In this project we are interested in using Hierarchical Multimodal Interfaces (HMI) to infer the activity of a person. The multimodality of such framework not only provides us with different sources of information to better decide about the ongoing activity of a person, but also makes activity recognition robust to loosing a source of information. This feature is especially important in environments where information from a modality (e.g., audio or video) is not available. In the following we have presented some of the applications for this technique.

ENTERTAINMENT: WHAT TYPE OF MUSIC YOU MAY LIKE TO HEAR?                                                                                                                                      

ACTIVITYMONITOR: USING MOBILE PHONES IN ASSISTIVE TECHNOLOGY (AT)                                                                                                                     

In this application, we have used a mobile phone to monitor physical activities and wellness of a user. A potential target group for such application would be elderly or handicapped people or people with need for special care and constant monitoring. The method is based on processing acceleration data provided by accelerometers integrated in mobile phones. This information is sent to a monitoring server, analyzed and translated into different health related factors for monitoring and assisting purposes. A monitoring agent, then, uses a desktop application to observe the patterns of physical activities of several patients in an online manner, and receives warnings in case of unexpected physical conditions. The data can also be stored offline for long term analysis of the physical behaviour of the patient. The desktop application also provides different options for managing, browsing, and searching activity related data. Video of live monitoring of health situation:

EMERGENCY DETECTION: DETECTING AN ACCIDENT WITH MOBILE PHONES                                                                                                                          

In this application we use a mobile phone to monitor physical activities of a user and detect unexpected emergency situations such as being in an accident or collapsing. Upon detection of such an event, the mobile phone can inform a designated centre (by automatically dialling a number or sending a text message) about the incident and its location. This can facilitate and speed up the help process especially if the user is alone or the accident has happened in a deserted place. Such an application can be particularly useful for elderly people or people with physical disabilities. The application operates based on analysis of user movements using data provided by accelerometers integrated in mobile phones. Video of emergency detection:

2010 

•       Towards Implicit Enhancement of Security and User Authentication in Mobile Devices Based on Movement and Audio Analysis

Hamed Ketabdar, Mehran Roshandel, Daria Skripko
The Fifth International Conference on Tangible, Embedded and Embodied Interaction, Madeira, Portugal, January 23-26, 2011

Abstract (.txt), paper (.pdf)

•       Enhanced Phone Posteriors for Improving Speech Recognition Systems

Hamed Ketabdar, Herve Bourlard
IEEE Transactions on Speech and Audio Processing, Vol. 18, Issue: 6, pp. 1094-1106, August, 2010
Abstract(.txt), paper(.pdf)

•       System and Methodology for Using Mobile Phones in Live Remote Monitoring of Physical Activities

Hamed Ketabdar, Matti Lyra
IEEE International Symposium on Technology and Society, pp. 350-356, Wollongong, Australia, June7-9, 2010
Abstract(.txt), paper(.pdf)

•       ActivityMonitor: Assisted Life Using Mobile Phones

Hamed Ketabdar, Matti Lyra
Proceedings of the Fourteenth International Conference on Intelligent User Interfaces, pp. 417-418, Hong Kong, China, 2010
Abstract(.txt), paper(.pdf)

•       Smart Ring: Controlling Call Alert Functionality Based on Audio and Movement Analysis

Hamed Ketabdar, Kamer Ali Yuksel
Proceedings of the Fourteenth International Conference on Intelligent User Interfaces, pp. 415-416, Hong Kong, China, 2010
Abstract(.txt), paper(.pdf)

2009

•       Tactile and Visual Alerts for Deaf People by Mobile Phones

Hamed Ketabdar, Tim Polzehl
Proceedings of the Eleventh International ACM SIGACCESS Conference on Computers and Accessibility, pp. 253-254, Pittsburgh, USA, 2009
Abstract(.txt), paper(.pdf)

•       Fall and Emergency Detection with Mobile Phones

Hamed Ketabdar, Tim Polzehl
Proceedings of the Eleventh International ACM SIGACCESS Conference on Computers and Accessibility, pp. 241-242, Pittsburgh, USA, 2009
Abstract(.txt), paper(.pdf)

•       Detecting Physical Shock by A Mobile Phone and Its Applications in Security and Emergency

Hamed Ketabdar, Tim Polzehl
Proceedings of the Eleventh International Conference on Human-Computer Interaction with Mobile Devices and Services, Bonn, Germany, 2009
Abstract(.txt), paper(.pdf)

•       Using Mobile Phones for Monitoring Physical Conditions

Hamed Ketabdar
Lecture Notes in Informatics, Proceedings of the Informatik 2009 Conference and the Third Workshop on Mobile and Embedded Interactive Systems, September, 2009
Abstract(.txt), paper(.pdf)

•       Enhancing Security and Emergency Functionalities in Mobile Phones by Audio and Movement Analysis

Hamed Ketabdar, Tim Polzehl
The Eleventh International Conference on Ubiquitous Computing, Florida, USA, September 30-October 3, 2009
Abstract(.txt), paper(.pdf)