Ubiquitous human activity recognition for healthcare applications: A study

Human activity recognition (HAR) has gained importance in recent years due to its applications in various fields such as healthcare, remote monitoring, security and surveillance, entertainment, and intelligent environments. HAR is a very vast field and consists of activities ranging from atomic (e.g., sitting) to very complex (e.g., dancing or preparing a meal). Also, based on the application area, activity can be monitoring the sleep of an elderly person or recognizing the gesture to interact with a system. Given the diverse nature of HAR, different methods have been used for recognizing human activities which can be broadly divided into vision-based and sensor-based. Sensor-based (also called ubiquitous) human activity recognition techniques have seen tremendous growth recently and are being used in various applications due to their advantages over the vision-based method such as low cost, privacy preserving, and easy deployment. Researchers have leveraged different sensor-based approaches, such as wearable, object-tagged, and device-free, to recognize human activities. Especially, in healthcare applications, sensor-based HAR has become very popular and many solutions have been presented to assist caregivers and help people with issues such as remote monitoring, medication management, sleep monitoring, exercise management, and independent living.

Although ubiquitous HAR techniques are being adopted in various applications, especially in healthcare, there are many associated challenges with it. Firstly, healthcare activities are very different than normal activities, e.g., monitoring the chest movements to estimate the breathing rate is very different than monitoring daily life activities. Some of these activities require very fine-grained monitoring such as vital signs monitoring or emotion recognition. Also, in some activities wearable approach may yield better results while in other, object-tagged or dense sensing may work better. The sensor-based approach suffers from environmental interference as the sensor's reading can be easily corrupted by the noise. Other limitations of the sensor-based approach include battery requirement, calibration, limited processing power, and storage.

In this thesis, we explore the use of ubiquitous HAR for healthcare applications and focus on two important problems which are sleep monitoring and toothbrushing activity monitoring. The first key contribution of this thesis is the comprehensive overview of the HAR area in general (Chapter 2) and sleep monitoring in specific (Chapter 3). We categorize the HAR into three major categories: action-based, motion-based, and interaction-based. We further classify these categories into 10-subtopics and review the latest works conducted in these areas. We compare the state-of-the-art solutions using 10 key factors. For sleep monitoring, we also provide a comprehensive review of the latest research in different aspects of sleep monitoring. We divide sleep monitoring into four categories, namely stages classification, posture recognition, disorder detection, and vital signs monitoring. We review and compare the recent works in these four aspects using 10 key metrics.

The second major contribution of this work is the development and implementation of a cost-effective and non-invasive system for sleep monitoring (Chapter 4). Using off-the-shelf available passive Radio Frequency Identification (RFID) technology, we present a system that can easily monitor the sleep in an in-home environment without the need for any special equipment. We attach two RFID tags to user's clothes while sleeping. Using an RFID reader and antenna, we energize these passive tags which capture the body and chest movements and modulate this information in the reflected signal. By applying multiple signal processing techniques, the proposed system can detect body movements (e.g., changing sides) and can accurately estimate the breathing rate. It can also detect sleep apnea which is a respiration-related sleep disorder and is among the most common sleep disorders. We evaluate the performance of the proposed system by conducting multiple experiments in real-world scenarios. The results show that our system achieves 100% accuracy for apnea detection and above 95% for breathing rate estimation.

The third main contribution of this work is the development of an off-body detachable sensor-based solution to monitor and assess the toothbrushing activity (Chapter 5). In this system, we model the problem of adherence to the standard toothbrushing method as an activity recognition problem. We divide the toothbrushing activity into 16 sub-activities corresponding to the 16 regions of the teeth. Exploiting the object-tagged approach, we attach an Inertial Measurement Unit (IMU) to the brush handle and capture the orientation and movements of the brush while reaching different regions of the teeth. We train a machine learning pipeline to recognize these 16 sub-activities which must be present in one standard toothbrushing session. We evaluate the performance of the proposed system in real-world scenarios and perform experiments with multiple users. The results show that our system performs better than wearable-based approach and can recognize the toothbrushing activity with 97.15% accuracy. We also evaluate our model for different types of brushes (manual and electric) and the results show that the proposed approach can work independently from the brush types.

The fourth significant contribution of this work is the collection of our own dataset for the toothbrushing activity (Chapter 6). As there was no publicly available dataset for toothbrushing activity, we collected our own dataset by conducting real-world experiments. We recruited 22 participants (11 males, 11 females) and recorded their toothbrushing sessions when they brushed their teeth over two weeks in seven different locations using different types of electric and manual brushes. We attached one IMU device with a brush handle while a second IMU device was worn by the participants on their brushing hand as a wristwatch. We collected the data in two different settings and collected a total of 120 toothbrushing sessions using both brush-attached and wearable approaches. We have made this dataset available for free public use by the research community.