EFFECTIVE PREPROCESSING AND FUSION OF AUDIO AND INERTIAL SENSORS FOR HUMAN ACTIVITY RECOGNITION WITH ATTENTION MECHANISMS

Abstract

This thesis presents a novel approach for human activity recognition (HAR) using sensors that incorporate advanced techniques such as Transformers and Attention mechanisms. The proposed system utilizes both audio and inertial sensors to estab lish an efficient preprocessing and fusion methodology for raw sensor signals. The preprocessing and fusion of the raw sensor signals help to remove any noise or incon sistencies present in the sensor data and combine the complementary aspects of the two sensors, respectively. The proposed approach then employs Convolutional Neural Networks (CNN) for feature extraction from the preprocessed and fused data. The extracted features are then fed into the Transformer-Encoder and Attention mechanisms, which are utilized for classification. These mechanisms are capable of modeling complex dependencies and temporal patterns in sequential data, allowing for more accurate recognition of primary and secondary activities, context, and phone placement. To evaluate the effectiveness of the proposed HAR system, experiments were conducted on the Extrasensory dataset. The results demonstrate that the proposed model outperforms state-of-the-art approaches, highlighting the effectiveness of advanced techniques such as Transformers and Attention mechanisms for HAR. This research represents a significant contribution to the field of HAR as it provides a novel approach that surpasses the current state-of-the-art methods. The proposed approach has the potential to impact various applications such as health monitoring, fitness tracking, context-aware services, and user behavior analysis, making it a valuable addition to the field of HAR