AN EMPIRICAL STUDY OF FEDERATED LEARNING AND VIDEO REPRESENTATIONS FOR HUMAN ACTION RECOGNITION

Abstract

This thesis explores the integration of Federated Learning in Human Action Recognition, with a focus on both supervised learning and Few-Shot Learning methodologies using RGB, skeleton, or fusion data. By employing advanced Deep Learning models and leveraging large-scale, diverse datasets, we demonstrate significant advancements in HAR, crucial for enhancing smart video surveillance systems. Our novel FL framework addresses the challenges associated with centralized learning, such as substantial resource allocation and potential confidentiality violations, by training models in a decentralized manner. This approach enhances privacy and efficiency, utilizing diverse data across devices to improve generalizability. We present a comprehensive evaluation of various 3D-CNN and Transformer-based architectures, emphasizing the effects of pre-training and comparing FL algorithms, specifically FedAvg and FedProx, under realistic settings. Our findings reveal that integrating pre-trained 3D-CNNs and Transformers with optimal FL configurations can significantly enhance HAR performance, enabling our models to compete with and, in some cases, surpass centralized learning counterparts.