2D SKELETON-BASED HUMAN ACTION RECOGNITION USING ACTION-SNIPPET REPRESENTATION AND DEEP SEQUENTIAL NEURAL NETWORK

Abstract

Human action recognition is one of the crucial and important tasks in data science. It aims to understand human behavior and assign a label on performed action and has diverse applications. Domains, where this application is used, includes visual surveillance, human–computer interaction and video retrieval. Hence, discriminating human actions is a challenging problem with a lot of issues like motion performance, occlusions and dynamic background, and different data representations. There are many researches that explore various types of approaches for human action recognition. In this work we propose advanced geometric features and adequate deep sequential neural networks (DSNN) for 2D skeleton-based HAR. The 2D skeleton data used in this project are extracted from RGB video sequences, allowing the use of the proposed model to enrich contextual information. The 2D skeleton joint coordinates of the human are used to capture the spatial and temporal relationship between poses. We employ BiLSTM and Transformer models to classify human actions as they are capable of concurrently modeling spatial relationships between geometric characteristics of different body parts.