POSE2ACT: TRANSFORMER-BASED 3D POSE ESTIMATION AND GRAPH CONVOLUTION NETWORKS FOR HUMAN ACTIVITY RECOGNITION

Simple item page
dc.contributor.authorAimyshev, Dias
dc.date.accessioned2023-05-24T10:04:07Z
dc.date.available2023-05-24T10:04:07Z
dc.date.issued2023
dc.description.abstractThe rise of deep learning has brought significant attention to two tasks in computer vision: pose estimation and human activity recognition. While human activity recognition has various applications in IoT systems, pose estimation is critical for motion tracking and prediction in virtual and augmented realities, robotics, and other fields. Despite being distinct tasks, they are closely linked, and this study focuses on merging pose estimation, which generates body joint coordinates, and skeleton-based activity recognition, which operates on the given joints. The study uses a visual transformer for 3D pose estimation, viewing joints as spatial features and neighboring frames as temporal features. Meanwhile, graph convolution networks are used for activity recognition based on a 3D skeleton, which has produced state-of-the-art results. However, these outcomes are based on 3D coordinates generated by motion capture systems and have limitations in their applicability and robustness. To overcome these limitations, the two models are merged into a single End2End network. The proposed approach is enhanced by applying various data transformations, modifications, pre-training, and fine-tuning of different architecture components. The research achieves a 90.3% activity recognition cross-subject accuracy score on the NTU RGB+D test dataset, comparable to the state-of-the-art using generated 3D input, and outperforms other models using 2D input by predicting 3D coordinates in the process.en_US
dc.identifier.citationAimyshev, D. (2023). Pose2Act: Transformer-based 3D Pose Estimation and Graph Convolution Networks for Human Activity Recognition. School of Engineering and Digital Sciencesen_US
dc.identifier.urihttp://nur.nu.edu.kz/handle/123456789/7069
dc.language.isoenen_US
dc.publisherSchool of Engineering and Digital Sciencesen_US
dc.rightsAttribution-NonCommercial-ShareAlike 3.0 United States*
dc.rights.urihttp://creativecommons.org/licenses/by-nc-sa/3.0/us/*
dc.subjectType of access: Embargoen_US
dc.subjectpose estimationen_US
dc.subjecthuman activity recognitionen_US
dc.subject3D skeletonen_US
dc.titlePOSE2ACT: TRANSFORMER-BASED 3D POSE ESTIMATION AND GRAPH CONVOLUTION NETWORKS FOR HUMAN ACTIVITY RECOGNITIONen_US
dc.typeMaster's thesisen_US
workflow.import.sourcescience

Files

Original bundle
Now showing 1 - 2 of 2
Thumbnail Image
Name:
Dias Aimyshev Computer Science Thesis Manuscript.pdf - Dias Aimyshev.pdf
Size:
1.89 MB
Format:
Adobe Portable Document Format
Description:
thesis
Download
No Thumbnail Available
Name:
Dias Aimyshev Computer Science Thesis Presentation - Dias Aimyshev.pptx
Size:
10.55 MB
Format:
Microsoft Powerpoint XML
Description:
presentation
Download
License bundle
Now showing 1 - 1 of 1
No Thumbnail Available
Name:
license.txt
Size:
6.28 KB
Format:
Item-specific license agreed upon to submission
Description:
Download

Collections

02. Master's Thesis