AI-BASED MULTIMODAL EMOTION RECOGNITION SYSTEM

Abstract

The project built an AI system for emotion recognition which integrated video data-processing with audio analysis as well as textual information assessment. Real-time facial emotion detection through Vision Transformers (ViT) and speech emotion recognition through Wav2Vec2 made up the core targets of the project with the aim of their integration. The project overcame dataset problems along with scope adjustments by concentrating on processing video and audio content instead of text analysis. The ultimate version of the prototype shows 90% accuracy in detecting emotions across high-definition video material thus creating a new framework which benefits applications in the areas of service interaction and psychiatric assessment.