MULTIMODAL EMOTION RECOGNITION USING DEEP LEARNING AND FUSION TECHNIQUES

Simple item page
dc.contributor.authorMukhametsharip, Zhanna
dc.contributor.authorKhamitova, Ainur
dc.contributor.authorKabdrakhmetova, Zhazira
dc.contributor.authorNurmakhan, Temirlan
dc.date.accessioned2024-06-19T05:10:16Z
dc.date.available2024-06-19T05:10:16Z
dc.date.issued2024-04-20
dc.description.abstractEmotion recognition plays a crucial role in human-computer interaction, significantly influencing the advancement of virtual assistants, mental health diagnosis tools, and customer experience analysis systems. Our senior project aims to develop an advanced multimodal emotion recognition (MER) model using modern deep learning techniques and fusion methods. Most traditional emotion recognition models rely on a single modality for decision-making, such as facial expressions or text. However, this approach can be limited in capturing the complexity of human emotions. To overcome this limitation, we will integrate multiple input types to create a more comprehensive model, reducing misclassifications and improving overall system performance. Our system includes an emotion recognition model and a user interface for interaction. The web application will serve as the interface, allowing users to upload video materials of a specified duration. The application extracts audio, video, and text from the uploaded video and feeds them into different deep-learning models customized for each modality. The outputs, representing probabilities for various emotion classes (e.g., ”happy,” ”sad,” ”fearful,” ”surprised,” ”angry,” ”disgusted,” and ”neutral”), will be combined using fusion techniques for enhanced accuracy. The web app then presents visual representations of the emotions through graphs and descriptions for user interpretation.en_US
dc.identifier.citationKhamitova A., Mukhametsharip Z., Kabdrakhmetova Z., Nurmakhan T. (2024). Multimodal Emotion Recognition Using Deep Learning and Fusion Techniques. Nazarbayev University School of Engineering and Digital Sciencesen_US
dc.identifier.urihttp://nur.nu.edu.kz/handle/123456789/7901
dc.language.isoenen_US
dc.publisherNazarbayev University School of Engineering and Digital Sciencesen_US
dc.rightsAttribution-NonCommercial-NoDerivs 3.0 United States*
dc.rights.urihttp://creativecommons.org/licenses/by-nc-nd/3.0/us/*
dc.subjectType of access: Embargoen_US
dc.subjectmultimodal deep learningen_US
dc.subjectemotion recognitionen_US
dc.titleMULTIMODAL EMOTION RECOGNITION USING DEEP LEARNING AND FUSION TECHNIQUESen_US
dc.typeBachelor's thesisen_US
workflow.import.sourcescience

Files

Original bundle
Now showing 1 - 1 of 1
No Thumbnail Available
Name:
group31.pdf
Size:
2.1 MB
Format:
Adobe Portable Document Format
Description:
capstone project
Download
License bundle
Now showing 1 - 1 of 1
No Thumbnail Available
Name:
license.txt
Size:
6.28 KB
Format:
Item-specific license agreed upon to submission
Description:
Download

Collections

03. Bachelor's Thesis