ML Model To Recognize Prosodic Or Emotional Cues In Audio Data

Abstract

his project presents a lightweight, real-time Speech Emotion Recognition (SER) system designed to run on a resource-constrained Raspberry Pi 5 with 2 GB RAM. The system classifies six emotions (happiness, sadness, anger, fear, surprise, and disgust) from live audio input. Multiple architectures were evaluated, including Linear SVM, optimized SVM, CNN-BiLSTM with attention, and a Multilayer Perceptron (MLP). The final MLP model, trained on a merged dataset of RAVDESS, CREMA-D, and TESS, achieved 78.8% test accuracy using a 217-dimensional feature vector comprising MFCCs, log-Mel spectrograms, chroma, spectral contrast, and prosodic features. The model meets all hardware constraints, with a size under 50 MB and inference time of 0.3–0.4 seconds per clip. Live testing confirmed real-time classification of five out of six emotions. Generalization limitations were observed when testing on unseen accents (SAVEE dataset), highlighting the need for greater speaker and accent diversity in future work.