AI-Powered Multimodal Evaluation System for Job Interviews: Integrating Technical Assessment with Emotion and Personality Recognition

Abstract

This project presents an AI-powered multimodal interview evaluation system designed to support more structured, transparent, and consistent candidate assessment. The system analyzes interview videos using textual, audio, and visual information to evaluate technical responses, emotional expression, and personality-related indicators. It combines deep learning, speech processing, computer vision, and language model-based analysis within a modular web application built with PyTorch, Whisper, Streamlit, and a Python backend. The platform provides secure video upload, automated multimodal processing, candidate comparison, and report generation through an interactive dashboard. A small internal evaluation dataset was collected for system-level testing, and the results suggest that the prototype can provide assessments that are reasonably aligned with expert human judgment. The system is intended as an assistive tool for human evaluators rather than a replacement for human decision-making.