AI-YM: AI-BASED SOLUTIONS FOR KAZAKH-RUSSIAN SIGN LANGUAGE

Abstract

AI-YM - is the interactive web application powered by the artificial intelligence designed to teach hearing individuals Kazakh and Russian Sign Language (K-RSL). This platform addresses the lack of accessible educational tools for learning K-RSL by employing AI to recognize signs performed by users and provide feedback, thereby enhancing the learning experience. The system addresses a significant gap in accessible K-RSL resources, especially for non-deaf users aiming to build inclusive communication skills. The methodology involved designing and integrating AI gesture recognition and developing a full-stack solution. The project’s final result is a fully functional web application that combines AI-based gesture recognition, user role management, interactive lesson creation, and a gamified user experience. Our AI model recognizes K-RSL signs from video input, compares to the original sign trajectory, and provides feedback to learners. Additionally, the learners can track their progress by earning XP points, use the streak system, and enroll in the desired courses. Finally, teachers can create their own learning content, and contribute their materials to the AI-YM platform. This project demonstrates the end-to-end design, implementation and evaluation of our solution to a real-world social challenge. The project employs machine learning, human- computer interaction, and web technologies to develop a responsive, inclusive, and adaptive educational tool.