AI-YM: AI-based 
Solutions for Kazakh 
Sign Language

Senior Project Final Presentation 2025

Computer Science Department

STUDENTS: Aruzhan Kaltay, Bekzat Ongdassynov, Dilnaz Sembekova, Yerzhan Yerbatyr, 
Karina Iskakova

ADVISORS/CO-ADVISOR: Anara Sandygulova, Antonio Cerone



11/18/2024



Review: The 
problem 
statement  

 a gap in inclusive 
communication

People with hearing loss cannot rely on 
other modes for effective communication 
and hearing people do not know sign 
languages.

 sign language 
processing

Interdisciplinary field focusing on creating 
technology for sign language recognition 
and translation.

 Education tools
Technologies greatly impact the learning 
and teaching processes.

SENIOR PROJECT FINAL PRESENTATION



Review: What we 
developed?

Fall semester 2024

11/18/2024



Now to Current 
Semester

Developed in the 
Spring Semester!

 Developed in the Fall 
Semester;

 Ai-YM Learn 
for 
students

 ai-ym  for 
teachers

 AI Integration 
 feedback model for 

sign-language;  content creators for AI-
YM learn, not only 
students!



AI Integration: 

How we did it?

How it works? 

 Video Processing 
& Landmark 
ExtractioN

We use  to detect and extract
 Full-body pose landmarks (33 points)
 Up to 2 hands (21 landmarks each).

MediaPipe

 Preprocessing & 
Feature 
Engineering

 We  and clean it, then 
normalize it.

 read CSV data

 Feature 
Extraction

 We compute joint angles between 
body landmarks to describe posture

 We extracts
 We generates speed-invariant features.

 trajectory features



AI Integration: 

How we did it?

How it works? 

  Sequence 
Matching & 
Comparison

 We use


 Dynamic Time Wrapping
To compare sequences even if they 
have different lengths or speeds

 Visualization
 We shows landmarks and skeletons 

from both sequences: real-time and 
from database.

 We display similarity scores and 
aligned frames.



Let’s see results. 

Test Cases with Known Labels



Let’s see results. 

Test Cases with Known Labels

Green landmarks are 
video real-time marks

Blue landmarks are 
marks of correct sign



How about AI-YM for 
Teachers?

We call it Content 
Creator Studio.

Let’s demonstrate it!



User evaluation with
 students
 teachers
 Deaf community
 native sign language users;
 sign language interpreters; 


10 people in total;

After evaluation
 we enhanced and fixed 

design; 



Website Performance 
Evaluation

Lighthouse tool for web apps 
performance evaluation

Reports for each page



Website Performance 
Evaluation



delivered fully functioning user-friendly 
platform for publishing and taking courses to 
learn sign language with feedback on the 
gesture accuracy.

Conclusion

11/18/2024