AUTOMATION OF CORPUS BUILDING FROM TRANSCRIPTION TO SEARCH: A CASE STUDY OF KAZAKH

Abstract

This thesis examines the challenges of corpus building for low-resource languages by developing an automated workflow for the Multimedia Corpus of Spoken Kazakh Language (MCSKL). Centering on Kazakh as a case study, this exploration examines the strategies for improving the efficiency of the corpus-building process by introducing automation. The challenge this thesis aimed to address is adapting tools initially designed for high-resource languages, such as English, to low-resource ones, like Kazakh, which was achieved via Whisper STT. However, since transcription is the early stage, the STT had to be a part of a more complex workflow. Thus, the second challenge was to create a workflow that could simplify the time-consuming steps of corpus building, ultimately aiming to convert language data from audio to a searchable database. The thesis also presents a closer look at MCSKL, a corpus of naturally occurring spoken interactional Kazakh, which served as the data source, and the case study for the automation effort feasibility. The study argues that adapting tools designed initially for high-resource languages, such as Whisper for speech-to-text and ELAN for annotation, requires not only technical fine-tuning but also methodological adjustments. In response, this thesis proposes a scalable and modular workflow that integrates these tools and supplements them with custom Python scripts, enabling researchers to efficiently process naturally occurring, unprompted spoken language data and output it in searchable formats compatible with search engines such as Apache Solr. Methodologically, the thesis adopts a discourse-functional approach to annotation, using intonation units rather than sentence-based segmentation to more faithfully represent spoken interaction. The research draws from both theoretical insights in corpus linguistics and practical implementation within a multilingual, interdisciplinary

research collaboration. It also highlights the broader implications of automating linguistic documentation for underrepresented languages in the digital realm. The findings demonstrate that while full automation remains elusive, targeted computational support for the time-intensive phases can significantly reduce annotation time, improve consistency, and empower resource-limited teams. Ultimately, this thesis contributes to sustainable corpus-building practices and reinforces the role of corpus linguistics as a discipline central to both linguistic research and the development of language technology.