Towards the establishment of sign agents’ evaluation apparatus tailored for native deaf signers

Abstract

Sign languages (SL) are a mode of communication that depends solely on the visual sense and is acknowledged as a complete natural language. These languages are inherently used by the deaf, hard-of-hearing, and certain hearing persons (family, friends, relatives, CODA (Children Of Deaf Adults) interpreters, etc.). The primary modality of spoken language is voice, with occasional writing, while sign language largely utilizes the visual-gestural modality. In daily life, adequate speech and writing are often inaccessible to unimodal/monolingual deaf users of sign language. Previous studies on sign language recognition have shown promising results in attaining highly accurate and resilient automated sign language recognition and have exhibited significant promise in tackling social barriers and concerns of inclusion. Sign language-generating avatars are virtual characters or animated creatures created to convey messages using sign language. These avatars are often used in apps designed to enhance communication with those who are deaf or hard of hearing. They may be configured to comprehend text or audio input and produce appropriate sign language motions. Research in this domain has concentrated on augmenting the realism and precision of sign language avatars, alongside strengthening their capacity to express emotions and subtleties in sign language communication. Research has investigated many methodologies, including text-to-sign conversion, motion capture technologies, machine learning algorithms, and natural language processing, to improve the functionality of sign language-generating avatars. Sign language generating avatars have the capacity to overcome communication disparities and provide more inclusive and accessible communication alternatives for those who depend on sign language as their primary means of communication. Continued research and development in this domain are crucial for enhancing the capabilities and efficacy of sign language avatars. The use of virtual characters as helpers has significantly increased during the last decade. Nonetheless, the advancement of medically plausible sign language synthesis and output remains in its infancy. This remark highlights the lack of advancement in virtual intelligent signing generation systems. Moreover, current models mostly rely on human rules and need specialized knowledge, while data-driven methodologies may provide superior solutions. This study highlights the deficiencies in the evaluation of sign language systems as viewed by users, as well as the advancements achieved in the creation of these systems. Signing avatars have been the focus of human-computer interaction research for a very brief period. Despite the recent advancements in sign language interpreting avatars, data indicate methodological deficiencies in research that impede the creation of effective sign language interpreting avatars.