Avaliação de modelos de linguagem para criação de chatbots para a comunicação com pessoas surdas que usam o português como segunda língua

Abstract

Large language models (LLMs) have established themselves as powerful tools for natural language processing, but their performance still presents challenges when applied to underrepresented linguistic variants. GLOSA—a textual representation that reflects the syntactic structure of LIBRAS—offers a controlled means of studying how models handle linguistic constructions similar to the written communication of deaf people, who frequently use Portuguese as a second language. This work investigated the adaptability of LLMs to GLOSA, with the aim of supporting the development of chatbots more accessible to the deaf community. Comparative experiments were conducted involving multiple open architectures of different sizes, evaluated in multilingual scenarios (English, Portuguese, GLOSA, and mixed data). Two fine-tuning strategies were applied to pre-trained models using databases constructed to reflect real GLOSA syntactic structures. The results confirm performance losses when migrating from English to Portuguese and, especially, to GLOSA, a pattern already observed in multilingual studies. However, targeted fine-tuning increased model interpretability, especially on larger architectures with robust multilingual pre-training, such as Phi-4-14B and Qwen2.5-14B. However, it was observed that excessive or misaligned adjustments can cause overfitting or decreased accuracy, highlighting the need for rigorous control in the adaptation process. This study offers three main contributions: (i) a novel dataset in GLOSA and Portuguese; (ii) a reproducible experimental pipeline for evaluating LLMs in unconventional linguistic scenarios; and (iii) practical recommendations for model tuning in accessibility-focused applications.