# An Investigation of Applying Large Language Models to Spoken Language Learning
## 4 Discussion  
Our work is the first to explore the potential applications of large language models (LLMs) in the field of spoken language learning. We aim to create a language learning AI tutor at a professional level that can accurately provide learning feedback. In the past, most work could only address errors at the phoneme level, such as substitution, insertion, or deletion errors. However, diagnostics at the suprasegmental level are highly customized, as a sentence can have different realizations of pronunciation, stress, break, and intonation depending on the context and intended meaning. CAPT systems equipped with SLI provided by LLMs can give spoken language learners more helpful instructions and feedback. Although this area is rarely explored, it holds the potential to fundamentally address the current shortcomings in intelligent language learning [2]. In this paper, we conducted comprehensive experiments to investigate the SLI of LLMs in language learning. The task of extracting conceptual knowledge posed few challenges for these LLMs. However, some models encountered difficulties when using this knowledge for inference. For some small models, due to excellent instruction fine-tuning, their responses were almost always valid. However, for many small or even relatively large models, their performance on the Application Questions subset was worse than random guessing. This is largely because they could not generate valid outputs following instructions. Therefore, we conclude that the results we reported are actually a comprehensive reflection of both SLI and the models' ability to follow instructions. Large language models can easily grasp the definitions of linguistic concepts owing to the availability of high-quality content on the Internet. For example, online platforms like Wikipedia clearly explain what a glottal stop is. However, it is highly likely that there is no content on the Internet that addresses questions like _\"How should I use a proper rhythm to articulate this sentence?--Large language models have the potential to advance the development of human-computer interaction systems and educational technologies by playing the role of speech/language experts\"_. Such questions are highly customized and require the utilization of knowledge structures for analysis, specifically the ability to reason. However, many researchers believe that current artificial intelligence systems still lack the reasoning ability that is representative of human intelligence [62; 63; 64]. Despite the encouraging performance of LLMs in certain reasoning tasks, e.g., arithmetic reasoning and common-sense reasoning, current LLMs still perform poorly in most tasks that are relatively easy for humans, e.g., empathy, emotional support, and subjective decision making [20; 65]. With spoken language learning, students need the help of proficient language teachers. Accordingly, LLMs should have SLI for the CALL applications. The results of the current study suggest that the LLMs could handle concept-related questions but still lacked sufficiently powerful capabilities to answer our \"expert-level\" questions. In this context, it is in line with the above-mentioned studies. Some widely proven effective prompting methods combined with domain-specific examples resulted in significant performance improvements for the models (e.g., GPT-3.5: 49.1% \(\rightarrow\) 64.4%; LLaMA2-70B-Chat: 42.2% \(\rightarrow\) 48.6%, see Tables 3 and 5 for details). However, these performance improvements were evident mostly in relatively larger models. Moreover, in most cases, appending more examples in direct and CoT scenarios did not yield continuous performance improvements. Considering the increasing number of consumed tokens, we consider 3\(\sim\)5 shots a reasonable choice. As for using external tools, GPT-3.5 did not improve the reasoning accuracy in application questions after leveraging knowledge from Google and Wikipedia compared