Applicability of Pretrained Language Models: Automatic Screening for Children’s Language Development Level (2022)

Abstract: The various potential of children can be limited by language delay or language impairments. However, there are many instances where parents are unaware of the child’s condition and do not obtain appropriate treatment as a result. Additionally, experts collecting children’s utterance to establish norms of language tests and evaluating children’s language development level takes a significant amount of time and work. To address these issues, dependable automated screening tools are required. In this paper, we used pretrained LM to assist experts in quickly and objectively screening the language development level of children. Here, evaluating the language development level is to ensure that the child has the appropriate language abilities for his or her age, which is the same as the child’s age. To do this, we analyzed the utterances of children according to age. Based on these findings, we use the standard deviations of the pretrained LM’s probability as a score for children to screen their language development level. The experiment results showed very strong correlations between our proposed method and the Korean language test REVT (REVT-R, REVT-E), with Pearson correlation coefficient of 0.9888 and 0.9892, respectively.

## introduction language development is directly related to cognitive and intellectual development and is impacted by environmental factors including social interactions such as conversation with parents, etc @xcite . language delay is the inability of a child to understand or use spoken language appropriately for their age, and it can result in language impairments. language impairments are disorders of language that has a negative impact on all facets of life, including academic performance and social interaction, and restricts a child's wide range of potential @xcite @xcite . in this situation, @xcite reported that many children with language impairment were not receiving appropriate treatment because their parents were unaware of the child's condition. in addition, many studies anticipate that following the covid-19 pandemic, quarantine measures including social distancing and mask wearing will include a negative impact on children's language development @xcite @xcite . to address this issue, experts have developed language tests that may be used prior to make diagnosing language impairments. standardized formal test analyzes linguistic abilities to screening a child's language development level. for example, ppvt-iv (peaboby picture vocabulary test-iv) @xcite and evt-2 (expressive vocabulary test-ii) (kathleen t. @xcite evaluate receptive vocabulary and expressive vocabulary, respectively. language sample analysis (lsa) analyzes linguistic abilities like grammar, pragmatics, and semantics as a measure @xcite . these methods evaluate a child's language development level compared with standardized norms from the same age group's children who have normally developed. in other words, it evaluates if a child has linguistic abilities that are age-appropriate. if a child's scores on these methods are lower than the norm for the same age group, tests to diagnose language impairment are performed. however, moving forward with the standardized formal test and lsa process requires ## transcription data in the field of speech therapy, a rule called conversation protocol is used to ensure reliability of norms and analysis when collecting children's data (i.e. utterances). conversation protocols allow only specific topics in the interview, and experts encourage children to speak on their own. as a result, we used the hallym conversation & pragmatic assessment protocol @xcite that the division of speech pathology and 1 the study was conducted according to the guidelines of the declaration of helsinki and is approved by the institutional review board of hallym university (hirb-2019-036, hirb-2021-093). audiology at hallym university created to collect data. standardized formal test and lsa examine the age-related differences in scores in children who have developed normally in order to verify their norms. so, children between the ages of 2 and 6 who the experts assessed to have developed normally were the subjects of data. the data we used were collected by experts after approved the institutional review board of hallym university 1 . it is a total of 143 children, and each child includes an average of 89 utterances. the words that are used by people of all ages and make it interrupt to analyze a language development level are indicated with special characters (i.e. symbols) in lsa and then excluded from the analysis. as a result, after analyzing them, we removed these words. these word types are shown in the following single-word utterances, such as "yes" or "no" and utterances using only proper nouns for people or things, are frequently appeared in children. because these utterances frequently appear in children of all ages, they interfered with the classification of children's age for language development levels in previous studies based on supervised learning @xcite . as children grow older, these utterances tend to become less frequent and utterances with complete sentence become more. we believe that this tendency is a useful linguistic characteristic for screening to children's language development level based on pretrained lm. the ratio of these utterances in age-specific children's overall utterances is shown in figure 1 . ## automatic screening based on pretrained lms children's language systems, including their grasp of grammar, steadily improve as they grow older. in this situation, the basis of our proposed method can evaluate the sentence's grammaticality using several values (e.g., probability, normalized log probability, etc) that can be calculated from a pretrained lm @xcite . these characteristics demonstrate the feasibility of using a pretrained lm to screen children's language development level. to screen the child's language development level, we only use the pretrained lm's probability as a score for the child's utterance and calculate the standard deviation of these scores. the rest of this section details more into pretrained lms which is used in this paper and the scoring method we used to screen for children's language development level. @xcite verified that the pretrained lm's probability may be used to assess the grammaticality of english sentences. based on gpt2, grammatical sentences ( 𝑠 𝑔𝑜𝑜𝑑 ) were evaluated highly scores in around 94% of all the data which is consist of (𝑠 𝑔𝑜𝑜𝑑 , 𝑠 𝑏𝑎𝑑 ) pairs. so, we verified if the korean pretrained lm provided the same observations as these experiments. ## results and discussion to ensure consistency and reliability of the analysis, lsa chooses 30 to 50 of the utterances made by children and analyzes them as a certain number of utterances @xcite @xcite . by omitting this procedure, we aim to provide an ## conclusion in this paper, we used pretrained lms for automated screening and tried to address limitations in the existing language tests, such as the number of data and the diversity of languages. at this time, we preprocessed the utterance by analyzing age-specific linguistic patterns of children to use the pretrained lm efficiently. additionally, the correlation with revt, a standardized formal test for korean language, was evaluated to demonstrate the reliability of our proposed method. the experimental results revealed a strong correlation between our proposed method, which is based on kogpt2-skt, and the norms for revt-r and revt-e, with pearson correlation coefficients of 0.9888 and 0.9892, respectively. these observations demonstrate the potential for the pretrained lm to automatically screen children's language development levels and are expected to address several issues with the limitations of language tests such as standardized formal tests and lsa. furthermore, we believe that the pretrained lm demonstrated the potential for applicability in various issues needing skills in natural language processing. future work will focus on make up for automatic screening based on pretrained lms and investigating automatic transcription methods for collecting children's utterance data using automatic speech recognition.