Synonym Replacement based on a Study of Basic-level Nouns in S wedish Texts of Different Complexity (2021)

Abstract: Basic-level terms have been described as the most important to human categorisation. They are the earliest emerging words in children’s language acquisition, and seem to be more frequently occurring in language in general. In this article, we explored the use of basic-level nouns in texts of different complexity, and hypothesise that hypernyms with characteristics of basic-level words could be useful for the task of lexical simplification. We conducted two corpus studies using four different corpora, two corpora of standard Swedish and two corpora of simple Swedish, and explored whether corpora of simple texts contain a higher proportion of basic-level nouns than corpora of standard Swedish. Based on insights from the corpus studies, we developed a novel algorithm for choosing the best synonym by rewarding high relative frequencies and monolexemity, and restricting the climb in the word hierarchy not to suggest synonyms of a too high level of inclusiveness.

## introduction the research concerned with automatically reducing the complexity of texts is called automatic text simplification @xcite . automatic text simplification was first proposed as a pre-processing step prior to other natural language processing tasks, such as machine translation or text summarisation. the assumption was that a simpler syntactic structure would lead to less ambiguity and, by extension, a higher quality of text processing @xcite . however, one of the main goals of modern automatic text simplification systems is to aid different types of target readers. the manual production of simple text is costly and if this process could be automated, this would have a beneficial effect on the targeted reader, as well as the society as a whole. previous ats studies have targeted different reader groups, such as second language (l2) learners @xcite , children @xcite @xcite , persons with aphasia @xcite @xcite , the hearing-impaired @xcite @xcite , and other persons with low literacy skills @xcite @xcite . reducing the complexity of a text can be done in numerous ways but one of the subtasks of ats is lexical simplification: the process of finding and replacing difficult words or phrases with simpler options. finding such simpler words can be done by using frequency measures to choose between substitution candidates with the intuition that the more common a word is, the simpler a synonym it is. as pointed out, for instance by @xcite , more frequent words can also be complex as they tend to be more polysemous. finding simpler words can also be done by studying how human writers do. to write simple texts, the writers usually consult guidelines. for swedish, such guidelines are given by myndigheten för tillgängliga medier (mtm) 1 . the mtm guidelines state, among other things, that the text should be adapted to the type of reader who is going to read the text, and that everyday words should be used @xcite . in this article, we explore the use of basic-level nouns in texts of different complexity, and hypothesise that hypernyms with characteristics of basiclevel words could be useful for the task of lexical simplification. we then use this knowledge to cre-ate an algorithm for synonym replacement. the conventional definition of a synonym is a word that have the same or nearly the same meaning as another word. however, for simplicity, in this article we extend this notion to also include nearsynonyms or other semantically similar words. hypernyms have been previously studied from the perspective of lexical simplification. for example, @xcite explored the types of lexical simplification operations that were present in a parallel corpus comprising 200 standard and simple news texts in spanish, and found that the exchanged words could be hypernyms, hyponyms and meronyms. @xcite used the vocabularies of wikipedia and simple english wikipedia to create word pairs of content words, and one of the methods for filtering out substitution word pairs was to consult the synonym and hypernym relations between the words. comparable synonym resources for swedish include synlex @xcite and swe-saurus @xcite . given what we know how simple texts are written, it seems probable that a corpus of simple text, targeting children and readers with different kinds of disabilities, is characterised by a higher proportion of basic-level nouns than, for example, a corpus comprising texts that are said to reflect general swedish language of the 90's. the aim of this study was to explore this claim in corpora of simple and standard texts, and to see how this could be used in the context of lexical text simplification. ## basic-level words prototype theory, as defined by @xcite , claims that there is a scale of human categorisation where some representing concepts are more representative than others. for example, furniture can be regarded as higher up in the taxonomy than chair or table, whereas kitchen chair or dining table can be found at a lower level with higher specificity. @xcite found that the basic level is the most important to human categorisation. for example, basic-level terms emerge early in a child's language acquisition, and such terms generally seem to be more frequently occurring in language. another characteristic of basic-level terms is that they often comprise one single lexeme, while subordinate terms more often consist of several lexemes @xcite . theories in cognitive linguistics are important for computational linguists as they adopt a usagebased approach. this means that language use is essential to how our knowledge of language is gained, and plays a large role in language change and language acquisition @xcite . when a child learns a language, the knowledge is gathered through extraction of constructions and patterns, a process grounded in general cognitive processes and abilities. one of the central ideas in the usagebased approach is that the relative frequency of linguistic constructions (such as words) affects the language system so that more frequent constructions are better entrenched in the system, thus further influencing language use. within the field of cognitive linguistics corpora is one of the proposed methods to study language @xcite . corpora make it relatively simple to perform large-scale analyses in order to get quantitative measures on how language is used in a naturalistic setting. the simplest measures we can use are frequency counts, which can provide insights in how commonly used certain constructions are, in comparison with others. ## corpus analysis we conducted two corpus studies using different corpora. the first study aimed to compare two corpora, where the first corpus contained texts that reflect the swedish language, and the second corpus contained easy-to-read texts. the stockholm-umeå corpus (suc) corpus @xcite is a balanced corpus of swedish texts written in the 1990's. in this study, we used the 3.0 version of the corpus (suc3). the läsbart corpus @xcite , is a corpus of swedish easy-to-read texts of four genres: easy-to-read news texts, fiction, community information, and children's fiction. the läsbart corpus was compiled in order to mirror simple language use in different domains and genres but it is not truly balanced in the traditional sense. the hypothesis was that the suc3 corpus would exhibit a higher average number of steps to the top-level noun than the läsbart corpus. the second study aimed to investigate whether the genre did play a role. in order to investigate this, we conducted an analysis of a corpus of the swedish newspaper 8 sidor, that comprises news articles in simple swedish, and a corpus with göteborgs-posten articles (gp2d). the cor-pora were of the same genre, but not parallel. the hypothesis was that the gp2d corpus would exhibit an even higher average number of steps to the top-level noun than the 8 sidor corpus. the suc3 corpus is balanced and, hence, also includes, for instance, simple texts that may affect the difference between the corpora. ## synonym replacement based on the analysis presented in section 3.3, we developed an algorithm for choosing the best synonym from the extracted nouns and their superordinate words. ## assessment of synonym replacement algorithm we compared the performance of our combined frequency/monolexemity algorithm (hereafter: fm) with two baseline algorithms. the first baseline (onelevel) always chose the word one level higher up in the hierarchy as the best synonym. if there was no superordinate word, the word remained unchanged. the second baseline (freq) always chose the word with the overall highest relative frequency as the best synonym, thus disregarding the monolexemity information. we ran all algorithms on the nouns extracted from the standard corpora: suc3 and gp2d. words monolex polylex 0 0.5 1 1.5 2 •10 5 206,609 206,609 206,609 206,609 135,673 157,329 148,624 170,921 70,936 49,280 57,985 35,688 number of words corpus freq onelevel alg1 sented in figure 3 and figure 4 respectively. the relative frequencies after running the algorithms are illustrated in figure 5 . regarding the suc3 corpus, all synonym replacement algorithms increased the number of monolexemic words. the largest increase was observed for the fm algorithm (+35,248), followed by freq (+21,656), and onelevel (+12,951). regarding the relative frequencies, all algorithms increased the average relative frequency of the exchanged words. the largest increase was seen for freq (+153.68), followed by fm (+120.92), and onelevel (+34.68). on the gp2d corpus, the number of monolexemic words increased for all algorithms. the largest increase was seen for the fm algorithm (+30,783), followed by the freq algorithm (+21,091), and onelevel (+9,482). replacement algorithms resulted in a higher average relative frequency, and the largest increase was observed for the freq algorithm (+149.54), followed by the fm algorithm (+110.58), and onelevel (+7.2). ## discussion the algorithm for finding synonyms proposed in this article is built on theory and corpus studies. this algorithm obviously needs to be evaluated and compared to other methods for extracting synonyms from corpora and lexical resources. it would be valuable to compare the algorithm with synonyms from, for example, the synlex lexicon, and to evaluate whether the exchanged synonyms are simpler, when consulting lexicons of base vo- suc3 gp2d 0 100 200 300 140.66 294.34 175.34 261.58 155.44 304.98 162.64 266.02 corpora relative frequency corpus freq onelevel alg1 cabularies, as well as humans. it can also be enhanced with techniques to utilise semantic and synonym similarity @xcite . the corpus analyses were not conclusive, and, although further analyses will probably not present results that argues against the proposed algorithm, further investigations may be important for the study of language use and we therefore present a more detailed discussion on the corpus study. we hypothesised that simple texts would exhibit a tendency towards the use of more basic-level words, when compared with texts written in standard swedish. however, there was no clear support for this hypothesis. in the statistical analysis, we compared very large samples, and the presence of statistical significance is not surprising. when comparing the means and medians of the datasets, it is clear that the differences are small and the results should be interpreted with caution. the results of the first study revealed that the suc3 corpus had a significantly lower average number of steps to the top-level noun, than the läsbart corpus. since our hypothesis was that the texts of the corpus of simple text would have a lower average number of steps to the top-level noun, these results showed a difference in the opposite direction. the second study was normalised for genre, in the sense that the compared corpora contained texts of the same genre. the simple news corpus 8 sidor had a significantly lower number of steps to the top-level noun than the standard news corpus gp2d. this tendency is further supported by the results of the relative frequency analysis, where we clearly see that the 8 sidor corpus has relatively high average relative frequency at the base level (level n), although exhibiting the highest frequencies at level n+3, whereas the gp2d corpus generally had lower average frequencies at level n and the highest frequencies at level n+7. regarding the analyses of the relative frequencies, we would expect the standard corpora to have lower relative frequencies at the base level (level n) than the corpora of simple text. this difference can be observed in the läsbart corpus, which had the highest relative frequency scores at level n, but is less prominent in the 8 sidor corpus. however, even if the 8 sidor corpus exhibits the highest relative frequencies at level n+3, it is noteworthy that the frequencies are relatively high even at the lower levels. the level n score is the second highest frequency score for this corpus, and much higher when compared to the level n score of the standard corpus of the same genre, gp2d. the gp2d corpus had the highest average frequency at level n+7, indicating that the words used in this corpus are more specific than in the other corpora. however, it should be noted that this high relative frequency score is based on a relatively low number of words (40), and that this corpus also exhibit the frequency peak at level n+3. for suc3 and 8 sidor, the most frequent words are found at level n+3. this would mean that the more basic-level nouns could be found if we choose the superordinate words three levels above the original word. however, it could also indicate that the words at this level are higher up at rosch's vertical axis, thus being more inclusive than the basic-level words, and therefore more frequent (compare: shetland pony, horse, animal). when designing this study, we made a number of assumptions that can be discussed, such as the assumption of the nature of texts in simple swedish versus texts in standard swedish. we made the assumption, according to rosch's claims of basic-level terms, that the proportion of such constructions would be higher in the simple corpora. this assumption should be tested, for example by counting the relative frequencies of some base vocabulary list words @xcite in both corpora. the usage-based thesis of cognitive linguistics implies that we gain knowledge about the linguistic system by studying authentic language in use. to this background, it seems reasonable that a corpus study would be suitable for studying linguistic phenomena. however, there are some drawbacks of using such methods. one of the problems is that we worked with four very different corpora. can we really say that a corpus reflects authentic and direct language use? for example, one commonly mentioned measure in this context is frequency. a frequency measure can provide information on how commonly used certain linguistic constructions are. however, what we see clearly in this study is that if we compare corpora of different characteristics, the frequency measures will differ between corpora depending on text type. a corpus of medical texts will have frequent constructions that do not even exist in a corpus of children's literature. the same issue will probably be manifested if we compare texts of different linguistic activities, such as spoken language with written language. this means that the insights that we can draw of the cognitive processes underlying the studied linguistic phenomenon will be very specific to the kind of corpus that we study. to compare corpora, we must make sure that the corpora are comparable, and consider the factor of language use reflected in the texts of the corpora when generalising our findings to a larger context. ## conclusion the aim of this paper was to develop an algorithm for synonym replacement based on theories of basic-level nouns. we also presented results from a study exploring whether corpora of simple texts contain a higher proportion of basic-level nouns than corpora of standard swedish, and to see how this could be used in the context of lexical text simplification. we observed that the corpus of simple news text did indeed include more basic-level nouns than the corpus of standard news. this in turn shows that lexical simplification, through the use of base-level nouns, may benefit from traversing a word hierarchy upwards. this could serve as a complement to the often-used replacement methods that rely on word length and word frequency measures. we presented techniques for finding the best synonym candidate in a given word hierarchy, based on information about relative frequencies and monolexemity. we saw that all synonym replacement techniques, including the baseline methods, increased the number of monolexemic words and relative frequencies. the fm algorithm aimed to reward high relative frequencies and monolexemity, while not climbing the word hierarchy too high, and seems to perform well with respect to these criteria. future work includes further evaluation of this algorithm, and comparison to other synonym replacement strategies.