Automatic detection of unexpected/erroneous collocations in learner corpus (2020)

Abstract: This research investigates the collocational errors made by English learners in a learner corpus. It focuses on the extraction of unexpected collocations. A system was proposed and implemented with open source toolkit. Firstly, the collocation extraction module was evaluated by a corpus with manually annotated collocations. Secondly, a standard collocation list was collected from a corpus of native speaker. Thirdly, a list of unexpected collocations was generated by extracting candidates from a learner corpus and discarding the standard collocations on the list. The overall performance was evaluated, and possible sources of error were pointed out for future improvement.

## introduction multiword expressions (mwes) are word combinations which present lexical, syntactic, semantic, pragmatic or statistical idiosyncrasies. the boundary between mwes and collocations is subtle. in @xcite , they defined collocations as combinations of words whose idiosyncrasy is purely statistical and show no substantial semantic idiosyncrasy. in this way they oppose mwes to collocations. some researchers @xcite regard collocations as any statistically significant cooccurrences, which include all kinds of mwes. some other researchers @xcite consider collocations as a subset of mwes. for @xcite , collocation is a category of semantic phraseme. as defined by @xcite , a phraseme is a set of phrase which is not free (without freedom of selection of its signified and without freedom of combination of its components). in this sense, the meaning of phraseme is quite similar to mwe. in this research, we considered collocation as a subset of semantic phraseme and a subset of mwes as well. to constrain the set of collocation candidates, we focus on the verb-noun (vn) construction. second language learners usually have problems with collocations. some researchers have reported that the errors are related to the learners' l1 @xcite . the correction of wrong collocations foot_0 , such as to *create [construct] a taller and safer building, in written essays can help learners increase their competence and thus their proficiency in english writing @xcite . therefore, the automatic detection and correction of erroneous collocations would be helpful for learners. designing such a system would support specific feedback messages that could be employed to guide learners in their meta-cognitive learning processes @xcite . such a system may be based on two kinds of corpora: a learner corpus which is used to extract known collocational errors, and a reference corpus to extract standard english collocations @xcite . @xcite proposed a method of bilingual collocation extraction from a parallel corpus to provide phrasal translation memory. their system performance was exceptionally good (preci-@xmath0.98, @xmath1.91). however, this approach required a bilingual dictionary, a parallel corpus for a specific l1 and english, as well as word-alignment matching of translations. this paper presents a preliminary research on a learner corpus. in the following sections, we will briefly explain the method, present the results, and give some discussions. ## method we propose a system to extract unexpected collocations in three stages: (a) implementation and evaluation of a collocation extraction module; (b) collection of standard collocations from a native corpus; (c) extraction of wrong collocations from a learner corpus. the main principle is, firstly, to extract all possible collocations in the learner corpus, and then identify standard collocations by the reference (collocations extracted from native corpus); the remainder of the items are considered as wrong collocations. three evaluation points were made, aiming at the collocation extraction module, the reference of standard collocations, and the extraction of wrong collocations, respectively. the system diagram and the three stages are shown in figure 1 . stage c. extraction of wrong collocations: we used the implemented module to extract candidate collocations (named as the nucle list) from the national university of singapore corpus of learner english (nucle foot_3 ) @xcite . the sentences manually annotated with erroneous collocations (wci tag) were also exported, and the vn terms in these sentences were detected and saved in the nucle wc list. it was used to evaluate the overall performance of our system. the scripts foot_4 were written in python with natural language toolkit (nltk) foot_5 @xcite . five lexical association measures were used in collocation extraction tasks, namely the raw frequency counting, t-test, chi-square test, log likelihood ratio, and pointwise mutual information. the formulas as well as an evaluation of 84 measures can be found in @xcite . ## discussions and conclusion as our experiment configuration is capable to extract wrong collocations from the leaner corpus, the overall performance is not satisfactory. hence, we reviewed the results and point out some possible sources of errors for future studies. first, regarding the parseme corpus, the gold standard was built based on the lvc tag, so it may be that the verbs of the collocations were biased. in fact, 44 out of 85 collocations on the list were constructed only by five verbs, namely do, get, give, have, and take. therefore, the evaluation of the module was also biased. regarding the bnc list, we have reached a precision of 0.57 due to the large size of corpus (100 million words) and a strict selection (top 10 for each sub-directory of the bnc). however, comparing with a previous study @xcite which extracted 631,638 vn collocations from the bnc, we found that our standard collocation reference list (bnc ⋃ parseme lvc) was much smaller (@xmath2. regarding the nucle, because the part-of-speech (pos) and the lemma are not available, we used a pos tagger and a lemmatizer. yet, their performances were not evaluated, so the gold standard nucle wc list was not perfectly accurate. as for the whole system, it may be helpful to incorporate a word dependency parser module to identify the object noun which received the action of the verb. our approach has shown a method to detect erroneous collocations in learner english. as it relies on the accurate extraction of a reference list, our next step will consist in exploring larger corpora for extraction. such an extraction module would be of great benefit as part of a computer aided language learning system dedicated to the analysis of phraseology in learner texts.