VMWE discovery: a comparative analysis between Literature and T witter Corpora (2020)

Abstract: We evaluate manually five lexical association measurements as regards the discovery of Modern Greek verb multiword expressions with two or more lexicalised components usingmwetoolkit3 (Ramisch et al., 2010). We use Twitter corpora and compare our findings with previous work on fiction corpora. The results of LL, MLE and T-score were found to overlap significantly in both the fiction and the Twitter corpora, while the results of PMI and Dice do not. We find that MWEs with two lexicalised components are more frequent in Twitter than in fiction corpora and that lean syntactic patterns help retrieve them more efficiently than richer ones. Our work (i) supports the enrichment of the lexicographical database for Modern Greek MWEs’ IDION’ (Markantonatou et al., 2019) and (ii) highlights aspects of the usage of five association measurements on specific text genres for best MWE discovery results.

## introduction most prior work on mwe discovery exploits lexical association measures (ams). mwe discovery experiments on twitter corpora has investigated alternatives to ams. @xcite , who study arabic tweets, generate candidate mwes by storing bi-/tri-grams and their frequencies, filter them on the basis of the frequency of relevant query responses and ensure filtering validity with a confidence ratio measure that takes into account the number of distinct tweets in which a mwe appears. @xcite propose a language agnostic, graph based method for multilingual mwe extraction (bigrams) from twitter corpora, evaluate it on various datasets and yield promising results in a number of languages; in the case of english, however, ams outperform their method. prior work on verb mwes (vmwes) has focused on expressions with two lexicalised components @xcite namely to particle verbs and light verb constructions @xcite . reported work on modern greek vmwe discovery with the use of ams includes stripeli et al. (2020), who work with light verbs, and stamou et al. (2020), who work with all types of vmwes in fiction corpora. to the best of our knowledge, this is the first work on mining vmwes with 2 or more lexicalised components in modern greek twitter corpora (with lexicography as a goal). am performance has been found to be subject to corpus characteristics (size, type etc.) and to the type of target mwes (see the ams' evaluation literature with regards to vmwes, e.g., german and english verb+preposition mwes @xcite , verb particle mwes @xcite , english, portuguese and spanish verb+object mwes @xcite . since different ams seem to achieve best results on different corpora, mwe types and frequencies @xcite , combinations of ams have been found to be more efficient for discovery purposes @xcite . we use the mwetoolkit3 @xcite , and discuss the application of five ams, namely dice, log likelihood (ll) 1 , mle (maximum likelihood estimation), pmi (pointwise mutual information) and t-score, on twitter corpora. the tool offers a complete pipeline for mwe extraction where candidate phrases are created with n-grams or predefined linguistic patterns applied on corpora annotated for lemma and pos tags. in section 2 we present the details reported in @xcite that are important to the work discussed in this paper. in section 3 we present and evaluate three experiments. in section 4, the differences observed between the two datasets, namely twitter and fiction corpora, are discussed. in section 5 we present our conclusions and comment on the use of the ams for lexicographic purposes. ## experiments with fiction corpora of modern greek stamou et al. (2020) used mwetoolkit3 @xcite to discover vmwes in fiction corpora tagged and lemmatised with the ilsp tools @xcite . a gold standard (gs) was defined from these corpora by three expert annotators. two experiments were conducted, one with simple and one with enriched linguistic patters. in the first experiment, six syntactic patterns were applied comprising the most frequent pos sequences in the gs. the results were evaluated both manually and automatically. manual evaluation was applied to identify true and false positives in the first 3000 candidates returned by each of the following ams: dice, ll, mle, pmi, t-score. in the second experiment, richer patterns were used featuring double prepositional phrases and conjunction structures, as shown in the automatic evaluation of both experiments against the gs returned the following order of scores: tscore, mle, ll, dice, pmi while the manual evaluation showed dice as the most reliable am; pmi scored last in the first experiment and ll in the second. reliability was expressed as interanotator agreement computed with fleiss values. similar observations can be found in @xcite and @xcite for nominal modern greek mwes and basque vmwes respectively. in the first experiment, t-score, mle and ll shared about 850 out of the first 3000 phrases returned by each one of these ams. dice and pmi promoted less frequent vmwes not included in the other am results. these facts suggested that annotators tended to select hapax legomena, in contrast to automatic evaluation that relies on an, inevitably, incomplete gs. summing up stamou et al. ( 2020 ) work: (i) rich syntactic patterns enhanced vmwe discovery results (ii) manual evaluation supported the discovery of less frequent vmwes (iii) dice (the most reliable am in both experiments) and pmi (it returned vmwes not found by the other ams) should be applied for a more efficient vmwe discovery procedure @xcite . ## vmwes in fiction and twitter corpora the plots of the number of lexicalised components in the vmwes (figure 2 ) reveal that vmwes with two lexicalised components prevail in twitter corpora and vmwes with more than two lexicalised components in fiction corpora. this (not unexpected) fact may partly explain the results of our second experiment where lean patterns returned several new vmwes. the first two phrases below were extracted from the fiction corpora and the next two from the twitter corpus: