A different perspective on collocation extraction focuses not only on their
retrieval, but on semantically classifying the obtained collocations, in order to
make them more useful for NLP applications (Wanner et al. 2006; 2016).

Concerning the extraction of bilingual collocations, most works rely on par-
allel corpora to find the equivalent of a collocation in a target language. In this
regard, Smadja (1992) and Smadja et al. (1996) first identify monolingual colloca-
tions in English (the source language), and then use MUTUAL INFORMATION (MI)
and the DICE COEFFICIENT to find the French equivalents of the source colloca-
tions.

Kupiec (1993) also uses parallel corpora to find noun phrase equivalents be-
tween English and French. Their method consists of applying an expectation
maximization (EM) algorithm to previously extracted monolingual collocations.
Similarly, Haruno et al. (1996) obtain Japanese-English chunk equivalents by com-
puting their MI scores and taking into account their frequency and position in the
aligned corpora.

Another work which uses parallel corpora is presented by Wu & Chang (2003).
The authors extract Chinese and English n-grams from aligned sentences by com-
puting their LOG-LIKELIHOOD ratio. Then, the competitive linking algorithm is
used to decide whether each bilingual pair actually corresponds to a translation
equivalent.

Seretan & Wehrli (2007) took advantage of syntactic parsing to extract bilin-
gual collocations from parallel corpora. The strategy consists of first extracting
monolingual collocations using log-likelihood, and then searching for equiva-
lents of each base using bilingual dictionaries. The method also uses the position
of the collocation in the corpus, and relies on the syntactic analysis by assuming
that equivalent collocations will occur with the same syntactic relations within
the collocations in both languages.

Rivera et al. (2013) present a framework for bilingual collocation retrieval that
can be applied (using different modules) to both parallel and comparable corpora.
As in other works, monolingual collocations based on n-grams are extracted in a
first step, and then bilingual dictionaries (or WordNet, in the comparable corpora
scenario) are used to find the equivalents of the base in the aligned sentence or
in a small window of adjacent sentences of the source collocation.

A different approach, which uses non-related monolingual corpora for finding
bilingual collocations, was presented in Lü & Zhou (2004). Here, the authors
apply dependency parsing and the log-likelihood ratio for obtaining English and
Chinese collocations. Then, they search for translations using word translation
equivalents with the same dependency relation in the target language (using the
EM algorithm and a bilingual dictionary).
Although not focused on collocations, Fung (1998) applied methods based on distributional semantics to build bilingual lexica from comparable corpora. This approach takes into account that in this type of resources the position and the frequency of the source and target words are not comparable, and also that the translations of the source words might not exist in the target document.

Similarly, the strategy presented in this chapter leverages noisy parallel corpora for building bilingual word embeddings. However, with a view to applying it to other resources such as comparable corpora, it identifies equivalents without using information about the position of the collocations or their comparative frequency in the corpora. Furthermore, it does not take advantage of external resources such as bilingual dictionaries, making it easy to extend to other languages. Garcia et al. (2018) had introduced a naive version of this approach, including experiments in Portuguese and Spanish with just one collocation pattern.

# 3 A new method for bilingual collocation extraction

This section presents the proposed method for automatically extracting bilingual collocations from corpora. First, the approach for identifying candidates of monolingual collocations using syntactic dependencies is briefly described. Then, the process of creating the bilingual word embeddings is shown, followed by the strategy for discovering the collocation equivalents between languages.

## 3.1 Monolingual dependency-based collocation extraction

Early works on n-gram based collocation extraction already pointed out the need for syntactic analysis to better identify collocations in corpora (Smadja 1993; Lin 1999). Syntactic analysis can, on the one hand, avoid the extraction of syntactically unrelated words which occur in small context windows. On the other hand, it can effectively establish a relation between lexical items occurring in long-distance dependencies (Evert 2008).

Besides, the method presented in this chapter assumes that most bilingual equivalents of collocations bear the same syntactic relation in both the source and the target languages, although it is not always the case (Lü & Zhou 2004).

In order to better capture the syntactic relations between the base and the collocate of each collocation, the strategy uses state-of-the-art dependency parsing. Apart from that, and aimed at obtaining harmonized syntactic information between languages, the method relies on Universal Dependencies annotation,
which makes it possible to use the same strategy for extracting and analyzing
the collocations in multiple languages.

**3.1.1 Preprocessing:**

Before extracting the collocation candidates from each corpus, a pipeline of NLP tools is applied in order to annotate the text with the desired information. Thus, the output of this process consists of a parsed corpus in CoNLL-U format,<fnref n="3" /> where each word is assigned to its surface form, its lemma, its POS-tag and morphosyntactic features, its syntactic head as well as the UD relation of the word in context.

From this analyzed corpus, the word pairs belonging to the desired relations (collocation candidates) are extracted. We keep their surface forms, POS-tags, and other syntactic dependents which may be useful for the identification of potential collocations. Besides, a list of triples is retained in order to apply association measures, containing (i) the syntactic relation, (ii) the head, and (iii) the dependent (using their lemmas together with the POS-tags). Thus, from a sentence such as *John took a great responsibility*, the following triples (among others) are obtained:

NSUBJ(take<sub>VERB</sub>,John<sub>PROP</sub>)
AMOD(responsibility<sub>NOUN</sub>,great<sub>ADJ</sub>)
DOBJ(take<sub>VERB</sub>,responsibility<sub>NOUN</sub>)

This information, along with the corpus size and the frequency of the different elements of the potential collocations, is stored in order to rank the candidates.

**3.1.2 Collocation patterns:**

In this chapter, candidates of five different syntactic patterns of collocations are extracted in three languages, Spanish (ES), Portuguese (PT), and English (EN):<fnref n="4" />

* Adjective—Noun (AMOD): these candidates are pairs of adjectives (as collocates) and nouns (as bases) where the former syntactically depends of the latter in a AMOD relation. Example: **killer<sub>base</sub>;serial<sub>collocate</sub>**.

* Noun—Noun (NMOD): this pattern consists of two common nouns related by the NMOD relation, where the head is the base and the dependent is

<footnote n="3">³http://universaldependencies.org/format.html</footnote>

<footnote n="4">⁴In this chapter we address the European variety of Portuguese. However, even if we use a European Portuguese corpus (see §4), it contains some texts in the Brazilian dialect.</footnote>
the collocate (optionally with a **CASE** marking dependent preposition: **of** in English, **de** in Portuguese and Spanish). Example: **rage**<sub>base</sub>; **fit**<sub>collocate</sub>.<sup>5</sup>

* Verb–Object (VOBJ): *verb-object* collocations consist of a verb (the collocation) and a common noun (the base) occurring in a DOBJ relation. Example: **care**<sub>base</sub>; **take**<sub>collocate</sub>.

* Subj–Verb (VSUBJ): the VSUBJ collocation pattern contains a common noun (the base, acting as a subject) and the verb it depends on (the collocate). Example: **ship**<sub>base</sub>; **sink**<sub>collocate</sub>.

* Verb–Adverb (ADVMOD): in this case, a collocate adverb modifies a verb (the base) in an ADVMOD relation. Example: **want**<sub>base</sub>; **really**<sub>collocate</sub>.

### 3.1.3 Identification of candidates:

For each of the five patterns of collocations, a list of potential candidates for the three languages is extracted. After that, the candidates are ranked using standard association measures that have been widely used in collocation extraction (Evert 2008).

In the current experiments, two statistical measures were selected, whose results complement each other: **T-SCORE**, which prefers frequent dependency pairs, and has been proved useful for collocation extraction (Krenn & Evert 2001), and **MUTUAL INFORMATION**, which is useful for a large corpus, even if it tends to assign high scores to candidates with very low-frequency (Pecina 2010).

The output of both association measures is merged in a final list for each language and collocation pattern, defining thresholds of T-SCORE≥2 and MI≥3 (Stubbs 1995), and extracting only collocations with a frequency of f≥10. This large value was defined to reduce the extraction of incorrect entries from a noisy corpus and from potential errors of the automatic analysis.

It must be noted that, since these lists of monolingual collocations have been built based on statistical measures of collocability, their members need not be *bona fide* collocations in the phraseological meaning. Thus, the lists can include idioms, e.g., *kick the bucket*, quasi-idioms, e.g., *big deal*, (Mel'čuk 1998), or free combinations, e.g., *buy a drink*.

<footnote n="5">⁵Some collocations belonging to this pattern are analyzed in UD – mainly in English – using the **COMPOUND** relation. These are not extracted in the experiments performed in this chapter.</footnote>
## 3.2 Bilingual word embeddings

Word embeddings are low-dimensional vector representations of words which capture their distributional context in corpora. Even though distributional semantics methods have been largely used in previous years, approaches based on word embeddings gained popularity with the publication of *word2vec* (Mikolov et al. 2013). Based on the *Skip-gram* model of *word2vec*, Luong et al. (2015) proposed *BiSkip*, a model of word embeddings which learns bilingual representations using aligned corpora, thus being able to predict words crosslinguistically.

The method presented in this chapter uses lemmas instead of surface forms to identify the collocation candidates, so the bilingual models of word embeddings are also trained on lemmatized corpora. Therefore, the raw parallel corpus is lemmatized keeping the original sentence alignment.

The bilingual models are built using *MultiVec*, an implementation of *word2vec* and *BiSkip* (Berard et al. 2016). As the approach is evaluated in three languages, three different bilingual models are needed: Spanish-English, Portuguese-English, and Spanish-Portuguese.

As it will be shown, the obtained models can predict the similarity between words in bilingual scenarios by computing the cosine similarity between their vectors. As the models learn the distribution of single words (lemmas), they deal with different semantic phenomena such as polysemy or homonymy. Concerning collocations, this means that, ideally, the bilingual models could predict not only the equivalents of a base, but also to capture the (less close) semantic relation between the bilingual collocates, if they occur frequently enough in the data.

## 3.3 Bilingual collocation alignment

In order to identify the bilingual equivalent of a collocation in a target language, the method needs (i) lists of monolingual collocations (ideally obtained from similar resources), and (ii) a bilingual *source-target* model of word embeddings.

With these resources, the following strategy is applied: For each collocation in the source language (e.g., *lío<sub>base</sub> tremendo<sub>collocate</sub>* 'huge mess' in Spanish) the system selects its base and obtains – using the bilingual model – the *n* most similar lemmas in the target language (where *n=5* in the experiments performed in this chapter): *trouble*, *mess*, etc. Then, starting from the most similar lemma, we search in the target list for collocations containing the equivalents of the base (*trouble*<sub>base</sub> *little*<sub>collocate</sub>, *trouble*<sub>base</sub> *deep*<sub>collocate</sub>, *mess*<sub>base</sub> *huge*<sub>collocate</sub>, *mess*<sub>base</sub> *fine*<sub>collocate</sub>, etc.). If a collocation with a base equivalent is found, the cosine similarity between both collocates (*tremendo* versus *little*, *deep*, *huge*, and *fine*)
is computed, and they are selected as potential candidates if their similarity is
higher than a given threshold (empirically defined in this chapter as 0.65), and if
the target candidate is among the *n* most similar words of the source collocate
(again, *n*=5). Finally, if these conditions are met, the source and target colloca-
tions are aligned, assigning the average distance between the bases and the col-
locates as a confidence value, as in the following Spanish-English example: *líοbase
tremendo<sub>collocate</sub> = mess<sub>base</sub> huge<sub>collocate</sub> → 0.721.*

4 Evaluation

This section presents the experiments carried out in order to evaluate the pro-
posed distributional method (henceforth DiS) in the three analyzed languages,
using the five collocation patterns defined in §3.1. The approach presented in
this chapter is compared to a baseline system (BAS), which uses hand-crafted
bilingual dictionaries, and to probabilistic translation dictionaries (NAT).<fnref n="6" />

**Corpora:** Monolingual collocations were extracted from a subset of the Open-Subtitles2016 corpus (Lison & Tiedemann 2016), which contains parallel corpora from TV and Movie subtitles. This resource was selected because it is a large and multilingual parallel corpus likely to contain different types of collocations, also from an informal register, thus being useful for comparative studies.<fnref n="7" />

From the English, Spanish, and Portuguese corpora, those senteces which ap-
pear in the three languages were selected, for a total of 13,017,016 sentences. These
sentences were tokenized, lemmatized and POS-tagged with a multilingual NLP
pipeline (Garcia & Gamallo 2015), obtaining three corpora of about 91M (ES and
PT), and about 98M (EN) tokens. The resulting data were enriched with syntactic
annotation using statistical models trained with MaltParser (Nivre et al. 2007) on
version 1.4 of the UD treebanks (Nivre et al. 2016).

**Collocations:** From each corpus, five patterns of collocation candidates were extracted: AMOD, NMOD, VOBJ, VSUBJ, and ADVMOD. For each language and pattern, a single list of collocations was obtained by merging the MI and T-SCORE outputs as explained in §3.1. Table 1 shows the number of filtered collocations in each case (colls).

<footnote n="6">⁶The extractions of these three methods are available at http://www.grupolys.org/~marcos/pub/pmwe-dis.tar.bz2</footnote>

<footnote n="7">⁷Note, however, that OpenSubtitles2016 includes non-professional translations with some noisy elements such as typos or case inconsistencies, among others.</footnote>
<table><caption>Table 1: Number of unique input dependencies for each syntactic pattern (*deps*), and final monolingual collocation candidates (*colls*).</caption><thead><tr><th rowspan="2">Lg</th><th colspan="2">AMOD</th><th colspan="2">NMOD</th><th colspan="2">VOBJ</th><th colspan="2">VSUBJ</th><th colspan="2">ADVMOD</th></tr><tr><th>deps</th><th>colls</th><th>deps</th><th>colls</th><th>deps</th><th>colls</th><th>deps</th><th>colls</th><th>deps</th><th>colls</th></tr></thead><tbody><tr><td>ES</td><td>373K</td><td>13,870</td><td>644K</td><td>5,673</td><td>423K</td><td>17,723</td><td>287K</td><td>4,914</td><td>124K</td><td>5,526</td></tr><tr><td>PT</td><td>361K</td><td>12,967</td><td>709K</td><td>5,643</td><td>544K</td><td>20,984</td><td>283K</td><td>3,927</td><td>142K</td><td>6,660</td></tr><tr><td>EN</td><td>381K</td><td>14,175</td><td>517K</td><td>3,133</td><td>483K</td><td>15,492</td><td>264K</td><td>2,663</td><td>162K</td><td>6,711</td></tr></tbody></table>

Another version of each corpus was created only with the lemma of each token, keeping the original sentence alignments. These corpora were used for training three bilingual word embeddings with MultiVec, with 100 dimensions and a window-size of eight words: ES-EN, ES-PT, and PT-EN.<fnref n="8" />

**Baseline (BAS):** The performance of the method described in §3.3 was compared to a baseline which follows the same strategy, but uses bilingual dictionaries instead of the word embeddings models. Thus, the BAS method obtains the equivalents of both the base and the collocate of a source collocation, and verifies whether there is a target collocation with the translations. The bilingual dictionaries provided by the *apertium* project were used for these experiments (Forcada et al. 2011).<fnref n="9" />

The Spanish-Portuguese dictionary has 14,364 entries, and the Spanish-English one contains 34,994. The Portuguese-English dictionary (not provided by *aper-tium*) was automatically obtained by transitivity from the two other lexica, with a size of 9,160 pairs.

**Probabilistic translation dictionaries (NAT):** The distributional method was also compared to probabilistic translation dictionaries. Probabilistic dictionaries are bilingual resources which contain, for each word in a source language, possible translations in the target language together with the probability of the translation being correct. To obtain these dictionaries NATools was used, which is a set of tools to work with parallel corpora that can be utilized for different tasks such as sentence and word alignment, or to extract bilingual translation dictionaries by means of statistical methods (Simões & Almeida 2003). The probabilistic dictionaries are obtained by applying the EM algorithm on sparse matrices of

<footnote n="8">⁸These models are available at http://www.grupolys.org/~marcos/pub/mwe17_models.tar.bz2</footnote>

<footnote n="9">⁹SVN revision 75,477, https://svn.code.sf.net/p/apertium/svn/</footnote>
bilingual word co-occurrences, previously built from parallel corpora (Hiemstra 1998).

For a better comparison to the DIS model, NAT dictionaries were extracted from the same lemmatized resources used for training the bilingual word embeddings. Thus, this method only differs from DIS in the bilingual resources used to search for equivalents of the bases and the collocates.<fnref n="10" />

## 4.1 Results

With a view to knowing the performance of BAS, NAT, and DIS in the different scenarios, 100 bilingual collocation pairs were randomly selected from each language and pattern, creating a total of 45 lists (15 from each of the three methods).<fnref n="11" />

Three reviewers worked during the evaluation process. Each bilingual collocation pair was labeled as (i) correct, (ii) incorrect, or (iii) dubious, which includes pairs where the translation might be correct in some contexts even if they were not considered faithful translations.<fnref n="12" /> Correct collocation equivalents are those pairs where the monolingual extractions were considered correct, both in terms of co-occurrence frequency and of collocation pattern classification, and whose translations were judged by the reviewers as potential translations in a real scenario. Two reviewers labeled each collocation pair in the BAS and DIS outputs, achieving 92% and 83% inter-annotator agreement, respectively, with an average $κ = 0.39$, which indicates the difficulty of this kind of annotation. Pairs with correct/incorrect disagreement were discarded for the evaluation. Those with at least one dubious label were checked by a third annotator, deciding in each case whether they were correct, incorrect, or dubious. This third annotator evaluated the outputs of NAT using exactly the same guidelines.

From these data, the precision values for each case were obtained by dividing the number of correct collocation equivalents by the number of correct, incorrect, and dubious cases (so dubious cases were considered incorrect). Recall ($r$) was obtained by multiplying the precision values ($p$) for the number of extracted equivalents ($e$), and dividing the result by the lowest number of input collocations for each pair ($i$, see Table 1). For instance, the Spanish-Portuguese baseline

<footnote n="10">¹⁰After preliminary evaluations, the translation probability thresholds of both lexical units were empirically defined as 0.1.</footnote>

<footnote n="11">¹¹Except for baseline extractions with less than 100 elements, where all of them were selected.</footnote>

<footnote n="12">¹²Some of these dubious equivalents are actual translations in the original corpus, such as the Spanish-English *copa de champaña* ‘champagne cup’, which was translated as *cup of wine*, even if they are semantically different.</footnote>
<table><caption>Table 2: Number of bilingual extractions of the baseline, NAT, and DIS systems.</caption><thead><tr><th>Pattern</th><th>model</th><th>ES-PT</th><th>ES-EN</th><th>PT-EN</th></tr></thead><tbody><tr><td rowspan="3">AMOD</td><td>BAS</td><td>657</td><td>248</td><td>213</td></tr><tr><td>NAT</td><td>1,329</td><td>1,113</td><td>1,005</td></tr><tr><td>DIS</td><td>9,464</td><td>7,778</td><td>7,083</td></tr><tr><td rowspan="3">NMOD</td><td>BAS</td><td>320</td><td>32</td><td>43</td></tr><tr><td>NAT</td><td>704</td><td>138</td><td>136</td></tr><tr><td>DIS</td><td>3,867</td><td>890</td><td>917</td></tr><tr><td rowspan="3">VOBJ</td><td>BAS</td><td>529</td><td>183</td><td>241</td></tr><tr><td>NAT</td><td>1,443</td><td>1,461</td><td>1,544</td></tr><tr><td>DIS</td><td>12,887</td><td>8,865</td><td>9,206</td></tr><tr><td rowspan="3">VSUBJ</td><td>BAS</td><td>188</td><td>27</td><td>55</td></tr><tr><td>NAT</td><td>382</td><td>346</td><td>323</td></tr><tr><td>DIS</td><td>2,522</td><td>1,344</td><td>1,298</td></tr><tr><td rowspan="3">ADVMOD</td><td>BAS</td><td>58</td><td>19</td><td>22</td></tr><tr><td>NAT</td><td>113</td><td>104</td><td>106</td></tr><tr><td>DIS</td><td>3,721</td><td>2,301</td><td>2,412</td></tr></tbody></table>

recall for the AMOD pattern was estimated as follows (see Table 1, Table 2, and Table 3): $r = \frac{p \cdot e}{i} = \frac{99 \cdot 657}{12,967} = 5.01$.<fnref n="13" /> Finally, f-score values (the harmonic mean between precision and recall) were obtained for each case, and the macro-average results were calculated for each language, pattern, and approach.

Table 2 contains the number of bilingual collocation equivalents extracted by each method in the 15 settings from the input lists of monolingual data (Table 1). These results clearly show that the baseline approach extracts a lower number of bilingual equivalents. NAT obtains much more bilingual collocations than BAS, but both methods extract less equivalents than the distributional approach. This might have happened due to the size of the dictionaries in BAS and because of

<footnote n="13">¹³Note that these recall results assume that every collocation in the shortest input list of each pair has an equivalent on the other language, which is not always the case. Thus, more realistic recall values (which would need an evaluation of every extracted pair) will be higher than the ones obtained in these experiments.</footnote>
<table><caption>Table 3: Precision, recall and f-score of the baseline (BAS) system (*average* is macro-average).</caption><thead><tr><th>Pattern</th><th></th><th>ES-PT</th><th>ES-EN</th><th>PT-EN</th><th>average</th></tr></thead><tbody><tr><td rowspan="3">AMOD</td><td>P</td><td>99.0</td><td>95.8</td><td>97.9</td><td>97.6</td></tr><tr><td>R</td><td>5.0</td><td>1.7</td><td>1.6</td><td>2.8</td></tr><tr><td>F1</td><td>9.6</td><td>3.4</td><td>3.2</td><td>5.4</td></tr><tr><td rowspan="3">NMOD</td><td>P</td><td>97.8</td><td>100</td><td>91.7</td><td>96.5</td></tr><tr><td>R</td><td>5.5</td><td>1.0</td><td>1.3</td><td>2.6</td></tr><tr><td>F1</td><td>10.5</td><td>2.0</td><td>2.5</td><td>5.1</td></tr><tr><td rowspan="3">VOBJ</td><td>P</td><td>98.7</td><td>100</td><td>92.1</td><td>96.9</td></tr><tr><td>R</td><td>3.0</td><td>1.2</td><td>1.4</td><td>1.9</td></tr><tr><td>F1</td><td>5.7</td><td>2.3</td><td>2.8</td><td>3.6</td></tr><tr><td rowspan="3">VSUBJ</td><td>P</td><td>93.8</td><td>96.3</td><td>92.7</td><td>94.3</td></tr><tr><td>R</td><td>4.5</td><td>1.0</td><td>1.9</td><td>2.5</td></tr><tr><td>F1</td><td>8.6</td><td>1.9</td><td>3.8</td><td>4.8</td></tr><tr><td rowspan="3">ADVMOD</td><td>P</td><td>96.7</td><td>100</td><td>95.7</td><td>97.4</td></tr><tr><td>R</td><td>1.0</td><td>0.3</td><td>0.3</td><td>0.6</td></tr><tr><td>F1</td><td>2.0</td><td>0.7</td><td>0.6</td><td>1.1</td></tr><tr><td rowspan="3">average</td><td>P</td><td>97.2</td><td>98.4</td><td>94.0</td><td>96.5</td></tr><tr><td>R</td><td>3.8</td><td>1.0</td><td>1.3</td><td>2.1</td></tr><tr><td>F1</td><td>7.3</td><td>2.1</td><td>2.6</td><td>4.0</td></tr></tbody></table>

the internal properties of the collocations in both BAS and NAT, where the collocates may not be direct translations of each other. Moreover, with all three strategies, the bilingual extractions including English are smaller than the Spanish-Portuguese ones.

Concerning the performance of the three approaches, Table 3 (BAS), Table 4 (NAT), and Table 5 (DIS) contain the precision, recall and f-score for each language pair and collocation pattern. BAS obtains high-precision results for every language and collocation pattern (91.7% in the worst scenario), with a macro-average value of 96.5%. These results are somehow expected due to the quality of the hand-crafted dictionaries. However, because of the poor recall numbers, the general performance of BAS is low, achieving F-scores around 4%. Interest-
<table><caption>Table 4: Precision, recall and f-score of the probabilistic (NAT) system (*average* is macro-average).</caption><thead><tr><th>Pattern</th><th></th><th>ES-PT</th><th>ES-EN</th><th>PT-EN</th><th>average</th></tr></thead><tbody><tr><td rowspan="3">AMOD</td><td>P</td><td>92.5</td><td>92.5</td><td>83.3</td><td>89.5</td></tr><tr><td>R</td><td>9.5</td><td>7.4</td><td>6.5</td><td>7.8</td></tr><tr><td>F1</td><td>17.2</td><td>13.8</td><td>12.0</td><td>14.3</td></tr><tr><td rowspan="3">NMOD</td><td>P</td><td>91.1</td><td>98.7</td><td>91.4</td><td>93.7</td></tr><tr><td>R</td><td>11.4</td><td>4.4</td><td>4.0</td><td>6.6</td></tr><tr><td>F1</td><td>20.2</td><td>8.3</td><td>7.6</td><td>12.1</td></tr><tr><td rowspan="3">VOBJ</td><td>P</td><td>95.2</td><td>80.0</td><td>92.7</td><td>89.3</td></tr><tr><td>R</td><td>7.8</td><td>7.5</td><td>9.2</td><td>8.2</td></tr><tr><td>F1</td><td>14.3</td><td>13.8</td><td>16.8</td><td>15.0</td></tr><tr><td rowspan="3">VSUBJ</td><td>P</td><td>82.4</td><td>78.6</td><td>79.2</td><td>80.0</td></tr><tr><td>R</td><td>8.0</td><td>10.2</td><td>9.6</td><td>9.3</td></tr><tr><td>F1</td><td>14.6</td><td>18.1</td><td>17.1</td><td>16.6</td></tr><tr><td rowspan="3">ADVMOD</td><td>P</td><td>59.2</td><td>78.8</td><td>83.3</td><td>73.8</td></tr><tr><td>R</td><td>1.2</td><td>1.5</td><td>1.3</td><td>1.3</td></tr><tr><td>F1</td><td>2.4</td><td>2.9</td><td>2.6</td><td>2.6</td></tr><tr><td rowspan="3">average</td><td>P</td><td>84.1</td><td>85.7</td><td>86.0</td><td>85.3</td></tr><tr><td>R</td><td>7.6</td><td>6.2</td><td>6.1</td><td>6.6</td></tr><tr><td>F1</td><td>13.8</td><td>11.4</td><td>11.2</td><td>12.1</td></tr></tbody></table>

ingly, the size of the dictionary does not seem crucial to the results of the baseline. In this respect, the Spanish-Portuguese results are much better, especially in terms of recall, than Spanish-English, whose dictionary is more than twice as large. Also, the Portuguese-English results are slightly better than the Spanish-Portuguese ones, the latter being obtained using a dictionary built by transitivity.

The use of probabilistic translation dictionaries (NAT) increases the recall by a factor of more than three when compared to the baseline, but with a cost in precision, which drops, in average, from 96.5% to 85.3%. However, these differences allow the NAT approach to obtain much better F-scores than BAS. When looking at the different collocation patterns, it is worth noting that while AMOD, NMOD, and VOBJ have precision values of about 90%, VSUBJ, and especially ADV-
<table><caption>Table 5: Precision, recall and f-score of DIS system (*average* is macro-average).</caption><thead><tr><th>Pattern</th><th></th><th>ES-PT</th><th>ES-EN</th><th>PT-EN</th><th>average</th></tr></thead><tbody><tr><td rowspan="3">AMOD</td><td>P</td><td>92.9</td><td>92.0</td><td>90.5</td><td>91.8</td></tr><tr><td>R</td><td>67.8</td><td>51.6</td><td>49.5</td><td>56.3</td></tr><tr><td>F1</td><td>78.4</td><td>64.3</td><td>64.0</td><td>68.9</td></tr><tr><td rowspan="3">NMOD</td><td>P</td><td>93.8</td><td>88.0</td><td>90.0</td><td>90.6</td></tr><tr><td>R</td><td>64.3</td><td>25.0</td><td>26.3</td><td>38.5</td></tr><tr><td>F1</td><td>76.3</td><td>38.9</td><td>40.1</td><td>51.9</td></tr><tr><td rowspan="3">VOBJ</td><td>P</td><td>90.1</td><td>84.0</td><td>83.9</td><td>86.2</td></tr><tr><td>R</td><td>66.0</td><td>48.1</td><td>49.9</td><td>54.7</td></tr><tr><td>F1</td><td>76.5</td><td>61.2</td><td>62.6</td><td>66.7</td></tr><tr><td rowspan="3">VSUBJ</td><td>P</td><td>80.3</td><td>81.2</td><td>74.1</td><td>78.5</td></tr><tr><td>R</td><td>51.6</td><td>41.0</td><td>36.1</td><td>42.9</td></tr><tr><td>F1</td><td>62.8</td><td>54.5</td><td>48.6</td><td>55.3</td></tr><tr><td rowspan="3">ADVMOD</td><td>P</td><td>77.6</td><td>83.3</td><td>67.4</td><td>76.1</td></tr><tr><td>R</td><td>52.2</td><td>34.7</td><td>24.4</td><td>37.1</td></tr><tr><td>F1</td><td>62.4</td><td>49.0</td><td>35.8</td><td>49.1</td></tr><tr><td rowspan="3">average</td><td>P</td><td>86.9</td><td>85.7</td><td>81.2</td><td>84.6</td></tr><tr><td>R</td><td>60.4</td><td>40.1</td><td>37.3</td><td>45.9</td></tr><tr><td>F1</td><td>71.3</td><td>53.6</td><td>50.2</td><td>58.4</td></tr></tbody></table>

MOD (also with very low recall values) do not surpass 80% (with one case, ES-PT, with < 60%). As it will be shown in §4.2, some preprocessing issues might be the source of the some errors of ADVMOD extractions.

As for the DIS model, its precision is again lower than BAS and very similar to the NAT approach, with average results of 84.6%. However, the distributional strategy finds much more bilingual equivalents than the dictionaries, so recall values increase to an average of more than 45%. Again, VSUBJ and ADVMOD show worse precision values than the other three patterns. Besides, the NMOD extractions of the pairs including English have very low recall when compared to the other results. This might be due to not extracting nouns analyzed as COMPOUND (§3.1). As for the other two methods, the DIS Spanish-Portuguese results are bet-
ter than the two other language pairs, so the linguistic distance seems to play an
important role in bilingual collocation extraction.

The method proposed in this chapter assigns a confidence value (obtained from
the cosine similarity between the vectors of the base and the collocate equiva-
lents) to each bilingual pair of collocations. In this respect, Figure 1 plots the av-
erage performance and confidence curves versus the total number of extracted
pairs. This figure shows that by using a high confidence value (> 90%), it is pos-
sible to extract about 40, 000 bilingual pairs with a high degree of precision. Be-
sides, filtering the extraction with confidence values higher than 90% does not
increase the precision of the system. This suggests that the errors produced in
the most confident pairs arise due to factors other than semantic similarity, such
as different degrees of compositionality.

However, as the confidence value decreases, the precision of the extraction
also gets worse, despite the rise in the number of extractions which involves
higher recall and consequently better f-score.

Finally, all the bilingual collocations extracted by DIS were merged into a sin-
gle list with the three languages, thus obtaining new bilingual equivalents (not ex-
tracted directly by the system) by transitivity.<fnref n="14" /> This final multilingual resource
has 74, 942 entries, 38, 629 of them with translations in all three languages.

## 4.2 Error analysis

The manually annotated lists of bilingual collocations were used to perform an
error analysis of the DIS system. These errors were classified in five types depend-
ing on their origin. Table 6 contains, for each error type, the macro-average rates
of each collocation pattern as well as the final distribution of the error typology.

1. **Bilingual model (BiModel):** Though useful, the bilingual word embedding approach produces some errors such as the identification of antonyms that have a similar distribution, which can align opposite collocation equivalents, such as the Portuguese-English pair *tecido<sub>base</sub> vivo<sub>collocate</sub> = tissue<sub>base</sub> dead<sub>collocate</sub>*, instead of *living tissue*, where the extracted equivalent of the collocate *vivo* ('living' - in this context - or 'alive', in Portuguese) was *dead*. In most cases, however, the system obtained similar (but not synonymous) collocations, such as *chá<sub>base</sub> preto<sub>collocate</sub>* 'black tea' in Portuguese aligned to *coffee<sub>base</sub> black<sub>collocate</sub>* 'black coffee' in English.

<footnote n="14">¹⁴The merging process obtained 6,969 new bilingual collocation equivalents not present in the original extractions, and it also includes more than one translation for some collocations.</footnote>
<figure><img src="image_60.png" /><figcaption>Figure 1: Average precision, recall, f-score, and confidence curves (from 0 to 1) versus total number of extractions of the DIS model.</figcaption></figure>

2. **Monolingual extraction (MonoExtract):** The extraction of base and collocate pairs produced incorrect collocations such as *plan*<sub>base</sub> figure<sub>collocate</sub>, instead of obtaining the phrasal verb *figure out* as collocate.

3. **Preprocessing (NLP):** Several errors derived from issues produced by the NLP pipeline, such as POS-tagging or dependency parsing: e.g., *pain*<sub>Noun</sub>, *end*<sub>Verb</sub> was labeled as *DOBJ* (instead of *NSUBJ*). A special case of preprocessing errors was the analysis of some Portuguese and Spanish adverbs ending in –*mente* (*-ly* adverbs in English), whose suffix was wrongly removed during the extraction process: e.g. *brutalmente* ‘brutally’ → *brutal*. These issues – which can be easily corrected – caused the alignment of incorrect Spanish and Portuguese collocations with English candidates, such as the Portuguese-English pair *matar*<sub>base</sub> *brutal*<sub>collocate</sub> = *kill*<sub>base</sub> *brutally*<sub>collocate</sub> instead of *matar*<sub>base</sub> *brutalmente*<sub>collocate</sub> = *kill*<sub>base</sub> *brutally*<sub>collocate</sub>. This was the main source of errors of the *ADVMOD* relation.

4. **Lemmatization and gender (Gender):** The lemmatization of some words differs from language to language, so working with lemmas instead of tokens also might involve some errors. For instance, the Spanish word *hija* ‘daughter’ is lemmatized as *hijo* ‘son’ (also in Portuguese: *filha*, *filho*),
while in English *son* and *daughter* appear as different entries. Thus, some bilingual collocations differ in the gender of their bases, such as the Spanish-English pair *hijo<sub>base</sub> encantador<sub>collocate</sub> = daughter<sub>base</sub> lovely<sub>collocate</sub>* instead of *hijo<sub>base</sub> encantador<sub>collocate</sub>=son<sub>base</sub> lovely<sub>collocate</sub>*.

5. **Other errors (Other):** Some other errors were caused by mixed languages in the original corpus. For example, the verb form *are*, in English, was analyzed as a form of the verb *arar* 'to plow' in Spanish. Some errors also arose from noise and misspellings in the corpora (proper nouns with lowercase letters, etc.).

It is worth mentioning that, in general, the error type distribution was similar across the different collocation patterns, showing much higher variation between different patterns of the same language pair. For instance, the distribution of Spanish-English *AMOD* errors is similar to the Portuguese-English *AMOD* one, while the typology of the Spanish-Portuguese *NMOD* errors is different to those of Spanish-Portuguese *AMOD* equivalents.

<table><caption>Table 6: Error rate of each of the defined error types of DIS system (*average* is macro-average).</caption><thead><tr><th>Type</th><th>AMOD</th><th>NMOD</th><th>VOBJ</th><th>VSUBJ</th><th>ADVMOD</th><th>average</th></tr></thead><tbody><tr><td>BiModel</td><td>70.57</td><td>93.52</td><td>59.23</td><td>45.74</td><td>32.61</td><td>60.33</td></tr><tr><td>MonoExtract</td><td>0</td><td>0</td><td>21.43</td><td>21.85</td><td>44.94</td><td>17.64</td></tr><tr><td>NLP</td><td>8.34</td><td>0</td><td>16.96</td><td>11.48</td><td>20.49</td><td>11.45</td></tr><tr><td>Gender</td><td>21.10</td><td>2.78</td><td>2.38</td><td>19.07</td><td>0</td><td>9.07</td></tr><tr><td>Other</td><td>0</td><td>3.70</td><td>0</td><td>1.85</td><td>1.96</td><td>1.50</td></tr></tbody></table>

Among the different errors produced by the presented method, an interesting case are *incongruent* collocations (Nesselhauf 2003). These expressions are those where the translation of both elements is not coherent, such as the English-Portuguese pair *requirement<sub>base</sub> meet<sub>collocate</sub> = condição<sub>base</sub> cumprir<sub>collocate</sub>* in which the verb *to meet* is usually translated into Portuguese as *conhecer*, not as *cumprir*. For these collocation equivalents to be correctly extracted by our method, they should appear with some frequency in the training corpus, which is not always the case. This fact may lead us to explore new compositional models, aimed at learning the distribution of the whole collocation, and not of its constituents, in further work.
## 4.3 Comparable corpora

A final experiment was carried out in order to find out (i) whether the bilingual word embeddings – trained on the same parallel corpora as those used for extracting the collocations – could be successfully applied to align collocations obtained from different resources, and (ii) the performance of the proposed method on comparable corpora.

Therefore, the same strategy for monolingual collocation extraction was applied in the Spanish and Portuguese *Wikipedia Comparable Corpus 2014*.<fnref n="15" /> Then, we calculated the semantic similarity between the collocations using the same word embedding models as in the previous experiments.

From these corpora, filtered lists of 89, 285 and 140, 900 candidate collocations in Portuguese and Spanish were obtained, from 140M, and 80M of tokens respectively. From the 59, 507 bilingual collocations obtained by the DiS approach, 150 Spanish-Portuguese pairs were randomly selected and evaluated.

The precision of the extraction was 87.25%, with a recall of 58.15% (again computed using the whole set of monolingual collocations), and 69.79% f-score. These results are in line with those obtained on the OpenSubtitles Spanish-Portuguese pair (about 2% lower), so the method works well on different corpora and domains. It is worth noting that 49, 259 of the extracted collocation equivalents (83%) had not been retrieved from the OpenSubtitles corpus.

This last experiment shows that (i) the bilingual word embeddings can be used to identify collocation equivalents in different corpora than those used for training, and that (ii) they can also be applied to corpora of different domains to obtain previously unseen multilingual collocations.

## 5 Conclusions

This chapter presents a new strategy to automatically discover multilingual collocation equivalents from both parallel and comparable corpora. First, monolingual collocation candidates of five different patterns are extracted using syntactic analysis provided by harmonized UD annotation, together with a combination of standard association measures. Besides, bilingual word embeddings are trained on lemmatized parallel corpora. These bilingual models are then used to find distributional equivalents of both the base and the collocate of each source collocation in the target language.

The performed experiments, using noisy parallel corpora in three languages, showed that the proposed method achieves an average precision of about 85%,

<footnote n="15">¹⁵http://linguatools.org/tools/corpora/wikipedia-comparable-corpora/</footnote>
with reasonable recall values. A systematic comparison to translation dictionaries pointed out that the distributional approach achieves similar precision results with much higher recall values than the probabilistic dictionaries. Furthermore, the evaluation showed that setting up a confidence value as a threshold is useful for retaining only high-quality bilingual equivalents, which could benefit the work on multilingual lexicography.

Finally, preliminary tests using comparable corpora suggested that the bilingual word embeddings can be efficiently applied to different corpora than those used for training, discovering new bilingual collocations not present in the original resources.

The multilingual resources generated by the proposed method can be used in several scenarios in which MWEs play an important role, such as machine translation or second language learning. In this respect, corpora from various registers and linguistic varieties could be used in order to obtain a wider diversity of collocation equivalents that can be useful for different purposes.

The work presented in this chapter enables us to propose a number of directions for further work. First, the results of the error analysis should be taken into account in order to reduce both the issues produced by the NLP pipeline, and those which arise from the word embedding models. On the one hand, understanding collocations as directional combinations may lead us to evaluate other association measures which are not symmetrical, e.g., *Delta-P*. On the other hand, it could be interesting to evaluate other approaches for the alignment of bilingual collocations which make use of better compositionality models, and which effectively learn the semantic distribution of collocations as single units, in order to deal with cases of incongruent collocation equivalents.

## Abbreviations

EM expectation maximization

EN English

MI mutual information

MWE multiword expression

NLP natural language processing

PT Portuguese

ES Spanish

UD Universal Dependencies

## Acknowledgements

This work has been supported by the Spanish Ministry of Economy, Industry and Competitiveness through the project with reference FFI2016-78299-P, by two *Juan de la Cierva* grants (FJCI-2014-22853 and IJCI-2016-29598), and by a 2017 Leonardo Grant for Researchers and Cultural Creators, BBVA Foundation.
## References

Alonso-Ramos, Margarita, Leo Wanner, Orsolya Vincze, Gerard Casamayor del Bosque, Nancy Vázquez Veiga, Estela Mosqueira Suárez & Sabela Prieto González. 2010. Towards a motivated annotation schema of collocation errors in learner corpora. In Nicoletta Calzolari, Khalid Choukri, Bente Maegaard, Joseph Mariani, Jan Odijk, Stelios Piperidis, Mike Rosner & Daniel Tapias (eds.), *Proceedings of the 7th Conference on International Language Resources and Evaluation* (LREC 2010), 3209–3214. European Language Resources Association (ELRA).

Altenberg, Bengt & Sylviane Granger. 2001. The grammatical and lexical patterning of MAKE in native and non-native student writing. *Applied Linguistics* 22(2). 173–195.

Berard, Alexandre, Christophe Servan, Olivier Pietquin & Laurent Besacier. 2016. MultiVec: A multilingual and multilevel representation learning toolkit for NLP. In Nicoletta Calzolari, Khalid Choukri, Thierry Declerck, Sara Goggi, Marko Grobelnik, Bente Maegaard, Joseph Mariani, Helene Mazo, Asuncion Moreno, Jan Odijk & Stelios Piperidis (eds.), *Proceedings of the 10th International Conference on Language Resources and Evaluation* (LREC 2016), 4188–4192. European Language Resources Association (ELRA).

Church, Kenneth Ward & Patrick Hanks. 1990. Word association norms, mutual information, and lexicography. *Computational Linguistics* 16(1). 22–29.

Evert, Stefan. 2008. Corpora and collocations. In Anke Lüdeling & Merja Kytö (eds.), *Corpus linguistics. An international handbook*, vol. 2, 1212–1248. Berlin: Mouton de Gruyter.

Evert, Stefan & Hannah Kermes. 2003. Experiments on candidate data for collocation extraction. In *Proceedings of the 10th Conference on European Chapter of the Association for Computational Linguistics*, vol. 2 (EACL 2003), 83–86.

Forcada, Mikel L., Mireia Ginestí-Rosell, Jacob Nordfalk, Jim O'Regan, Sergio Ortiz-Rojas, Juan Antonio Pérez-Ortiz, Felipe Sánchez-Martínez, Gema Ramírez-Sánchez & Francis M. Tyers. 2011. Apertium: A free/open-source platform for rule-based machine translation. *Machine Translation* 25(2). 127–144.

Fung, Pascale. 1998. A statistical view on bilingual lexicon extraction: From parallel corpora to non-parallel corpora. In *Proceedings of the 3rd Conference of the Association for Machine Translation in the Americas. Machine Translation and the Information Soup* (AMTA 1998), 1–17. Springer.

Garcia, Marcos & Pablo Gamallo. 2015. Yet another suite of multilingual NLP tools. In *Sierra-Rodríguez, José-Luis and Leal, José Paulo and Simões, Alberto*
(ed.), *Languages, applications and technologies. Communications in computer and information science* (International Symposium on Languages, Applications and Technologies (SLATE 2015)), 65–75. Springer.

Garcia, Marcos, Marcos García-Salido & Margarita Alonso-Ramos. 2017. Using bilingual word-embeddings for multilingual collocation extraction. In *Proceedings of the 13th Workshop on Multiword Expressions* (MWE ’17), 21–30. Association for Computational Linguistics.

Garcia, Marcos, Marcos García-Salido & Margarita Alonso-Ramos. 2018. Discovering bilingual collocations in parallel corpora: A first attempt at using distributional semantics. In Irene Doval & María Teresa Sánchez Nieto (eds.), *Parallel corpora for contrastive and translation studies: New resources and applications*. John Benjamins Publishing Company.

Haruno, Masahiko, Satoru Ikehara & Takefumi Yamazaki. 1996. Learning bilingual collocations by word-level sorting. In *Proceedings of the 16th Conference on Computational Linguistics*, vol. 1 (COLING 1996), 525–530. Association for Computational Linguistics.

Hiemstra, Djoerd. 1998. Multilingual domain modeling in twenty-one. Automatic creation of a bi-directional translation lexicon from a parallel corpus. In *Computational linguistics in the Netherlands 1997: Selected papers from the eighth clin meeting*, 41–57.

Kilgarriff, Adam. 2006. Collocationality (and how to measure it). In Elisa Corino, Carla Marello & Cristina Onesti (eds.), *Proceedings of the 12th EURALEX international congress*, vol. 2, 997–1004.

Krenn, Brigitte & Stefan Evert. 2001. Can we do better than frequency? A case study on extracting PP-verb collocations. In *Proceedings of the ACL Workshop on Collocations*, 39–46. Association for Computational Linguistics.

Kupiec, Julian. 1993. An algorithm for finding noun phrase correspondences in bilingual corpora. In *Proceedings of the 31st annual meeting on association for computational linguistics* (ACL 1993), 17–22. Association for Computational Linguistics.

Lin, Dekang. 1999. Automatic identification of non-compositional phrases. In *Proceedings of the 37th annual meeting of the association for computational linguistics on computational linguistics* (ACL 1999), 317–324.

Lison, Pierre & Jörg Tiedemann. 2016. OpenSubtitles2016: Extracting large parallel corpora from movie and TV subtitles. In Nicoletta Calzolari, Khalid Choukri, Thierry Declerck, Sara Goggi, Marko Grobelnik, Bente Maegaard, Joseph Mariani, Helene Mazo, Asuncion Moreno, Jan Odijk & Stelios Piperidis (eds.), *Pro-
ceedings of the 10th International Conference on Language Resources and Evalua-
tion (LREC 2016), 923–929. European Language Resources Association (ELRA).

Lü, Yajuan & Ming Zhou. 2004. Collocation translation acquisition using mono-
lingual corpora. In *Proceedings of the 42nd annual meeting of the association for
computational linguistics* (ACL 2004), 167–174. Association for Computational
Linguistics.

Luong, Minh-Thang, Hieu Pham & Christopher D. Manning. 2015. Bilingual word representations with monolingual quality in mind. In *Proceedings of the First Workshop on Vector Space Modeling for Natural Language Processing* (VSM-NLP) at the 2015 Conference of the North American Chapter of the Association for Computational Linguistics – Human Language Technologies (NAACL HLT 2015), 151–159. Association for Computational Linguistics.

Mel'čuk, Igor A. 1998. Collocations and lexical functions. In Anthony Paul Cowie (ed.), *Phraseology. Theory, analysis and applications*, 23–53. Oxford: Clarendon Press.

Mikolov, Tomas, Kai Chen, Greg Corrado & Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. In *Workshop Proceedings of the International Conference on Learning Representations (ICLR) 2013*. arXiv preprint arXiv:1301.3781.

Nesselhauf, Nadja. 2003. The use of collocations by advanced learners of English and some implications for teaching. *Applied linguistics* 24(2). 223–242.

Nivre, Joakim, Marie-Catherine de Marneffe, Filip Ginter, Yoav Goldberg, Jan Hajic, Christopher D. Manning, Ryan McDonald, Slav Petrov, Sampo Pyysalo, Natalia Silveira, Reut Tsarfaty & Daniel Zeman. 2016. Universal dependencies v1: A multilingual treebank collection. In Nicoletta Calzolari, Khalid Choukri, Thierry Declerck, Sara Goggi, Marko Grobelnik, Bente Maegaard, Joseph Mariani, Helene Mazo, Asuncion Moreno, Jan Odijk & Stelios Piperidis (eds.), *Proceedings of the Tenth International Conference on Language Resources and Evaluation* (LREC 2016), 1659–1666. European Language Resources Association (ELRA). 23-28 May, 2016.

Nivre, Joakim, Johan Hall, Jens Nilsson, Atanas Chanev, Gülsen Eryigit, Sandra Kübler, Svetoslav Marinov & Erwin Marsi. 2007. MaltParser: A language-independent system for data-driven dependency parsing. *Natural Language Engineering* 13(02). 95–135.

Orliac, Brigitte & Mike Dillinger. 2003. Collocation extraction for Machine Translation. In *Proceedings of ninth machine translation summit* (MT Summit IX), 292–298.
Pecina, Pavel. 2010. Lexical association measures and collocation extraction. *Language Resources and Evaluation* 44(1-2). 137–158.

Rivera, Oscar Mendoza, Ruslan Mitkov & Gloria Corpas Pastor. 2013. A flexible framework for collocation retrieval and translation from parallel and comparable corpora. In *Proceedings of the Workshop on Multi-word Units in Machine Translation and Translation Technology*, 18–25.

Sag, Ivan A., Timothy Baldwin, Francis Bond, Ann A. Copestake & Dan Flickinger. 2002. Multiword expressions: A pain in the neck for NLP. In *Proceedings of the 3rd International Conference on Computational Linguistics and Intelligent Text Processing*, vol. 2276/2010 (CICLing ’02), 1–15. Springer-Verlag.

Seretan, Violeta. 2011. *Syntax-based collocation extraction* (Text, Speech and Language Technology). Dordrecht, Heidelberg, London, New York: Springer.

Seretan, Violeta & Eric Wehrli. 2007. Collocation translation based on sentence alignment and parsing. In *Actes de la 14e conference sur le traitement automatique des langues naturelles* (TALN 2007), 401–410. IRIT Press.

Simões, Alberto Manuel & José João Almeida. 2003. NATools – a statistical word aligner workbench. *Procesamiento del lenguaje natural* 31. 217–224.

Smadja, Frank. 1992. How to compile a bilingual collocational lexicon automatically. In *Proceedings of the AAAI Workshop on Statistically-Based NLP Techniques*, 57–63. AAAI Press.

Smadja, Frank. 1993. Retrieving collocations from text: Xtract. *Computational Linguistics* 19(1). 143–177.

Smadja, Frank, Kathleen R. McKeown & Vasileios Hatzivassiloglou. 1996. Translating collocations for bilingual lexicons: A statistical approach. *Computational Linguistics* 22(1). 1–38.

Stubbs, Michael. 1995. Collocations and semantic profiles: On the cause of the trouble with quantitative studies. *Functions of language* 2(1). 23–55.

Wanner, Leo, Bernd Bohnet & Mark Giereth. 2006. Making sense of collocations. *Computer Speech & Language* 20(4). 609–624.

Wanner, Leo, Gabriela Ferraro & Pol Moreno. 2016. Towards distributional semant-ics-based classification of collocations for collocation dictionaries. *International Journal of Lexicography* 30. 167–186.

Wu, Chien-Cheng & Jason S. Chang. 2003. Bilingual collocation extraction based on syntactic and statistical analyses. In *Proceedings of the 15th Conference on Computational Linguistics and Speech Processing*, 1–20. Association for Computational Linguistics and Chinese Language Processing.
<figure><img src="image_61.png" /></figure>

# Exploiting multilingual lexical resources to predict MWE compositionality

Bahar Salehi

The University of Melbourne

Paul Cook

University of New Brunswick

Timothy Baldwin

The University of Melbourne

Semantic idiomaticity is the extent to which the meaning of a multiword expression (MWE) cannot be predicted from the meanings of its component words. Much work in natural language processing on semantic idiomaticity has focused on compositionality prediction, wherein a binary or continuous-valued compositionality score is predicted for an MWE as a whole, or its individual component words. One source of information for making compositionality predictions is the translation of an MWE into other languages. This chapter extends two previously-presented studies – Salehi & Cook (2013) and Salehi et al. (2014) – that propose methods for predicting compositionality that exploit translation information provided by multilingual lexical resources, and that are applicable to many kinds of MWEs in a wide range of languages. These methods make use of distributional similarity of an MWE and its component words under translation into many languages, as well as string similarity measures applied to definitions of translations of an MWE and its component words. We evaluate these methods over English noun compounds, English verb-particle constructions, and German noun compounds. We show that the estimation of compositionality is improved when using translations into multiple languages, as compared to simply using distributional similarity in the source language. We further find that string similarity complements distributional similarity.

<figure><img src="image_62.png" /></figure>
# 1 Compositionality of MWEs

Multiword expressions (hereafter MWEs) are combinations of words which are lexically, syntactically, semantically or statistically idiosyncratic (Sag et al. 2002; Baldwin & Kim 2010). Much research has been carried out on the extraction and identification of MWEs<fnref n="1" /> in English (Schone & Jurafsky 2001; Pecina 2008; Fazly et al. 2009) and other languages (Dias 2003; Evert & Krenn 2005; Salehi et al. 2012). However, considerably less work has addressed the task of predicting the meaning of MWEs, especially in non-English languages. As a step in this direction, the focus of this study is on predicting the compositionality of MWEs.

An MWE is fully compositional if its meaning is predictable from its component words, and it is non-compositional (or idiomatic) if not. For example, *stand up* “rise to one’s feet” is compositional, because its meaning is clear from the meaning of the components *stand* and *up*. However, the meaning of *strike up* “to start playing” is largely unpredictable from the component words *strike* and *up*.

In this study, following McCarthy et al. (2003) and Reddy et al. (2011), we consider compositionality to be graded, and aim to predict the *degree* of compositionality. For example, in the dataset of Reddy et al. (2011), *climate change* is judged to be 99% compositional, while *silver screen* is 48% compositional and *ivory tower* is 9% compositional. Formally, we model compositionality prediction as a regression task.

An explicit handling of MWEs has been shown to be useful in NLP applications (Ramisch 2012). As an example, Carpuat & Diab (2010) proposed two strategies for integrating MWEs into statistical machine translation. They show that even a large scale bilingual corpus cannot capture all the necessary information to translate MWEs, and that in adding the facility to model the compositionality of MWEs into their system, they could improve translation quality. Acosta et al. (2011) showed that treating non-compositional MWEs as a single unit in information retrieval improves retrieval effectiveness. For example, while searching for documents related to *ivory tower*, we are almost certainly not interested in documents relating to elephant tusks.

Our approach is to use a large-scale multi-way translation lexicon to source translations of a given MWE and each of its component words, and then model the semantic similarity between each component word and the MWE.<fnref n="2" /> We consider similarity measures based on distributional similarity from monolingual

<footnote n="1">¹In this chapter, we follow Baldwin & Kim (2010) in considering MWE “identification” to be a token-level disambiguation task, and MWE “extraction” to be a type-level lexicon induction task.</footnote>

<footnote n="2">²Note that we will always assume that there are two component words, but the method is easily generalisable to MWEs with more than two components.</footnote>
13 Exploiting multilingual lexical resources to predict MWE compositionality

<figure><img src="image_63.png" /><figcaption>Figure 1: Outline of our approach to computing the similarity of translations of an MWE with each of its component words, for a given target language. simᵢ is the similarity between the first or second component of the MWE, and the MWE itself, based on either string or distributional similarity, as measured using language i.</figcaption></figure>

corpora for the source language and each of the target languages, as well as string similarity measures applied to definitions of translations of an MWE and its component words as shown in Figure 1. We then consider a variety of approaches to combining similarity scores from the various languages to produce a final compositionality score for the source language expression, as illustrated in Figure 2. We hypothesise that by using multiple translations we will be able to better predict compositionality, and that string similarity measures will complement distributional similarity. Our results confirm our hypotheses, and we further achieve state-of-the-art results over two compositionality prediction datasets.

This chapter combines two previous works – Salehi & Cook (2013) and Salehi et al. (2014) – and extends them in the following ways:

* two new string similarity measures in §4.1.1;

* updated results in §4.2 for the method of Salehi & Cook (2013) such that they are now comparable with the results of the method of Salehi et al. (2014) in §6 – previously these results were not comparable because they used different cross-validation folds during evaluation;

* new results for a dataset of German noun compounds based on the string similarity methods in §4.2;
<figure><img src="image_64.png" /><figcaption>Figure 2: Outline of the method for combining similarity scores from multiple languages, across the components of the MWE.</figcaption></figure>

• additional error analysis in §4.2.1 for English verb-particle constructions;

• two new translation-based similarity approaches, and results for these methods, in §4.2.2;

• experiments considering an alternative translation dictionary in §5;

• analysis of the impact of window size on the distributional similarity approach in §6.1.1.

## 2 Related work

Most recent work on predicting the compositionality of MWEs can be divided into two categories: language/construction-specific and general-purpose. This can be at either the token-level (over token occurrences of an MWE in a corpus) or type-level (over the MWE string, independent of usage). The bulk of work on compositionality has been language/construction-specific and operated at the token-level, using dedicated methods to identify instances of a given MWE, and specific properties of the MWE in that language to predict compositionality (Lin 1999; Kim & Baldwin 2007; Fazly et al. 2009).

General-purpose token-level approaches such as distributional similarity have
been commonly applied to infer the semantics of a word/MWE (Schone & Ju-
rafsky 2001; Baldwin et al. 2003; Reddy et al. 2011). These techniques are based
on the assumption that the meaning of a word is predictable from its context
of use, via the neighbouring words of token-level occurrences of the MWE. In
order to predict the compositionality of a given MWE using distributional simi-
larity, the different contexts of the MWE are compared with the contexts of its
components, and the MWE is considered to be compositional if the MWE and
component words occur in similar contexts.

Identifying token instances of MWEs is not always easy, especially when the
component words do not occur sequentially. For example, consider *put on* in *put
your jacket on*, and *put your jacket on the chair*. In the first example *put on* is an
MWE, while in the second example, *put on* is a simple verb with prepositional
phrase and not an instance of an MWE. Moreover, if we adopt a conservative
identification method, the number of token occurrences will be limited and the
distributional scores may not be reliable. Additionally, for morphologically-rich
languages, it can be difficult to predict the different word forms a given MWE
type will occur across, posing a challenge for our requirement of no language-
specific preprocessing.

Pichotta & DeNero (2013) proposed a token-based method for identifying En-
glish phrasal verbs based on parallel corpora for 50 languages. They show that
they can identify phrasal verbs better when they combine information from mul-
tiple languages, in addition to the information they get from a monolingual cor-
pus. This finding lends weight to our hypothesis that using translation data and
distributional similarity from each of a range of target languages, can improve
compositionality prediction. Having said that, the general applicability of their
method is questionable – there are many parallel corpora involving English, but
for other languages, this tends not to be the case.

In the literature, compositionality has been viewed as either compositionality
of the whole MWE as one unit (McCarthy et al. 2003; Venkatapathy & Joshi 2005;
Katz 2006; Biemann & Giesbrecht 2011; Farahmand et al. 2015), or compositiona-
lity relative to each component (Reddy et al. 2011; Hermann et al. 2012; Schulte
im Walde et al. 2013). There have also been studies which focus only on one com-
ponent of the MWE. For example, Korkontzelos & Manandhar (2009) induce the
most probable sense of an MWE first, and then measure the semantic similarity
between the MWE and its semantic head. This approach of considering only the
head component has been shown to be quite accurate for English verb-particle
constructions (Bannard et al. 2003). However, this might not always be the case.
For example, as shown in Reddy et al. (2011), the compositionality of the first
noun (the modifier) has more impact than the second noun (the head) for En-
glish noun compounds.

Elsewhere, a lot of work has been done on specific types of MWE in specific
languages. In English, studies have been done specifically on VPCs (McCarthy
et al. 2003; Bannard et al. 2003), verb+noun MWEs (Venkatapathy & Joshi 2005;
McCarthy et al. 2007; Fazly et al. 2009), noun compounds (Reddy et al. 2011),
and adjective+noun compounds (Vecchi et al. 2011). There have also been studies
focusing on a specific language other than English, such as Arabic (Saif et al. 2013)
and German (Schulte im Walde et al. 2013). This chapter investigates language
independent approaches applicable to any type of MWE in any language.

3 Resources

In this section, we describe the datasets used to evaluate our method and the
multilingual dictionary it requires. These are the same resources as used by Salehi
& Cook (2013) and Salehi et al. (2014).

3.1 Datasets

We evaluate our proposed method over three datasets (two English, one German),
as described below.

3.1.1 English noun compounds (ENC)

Our first dataset is made up of 90 binary English noun compounds, from the work
of Reddy et al. (2011). Each noun compound was annotated by multiple annota-
tors using the integer scale 0 (fully non-compositional) to 5 (fully compositional).
A final compositionality score was then calculated as the mean of the scores from
the annotators. If we simplistically consider 2.5 as the threshold for compositio-
nality, the dataset is relatively well balanced, containing 48% compositional and
52% non-compositional noun compounds.

Spearman correlation was used to get an estimate of inter-annotator agree-
ment. The average correlation for compound compositionality was ρ = 0.522.
This score was slightly higher for the compositionality of components (ρ = 0.570
for the first component and ρ = 0.616 for the second component).
### 3.1.2 English verb-particle constructions (EVPC)

The second dataset contains 160 English verb-particle constructions (VPCs), from the work of Bannard (2006). In this dataset, a verb-particle construction consists of a verb (the head) and a prepositional particle (e.g. *hand in*, *look up* or *battle on*). For each component word (the verb and particle, respectively), multiple annotators were asked whether the VPC entails the component word. In order to translate the dataset into a regression task, we calculate the overall compositionality as the number of annotations of entailment for the verb, divided by the total number of verb annotations for that VPC. That is, following Bannard et al. (2003), we only consider the compositionality of the verb component in our experiments. The Kappa score between the multiple annotators is 0.372 for verb and 0.352 for the particle component.

### 3.1.3 German noun compounds (GNC)

Our final dataset is made up of 246 German noun compounds (von der Heide & Borgwaldt 2009; Schulte im Walde et al. 2013). Multiple annotators were asked to rate the compositionality of each German noun compound on an integer scale of 1 (non-compositional) to 7 (compositional). The overall compositionality score is then calculated as the mean across the annotators. Note that the component words are provided as part of the dataset, and that there is no need to perform decompounding. This dataset is significant as it is non-English and because of the fact that German has relatively rich morphology, which we expect to impact on the identification of both the MWE and the component words.

## 3.2 Multilingual dictionary

To translate the MWEs and their components, we use PanLex (Baldwin et al. 2010). This online dictionary is massively multilingual, covering more than 1353 languages. The translations are sourced from handmade electronic dictionaries. It contains lemmatised words and MWEs in a large variety of languages, with lemma-based (and less frequently sense-based) links between them.

For each MWE dataset (see §3.1), we translate each MWE, and its component words, from the source language into many target languages. These translations will be used in §4 and §6. In instances where there is no direct translation in a given language for a term, we use a pivot language to find translation(s) in the target language. For example, the English noun compound *silver screen* has direct translations in only 13 languages in PanLex, including Vietnamese (*mân bac*) but
not French. There is, however, a translation of *màn bac* into French (cinéma), allowing us to infer an indirect translation between *silver screen* and *cinéma*. In this way, if there are no direct translations into a particular target language, we search for a single-pivot translation via each of our other target languages, and combine them all together as our set of translations for the target language of interest.

# 4 String similarity

In this section we present our string similarity-based method for predicting compositionality, followed by experimental results using this method. This section extends Salehi & Cook (2013) as described in §1.

## 4.1 Compositionality prediction based on string similarity

We hypothesize that compositional MWEs are more likely to be word-for-word translations in a given language than non-compositional MWEs. Hence, if we can locate the translations of the components in the translation of the MWE, we can deduce that it is compositional. As an example of our method, consider the English-to-Persian translation of *kick the bucket* as a non-compositional MWE and *make a decision* as a semi-compositional MWE (Table 1).<fnref n="3" /> By locating the translation of *decision* (*tasnim*) in the translation of *make a decision* (*tasnim gereftan*), we can deduce that it is semi-compositional. However, we cannot locate any of the component translations in the translation of *kick the bucket*. Therefore, we conclude that it is non-compositional. Note that in this simple example, the match is word-level, but that due to the effects of morphophonology, the more likely situation is that the components don’t match exactly (as we observe in the case of *khadamaat* and *khedmat* for the *public service* example), which motivates our use of string similarity measures which can capture partial matches.

### 4.1.1 String similarity measures

We consider the following string similarity measures to compare the translations. In each case, we normalize the output value to the range [0, 1], where 1 indicates identical strings and 0 indicates completely different strings. We will indicate the translation of the MWE in a particular language *t* as *mwe*^t, and the translation of a given component in language *t* as *component*^t.

<footnote n="3">³Note that the Persian words are transliterated into English for ease of understanding.</footnote>
<table><caption>Table 1: English MWEs and their components with their translation in Persian. Direct matches between the translation of an MWE and its components are shown in **bold**; partial matches are shown in *italics*.</caption><thead><tr><th>English</th><th>Persian translation</th></tr></thead><tbody><tr><td>kick the bucket</td><td>mord</td></tr><tr><td>kick</td><td>zad</td></tr><tr><td>the</td><td>-</td></tr><tr><td>bucket</td><td>satl</td></tr><tr><td>make a decision</td><td><b>tasmim</b> gereft</td></tr><tr><td>make</td><td>sakht</td></tr><tr><td>a</td><td>yek</td></tr><tr><td>decision</td><td><b>tasmim</b></td></tr><tr><td>public service</td><td><b>khadamaat omumi</b></td></tr><tr><td>public</td><td><b>omumi</b></td></tr><tr><td>service</td><td><b>khedmat</b></td></tr></tbody></table>

**Longest common substring (LCS):** The LCS measure finds the longest common substring between two strings. For example, the LCS between ABABC and BABCAB is BABC. We calculate a normalized similarity value based on the length of the LCS as follows:

$$ \frac{\text{LCS}(mwe^t, \text{component}^t)}{\min(\text{len}(mwe^t), \text{len}(\text{component}^t))} \qquad (13.1) $$

**Levenshtein (LEV1):** The Levenshtein distance calculates the number of basic edit operations required to transform one word into the other. Edits consist of single-letter insertions, deletions or substitutions. We normalize LEV1 as follows:

$$ 1 - \frac{\text{LEV1}(mwe^t, \text{component}^t)}{\max(\text{len}(mwe^t), \text{len}(\text{component}^t))} \qquad (13.2) $$

**Levenshtein with substitution penalty (LEV2):** One well-documented feature of Levenshtein distance (Baldwin 2009) is that substitutions are in fact the combination of an addition and a deletion, and as such can be considered to be two edits. Based on this observation, we experiment with a variant of LEV1 with this penalty applied for substitutions. Similarly to LEV1, we normalize as follows:
$$1 - \frac{\text{LEV2}(mwe^t, \text{component}^t)}{\text{len}(mwe^t) + \text{len}(\text{component}^t)} \qquad (13.3)$$

**Smith Waterman (SW):** This method is based on the Needleman-Wunsch algorithm,<fnref n="4" /> and was developed to locally-align two protein sequences (Smith & Waterman 1981). It finds the optimal similar regions by maximizing the number of matches and minimizing the number of gaps necessary to align the two sequences. For example, the optimal local sequence for the two sequences below is AT--ATCC, in which “-” indicates a gap:

Seq1: ATGCATCCCATGAC

Seq2: TCTATATCCGT

As the example shows, it looks for the longest common string but has a built-in mechanism for including gaps in the alignment (with penalty). This characteristic of SW might be helpful in our task, because there may be morphophonological variations between the MWE and component translations (as seen above in the *public service* example). We normalize SW similarly to LCS:

$$\frac{\text{len(alignedSequence)}}{\min(\text{len}(mwe^t), \text{len}(\text{component}^t))} \qquad (13.4)$$

The aligned sequence is the combination of the common characters in the optimal local sequence we found using SW. In the above example, the aligned sequence is **ATATCC**.

**Jaccard and Dice similarity:** For further analysis, we experiment with Jaccard and Dice similarity, which are well-known for measuring the similarity between two sentences or bodies of text (Gomaa & Fahmy 2013). Both methods view the texts as sets of words, with similarity based on the size of the intersection between the sets, but differ in the way they are normalized. In our case, we expect relatively low overlap at the word level due to morphophonology, and therefore

<footnote n="4">⁴The Needleman-Wunsch (NW) algorithm was designed to align two sequences of amino-acids (Needleman & Wunsch 1970). The algorithm looks for the sequence alignment which maximizes the similarity. As with the LEV score, NW minimizes edit distance, but also takes into account character-to-character similarity based on the relative distance between characters on the keyboard. We exclude this score because it is highly similar to the LEV scores and we did not obtain encouraging results using NW in our preliminary experiments.</footnote>
calculate Jaccard (J) and Dice (D) at the character- instead of word-level as follows:

$$J = \frac{|mwe^t \cap component^t|}{|mwe^t| + |component^t| - |mwe^t \cap component^t|} \qquad (13.5)$$

$$D = \frac{2 * |mwe^t \cap component^t|}{|component^t| + |mwe^t|} \qquad (13.6)$$

### 4.1.2 Calculating compositionality

Given the string similarity scores calculated between the translations for a given component word and the MWE, we need some way of combining scores across component words. First, we measure the compositionality of each component within the MWE ($s_1$ and $s_2$):

$$s_1 = f_1(\text{sim}_1(w_1, mwe), \dots, \text{sim}_1(w_1, mwe)) \qquad (13.7)$$

$$s_2 = f_2(\text{sim}_2(w_2, mwe), \dots, \text{sim}_2(w_2, mwe)) \qquad (13.8)$$

where sim is a similarity measure, $sim_i$ indicates that the calculation is based on translations in language i, and $f_1$ is a score combination function.

Then, we compute the overall compositionality of the MWE ($s_3$) from $s_1$ and $s_2$ using $f_2$:

$$s_3 = f_2(s_1, s_2) \qquad (13.9)$$

Since we often have multiple translations for a given component word/MWE in PanLex, we exhaustively compute the similarity between each MWE translation and component translation, and use the highest similarity as the result of $sim_i$. If an instance does not have a direct/indirect translation in PanLex, we assign a default value, which is the mean of the highest and lowest annotation score for the dataset under consideration. Note that word order is not an issue in our method, as we calculate the similarity independently for each MWE component.

We consider simple functions for $f_1$ such as mean, median, product, minimum and maximum. $f_2$ was selected to be the same as $f_1$ in all situations, except when we use mean for $f_1$. Here, following Reddy et al. (2011), we experimented with weighted mean:
$$f_2(s_1, s_2) = \alpha s_1 + (1 - \alpha)s_2 \qquad (13.10)$$

Based on 3-fold cross-validation, we chose $\alpha = 0.7$ for ENC.<fnref n="5" /> We found $\alpha = 0.7$ is also optimal for GNC.

Since we do not have judgements for the compositionality of the full VPC in EVPC (we instead have separate judgements for the verb and particle), we cannot use $f_2$ for this dataset. Bannard et al. (2003) observed that nearly all of the verb-compositional instances were also annotated as particle-compositional by the annotators. In line with this observation, we use $s_1$ (based on the verb) as the compositionality score for the full VPC.

### 4.1.3 Language selection

Our method is based on the translation of an MWE into many languages. First, we chose 54 languages for which relatively large corpora were available.<fnref n="6" /> The coverage, or the number of instances which have direct/indirect translations in PanLex, varies from one language to another. In preliminary experiments, we noticed that there is a high correlation (between roughly $r = 0.6$ and $0.8$ across the three datasets) between the usefulness of a language and its translation coverage on MWEs. Therefore, we excluded languages with MWE translation coverage of less than 50%. Based on nested 10-fold cross-validation in our experiments, we select the 10 most useful languages for each cross-validation training partition, based on the Pearson correlation between the given scores in that language and human judgements.<fnref n="7" /> The 10 best languages are selected based only on the training set for each fold. (The languages selected for each fold will later be used to predict the compositionality of the items in the testing portion for that fold.)

## 4.2 Results

As mentioned above, we perform nested 10-fold cross-validation to select the 10 best languages on the training data for each fold. The selected languages for a given fold are then used to compute $s_1$ and $s_2$ (and $s_3$ for NCs) for each instance

<footnote n="5">⁵We considered values of $\alpha$ from 0 to 1, incremented by 0.1.</footnote>

<footnote n="6">⁶In §6 these corpora will be used to compute distributional similarity. Note that the string similarity methods of interest here do not rely on the availability of large corpora.</footnote>

<footnote n="7">⁷Note that for VPCs, we calculate the compositionality of only the verb part, because we don't have the human judgements for the whole VPC.</footnote>
<table><caption>Table 2: Correlation (r) on each dataset, for each string similarity measure. The best correlation for each dataset is shown in boldface.</caption><thead><tr><th>Method</th><th>ENC</th><th>EVPC</th><th>GNC</th></tr></thead><tbody><tr><td>SW</td><td><strong>0.644</strong></td><td>0.349</td><td>0.379</td></tr><tr><td>LCS</td><td><strong>0.644</strong></td><td><strong>0.385</strong></td><td>0.372</td></tr><tr><td>LEV1</td><td>0.502</td><td>0.328</td><td>0.318</td></tr><tr><td>LEV2</td><td>0.566</td><td>0.327</td><td><strong>0.389</strong></td></tr><tr><td>Jaccard</td><td>0.474</td><td>0.335</td><td>0.299</td></tr><tr><td>Dice</td><td>0.557</td><td>0.331</td><td>0.370</td></tr><tr><td>Unsupervised (family)</td><td>0.556</td><td>0.257</td><td>0.164</td></tr><tr><td>Unsupervised (coverage)</td><td>0.642</td><td>0.323</td><td>0.343</td></tr></tbody></table>

in the test set for that fold. The scores are compared with human judgements using Pearson's correlation.

We experimented with five functions for $f_1$, namely mean, median, product, maximum and minimum. Among these functions, mean performed consistently better than the others, and as such we only present results using mean in Table 2.

For ENC, LCS and SW perform best, while for EVPC, LCS performs best with SW being the next best measure. Both LCS and SW look for a sequence of similar characters, unlike LEV1 and LEV2, which are not affected by match contiguity. For GNC, LEV2, SW and LCS perform better than LEV1. However, unlike the other two datasets, LEV2 is the best performing method, and SW is slightly better than LCS.

For all datasets, Jaccard and Dice perform worse than SW and LCS. This shows that, despite being useful in measuring the similarity between sentences, these two measures do not perform well in this compositionality prediction task. The relatively poor performance of these measures could be because, unlike the other measures, Jaccard and Dice are calculated independently of the order of characters. Dice performs better than Jaccard for ENC and GNC, while Jaccard performs slightly better than Dice for EVPC.

The results support our hypothesis that using multiple target languages rather than one, results in a more accurate prediction of MWE compositionality. Our best result using the 10 selected languages on ENC is $r = 0.644$, as compared to the best single-language correlation of $r = 0.543$ for Portuguese. On EVPC, the best LCS result for the verb component is $r = 0.385$, as compared to the
best single-language correlation of *r* = 0.342 for Lithuanian. For GNC, the best correlation of *r* = 0.389 is well above the highest correlation of a single language of roughly *r* = 0.32.

In §6 we will combine this string similarity approach with an approach based on distributional similarity, and compare it against a baseline and state-of-the-art approaches.

### 4.2.1 Error analysis

We analysed items in ENC which have a high absolute difference (more than 2.5) between the human annotation and our scores (using LCS and mean). The words are *cutting edge*, *melting pot*, *gold mine* and *ivory tower*, which are non-compositional according to ENC. After investigating their translations, we came to the conclusion that the first three MWEs have word-for-word translations in most languages. Hence, they disagree with our hypothesis that word-for-word translation is a strong indicator of compositionality. The word-for-word translations might be because of the fact that they have both compositional and non-compositional senses, or because they are calques (loan translations). However, we have tried to avoid such problems with calques by using translations into several languages.

For *ivory tower* (“a state of mind that is discussed as if it were a place”)<fnref n="8" /> we noticed that we have a direct translation into 13 languages. Other languages have indirect translations. By checking the direct translations, we noticed that, in French, the MWE is translated to *tour* and *tour d'ivoire*. A noisy (wrong) translation of *tour* “tower” resulted in wrong indirect translations for *ivory tower* and an inflated estimate of compositionality.

We repeat the same error analysis for the EVPC dataset. The items with a high difference between the human annotation and our scores are: *carry out*, *drop out*, *get in*, *carry away*, *wear down* and *turn on*. All of these items are annotated as non-compositional. These VPCs also have a compositional sense beside the non-compositional meaning. Also, as with the ENC dataset, we have problems of calques. For example, *drop out* when translated to German (*ausfallen*) includes the word *fallen*, which is one of the translations of *drop*.

### 4.2.2 Unsupervised approach

The proposed translation-based string similarity approach has been supervised so far, in that the best target languages are selected based on training data. In this

<footnote n="8">⁸This definition is from Wordnet 3.1.</footnote>
section, we propose two unsupervised approaches in which: (1) only the target languages of the same language family as the source language are considered; and (2) only the 10 target languages with the highest translation coverage are considered.

**Languages of the same family:** We hypothesize that translations into target languages in the same language family as the source language might be particularly useful for compositionality prediction for MWEs in the source language. To test this hypothesis, we consider an unsupervised approach in which only languages in the same family as the source language are used when computing the compositionality scores.

In this unsupervised approach, LCS scores of the languages of the same family as the source language (here Germanic, for both the English and German datasets) are considered. The Germanic languages among our 54 languages are: English, German, Danish, Dutch, Icelandic, Luxembourgish, Norwegian and Swedish.

Results for this unsupervised approach are shown in Table 2 ("Unsupervised (family)"). This approach performs substantially worse than the corresponding supervised approach based on LCS, for each dataset. This drop in performance could be because almost none of the 10 best languages selected in the supervised approach are in the same language family as the source language. The shared languages between the supervised approach and this approach are Dutch and Norwegian for ENC, English for GNC. There is no shared language between the two approaches when using EVPC.

**Languages with the highest translation coverage:** In the proposed supervised setup, the best target languages are those whose scores have the highest correlation with gold-standard annotations. According to our experiments, we showed that there is a strong correlation between being a good language for this compositionality prediction task and its coverage in PanLex (in the range of roughly 0.6 < r < 0.8 across the three datasets). In other words, the target languages to which most of the source language MWEs have a translation in PanLex, result in higher correlation for compositionality prediction.

We now consider an unsupervised approach, in which only the 10 target languages with the highest translation coverage are considered. The results of this unsupervised approach, again using LCS, are shown in Table 2 ("Unsupervised (coverage)"). According to the results, despite the lower correlation scores for the proposed unsupervised method, this method is comparable to the supervised
<table><caption>Table 3: The 10 languages with the highest translation coverage for ENC, EVPC and GNC. Languages also selected by the supervised approach are shown in **boldface**.</caption><thead><tr><th>ENC</th><th>EVPC</th><th>GNC</th></tr></thead><tbody><tr><td>German</td><td>German</td><td><strong>English</strong></td></tr><tr><td>Finnish</td><td>Finnish</td><td>Japanese</td></tr><tr><td><strong>French</strong></td><td>French</td><td>French</td></tr><tr><td>Italian</td><td>Italian</td><td>Italian</td></tr><tr><td>Russian</td><td>Japanese</td><td>Russian</td></tr><tr><td>Spanish</td><td>Hungarian</td><td>Hungarian</td></tr><tr><td><strong>Portuguese</strong></td><td>Dutch</td><td>Dutch</td></tr><tr><td>Japanese</td><td><strong>Polish</strong></td><td>Turkish</td></tr><tr><td><strong>Chinese</strong></td><td>Chinese</td><td>Chinese</td></tr><tr><td>Czech</td><td>Czech</td><td>Czech</td></tr></tbody></table>

approach. Therefore, in the case of not having a training set for a group of MWEs (no matter in what language or what type of MWE), we suggest using the target languages to which the majority of those MWEs have a translation.

The 10 languages with highest correlation for ENC, EVPC and GNC are shown in Table 3. There is some overlap between the list of languages with the highest coverage and the 10 best languages selected in our supervised approach, as shown in boldface for each dataset.

## 5 An alternative multilingual dictionary

In this section we consider the same string similarity-based approach to predicting compositionality as in §4.1, but using an alternative multilingual dictionary to PanLex, specifically dict.cc.<fnref n="9" />

dict.cc is a translation dictionary that provides translations for both English and German into 26 languages spoken in Europe. It is a crowd-sourced dictionary, with translations being contributed, and refined, by users. Due to the relatively small number of languages it covers, relying on dict.cc goes against our goals of developing compositionality prediction methods that are applicable to any language; we could not use dict.cc to predict the compositionality of, for example, a French MWE, because translations are not available for French into many languages (only English and German). Nevertheless, by considering the

<footnote n="9">⁹https://www.dict.cc/</footnote>
<table><caption>Table 4: Correlation (r) on each dataset, for each string similarity measure, using dict.cc and PanLex as the translation dictionary. The best correlation for each dataset is shown in boldface.</caption><thead><tr><th>Dictionary</th><th>Method</th><th>ENC</th><th>EVPC</th><th>GNC</th></tr></thead><tbody><tr><td rowspan="6">dict.cc</td><td>SW</td><td>.269</td><td>.217</td><td>.514</td></tr><tr><td>LCS</td><td>.251</td><td>.262</td><td>.523</td></tr><tr><td>LEV1</td><td>.181</td><td>.161</td><td>.482</td></tr><tr><td>LEV2</td><td>.163</td><td>.189</td><td>.474</td></tr><tr><td>Jaccard</td><td>.158</td><td>.127</td><td>.442</td></tr><tr><td>Dice</td><td>.230</td><td>.192</td><td>.420</td></tr><tr><td rowspan="6">PanLex</td><td>SW</td><td>.559</td><td>.294</td><td>.270</td></tr><tr><td>LCS</td><td>.551</td><td>.276</td><td>.290</td></tr><tr><td>LEV1</td><td>.388</td><td>.274</td><td>.276</td></tr><tr><td>LEV2</td><td>.512</td><td>.281</td><td>.262</td></tr><tr><td>Jaccard</td><td>.459</td><td>.241</td><td>.267</td></tr><tr><td>Dice</td><td>.541</td><td>.235</td><td>.197</td></tr></tbody></table>

use of an alternative translation dictionary (which is applicable to the English and German datasets we use for evaluation) we can learn whether our approach to predicting compositionality implicitly relies on information particular to PanLex, or whether an alternative dictionary can be substituted in its place.

We chose target languages available in dict.cc that overlap with the set of 54 target languages used in experiments with PanLex in §4.1. This resulted in 22 target languages. We introduced this restriction, as opposed to using all languages available in dict.cc, to allow us to compare PanLex and dict.cc when using the exact same set of target languages.

Results for the string similarity-based approach to predicting compositionality, using dict.cc and PanLex, each with the same 22 target languages, are shown in Table 4. The 10 best languages are selected using the same method as in §4.1.3.

For each translation dictionary and dataset, the best method is always one of either SW or LCS, and in many cases these are the top two methods (with the exceptions being EVPC and GNC using PanLex). These methods were also found to perform well in §4.2 when using PanLex and 54 target languages. This
<figure><img src="image_65.png" /><figcaption>Figure 3: Boxplots showing the percentage of expressions in each dataset covered by dict.cc and PanLex, over the 22 target languages.</figcaption></figure>

demonstrates that the methods are robust to the choice of specific translation dictionary, and when the number of target languages is substantially reduced.

There are, however, substantial differences between the results using different translation dictionaries. For any combination of dataset and method, the results using PanLex are always better than those using dict.cc for ENC and EVPC, while for GNC, the results using dict.cc are always better. To understand why this is the case, for each dataset and dictionary, and for each of the 22 target languages, we computed the proportion of expressions for which translations are available. Boxplots illustrating these findings are shown in Figure 3. On average across the target languages, many more expressions are covered by PanLex than dict.cc for ENC and EVPC, while for GNC the coverage is higher for dict.cc. For example, according to Figure 3, for EVPC the coverage for almost all of the 22 target languages is close to 100% in PanLex.

Because it in keeping with our goal of building methods for compositionality prediction that are applicable to any language, and because it gives the best results in two out of three cases for the datasets used for evaluation, we will only consider PanLex as the translation dictionary for the remainder of this chapter.
# 6 Distributional similarity

In this section we describe a method for predicting compositionality based on the same framework as in §4, but using distributional similarity instead of string similarity. This section extends Salehi et al. (2014) as described in §1.

## 6.1 Compositionality prediction based on distributional similarity

To predict the compositionality of a given MWE, we first measure the semantic similarity between the MWE and each of its component words using distributional similarity based on a monolingual corpus in the source language. We then repeat the process for translations of the MWE and its component words into each of a range of target languages, calculating distributional similarity using a monolingual corpus in the target language. We additionally use supervised learning to identify which target languages (or what weights for each language) optimise the prediction of compositionality. We hypothesise that by using multiple translations – rather than only information from the source language – we will be able to better predict compositionality. We further optionally combine our proposed approach with the LCS-based string similarity method from §4.

Below, we detail our method for calculating distributional similarity in a given language, the different methods for combining similarity scores into a single estimate of compositionality, and finally the method for selecting the target languages to use in calculating compositionality.

### 6.1.1 Calculating distributional similarity

We collected monolingual corpora for each of the 52 languages (51 target languages + 1 source language) from XML dumps of Wikipedia. These languages are based on the 54 target languages used in §4, excluding Spanish because we happened not to have a dump of Spanish Wikipedia, and also Chinese and Japanese because of the need for a language-specific word tokeniser. The raw corpora were preprocessed using the WP2TXT toolbox<fnref n="10" /> to eliminate XML tags, HTML tags and hyperlinks, and then tokenisation based on whitespace and punctuation was performed. The corpora vary in size from roughly 750M tokens for English, to roughly 640K tokens for Marathi.

In order to be consistent across all languages and to be as language-independent as possible, we calculate distributional similarity in the following manner for a given language.

<footnote n="10">¹⁰http://wp2txt.rubyforge.org/</footnote>
<table><caption>Table 5: Results of distributional similarities using 10 best languages on ENC dataset (N is window size)</caption><thead><tr><td>Context window</td><td>Correlation (r)</td></tr></thead><tbody><tr><td>Sentence</td><td>0.425</td></tr><tr><td>Window (N=3)</td><td>0.175</td></tr><tr><td>Window (N=3, with positional index)</td><td>0.031</td></tr></tbody></table>

Tokenisation is based on whitespace delimiters and punctuation; no lemmatisation or case-folding is carried out. Token instances of a given MWE or component word are identified by full-token *n*-gram matching over the token stream. We assume that all full stops and equivalent characters for other orthographies are sentence boundaries, and chunk the corpora into (pseudo-)sentences on the basis of them. For each language, we identify the 51st–1050th most frequent words, and consider them to be content-bearing words, in the manner of Schütze (1997). This is based on the assumption that the top-50 most frequent words are stop words, and not a good choice of word for calculating distributional similarity over. That is not to say that we can’t calculate the distributional similarity for stop words, however (as we will for the EVPC dataset) they are simply not used as the dimensions in our calculation of distributional similarity.

We form a vector of content-bearing words across all token occurrences of the target word, on the basis of these 1000 content-bearing words. Our preliminary results on selecting the best context window size are shown in Table 5. According to this table, for predicting the compositionality using the best 10 languages, the sentence context window results in a higher correlation. We use sentence boundaries as the context window in the rest of our experiments. According to Weeds (2003) and Padó & Lapata (2007), using dependency relations with the neighbouring words of the target word can better predict the meaning of the target word. However, in line with our assumption of no language-specific preprocessing, we just use word co-occurrence. Finally, distributional similarity is calculated over these context vectors using cosine similarity.

### 6.1.2 Calculating compositionality

The procedure of calculating the compositionality is similar to what we used in §4.1.2: after translating the MWE and its components into multiple languages and measuring the distributional similarity between the translations of the MWE and
its components (Figure 1), we find the best languages according to the training set. Then, we combine the scores from those best languages and finally calculate a combined compositionality score from the individual distributional similarities between each component word and the MWE. Based on our findings in §4.1.2, we combine the component scores using the weighted mean (Figure 2):

$$ \text{Compositionality} = \alpha s_1 + (1 - \alpha)s_2 \qquad (13.11) $$

where $s_1$ and $s_2$ are the scores for the first and the second component, respectively. We use different $\alpha$ settings for each dataset, based on the settings from §4.1.2.

We experiment with a range of methods for calculating compositionality, as follows:

*   **CS<sub>L1</sub>:** calculate distributional similarity using only distributional similarity in the source language corpus. (This is the approach used by Reddy et al. (2011), as discussed in §2.)

*   **CS<sub>L2N</sub>:** exclude the source language and compute the mean of the distributional similarity scores for the best-N target languages. The value of N is selected according to training data, as detailed in §6.1.3.<fnref n="11" />

*   **CS<sub>L1+L2N</sub>:** calculate distributional similarity over both the source language (CS<sub>L1</sub>) and the mean of the best-N languages (CS<sub>L2N</sub>), and combine via the arithmetic mean.<fnref n="12" /> This is to examine the hypothesis that using multiple target languages is better than just using the source language.

*   **CS<sub>SVR(L1+L2)</sub>:** train a support vector regressor (SVR: Smola & Schölkopf (2004)) over the distributional similarities for all 52 languages (source and target languages).

*   **CS<sub>string</sub>:** calculate string similarity using the LCS-based method of §4. LCS is chosen because, in general, it performs better than the other string similarity measures.

<footnote n="11">¹¹In the case that no translation (direct or indirect) can be found for a given source language term into a particular target language, the compositionality score for that target language is set to the average across all target languages for which scores can be calculated for the given term. If no translations are available for any target language (e.g. the term is not in PanLex) the compositionality score for each target language is set to the average score for that target language across all other source language terms.</footnote>

<footnote n="12">¹²We also experimented with taking the mean over all the languages – target and source – but found it best to combine the scores for the target languages first, to give more weight to the source language.</footnote>
*CS<sub>string+L1</sub>*: calculate the mean of the string similarity (*CS*<sub>string</sub>) and distributional similarity in the source language.

*CS<sub>all</sub>*: calculate the mean of the string similarity (*CS*<sub>string</sub>) and distributional similarity scores (*CS*<sub>L1</sub> and *CS*<sub>L2N</sub>).

### 6.1.3 Selecting target languages

We experiment with two approaches for combining the compositionality scores from multiple target languages.

First, in *CS*<sub>L2N</sub> (and *CS*<sub>L1+L2N</sub> and *CS*<sub>all</sub> that build off it), following the approach from §4.1.3, we use training data to rank the target languages according to Pearson's correlation between the predicted compositionality scores and the gold-standard compositionality judgements. However, in this case, based on this ranking, we take the best-*N* languages (instead of the best-10 languages as in §4.1.3) and again combine the individual compositionality scores by taking the arithmetic mean. We select *N* by determining the value that optimises the correlation over the training data. In other words, the selection of *N* and accordingly the best-*N* languages are based on nested cross-validation over training data, independently of the test data for that iteration of cross-validation.

Second in *CS*<sub>SVR(L1+L2)</sub>, we take the compositionality scores from the source and all 51 target languages, combine them into a feature vector, and train an SVR over the data using LIBSVM.<sup>13</sup>

## 6.2 Results

All experiments are carried out using 10 iterations of 10-fold cross validation, randomly partitioning the data independently on each of the 10 iterations, and averaging across all 100 test partitions in our presented results (Table 6). In the case of *CS*<sub>L2N</sub> and other methods that make use of it (i.e. *CS*<sub>L1+L2N</sub> and *CS*<sub>all</sub>), the languages selected for a given training fold are then used to compute the compositionality scores for the instances in the test set.

Figure 4 shows histograms of the number of times each *N* is selected over 100 folds on ENC, EVPC and GNC datasets, respectively. From the histograms, *N* = 6, *N* = 15 and *N* = 2 are the most commonly selected settings for ENC, EVPC and GNC, respectively. That is, multiple languages are generally used, but more languages are used for English VPCs than either of the compound noun datasets.

<footnote n="13">13 http://www.csie.ntu.edu.tw/~cjlin/libsvm</footnote>
<table><caption>Table 6: Pearson's correlation on the ENC, EVPC and GNC datasets</caption><thead><tr><th>Method</th><th>Summary of the Method</th><th>ENC</th><th>EVPC</th><th>GNC</th></tr></thead><tbody><tr><td>CS<sub>L1</sub></td><td>Source language</td><td>0.700</td><td>0.177</td><td>0.141</td></tr><tr><td>CS<sub>L2N</sub></td><td>Best-N target languages</td><td>0.434</td><td>0.398</td><td>0.113</td></tr><tr><td>CS<sub>L1+L2N</sub></td><td>Source + best-N target languages</td><td>0.725</td><td>0.312</td><td>0.178</td></tr><tr><td>CS<sub>SVR(L1+L2)</sub></td><td>SVR (Source + all 51 target languages)</td><td><strong>0.744</strong></td><td>0.389</td><td>0.085</td></tr><tr><td>CS<sub>string</sub></td><td>String Similarity</td><td>0.644</td><td>0.385</td><td><strong>0.372</strong></td></tr><tr><td>CS<sub>string+L1</sub></td><td>CS<sub>string</sub> + CS<sub>L1</sub></td><td>0.739</td><td>0.360</td><td>0.353</td></tr><tr><td>CS<sub>all</sub></td><td>CS<sub>L1</sub> + CS<sub>L2N</sub> + CS<sub>string</sub></td><td>0.732</td><td><strong>0.417</strong></td><td>0.364</td></tr></tbody></table>

<figure><img src="image_66.png" /><figcaption>Figure 4: Histograms displaying how many times a given *N* is selected as the best number of languages over each dataset. For example, according to the GNC chart, there is a peak for *N* = 2, which shows that over 100 folds, the best-2 languages achieved the highest correlation on 18 folds.</figcaption></figure>

Further analysis reveals that 32 (63%) target languages for ENC, 25 (49%) target languages for EVPC, and only 5 (10%) target languages for GNC have a correlation of *r* ≥ 0.1 with gold-standard compositionality judgements. On the other hand, 8 (16%) target languages for ENC, 2 (4%) target languages for EVPC, and no target languages for GNC have a correlation of *r* ≤ -0.1.
## 6.2.1 ENC results

English noun compounds are relatively easy to identify in a corpus,<fnref n="14" /> because the components occur sequentially, and the only morphological variation is in noun number (singular vs. plural). In other words, the precision for our token matching method is very high, and the recall is also acceptably high. Partly as a result of the ease of identification, we get a high correlation of *r* = 0.700 for *CS*<sub>L1</sub> (using only source language data). Using only target languages (*CS*<sub>L2N</sub>), the results drop to *r* = 0.434, but when we combine the two (*CS*<sub>L1+L2N</sub>), the correlation is higher than using only source or target language data, at *r* = 0.725. When we combine all languages using SVR, we achieve our best results on this dataset of *r* = 0.744, an improvement over the previous state of the art of Reddy et al. (2011) (*r* = 0.714). These last two results support our hypothesis that using translation data can improve the prediction of compositionality. The results for string similarity on its own (*CS*<sub>string</sub>, *r* = 0.644) are slightly lower than those using only source language distributional similarity, but when combined with *CS*<sub>L1+L2N</sub> (i.e. *CS*<sub>all</sub>) there is a slight rise in correlation (from *r* = 0.725 to *r* = 0.732).

## 6.2.2 EVPC results

English VPCs are hard to identify. As discussed in §2, VPC components may not occur sequentially, and even when they do occur sequentially, they may not be a VPC. As such, our simplistic identification method has low precision and recall (hand analysis of 927 identified VPC instances would suggest a precision of around 74%). There is no question that this is a contributor to the low correlation for the source language method (*CS*<sub>L1</sub>; *r* = 0.177). When we use target languages instead of the source language (*CS*<sub>L2N</sub>), the correlation jumps substantially to *r* = 0.398.

When we combine English and the target languages (*CS*<sub>L1+L2N</sub>), the results are actually lower than just using the target languages, because of the high weight on the target language, which is not desirable for VPCs, based on the source language results. Even for *CS*<sub>SVR(L1+L2)</sub>, the results (*r* = 0.389) are slightly below the target language-only results. This suggests that when predicting the compositionality of MWEs which are hard to identify in the source language, it may actually be better to use target languages only. The results for string similarity (*CS*<sub>string</sub>: *r* = 0.385) are similar to those for *CS*<sub>L2N</sub>. However, as with the ENC

<footnote n="14">¹⁴Although see Lapata & Lascarides (2003) for discussion of the difficulty of reliably identifying low-frequency English noun compounds.</footnote>
dataset, when we combine string similarity and distributional similarity ($CS_{all}$), the results improve, and we achieve the state of the art for the dataset.

In Table 7, we present classification-based evaluation over a subset of EVPC, binarising the compositionality judgements in the manner of Bannard et al. (2003). Our method achieves state-of-the-art results in terms of overall F-score and accuracy.

<table><caption>Table 7: Results (%) for the binary compositionality prediction task on the EVPC dataset</caption><thead><tr><th>Method</th><th>Precision</th><th>Recall</th><th>F-score (β = 1)</th><th>Accuracy</th></tr></thead><tbody><tr><td>Bannard et al. (2003)</td><td>60.8</td><td>66.6</td><td>63.6</td><td>60.0</td></tr><tr><td>CS<sub>string</sub></td><td><strong>86.2</strong></td><td>71.8</td><td>77.4</td><td>69.3</td></tr><tr><td>CS<sub>all</sub></td><td>79.5</td><td><strong>89.3</strong></td><td><strong>82.0</strong></td><td><strong>74.5</strong></td></tr></tbody></table>

### 6.2.3 GNC results

German is a morphologically-rich language, with marking of number and case on nouns. Given that we do not perform any lemmatisation or other language-specific preprocessing, we inevitably achieve low recall for the identification of noun compound tokens, although the precision should be nearly 100%. Partly because of the resultant sparseness in the distributional similarity method, the results for $CS_{L1}$ are low ($r = 0.141$), although they are lower again when using target languages ($r = 0.113$). However, when we combine the source and target languages ($CS_{L1+L2N}$) the results improve to $r = 0.178$. The results for $CS_{SVR(L1+L2)}$, on the other hand, are very low ($r = 0.085$). Ultimately, simple string similarity achieves the best results for the dataset ($r = 0.372$), and this result actually drops slightly when combined with the distributional similarities.

To better understand the reason for the lacklustre results using SVR, we carried out error analysis and found that, unlike the other two datasets, about half of the target languages return scores which correlate negatively with the human judgements. When we filter these languages from the data, the score for SVR improves appreciably. For example, over the best-3 languages overall, we get a correlation score of $r = 0.179$, which is slightly higher than $CS_{L1+L2N}$.

We further investigated the reason for getting very low and sometimes negative correlations with many of our target languages. We noted that about 24% of the German noun compounds in the dataset do not have entries in PanLex.
This contrasts with ENC where only one instance does not have an entry in
PanLex, and EVPC where all VPCs have translations in at least one language
in PanLex. We experimented with using string similarity scores in the case of
such missing translations, as opposed to the strategy described in §3.2. The re-
sults for CS<sub>SVR(L1+L2)</sub> rose to r = 0.269, although this is still below the correlation
for just using string similarity.

Our results on the GNC dataset using string similarity to measure the com-
positionality of the whole compound are competitive with the state-of-the-art
results (r = 0.45) using a window-based distributional similarity approach over
monolingual German data by adding the modifier and head predictions (Schulte
im Walde et al. 2013).<sup>15</sup> Note, however, that their method used part-of-speech in-
formation and lemmatisation, where ours does not, in keeping with the language-
independent philosophy of this research. Furthermore, as shown in §5, our string
similarity measure can be substantially improved on GNC by using a multilingual
dictionary with higher coverage for the expressions in this dataset.

7 Conclusion

This chapter presented an extension of two previous studies – Salehi & Cook (2013) and Salehi et al. (2014) – that proposed supervised and unsupervised methods to predict the compositionality of MWEs based on measures of string similarity between the translations of an MWE, and translations of its component words, into many target languages, and based on distributional similarity between an MWE and its component words, both in the original source language and under translation.

In experiments using the string similarity approach, we showed that informa-
tion from translations into multiple target languages can be effectively combined
to give improvements over using just a single target language. We also showed
that string similarity measures which capture information about character se-
quences perform better than measures that do not. From the experiments on
unsupervised approaches, we learned that languages of the same family as the
source language cannot predict the compositionality of MWEs as well as the lan-
guages for which we have good translations coverage.

For distributional similarity, our experimental results showed that incorpo-
rating information from translations into target languages improved over using

<footnote n="15"><sup>15</sup>Additionally, Schulte im Walde et al. (2013) showed that their method achieves the state-of-the-art results (*r* = 0.65) in predicting the compositionality of each individual component within the compound.</footnote>
distributional similarity in just the source language. Furthermore, we learned
that there is a strong complementarity between approaches based on string and
distributional similarity.

Abbreviations

MWE multiword expression

ENC English Noun Compound dataset of Reddy et al. (2011)

EVPC English Verb-Particle Construction dataset of Bannard et al. (2003)

GNC German Noun Compound dataset of Schulte im Walde et al. (2013)

LCS longest common substring

LEV1 Levenshtein

LEV2 Levenshtein with substitution penalty

SW Smith Waterman algorithm

Acknowledgements

We thank the anonymous reviewers for their valuable comments, and the editors
for their time and effort in compiling this volume.

References

Acosta, Otavio, Aline Villavicencio & Viviane Moreira. 2011. Identification and treatment of multiword expressions applied to information retrieval. In *Proceedings of the Workshop on Multiword Expressions: From Parsing and Generation to the Real World (MWE’ 11)*, 101–109. Association for Computational Linguistics.

Baldwin, Timothy. 2009. The hare and the tortoise: Speed and accuracy in translation retrieval. *Machine Translation* 23(4). 195–240.

Baldwin, Timothy, Colin James Bannard, Takaaki Tanaka & Dominic Widdows. 2003. An empirical model of multiword expression decomposability. In *Proceedings of the ACL-2003 Workshop on Multiword Expressions: Analysis, Acquisition and Treatment (MWE ’03)*, 89–96. Association for Computational Linguistics. DOI:10.3115/1119282.1119294

Baldwin, Timothy & Su Nam Kim. 2010. Multiword expressions. In Nitin Indurkhya & Fred J. Damerau (eds.), *Handbook of Natural Language Processing, Second edition*, 267–292. Boca Raton: CRC Press.
Baldwin, Timothy, Jonathan Pool & Susan M. Colowick. 2010. PanLex and LEX-TRACT: Translating all words of all languages of the world. In *Proceedings of the 23rd International Conference on Computational Linguistics: Demonstrations* (COLING '10), 37–40. Association for Computational Linguistics.

Bannard, Colin James. 2006. *Acquiring phrasal lexicons from corpora*. University of Edinburgh dissertation.

Bannard, Colin James, Timothy Baldwin & Alex Lascarides. 2003. A statistical approach to the semantics of verb-particles. In *Proceedings of the ACL-2003 Workshop on Multiword Expressions: Analysis, Acquisition and Treatment* (MWE '03 1), 65–72. Association for Computational Linguistics.
DOI:10.3115/1119282.1119291

Biemann, Chris & Eugenie Giesbrecht. 2011. Distributional semantics and compositionality 2011: Shared task description and results. In *Proceedings of the Distributional Semantics and Compositionality Workshop* (DISCo 2011) in *conjunction with ACL 2011*, 21–28.

Carpuat, Marine & Mona Diab. 2010. Task-based evaluation of multiword expressions: A pilot study in Statistical Machine Translation. In *Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics*, 242–245. Association for Computational Linguistics. http://www.aclweb.org/anthology/N10-1029.

Dias, Gaël. 2003. Multiword unit hybrid extraction. In *Proceedings of the ACL 2003 Workshop on Multiword Expressions: Analysis, Acquisition and Treatment* (MWE '03), 41–48. Association for Computational Linguistics.

Evert, Stefan & Brigitte Krenn. 2005. Using small random samples for the manual evaluation of statistical association measures. *Computer Speech and Language* 19(4). 450–466.

Farahmand, Meghdad, Aaron Smith & Joakim Nivre. 2015. A multiword expression data set: Annotating non-compositionality and conventionalization for English noun compounds. In *Proceedings of the 11th Workshop on Multiword Expressions* (MWE '15), 29–33. Association for Computational Linguistics.

Fazly, Afsaneh, Paul Cook & Suzanne Stevenson. 2009. Unsupervised type and token identification of idiomatic expressions. *Computational Linguistics* 35(1). 61–103. http://aclweb.org/anthology/J09-1005.

Gomaa, Wael H & Aly A Fahmy. 2013. A survey of text similarity approaches. *International Journal of Computer Applications* 68(13). 13–18.

Hermann, Karl Moritz, Phil Blunsom & Stephen Pulman. 2012. An unsupervised ranking model for noun-noun compositionality. In *Proceedings of the First Joint*
13 Exploiting multilingual lexical resources to predict MWE compositionality

*Conference on Lexical and Computational Semantics (*SEM)*, 132–141. June 7-8, 2012.

Katz, Graham. 2006. Automatic identification of non-compositional multi-word expressions using latent semantic analysis. In *Proceedings of the ACL/COLING-06 Workshop on Multiword Expressions: Identifying and Exploiting Underlying Properties* (MWE ’06), 12–19. Association for Computational Linguistics.

Kim, Su Nam & Timothy Baldwin. 2007. Detecting compositionality of English verb-particle constructions using semantic similarity. In *Proceedings of the 7th meeting of the Pacific association for computational linguistics* (PACLING 2007), 40–48.

Korkontzelos, Ioannis & Suresh Manandhar. 2009. Detecting compositionality in multi-word expressions. In *Proceedings of the ACL-IJCNLP 2009 Conference-Short papers*, 65–68. August 4, 2009.

Lapata, Mirella & Alex Lascarides. 2003. Detecting novel compounds: The role of distributional evidence. In *Proceedings of the 11th Conference of the European Chapter for the Association of Computational Linguistics* (EACL-2003), 235–242.

Lin, Dekang. 1999. Automatic identification of non-compositional phrases. In *Proceedings of the 37th annual meeting of the association for computational linguistics on computational linguistics* (ACL 1999), 317–324.

McCarthy, Diana, Bill Keller & John Carroll. 2003. Detecting a continuum of compositionality in phrasal verbs. In *Proceedings of the ACL 2003 workshop on multiword expressions: Analysis, acquisition and treatment* (MWE ’03), 73–80. Association for Computational Linguistics. DOI:10.3115/1119282.1119292

McCarthy, Diana, Sriram Venkatapathy & Aravind K. Joshi. 2007. Detecting compositionality of verb-object combinations using selectional preferences. In *Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning* (EMNLP-CoNLL), 369–379. Association for Computational Linguistics. June 28–30, 2007.

Needleman, Saul B. & Christian D. Wunsch. 1970. A general method applicable to the search for similarities in the amino acid sequence of two proteins. *Journal of Molecular Biology* 48(3). 443–453.

Padó, Sebastian & Mirella Lapata. 2007. Dependency-based construction of semantic space models. *Computational Linguistics* 33(2). 161–199.

Pecina, Pavel. 2008. *Lexical association measures: Collocation extraction*. Prague, Czech Republic: Faculty of Mathematics and Physics, Charles University in Prague, Prague, Czech Republic dissertation.
Pichotta, Karl & John DeNero. 2013. Identifying phrasal verbs using many bilingual corpora. In *Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing* (EMNLP 2013). October 18-21, 2013.

Ramisch, Carlos. 2012. A generic framework for multiword expressions treatment: From acquisition to applications. In *Proceedings of ACL 2012 Student Research Workshop*, 61–66.

Reddy, Siva, Diana McCarthy & Suresh Manandhar. 2011. An empirical study on compositionality in compound nouns. In *Proceedings of the 5th International Joint Conference on Natural Language Processing (IJCNLP)*, 210–218.

Sag, Ivan A., Timothy Baldwin, Francis Bond, Ann A. Copestake & Dan Flickinger. 2002. Multiword expressions: A pain in the neck for NLP. In *Proceedings of the 3rd International Conference on Computational Linguistics and Intelligent Text Processing*, vol. 2276/2010 (CICLing ’02), 1–15. Springer-Verlag.

Saif, Abdulgabbar, Mohd Juzaiddin Ab Aziz & Nazlia Omar. 2013. Measuring the compositionality of Arabic multiword expressions. In *Proceedings of the second international multi-conference on artificial intelligence technology*, 245–256.

Salehi, Bahar, Narjes Askarian & Afsaneh Fazly. 2012. Automatic identification of Persian light verb constructions. In *Proceedings of the 13th International Conference on Intelligent Text Processing Computational Linguistics (CICLing ’12)*), 201–210.

Salehi, Bahar & Paul Cook. 2013. Predicting the compositionality of multiword expressions using translations in multiple languages. In *Second Joint Conference on Lexical and Computational Semantics*, vol. 1 (*SEM 2013*), 266–275. June 13-14, 2013.

Salehi, Bahar, Paul Cook & Timothy Baldwin. 2014. Using distributional similarity of multi-way translations to predict multiword expression compositionality. In *Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics* (EACL 2014), 472–481. Gothenburg. http://aclweb.org/anthology/E/E14/E14-1050.pdf.

Schone, Patrick & Daniel Jurafsky. 2001. Is knowledge-free induction of multiword unit dictionary headwords a solved problem. In *Proceedings of the 6th Conference on Empirical Methods in Natural Language Processing* (EMNLP 2001), 100–108. http://pascasarjana.mercubuana.ac.id/49/W01-0513.pdf.

Schulte im Walde, Sabine, Stefan Müller & Stefan Roller. 2013. Exploring vector space models to predict the compositionality of German noun-noun compounds. In *Second Joint Conference on Lexical and Computational Semantics (*SEM), Volume 1: Proceedings of the Main Conference and the Shared Task*, 255–265. Association for Computational Linguistics. June 13-14, 2013.
Schütze, Hinrich. 1997. *Ambiguity resolution in language learning*. Stanford, USA: CSLI Publications.

Smith, Temple F. & Michael S. Waterman. 1981. Identification of common molecular subsequences. *Molecular Biology* 147. 195–197.

Smola, Alex J. & Bernhard Schölkopf. 2004. A tutorial on support vector regression. *Statistics and Computing* 14(3). 199–222.

Vecchi, Eva Maria, Marco Baroni & Roberto Zamparelli. 2011. Linear maps of the impossible: Capturing semantic anomalies in distributional space. In *Proceedings of the Workshop on Distributional Semantics and Compositionality*, 1–9.

Venkatapathy, Sriram & Aravind K. Joshi. 2005. Measuring the relative compositionality of verb-noun (V-N) collocations by integrating features. In *Proceedings of the Conference on Human Language Technology and Empirical Methods in Natural Language Processing (HLT-EMNLP 2005)*, 771–778.

von der Heide, Claudia & Susanne Borgwaldt. 2009. Assoziationen zu Unter, Basis und Oberbegriffen. Eine explorative Studie. In *Proceedings of the 9th norddeutsches linguistisches Kolloquium*, 51–74.

Weeds, Julie Elizabeth. 2003. *Measures and applications of lexical distributional similarity*. University of Sussex dissertation.
<figure><img src="image_67.png" /></figure>
Name index

Abeillé, Anne, 134, 268
Acosta, Otavio, 344
Adali, Kübra, 110, 135, 136
Akimoto, Minoji, 38
Al Saied, Hazem, vi, 91, 166, 210, 240, 301
Algeo, John, 37
Ali, Aida, 310
Allen, James F., 62, 63, 80
Almeida, José João, 321, 328
Alonso-Ramos, Margarita, 39, 320
Alsina, Alex, 37
Altenberg, Bengt, 320
Amberber, Mengistu, 37
Arppe, Antti, 22
Attia, Mohammed, 179
Ayto, John, 4

Baayen, R. Harald, 3, 26, 27
Baggio, Giosuè, 88
Baldwin, Timothy, 35, 39, 65, 66, 92, 93, 131, 132, 135, 179, 223, 246, 301, 303, 344, 346, 347, 349, 351
Balota, David A., 16
Bannard, Colin James, 39, 64–66, 347–349, 354, 367, 369
Baptista, Jorge, 223
Bar, Kfir, 179
Barančíková, Petra, v, 36
Barlow, Michael, 1, 2
Baroni, Marco, 268

Barzilay, Regina, 36
Bates, Douglas, 8
Bauer, Laurie, 94
Bejček, Eduard, 42, 110, 135, 248
Bell, Susan M., 1
Berard, Alexandre, 326
Bernardini, Silvia, 268
Bhatia, Archna, v, 62, 63, 72
Bhutada, Pravin, 223, 266
Biemann, Chris, 265–268, 270, 347
Bies, Ann, 246, 247
Blunsom, Phil, 179, 223
Bobrow, Samuel A., 1
Bock, Kathryn, 4, 5, 12, 24
Bohnet, Bernd, 231
Bojar, Ondřej, 43, 52
Borgwaldt, Susanne, 349
Boroş, Tiberiu, 165, 223, 240
Boukobza, Ram, 179, 264
Boyd, Adriane, 248
Branco, António, 134
Brants, Sabine, 135
Bréal, Michel, 80
Bresnan, Joan, 27
Brinton, Laurel, 38
Brooke, Julian, vi
Buchholz, Sabine, 246, 273
Burger, Harald, 40
Butt, Miriam, 37

Cacciari, Cristina, 2, 8, 26, 27
Callison-Burch, Chris, 36, 41
Candito, Marie, 229, 247, 248, 264, 266

Carpuat, Marine, 264, 344

Čermák, František, 40, 43

Chafe, Wallace L., 36

Chan, King, 247

Chang, Jason S., 320, 322

Charles, Walter G., 3

Chen, Ping, 65

Chen, Wei-Te, 39

Church, Kenneth Ward, 321

Clark, Eve V., 80

Cohen, Jacob, 68

Collobert, Ronan, 179, 203, 223, 302

Connine, Cynthia M., 2, 9, 11, 18, 25, 27

Constant, Matthieu, 179, 211, 223, 229, 246–248, 264–268, 270, 291, 300–302

Cook, Paul, 40, 62, 64, 300, 302, 305, 312, 343, 345, 348, 350, 368

Cordeiro, Silvio, 288

Cowie, Anthony, 39, 40

Csirik, János, 136, 223

Cutler, Anne, 2, 26

Cutting, J. Cooper, 4, 5, 12, 24

Danchik, Emily, 252, 253, 264–268

de Marneffe, Marie-Catherine, 94, 247

de Medeiros Caseli, Helena, 39

De Smedt, Koenraad, 135, 256

Declerck, Thierry, 246

Dehé, Nicole, 65

DeNero, John, 347

Diab, Mona, 223, 264, 266, 344

Dias, Gaël, 344

Dickinson, Markus, 246, 249

Dillinger, Mike, 320

Dorr, Bonnie J., 36

Dryer, Matthew S., v

Duffley, Patrick J., 1, 2, 6

Ekbal, Asif, 291

Elman, Jeffrey L., 27

Emms, Martin, 179, 184, 189

Erjavec, Tomaz, 134

Eryiğit, Gülşen, 136, 223, 247, 248

Eskin, Eleazar, 249

Evans, Vyvyan, 62

Everaert, Martin, 40

Evert, Stefan, 320, 321, 323, 325, 344

Fahmy, Aly A, 352

Fanari, Rachele, 2, 9, 24

Farahmand, Meghdad, 300, 347

Fay, David, 4, 12, 24

Fazly, Afsaneh, 39, 216, 223, 264, 302, 305, 344, 346, 348

Fellbaum, Christiane, 62

Fillmore, Charles J., 36, 63

Finkel, Jenny Rose, 247

Finlayson, Mark, 264

Fleiss, Jacob L., 68

Fletcher, Jeremy, 64

Forcada, Mikel L., 328

Fort, Karën, 115

Fothergill, Richard, 303

Fotopoulou, Angeliki, 247

Fraser, Bruce, 39, 64, 65

Fujita, Atsushi, 39

Fung, Pascale, 323

Funke, Frederik, 7

Gamallo, Pablo, 327

Ganitkevitch, Juri, 41

Garcia, Marcos, vi, 321, 323, 327

Geeraert, Kristina, v, 3, 9, 28
Gharbieh, Waseem, 301, 303, 312
Gibbs, Raymond W., 1-3, 19, 24, 26
Giesbrecht, Eugenie, 347
Goldberg, Yoav, 213, 224
Gomaa, Wael H, 352
Granger, Sylviane, 40, 320
Green, Spence, 179, 229, 247, 266
Gries, Stefan Th., 161
Gross, Gaston, 93
Gross, Maurice, 133

Habash, Nizar, 246
Hajič, Jan, 110, 135
Hall, Mark, 191
Han, Aaron Li-Feng, 180
Hanks, Patrick, 321
Harris, Zellig, 41
Haruno, Masahiko, 322
Haspelmath, Martin, v
Healy, Adam, 39
Hermann, Karl Moritz, 347
Hiemstra, Djoerd, 321, 329
Hirotani, Masako, 20
Hollenstein, Nora, 248
Hosseini, Mohammad Javad, 179
Hovy, Eduard, 246
Hwa, Rebecca, 41

Ide, Nancy, 246
Ishizaki, Yasuaki, 64

Jackendoff, Ray, 62, 64, 65, 300
Janssen, Theo M. V., 88
Järvikivi, Juhani, 22
Jespersen, Otto, 102
Johnson, Christopher, 63
Joshi, Aravind K., 179, 347, 348
Jurafsky, Daniel, 344, 347
Kaalep, Heiki-Jaan, 136

Kato, Yoshihide, 249
Katz, Graham, 41, 300, 302, 347
Kauchak, David, 36
Kermes, Hannah, 320
Kettnerová, Václava, v, 36, 39, 42
Kilgarriff, Adam, 163, 305, 320
Kim, Su Nam, 35, 39, 88, 92, 93, 131,
132, 246, 301, 344, 346
Kiparsky, Paul, 80
Klyueva, Natalia, 165, 223, 240
Koehn, Philipp, 188
Koeva, Svetla, 115
Kordoni, Valia, 300
Korkontzelos, Ioannis, 247, 347
Kracht, Marcus, 88, 104
Křen, Michal, 43
Krenn, Brigitte, 135, 325, 344
Krippendorff, Klaus, 51
Kuhn, Jonas, 39
Kulkarni, Nidhi, 264
Kupiec, Julian, 320, 322

Lafferty, John D., 178, 180, 267
Landauer, Thomas K., 16
Langlotz, Andreas, 2
Lapata, Mirella, 362, 366
Laporte, Éric, 131-134, 272
Lascarides, Alex, 366
Le Roux, Joseph, 265, 287, 288
Legrand, Joël, 179, 203, 223, 302
Li, Linlin, 40
Libben, Maya R., 26
Lin, Dekang, 321, 323, 346
Lipka, Leonhard, 94
Lison, Pierre, 321, 327
Liu, Changsheng, 41
Loftsson, Hrafn, 249
Lopatková, Markéta, 39
Lü, Yajuan, 320, 322, 323
Luong, Minh-Thang, 326

Ma, Qing, 249

Madnani, Nitin, 36

Maldonado, Alfredo, vi, 91, 125, 150,
159, 165, 178–180, 184, 189,
223, 240, 312

Manandhar, Suresh, 247, 347

Manning, Christopher D., 158, 247

Marcus, Mitchell P., 115, 268

Marsi, Erwin, 246, 273

Martins, Ronaldo, 300

Marton, Yuval, 36

Mathet, Yann, 111, 121

Matsubara, Shigeki, 249

McCarthy, Diana, 65, 344, 347, 348

McDonald, Ryan, 134, 136

McGlone, Matthew S., 1, 3, 24, 27

Mel'čuk, Igor A., 39, 93, 131, 132, 320,
325

Mertens, Piet, 268

Meunier, Fanny, 40

Meurers, W. Detmar, 246, 249

Mikolov, Tomas, 41, 326

Miller, George A., 3, 62

Moon, Rosamund, 1, 2, 4–6, 24

Moreau, Erwan, vi, 91, 165

Moreno-Ortiz, Antonio, 62

Muischnek, Kadri, 136

Muzny, Grace, 40

Nagy T., István, 216, 223

Nasr, Alexis, 229, 264–269, 271, 274,
280, 285–287, 300

Nayak, Nandini P., 2, 3, 19

Needleman, Saul B., 352

Nerima, Luka, 166, 240

Nesselhauf, Nadja, 336

Newman, John, 3, 28

Niinuma, Fumikazu, 64

Nilsson, Jens, 264, 266, 300

Nivre, Joakim, 91, 134, 211, 213, 217,
223, 224, 229, 234, 246–248,
264–267, 300, 327

Nunberg, Geoffrey, 39, 131, 133

O'Dowd, Elizabeth M., 65

Oepen, Stephan, 248

Okazaki, Naoaki, 268

Onuffer, Spencer, 246, 247, 270

Orliac, Brigitte, 320

Owoputi, Olutobi, 253

Padó, Sebastian, 362

Pagin, Peter, 88

Pal, Santanu, 301

Papineni, Kishore, 50

Partee, Barbara H., 88

Pasquer, Caroline, 127

Patrick, Jon, 64

Paul, Hermann, 80

Pausé, Marie-Sophie, 132

Pecina, Pavel, 325, 344

Peng, Jing, 41

Pennington, Jeffrey, 307

Pershina, Maria, 41

Pichotta, Karl, 347

Powers, David M. W., 170

QasemiZadeh, Behrang, vi, 91, 125,
136, 184, 312

Qu, Lizhen, 256, 302

Quinlan, J. Ross, 191

Radimský, Jan, 37

Rahimi, Saeed, 136

Ramisch, Carlos, 265, 268, 300–302,
309, 344
Ramscar, Michael, 27

Ramshaw, Lance, 268

Rappoport, Ari, 179, 264

Rayner, Keith, 20

Reddy, Siva, 344, 347, 348, 353, 363, 366, 369

Řehůřek, Radim, 46

Reips, Ulf-Dietrich, 7

Reynaert, Martin, 113

Riedl, Martin, 265–268, 270

Rivera, Oscar Mendoza, 320, 322

Rose, Tony, 163

Rosén, Victoria, 134

Roth, Dan, 135

Sag, Ivan A., 35, 39, 88, 93, 131, 132, 150, 178, 179, 247, 264, 300, 320, 344

Sagot, Benoît, 115

Saha, Sriparna, 291

Saif, Abdulgabbar, 348

Salehi, Bahar, vi, 179, 300, 343–345, 348, 350, 361, 368

Salton, Giancarlo, 50, 305

Savary, Agata, vi–viii, 2, 5, 6, 91, 112, 117, 120, 122, 125, 134, 138, 150, 159, 161, 166, 170, 178, 209, 210, 219, 228, 233, 272, 276, 301, 312

Schneider, Nathan, 131–133, 136, 179, 183, 223, 246, 247, 252, 253, 264–268, 270, 300–302

Scholivet, Manon, vi, 2, 265, 302, 309

Schone, Patrick, 344, 347

Schröder, Daniela, 2, 4, 5, 24

Schulte im Walde, Sabine, 347–349, 368, 369

Schütze, Hinrich, 158, 179, 184, 186, 362

Schölkopf, Bernhard, 363

Seddah, Djamé, 272

Seraji, Mojgan, 136

Seretan, Violeta, 216, 247, 320–322

Sheinfux, Livnat Herzig, 131–133

Shigeto, Yutaro, 264, 266, 267

Siemieniec-Gołaś, Ewa, 110

Sigogne, Anthony, 265–268, 270, 291

Silberztein, Max, 264

Simi, Maria, 246

Simkó, Katalin Ilona, vi, 91, 166, 228, 233, 234

Simões, Alberto Manuel, 321, 328

Simov, Kiril, 249

Sinclair, John, 4, 178

Sinha, R. Mahesh K., 39

Smadja, Frank, 320–323

Smith, Temple F., 352

Smola, Alex J., 363

Sojka, Petr, 46

Sporleder, Caroline, 40

Stefanowitsch, Anatol, 161

Stevenson, Suzanne, 62, 64

Straňák, Pavel, 135

Stubbs, Michael, 325

Suderman, Keith, 246

Sun, Xiao, 179

Swinney, David A., 2, 26

Szarvas, György, 91

Tabossi, Patrizia, 2, 9, 26, 27

Taslimipoor, Shiva, vi, 91, 125, 301, 304

Tellier, Isabelle, 300, 301

Teng, Choh Man, 62, 63

Tiedemann, Jörg, 321, 327

Titone, Debra A., 2, 9, 11, 18, 25–27

Tsvetkov, Yulia, 179

Tu, Yuancheng, 135
Tyler, Andrea, 62

Ule, Tylman, 249

van den Eynde, Karel, 268

van der Linden, Erik-Jan, 39

van Gompel, Maarten, 113, 115

Vecchi, Eva Maria, 348

Venkatapathy, Sriram, 179, 347, 348

Villavicencio, Aline, 62, 64, 68

Vincze, Veronika, 64, 134–136, 179, 216, 223, 229, 247, 266

von der Heide, Claudia, 349

Wallis, Peter, 36

Wanner, Leo, 322

Waszczuk, Jakub, 229

Waterman, Michael S., 352

Weeds, Julie Elizabeth, 362

Wehrli, Eric, 223, 229, 247, 322

Westerståhl, Dag, 88

Wickham, Hadley, 203

Wintner, Shuly, 179

Wolf, Kinou, 2, 9

Wu, Chien-Cheng, 320, 322

Wunsch, Christian D., 352

Zarrieß, Sina, 39

Zeman, Daniel, 273

Zettlemoyer, Luke S., 40

Zhang, Yue, 217

Zhou, Liang, 36

Zhou, Ming, 320, 322, 323

Žolkovskij, Alexander K., 39
Language index

BG, see Bulgarian

Bulgarian, 91, 109, 166, 167, 180, 210, 217, 220

CZ, see Czech

Czech, v, 35–38, 40–43, 47, 49, 51–53, 91, 109, 110, 135, 150, 154, 161, 180, 183, 184, 210, 217, 221

Danish, 357

DE, see German

Dutch, 357

EL, see Greek

EN, see English

English, v–vii, 1, 51, 61, 135, 223, 237, 246, 247, 250, 253, 255, 266, 288, 302, 304, 305, 307–310, 312, 320–322, 324–328, 331–333, 335, 336, 344, 346–350, 357–359, 361, 364, 366

ES, see Spanish

Estonian, 136

FA, see Farsi

Farsi, vii, 91, 108, 136, 150, 156, 159, 161, 166, 180, 184, 188, 217, 220, 233, 236, 350

FR, see French

French, vi, 91, 127, 151, 161, 163, 165, 166, 180, 182–184, 187, 188, 200, 217, 232, 233, 265, 266,

268, 271–273, 275, 280, 291, 302, 322, 350, 356, 358

German, vi, 91, 109, 135, 156, 161, 167, 180, 182–184, 210, 212, 217, 229, 233, 236–238, 345, 348, 349, 356–359, 367, 368

Greek, 91, 161, 184, 233

HE, see Hebrew

Hebrew, vii, 91, 110, 150, 163, 166, 167, 180, 210, 217, 220

HU, see Hungarian

Hungarian, 91, 135, 156, 158, 159, 161, 166, 168, 184, 210, 220, 228–230, 233, 236–239

Icelandic, 357

IT, see Italian

Italian, vi, 91, 109, 165, 183, 184, 191, 220, 232, 233, 235, 236, 304, 305, 307, 308, 310, 312

Lithuanian, 91, 110, 128, 150, 156, 158, 166, 167, 180, 210, 217, 356

LT, see Lithuanian

Luxembourgish, 357

Maltese, 91, 128, 150, 154, 156, 158, 180, 184, 188, 210, 217, 233, 235, 236

MT, see Maltese
Language index

Norwegian, 357

Persian, see Farsi

PL, see Polish

Polish, 91, 150, 161, 166, 167, 180, 182–
184, 232, 233

Portuguese, vi, 233, 320, 321, 323–
329, 331–337, 355

Brazilian, 91, 161, 165, 183, 184,
221

PT, see Portuguese

RO, see Romanian

Romanian, 91, 150, 154, 156, 161, 165,
166, 172, 180, 184, 210, 217,
220, 232, 233, 236

SL, see Slovene

Slovene, 91, 109, 220, 233

Spanish, vi, 91, 154, 158, 168, 183, 184,
210, 217, 233, 304, 305, 307,
308, 312, 320, 321, 323–329,
331–333, 335–337, 361

SV, see Swedish

Swedish, 91, 118, 150, 154, 184, 233,
236, 357

TR, see Turkish

Turkish, 91, 110, 136, 161, 165, 168, 184,
188, 232, 233
Subject index

ambiguity, 264

categorical, 89

literal-idiomatic, 89

syntactic, 89

annotation

arbitration, 246, 250

consistency, vi, 116, 117, 246, 248,
250, 256

guidelines, vi, 91, 99

hard cases, 249, 251, 254

harmonization, 246

manual, 42, 46, 48, 51

association measure, 276, 282

BIO encoding, 268, 270, 272

classification, vi, 170, 180, 190, 223,
300–305, 309, 329

Cohen's kappa, 169, 170

collocation, vi, 93, 134, 189, 320–324,
326–332, 334, 336, 337

base, 321

collocate, 321

compositionality, v, vi, 1, 2, 9, 11, 12,
16, 19, 26, 27, 88, 104

conditional random fields, 178, 265,
267

context vector, 184, 186, 187, 189, 201,
203

corpus

heterogeneity, 152, 163

homogeneity, 151, 163

cranberry word, 99

CRF, see conditional random fields

cross-language divergence, 90

dependency

graph-based, 267

parsing, see parsing

relation, 182, 183, 229, 231, 266,
267, 322

treebank, 246, 248

dictionary lookup baseline, 167

dictionary of paraphrases, 36, 49

distributional semantics, 184, 186,
203, 321

distributional similarity, 65, 344, 346,
347, 354, 356, 361–364, 366–
369

language-independent measure,
361

English Web Treebank, 246, 247, 253

Europarl corpus, 179, 186, 188, 198

eye-tracking, v, 3, 14, 15, 18, 20–22

feature, 267, 304

selection, 277, 279, 281, 282

sparsity, 278

template, 180, 182, 210, 215, 268

figuration, 40, 133

FLAT, 113

FoLiA, 113

French Treebank, 272, 283
frequency distribution, 156, 158, 159

grammatical aspect, 36, 38, 50, 129

IAA, see inter-annotator agreement

ID, see verbal idiom

idiom, 1–5, 9, 11, 13–16, 18, 19, 21–27

blend, see idiomatic variation

verbal, 96, 104, see idiomatic verbal construction

idiomatic reading, 89

idiomatic translation, vii, viii

idiomatic variation, v, 2, 9, 13, 21–24, 26, 89, 127, 131, 236, 264, 289, 290

acceptability, v, 3, 4, 7, 9–11, 13–16, 19, 22–26

comprehension, v, 3, 4, 14, 16–27

idiom blend, 4, 5, 7, 9, 10, 12, 15, 18, 20, 21, 23–25, 28

integrated concept, 2, 4, 5, 7, 9, 10, 13–16, 18, 20, 21, 23, 24, 28

lexical, 2–6, 9, 10, 14, 15, 18, 20, 23, 28

morphological, 271, 290

partial form, 2, 4, 5, 7, 9, 10, 13, 15, 16, 18, 20, 21, 23, 24, 28

syntactic, 2, 5

idiomatic verbal construction

linguistic properties, 36, 39

selection, 36, 43

inflexibility

lexical, 89, 99

morpho-syntactic, 99

morphological, 99

syntactic, 89, 99

inherently reflexive verb, 96, 106, 236

inter-annotator agreement, 51, 63, 67, 70, 74, 119, 305, 329

IRV, see inherently reflexive verb

IVC, see idiomatic verb construction

language learning, 320

lexicalised component, 92, 94

light-verb construction, 37, 96, 102, 128, 236

hypothesis, 99

linguistic properties, 36, 37

selection, 36, 38, 42

literal meaning, 1, 2, 5, 6, 9, 11, 18, 23, 25–27, 89, 300

literal reading, see literal meaning

literal translation, vii

LVC, see light-verb construction

machine translation, v, 36, 50–53, 179, 320

metaphor, 40, 130

mixed-effects linear regression, 8, 15

MT, see machine translation

MTW, see multitoken word

multilingual dictionary, 348, 349, 358, 368

multitoken word, 92

multiword expression, 1, 2, 16, 22, 92

annotated corpus, 88, 91, 246, 247, 249, 256, 272

annotation, 134

compositionality, 179, 344–350, 353–368

context-based identification, 304

continuity, 183, 186, 189, 201, 202, 268

identification, iv, vi, 39, 40, 43, 178–180, 247, 252, 256, 263,
265, 266, 285, 300, 301, 303, 309, 312
identification evaluation, 303, 306, 307
identification modelling, 300, 303, 309
nominal, 271, 281
notational conventions, vii
overlapping, 276
sparsity, 277
strong, 248
strongish, 249, 252, 254
type-aware classification, 307
type-aware evaluation, 307, 310
typology, 131
unseen, 289, 290
variability, see idiomatic variation
weak, 248, 249, 252, 254, 255
multiword token, 92, 212
multiword unit, see multiword expression
MWE, see multiword expression
MWE 2017 workshop, ix
MWT, see multiword token
named entity, 254, 255, 266, 272
non-compositionality, 302
non-compositionality, 62, 65–68, 70, 71, 74, 99, 131, 178, 251
semantic, iii, 88, 229, 344
non-literal translatability, 90
noun compound, 348, 349, 366, 367
parallel corpora, 39, 320–323, 327–329, 337, 347
paraphrasing model, 36, 39, 41
PARSEME, iv, viii, 91, 150, 178, 229
corpus, vi, viii, 88, 91, 152, 154

shared task, iv, vi, viii, 150, 161, 163, 165, 178, 191, 210, 219, 228, 229, 232, 234, 276, 301
parseme-tsv, 119
parsing
dependency, vi, 211, 228, 229, 231–234, 239, 266, 273, 321, 323
semantic, 62, 63, 78, 83
transition-based, vi, 210–212, 215–217
part-of-speech tagging, 228, 229, 231–233, 239, 249, 266
particle sense classes, 61, 63, 64, 73
aspectual, 73, 74
completely, 66, 72, 73, 82
continuing, 73, 74
direction, 62, 74, 81
direction-away, 73
direction-out, 73
direction-up, 73
disappearance, 74
idiomatic, 74
in-working-order-val, 74
not-in-working-order-val, 74
pre-annotation, 115
predictability, 3, 19, 25–27
productivity, 130
prototypical verbal phrase, 94
reranking, vi, 178, 179, 184, 188, 190–198, 202
sequence labelling, vi, 286, 302, 303, 309
sequence model, 178, 265
SIGLEX, ix
string similarity, 345, 350, 353, 354, 356, 361, 363, 364, 366–368
support vector machine, 214, 223

SVM, see support vector machine

tagging, see sequence labelling

token, 92

transparency, 133

type-aware classification, 312

UD, see Universal Dependencies

Universal Dependencies, 229, 234, 235, 239, 247, 267, 323, 327

valence lexicon, 268, 275

verb detection baseline, 167

verb-particle construction, 61, 62, 65–67, 96, 107, 233, 236, 347, 349

classification, 64–67

compositionality, 61–68

heuristics for compositionality, 63, 69–71, 74

verbal multiword expression, 88, 93

canonical form, 95

categorization, 209, 214, 220

decomposability, 131

difficulty factors, 152

discontinuity, 89, 122, 150, 161, 167

embedding, see verbal multi-word expression nesting

identification, 99, 150, 165, 209, 216, 217, 223, 228, see multi-word expression identification

nesting, 126, 150, 161, 167, 183, 184

other, 97

overlapping, 126, 150, 161, 167, 183, 184, 214, 234

paraphrasability, v, 35, 37, 39, 40, 52, 53

quasi-universal category, 96

seen, 159

sparsity, 44, 150, 152, 154, 156, 158

structural test, 100

typology, 49, 96

universal category, 96

unseen, 159, 163

variability, see idiomatic variation

verb-noun, 179, 223, 266, 305

VMWE, see verbal multiword expression

VPC, see verb-particle construction

word, 92

word embeddings, vi, 41, 43, 179, 203, 307, 309, 310, 321, 323, 326, 328, 329, 334, 337

wordplay proneness, 90
<figure><img src="image_68.png" /></figure>
Did you like this
book?

This book was brought to you for free

Please help us in providing free access
to linguistic research worldwide. Visit
http://www.langsci-press.org/donate to
provide financial support or register as
a community proofreader or typesetter
at http://www.langsci-press.org/register.

<figure><img src="image_69.png" /></figure>

language
science
press
<figure><img src="image_70.png" /></figure>
Multiword expressions at length and
in depth

The annual workshop on multiword expressions takes place since 2001 in conjunction
with major computational linguistics conferences and attracts the attention of an ever-
growing community working on a variety of languages, linguistic phenomena and re-
lated computational processing issues. MWE 2017 took place in Valencia, Spain, and
represented a vibrant panorama of the current research landscape on the computational
treatment of multiword expressions, featuring many high-quality submissions. Further-
more, MWE 2017 included the first shared task on multilingual identification of verbal
multiword expressions. The shared task, with extended communal work, has developed
important multilingual resources and mobilised several research groups in computa-
tional linguistics worldwide.

This book contains extended versions of selected papers from the workshop. Authors
worked hard to include detailed explanations, broader and deeper analyses, and new ex-
citing results, which were thoroughly reviewed by an internationally renowned com-
mittee. We hope that this distinctly joint effort will provide a meaningful and useful
snapshot of the multilingual state of the art in multiword expressions modelling and
processing, and will be a point of reference for future work.

ISBN 978-3-96110-123-8

<figure><img src="image_71.png" /></figure>

9 783961 101238