# Graph-based multi-layer querying in Parseme Corpora

Bruno Guillaume

► To cite this version:

Bruno Guillaume. Graph-based multi-layer querying in Parseme Corpora. Proceedings of the 19th Workshop on Multiword Expressions (MWE 2023), Jun 2023, Dubrovnic, Croatia. pp.58-64, 10.18653/v1/2023.mwe-1.9. hal-04387852

HAL Id: hal-04387852

https://inria.hal.science/hal-04387852v1

Submitted on 11 Jan 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
Graph-based multi-layer querying in Parseme Corpora

Bruno Guillaume

Université de Lorraine, CNRS, Inria, LORIA, F-54000 Nancy, France

Bruno.Guillaume@inria.fr

**Abstract**

We present a graph-based tool which can be
used to explore Verbal Multi-Word Expres-
sion (VMWE) annotated in the Parseme project.
The tool can be used for linguistic exploration
on the data, for helping the manual annotation
process and to search for errors or inconsisten-
cies in the annotations.

# 1 Introduction

The Parseme project (Monti et al., 2018) proposes a
large set of annotated data with Verbal Multi-Word
Expressions (VMWE). In version 1.2 (Ramisch
et al., 2020), 14 languages were covered but with
older versions and ongoing work<fnref n="1" />, there are now
data in 26 languages (See Table 4 in appendix for
list of languages and the number of sentences for
each language). In the last release, Parseme 1.3,
only “verbal” Multi-Word Expressions are anno-
tated; the annotation of other categories is planed
for future releases.

Parseme data is published with associated
morpho-syntactic annotations, in accordance with
the Universal Dependencies (de Marneffe et al.,
2021) framework. Some parseme annotations are
directly built on data available in the UD project.
In this case, we have both high-quality morpho-
syntactic annotations and VMWE annotations on
the same data. Other parts of the Parseme data,
which are not built on existing UD data, are accom-
panied by an automatic morpho-syntactic annota-
tion, obtained with UDPipe (Straka et al., 2016),
thus also following the UD annotation frame-
work. This means that all annotated data from the
Parseme project can be considered as multi-layer
annotated data, with morphosyntactic annotations
encoded following UD and VMWE.

In this article, we propose an encoding of the
two annotation layers in a common structure, using

a graph encoding of both UD and VMWE anno-
tations. With this encoding, it is possible to use
graph-based tools to work with the data. In this
work, we use our GREW tool (Guillaume, 2021) to
make queries on the two layers.

The Parseme 1.3 data will be released on
http://hdl.handle.net/11372/LRT-5124. At
the time of the final version of the paper, these data
are not available. The experiments reported in the
paper are done on a preliminary version of the data
provided by the Parseme team. We cannot exclude
minor differences between the data we used in our
observations and the official 1.3 data.

In Section 2, we explain the encoding. The next
sections give examples of usage with general obser-
vations in Section 3, applications to error mining
in Section 4 and some more comprehensive study
of the consistency between UD and Parseme anno-
tation layers in Section 5.

2 Graph encoding

The two annotation layers (UD and VMWE) are
stored in a common technical format (CUPT)<fnref n="2" />, but
it is not straightforward to consider both in the
same structure. In UD, each sentence is split in a
sequence of tokens and each Parseme annotation
consists in identifying a subset of the tokens of the
sentence which correspond to a VMWE. In addi-
tion to the subset, a tag is given to each VMWE.

The Parseme guidelines³ describes the set of
tags and their definitions. The tagset contains
three universal tags: LVC.FULL, LVC.CAUSE for
light verb constructions and VID for verbal id-
ioms. Three quasi-universal categories are also
defined: VPC.FULL, VPC.SEMI for verb-particle
constructions and MVC for multi-verb construc-
tions. A few other tags are used in the corpora: the
IAV for inherently adpositional verbs, presented

<footnote n="1">¹https://gitlab.com/parseme/corpora</footnote>

<footnote n="2">²http://multiword.sourceforge.net/cupt-format
³https://parsemefr.lis-lab.fr/parseme-st-guidelines/1.3/</footnote>
as experimental in the Parseme guidelines and the
tag LS.ICV for inherently clitic verbs (currently
only used in Italian data). Some development ver-
sions of the data makes also use of the special
tag NOTMWE which, as its name indicates, does
not encode a VMWE, it is used in the consistency
checking mechanism<fnref n="4" />.

There are two difficulties in the encoding.

* A VMWE is not always a span of the original text, or in other words it does not always contains a subset of consecutive tokens of the sentence. For instance, in the sentence *Take a look !!!*, the subset containing *Take* and *look* is annotated with the tag LVC.FULL the token *a* between the two elements not being part of the VMWE.

* The second problem is that the same token can be included in more than one VMWEs. In the sentence [...] to get rid of the moral burden [...], two subsets are annotated independently: the 3 elements *get*, *rid* and *of* are tagged as IAV (Inherently adpositional verbs) and the 2 elements *get* and *rid* are tagged as MVC (Multi-verb constructions). Such VMWE annotations will be called *overlapping* VMWEs.

In order to take into account these difficulties,
we propose to encode the two layers in a single
graph structure. Our graphs contain two kinds of
nodes and two kinds of edges:

* UD nodes and UD edges which encode the lexical tokens and the dependency relations between tokens

* Parseme nodes and Parseme edges which encode VMWEs: each VMWE is represented by a new node with a feature named *label* which stores the tag. The node associated with a VMWE is linked with Parseme edges to all the UD token it contains.

Figure 1 shows a simplified picture of the encod-
ing of the two overlapping VMWEs in the sentence
[...] to get rid of the moral burden [...]. UD
nodes and relations are drawn in black whereas
Parseme nodes and edges are drawn in blue and
below the sentence.

<figure><img src="image_2.png" /><figcaption>Figure 1: Graph representation (simplified) of two overlapping VMWE annotations</figcaption></figure>

Note that Parseme nodes are not really inserted
in the linear structure of the sentence. By conven-
tion, these nodes are drawn before the first token
of the subset, just to ease the reading of the figures.

3 Multi-layer queries

The benefit of having the two annotation layers
in the same structure is that it is possible to make
queries which refer to both layers and then to make
cross observations. We use the GREW tool which
allows to write graph queries that can be executed
on the Parseme corpora represented has above. The
tool is available in an online web interface: GREW-
MATCH<fnref n="6" /> on a predefined set of treebanks. A Python
library, named GREWPy, is also available to use
queries in scripts.

We give a few examples of GREW queries on the
Parseme graph encoding.

3.1 VWMEs by types

Using the fact that all Parseme nodes have a feature
named *label* (and that UD nodes do not have such
a feature), the simple request below returns the set
of all annotated VMWEs.

1 pattern { MWE [label] }

In the request, MWE is an node identifier. The
query can be rephrased as: “search for any node
having a feature named label and call this node
MWE”.

GREW proposes a mechanism to cluster the output of a query following some criterion. With the clustering key MWE.label, the set of solutions of the previous query is clustered in a partition of subsets according to the value of the feature label of the node MWE.

In Table 1 in appendix, each line correspond to
the size of the clusters obtained for each language
in the Parseme data.

<footnote n="4">⁴Script consistencyCheckWebpage.py available in
https://gitlab.com/parseme/utilities</footnote>

<footnote n="5">⁵English examples come from the English Parseme corpus:
https://gitlab.com/parseme/parseme_corpus_en</footnote>

<footnote n="6">⁶http://parseme.grew.fr</footnote>
## 3.2 VWMEs by sizes

We keep the same basic request used in the previous subsection. In GREW, the clustering key `MWE.__out__` splits the occurrences returned by a request depending on the number of outgoing edges on the node `MWE`. Following the encoding described in Section 2, this corresponds to the number of tokens implied in the VMWE. Table 2 in appendix reports the sizes of the clusters obtained with this clustering for all languages.

According to the notion of Multi-Word Expression, we do not expect to have one-token annotation as VMWE. All languages have a few number of such VMWEs (above 50 occurrences) except for four languages: Hungarian, Chinese, Swedish and German.

In Hungarian data, there are 5745 one-token VMWEs; this is probably linked to the fact that Hungarian is an agglutinative language; among the cases, the same noun with lemma *bekezdés* 'paragraph' appears 995 times, it is tagged as VPC.FULL and it is built from the verb *bekezd* 'to indent' and with a noun-forming suffix -és<fnref n="7" />.

There are 5382 cases in Chinese, but Chinese, not using whitespaces, is well known to be a language in which tokenization is challenging.

In Swedish, there are 1614 occurrences and 1268 occurrences in German. In both languages, the major part of cases are particle verbs. In German, the four most frequent lemmas are: *einsetzen* 'to insert', *anbieten* 'to offer', *ankündigen* 'to announce', *mitteilen* 'to share'. Unlike English where particles of particle verbs always remain separate words (*put off*, *to put off*), German particles of infinitival forms are fronted and spelled as one word, joining the main verb (*abschrecken* 'to put off', *abgeschreckt* 'be put off'). whereas particles of finite verb forms are positioned behind the main verb and spelled as separate words (*schreckt* ab 'put off') just as in English. In Swedish, just as in German, there are many particle verbs that alternate between realizing the particle as a separate word and as a prefix. This shows that the notion of tokenisation is considered quite differently in both projects.

Apart from one-token annotations, the size 2 is the most common setting in all languages. Size 6 and higher are quite rare and the maximum is reached by Hebrew with one VMWE containing 13 tokens.

## 3.3 Ratio of overlapping VMWEs

With a graph request, we can distinguish VMWE annotations with or without overlapping. The request below corresponds to the “without overlapping” case:

<figure><img src="image_3.png" /></figure>

Lines 1-3 is a request for any VWME. Lines 4-7 use the *without* construction of GREW which filters the output of a query: each occurrence of the basic query (line 1-3) which satisfies the constraint expressed in the *without* part is filtered out. In our example, cases where some MWE2 exists, which shares a token X with the one previously found MWE1 are removed. The result of the full query is then only the non-overlapping VMWEs.

For the “with overlapping” case, the request is the same where the keyword *without* is replaced by the keyword *with* (line 4). Table 3 shows the ratio of overlapping VMWEs for each language.

# 4 Error mining

One of the common usage of GREW and mainly GREW-MATCH is error mining. By looking at all examples of a given query, we can spot inconsistencies and potential annotation errors. A first example of error mining is to explore the occurrences of one-token VMWEs (see Subsection 3.2) which are unexpected and require manual inspection. Let us see a few other examples.

## 4.1 VMWEs without any verb

We can test whether each annotation does contain a verb. This is expected as the current version of Parseme focuses on “Verbal” Multi-Word Expressions. The following request searches for Parseme VMWEs without any verb, according to UD annotation (UD uses the two POS tags AUX and VERB for the verbal forms).

<figure><img src="image_4.png" /></figure>

<footnote n="7">⁷https://en.wiktionary.org/wiki/bekezdés</footnote>
Table 4 in appendix gives the numbers of occurrences in each language for this request (column **no_verb**). The median of the number of occurrences in the 26 treebanks is 91.5, with two treebanks above 1000 occurrences. The two exceptions are Hungarian (we have already seen that many nouns are tagged as VMWE because there are built from a verb and a noun-forming affix) and Arabic (where we also observe many cases of noun describing an action, build on a verbal root). This shows that the definition of what is a “verb” in Parseme is not fully aligned with the UD policy.

## 4.2 Inherently reflexive verbs

Parseme consider the tag IRV for Inherently reflexive verbs. In the meantime in UD, there is the feature Reflex=Yes which can be used on reflexive pronouns (but this is not mandatory). We can expect that a VMWE annotated as IRV contains such a reflexive pronoun. The request above allows to search for the exceptions to this rule.

<figure><img src="image_5.png" /></figure>

Table 4 in appendix gives the numbers of occurrences in each language for this request (column **IRV_no_reflex**). For the three highest numbers are 1144 in Italian, 1021 in Portuguese and 237 in Swedish. This is due to the fact that these three languages does not annotate feature Reflex in the UD data. Romanian and French have both annotations IRV in Parseme and Reflex=Yes on pronoun in UD, but there are many inconsistencies.

Is it worth noting that Slovenian also appeared in the problematic languages at the submission time. but it was due to a bug in the data which was found thanks to the current work and which was fixed in the mean time.

# 5 Consistency UD/Parseme

In this section, we give a few examples of requests which can be used in GREW-MATCH to explore how some specific class of VWME is annotated in one treebank.

<figure><img src="image_6.png" /><figcaption>Figure 2: POS of the tokens used in the VPC construction in English</figcaption></figure>

## 5.1 Verb-particle constructions in English

The example runs on Verb-particle constructions (VPC) and on English data. According to Parseme guidelines, two subtags must be used: VPC.FULL for fully non-compositional VPC and VPC.SEMI for semi-non-compositional.

First, we can have a look at the distribution of this kind of VMWE according to the number of tokens implied<fnref n="8" />. We observed 421 occurrences (368 VPC.FULL and 53 VPC.SEMI)<fnref n="9" /> of this label, all of them having exactly two tokens.

Another request<fnref n="10" />, specifying the two tokens N1 and N2, can display the distribution of the POS of the tokens in the Figure 2 which shows that two constructions VERB-ADP and VERB-ADV covers all but 6 cases.

Exploring further<fnref n="11" />, we observed in the 217 VERB-ADP cases, a large majority (202) of annotation where the VERB is linked to the ADP with relation **compound:prt**. Other cases are: no direct relation between the two nodes (10 cases), relation **advmod** (3 cases), compound (2 cases). Similarly<fnref n="12" />, we observed in the 198 VERB-ADV cases, a majority (118) of annotation where the VERB is linked to the ADP with relation **compound:prt**. Other cases are: relation **advmod** (75 cases), no direct relation between the two nodes (2 cases), compound, **obl** and **xcomp** (1 case for each).

These irregularities in the annotation would require a careful inspection by a native English speaker but we can already see a bunch of annotation inconsistencies either in the UD annotation layer or in the Parseme one.

<footnote n="8">⁸http://parseme.grew.fr/?custom=63f1ee845234a</footnote>

<footnote n="9">⁹The numbers of this section are based on requests done on 2023/02/19, they may changed when the data is updated. Requests on a stable data from a release will be provided for final version.</footnote>

<footnote n="10">¹⁰http://parseme.grew.fr/?custom=63f1eeda64fe8</footnote>

<footnote n="11">¹¹http://parseme.grew.fr/?custom=63f1f03dea172</footnote>

<footnote n="12">¹²http://parseme.grew.fr/?custom=63f1f0d94e4ec</footnote>
<figure><img src="image_7.png" /><figcaption>Figure 3: Lemmas used with MVC label in French data. Translations of columns lemmas are ‘to do’, ‘to hear’ and ‘to let’. Translations of rows lemmas are ‘to speak’, ‘to remark’, ‘to leave’, ‘to fall’, ‘to be worth’ and ‘to pass’.</figcaption></figure>

## 5.2 MVC in French

There are 22 occurrences of the MVC in the French data, all having two tokens. All are continuous except one containing a negation on *n'entendra plus parler de... 'one will no more hear about...'. The Figure 3 shows the distribution of lemmas of the two tokens N1 and N2 (following the linear order).

The syntactic annotation is regular with lemma *faire 'to do' for N1 as a causative auxiliary of N2 and for the two other lemmas (*entendre 'to hear' et laisser 'to let'), an xcomp relation from N1 to N2.

By searching the corresponding lemmas, we found a few annotation errors or annotation inconsistencies.

## 6 Conclusion

We have shown in this paper that using a graph encoding to represent a multi-layer annotation in a common structure is useful and can be exploited for different purposes, like error mining or linguistic exploration of the data. This methodology opens new perspectives for corpora maintenance and is complementary to existing tools like the UD validation script<fnref n="13" /> and the Parseme consistency checking. Using the same idea, it would be possible to encode other annotation layers, like the ones available in a corpus like the GUM corpus<fnref n="14" /> (Zeldes, 2017).

## Acknowledgments

We would like to thank the reviewers for their helpful comments. Thanks also to Kim, Joakim and Khensa for their help on German, Swedish and Arabic respectively.

## References

Marie-Catherine de Marneffe, Christopher D. Manning, Joakim Nivre, and Daniel Zeman. 2021. Universal Dependencies. *Computational Linguistics*, 47(2):255–308.

Bruno Guillaume. 2021. Graph Matching and Graph Rewriting: GREW tools for corpus exploration, maintenance and conversion. In *EACL 2021 - 16th conference of the European Chapter of the Association for Computational Linguistics*, Kiev/Online, Ukraine.

Johanna Monti, Savary Agata, Marie Candito, Veriginica Barbu Mititelu, Bejček Eduard, Cap Fabienne, Čéplö Slavomir, Silvio Ricardo Cordeiro, Eryiğit Gülşen, Voula Giouli, Maarten van Gompel, HaCohen-Kerner Yaakov, Kovalevskaitė Jolanta, Krek Simon, Liebeskind Chaya, Carla Parra Escartín, Lonneke van der Plas, Qasemizadeh Behrang, Ramisch Carlos, Federico Sangati, Stoyanova Ivelina, and Vincze Veronika. 2018. Parseme multilingual corpus of verbal multiword expressions.

Carlos Ramisch, Agata Savary, Bruno Guillaume, Jakub Waszczuk, Marie Candito, Ashwini Vaidya, Veriginica Barbu Mititelu, Archna Bhatia, Uxoa Iñurrieta, Voula Giouli, Tunga Güngör, Menghan Jiang, Timm Lichte, Chaya Liebeskind, Johanna Monti, Renata Ramisch, Sara Stymne, Abigail Walsh, and Hongzhi Xu. 2020. Edition 1.2 of the PARSEME shared task on semi-supervised identification of verbal multiword expressions. In *Proceedings of the Joint Workshop on Multiword Expressions and Electronic Lexicons*, pages 107–118, online. Association for Computational Linguistics.

Milan Straka, Jan Hajič, and Jana Straková. 2016. UD-Pipe: Trainable pipeline for processing CoNLL-U files performing tokenization, morphological analysis, POS tagging and parsing. In *Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16)*, pages 4290–4297, Portorož, Slovenia. European Language Resources Association (ELRA).

Amir Zeldes. 2017. The GUM corpus: Creating multilayer resources in the classroom. *Language Resources and Evaluation*, 51(3):581–612.

## A Example Appendix

<footnote n="13">¹³https://universaldependencies.org/ validation-rules.html</footnote>

<footnote n="14">¹⁴https://gucorpling.org/gum/</footnote>
<table><caption>Table 1: Numbers of occurrences of VMWEs with their labels.</caption><thead><tr><th>Language</th><th>IAV</th><th>IRV</th><th>LS.ICV</th><th>LVC.cause</th><th>LVC.full</th><th>MVC</th><th>VID</th><th>VPC.full</th><th>VPC.semi</th></tr></thead><tbody><tr><td>Arabic</td><td>581</td><td>0</td><td>0</td><td>303</td><td>2678</td><td>5</td><td>1182</td><td>0</td><td>0</td></tr><tr><td>Basque</td><td>0</td><td>0</td><td>0</td><td>214</td><td>3152</td><td>0</td><td>880</td><td>0</td><td>0</td></tr><tr><td>Bulgarian</td><td>90</td><td>3223</td><td>0</td><td>222</td><td>1909</td><td>0</td><td>1260</td><td>0</td><td>0</td></tr><tr><td>Croatian</td><td>1388</td><td>1193</td><td>0</td><td>147</td><td>880</td><td>0</td><td>293</td><td>1</td><td>0</td></tr><tr><td>Chinese</td><td>0</td><td>0</td><td>0</td><td>177</td><td>1214</td><td>3826</td><td>973</td><td>0</td><td>4629</td></tr><tr><td>Czech</td><td>0</td><td>10000</td><td>0</td><td>0</td><td>2923</td><td>0</td><td>1613</td><td>0</td><td>0</td></tr><tr><td>English</td><td>71</td><td>0</td><td>0</td><td>51</td><td>333</td><td>51</td><td>187</td><td>368</td><td>53</td></tr><tr><td>Farsi</td><td>0</td><td>1</td><td>0</td><td>0</td><td>3435</td><td>0</td><td>17</td><td>0</td><td>0</td></tr><tr><td>French</td><td>0</td><td>1501</td><td>0</td><td>97</td><td>1878</td><td>22</td><td>2157</td><td>0</td><td>0</td></tr><tr><td>German</td><td>0</td><td>322</td><td>0</td><td>33</td><td>311</td><td>0</td><td>1437</td><td>1744</td><td>194</td></tr><tr><td>Greek</td><td>0</td><td>1</td><td>0</td><td>179</td><td>5293</td><td>51</td><td>2841</td><td>143</td><td>0</td></tr><tr><td>Hebrew</td><td>0</td><td>0</td><td>0</td><td>223</td><td>1049</td><td>0</td><td>1108</td><td>153</td><td>0</td></tr><tr><td>Hindi</td><td>0</td><td>0</td><td>0</td><td>26</td><td>641</td><td>306</td><td>61</td><td>0</td><td>0</td></tr><tr><td>Hungarian</td><td>0</td><td>0</td><td>0</td><td>401</td><td>1143</td><td>0</td><td>104</td><td>5156</td><td>956</td></tr><tr><td>Irish</td><td>187</td><td>0</td><td>0</td><td>118</td><td>200</td><td>0</td><td>106</td><td>28</td><td>20</td></tr><tr><td>Italian</td><td>497</td><td>1144</td><td>37</td><td>174</td><td>734</td><td>33</td><td>1484</td><td>105</td><td>2</td></tr><tr><td>Lithuanian</td><td>0</td><td>0</td><td>0</td><td>25</td><td>479</td><td>0</td><td>308</td><td>0</td><td>0</td></tr><tr><td>Maltese</td><td>0</td><td>1</td><td>0</td><td>1</td><td>700</td><td>2</td><td>518</td><td>4</td><td>0</td></tr><tr><td>Polish</td><td>0</td><td>3688</td><td>0</td><td>314</td><td>2478</td><td>0</td><td>833</td><td>0</td><td>0</td></tr><tr><td>Portuguese</td><td>0</td><td>1021</td><td>0</td><td>127</td><td>3954</td><td>18</td><td>1306</td><td>0</td><td>0</td></tr><tr><td>Romanian</td><td>3340</td><td>3826</td><td>0</td><td>182</td><td>516</td><td>0</td><td>1644</td><td>0</td><td>0</td></tr><tr><td>Serbian</td><td>0</td><td>564</td><td>0</td><td>69</td><td>402</td><td>0</td><td>269</td><td>0</td><td>0</td></tr><tr><td>Slovenian</td><td>710</td><td>1626</td><td>0</td><td>64</td><td>239</td><td>0</td><td>724</td><td>0</td><td>0</td></tr><tr><td>Spanish</td><td>511</td><td>714</td><td>0</td><td>81</td><td>392</td><td>713</td><td>327</td><td>1</td><td>0</td></tr><tr><td>Swedish</td><td>0</td><td>237</td><td>0</td><td>10</td><td>417</td><td>0</td><td>441</td><td>1461</td><td>589</td></tr><tr><td>Turkish</td><td>0</td><td>0</td><td>0</td><td>0</td><td>3583</td><td>5</td><td>4141</td><td>0</td><td>0</td></tr></tbody></table>

<table><caption>Table 2: Numbers of tokens of VMWEs.</caption><thead><tr><th>Language</th><th>1</th><th>2</th><th>3</th><th>4</th><th>5</th><th>6</th><th>7</th><th>8</th><th>9</th><th>10</th><th>11</th><th>12</th><th>13</th></tr></thead><tbody><tr><td>Arabic</td><td>17</td><td>3673</td><td>946</td><td>91</td><td>11</td><td>10</td><td>0</td><td>1</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td></tr><tr><td>Basque</td><td>0</td><td>4164</td><td>70</td><td>12</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td></tr><tr><td>Bulgarian</td><td>11</td><td>5974</td><td>604</td><td>102</td><td>13</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td></tr><tr><td>Croatian</td><td>0</td><td>3182</td><td>640</td><td>75</td><td>3</td><td>2</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td></tr><tr><td>Chinese</td><td>5382</td><td>5224</td><td>136</td><td>35</td><td>15</td><td>14</td><td>6</td><td>5</td><td>1</td><td>0</td><td>1</td><td>0</td><td>0</td></tr><tr><td>Czech</td><td>0</td><td>11178</td><td>2571</td><td>664</td><td>97</td><td>18</td><td>8</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td></tr><tr><td>English</td><td>4</td><td>1001</td><td>73</td><td>25</td><td>7</td><td>3</td><td>0</td><td>1</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td></tr><tr><td>Farsi</td><td>1</td><td>3004</td><td>404</td><td>38</td><td>4</td><td>2</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td></tr><tr><td>French</td><td>5</td><td>4353</td><td>1048</td><td>180</td><td>34</td><td>28</td><td>6</td><td>1</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td></tr><tr><td>German</td><td>1268</td><td>1976</td><td>644</td><td>129</td><td>15</td><td>7</td><td>1</td><td>0</td><td>1</td><td>0</td><td>0</td><td>0</td><td>0</td></tr><tr><td>Greek</td><td>1</td><td>6253</td><td>1511</td><td>523</td><td>166</td><td>31</td><td>9</td><td>7</td><td>5</td><td>1</td><td>1</td><td>0</td><td>0</td></tr><tr><td>Hebrew</td><td>42</td><td>1781</td><td>584</td><td>87</td><td>21</td><td>5</td><td>8</td><td>2</td><td>2</td><td>0</td><td>0</td><td>0</td><td>1</td></tr><tr><td>Hindi</td><td>0</td><td>961</td><td>15</td><td>46</td><td>9</td><td>1</td><td>1</td><td>0</td><td>1</td><td>0</td><td>0</td><td>0</td><td>0</td></tr><tr><td>Hungarian</td><td>5745</td><td>2010</td><td>5</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td></tr><tr><td>Irish</td><td>3</td><td>477</td><td>152</td><td>21</td><td>5</td><td>1</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td></tr><tr><td>Italian</td><td>9</td><td>2693</td><td>1118</td><td>288</td><td>64</td><td>27</td><td>11</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td></tr><tr><td>Lithuanian</td><td>0</td><td>683</td><td>99</td><td>21</td><td>7</td><td>1</td><td>0</td><td>1</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td></tr><tr><td>Maltese</td><td>13</td><td>680</td><td>391</td><td>100</td><td>32</td><td>3</td><td>4</td><td>1</td><td>1</td><td>0</td><td>1</td><td>0</td><td>0</td></tr><tr><td>Polish</td><td>0</td><td>6550</td><td>653</td><td>88</td><td>13</td><td>6</td><td>0</td><td>2</td><td>0</td><td>0</td><td>0</td><td>1</td><td>0</td></tr><tr><td>Portuguese</td><td>1</td><td>5449</td><td>650</td><td>263</td><td>32</td><td>20</td><td>6</td><td>4</td><td>0</td><td>1</td><td>0</td><td>0</td><td>0</td></tr><tr><td>Romanian</td><td>0</td><td>8009</td><td>1368</td><td>74</td><td>45</td><td>12</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td></tr><tr><td>Serbian</td><td>0</td><td>1151</td><td>128</td><td>17</td><td>4</td><td>3</td><td>1</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td></tr><tr><td>Slovenian</td><td>0</td><td>2732</td><td>531</td><td>72</td><td>21</td><td>4</td><td>2</td><td>1</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td></tr><tr><td>Spanish</td><td>2</td><td>2089</td><td>569</td><td>69</td><td>10</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td></tr><tr><td>Swedish</td><td>1614</td><td>1336</td><td>188</td><td>14</td><td>3</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td></tr><tr><td>Turkish</td><td>6</td><td>7233</td><td>445</td><td>41</td><td>4</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td></tr></tbody></table>
<table><caption>Table 3: Ratio of VMWEs which overlap with another annotation.</caption><thead><tr><th>Language</th><th>Yes</th><th>No</th></tr></thead><tbody><tr><td>Bulgarian</td><td>0.0%</td><td>100.0%</td></tr><tr><td>Maltese</td><td>0.16%</td><td>99.84%</td></tr><tr><td>Turkish</td><td>0.57%</td><td>99.43%</td></tr><tr><td>Farsi</td><td>0.58%</td><td>99.42%</td></tr><tr><td>Lithuanian</td><td>0.74%</td><td>99.26%</td></tr><tr><td>Serbian</td><td>1.38%</td><td>98.62%</td></tr><tr><td>Slovenian</td><td>1.4%</td><td>98.6%</td></tr><tr><td>Basque</td><td>1.77%</td><td>98.23%</td></tr><tr><td>Swedish</td><td>2.31%</td><td>97.69%</td></tr><tr><td>Hebrew</td><td>2.88%</td><td>97.12%</td></tr><tr><td>Polish</td><td>2.95%</td><td>97.05%</td></tr><tr><td>French</td><td>3.04%</td><td>96.96%</td></tr><tr><td>Chinese</td><td>3.38%</td><td>96.62%</td></tr><tr><td>Czech</td><td>3.78%</td><td>96.22%</td></tr><tr><td>Portuguese</td><td>4.17%</td><td>95.83%</td></tr><tr><td>Arabic</td><td>4.27%</td><td>95.73%</td></tr><tr><td>English</td><td>4.67%</td><td>95.33%</td></tr><tr><td>Greek</td><td>4.82%</td><td>95.18%</td></tr><tr><td>Irish</td><td>4.86%</td><td>95.14%</td></tr><tr><td>Hungarian</td><td>5.54%</td><td>94.46%</td></tr><tr><td>German</td><td>6.9%</td><td>93.1%</td></tr><tr><td>Italian</td><td>12.19%</td><td>87.81%</td></tr><tr><td>Hindi</td><td>12.86%</td><td>87.14%</td></tr><tr><td>Romanian</td><td>18.46%</td><td>81.54%</td></tr><tr><td>Spanish</td><td>22.82%</td><td>77.18%</td></tr><tr><td>Croatian</td><td>28.86%</td><td>71.14%</td></tr></tbody></table>

<table><caption>Table 4: Numbers of occurrences of VMWEs without any verbal token (column no_verb) and of VMWEs tagged IRV without any reflexive pronoun (column irv_no_reflex).</caption><thead><tr><th>Language</th><th># sentences</th><th>no_verb</th><th>irv_no_reflex</th></tr></thead><tbody><tr><td>Arabic</td><td>7483</td><td>1302</td><td>0</td></tr><tr><td>Basque</td><td>11158</td><td>4</td><td>0</td></tr><tr><td>Bulgarian</td><td>21599</td><td>416</td><td>2</td></tr><tr><td>Croatian</td><td>6133</td><td>146</td><td>2</td></tr><tr><td>Chinese</td><td>48929</td><td>526</td><td>0</td></tr><tr><td>Czech</td><td>49431</td><td>790</td><td>0</td></tr><tr><td>English</td><td>7436</td><td>11</td><td>0</td></tr><tr><td>Farsi</td><td>3617</td><td>1</td><td>1</td></tr><tr><td>French</td><td>20961</td><td>2</td><td>107</td></tr><tr><td>German</td><td>8996</td><td>126</td><td>3</td></tr><tr><td>Greek</td><td>26175</td><td>26</td><td>1</td></tr><tr><td>Hebrew</td><td>19200</td><td>264</td><td>0</td></tr><tr><td>Hindi</td><td>1684</td><td>0</td><td>0</td></tr><tr><td>Hungarian</td><td>6159</td><td>5901</td><td>0</td></tr><tr><td>Irish</td><td>1705</td><td>214</td><td>0</td></tr><tr><td>Italian</td><td>15728</td><td>65</td><td>1144</td></tr><tr><td>Lithuanian</td><td>11104</td><td>12</td><td>0</td></tr><tr><td>Maltese</td><td>10600</td><td>59</td><td>1</td></tr><tr><td>Polish</td><td>23547</td><td>836</td><td>0</td></tr><tr><td>Portuguese</td><td>32062</td><td>26</td><td>1021</td></tr><tr><td>Romanian</td><td>56664</td><td>5</td><td>206</td></tr><tr><td>Serbian</td><td>3586</td><td>91</td><td>0</td></tr><tr><td>Slovenian</td><td>27825</td><td>0</td><td>5</td></tr><tr><td>Spanish</td><td>5515</td><td>23</td><td>8</td></tr><tr><td>Swedish</td><td>6026</td><td>92</td><td>237</td></tr><tr><td>Turkish</td><td>22306</td><td>330</td><td>0</td></tr></tbody></table>