<figure><img src="image_1.png" /></figure>

HAL
open science

# Multi-layer querying in Corpora: Example of Parseme and UD

Bruno Guillaume

<figure><img src="image_2.png" /></figure>

## To cite this version:

Bruno Guillaume. Multi-layer querying in Corpora: Example of Parseme and UD. UniDive 1st general meeting, Mar 2023, Saclay, France. . hal-04387830

HAL Id: hal-04387830

https://inria.hal.science/hal-04387830v1

Submitted on 11 Jan 2024

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_3.png" /></figure>

Distributed under a Creative Commons Attribution - ShareAlike 4.0 International License
# Multi-layer querying in Corpora: Example of Parseme and UD

Bruno Guillaume

Université de Lorraine, CNRS, Inria, LORIA, F-54000 Nancy, France

Bruno.Guillaume@inria.fr

Relevant UniDive working groups: WG1

## 1 Introduction

The Parseme project (Monti et al., 2018) proposes a large set of annotated data with Verbal Multi-Word Expressions (VMWE). In version 1.2 (Ramisch et al., 2020), 14 languages were covered but with older versions and ongoing work<fnref n="1" />, there are data in 26 languages (See Table 3 in appendix for list of languages and the number of sentences for each language).

Most of these data are also released with morpho-syntactic annotations, following the Universal Dependencies (de Marneffe et al., 2021) framework. Some Parseme data are directly annotated on data available in the UD project and then we have both high quality morpho-syntactic annotation and VMWE annotations on the same data. Some other parts of Parseme data, which are not on existing UD data, comes with an automatic morpho-syntactic annotation, built with UDPipe (Straka et al., 2016), thus following the UD annotation framework.

In this abstract, we present an encoding of the two annotation layers in a common structure, using graph encoding of both UD and VMWE annotations. With this encoding, it is possible to use the graph-based tools to work with the data. In this abstract, we use the GREW tool (Guillaume, 2021) to make queries on the two layers.

In Section 2, we explain the encoding and in Section 3, we show how queries can be run and give a few examples.

## 2 Graph encoding

The two annotation layers are technically stored in a common format (CUPT<fnref n="2" />), but it is not straightforward to consider both in the same structure. In UD, each sentence is split in a sequence of tokens and a annotated VMWE contains several tokens, but not necessarily contiguous (for instance in *Take a*

<figure><img src="image_4.png" /><figcaption>Figure 1: Graph representation (simplified) of two overlapping VMWE annotations</figcaption></figure>

look !!!). Moreover, the same token can be implied in more than one VMWE annotation items. In Parseme-EN for instance, the same sentence have the two following overlapping annotations: [...] to get rid of the moral burden [...] with label IAV (Inherently adpositional verbs) and [...] to get rid of the moral burden [...] with label MVC (Multi-verb constructions).

In order to cover these cases, we propose to represent each sentence as a graph with two kinds of nodes and two kinds of edges. In addition to morphosyntactic nodes and edges (drawn in black in Figure 1), for each VMWE annotation a new node is added, labelled with the kind of VMWE and the new node is to each tokens which is part of the VMWE (with a specific type of edges). All VMWE related structure (nodes and edges) is drawn in blue in Figure 1.

## 3 Multi-layer queries

The benefit of having the two annotation layers in the same structure is that it is possible to make queries which refer to both layers and to make cross observation. To do this, we use the GREW tools which allows to write graph queries that can be executed on the Parseme corpora represented has above.

For querying treebanks, the GREW tool can be used online with GREW-MATCH<fnref n="3" /> (on a predefined set of treebanks) on through the GREWPy Python library. With a simple request, and using the clustering feature, we can observe the number of VMWEs

<footnote n="1">¹https://gitlab.com/parseme/corpora</footnote>

<footnote n="2">²http://multiword.sourceforge.net/cupt-format</footnote>

<footnote n="3">³http://parseme.grew.fr</footnote>
by type in each language (Table 1 in appendix).

Graph requests can be used to distinguish VMWE annotation which are overlapping with another one with the ones without overlapping. The request below correspond to the “without overlapping” case: lines 1-3 is a request for any VWME (without the tag NotMWE) and lines 4-7 filter out cases where some MWE2 exists, which shares a token X with the one previously found MWE1.

<figure><img src="image_5.png" /></figure>

For the “with overlapping” case, the request is the same where the keyword *without* is replaced by the keyword *without* (line 4). Table 1 shows the ratio of overlapping VMWE for each language.

## 3.1 Error mining

Queries can be used to do error mining in the treebanks. We give here two simple queries which can be used on all languages. Of course, as seen in the next subsection, exploration can be done more in depth for given language, through specific queries.

First, we query the different treebanks to search for VMWE containing only one token. Intuitively this should not happen, by definition of multi-word expression. With the request below, we request for such a VMWE:

<figure><img src="image_6.png" /></figure>

Secondly, we may wonder if we can have a VMWE which contains neither a VERB nor an AUX in the corresponding tokens. This can be explored with the request:

<figure><img src="image_7.png" /></figure>

Table 3 in appendix gives the numbers of occurrences in each language for these two requests. For the first one, there are 4 languages with an high number of occurrences: Hungarian (5735), Chinese (4420), Swedish (1627) and German (1270). All remaining languages have no or few (above 20) occurrences of the request. This shows that the

notion of tokenisation is considered differently in both projects. For the second request, the median of the number of occurrences in the 26 treebanks is 92, with 3 treebanks above 1000 occurrences. This shows that the definition what is a “verb” in Parseme is not fully aligned with the UD policy.

## 3.2 Example: VPC.FULL

In this section, we give a few examples of requests which can be used in GREW-MATCH to explore how some specific class of VWME is annotated in one treebank. The example runs on VPC.FULL and on English data.

First, we can have a look at the distribution of this kind of VMWE according to the number of tokens implied<fnref n="4" />. We observed 368 occurrences<fnref n="5" /> of this label, all of them having exactly two tokens.

Another request<fnref n="6" />, specifying the two tokens N1 and N2, can display the distribution of the upos of the tokens in the following table which shows that two constructions VERB-ADP and VERB-ADV covers all but 5 cases.

<figure><img src="image_8.png" /></figure>

Exploring further<fnref n="7" />, we observed in the 199 VERB-ADP cases, a large majority (185) of annotation where the VERB is linked to the ADP with relation *compound:prt*. Other cases are: no direct relation between the two nodes (9 cases), relation *advmod* (3 cases), *compound* (2 cases). Similarly<fnref n="8" />, we observed in the 164 VERB-ADV cases, a majority (93) of annotation where the VERB is linked to the ADP with relation *compound:prt*. Other cases are: relation *advmod* (66 cases), no direct relation between the two nodes (2 cases), *compound*, *obl* and *xcomp* (1 case for each).

These irregularities in the annotation would require a careful inspection by a native English speaker but we can already see a bunch of annotation inconsistencies either in the UD annotation layer or in the Parseme one.

<footnote n="4">⁴http://parseme.grew.fr/?custom=63cfdecd4ed4e</footnote>

<footnote n="5">⁵The numbers of this section are based on requests done on 2023/02/02, they may changed when the data is updated. Requests on a stable data from a release will be provided for final version.</footnote>

<footnote n="6">⁶http://parseme.grew.fr/?custom=63cfe3d7095fe</footnote>

<footnote n="7">⁷http://parseme.grew.fr/?custom=63cfe5673f17f</footnote>

<footnote n="8">⁸http://parseme.grew.fr/?custom=63cfe782a12cb</footnote>
## References

Marie-Catherine de Marneffe, Christopher D. Manning, Joakim Nivre, and Daniel Zeman. 2021. *Universal Dependencies*. *Computational Linguistics*, 47(2):255–308.

Bruno Guillaume. 2021. *Graph Matching and Graph Rewriting: GREW tools for corpus exploration, maintenance and conversion*. In *EACL 2021 - 16th conference of the European Chapter of the Association for Computational Linguistics*, Kiev/Online, Ukraine.

Johanna Monti, Savary Agata, Marie Candito, Verginica Barbu Mititelu, Bejček Eduard, Cap Fabienne, Čeplö Slavomir, Silvio Ricardo Cordeiro, Eryiğit Gülşen, Voula Giouli, Maarten van Gompel, HaCohen-Kerner Yaakov, Kovalevskaitė Jolanta, Krek Simon, Liebeskind Chaya, Carla Parra Escartín, Lonneke van der Plas, Qasemizadeh Behrang, Ramisch Carlos, Federico Sangati, Stoyanova Ivelina, and Vincze Veronika. 2018. Parseme multilingual corpus of verbal multiword expressions.

Carlos Ramisch, Agata Savary, Bruno Guillaume, Jakub Waszczuk, Marie Candito, Ashwini Vaidya, Verginica Barbu Mititelu, Archna Bhatia, Uxoa Iñurrieta, Voula Giouli, Tunga Güngör, Menghan Jiang, Timm Lichte, Chaya Liebeskind, Johanna Monti, Renata Ramisch, Sara Stymne, Abigail Walsh, and Hongzhi Xu. 2020. Edition 1.2 of the PARSEME shared task on semi-supervised identification of verbal multiword expressions. In *Proceedings of the Joint Workshop on Multiword Expressions and Electronic Lexicons*, pages 107–118, online. Association for Computational Linguistics.

Milan Straka, Jan Hajič, and Jana Straková. 2016. *UD-Pipe: Trainable pipeline for processing CoNLL-U files performing tokenization, morphological analysis, POS tagging and parsing*. In *Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16)*, pages 4290–4297, Portorož, Slovenia. European Language Resources Association (ELRA).

## A Example Appendix

Note: At the time of the final version, the last version (1.3) was not released. The tables are built from the data freezed by the Parseme core team on 2023/02/27. The data of the official release 1.3 may be slightly different.
<table><caption>Table 1: Numbers of occurrences of VMES with theirs labels.</caption><thead><tr><th>Language</th><th>MVC</th><th>LS.ICV</th><th>LVC.cause</th><th>IRV</th><th>VID</th><th>VPC.full</th><th>VPC.semi</th><th>IAV</th><th>LVC.full</th></tr></thead><tbody><tr><td>Arabic</td><td>303</td><td>0</td><td>5</td><td>2678</td><td>0</td><td>0</td><td>581</td><td>0</td><td>1182</td></tr><tr><td>Bulgarian</td><td>0</td><td>0</td><td>222</td><td>3223</td><td>1260</td><td>0</td><td>0</td><td>90</td><td>1909</td></tr><tr><td>Czech</td><td>0</td><td>0</td><td>0</td><td>10000</td><td>1613</td><td>0</td><td>0</td><td>0</td><td>2923</td></tr><tr><td>German</td><td>0</td><td>0</td><td>33</td><td>322</td><td>1437</td><td>1744</td><td>194</td><td>0</td><td>311</td></tr><tr><td>Greek</td><td>51</td><td>0</td><td>179</td><td>1</td><td>2841</td><td>143</td><td>0</td><td>0</td><td>5293</td></tr><tr><td>English</td><td>51</td><td>0</td><td>51</td><td>0</td><td>187</td><td>368</td><td>53</td><td>71</td><td>333</td></tr><tr><td>Spanish</td><td>713</td><td>0</td><td>81</td><td>714</td><td>327</td><td>1</td><td>0</td><td>511</td><td>392</td></tr><tr><td>Basque</td><td>0</td><td>0</td><td>214</td><td>0</td><td>880</td><td>0</td><td>0</td><td>0</td><td>3152</td></tr><tr><td>Farsi</td><td>0</td><td>0</td><td>0</td><td>1</td><td>17</td><td>0</td><td>0</td><td>0</td><td>3435</td></tr><tr><td>French</td><td>22</td><td>0</td><td>97</td><td>1501</td><td>2157</td><td>0</td><td>0</td><td>0</td><td>1878</td></tr><tr><td>Irish</td><td>0</td><td>0</td><td>118</td><td>0</td><td>106</td><td>28</td><td>20</td><td>187</td><td>200</td></tr><tr><td>Hebrew</td><td>0</td><td>0</td><td>223</td><td>0</td><td>1108</td><td>153</td><td>0</td><td>0</td><td>1049</td></tr><tr><td>Hindi</td><td>306</td><td>0</td><td>26</td><td>0</td><td>61</td><td>0</td><td>0</td><td>0</td><td>641</td></tr><tr><td>Croatian</td><td>0</td><td>0</td><td>147</td><td>1193</td><td>293</td><td>1</td><td>0</td><td>1388</td><td>880</td></tr><tr><td>Hungarian</td><td>0</td><td>0</td><td>401</td><td>0</td><td>104</td><td>5156</td><td>956</td><td>0</td><td>1143</td></tr><tr><td>Italian</td><td>33</td><td>37</td><td>174</td><td>1144</td><td>1484</td><td>105</td><td>2</td><td>497</td><td>734</td></tr><tr><td>Lithuanian</td><td>0</td><td>0</td><td>25</td><td>0</td><td>308</td><td>0</td><td>0</td><td>0</td><td>479</td></tr><tr><td>Maltese</td><td>2</td><td>0</td><td>1</td><td>1</td><td>518</td><td>4</td><td>0</td><td>0</td><td>700</td></tr><tr><td>Polish</td><td>0</td><td>0</td><td>314</td><td>3688</td><td>833</td><td>0</td><td>0</td><td>0</td><td>2478</td></tr><tr><td>Portuguese</td><td>18</td><td>0</td><td>127</td><td>1021</td><td>1306</td><td>0</td><td>0</td><td>0</td><td>3954</td></tr><tr><td>Romanian</td><td>0</td><td>0</td><td>182</td><td>3826</td><td>1644</td><td>0</td><td>0</td><td>3340</td><td>516</td></tr><tr><td>Slovenian</td><td>0</td><td>0</td><td>64</td><td>1626</td><td>724</td><td>0</td><td>0</td><td>710</td><td>239</td></tr><tr><td>Serbian</td><td>0</td><td>0</td><td>69</td><td>564</td><td>269</td><td>0</td><td>0</td><td>0</td><td>402</td></tr><tr><td>Turkish</td><td>5</td><td>0</td><td>0</td><td>0</td><td>4141</td><td>0</td><td>0</td><td>0</td><td>3583</td></tr><tr><td>Chinese</td><td>3826</td><td>0</td><td>177</td><td>0</td><td>973</td><td>0</td><td>4629</td><td>0</td><td>1214</td></tr></tbody></table>

<table><caption>Table 2: Ratio of VMES which overlap with another annotation (languages sorted by values of the ratio).</caption><thead><tr><th>Language</th><th>Yes</th><th>No</th><th># sentences</th><th>one_token</th><th>no_verb</th></tr></thead><tbody><tr><td>Arabic</td><td>7483</td><td>17</td><td>7483</td><td>17</td><td>1302</td></tr><tr><td>Bulgarian</td><td>21599</td><td>11</td><td>21599</td><td>11</td><td>416</td></tr><tr><td>Czech</td><td>49431</td><td>0</td><td>49431</td><td>0</td><td>790</td></tr><tr><td>German</td><td>8996</td><td>1268</td><td>8996</td><td>1268</td><td>126</td></tr><tr><td>Greek</td><td>26175</td><td>1</td><td>26175</td><td>1</td><td>26</td></tr><tr><td>English</td><td>7436</td><td>4</td><td>7436</td><td>4</td><td>11</td></tr><tr><td>Spanish</td><td>5515</td><td>2</td><td>5515</td><td>2</td><td>23</td></tr><tr><td>Basque</td><td>11158</td><td>0</td><td>11158</td><td>0</td><td>4</td></tr><tr><td>Farsi</td><td>3617</td><td>1</td><td>3617</td><td>1</td><td>1</td></tr><tr><td>French</td><td>20961</td><td>5</td><td>20961</td><td>5</td><td>2</td></tr><tr><td>Irish</td><td>1705</td><td>3</td><td>1705</td><td>3</td><td>214</td></tr><tr><td>Hebrew</td><td>19200</td><td>42</td><td>19200</td><td>42</td><td>264</td></tr><tr><td>Hindi</td><td>1684</td><td>0</td><td>1684</td><td>0</td><td>0</td></tr><tr><td>Croatian</td><td>6133</td><td>0</td><td>6133</td><td>0</td><td>146</td></tr><tr><td>Hungarian</td><td>6159</td><td>5745</td><td>6159</td><td>5745</td><td>5901</td></tr><tr><td>Italian</td><td>15728</td><td>9</td><td>15728</td><td>9</td><td>65</td></tr><tr><td>Lithuanian</td><td>11104</td><td>0</td><td>11104</td><td>0</td><td>12</td></tr><tr><td>Maltese</td><td>10600</td><td>13</td><td>10600</td><td>13</td><td>59</td></tr><tr><td>Polish</td><td>23547</td><td>0</td><td>23547</td><td>0</td><td>836</td></tr><tr><td>Portuguese</td><td>32062</td><td>1</td><td>32062</td><td>1</td><td>26</td></tr><tr><td>Romanian</td><td>56664</td><td>0</td><td>56664</td><td>0</td><td>5</td></tr><tr><td>Slovenian</td><td>27825</td><td>0</td><td>27825</td><td>0</td><td>0</td></tr><tr><td>Serbian</td><td>3586</td><td>0</td><td>3586</td><td>0</td><td>91</td></tr><tr><td>Swedish</td><td>6026</td><td>1614</td><td>6026</td><td>1614</td><td>92</td></tr><tr><td>Turkish</td><td>22306</td><td>6</td><td>22306</td><td>6</td><td>330</td></tr><tr><td>Chinese</td><td>48929</td><td>5382</td><td>48929</td><td>5382</td><td>526</td></tr></tbody></table>

<caption>Table 3: Numbers of occurrences of VMES with one token (column one_token), of VMES without any verbal token (column no_verb).</caption>