# Multiword expressions at length and in depth

Stella Markanotonatou, Carlos Ramisch, Agata Savary, Veronika Vincze

► To cite this version:

Stella Markanotonatou, Carlos Ramisch, Agata Savary, Veronika Vincze (Dir.). Multiword expressions at length and in depth: Extended papers from the MWE 2017 workshop. Language Science Press, 2018. hal-01917075

HAL Id: hal-01917075

https://hal.science/hal-01917075

Submitted on 4 Dec 2018

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
Multiword
expressions at
length and in
depth

Extended papers from the MWE 2017
workshop

Edited by

Stella Markantonatou

Carlos Ramisch

Agata Savary

Veronika Vincze

Phraseology and Multiword Expressions 2

<figure><img src="image_2.png" /></figure>

language
science
press
# Phraseology and Multiword Expressions

## Series editors

Agata Savary (University of Tours, Blois, France), Manfred Sailer (Goethe University Frankfurt a. M., Germany), Yannick Parmentier (University of Orléans, France), Victoria Rosén (University of Bergen, Norway), Mike Rosner (University of Malta, Malta).

In this series:

1. Sailer, Manfred & Stella Markantonatou (eds.). Multiword expressions: Insights from a multilingual perspective.

2. Markantonatou, Stella, Carlos Ramisch, Agata Savary & Veronika Vincze (eds.). Multiword expressions at length and in depth: Extended papers from the MWE 2017 workshop.

ISSN: 2625-3127
Multiword
expressions at
length and in
depth

Extended papers from the MWE 2017
workshop

Edited by

Stella Markantonatou

Carlos Ramisch

Agata Savary

Veronika Vincze

<figure><img src="image_3.png" /></figure>

language
science
press
Markantonatou, Stella, Carlos Ramisch, Agata Savary & Veronika Vincze (eds.).
2018. *Multiword expressions at length and in depth: Extended papers from the MWE 2017 workshop* (Phraseology and Multiword Expressions 2). Berlin:
Language Science Press.

This title can be downloaded at:

http://langsci-press.org/catalog/book/204

© 2018, the authors

Published under the Creative Commons Attribution 4.0 Licence (CC BY 4.0):

http://creativecommons.org/licenses/by/4.0/

ISBN: 978-3-96110-123-8 (Digital)

978-3-96110-124-5 (Hardcover)

ISSN: 2625-3127

DOI:10.5281/zenodo.1469527

Source code available from www.github.com/langsci/204

Collaborative reading: paperhive.org/documents/remote?type=langsci&id=204

Cover and concept of design: Ulrike Harbort

Typesetting: Sebastian Nordhoff, Agata Savary

Proofreading: Amr Zawawy, Beatriz Sánchez-Cárdenas, Bojana Đorđević, Brett Reynolds, Conor Pyle, Eitan Grossman, Eleni Koutsomitopoulou, Felix Hoberg, Felix Kopecky, Glorianna Jagfeld, Harald Ulland, Ikmi Nur Oktavianti, Ivelina Stoyanova, Ivica Jeđud, Jeroen van de Weijer, Melanie Röthlisberger, Jean Nitzke, Steven Moran, Trinka D'Cunha, Vadim Kimmelman, Valeria Quochi

Reynolds, Conor Pyle, Eitan Grossman, Eleni Koutsomitopoulou, Felix Hoberg,

Felix Kopecky, Glorianna Jagfeld, Harald Ulland, Ikmi Nur Oktavianti, Ivelina

Stoyanova, Ivica Jeđud, Jeroen van de Weijer, Melanie Röthlisberger, Jean

Nitzke, Steven Moran, Trinka D'Cunha, Vadim Kimmelman, Valeria Quochi

Fonts: Linux Libertine, Libertinus Math, Arimo, DejaVu Sans Mono

Typesetting software: XeLATEX

Language Science Press
Unter den Linden 6
10099 Berlin, Germany
langsci-press.org

Storage and cataloguing done by FU Berlin
# Contents

## Preface

Stella Markantonatou, Carlos Ramisch, Agata Savary & Veronika Vincze iii

## 1 "Spilling the bag" on idiomatic variation

Kristina Geeraert, R. Harald Baayen & John Newman 1

## 2 Paraphrases of verbal multiword expressions: the case of Czech light verbs and idioms

Petra Barančíková & Václava Kettnerová 35

## 3 Identifying senses of particles in verb-particle constructions

Archna Bhatia, Choh Man Teng & James F. Allen 61

## 4 PARSEME multilingual corpus of verbal multiword expressions

Agata Savary, Marie Candito, Verginica Barbu Mititelu, Eduard Bejček, Fabienne Cap, Slavomír Čeplö, Silvio Ricardo Cordeiro, Gülşen Eryiğit, Voula Giouli, Maarten van Gompel, Yaakov HaCohen-Kerner, Jolanta Kovalevskaitė, Simon Krek, Chaya Liebeskind, Johanna Monti, Carla Parra Escartín, Lonneke van der Plas, Behrang QasemiZadeh, Carlos Ramisch, Federico Sangati, Ivelina Stoyanova & Veronika Vincze 87

## 5 Analysis and Insights from the PARSEME Shared Task dataset

Alfredo Maldonado & Behrang QasemiZadeh 149

## 6 Semantic reranking of CRF label sequences for verbal multiword expression identification

Erwan Moreau, Ashjan Alsulaimani, Alfredo Maldonado, Lifeng Han, Carl Vogel & Koel Dutta Chowdhury 177

## 7 A transition-based verbal multiword expression analyzer

Hazem Al Saied, Marie Candito & Matthieu Constant 209
Contents

<table><tr><td>8</td><td>Identifying verbal multiword expressions with POS tagging and parsing techniques<br/>Katalin Ilona Simkó, Viktória Kovács & Veronika Vincze</td><td>227</td></tr><tr><td>9</td><td>Semi-automated resolution of inconsistency for a harmonized multiword-expression and dependency-parse annotation<br/>Julian Brooke, King Chan & Timothy Baldwin</td><td>245</td></tr><tr><td>10</td><td>Sequence models and lexical resources for MWE identification in French<br/>Manon Scholivet, Carlos Ramisch & Silvio Cordeiro</td><td>263</td></tr><tr><td>11</td><td>Identification of multiword expressions: A fresh look at modelling and evaluation<br/>Shiva Taslimipoor, Omid Rohanian, Ruslan Mitkov & Afsaneh Fazly</td><td>299</td></tr><tr><td>12</td><td>Comparing bilingual word embeddings to translation dictionaries for extracting multilingual collocation equivalents<br/>Marcos Garcia</td><td>319</td></tr><tr><td>13</td><td>Exploiting multilingual lexical resources to predict MWE compositionality<br/>Bahar Salehi, Paul Cook & Timothy Baldwin</td><td>343</td></tr><tr><td colspan="2">Index</td><td>375</td></tr></table>
# Preface

## Stella Markantonatou

Institute for Language and Speech Processing, Athena RIC, Greece

## Carlos Ramisch

Aix Marseille Univ, Université de Toulon, CNRS, LIS, Marseille, France

## Agata Savary

University of Tours, LIFAT, France

## Veronika Vincze

University of Szeged, Hungary

In this introductory chapter we present the rationale for the volume at hand. We explain the origin and the selection process of the contributing chapters, and we sketch the contents and the organization of the volume. We also describe notational conventions put forward for citing and glossing multilingual examples of multiword expressions. We finally acknowledge the efforts which paved the way for setting up this book project, ensuring its quality and publication.

Multiword expressions (MWEs) belong to those language phenomena which pose the hardest challenges both in linguistic modelling and in automatic processing. This is due notably to their semantic non-compositionality, that is, the impossibility to predict their meaning from their syntactic structure and from the semantics of their component words in a way deemed regular for the given language. But MWEs also exhibit unpredictable behaviour on other levels of language modelling such as the lexicon, morphology and syntax, and call, therefore, for dedicated procedures in natural language processing (NLP) applications.

These challenges have been addressed by an ever-growing and increasingly multilingual community gathering at the Multiword Expressions Workshop, organized yearly since 2003, often jointly with major NLP conferences. The 13th

<figure><img src="image_4.png" /></figure>
edition of the Workshop, co-located with the EACL 2017 conference in Valencia, Spain, saw a major evolution of the topics and methods addressed by the community. This evolution resulted notably from the efforts coordinated by PARSEME, a European research network dedicated to parsing and MWEs, gathering, since 2013, researchers from 31 countries and working on as many languages.<fnref n="1" />

One of PARSEME’s main outcomes was a corpus in 18 languages annotated for verbal MWEs (VMWEs), based on a unified methodology and terminology, and published under open licenses. This considerable collective and inclusive effort mobilized experts from different linguistic traditions, triggered cross-language and cross-domain discussions, and brought convergence to modelling and processing of MWE-related phenomena. The availability of this new open resource also made it possible to organize the PARSEME Shared Task on Automatic Identification of Verbal Multiword Expressions, i.e. a competition of VMWE identification tools, whose culminating event was hosted by the MWE 2017 workshop in Valencia. The 7 participating systems covered jointly all 18 languages represented in the corpus. They also offered a large panorama of VMWE identification techniques. These assets, as well as some other contributions published in the main track of the MWE 2017 workshop, showed a growing awareness by the MWE community of specific challenges posed, in particular, by verbal MWEs, such as their discontinuity and high morpho-syntactic flexibility. The workshop programme addressed a large variety of MWE-dedicated tasks such as: lexical and grammatical encoding, annotation, tokenization, extraction, identification, classification, variation study, parsing, compositionality prediction, and translation. Finally, it testified that MWE research has reached a highly multilingual stage.

# 1 Organization and contents of the volume

This volume is a collection of selected extended papers from the MWE 2017 workshop in Valencia: 8 of them from the main track, and 5 from the shared task track. They address 19 languages from 9 language families, as shown in Figure 1. The chapter selection process was initiated by an open call, addressed to all co-authors of the workshop papers. The call included the requirement of extending the original contributions by at least 30% with unpublished content. An international programme committee reviewed 15 submissions and selected 14 of them for publication. One of the selected chapters was further withdrawn. As a result, the volume consists of 13 chapters covering a large variety of aspects related to MWE representation and processing, with a particular focus on verbal MWEs.

<footnote n="1">¹http://www.parseme.eu</footnote>
<figure><img src="image_5.png" /><figcaption>Figure 1: Languages addressed in the chapters of this volume, together with their two-letter language code, language families and genera (middle columns), according to WALS (World Atlas of Language Structures, Dryer & Haspelmath 2013).</figcaption></figure>

Chapters 1 to 3 address outstanding linguistic properties of VMWEs and their automatic assessment. Geeraert et al. (2018 [this volume]) discuss idiomatic variation of several types of English verbal idioms. They draw on multimodal evidence, namely acceptability rating and eye-tracking measurements, to investigate comprehension mechanisms. Barančíková & Kettnerová (2018 [this volume]) deal with light-verb constructions and verbal idioms in Czech, and explore their paraphrasability by single verbs. They also propose a lexicographic scheme for VMWE paraphrase encoding, and show its usefulness in machine translation. Bhatia et al. (2018 [this volume]) focus on English verb-particle constructions, and estimate their compositionality degree, so as to further compute the semantics of sentences containing verb-particle constructions on the basis of lexical, grammatical and ontological resources.
Chapters 4 to 8 are dedicated to the PARSEME shared task on automatic identi-
fication of verbal MWEs. Savary et al. (2018 [this volume]) describe the PARSEME
multilingual VMWE-annotated corpus underlying the shared task. In a first step,
the annotation guidelines and methodology are presented, then the properties of
the annotated corpus are analysed across the 18 participating languages. Maldon-
do & QasemiZadeh (2018 [this volume]) offer a critical analysis of the shared
task organization and of its results across languages and participating systems.
Chapters 6 to 8 are dedicated to three of the seven VMWE identification systems
participating in the shared task. They show a representative panorama of recent
techniques used to address the VMWE identification task. Moreau et al. (2018
(this volume)) model the task as sequence labelling with reranking. Al Saied et
al. (2018 [this volume]) present a dedicated transition-based dependency parser,
which jointly predicts a syntactic dependency tree and a forest of VMWEs. Fi-
nally, Simkó et al. (2018 [this volume]) rely on a generic dependency parser
trained on a corpus with merged syntactic and VMWE labels.

Chapters 9 to 11 further discuss MWE identification issues in various settings
and scopes. Brooke et al. (2018 [this volume]) show how comparing various an-
notations of the same MWE in an English corpus can help correct annotation er-
rors, enhance the consistency of corpus annotation, and consequently increase
the quality of downstream MWE identification systems. Scholivet et al. (2018
(this volume)) address identification of French continuous MWEs via sequence
labelling, compare its results to more sophisticated parsing-based approaches,
and show that feature engineering based on external lexical data (whether hand-
crafted or automatically extracted) systematically enhances the tagging perfor-
mance. Taslimipoor et al. (2018 [this volume]), conversely, advocate modelling
MWE identification as classification rather than tagging. They exploit word em-
beddings as classification features in Italian, Spanish and English, and put for-
ward a MWE-specific methodology of train vs. test corpus split.

The last two chapters of the book, 12 and 13, are dedicated to multilingual
MWE-oriented applications. Garcia (2018 [this volume]) describes automatic ex-
traction of bilingual collocation equivalents in English, Spanish, and Portuguese,
using syntactic dependencies, association measures and distributional models.
Finally, Salehi et al. (2018 [this volume]) predict the compositionality of English
and German MWEs on the basis of their translations extracted from highly mul-
tilingual lexical resources.
## 2 Conventions for citing and glossing multilingual MWE examples

As mentioned above, this volume addresses a large number of languages, particularly in the chapters related to the PARSEME corpus and shared task. Therefore, the editorial effort around this volume includes putting forward notational conventions which might become a standard for citing and glossing multilingual MWE examples. We illustrate the proposed conventions by the numbered examples (1) to (4). Each numbered example contains:

(i) a sample use of the VMWE, followed by the 2-letter ISO 639-1 language code (cf. Figure 1),

(ii) a transcription, if the language of the example is written with a script different from the one used for the main text,<fnref n="2" />

(iii) a gloss following the Leipzig Glossing Rules,<fnref n="3" />

(iv) a literal translation, followed by an idiomatic translation in single quotes.

For English examples, items (ii)–(iv) are irrelevant or optional but idiomatic translation might sometimes be useful to ease the comprehension by non-native readers. For right-to-left languages (e.g. Farsi or Hebrew), item (i) is spelled right-to-left, item (iv) left-to-right and items (ii)–(iii) left-to-right within components, and right-to-left from one component to another. Lexicalized components of the VMWE, i.e. those which are always realized by the same lexeme (cf. Savary et al. 2018 [this volume], §2, p. 92) are highlighted in bold face.

(1) She reluctantly took on this task. (EN)

‘She reluctantly agreed to be in charge of this task.’

(2) Ida skriva glavo v pesek. (SL)

Ida hide.3.SG head in sand

Ida hides her head in the sand. ‘Ida pretends not to see a problem.’

<footnote n="2">²For instance, transcription is needed for Bulgarian, Greek, Farsi and Hebrew examples in this volume. Conversely, examples in English, or any other language using Latin script, would require transcriptions in texts written in Cyrillic, Greek, Arabic or Hebrew script.</footnote>

<footnote n="3">³https://www.eva.mpg.de/lingua/pdf/Glossing-Rules.pdf</footnote>
(3) H Ζωή παίρνει μία απόφαση. (EL)
i Zoi perni mia apofasi
the.FEM.SG Zoe.FEM.SG take.3.SG a decision
Zoe takes a decision. ‘Zoe makes a decision.’

(4) به قدر كافي برای من خواب دیده است ast dide khab man baraye kafi qadre be is seen sleep me for enough quantity to He had enough sleep for me. ‘He has many plans for me.’

*In-line examples*, used for brevity, are preceded by the 2-letter language code, contain items (i), (ii) if relevant, and (iv) only, and the idiomatic translation (if any) is introduced by a double arrow ‘⇒’. For instance, an in-line example corresponding to numbered example (2) would be the following: (SL) *Ida skriva glavo v pesek* ‘Ida hides her head in the sand’ ⇒ ‘Ida pretends not to see a problem’. If the language under study is written with a non-Latin alphabet, the inline example should not be in italics, and the transliteration should be included in parentheses, e.g. (EL) Η Ζωή παίρνει μία απόφαση (I Zoi perni mia apofasi) ‘The Zoe takes a decision’ ⇒ ‘Zoe makes a decision’. To keep such examples reasonably short, the first item can be omitted and only the transliterated example is kept: (EL) I Zoi perni mia apofasi ‘The Zoe takes a decision’ ⇒ ‘Zoe makes a decision’. The literal or the idiomatic translation are sometimes superfluous or too verbose, and can be skipped, as in: (EL) I Zoi perni mia apofasi ‘Zoe makes a decision’.

The typesetting commands for both numbered and in-line examples for LATEX can be found in the GitHub repository containing the source codes of this volume, accessible from its webpage.<fnref n="4" />

## 3 Acknowledgements

Huge collective efforts paved the way towards the publication of this volume.

The PARSEME Shared Task on Automatic Identification of VMWEs was made possible by the European research network PARSEME<fnref n="5" />, funded by the IC1207 COST<fnref n="6" /> Action in 2013–2017. The PARSEME core group and eighteen language teams (cf. acknowledgements in Savary et al. 2018 [this volume]) prepared the annotation guidelines and tools, and created the multilingual corpus underlying

<footnote n="4">⁴http://langsci-press.org/catalog/book/204</footnote>

<footnote n="5">⁵http://www.parseme.eu</footnote>

<footnote n="6">⁶http://www_cost.eu/</footnote>
the shared task. We are also grateful to the COST Officials, notably to Ralph Stüb-
ner, for their continuous support in the scientific and financial administration of
the Action.

The 13th Workshop on Multiword Expressions (MWE 2017)<fnref n="7" /> was organized and sponsored by PARSEME jointly with the Special Interest Group on the Lexicon (SIGLEX)<fnref n="8" /> of the Association for Computational Linguistics. An international Programme Committee of over 80 researchers from 27 countries reviewed the workshop submissions and ensured a high-quality paper selection.

Our warm acknowledgements go also to the Editorial Staff of Language Sci-
ence Press, and in particular to Sebastian Nordhoff, for his expert and friendly
editorial assistance. We are also grateful to the editors of the *Phraseology and
Multiword Expressions* book series for their support. In particular, Yannick Par-
mentier played the role of the editor-in-charge of the volume, and offered advice
on technical editorial issues.

Finally, we thank the following reviewers, who provided insightful reviews to
the chapters submitted to this volume:

* Dimitra Anastasiou (Luxembourg Institute of Science and Technology, Luxembourg)

• Doug Arnold (University of Essex, UK)

• Timothy Baldwin (University of Melbourne, Australia)

• Eduard Bejček (Charles University in Prague, Czech Republic)

• António Branco (University of Lisbon, Portugal)

• Marie Candito (Paris Diderot University, France)

• Fabienne Cap (Uppsala University, Sweden)

• Matthieu Constant (Université de Lorraine, France)

• Paul Cook (University of New Brunswick, Canada)

• Lucia Donatelli (Georgetown University, USA)

• Silvio Ricardo Cordeiro (Aix-Marseille University, France)

<footnote n="7">⁷http://multiword.sourceforge.net/mwe2017</footnote>

<footnote n="8">⁸http://siglex.org/</footnote>
• Béatrice Daille (University of Nantes, France)

• Gaël Dias (University of Caen Basse-Normandie, France)

• Voula Giouli (Institute for Language and Speech Processing/Athena RIC, Greece)

• Tracy Holloway King (eBay, USA)

• Philipp Koehn (Johns Hopkins University, USA)

• Dimitrios Kokkinakis (University of Gothenburg, Sweden)

• Yannis Korkontzelos (Edge Hill University, UK)

• Eric Laporte (Université Paris-Est Marne-la-Vallee, France)

• Timm Lichte (University of Düsseldorf, Germany)

• Gyri S. Losnegaard (University of Bergen, Norway)

• Héctor Martínez Alonso (Thomson Reuters Labs, Canada)

• Verginica Mititelu (Romanian Academy Research Institute for Artificial In-
telligence, Romania)

• Preslav Nakov (Qatar Computing Research Institute, HBKU, Qatar)

• Joakim Nivre (Uppsala University, Sweden)

• Jan Odijk (University of Utrecht, Netherlands)

• Petya Osenova (Bulgarian Academy of Sciences, Bulgaria)

• Harris Papageorgiou (Institute for Language and Speech Processing/Athena RIC, Greece)

• Yannick Parmentier (Université de Lorraine, France)

• Carla Parra Escartín (Dublin City University, ADAPT Centre, Ireland)

• Agnieszka Patejuk (Institute of Computer Science, Polish Academy of Sciences, Poland)

• Pavel Pecina (Charles University in Prague, Czech Republic)
* Scott Piao (Lancaster University, UK)

* Martin Riedl (University of Stuttgart, Germany)

* Manfred Sailer (Goethe-Universität Frankfurt am Main, Germany)

* Nathan Schneider (Georgetown University, USA)

* Sabine Schulte Im Walde (University of Stuttgart, Germany)

* Ruben Urizar (University of the Basque Country, Spain)

* Aline Villavicencio (Federal University of Rio Grande do Sul, Brazil)

* Jakub Waszczuk (University of Tours, France)

* Shuly Wintner (University of Haifa, Israel)

We hope that the quality of this volume will be a valuable reward to all these contributors, and a source of information and inspiration for the international MWE community.

## References

Al Saied, Hazem, Marie Candito & Matthieu Constant. 2018. A transition-based verbal multiword expression analyzer. In Stella Markantonatou, Carlos Ramisch, Agata Savary & Veronika Vincze (eds.), *Multiword expressions at length and in depth: Extended papers from the MWE 2017 workshop*, 209–226. Berlin: Language Science Press. DOI:10.5281/zenodo.1469561

Barančíková, Petra & Václava Kettnerová. 2018. Paraphrases of verbal multiword expressions: The case of Czech light verbs and idioms. In Stella Markantonatou, Carlos Ramisch, Agata Savary & Veronika Vincze (eds.), *Multiword expressions at length and in depth: Extended papers from the MWE 2017 workshop*, 35–59. Berlin: Language Science Press. DOI:10.5281/zenodo.1469553

Bhatia, Archna, Choh Man Teng & James F. Allen. 2018. Identifying senses of particles in verb-particle constructions. In Stella Markantonatou, Carlos Ramisch, Agata Savary & Veronika Vincze (eds.), *Multiword expressions at length and in depth: Extended papers from the MWE 2017 workshop*, 61–86. Berlin: Language Science Press. DOI:10.5281/zenodo.1469555
Brooke, Julian, King Chan & Timothy Baldwin. 2018. Semi-automated resolution of inconsistency for a harmonized multiword-expression and dependency-parse annotation. In Stella Markantonatou, Carlos Ramisch, Agata Savary & Veronika Vincze (eds.), *Multiword expressions at length and in depth: Extended papers from the MWE 2017 workshop*, 245–262. Berlin: Language Science Press.
DOI:10.5281/zenodo.1469565

Dryer, Matthew S. & Martin Haspelmath (eds.). 2013. *The world atlas of language structures online*. Leipzig: Max Planck Institute for Evolutionary Anthropology. https://wals.info/, accessed 2018-7-25. Accessed on.

Garcia, Marcos. 2018. Comparing bilingual word embeddings to translation dictionaries for extracting multilingual collocation equivalents. In Stella Markantonatou, Carlos Ramisch, Agata Savary & Veronika Vincze (eds.), *Multiword expressions at length and in depth: Extended papers from the MWE 2017 workshop*, 319–342. Berlin: Language Science Press. DOI:10.5281/zenodo.1469571

Geeraert, Kristina, R. Harald Baayen & John Newman. 2018. “Spilling the bag” on idiomatic variation. In Stella Markantonatou, Carlos Ramisch, Agata Savary & Veronika Vincze (eds.), *Multiword expressions at length and in depth: Extended papers from the MWE 2017 workshop*, 1–33. Berlin: Language Science Press. DOI:10.5281/zenodo.1469551

Maldonado, Alfredo & Behrang QasemiZadeh. 2018. Analysis and Insights from the PARSEME Shared Task dataset. In Stella Markantonatou, Carlos Ramisch, Agata Savary & Veronika Vincze (eds.), *Multiword expressions at length and in depth: Extended papers from the MWE 2017 workshop*, 149–175. Berlin: Language Science Press. DOI:10.5281/zenodo.1469557

Moreau, Erwan, Ashjan Alsulaimani, Alfredo Maldonado, Lifeng Han, Carl Vogel & Koel Dutta Chowdhury. 2018. Semantic reranking of CRF label sequences for verbal multiword expression identification. In Stella Markantonatou, Carlos Ramisch, Agata Savary & Veronika Vincze (eds.), *Multiword expressions at length and in depth: Extended papers from the MWE 2017 workshop*, 177–207. Berlin: Language Science Press. DOI:10.5281/zenodo.1469559

Salehi, Bahar, Paul Cook & Timothy Baldwin. 2018. Exploiting multilingual lexical resources to predict MWE compositionality. In Stella Markantonatou, Carlos Ramisch, Agata Savary & Veronika Vincze (eds.), *Multiword expressions at length and in depth: Extended papers from the MWE 2017 workshop*, 343–373. Berlin: Language Science Press. DOI:10.5281/zenodo.1469573

Savary, Agata, Marie Candito, Verginica Barbu Mititelu, Eduard Bejček, Fabienne Cap, Slavomír Čeplö, Silvio Ricardo Cordeiro, Gülşen Eryiğit, Voula Giouli, Maarten van Gompel, Yaakov HaCohen-Kerner, Jolanta Kovalevskaitė, Simon
Krek, Chaya Liebeskind, Johanna Monti, Carla Parra Escartín, Lonneke van der Plas, Behrang QasemiZadeh, Carlos Ramisch, Federico Sangati, Ivelina Stoyanova & Veronika Vincze. 2018. PARSEME multilingual corpus of verbal multiword expressions. In Stella Markantonatou, Carlos Ramisch, Agata Savary & Veronika Vincze (eds.), *Multiword expressions at length and in depth: Extended papers from the MWE 2017 workshop*, 87–147. Berlin: Language Science Press.
DOI:10.5281/zenodo.1471591

Scholivet, Manon, Carlos Ramisch & Silvio Cordeiro. 2018. Sequence models and lexical resources for MWE identification in French. In Stella Markantonatou, Carlos Ramisch, Agata Savary & Veronika Vincze (eds.), *Multiword expressions at length and in depth: Extended papers from the MWE 2017 workshop*, 263–297. Berlin: Language Science Press. DOI:10.5281/zenodo.1469567

Simkó, Katalin Ilona, Viktória Kovács & Veronika Vincze. 2018. Identifying verbal multiword expressions with POS tagging and parsing techniques. In Stella Markantonatou, Carlos Ramisch, Agata Savary & Veronika Vincze (eds.), *Multiword expressions at length and in depth: Extended papers from the MWE 2017 workshop*, 227–243. Berlin: Language Science Press.
DOI:10.5281/zenodo.1469563

Taslimipoor, Shiva, Omid Rohanian, Ruslan Mitkov & Afsaneh Fazly. 2018. Identification of multiword expressions: A fresh look at modelling and evaluation. In Stella Markantonatou, Carlos Ramisch, Agata Savary & Veronika Vincze (eds.), *Multiword expressions at length and in depth: Extended papers from the MWE 2017 workshop*, 299–317. Berlin: Language Science Press.
DOI:10.5281/zenodo.1469569
<figure><img src="image_6.png" /></figure>
# Chapter 1

## “Spilling the bag” on idiomatic variation

Kristina Geeraert

KU Leuven

R. Harald Baayen

University of Tübingen & University of Alberta

John Newman

University of Alberta & Monash University

Recent corpus-based studies have shown that idioms can vary much more exten-
sively than previously claimed (Moon 1998; Barlow 2000; Duffley 2013), but little
research has been conducted on how we understand or regard these variants (Gibbs
et al. 1989; McGlone et al. 1994). This study further examines idiomatic variation,
specifically investigating the acceptability and processing of several types of vari-
ants through rating and eye-tracking experiments. Four types of idiom variants
were included, in addition to the canonical form and a literal meaning of the ex-
pression (i.e. in a different context). The results show that altering an idiom does
not render it completely unacceptable nor incomprehensible, but rather, certain
variants are more preferred and easier to understand. These results show support
for probabilistic accounts of language.

### 1 Introduction

Idioms have traditionally been regarded as multiword units whose meaning can-
not be derived from the meaning of its parts (Bobrow & Bell 1973). The literal
meaning of an idiom is the word-by-word compositional meaning of the words,
whereas the idiomatic meaning is stored separately with the idiom, as if a large

<figure><img src="image_7.png" /></figure>

Kristina Geeraert, R. Harald Baayen & John Newman. 2018. “Spilling the bag” on idiomatic variation. In Stella Markantonatou, Carlos Ramisch, Agata Savary & Veronika Vincze (eds.), *Multiword expressions at length and in depth: Extended papers from the MWE 2017 workshop*, 1–33. Berlin: Language Science Press.

DOI:10.5281/zenodo.1469551
word. Furthermore, if the idiom is stored whole, then idioms must also be struc-
turally fixed. This rationale has led researchers to predominantly investigate id-
ioms in their canonical form, and how they are understood in comparison to a
literal paraphrase (Swinney & Cutler 1979; Gibbs 1980; Cacciari & Tabossi 1988;
Titone & Connine 1999).

Recent corpus-based research however has shown that idioms can occur with a range of variation (Moon 1998; Barlow 2000; Langlotz 2006; Schröder 2013), such as syntactic variation (e.g. *her new-found reputation was a bubble that would burst* [from *burst one's bubble* 'shatter one's illusions about something'], *the question begged by* [beg the question 'invite an obvious question}], lexical variation (e.g., *set/start the ball rolling* 'set an activity in motion', *a skeleton in the closet/cupboard* 'a discreditable fact that is kept secret'), truncations (e.g., *make hay* [while the sun shines] 'take advantage of favourable conditions'), and even adverbial or adjective modifications (e.g., *pulling political strings* [pull strings 'use one's power or influence to gain an advantage)], *rock the party boat* [rock the boat 'upset the status-quo)]. This variation can even occur with nondecomposable idioms (Duffley 2013), or idioms thought to be semantically frozen and syntactically fixed, such as *kick the bucket* 'die' (e.g., *no buckets were kicked, reluctant to kick his brimming bucket of life, and my phone kicked the pail last week*). These studies demonstrate that idioms are not nearly as fixed as previously assumed. This variability in idioms, and MULTIWORD EXPRESSIONS (MWEs) more generally, is acknowledged as a key challenge in the automated identification of MWEs in corpora (cf. Savary et al. 2018 [this volume] and Scholivet et al. 2018 [this volume]).

Few studies have investigated idiomatic variation from an experimental per-
spective. Gibbs and colleagues (Gibbs et al. 1989; Gibbs & Nayak 1989) explored
lexical and syntactic variation of decomposable and nondecomposable idioms
using a semantic similarity rating task. They found that decomposable idioms
(i.e. idioms whose constituents contribute to the meaning of the whole, as in
*pop the question* ‘propose marriage’) were rated as more similar in meaning to
a literal paraphrase than were nondecomposable idioms, or idioms whose con-
stituents do not contribute meaning, as in *kick the bucket*. But as Duffley (2013)
has shown, nondecomposable idioms can be modified in context and still retain
their meaning. In addition, the semantic decomposability measure used by Gibbs
and colleagues has not proven a reliable measure, with participants performing
at chance in replication studies (Titone & Connine 1994b, Tabossi, Fanari & Wolf
2008). Replication studies have also shown inconsistent results – decomposable
and nondecomposable idioms are not always found to be statistically different
(Tabossi et al. 2008). Finally, a measure of semantic similarity between an idiom
variant and a literal paraphrase may not be the best method for determining the
comprehension of idiomatic variation. Semantic similarity has been shown to be
predicted by the same local contexts as observed in corpora (Miller & Charles
1991), suggesting that this measure may simply be reflecting how interchange-
able the variant is with its paraphrase, or how acceptable the variant may be at
conveying the meaning in the paraphrase.

Meanwhile, McGlone et al. (1994) explored the semantic productivity of idiom
variation. Variants in this study produced an enhanced idiomatic meaning based
on the original (e.g. *shatter the ice*, from *break the ice*, meaning ‘to break an un-
comfortable or stiff social situation in one fell swoop’). In a self-paced reading
study, they measured the reaction time for participants to read the final sentence
of a story, which contained idioms, variants, or literal paraphrases. They found
that participants were significantly faster at reading the canonical form of the
idiom, but that variants were read as fast as literal paraphrases. They suggest
that canonical forms of idioms are accessed whole, while variants are processed
like literal language and are therefore processed slower. However, they did not
control for the type of variation used in their study. They included instances of
lexical variation (e.g. *shatter the ice*), quantification (e.g. *not spill a single bean*
[*spill the beans* ‘reveal secret information’]), and hyperboles (e.g. *it’s raining the
whole kennel* [*rain cats and dogs* ‘rain very hard’]). It is unclear whether some
types of variants are easier to comprehend than others.

The current study attempts to improve upon these previous studies. We ex-
plore the acceptability and processing of several types of idiom variants through
a rating task and an eye-tracking experiment, respectively. While both of these
experiments have been presented independently elsewhere – the eye-tracking
study was presented in Geeraert, Baayen, et al. (2017) and part of the acceptabil-
ity ratings study was presented in Geeraert, Newman, et al. (2017), but presented
in full below – they have been brought together here in a multi-methodological
study in order to tease apart and contrast speaker judgements from online pro-
cessing. Previous research has sometimes conflated these two methods, making
interpretation difficult (cf. Gibbs et al. 1989; Gibbs & Nayak 1989). But here we dis-
tinctly separate them, utilizing an online measure of processing in addition to a
subjective evaluative measure, to determine any converging or diverging results
between the two methods, which are important for understanding idioms and
idiomatic variation. These two studies utilize the same data, yet are different per-
spectives. Thus, this chapter provides a complete reportage of the larger study, a
discussion of the variables useful for predicting each modality (with suggestions
as to why), as well as a unique perspective in the idiom literature.
Two main research questions are explored. First, how do variants compare with the canonical form? This question explores whether differences between the canonical form and the variants are still present when the type of variation is controlled. Second, how do variants compare with each other? This question explores whether any differences emerge between the variant types – are certain variant types more preferred or easier to process?

This study is largely exploratory, but we do have some predictions about the results. For example, formal idiom blends are often regarded in the literature as being errors or slips of the tongue (Fay 1982; Cutting & Bock 1997). We therefore hypothesized that blends would be least preferred and more difficult to process due to this perceived error-like nature. Meanwhile, some idioms have been shown to occur in “idiom sets” (Moon 1998), such as *shake/quake/quiver in one’s boots* ‘tremble with apprehension’ or *down the drain/chute/tube/toilet* ‘completely lost or wasted’. Given this, we predict that lexical variation will not be more difficult to understand than the canonical form, and may be considered an acceptable variant strategy. A modifier inserted into the idiom should take additional time to process due to the presence of the extra word, but given their relative frequency and overall productivity in corpora (Moon 1998; Schröder 2013), may be the most preferred variant. Lastly, a partial or truncated form will likely be faster to process, due to the omission of a word, but may not be widely accepted due to their limited occurrence in corpora (Moon 1998).

The remainder of the chapter proceeds as follows: We discuss each experiment in turn, beginning with the acceptability ratings, and then the eye-tracking experiment. We discuss the design of each experiment and the results obtained. We conclude with a discussion of our findings, how the results of the two experiments converge or diverge, as well as how the results fit into the larger discussion on idioms, and specifically within a probabilistic approach to language.

# 2 Acceptability rating experiment

## 2.1 Methodology

### 2.1.1 Materials

Sixty idioms were extracted from the Oxford Dictionary of English Idioms (Ayto 2009) and the Collins COBUILD Idioms Dictionary (Sinclair 2011), listed in Appendix A. These idioms varied in length and syntactic structure: 20 three-word idioms consisting of a verb and a noun phrase (V-NP, e.g. *rock the boat*); 20 four-word idioms consisting of a verb and a prepositional phrase (V-PP, e.g. *jump on*
1 “Spilling the bag” on idiomatic variation

the bandwagon ‘join others doing something fashionable’); and 20 five- or six-
word idioms (10 each) consisting of a verb, noun phrase, and a prepositional
phrase (V-NP-PP, e.g. *hear something through the grapevine* ‘hear gossip’). Two
contexts were created for each idiom: one literal and one figurative (e.g. *I used
to pretend I could talk to plants, and I would hear things through the grapevine =
literal; and I used to be a socialite, and I would hear things through the grapevine
= figurative*). Both contexts had identical final clauses, with the idiom in sentence-
final position. As syntactic variation is possible with idioms (Moon 1998; Schröder
2013), the contexts were not restricted to the present tense.

The form listed in the dictionary was regarded as the canonical form (for a dif-
ferent approach to establishing canonical forms of MWEs (see Savary et al. 2018
>this volume)). If more than one form was listed then the form most familiar to
the first author was used, as she spoke the same variety as the participants in
the study. In addition to the canonical form, these idioms were manipulated for
four types of variation within the figurative context (i.e. the context was identi-
cal for all variants). First, lexical variation, where one of the lexical items within
the expression was altered to a synonymous or near-synonymous word (e.g. *dis-
cover something through the grapevine*). Synonyms were selected based on their
naturalness in the context to convey a similar meaning.<fnref n="1" /> Second, partial form
of the idiom, where only a portion of the idiom was presented, usually a key
word or words (e.g. *use the grapevine*). In order for the sentence to still be gram-
matically correct, pronouns or lexically-vague words replaced the missing ele-
ments of the expression, such as *it, them, things* for nouns, or *have, be, do, use*
for verbs. Third, integrated concept, where an additional concept was integrated
into the idiom (e.g. *hear something through the judgemental grapevine*). These ad-
ditional concepts expanded or emphasized the figurative contexts in which the
idiom occurred. Finally, formal idiom blend, where two idioms were blended to-
gether (e.g. *get wind through the grapevine* – blending *hear something through
the grapevine* with *get wind of something* ‘hear a rumour’). Each “experimental”
idiom (i.e. the 60 idioms selected) was paired with a non-experimental idiom for
use in the idiom blend condition. These paired “blending” idioms were chosen for
their intuitive plausibility, but controlled for their syntax and semantics (Cutting
& Bock 1997). Four types of blends were created: same syntax, similar semantics
(sSYN, sSEM); same syntax, different semantics (sSYN, dSEM); different syntax,
similar semantics (dSYN, sSEM); and different syntax, different semantics (dSYN,
dSEM), illustrated in Table 1. Five instances of each type of blend occurred with
the three syntactic types (i.e. V-NP, V-PP, or V-NP-PP), totalling 15 of each blend

<footnote n="1">¹An online thesaurus (http://www.thesaurus.com) was often utilized for synonymous words.</footnote>
type. There is clearly a linguistic playfulness at work in the creation of the id-
iom blends in Table 1, just as there is in many of the non-canonical idiom forms
found in naturally occurring usage (cf. Moon 1998; Duffley 2013). This playful-
ness, it should be noted, presents a special challenge to modelling of MWEs in
the context of NLP work on multiword expressions or annotation of MWEs in
corpora, as noted in Savary et al. (2018 [this volume]). Indeed, Savary et al. (2018
(this volume)) consider “wordplay proneness”, as they call it, a challenge that
“goes far beyond the state of the art in semantic modelling and processing of
VMWEs [verbal MWEs]”.

<table><caption>Table 1: Four types of blends used in the idiom blend condition.</caption><thead><tr><th>Type of blend</th><th>Example</th><th>Source idioms</th><th>Total</th></tr></thead><tbody><tr><td>sSYN, sSEM</td><td>rock the applecart</td><td>rock the boat<br>upset the applecart</td><td>15</td></tr><tr><td>sSYN, dSEM</td><td>shoot your tongue</td><td>shoot the breeze<br>hold your tongue</td><td>15</td></tr><tr><td>dSYN, sSEM</td><td>pass the mustard</td><td>cut the mustard<br>pass muster</td><td>15</td></tr><tr><td>dSYN, dSEM</td><td>face the wringer</td><td>face the music<br>put through the wringer</td><td>15</td></tr></tbody></table>

Half of the idioms had the beginning portion of the expression altered (verb), while the other half had alternations made to the final portion of the expression (noun). In total, there are six conditions: one in a literal context and five in a figurative context (i.e. one canonical form and four variants). The experiment utilized a Latin-square design, where every participant saw each idiom once in one of the six conditions. Six versions of the experiment were created, each one containing 10 idioms in each of the six conditions.

**Conditions:**

1. **Literal meaning** of the idiom in its canonical form
(While the guys were reshingling, they suddenly went through the roof.)

2. **Canonical form** of the idiom in a figurative context
(Although these were new stocks, they suddenly went through the roof.)

3. **Lexical variation** of the idiom in a figurative context
(Although these were new stocks, they suddenly went through the ceiling.)
1 "Spilling the bag" on idiomatic variation

4. **Partial form** of the idiom in a figurative context

(Although these were new stocks, they suddenly went through it.)

5. **Integrated concept** within the idiom in a figurative context

(Although these were new stocks, they suddenly went through the investment roof.)

6. **Idiom blend** of two idioms in a figurative context

(Although these were new stocks, they suddenly went through the charts.)

Since the blending idioms only occurred in one condition (i.e. idiom blend), they were used as fillers in their canonical form in the other five versions of the experiment, occurring in either a figurative or literal context. Each blending idiom was excluded as a control in the version of the experiment where it occurred in the idiom blend condition in order to avoid a bias in the materials. Therefore, in each version of the experiment, 10 of the blending idioms occurred in the idiom blend condition, while the remaining 50 appeared as fillers. Of these fillers, 20 occurred in a figurative context and 30 occurred in a literal context. This was done to increase the number of literal contexts in the experiment so that they were not so underrepresented. In sum, each participant saw 110 items: 60 experimental idioms (10 in each condition) and 50 blending idioms as fillers.

Finally, six practice sentences were created using a different six idioms. These idioms all occurred in their canonical form. Three were in a figurative context and three in a literal context. These were the same for all participants.

2.1.2 Procedure

Using E-prime 2.0 standard edition software, each sentence was presented in random order at the top of the computer screen. The text was presented in a black, bold, 24-point Courier New font, centered on a white background. Below each sentence was a VISUAL ANALOGUE SCALE (VAS), which is a continuous graphical rating scale that allows fine gradations to be measured (Funke & Reips 2012).

Participants were explicitly told that they would be reading sentences contain-
ing English expressions, but that some of the expressions had been modified in
various ways. They were asked to rate the acceptability of the expression, as it
occurred in the sentence, by clicking the mouse anywhere on the provided scale,
which was labelled with "acceptable" on the right and "unacceptable" on the left.
The mouse was repositioned to the centre of the scale on each trial. Participants
were encouraged to use the whole scale before the experiment began, and were
given an opportunity to take a short break halfway through the experiment.

After the participants had completed the rating task, they were asked whether
they knew each idiom. As different speakers are familiar with different subsets
of idioms, this information allowed us to control, at the level of the individual, whether they knew the idiom (Cacciari et al. 2005), while maximizing the number of idioms in the study. Each idiom appeared, in its canonical form, in a black, bold, 22-point Courier New font, centered on a white background. Above the idiom was the question “Do you know this expression?” and below were two boxes, one labelled “yes” and the other labelled “no”. Using the mouse, participants clicked on the appropriate box to respond. The mouse repositioned itself to the center of the screen on each trial.

At the end of this second task, participants were presented with a few addi-
tional questions pertaining to their idiom usage (e.g. How often do you use these
expressions?, Do you like using these expressions?). Participants responded to
these questions using the same VAS scale as the rating task, this time labelled
with a thumbs-up image on the right for a positive response and a thumbs-down
image on the left for a negative one. Lastly, participants were asked to rate the
acceptability of seven prescriptively “incorrect” sentences (LQs), shown below,
also using this VAS scale. These sentences attempted to elicit a measure of the
participant’s flexibility with language and non-standard usage.

**Language questions (LQs):**

1. The only option the school board has is to lay off a large *amount* of people.

2. Slot machines are thought to be more *addicting* than table games.

3. The document had to be signed by both Susan and I.

4. While cleaning the kitchen, Sally looked up and saw a spider on the *roof*.

5. I thought it could’ve *went* either way.

6. She *could care* less what he had to say about it.

7. You have to balance your life, *irregardless* of what anybody thinks.

2.1.3 Participants

Forty-eight undergraduate linguistics students from the University of Alberta participated in this experiment. All participants were native speakers of English. There were 37 female and 11 male participants, ranging from 17–43 years of age. All participants were reimbursed for their time with course credit.

## 2.2 Results

The results were analyzed with mixed-effects linear regression using the lme4
package (Bates et al. 2015) in R (R Core Team 2012). We focus on two analyses:
the rating responses and the rating reaction times. Only the 60 experimental idioms were included in these analyses (i.e. the fillers were not included outside of the idiom blend condition).

Six predictor variables are discussed below. Condition is a factor indicating the condition in which the idiom occurred (e.g. canonical form, lexical variation, idiom blend). Length specifies the number of words within the idiom's canonical form. PairedSemantics is a factor specifying whether the two idioms used in the formal idiom blend have similar or different semantics (e.g. *spill the beans & let the cat out of the bag* 'reveal a secret' = similar; *shoot the breeze* 'have a casual conversation' & *hold your tongue* 'remain silent' = different). Meanwhile, KnowIdiom is a factor indicating the participant's knowledge of the idiom (i.e. yes or no). And Trial is the scaled (i.e. standardized) order of presentation of the stimuli in the experiment. Since the stimuli was presented randomly, this order will be different for each participant.

meanTransparencyRating is the scaled average rating for the transparency (or clarity) of the idiom's meaning as a whole. Since speakers differ in how they interpret the decomposability (i.e. compositionality) of idioms, as evidenced by the low reliability of the decomposability classification task (Titone & Connine 1994b, Tabossi, Fanari & Wolf 2008), we were interested in a measure for how clear or obvious people find the meaning of the idiom as a whole. This measure then, may provide some indication of how literal or figurative people consider an idiom. These ratings were collected in a separate experiment, specifically designed to elicit ratings of transparency. Those participants saw each idiom, along with a definition and an example sentence, and were asked to rate the transparency of the idiom (for further details, see Geeraert 2016). The average rating for each idiom was included as a separate predictor to determine whether transparency influences people's preferences of variation.

### 2.2.1 Rating responses

The first model examines the rating responses. The fixed effects of this model are shown in Table 2. This model has three significant interactions with Condition. The first, between Condition and KnowIdiom, is shown in the top-left panel of Figure 1. As expected, participants are not sensitive to variation when an idiom is unfamiliar. But when the idiom is known, there is a clear preference for the canonical form. Two variants types, integrated concepts and lexical variation, are rated as more acceptable than the others, with a slight preference for variants which have an additional concept inserted into the idiom. The remaining variants: idiom blends, partial forms, and a literal reading of the idiom, are all rated as the least preferred variants.
<table><caption>Table 2: Fixed effects for the acceptability rating responses.</caption><thead><tr><th></th><th>Estimate</th><th>Std. Error</th><th>t-value</th><th>ΔAIC</th></tr></thead><tbody><tr><td>Intercept</td><td>88.81</td><td>6.54</td><td>13.59</td><td></td></tr><tr><td>Condition=Concept</td><td>-19.68</td><td>7.34</td><td>-2.68*</td><td>187.74</td></tr><tr><td>Condition=Blend</td><td>-37.56</td><td>7.36</td><td>-5.10*</td><td></td></tr><tr><td>Condition=Lexical</td><td>-22.69</td><td>7.36</td><td>-3.08*</td><td></td></tr><tr><td>Condition=Literal</td><td>-46.46</td><td>7.36</td><td>-6.31*</td><td></td></tr><tr><td>Condition=Partial</td><td>-45.25</td><td>7.37</td><td>-6.14*</td><td></td></tr><tr><td>KnowIdiom=No</td><td>-30.17</td><td>3.58</td><td>-8.43*</td><td>30.24</td></tr><tr><td>Length</td><td>-2.02</td><td>1.49</td><td>-1.35</td><td>1.98</td></tr><tr><td>meanTransparencyRating</td><td>3.82</td><td>1.91</td><td>2.00</td><td>18.30</td></tr><tr><td>Trial</td><td>1.77</td><td>0.69</td><td>2.58</td><td>4.26</td></tr><tr><td>I(KnowIdiom=No|Condition=Concept)</td><td>13.66</td><td>4.92</td><td>2.78*</td><td>52.10</td></tr><tr><td>I(KnowIdiom=No|Condition=Blend)</td><td>31.76</td><td>4.74</td><td>6.71*</td><td></td></tr><tr><td>I(KnowIdiom=No|Condition=Lexical)</td><td>23.31</td><td>4.94</td><td>4.71*</td><td></td></tr><tr><td>I(KnowIdiom=No|Condition=Literal)</td><td>31.26</td><td>4.74</td><td>6.59*</td><td></td></tr><tr><td>I(KnowIdiom=No|Condition=Partial)</td><td>22.63</td><td>4.85</td><td>4.66*</td><td></td></tr><tr><td>I(Length|Condition=Concept)</td><td>1.09</td><td>1.72</td><td>0.64</td><td>2.63</td></tr><tr><td>I(Length|Condition=Blend)</td><td>2.52</td><td>1.71</td><td>1.48</td><td></td></tr><tr><td>I(Length|Condition=Lexical)</td><td>0.52</td><td>1.71</td><td>0.31</td><td></td></tr><tr><td>I(Length|Condition=Literal)</td><td>4.63</td><td>1.71</td><td>2.71*</td><td></td></tr><tr><td>I(Length|Condition=Partial)</td><td>4.11</td><td>1.71</td><td>2.40</td><td></td></tr><tr><td>I(meanTransparencyRating|Condition=Concept)</td><td>0.72</td><td>2.25</td><td>0.32</td><td>1.32<sup>a</sup></td></tr><tr><td>I(meanTransparencyRating|Condition=Blend)</td><td>2.01</td><td>2.22</td><td>0.90</td><td></td></tr><tr><td>I(meanTransparencyRating|Condition=Lexical)</td><td>3.59</td><td>2.31</td><td>1.56</td><td></td></tr><tr><td>I(meanTransparencyRating|Condition=Literal)</td><td>6.35</td><td>2.27</td><td>2.80*</td><td></td></tr><tr><td>I(meanTransparencyRating|Condition=Partial)</td><td>0.37</td><td>2.31</td><td>0.16</td><td></td></tr></tbody></table>

Length also occurs in a significant interaction with Condition, shown in the top-centre panel of Figure 1. Participants tend to rate idioms as less acceptable in their canonical form if they are longer. This pattern holds for most variants as well: integrated concepts, lexical variation, and formal idiom blends have slopes which are not significantly different from the canonical form and are therefore depicted in grey. Literal meanings and partial forms however are rated as more acceptable if the idiom is longer. Apparently, literal interpretations (which likely may also characterize partial forms) benefit from the presence of many words.

<footnote>* = Factors that remain significant after a Bonferroni correction</footnote>

<footnote><sup>a</sup>An ANOVA test run during model comparison indicates that the inclusion of this interaction is significant (p = 0.045).</footnote>
<figure><img src="image_8.png" /><figcaption>Figure 1: Interactions in the mixed-effects linear regression models for the acceptability rating responses and reaction times. Lines in grey represent factors levels which are not significantly different.</figcaption></figure>

This might suggest that as expressions become longer, the non-idiomatic reading becomes stronger and begins to interfere with the idiomatic reading.

The last interaction, between meanTransparencyRating and Condition, is illustrated in the top-right panel of Figure 1. Higher ratings of acceptability are given to idioms judged to be more transparent. For the condition in which the context enforced a literal reading, the effect of transparency was stronger than for any of the other idiom variants. This result is not unexpected, given that not all idioms have a plausible literal meaning (Titone & Connine 1994b).

Trial was significant as a main effect; participants became more accepting of the stimuli (both variants and the canonical form) the further they advanced through the experiment. But participants differed in exactly how much more accepting they became, as evidenced by the by-Subject random slopes for Trial. These slopes in the random effects structure are in addition to Subject and Idiom included as random effects.<fnref n="2" /> Finally, it is interesting to note that the frequency or syntax of the idiom, as well as whether modifications were made to the verb or the noun, did not affect the acceptability of idioms or variants.

<footnote n="2">²The rating response model and the RT model show the same random effects structure.</footnote>
We also looked specifically at formal idiom blends, given their error-like status in the literature (Fay 1982; Cutting & Bock 1997), in order to explore whether any factors influence their acceptability. Two interactions appear significant, shown in Table 3: both between the participant's knowledge of an idiom and the paired semantics of the two idioms involved in the blend. The bottom-left panel in Figure 1 shows the interaction with knowledge of the experimental idiom, while the bottom-centre panel shows the knowledge of the blending idiom. Both interactions indicate that blends are rated as more acceptable when the meanings of the two idioms differ, and less acceptable when they are similar. Participants significantly rate blends with similar semantics with a lower acceptability if one of the idioms is unknown. A three-way interaction between these variables (i.e. knowledge of both idioms and the semantic similarity of the idioms) is not significant, suggesting that speakers only need to be unfamiliar with one of the idioms to regard semantically similar idiom blends as less acceptable. The noticeability of the unknown idiom likely causes this increase in unacceptability, which is perhaps not as noticeable for those who are familiar with both blended idioms – presumably, they are able to access the meaning of the blend easier, as they are familiar with both idioms from which the parts belong, and therefore are not as surprised or unimpressed by the blend. Finally, meanTransparencyRating is also significant in this model – speakers prefer idiom blends that are more transparent and clearer in meaning.

<table><caption>Table 3: Fixed effects for the acceptability ratings of idiom blends.</caption><thead><tr><th></th><th>Estimate</th><th>Std. Error</th><th>t-value</th><th>ΔAIC</th></tr></thead><tbody><tr><td>Intercept</td><td>63.62</td><td>6.06</td><td>10.50</td><td></td></tr><tr><td>meanTransparencyRating</td><td>5.21</td><td>2.30</td><td>2.26</td><td>3.00</td></tr><tr><td>KnowExpIdiom=Yes</td><td>-10.58</td><td>4.64</td><td>-2.28*</td><td>1.02</td></tr><tr><td>KnowBlendingIdiom=Yes</td><td>-2.13</td><td>4.77</td><td>-0.45</td><td>0.59</td></tr><tr><td>Semantics=Similar</td><td>-21.80</td><td>7.43</td><td>-2.93*</td><td>2.00</td></tr><tr><td>I(Semantics=Similar|KnowExpIdiom=Yes)</td><td>14.87</td><td>6.47</td><td>2.30*</td><td>3.23</td></tr><tr><td>I(Semantics=Similar|KnowBlendingIdiom=Yes)</td><td>14.19</td><td>6.50</td><td>2.18*</td><td>2.74</td></tr></tbody></table>

<footnote>* = Factors that remain significant after a Bonferroni correction</footnote>
### 2.2.2 Rating reaction times

We also analyzed the REACTION TIMES (RTs) for how quickly the participants responded to the acceptability rating task. Faster reaction times are associated with easier judgements of acceptability. The fixed effects for this model are shown in Table 4. Only one interaction, between KnowIdiom and Condition, is significant in this model, illustrated in the bottom-right panel in Figure 1. The RTs associated with each condition are similar for both those who know the idiom and those who do not. Significantly longer RTs are observed with integrated concepts, while significantly shorter RTs are observed with partial forms. These results may simply reflect the fact that the integrated concept condition has an additional word inserted into the idiom, whereas the partial form condition has a word omitted from the expression. This RT difference likely corresponds to length of the expression. The most notable observation is that participants are significantly faster rating the canonical form of the expression if the idiom is known. If the idiom is unknown, the RT to rate the canonical form does not differ significantly from the other variants. These results illustrate that the canonical form has an advantage if it is familiar, but that variants of the same length as the canonical are rated as quickly as if one does not know the expression.

<table><caption>Table 4: Fixed effects for the acceptability rating reaction times.</caption><thead><tr><th></th><th>Estimate</th><th>Std. Error</th><th>t-value</th><th>ΔAIC</th></tr></thead><tbody><tr><td>Intercept</td><td>8.51</td><td>0.04</td><td>226.18</td><td></td></tr><tr><td>Condition=Concept</td><td>0.24</td><td>0.02</td><td>9.65*</td><td>93.85</td></tr><tr><td>Condition=Blend</td><td>0.12</td><td>0.02</td><td>4.70*</td><td></td></tr><tr><td>Condition=Lexical</td><td>0.15</td><td>0.02</td><td>6.22*</td><td></td></tr><tr><td>Condition=Literal</td><td>0.15</td><td>0.02</td><td>5.92*</td><td></td></tr><tr><td>Condition=Partial</td><td>0.06</td><td>0.02</td><td>2.27</td><td></td></tr><tr><td>KnowIdiom=No</td><td>0.17</td><td>0.04</td><td>4.40*</td><td>1.69</td></tr><tr><td>Trial</td><td>-0.08</td><td>0.01</td><td>-8.16</td><td>39.96</td></tr><tr><td>I(KnowIdiom=No|Condition=Concept)</td><td>-0.15</td><td>0.05</td><td>-2.82*</td><td>13.10</td></tr><tr><td>I(KnowIdiom=No|Condition=Blend)</td><td>-0.15</td><td>0.05</td><td>-2.91*</td><td></td></tr><tr><td>I(KnowIdiom=No|Condition=Lexical)</td><td>-0.24</td><td>0.05</td><td>-4.67*</td><td></td></tr><tr><td>I(KnowIdiom=No|Condition=Literal)</td><td>-0.16</td><td>0.05</td><td>-3.08*</td><td></td></tr><tr><td>I(KnowIdiom=No|Condition=Partial)</td><td>-0.11</td><td>0.05</td><td>-2.18</td><td></td></tr></tbody></table>

<footnote>* = Factors that remain significant after a Bonferroni correction</footnote>
In sum, this experiment explored the acceptability of idiomatic variation, using
several types of variants. The canonical form is the most preferred and partici-
pants are quicker at rating this form, but only when the expression is known.
Modifying an idiom makes it less acceptable, but the decrease in acceptability
varies according to the type of alternation – integrating an additional element
(go through the investment roof) or replacing a word with a near-synonym (go
through the ceiling) were considered more acceptable variants. We now turn our
attention to how these variants are understood.

3 Eye-tracking experiment

**3.1 Methodology**

**3.1.1 Materials**

This experiment utilized the same materials as the previous experiment.

**3.1.2 Procedure**

This experiment used the Eye-Link 1000, desk-top mounted video-based eye-
tracking device, manufactured by SR Research. The eye-tracker sampled the pupil
location and size at a rate of 1000Hz, and was calibrated using a 9-point calibra-
tion grid. Calibration occurred at the beginning of the experiment, after the prac-
tice, and again after every 22 sentences, for a total of five blocks. The computer
screen resolution was set to 1920 x 1080 pixels.

The stimuli were presented in two parts. Participants first saw the “context
clause” (e.g., *Although these were new stocks*.), followed by the “idiom clause” (e.g.
*they suddenly went through the roof*) on a separate screen. Each trial began with
a fixation cross presented for 1,000 msec on the left side of a light-grey screen.
Next, they saw the context clause, also on a light-grey background, in a bold,
black, Courier New 30-point font. Every clause was displayed in full and fit on
one line. To exit this screen, participants had to trigger an invisible boundary in
the bottom right corner. A blank, light-grey screen was presented for 1,000 msec
before the fixation cross preceding the idiom clause appeared. The sequence of
screens for the idiom clause was identical to the context clause.

Ten percent of the stimuli were followed by a true/false comprehension ques-
tion, which pertained to the immediately preceding sentence, and were presented
randomly throughout the experiment. Participants pushed one of two buttons on
a game controller to answer these questions, which were clearly labelled on the
question screen. The experiment began with a practice session, which consisted
of six practice sentences and three questions. These were the same for all partic-
ipants, although their order varied.

All participants had normal or corrected-to-normal vision. The right eye of each participant was tracked. Participants sat approximately 85cm from the computer screen, with the camera placed on the desk about 35cm in front of the computer screen. The participants sat in a sound-proof booth, while the experimenter sat outside the booth, running the experiment. The lights were kept on. The experiment was self-paced and took about 45 minutes to complete. Each participant was given an opportunity for a short break half-way through the experiment.

After the participants had completed the eye-tracking portion, they were asked to complete three additional tasks: (1) to indicate their knowledge of each expression; (2) to answer questions pertaining to their idiom usage; and (3) to rate the acceptability of the seven prescriptively “incorrect” sentences (LQs). These tasks were identical to the ones in the acceptability rating experiment.

3.1.3 Participants

Sixty linguistics undergraduate students from the University of Alberta partici-
pated in this experiment. All were native speakers of English, and all were differ-
ent participants than those who participated in the previous study. There were 43
female and 17 male participants, ranging from 17–29 years of age. All participants
were reimbursed for their time with course credit.

3.2 Results

The results were analyzed using mixed-effects linear regression. We focus on the total fixation duration (i.e. the total amount of time spent fixating on the AREA OF INTEREST, or AOI) within two AOIs: the idiom as a whole (i.e. the summed fixations on all words within the idiom) and the altered word within the idiom (i.e. the synonymous word in lexical variation, the additional word in the integrated concept, the semantically vague “replacement” word in partial forms, and the word from another idiom in the idiom blend). As above, the analyses only include the 60 experimental idioms.

Ten predictor variables appeared significant in the models. Condition, Know Idiom, Length, meanTransparencyRating, and Trial are the same variables used in the previous experiment. Gender is a factor specifying whether the participant is male or female. PortionAltered is a factor specifying which part of the idiom (i.e. beginning/verb or ending/noun) was manipulated in the variant. And
pression or not; thus a facilitation effect for knowing the idiom is not observed as it is with the other variants. This particular variant is fixated upon less than the canonical form, likely due to it being shorter in length (i.e. fewer number of words). This is in line with longer fixations observed on integrated concepts – an additional word is integrated into the idiom, making it longer in length and requiring additional fixations.

<table><caption>Table 5: Fixed effects for the idiom as AOI.</caption><thead><tr><th></th><th>Estimate</th><th>Std. Error</th><th>t-value</th><th>ΔAIC</th></tr></thead><tbody><tr><td>Intercept</td><td>6.71</td><td>0.09</td><td>75.97</td><td></td></tr><tr><td>Condition=Concept</td><td>0.49</td><td>0.10</td><td>5.04*</td><td>130.12</td></tr><tr><td>Condition=Blend</td><td>0.08</td><td>0.10</td><td>0.75</td><td></td></tr><tr><td>Condition=Lexical</td><td>0.01</td><td>0.10</td><td>0.05</td><td></td></tr><tr><td>Condition=Literal</td><td>-0.19</td><td>0.10</td><td>-1.94</td><td></td></tr><tr><td>Condition=Partial</td><td>-0.75</td><td>0.16</td><td>-4.80*</td><td></td></tr><tr><td>KnowIdiom=Yes</td><td>-0.18</td><td>0.04</td><td>-4.32*</td><td>34.84</td></tr><tr><td>Length</td><td>0.11</td><td>0.02</td><td>6.76</td><td>40.19</td></tr><tr><td>PortionIdiomAltered=Ending</td><td>-0.06</td><td>0.02</td><td>-2.52*</td><td>3.50</td></tr><tr><td>PC2.LQ</td><td>-0.07</td><td>0.03</td><td>-2.42</td><td>3.60</td></tr><tr><td>LSA Scores</td><td>0.24</td><td>0.07</td><td>3.49</td><td>8.21</td></tr><tr><td>meanVariationRating</td><td>-0.06</td><td>0.01</td><td>-7.23</td><td>43.80</td></tr><tr><td>Gender=Male</td><td>-0.17</td><td>0.08</td><td>-2.17</td><td>2.53</td></tr><tr><td>TrialScaled</td><td>-0.04</td><td>0.01</td><td>-3.78</td><td>10.80</td></tr><tr><td>I(KnowIdiom=Yes|Condition=Concept)</td><td>0.06</td><td>0.05</td><td>1.16</td><td>1.26</td></tr><tr><td>I(KnowIdiom=Yes|Condition=Blend)</td><td>0.08</td><td>0.06</td><td>1.42</td><td></td></tr><tr><td>I(KnowIdiom=Yes|Condition=Lexical)</td><td>0.08</td><td>0.06</td><td>1.52</td><td></td></tr><tr><td>I(KnowIdiom=Yes|Condition=Literal)</td><td>0.03</td><td>0.06</td><td>0.55</td><td></td></tr><tr><td>I(KnowIdiom=Yes|Condition=Partial)</td><td>0.17</td><td>0.06</td><td>2.75*</td><td></td></tr><tr><td>I(Length|Condition=Concept)</td><td>-0.05</td><td>0.02</td><td>-2.62</td><td>14.11</td></tr><tr><td>I(Length|Condition=Blend)</td><td>-0.01</td><td>0.02</td><td>-0.36</td><td></td></tr><tr><td>I(Length|Condition=Lexical)</td><td>0.00</td><td>0.02</td><td>0.20</td><td></td></tr><tr><td>I(Length|Condition=Literal)</td><td>0.02</td><td>0.02</td><td>1.04</td><td></td></tr><tr><td>I(Length|Condition=Partial)</td><td>0.08</td><td>0.03</td><td>2.48</td><td></td></tr></tbody></table>

<footnote>* = Factors that remain significant after a Bonferroni correction</footnote>
<figure><img src="image_9.png" /></figure>

Figure 2: Interactions in the mixed-effects linear regression models for the summed total fixation duration on the whole idiom and the altered word as an AOI. Lines in grey represent factor levels which are not significantly different or slopes which are not significant.

The second interaction, shown in the second panel of Figure 2, is between Condition and Length. Longer idioms show longer summed fixation durations, as expected, due to the increased number of words in the idiom. Lexical variation, formal idiom blends, and the literal meaning of the idiom are not significantly different from the canonical form (shown in grey). The other two variants show a pattern that is significantly different from the canonical form. Integrated concepts show a slight inhibitory effect of length, where an additional concept is more difficult to integrate into shorter idioms (i.e. extra time is required). Whereas partial forms of shorter idioms have even fewer words to fixate upon and therefore show considerably shorter fixation durations. Thus, durations on integrated concepts and partial forms are more comparable to the canonical form when the idiom is longer.<fnref n="4" />

Interestingly, the literal meaning of the idiom shows shorter fixation durations than the canonical form, albeit not quite significantly shorter ($t = -1.94$). The literality of the expression (Titone & Connine 1994a) may be contributing to this result. Nevertheless, a general pattern is evident based on these two above interactions with Condition: variants of the same length as the canonical form are not processed significantly different from this canonical form.

Six main effects are observed in this model. Longer fixation durations are observed on the whole idiom if the beginning (the verb) was altered (i.e. Portion Altered). This is not dependent on the type of variation; all variants are easier to process if the change comes later in the idiom. This is a different result than

<footnote n="4">⁴PC1.logFrequency was also significant in the idiom as AOI model. However, this variable is strongly correlated with Length ($r = -0.9$). This correlation is unsurprising given that PC1.logFrequency was created using adjacent co-occurrence frequencies. Model comparison shows that Length is the more significant predictor in this model, producing a considerably lower AIC value, and therefore was retained at the expense of PC1.logFrequency.</footnote>
that of Gibbs and colleagues (Gibbs et al. 1989; Gibbs & Nayak 1989) who found
no difference with similarity ratings in whether the noun or verb was altered.

MeanVariationRating is also significant. Variants which received higher ac-
ceptability ratings are fixated on less long, suggesting preferred variants are
easier to understand and interpret (or perhaps variants easier to interpret are
preferred). Longer fixation durations appear on idioms which have higher LSA
scores for the idiom’s paraphrase (i.e. LSA Score Paraphrase). This finding seems
initially surprising, as previous analyses on the comprehension of idioms suggest
that idioms are easier to understand when the individual components contribute
meaning to the whole (Gibbs et al. 1989). However, the LSA scores indicate how
similar the local contexts are between the idiom and its paraphrase (i.e. how in-
terchangable is the idiom and its paraphrase). When the LSA score is high (i.e.
the paraphrase is easily interchangable), looking time increases as the contexts
are not distinctive for the idiom. But if the LSA score is low, then the idiom and
its paraphrase are less interchangable, making the context more distinctive and
the idiom more predictable. Interestingly, meanTransparencyRating is not signif-
icant. The degree to which the idiom is considered obvious in meaning does not
seem to influence the comprehension of idioms or variants.

A main effect is also observed for PC2.LQ, a latent variable representing the
participants’ “flexibility” with language (i.e. the more they consider nonstandard
or erroneous forms acceptable). Shorter fixations are observed on idioms, both
the canonical form and variants, if speakers are more flexible with language. It is
interesting to note that this finding is not restricted to only the variants. Gender
also shows a significant main effect – males tend to fixate less long on the id-
iom than females, although we are not quite sure why. Finally, a main effect of
Trial is also significant; participants fixate less long on the idiom the further into
the experiment they get. But the degree to which each participant is affected by
the order of presentation varies, as evidenced by significant by-Subject random
slopes for Trial.<fnref n="5" /> By-Item random slopes for Condition with correlation param-
eters are also significant in this model. These slopes indicate that participants’
fixation durations vary depending on which idiom occurred in which condition
– participants found certain idioms easier or more difficult to understand depend-
ing on the condition in which they occurred.

3.2.2 Altered word as AOI

We next investigate the fixation duration on the altered word (i.e. the word in the
idiom that was manipulated). The fixed effects for this model are shown in Table 6.

<footnote n="5">⁵Both idiom as AOI and altered word as AOI models have the same random effects structure.</footnote>
<table><caption>Table 6: Fixed effects for the altered word as AOI.</caption><thead><tr><th></th><th>Estimate</th><th>Std. Error</th><th>t-value</th><th>ΔAIC</th></tr></thead><tbody><tr><td>Intercept</td><td>5.70</td><td>0.06</td><td>98.48</td><td></td></tr><tr><td>Condition=Concept</td><td>0.47</td><td>0.06</td><td>8.28*</td><td>58.40</td></tr><tr><td>Condition=Blend</td><td>0.15</td><td>0.06</td><td>2.67*</td><td></td></tr><tr><td>Condition=Lexical</td><td>0.09</td><td>0.06</td><td>1.54</td><td></td></tr><tr><td>Condition=Partial</td><td>0.30</td><td>0.07</td><td>4.61*</td><td></td></tr><tr><td>PortionIdiomAltered=Ending</td><td>0.27</td><td>0.06</td><td>4.49*</td><td>17.88</td></tr><tr><td>KnowIdiom=Yes</td><td>-0.04</td><td>0.03</td><td>-1.29</td><td>0.21</td></tr><tr><td>PC2.LQ</td><td>-0.10</td><td>0.03</td><td>-3.12</td><td>2.59</td></tr><tr><td>PC1.logFrequency</td><td>0.03</td><td>0.01</td><td>4.70</td><td>15.43</td></tr><tr><td>meanVariationRating</td><td>-0.07</td><td>0.02</td><td>-4.27</td><td>14.09</td></tr><tr><td>TrialScaled</td><td>-0.04</td><td>0.01</td><td>-2.79</td><td>5.28</td></tr><tr><td>I(PortionIdiomAltered=Ending|Condition=Concept)</td><td>-0.12</td><td>0.08</td><td>-1.46</td><td>9.81</td></tr><tr><td>I(PortionIdiomAltered=Ending|Condition=Blend)</td><td>-0.09</td><td>0.08</td><td>-1.17</td><td></td></tr><tr><td>I(PortionIdiomAltered=Ending|Condition=Lexical)</td><td>-0.02</td><td>0.08</td><td>-0.26</td><td></td></tr><tr><td>I(PortionIdiomAltered=Ending|Condition=Partial)</td><td>-0.40</td><td>0.09</td><td>-4.42*</td><td></td></tr><tr><td>I(PC2.LQ|KnowIdiom=Yes)</td><td>0.06</td><td>0.02</td><td>2.27*</td><td>3.09</td></tr></tbody></table>

As there is no altered word in the literal condition, this section focuses on the four idiom variants: lexical variation, partial forms, idiom blends, and integrated concepts, and how they compare to the canonical form.

The interaction between Condition and PortionAltered is seen in the third panel of Figure 2. The overall pattern is that longer fixation durations occur at the end of the idiom, which is also true for the canonical form. Since the idiom occurs at the end of a sentence, these longer fixations may reflect a sentence wrap-up effect (Rayner et al. 2000; Hirotani et al. 2006). Nevertheless, the altered word for most variants shows significantly longer fixations than the canonical form. This is not true of lexical variation, which is the only variant that does not have significantly longer fixations than the canonical form ($t = 1.54$). Thus, a lexically altered variant is just as easy to process as the canonical form. Partial forms however, appear considerably different from the canonical form. Longer fixations are observed on the altered word when the beginning has been altered, as in *use the grapevine*. But when the ending is altered (e.g., *spilled it*), fixations on the altered word are not significantly different from the canonical form ($t = -1.44$). Altering the verb does not always result in significantly longer fixations (cf. the non-significantly different lexical variant when the beginning is altered),

<footnote>* = Factors that remain significant after a Bonferroni correction</footnote>
however altering the verb to a semantically vague verb (i.e. *be, do, used* – in order to make the sentence grammatical) does significantly inhibit processing.

The second interaction, shown in the last panel of Figure 2, is between knowl-
edge of the idiom (i.e. *KnowIdiom*) and the participant’s flexibility with language
(i.e. PC2.LQ). Flexibility with language only appears facilitative for those who do
not know the idiom, illustrated by the non-significant slope (in grey) for those
who know the expression. Other strategies are apparently relied upon to inter-
pret the idiom when knowledge of it is not available.

Additional main effects are also observed. Fixation durations are longer on the altered word when the co-occurrence frequencies of the idiom are higher. Thus, altering part of a more frequent sequence causes greater processing costs. In addition, participants have shorter fixation durations when the variant is rated as more acceptable (i.e. *meanVariationRating*). The more the variation strategy is preferred with a particular idiom, the easier it is to interpret. Finally, the further the participants get into the experiment (i.e. *Trial*), the shorter their fixation durations on the altered word.

We also specifically looked at idioms blends, to determine whether the syntax
or the semantics of the two merged idioms affects the processing of this variant.
Interestingly, neither of these variables were predictive of fixation duration – we
can understand idiom blends regardless of the syntax or the semantics of the two
idioms used in the blend.

Some of these alternations may have been surprising to the participants, re-
sulting in effects that continued beyond the altered word. We therefore ran a
model to explore any spillover effects from the altered word, shown in Table 7.
As the idiom occurred in sentence-final position, spillover effects from an altered
noun (i.e. the end of the idiom) are not able to be determined; thus, this model
only focuses on spillover effects from an altered verb. We examined the fixation
duration on the first content word after the verb when the verb was manipulated
(i.e. the alternation occurred at the beginning of the idiom).

Spillover effects are observed for all variant types (i.e. *Condition*), but the
longest durations are for integrated concepts and partial forms. Incorporating
an additional word into an idiom results in a processing cost likely due to the
surprisal of this extra word. Integrating this additional information into the id-
iom and context requires extra time. The largest spillover effect is with partial
forms. It appears that the semantically vague words used in these sentences (to
make them grammatical) make these partial forms more difficult to comprehend
and cause considerable spillover effects. It remains to be determined whether
partial forms from more naturalistic language produce this same effect.
<table><caption>Table 7: Fixed effects for the first content word after the verb.</caption><thead><tr><th></th><th>Estimate</th><th>Std. Error</th><th>t-value</th><th>ΔAIC</th></tr></thead><tbody><tr><td>Intercept</td><td>5.95</td><td>0.08</td><td>73.41</td><td></td></tr><tr><td>Condition=Concept</td><td>0.27</td><td>0.07</td><td>3.76*</td><td>11.6</td></tr><tr><td>Condition=Blend</td><td>0.17</td><td>0.06</td><td>2.75*</td><td></td></tr><tr><td>Condition=Lexical</td><td>0.14</td><td>0.05</td><td>2.92*</td><td></td></tr><tr><td>Condition=Partial</td><td>0.30</td><td>0.06</td><td>4.62*</td><td></td></tr><tr><td>PC1.logFrequency</td><td>0.04</td><td>0.01</td><td>3.54</td><td>6.38</td></tr><tr><td>KnowIdiom=Yes</td><td>-0.11</td><td>0.05</td><td>-2.32*</td><td>3.20</td></tr></tbody></table>

* = Factors that remain significant after a Bonferroni correction

The last two effects are PC1.Frequency and KnowIdiom. The higher the co-occurrence frequencies of the idiom, the longer the fixation duration on the first content word after the alternation. Modifying a frequent multiword sequence inhibits processing. However, these spillover effects are reduced if the idiom is familiar (i.e. KnowIdiom).

## 4 Discussion

This study employed a multi-methodological approach to investigate the acceptability and processing of idiomatic variation. One advantage of using multiple methods is that they can reveal greater insights, by contrasting converging and diverging results between the different methods. Converging results can provide greater confidence that a particular result or predictor variable is robust; whereas diverging results can uncover differences due to a specific modality or shed light on findings concerning the larger picture that would otherwise be overlooked or thought contradictory (Arppe & Järvikivi 2007). The findings between the acceptability rating and eye-tracking experiments presented together in this chapter do in fact show converging and diverging results worthy of discussion.

Interestingly, the findings between the experiments primarily show diverging results with regard to our two research questions. For example, our first research question asks how the variants compare with the canonical form, and we see from the acceptability ratings that the canonical form is rated as more acceptable than variants or a literal reading, with speakers clearly preferring this form. However, the processing differences are not nearly as straightforward. Some vari-
ants are processed differently than the canonical form. The variant showing the greatest difference from the canonical form is the partial form of the idiom (e.g., *use the grapevine*). This idiom variant is fixated on less than the canonical form, as expected, largely due to the omission of a word (or words) from the expression. Yet despite this shorter fixation on the whole idiom, participants fixated significantly longer on the “replacement” verbs (i.e. the semantically vague verbs used to connect the idiom to the sentence) and significant spillover effects were observed on the first content word after these verbs. A similar inhibitory effect was not observed if the ending was modified (e.g., *spilled it*). These results are likely due to the design of the experiment. The tightly controlled stimuli used in this study made these partial forms unnatural and difficult to interpret. A study investigating partial forms in naturally occurring language may shed more light on the degree of difficulty for processing this variant.

Idioms with additional concepts integrated into the expression are also pro-
cessed differently from the canonical form. These variants require additional pro-
cessing time, as anticipated, but this longer reading time is largely attributable
to the extra word in the expression. The longer duration on the whole idiom is
very similar to the altered word AOI, suggesting that this variant experiences
very little processing costs over and above having to read an extra word.

However, modification of an idiom's form does not always result in a process-
ing disadvantage. Some variants – lexical variation, formal idiom blends, and
a literal reading of the idiom – are not processed significantly slower than the
canonical. Differences between these variants and the canonical form are ob-
served, such as longer fixations on the altered word (at least for idiom blends)
or some spillover effects if the verb was altered, but these differences do not re-
sult in longer reading times for the idiom as a whole. These findings are partly
in line with our predictions. Only idiom blends were predicted to be processed
slower than the canonical form, due to the potential surprisal at or unrecogniz-
ability with this so-called error. But as observed, they do not present difficulties
in comprehension. Thus, intentional or not, altering a word within an idiom to a
synonymous or non-synonymous word does not result in a processing cost.

Our second research question asks how variants compare with each other. Once again, diverging results between the two methods are evident. Lexical variation and idiom blends are processed quite similarly, showing comparable fixation durations, to each other and to the canonical form. The length of the original idiom is maintained in these variants, possibly explaining these comparable durations. However, they do not share similar acceptabilities. Lexically modified variants are considered much more acceptable than idiom blends. In fact, idiom
blends are even less preferred when the two idioms used to make the blend share similar semantics, possibly explaining why blends are often viewed as errors (Fay 1982; Cutting & Bock 1997). Meanwhile, integrated concepts, which add extra information into the idiom, show longer reading times than the other variants, yet are the most preferred. This higher acceptability was expected, given their relatively frequent occurrence in corpora (Moon 1998; Schröder 2013), and leaves us wondering whether semantically productive lexical variants (cf. McGlone et al. 1994) would show higher levels of acceptability (on par with integrated concepts) compared with the synonymous lexical variants utilized here (following Gibbs et al. 1989). Finally, partial forms and a literal reading of the idiom are not acceptable variation strategies, even though they have comparable (or shorter) reading times to the canonical form.

The findings from these two research questions present two main observations. First, variants which add an extra element or are truncated in some way show longer or shorter reading times, respectively, while modifications that maintain the same length as the canonical form show comparable reading times to the canonical form. Second, variants that preserve more of the canonical form (e.g. integrated concepts, lexical variation) are considered more acceptable, although preference remains with the canonical form (which likely facilitates the learning of idioms and leads to faster recognition).

One cautionary note must be made. These aggregated results show patterns and preferences, but they do not imply that all idioms can be altered using all variation strategies. Much variability, particularly when it comes to comprehension, is also observed. Including the mean acceptability for each idiom in each condition as a control variable in the comprehension models resulted in preferred variants showing shorter reading times. In other words, the way in which an idiom is modified can affect how easy it is to understand. Variability is also observed in the random effects structure of the comprehension models, which have by-Item random slopes with correlation parameters for Condition, indicating that specific idioms can be easier or more difficult to process depending with which condition they occurred. Thus, while variation is possible and general patterns can be observed, there are also idiom-specific preferences that factor into how an idiom is altered, understood, and appreciated.

Converging and diverging results are also observed with the predictor variables in the analyses. Two variables converge between the two methods. Length is shown to be an important predictor and yet is rarely included in the idiom literature (cf. Fanari et al. 2010). Longer idioms require additional processing time, as expected, and there can be some facilitation or inhibitory processing effects de-
pending on the type of variation encountered. A literal reading gains additional approval when the idiom is longer (and perhaps also more transparent), while shorter idioms are even more preferred with the idiomatic reading. Perhaps the extra words in longer idioms clearly identify the metaphorical links associated with the idiom, making a literal reading also more interpretable.

The participant's knowledge of the expression is another important predictor that converges between the two studies. Participants fixated less on the idiom (i.e. shorter reading times) and were faster to rate the expression when they knew the idiom. They also considered the idiom and its variants as more acceptable when it was familiar. Yet surprisingly, research on idioms tends to include an average measure of familiarity (cf. Titone & Connine 1994b), as a control for frequency or as a measure of subjective familiarity. This study demonstrates that a speaker-specific measure of familiarity is important for idiom research, as it incorporates speaker-specific experiences into the model.

Not all participant-related variables show converging results. The language questions (LQs) that were collected to provide a latent measure of the partici- pant's flexibility with language only appeared significant in predicting the com- prehension of idioms, and not their acceptability. Participants who are more flex- ible (i.e. more accepting of non-standard or erroneous forms) have an easier time processing idioms and variants. This of course makes sense; these speakers are not distracted by the specific form used, but focus solely on the message being conveyed.

Frequency was also not predictive of acceptability. Even highly frequent idioms can be regarded as acceptable when altered. But frequency is predictive of comprehension. This variable only appeared in the altered word model (due to the high correlation with length in the idiom model), and revealed that alter- nations made to frequent idioms result in a processing cost. When a sequence of words that typically occur together has been modified, additional time is re- required to interpret the new sequence, as the advantage it once received due to its predictability is no longer available. The opposite pattern is observed for the semantics of formal idiom blends – a significant predictor of acceptability, but not comprehension. Speakers find blends unacceptable when the merged idioms share similar semantics, but appear to have no difficulty interpreting them.

A divergence is also seen with the variable PortionAltered (i.e. where in the idiom the alternation occurred: beginning/verb or ending/noun). This variable is not predictive of acceptability – participants’ judgements were not affected by where in the idiom the alternation occurred (i.e. modifications to nouns and verbs are equally acceptable). However, this variable is predictive of comprehension –
alternations made earlier in the idiom (the verb) result in greater processing costs. Gibbs et al. (1989) found no difference between modifications made to nouns or verbs in their similarity rating task, providing further confirmation that a subjective rating of similarity is not measuring comprehension. In addition, these results may also provide support for a time-dependent nature of idiom processing (Titone & Libben 2014). As one advances through the idiom, the predictability of the idiom becomes greater and the idiomatic meaning accumulates resulting in greater priming effects for later words. It seems reasonable then that changes made later in the expression will be less costly – the meaning is more predictable even if changes have been made.

Finally, a divergence is also evident between which compositionality measure was determined to be predictive for each modality. A measure of transparency is predictive of the acceptability rating responses; speakers prefer idioms that are transparent and clear in meaning. But an objective measure of contextual similarity is predictive of comprehension. Idioms are faster to process in unique or distinctive contexts (i.e. lower LSA scores), because they are more predictable. Thus, evaluative judgements are influenced by the clarity of the expression, whereas comprehension is affected by the local contexts in which the idiom occurs.

These (largely diverging) results, in regards to the predictor variables, nicely capture patterns between the two methods. Clarity of the expression and motivation for the alternation are important for the acceptability of idioms and variants, whereas the placement of the alternation and the local context (i.e. distinctiveness, as well as disruptions to this context) are important for comprehension.

This study has shown that not all variants are processed significantly different than the canonical form and that the predictability of idioms is important, especially during processing. Yet these findings conflict with traditional views of idioms, which claim that idioms cannot be modified without losing their idiomatic meaning, or that idioms are stored and accessed whole along with their idiomatic meaning, since they do not equal the sum of their parts. These traditional approaches proposed a dual-route model to account for the processing of idioms – literal language would be understood incrementally through ordinary linguistic processing and idioms would be accessed directly along with their meaning (cf. Swinney & Cutler 1979; Cacciari & Tabossi 1988). For instance, they could be activated by accessing the “idiom key” (Cacciari & Tabossi 1988), which is the idiomatic configuration indicating that sufficient input has been received. But how does one receive sufficient input if the form has been altered? One proposal is to store each variant, but this inefficient method of handling variation would result in a large burden being placed on the mental lexicon (Baayen et al.
2013). McGlone et al. (1994) proposed that idioms are accessed whole for faster processing, but that they could be understood through ordinary linguistic processing, while variants are understood like literal language (which is why they are processed slower), using various strategies in order to understand them. But this study showed that not all variants are processed slower – variants of the same length as the canonical are processed just as quickly.

This study also shows the importance of predictability in understanding idioms – when the local context is distinctive, idioms are faster to process; when alternations are made later in the expression, variants are easier to process; when frequent sequences are altered, variants are slower to process; and even the more flexible a speaker is with language, the easier idioms are to process. These results are in line with other aspects of predictability or probability seen elsewhere in language. Idioms that have a higher cloze probability have an idiomatic meaning that is available earlier (Cacciari & Tabossi 1988; Titone & Connine 1994a). The combination of words can lead to certain predictions or expectations (Elman 2011): subject-verb combinations lead to predictions about the upcoming object (e.g. *lumberjack cuts* primes *wood*, whereas *surgeon cuts* primes *bone*), and the type of semantic theme can be predicted based on voice (e.g. *she arrested* primes *crook*, but *she was arrested* primes *cop*). Speakers have even been shown to make accurate probabilistic predictions about the type of syntactic choice made by others; for example, in the dative alternation: *because he brought the pony to my children* vs. *because he brought my children the pony* (Bresnan 2007).

These predictions would not be possible if language was understood in a truly compositional way. Some scholars are therefore challenging the traditional view of the mental lexicon, as a list of dictionary entries, and instead are proposing probabilistic approaches to language, where words do not possess meaning but are simply cues to meaning, modulated by context and experience (Elman 2004; 2011; Ramscar & Baayen 2013). These approaches highlight the vast amount of information speakers have available to them, besides simply the meaning of the word – speakers are able to draw upon past experience, cultural norms, event and world knowledge and even the feelings of the speaker to interpret the meaning being communicated.

In one such framework, Implicit Grammar (Baayen & Ramscar 2015), learning a language is about learning which cues (i.e. sounds, morphemes, words, contexts) are informative, or discriminative, for a particular outcome (i.e. meaning). Thus, learning occurs when cues successfully predict outcomes, but also when predictions fail to result in those outcomes. Under this view, idioms and their variants would be processed similarly to literal language: being a sequence of words
which are cues to the intended meaning (cf. Geeraert, Newman, et al. 2017). This is likely why speakers prefer the canonical form – using perfectly good cues for accessing the intended idiomatic meaning. But altering these cues is still possible. Integrated concepts still use the canonical form, but with an additional word inserted into the expression. This extra information takes additional time to integrate, but does not alter the already established cues, making this the most preferred variant. Whereas lexical variation and idiom blends alter one of the cues in the idiom, causing relearning in order to discriminate the new cue with the intended meaning, and making these variants less appreciated. This approach may also explain why idioms can become shorter or truncated over time: certain cues are better at discriminating the intended meaning, while others become irrelevant and are eventually dropped. But before this natural development happens, omitting (potentially useful) cues is considerably less appreciated.

## Acknowledgements

This study was generously supported by an Izaak Walton Killam memorial scholarship, awarded to the first author. In addition, we would like to thank Lauren Rudat for her suggestions on improving the stimuli, and to the editors and anonymous reviewers for their suggestions on improving the text.

## Abbreviations

AOI area of interest

LQ language question

LSA latent semantic analysis

MWE multiword expression

PCA principal components analysis

RT reaction time

VAS visual analogue scale
Appendix A:

Canonical form for the 60 idioms used in the two studies

beat around the bush
burn a hole in your pocket
bury the hatchet
cut the mustard
drink someone under the table
drown your sorrows
fall by the wayside
fly off the handle
get the show on the road
give up the ghost
go against the grain
go through the roof
grind to a halt
have a card up your sleeve
have someone over a barrel
hear something through the grapevine
keep someone on their toes
keep your nose to the grindstone
line your pockets
nip something in the bud
pick your brain
pull the strings
put your foot in your mouth
run the gauntlet
shoot the breeze
skate on thin ice
spin your wheels
swallow your pride
tear a strip off someone
wear your heart on your sleeve

bend the rules
burn your bridges
chomp at the bit
dragged through the mud
drive someone up the wall
face the music
flip your lid
foot the bill
get under someone's skin
give up the ship
go behind someone's back
go with the flow
hang up your boots
have many irons in the fire
have your finger on the pulse
jump on the bandwagon
keep your eye on the ball
lie through your teeth
lose your marbles
paint yourself into a corner
pull someone's leg
pull up your socks
rock the boat
shake in your boots
shoot the messenger
spill the beans
stick to your guns
sweep something under the rug
wash your hands of something
wrap someone around your finger
## References

Arppe, Antti & Juhani Järvikivi. 2007. Every method counts – combining corpus-based and experimental evidence in the study of synonymy. *Corpus Linguistics and Linguistic Theory* 3. 131–160.

Ayto, John (ed.). 2009. *From the horse's mouth: Oxford dictionary of English idioms*. Oxford: Oxford University Press.

Baayen, R. Harald, Peter Hendrix & Michael Ramscar. 2013. Sidestepping the combinatorial explosion: An explanation of n-gram frequency effects based on naive discriminative learning. *Language and Speech* 56. 329–347.

Baayen, R. Harald & Michael Ramscar. 2015. Abstraction, storage and naive discriminative learning. In Ewa Dabrowska & Dagmar Divjak (eds.), *Handbook of cognitive linguistics*, 100–120. Berlin: Mouton de Gruyter.

Balota, David A., Melvin J. Yap, Michael J. Cortese, Keith A. Hutchison, Brett Kessler, Bjorn Loftis, James H. Neely, Douglas L. Nelson, Greg B. Simpson & Rebecca Treiman. 2007. English lexicon project. *Behavior Research Methods* 39(3). 445–459.

Barlow, Michael. 2000. Usage, blends and grammar. In Michael Barlow & Suzanne Kemmer (eds.), *Usage-based models of language*, 315–345. Stanford, CA: CSLI Publications.

Bates, Douglas, Martin Mächler, Ben Bolker & Steve Walker. 2015. Fitting linear mixed-effects models using lme4. *Journal of Statistical Software* 67(1). 1–48.
DOI:10.18637/jss.v067.i01

Bobrow, Samuel A. & Susan M. Bell. 1973. On catching on to idiomatic expressions. *Memory & Cognition* 1. 343–346.

Bresnan, Joan. 2007. Is syntactic knowledge probabilistic? Experiments with the English dative alternation. In Sam Featherston & Wolfgang Sternefeld (eds.), *Roots: Linguistics in search of its evidential base*, 75–96. Berlin/New York: Mouton de Gruyter.

Cacciari, Cristina, Paola Corradini & Roberto Padovani. 2005. Speed of processing effects on spoken idioms comprehension. In Bruno G. Bara, Lawrence Barsalou & Monica Bucciarelli (eds.), *Proceedings of the XXVII annual meeting of the cognitive science society*, 21–23. New Jersey: Lawrence Erlbaum.

Cacciari, Cristina & Patrizia Tabossi. 1988. The comprehension of idioms. *Journal of Memory and Language* 27. 668–683.

Cutting, J. Cooper & Kathryn Bock. 1997. That's the way the cookie bounces: Syntactic and semantic components of experimentally elicited idiom blends. *Memory & Cognition* 25(1). 57–71.
Duffley, Patrick J. 2013. How creativity strains conventionality in the use of idiomatic expressions. In Mike Borkent, Barbara Dancygier & Jennifer Hinnell (eds.), *Language and the creative mind*, 49–61. Stanford, CA: CSLI Publications.

Elman, Jeffrey L. 2004. An alternative view of the mental lexicon. *Trends in Cognitive Sciences* 8(7). 301–306.

Elman, Jeffrey L. 2011. Lexical knowledge without a lexicon. *The Mental Lexicon* 6. 1–33.

Fanari, Rachele, Cristina Cacciari & Patrizia Tabossi. 2010. The role of idiom length and context in spoken idiom comprehension. *European Journal of Cognitive Psychology* 22(3). 321–334.

Fay, David. 1982. Substitutions and splices: A study of sentence blends. In Anne Cutler (ed.), *Slips of the tongue and language production*, 163–195. Amsterdam: Mouton de Gruyter.

Funke, Frederik & Ulf-Dietrich Reips. 2012. Why semantic differentials in web-based research should be made from visual analogue scales and not from 5-point scales. *Field Methods* 24(3). 310–327.

Geeraert, Kristina. 2016. *Climbing on the bandwagon of idiomatic variation: A multi-methodological approach*. University of Alberta Doctoral Dissertation.

Geeraert, Kristina, R. Harald Baayen & John Newman. 2017. Understanding idiomatic variation. In *Proceedings of the 13th Workshop on Multiword Expressions (MWE2017)*, 80–90. Association for Computational Linguistics. April 4, 2017.

Geeraert, Kristina, John Newman & R. Harald Baayen. 2017. Idiom variation: Experimental data and a blueprint of a computational model. *Topics in Cognitive Science* 9(3). 1–17. DOI:10.1111/tops.12263

Gibbs, Raymond W. 1980. Spilling the beans on understanding and memory for idioms in conversation. *Memory & Cognition* 8(2). 149–156.

Gibbs, Raymond W. & Nandini P. Nayak. 1989. Psycholinguistic studies on the syntactic behavior of idioms. *Cognitive Psychology* 21. 100–138.

Gibbs, Raymond W., Nandini P. Nayak, John L. Bolton & Melissa E. Keppel. 1989. Speakers’ assumptions about the lexical flexibility of idioms. *Memory & Cognition* 17(1). 58–68.

Hirotani, Masako, Lyn Frazier & Keith Rayner. 2006. Punctuation and intonation effects on clause and sentence wrap-up: Evidence from eye movements. *Journal of Memory and Language* 54. 425–443.

Landauer, Thomas K., Peter W. Foltz & Darrell Laham. 1998. Introduction to latent semantic analysis. *Discourse Processes* 25. 259–184.
Langlotz, Andreas. 2006. *Idiomatic creativity: A cognitive-linguistic model of idiom-representation and idiom-variation in English*. Amsterdam: John Benjamins.

McGlone, Matthew S., Sam Glucksberg & Cristina Cacciari. 1994. Semantic productivity and idiom comprehension. *Discourse Processes* 17. 167–190.

Miller, George A. & Walter G. Charles. 1991. Contextual correlates of semantic similarity. *Language and Cognitive Processes* 6(1). 1–28.

Moon, Rosamund. 1998. *Fixed expressions and idioms in English*. Oxford: Oxford University Press.

R Core Team. 2012. *R: A language and environment for statistical computing*. R Foundation for Statistical Computing. Vienna, Austria. https://www.R-project.org/. ISBN 3-900051-07-0.

Ramscar, Michael & R. Harald Baayen. 2013. Production, comprehension, and synthesis: A communicative perspective on language. *Frontiers in Psychology* 4. 233. DOI:10.3389/fpsyg.2013.00233

Rayner, Keith, Gretchen Kambe & Susan A. Duffy. 2000. The effect of clause wrap-up on eye movements during reading. *The Quarterly Journal of Experimental Psychology* 53A(4). 1061–1080.

Savary, Agata, Marie Candito, Verginica Barbu Mititelu, Eduard Bejček, Fabienne Cap, Slavomír Čeplö, Silvio Ricardo Cordeiro, Gülşen Eryiğit, Voula Giouli, Maarten van Gompel, Yaakov HaCohen-Kerner, Jolanta Kovalevskaitė, Simon Krek, Chaya Liebeskind, Johanna Monti, Carla Parra Escartín, Lonneke van der Plas, Behrang QasemiZadeh, Carlos Ramisch, Federico Sangati, Ivelina Stoyanova & Veronika Vincze. 2018. PARSEME multilingual corpus of verbal multiword expressions. In Stella Markantonatou, Carlos Ramisch, Agata Savary & Veronika Vincze (eds.), *Multiword expressions at length and in depth: Extended papers from the MWE 2017 workshop*, 87–147. Berlin: Language Science Press. DOI:10.5281/zenodo.1471591

Scholivet, Manon, Carlos Ramisch & Silvio Cordeiro. 2018. Sequence models and lexical resources for MWE identification in French. In Stella Markantonatou, Carlos Ramisch, Agata Savary & Veronika Vincze (eds.), *Multiword expressions at length and in depth: Extended papers from the MWE 2017 workshop*, 263–297. Berlin: Language Science Press. DOI:10.5281/zenodo.1469567

Schröder, Daniela. 2013. *The syntactic flexibility of idioms: A corpus-based approach*. Munich: AVM.

Sinclair, John (ed.). 2011. *Collins COBUILD idioms dictionary*. Harper Collins.

Swinney, David A. & Anne Cutler. 1979. The access and processing of idiomatic expressions. *Journal of Verbal Learning and Verbal Behaviour* 18. 523–534.
Tabossi, Patrizia, Rachele Fanari & Kinou Wolf. 2008. Processing idiomatic expressions: Effects of semantic compositionality. *Journal of Experimental Psychology: Learning, Memory, and Cognition* 34(2). 313–327.

Titone, Debra A. & Cynthia M. Connine. 1994a. Comprehension of idiomatic expressions: Effects of predictability and literality. *Journal of Experimental Psychology: Learning, Memory, and Cognition* 20(5). 1126–1138.

Titone, Debra A. & Cynthia M. Connine. 1994b. Descriptive norms for 171 idiomatic expressions: Familiarity, compositionality, predictability, and literality. *Metaphor and Symbolic Activity* 9(4). 247–270.

Titone, Debra A. & Cynthia M. Connine. 1999. On the compositional and non-compositional nature of idiomatic expressions. *Journal of Pragmatics* 31. 1655–1674.

Titone, Debra A. & Maya R. Libben. 2014. Time-dependent effects of decomposability, familiarity and literal plausibility on idiom meaning activation: A cross-modal priming investigation. *The Mental Lexicon* 9. 473–496.
<figure><img src="image_10.png" /></figure>
# Chapter 2

## Paraphrases of verbal multiword expressions: the case of Czech light verbs and idioms

Petra Barančíková

Charles University

Václava Kettnerová

Charles University

In this chapter, we deal with two types of Czech verbal MWEs: light verb constructions and verbal idiomatic constructions. Many verbal MWEs are characterized by the possibility of being paraphrased by single words. We explore paraphrasability of Czech verbal MWEs by single verbs in a semiautomatic experiment using word embeddings. Further, we propose a lexicographic representation of the obtained paraphrases enriched with morphological, syntactic and semantic information. We demonstrate one of its practical application in a machine translation experiment.

### 1 Introduction

Multiword expressions (MWEs) are widely acknowledged as a serious challenge for both foreign speakers and many NLP tasks (Sag et al. 2002). Out of various MWEs, those that involve verbs are of great significance as verbs represent the syntactic center of a sentence. Baldwin & Kim (2010) distinguish the following four types of verbal MWEs:

* verb-particle constructions (also referred to as particle verbs, or phrasal verbs), e.g., *catch up*, *put on*, *swallow down*;

<figure><img src="image_11.png" /></figure>
* prepositional verbs, e.g., *come across, refer to*;

* light-verb constructions (also referred to as verb-complement pairs, or support verb constructions), e.g., *do a report, give a kiss, make an attempt*;

* verb-noun idiomatic constructions (also referred to as VP idioms), e.g., *spill the beans, pull strings, shoot the breeze*.

In this chapter, we focus on two particular types of Czech verbal MWEs: light-verb constructions (LVCs) and idiomatic verbal constructions (IVCs) as they also represent MWEs in Czech in contrast to the first two types that are primarily expressed as single prefixed verbs.

We explore the possibility of expressing these two types of MWEs by single synonymous verbs, which is considered to be one of their prototypical features, see e.g. Chafe (1968) and Fillmore et al. (1988). The motivation for this work lies in the fact that paraphrases greatly assist in a wide range of NLP applications such as information retrieval (Wallis 1993), machine translation (Madnani & Dorr 2013; Callison-Burch et al. 2006; Marton et al. 2009) or machine translation evaluation (Kauchak & Barzilay 2006; Zhou et al. 2006; Barančíková et al. 2014).

The content of this chapter is an extended version of Barančíková & Kettnerová (2017). In addition, it is further explored with IVCs and linguistic properties of LVCs and IVCs relevant to the paraphrasing task are discussed in detail. The new version of the dictionary of paraphrases is larger and it provides a more elaborated set of morphological, syntactic and semantic features, including information on aspects and aspectual counterparts of verbs.

This chapter is structured as follows. First, linguistic properties of LVCs and IVCs are discussed (§2) and related work on their paraphrases is introduced. Second, a paraphrasing model is proposed, namely the selection of LVCs and IVCs, an automatic extraction of candidates for their paraphrases and their manual evaluation are described in detail (§3). Third, the resulting data and their representation in a dictionary of paraphrases are introduced (§4). Finally, in order to present one of the many practical applications of this dictionary, a random sample of paraphrases of LVCs is used in a machine translation experiment (§5).

## 2 Linguistic properties of LVCs and IVCs

Both LVCs and IVCs represent verbal multiword units: they are composed of separate words that, however, refer to an extralinguistic reality as a whole. Their
linguistic properties relevant for their paraphrasability by single verbs are intro-
duced below.

2.1 Light-verb constructions

The theoretical research on light-verb constructions is characterized by an enor-
mous diversity in terms and analyses used, see esp. Amberber et al. (2010) and
Alsina et al. (1997). Here, we use the term LVC for a multiword unit within which
the verb – not retaining its full semantic content – provides grammatical func-
tions and to which the main predicative content is contributed by a noun; as
a result, such a multiword unit serves as a single predicative unit, see e.g. Algeo
(1995), Alsina et al. (1997) and Butt (2010).<fnref n="1" /> In contrast to IVCs, predicative nouns
in LVCs have the same meanings as in nominal structures, meanings of light
verbs are rather impoverished when compared with their full verb counterparts,
see §2.2.

In the Czech language, the central type of LVCs are represented by LVCs in
which predicative nouns are expressed as a direct or indirect object of a light
verb (e.g., *dostat strach* ‘to get fear’ ⇒ ‘to become afraid’, *vzdát úctu* ‘to pay
tribute’, and *vyvolat pobouření* ‘to provoke indignation’ ⇒ ‘to cause uproar’).
The LVCs in which a predicative noun occupies an adverbial of the light verb,
(e.g., *dát do pořádku* ‘to put in order’, *mít pod kontrolou* ‘to have under control’,
*mít na starosti* ‘to have on care’ ⇒ ‘to be responsible’) are more syntactically and
morphologically fixed than the central type of LVCs (Radimský 2010).

As single predicative units, most LVCs have their single predicative counter-
parts by which they can be paraphrased. A single verb paraphrase can be either
morphologically related, or non-related with the predicative noun representing
the nominal component of the paraphrased LVC. For example, the LVCs *dát
polibek* and *dát pusu* ‘give a kiss’ can be both paraphrased by the verb *políbit*
‘to kiss’, which is morphologically related only with the nominal component of
the first LVC. There is no synonymous verb morphologically related to the nom-
inal component of the second LVC.

In contrast to their single predicative paraphrases, LVCs manifest greater flex-
ibility in their modification, compare e.g. adjectival modifiers of the LVC *dát
polibek* ‘give a kiss’ and the corresponding adverbial modifiers of its single verb
paraphrase *políbit* ‘to kiss’: *dát vášnivý/něžný/letmý/manželský/májový/smrtící
polibek* ‘give a passionate/tender/fleeting/marriage/May/fatal kiss’ vs. *vášnivě/*

<footnote n="1">¹Besides predicative nouns, adjectives, adverbs and verbs can also serve as predicative elements. These cases are left aside here.</footnote>
něžně/letmo/*manželsky/*májově/*smrtelně políbit ‘to kiss passionately/tenderly/fleetingly/*marriagely/*Mayly/?fatally’. Easier modification of LVCs is often considered a motivation for their use (Brinton & Akimoto 1999).

Another motivation lies in the possibility to structure the expressed event in a more subtle way than what single verbs allow. For example, in Czech various combinations of the grammatical aspect of light verbs and the number of predicative nouns allow for the expression of several meanings that cannot be expressed with single verbs; these cases require lexical modification, see Table 1.

Finally, in many cases, the selection of different light verbs allows for per-

<table><caption>Table 1: Possible combinations of the grammatical aspect of the light verbs *dát*<sup>pf</sup>, *dávat*<sup>pf</sup> ‘to give’ and the number of the noun *polibek* ‘kiss’ and their paraphrasability by the perfective and imperfective single verbs *políbit*<sup>pf</sup> and *líbat*<sup>impf</sup> ‘to kiss’, respectively.</caption><thead><tr><th>LVC</th><th>Single verb paraphrase</th><th>Lexical modification</th><th>Example<sup>a</sup></th></tr></thead><tbody><tr><td>sg & pf</td><td>pf</td><td>no</td><td><i>Petr dal Janě polibek.</i><br>‘Peter gave a kiss to Jane.’<br>~ <i>Petr Janu políbil.</i><br>‘Peter kissed Jane.’</td></tr><tr><td>pl & impf</td><td>impf</td><td>no</td><td><i>Petr dával Janě polibky.</i><br>‘Peter gave kisses to Jane.’<br>~ <i>Petr Janu líbal.</i><br>‘Peter was kissing Jane.’</td></tr><tr><td>pl & pf</td><td>pf</td><td>yes</td><td><i>Petr dal Janě polibky.</i><br>‘Peter gave several kisses to Jane.’<br>~ <i>Petr Janu několikrát políbil.</i><br>‘Peter kissed Jane several times.’</td></tr><tr><td>sg & impf</td><td>impf</td><td>yes</td><td><i>Petr Janě dával polibek.</i><br>‘Peter was giving a kiss to Jane.’<br>~ <i>Petr Janu právě líbal.</i><br>‘Peter was just kissing Jane.’</td></tr></tbody></table>

<footnote><sup>a</sup>Let us emphasize that the single verb paraphrases of the last two combinations require to be lexically modified – by the words několikrát ‘several times’ and právě ‘just’, respectively.</footnote>
spectivization of the expressed event from the point of view of its different par-
ticipants, see esp. Kettnerová & Lopatková (2015). For example, besides the light
verb *dát* ‘to give’, the noun *polibek* ‘kiss’ can select the light verb *dostat* ‘to get’ as
well. The LVC *dát polibek* ‘to give a kiss’ promotes a kisser in the subject position
while the LVC *dostat polibek* ‘to get a kiss’ puts a kissee into this position. Both
these LVCs are paraphrasable by a single verb *políbit* ‘to kiss’, however, with dif-
ferent values of the grammatical voice: the LVC *dát polibek* ‘to give a kiss’ can
be paraphrased by the verb *políbit* ‘to kiss’ in the active voice (e.g., *Petr dal Janě
polibek*. ‘Peter gave a kiss to Jane.’ ~ *Petr Janu políbil*. ‘Peter kissed Jane.’) while
the LVC *dostat polibek* ‘to get a kiss’ requires the passive voice of the verb *políbit*
‘to kiss’ (e.g., *Jana dostala od Petra polibek*. ‘Jane got a kiss from Peter.’ ~ *Jana
byla políbena od Petra*. ‘Jane was kissed by Peter’.)

LVCs in NLP. One of the trending topics concerning LVCs in the NLP com-
munity is their automatic identification. In this task, various statistical measures
often combined with information on syntactic and/or semantic properties of
LVCs are employed, see e.g. Bannard (2007) and Fazly et al. (2005). The automatic
detection benefits especially from parallel corpora representing valuable sources
of data in which LVCs can be automatically recognized via word alignment, see
e.g. Chen et al. (2015), de Medeiros Caseli et al. (2010), Sinha (2009), Zarrieß &
Kuhn (2009). However, work on paraphrasing LVCs is still not extensive. For
example, a paraphrasing model has been proposed within the Meaning↔Text
Theory (Žolkovskij & Mel'čuk 1965); its representation of LVCs by means of lex-
ical functions and rules applied in the paraphrasing model are thoroughly de-
scribed in Alonso-Ramos (2007). Further, Fujita et al. (2004) presents a paraphras-
ing model which takes advantage of semantic representation of LVCs by lexical
conceptual structures. As with our method proposed in §3, their model also takes
into account several morphological and syntactic features of LVCs, which have
turned out to be highly relevant for the paraphrasing task.

## 2.2 Idiomatic Verbal Constructions

Despite their low frequency, IVCs form a substantial part of a lexis, see e.g. Bald-
win & Kim (2010), Sag et al. (2002) and Cowie (2001). Similarly to LVCs, defini-
tions of idioms vary depending on diverse purposes of their description, see e.g.
Healy (1968), Fraser (1970), van der Linden (1992) and Nunberg et al. (1994).

Here, we define an IVC as a verbal multiword unit that exhibits strong lexical
co-occurrence restrictions so that at least one of its parts cannot be used with
the same meaning outside the given multiword unit. The idiomatic meaning of
individual components of IVCs is reflected in the fact that they are only rarely
interchangeable with words of similar meanings. IVCs thus represent highly con-
ventionalized multiword units, see e.g. Everaert et al. (2014), Granger & Meunier
(2008) and Cowie (2001). IVCs can exhibit the following specific properties, see
e.g. Burger et al. (2007), Čermák (2001) and Everaert et al. (2014):

* markedness at the syntactic and/or morphological level: e.g., *vzít za své*
  'take as one's own' ⇒ 'to be no more' (syntactically marked as the re-
  flexive adjective *své* does not modify any noun), and *nalít někomu čistého*
  *vína* 'to pour someone pure wine' ⇒ 'to tell someone the honest truth'
  (morphologically marked due to the partitive genitive of the noun *víno*
  'wine', which is highly restricted in contemporary Czech);

* figuration: e.g., *vstát z mrtvých*, 'raise the dead' (as it involves a metaphor), *pověsit se někomu na krk* 'to hang around someone's neck' (as it involves a metonymy);

* fixedness at syntactic and/or morphological level: e.g., *postavit někoho na nohy* 'to put someone back on his feet' (syntactically fixed as it cannot be transformed into the passive structure), and *přijít na jiné myšlenky* 'to come to different ideas' ⇒ 'to find something else to think about' (morphologically fixed as the noun *myšlenka* 'idea' can have only the plural form);

* proverbiality: IVCs are typically used for recurrent socially significant situations, implying often their subjective evaluation (e.g., *vidět někomu do duše* 'to see right through someone');

* informality: IVCs are typically of informal register (e.g., *strčit si něco za klobouk* 'to put something behind a hat' ⇒ 'to stick it up one's jumper').

Some IVCs can be paraphrased by a single word verb, see e.g. the IVC *po-dat někomu pomocnou ruku* ‘to give someone helping hand’ and its single verb paraphrase *pomoci* ‘to help’. However, many IVCs are paraphrasable rather by a whole syntactic structure, see e.g. the IVC *mít slovo* ‘to have a word’ ⇒ ‘to be someone’s turn to speak’.

**IVCs in NLP.** There is considerable work focused on automatic identification of idioms in the text and their extraction (Cook et al. 2007; Li & Sporleder 2009;
Muzny & Zettlemoyer 2013; Peng et al. 2015; Katz 2006). However, little attention has been paid to paraphrases of idioms. Let us introduce two works focused on paraphrases of idioms. First, Pershina et al. (2015) identifies synonymous idioms based on their dictionary definitions and their occurrences in tweets. Similarly, Liu & Hwa (2016) generate paraphrases of idioms using dictionary entries. However, there are no lexical resources available for NLP applications providing information on idioms in Czech.

## 3 Paraphrase model

In this section, the process of extracting paraphrases is described in detail. First, we present the selection of LVCs and IVCs (§3.1). For their paraphrasing, we had initially intended to use some of the existing resources, however, they turned out to be completely unsatisfactory for our task.

First, we used the *ParaPhrase DataBase* (PPDB) (Ganitkevitch & Callison-Burch 2014), the largest paraphrase database available for the Czech language. PPDB was created automatically from large parallel data. Unfortunately, there were only 54 candidates for single verb paraphrases of LVCs present. A manual analysis of these candidates showed that only 2 of them were detected correctly, the rest was noise in PPDB. Similarly for idioms, PPDB contained a correct single verb paraphrase for only 6 IVCs from our data (i.e. about 1%). As this number is clearly insufficient, we chose not to use parallel data for paraphrasing.

Therefore, we adopted another approach to the paraphrasing task applying *word2vec* (Mikolov et al. 2013), a neural network model. *Word2vec* is a group of shallow neural networks generating word embeddings, i.e. representations of words in a continuous vector space depending on the contexts in which they appear. In line with the distributional hypothesis (Harris 1954), semantically similar words are mapped close to each other (measured by the cosine similarity) so we can expect LVCs and IVCs to have similar vector space distribution to their single verb paraphrases.

*Word2vec* computes vectors for single tokens. As both LVCs and IVCs represent multiword units, their preprocessing was thus necessary: each LVC and IVC had to be first identified and connected into a single token (§3.2). Particular settings of our model for an automatic extraction of candidates for single verb paraphrases are described in §3.3.

The advantage of this approach is that only monolingual data – generally easily obtainable in a large amount – is necessary for word embeddings training. The disadvantage is that not only paraphrases can have similar word embeddings.
Antonyms and words with more specific or even different meaning can appear in similar contexts as well. Therefore, a manual evaluation of the extracted candidates is necessary (§3.4).

## 3.1 Data selection

### 3.1.1 LVCs selection

Three different datasets of LVCs – containing together 2,389 unique LVCs<fnref n="2" /> – were used in our experiment. As all the datasets were manually created, they allow us to achieve the desired quality of the resulting data.

The first dataset resulted from the experiment examining the native speakers' agreement on the interpretation of light verbs (Kettnerová et al. 2013). This dataset consists of both LVCs in which predicative nouns are expressed as a direct or indirect object by a prepositionless case (e.g. *položit otázku* 'put a question') and LVCs in which predicative nouns are expressed as an adverbial by a simple prepositional case (e.g., *dát do pořádku* 'put in order') or by a complex prepositional group (e.g., the verb *přejít* 'go' plus the complex prepositional group *ze smíchu do pláče* 'from laughing to crying').

The second dataset resulted from a project aiming to enhance the high coverage valency lexicon of Czech verbs VALLEX<fnref n="3" /> with the information on LVCs (Kettnerová et al. 2016). In this case, only the predicative nouns expressed as the direct object by the prepositionless accusative were selected. For identification of LVCs, the modified test of coreference was applied (Kettnerová & Bejček 2016). As the frequency and saliency have been taken as the main criteria for their selection, the resulting set represents a valuable source of LVCs for Czech.

The third small dataset is represented by LVCs in which the predicative noun is expressed as an adverbial. These LVCs were obtained from the VALLEX lexicon as a result of manual analysis of verbal multiword units marked as idioms. As these multiword units were treated inconsistently in the annotation, including not only IVCs but sometimes also LVCs with predicative nouns in adverbial positions, the obtained dataset had to be manually selected.

As in the VALLEX lexicon, information on aspectual counterparts of the given verbs is available, we have used it to expand these datasets by adding missing aspectual counterparts. The overall number of LVCs in the datasets is presented below in Table 2. The union of LVCs from these datasets has been used in the paraphrase candidates extraction task.

<footnote n="2">²When counting aspectual counterparts separately, the number increases to 3,509 unique LVCs</footnote>

<footnote n="3">³http://ufal.mff.cuni.cz/vallex/3.0/</footnote>
### 3.1.2 IVCs selection

The dataset of IVCs was extracted from the VALLEX lexicon after the manual filtering of LVCs with predicative nouns in adverbial positions, see the third dataset in §3.1.1. From the obtained IVCs, those IVCs that include the highly polysemous pronoun *to* ‘it’ were removed as their automatic identification could be unreliable. The final set consists of 595 IVCs (counting aspectual counterparts separately 621 IVCs), see the statistics provided in Table 2.

<table><caption>Table 2: The number of LVCs and IVCs, verbs and nominal components in the three datasets described in §3.1.1, before (first number) and after (second number) the aspectual counterparts expansion.</caption><thead><tr><th>Dataset</th><th>LVCs</th><th>IVCs</th><th>Verbs</th><th>Nominal components</th></tr></thead><tbody><tr><td>First</td><td>726/1,167</td><td>0/0</td><td>49/84</td><td>612</td></tr><tr><td>Second</td><td>1,640/2,366</td><td>0/0</td><td>126/131</td><td>699</td></tr><tr><td>Third</td><td>104/106</td><td>595/621</td><td>310/324</td><td>324</td></tr><tr><td>Union<sup>a</sup></td><td>2,389/3,509</td><td>595/621</td><td>417/446</td><td>1444</td></tr></tbody></table>

## 3.2 Data preprocessing

We used four large lemmatized and POS-tagged corpora of Czech texts: SYN2000 (Čermák et al. 2000), SYN2005 (Čermák et al. 2005), SYN2010 (Křen et al. 2010) and CzEng 1.0 (Bojar et al. 2011). These corpora were further extended with the data from the Czech Press – a large collection of contemporary news texts containing more than 2,000 million lemmatized and POS-tagged tokens. The overall statistics on all datasets is presented in Table 3.

To generate LVCs and IVCs paraphrases, all the selected LVCs and IVCs (§3.1) had to be automatically identified in the given corpora. For their identification, we started with verbs. First, all verbs in the corpora were detected. From these verbs, only those verbs that represent parts of the selected LVCs and IVCs were further processed. For each selected verb, each noun phrase in the context ± 4 words from the given verb was identified based on POS tags and extracted in case the verb and the given noun phrase can combine in some of the selected LVCs or IVCs.

Further, as word embeddings are generated for single words, each detected noun phrase was connected with its respective verb into a single word unit. In

<footnote><sup>a</sup>The numbers do not add up due to a small overlap among the datasets.</footnote>
<table><caption>Table 3: Basic statistics of datasets (numbers in millions of units).</caption><thead><tr><th>Corpus</th><th>Sentences</th><th>Tokens</th></tr></thead><tbody><tr><td>CNK2000</td><td>2.78</td><td>121.81</td></tr><tr><td>CNK2005</td><td>7.95</td><td>122.99</td></tr><tr><td>CNK2010</td><td>8.18</td><td>122.48</td></tr><tr><td>Czeng 1.0</td><td>14.83</td><td>206.05</td></tr><tr><td>Czech Press</td><td>57.03</td><td>2447.68</td></tr><tr><td>Total</td><td>90.77</td><td>3021.01</td></tr></tbody></table>

cases where some verb could combine with more than one noun phrase into LVCs or IVCs, or in cases where a particular noun phrase could be connected with more than one verb, we followed the principle that every verb should be connected to at least one noun phrase in order to maximize the number of identified LVCs and IVCs. For example, if there were two verbs $v_1$ and $v_2$ in a sentence and $v_1$ had a candidate noun phrase $c_1$, while $v_2$ had two candidate noun phrases $c_1$ and $c_2$, $v_1$ was connected with $c_1$ and $v_2$ with $c_2$. In case this principle was not sufficient, a verb was assigned the closest noun phrase on the basis of word order. When each noun phrase was connected maximally with one verb and each verb was connected maximally with one noun phrase, we have joined the noun phrases to their respective verbs into single word units with the underscore character and deleted the noun phrases from their original positions in sentences.

Further, to compensate sparsity of LVCs and IVCs in the data, after identifying a verb from the selected LVCs and IVCs in the data, its aspectual counterpart – if relevant – has been automatically added. For example, after detecting the imperfective verb *vcházet*<sup>impf</sup> ‘enter’ in the data and the prepositional noun phrase *do dějin* ‘to history’ in its context, not only the given imperfective verb, but also its perfective counterpart *vejít*<sup>pf</sup> have been connected with the given noun phrase into the resulting unit *vcházet\_vejít\_do\_dějin*. We refer to such an artificially constructed unit as an *abstract unit* from now on. The abstract unit *vcházet\_vejít\_do\_dějin* then replaced the verb *vcházet* in the sentence, while the noun phrase *do dějin* was deleted from the sentence. Each LVC and IVC identified in the data is thus represented by a single abstract unit representing also its relevant aspectual counterparts.

On this basis, almost 7 million instances of LVC and IVC abstract units were generated in the corpora, see Table 4. The rank and frequency of the most and the least common ones are presented in Table 5.
<table><caption>Table 4: The number of LVCs and IVCs detected in the data. The first row shows the total number of LVC and IVC abstract units identified in the data. The second row represents the number of their unique instances. The third row provides the number of those unique units with higher frequency than 100 occurrences. The last row shows the number of unique LVCs and IVCs without aspectual counterparts expansion, i.e. after splitting the generated abstract units back to a single verb-a single noun phrase pairs.</caption><thead><tr><th></th><th>LVCs</th><th>IVCs</th></tr></thead><tbody><tr><td>abstract units</td><td>6,541,394</td><td>374,493</td></tr><tr><td>unique abstract units</td><td>1,776</td><td>211</td></tr><tr><td>unique abstract units &gt; 100</td><td>1,361</td><td>153</td></tr><tr><td>unique MWEs</td><td>2,954</td><td>353</td></tr></tbody></table>

<table><caption>Table 5: The ranking of LVC and IVC abstract units identified in the data, based on their frequency.</caption><thead><tr><th>rank</th><th>type</th><th>abstract unit</th><th>frequency</th></tr></thead><tbody><tr><td>1.</td><td>LVC</td><td>mít_problém<br>'have a problem'</td><td>211,296</td></tr><tr><td>2.</td><td>LVC</td><td>mítMožnost<br>'have a possibility'</td><td>207,330</td></tr><tr><td>...</td><td>...</td><td>...</td><td>...</td></tr><tr><td>29.</td><td>IVC</td><td>mít_na_mysli<br>'have in mind'</td><td>43,521</td></tr><tr><td>...</td><td>...</td><td>...</td><td>...</td></tr><tr><td>1986.</td><td>IVC</td><td>chytnout_chytat_chytit_za_spatný_konec<br>'get hold of the wrong end of the stick'</td><td>1</td></tr><tr><td>1987.</td><td>LVC</td><td>přechodit_přecházet_přejít_ze_smíchu_do_pláče<br>'go from laughing to crying'</td><td>1</td></tr></tbody></table>
## 3.3 Word2vec model

To the resulting data, we applied *gensim*, a freely available *word2vec* implementation (Řehůřek & Sojka 2010). In particular, we used a model of vector size 500 with continuous bag of word (CBOW) training algorithm and negative sampling.

As it is impossible for the model to learn anything about a rarely seen word, we set a minimum number of word occurrences to 100 in order to limit the size of the vocabulary to reasonable words. Even though we increased frequencies of LVCs and IVCs by the unified representation for their aspectual counterparts, this limit still filtered more than 300 rarely used LVC and 50 IVC abstract units; the resulting number is provided in the third row of Table 4.

After training the model, for each of 1,361 LVC and 153 IVC abstract units with more than 100 occurrences we extracted 30 words with the most similar vectors. From these 30 words, we selected up to 15 single verbs closest to a given LVC or IVC abstract unit. These verbs were taken as candidates for single verb paraphrases of LVCs or IVCs in that abstract unit. On average, there were 7 candidates for each LVC abstract unit and 10 candidates for each IVC abstract unit.

Before the manual evaluation of the candidates, the abstract units were divided back to individual IVCs or LVCs and their paraphrase candidates were again enriched with their aspectual counterparts from the VALLEX lexicon. This way, annotators could select a paraphrase with a proper aspect for each verbal MWE.

## 3.4 Annotation process

In this section, the annotation process of the candidates for single verb paraphrases of LVCs and IVCs is thoroughly described. Let us repeat that *word2vec* generates semantically similar words depending on the context in which they appear. However, not only words having the same meaning can have similar space representations, but words with an opposite meaning, more specific meaning or even different meaning can be extracted as they can appear in similar contexts as well. Manual processing of the extracted single verbs was thus necessary for evaluating the results of the adopted method.

In the manual evaluation, two annotators were asked to indicate for each instance of the unique paraphrase candidates of an LVC or IVC whether it represents a single verb paraphrase of the given LVC or IVC, or not. For example, the single word verbs *upřednostňovat* and *preferovat* ‘to prefer’ were indicated as paraphrases of the LVC *dávat přednost* ‘to give a preference’. Similarly, for the IVC *prásknout do bot* ‘to bang to the shoes’ ⇒ ‘to take to one’s heels’, the single verbs *utéci* ‘to run away’ and *zdrhnout* ‘to make off’ among others were chosen as paraphrases.
Moreover, single verbs anonymous to LVCs or IVCs were marked as well since they can also function as paraphrases in a modified context. For example, for the LVC *vypovídat pravdu* ‘to tell the truth’ the anonymous verb *lhát* ‘to lie’ was selected as well, as the sentence *Nevypovídá pravdu*. ‘He is not telling the truth.’ can be paraphrased as *Lže*. ‘He is lying.’

Further, when the annotators determined a certain candidate as a single verb paraphrase of an LVC or IVC, they took into account the following four morphological, syntactic and semantic aspects.

First, they had to pay special attention to the morphosyntactic expression of arguments. As Czech encodes syntactic relations via morphological forms, changes in the morphological expression of arguments reflect different perspectives from which the event denoted by an LVC or IVC on the one hand and its single verb paraphrase on the other hand is viewed. For example, the single verb *potrestat* ‘to punish’ paraphrases the LVC *dostat trest* ‘to get a punishment’, however, the morphological forms of the punisher and the punishee, two semantic roles evoked by the given LVC and the single verb, differ. In the LVC *dostat trest* ‘to get punishment’, the punishee (*Petr* ‘Peter’) is expressed by the nominative and the punisher (*otec* ‘father’) has the form of the prepositional group *od+genitive* (e.g., *Petrnom dostal od otceod+gen trest*. ‘Peter got punishment from his father.’), while with its single verb paraphrase *potrestat* ‘to punish’ the nominative encodes the punisher and the accusative expresses the punishee (e.g., *Otecnom Petraacc potrestal*. ‘Father punished Peter.’).

Second, the annotators had to take into account differences between the syntactic structure of a sentence created by an LVC or IVC and by its respective paraphrase. Particularly, the difference between sentences with a subject and subjectless sentences had to be indicated. For example, the LVC *dojít k oddělení* ‘to happen to the separation’ ⇒ ‘the separation happens’ is paraphrasable by the single verb *oddělit se* ‘to separate’, although the LVC forms a subjectless structure, the syntactic structure of its single verb paraphrase needs a subject.

Third, in some cases the reflexive morpheme *se/si*, marking usually intransitive verbs, has to be added to a single verb paraphrase so that its meaning corresponds to a meaning of its respective multiword counterpart. For example, the IVC *vejít do dějin* ‘to come into history’ ⇒ ‘to go down in history’ can be paraphrased by the verb *proslavit* only on the condition that the reflexive morpheme *se* is attached to the verb lemma *proslavit se* ‘to achieve fame’.

Lastly, some verbs function as paraphrases of particular LVCs or IVCs only if nouns in these LVCs or IVCs have certain adjectival modifications. These paraphrases were paired with appropriate adjectives during the annotation. For ex-
ample, if the LVC *provozovat praxi* ‘to run a practice’ is to be paraphrased by the single verb *ordinovat* ‘to see patients’, the adjective *lékařský* ‘medical’ has to modify the noun *praxe* ‘practice’.

The above given four features are not mutually exclusive – they can combine. For example, the verb *zaměstnat* ‘to hire’ is a paraphrase of the LVC *nalézt uplatnění* ‘to find an use’ but both the reflexive morpheme *se* and the adjectival modification *pracovní* ‘working’ are required.

To summarize, for each identified single verb paraphrase *v* of an LVC or IVC *l*, the annotators have chosen from the following options:

* *v* is a paraphrase of *l*
  e.g., *mít zájem* 'to be interested' and *chtít* 'to want';

* *v* is an antonym of *l* (the modification of the context is necessary)
  e.g., *zaznamenat propad* 'to experience a drop' and *stoupnout* 'to rise';

* *v* is a paraphrase of *l* but changes in the morphosyntactic expression of arguments are necessary
  e.g., *dostat nabídku* 'to get an offer' and *nabídnout* 'to offer';

* *v* is a paraphrase of *l* but the change in a sentence structure is required
  e.g., *dojít k poruše* 'to happen to the failure' ⇒ 'the failure happens' and *porouchat se* 'to breakdown';

* *v* is a paraphrase of *l* but the modification of the verb lemma by the reflexive morpheme *se/si* is necessary
  e.g., *nést název* 'bear a name' and *nazývat se* 'to be called';

* *v* is a paraphrase of *l* only if a noun component of *l* is modified by a particular adjectival modification
  e.g., *podat oznámení* 'to make an announcement' can be paraphrased as *žalovat* 'to sue' only if the noun *oznámení* is modified with the adjective *trestní* 'criminal';

* *v* is a not a paraphrase of *l*.

As a result of the annotation, for 1,421 of 2,954 LVCs identified in the data (48,1%) and for 200 of 353 IVCs (56,6%) at least one single verb paraphrase was found. The highest number of single verb paraphrases indicated for one multi-word unit was nine and that was the LVC *provést řez* ‘to make an incision’ and the LVC *dát do pořádku* ‘to put in order’. The total number of the indicated single
verb paraphrases of LVCs and IVCs was 2,912 and 498, respectively, see Table 6 providing results of the annotation including the frequency of the additional morphological, syntactic and semantic features used in the annotation.

<table><caption>Table 6: The basic statistics on the annotation.</caption><thead><tr><th></th><th>LVC</th><th>IVC</th></tr></thead><tbody><tr><td>no constraints</td><td>2063</td><td>336</td></tr><tr><td>+ antonymous</td><td>115</td><td>47</td></tr><tr><td>+ reflexive morpheme</td><td>473</td><td>85</td></tr><tr><td>+ morphosyntactic change</td><td>270</td><td>38</td></tr><tr><td>+ syntactic change</td><td>43</td><td>0</td></tr><tr><td>+ an adjective</td><td>30</td><td>1</td></tr><tr><td>total<sup>4</sup></td><td>2912</td><td>498</td></tr></tbody></table>

## 4 Dictionary of paraphrases

3,410 single verbs indicated by the annotators as paraphrases or antonyms of 1,421 LVCs and 200 IVCs (§3.4) form the lexical stock of *ParaDi* 2.0, a dictionary of single verb paraphrases of Czech multiword units of the selected types.<fnref n="5" />

The format of *ParaDi* 2.0 has been designed with respect to both human and machine readability. The dictionary is thus represented as a plain table in the TSV format, as it is a flexible and language-independent data format.

Each lexical entry in the dictionary describes an individual LVC or IVC, providing the following information:

(i) *type* – the type of the given verbal multiword expression with the following three possible values: LVC (indicating an LVC with the predicative noun in the direct or indirect object position), ILVC (representing an LVC with the predicative noun in the adverbial position), or IVC;

(ii) *verb* – a lemma of the verbal component of the given multiword unit;

(iii) *reflexive* – the reflexive morpheme of the lemma, if relevant;

<footnote n="4">⁴The columns do not add up as the features are not mutually exclusive as mentioned earlier.</footnote>

<footnote n="5">⁵*ParaDi* 2.0 is freely available at the following URL: http://hdl.handle.net/11234/1-2377.</footnote>
(iv) *aspect* – a value of the grammatical aspect of the verb;

(v) *aspectual counterpart* – the aspectual counterpart of the verb, if relevant;

(vi) *noun phrase* – the nominal component of the given multiword unit;

(vii) *morphology* – the morphemic form of the given noun phrase;

(viii) *lemmatized noun phrase* – a lemma representing the noun phrase;

(ix) *synonyms* – a list of synonymous single verb paraphrases;

(x) *antonyms* – a list of antonymous single verbs;

(xi) *adj-modification* – a list of single verb paraphrases and adjectival modifications of the nominal component of the LVC or IVC;

(xii) *structural_change* – a list of single verb paraphrases requiring a change in their sentence structure;

(xiii) *voice_change* – a list of single verb paraphrases requiring changes in the morphosyntactic expression of arguments.

While the information provided in the columns (i)-(viii) concerns multiword units, the information given in (ix)-(xiii) is relevant for their single verb paraphrases. A single verb paraphrase can appear in several columns if it is relevant. For example, the verb paraphrase *zalíbit se* ‘to find appealing’ of the LVC *nalézt zalíbení* ‘to find a delight’ ⇒ ‘to find appealing’ is present in both columns *reflexive* and *voice_change* as it represents the verb paraphrase, which requires both adding the reflexive morpheme *se* to the verb lemma and changes in the morphosyntactic expression of its arguments.

## 5 Machine translation experiment

In this section, we show how the dictionary providing high quality data can be integrated into an experiment with improving statistical machine translation quality. If translated separately, multiword expressions often cause errors in machine translation. For example, IVCs have been reported to negatively affect statistical machine translation systems which might achieve only half of the BLEU score (Papineni et al. 2002) on the sentences containing IVCs compared to those that do not (Salton et al. 2014).
Please rate quality of the following sentences from **best** (1) to **worst** (4). Ties are allowed.

**Source sentence:** Můžeme si tak představit jejich život v Tibetu.

1 2 3 4 We can not imagine their life in Tibet.

1 2 3 4 We can thus imagine their life in Tibet.

1 2 3 4 This way we can get an idea about their life in Tibet.

1 2 3 4 So we can do about their life in Tibet.

Send

<caption>Figure 1: Example of the annotation interface for the MT experiment.</caption>

We took advantage of the *ParaDi* dictionary in a machine translation experiment in order to verify its benefit for one of the key NLP tasks. We experimented only with LVCs as we expected quality of LVC translations higher than those of IVCs due to their weaker lexical markedness and their more common use as their higher frequencies in the data suggested (see Table 4).

We selected 50 random LVCs from the dictionary. For each of them, we randomly extracted one sentence from our data containing the given LVC. This set of sentences is referred to as BEFORE. By substituting the LVC for its first paraphrase, i.e. the closest paraphrase in the vector space, we have created a new dataset, referred to as AFTER. We have translated both these datasets – BEFORE and AFTER – to English using two freely available MT systems – *Google Translate*<fnref n="6" /> (GT) and *Moses*.<fnref n="7" />

We used crowdsourcing for evaluation of the resulting translations. Six annotators were presented randomly a Czech source sentence either from the dataset BEFORE or from AFTER and their English translations in a randomized order. The annotation interface is displayed in Figure 1. For each translated sentence, the annotators had to indicate its quality, allowing for the same ranking of more than one translated sentences.

We collected almost 300 comparisons. The inter-annotator agreement measured by Krippendorff's alpha (Krippendorff 2007), a reliability coefficient developed to measure the agreement between judges, has achieved 0.58, i.e. a moder-

<footnote n="6">⁶http://translate.google.com</footnote>

<footnote n="7">⁷http://quest.ms.mff.cuni.cz/moses/demo.php</footnote>
ate agreement. The results of replacing the selected verbal MWEs by their single verb paraphrases in machine translation are very promising: annotators clearly preferred translations of AFTER (i.e. the translations with single verbs) to BEFORE (i.e. with LVCs), in 45% of cases for Moses and in 44% of cases for Google Translate. The results are consistent for both translation systems, see Table 7.

<table><caption>Table 7: Results of the manual evaluation of the MT experiment. The first column shows the source of the better ranked sentence in the pairwise comparison within one translation model or whether they tied.</caption><thead><tr><td>Source</td><td>Moses</td><td>GT</td></tr></thead><tbody><tr><td>BEFORE</td><td>30%</td><td>33%</td></tr><tr><td>AFTER</td><td>45%</td><td>44%</td></tr><tr><td>TIE</td><td>25%</td><td>23%</td></tr></tbody></table>

However, the example in Table 8 illustrates that even minimal change in a source sentence can substantially change its translations as both the translation models are phrase-based.<fnref n="8" /> Based on this fact, we can expect that the evaluation of the translations was not affected only by differences between translations of LVCs and their respective single verb paraphrases but by overall low quality of the translations, which is inevitably reflected in the lower inter-annotator agreement, typical of machine translation evaluation (Bojar et al. 2013). The judges unanimously agreed that the translations of the AFTER source sentence are better than the translations of the BEFORE source sentence. Both systems exhibited a tendency to translate the LVC *dát branku* literally word by word, resulting in incorrect translations of the BEFORE source sentence.

# 6 Conclusion

We have explored the paraphrasability of Czech light-verb constructions and idiomatic verbal constructions. We have shown that their single verb paraphrases are automatically obtainable from large monolingual data with a manual verification in a significantly larger scale than from paraphrase tables generated from parallel data. Our semiautomatic experiment further revealed that although these verbal multiword units exhibit different linguistic properties, the possibility to

<footnote n="8">⁸The translations were performed on 9th July 2016, i.e. before a massive expansion of neural translation systems.</footnote>
<table><caption>Table 8: An example of translated sentences.</caption><tbody><tr><td rowspan="2">Source</td><td>BEFORE</td><td><b>Fotbalisté Budějovic opět nedali branku</b><br/>Footballers Budějovice again did not give gate<br/>'Footballers of Budějovice didn't make a goal again.'</td></tr><tr><td>AFTER</td><td><b>Fotbalisté Budějovic opět neskórovali</b><br/>Footballers Budějovice again did not score<br/>'Footballers of Budějovice didn't score again.'</td></tr><tr><td rowspan="2">GT</td><td>BEFORE</td><td>Footballers Budějovice again not given goal</td></tr><tr><td>AFTER</td><td>Footballers did not score again Budejovice</td></tr><tr><td rowspan="2">Moses</td><td>BEFORE</td><td>Footballers Budějovice again gave the gate</td></tr><tr><td>AFTER</td><td>Footballers Budějovice score again</td></tr></tbody></table>

paraphrase them is very similar; for about one half of the selected light-verb constructions and idiomatic verbal constructions single verb paraphrases have been detected.

The results of our experiment form the lexical stock of a new version of the freely available *ParaDi* dictionary. We have demonstrated one of its possible applications, namely an experiment with improving machine translation quality. However, the dictionary can be used in many other NLP tasks (text simplification, information retrieval, etc.). We have used largely language independent methods, a similar dictionary can be thus created for other languages as well.

## Acknowledgments

The research reported in this chapter was supported by the Czech Science Foundation GA ČR, grant No. GA15-09979S. This work used language resources developed and/or stored and/or distributed by the LINDAT-Clarin project of the Ministry of Education, Youth and Sports of the Czech Republic, project No. LM2015071.

## Abbreviations

<table><tr><td>GT</td><td>Google Translate</td><td>LVCS</td><td>light-verb constructions</td></tr><tr><td>IVCS</td><td>idiomatic verbal constructions</td><td>MT</td><td>Machine Translation</td></tr></table>
## References

Algeo, John. 1995. Having a look at the expanded predicate. In Bas Aarts & Charles F. Meyer (eds.), *The verb in contemporary English: Theory and description*, 203–217. Cambridge: Cambridge University Press.

Alonso-Ramos, Margarita. 2007. Towards the synthesis of support verb constructions: Distribution of syntactic actants between the verb and the noun. In Leo Wanner (ed.), *Selected lexical and grammatical issues in the Meaning-Text Theory*, 97–137. Amsterdam, Philadelphia: John Benjamins Publishing Company.

Alsina, Alex, Joan Bresnan & Peter Sells (eds.). 1997. *Complex predicates*. Stanford: CSLI Publications.

Amberber, Mengistu, Brett Baker & Mark Harvey (eds.). 2010. *Complex predicates in cross-linguistic perspective*. Cambridge: Cambridge University Press.

Baldwin, Timothy & Su Nam Kim. 2010. Multiword expressions. In Nitin Indurkhya & Fred J. Damerau (eds.), *Handbook of Natural Language Processing, Second edition*, 267–292. Boca Raton: CRC Press.

Bannard, Colin James. 2007. A measure of syntactic flexibility for automatically identifying multiword expressions in corpora. In *Proceedings of the Workshop on a Broader Perspective on Multiword Expressions* (MWE ’07), 1–8. Association for Computational Linguistics.

Barančíková, Petra & Václava Kettnerová. 2017. ParaDi: Dictionary of paraphrases of Czech complex predicates with light verbs. In *Proceedings of the 13th Workshop on Multiword Expressions* (MWE ’17), 1–10. Association for Computational Linguistics. April 4, 2017.

Barančíková, Petra, Rudolf Rosa & Aleš Tamchyna. 2014. Improving evaluation of English-Czech MT through paraphrasing. In Nicoletta Calzolari, Khalid Choukri, Thierry Declerck, Hrafn Loftsson, Bente Maegaard & Joseph Mariani (eds.), *Proceedings of the 9th International Conference on Language Resources and Evaluation* (LREC 2014), 596–601. European Language Resources Association (ELRA).

Bojar, Ondřej, Christian Buck, Chris Callison-Burch, Christian Federmann, Barry Haddow, Philipp Koehn, Christof Monz, Matt Post, Radu Soricut & Lucia Spacia. 2013. Findings of the 2013 Workshop on Statistical Machine Translation. In *Proceedings of the 8th Workshop on Statistical Machine Translation*, 1–44. Association for Computational Linguistics.

Bojar, Ondřej, Zdeněk Žabokrtský, Ondřej Dušek, Petra Galuščáková, Martin Majliš, David Mareček, Jiří Maršík, Michal Novák, Martin Popel & Aleš Tamchyna. 2011. *Czech-English parallel corpus 1.0 (CzEng 1.0)*. LINDAT/CLARIN
digital library at Institute of Formal and Applied Linguistics, Charles University in Prague.

Brinton, Laurel & Minoji Akimoto (eds.). 1999. *Collocational and idiomatic aspects of composite predicates in the history of English*. Amsterdam, Philadelphia: John Benjamins Publishing Company.

Burger, Harald, Dmitrij O. Dobrovol’skij, Peter Kühn & Neal R. Norrick (eds.). 2007. *Phraseologie / phraseology: Ein internationales Handbuch zeitgenössischer Forschung / an international handbook of contemporary research: Volume 1*. Berlin/ New York: Walter de Gruyter.

Butt, Miriam. 2010. The light verb jungle: Still hacking away. In Mengistu Amberber, Brett Baker & Mark Harvey (eds.), *Complex predicates in cross-linguistic perspective*, 48–78. Cambridge: Cambridge University Press.

Callison-Burch, Chris, Philipp Koehn & Miles Osborne. 2006. Improved statistical machine translation using paraphrases. In *Proceedings of the Main Conference on Human Language Technology Conference of the North American Chapter of the Association of Computational Linguistics (HLT-NAACL ’06)*, 17–24. Association for Computational Linguistics.

Čermák, František. 2001. Substance of idioms: Perennial problems, lack of data or theory? *International Journal of Lexicography* 14(1). 1–20.

Čermák, František, Renata Blatná, Jaroslava Hlaváčová, Jan Kocek, Marie Kopřivová, Michal Křen, Vladimír Petkevič, Věra Schmiedtová & Michal Šulc. 2000. *SYN2000: Balanced corpus of written Czech*. LINDAT/CLARIN digital library at Institute of Formal and Applied Linguistics, Charles University in Prague.

Čermák, František, Jaroslava Hlaváčová, Milena Hnátková, Tomáš Jelínek, Jan Kocek, Marie Kopřivová, Michal Křen, Renata Novotná, Vladimír Petkevič, Věra Schmiedtová, Hana Skoumalová, Johanka Spoustová, Michal Šulc & Zdeněk Velíšek. 2005. *SYN2005: Balanced corpus of written Czech*. LINDAT/CLARIN digital library at Institute of Formal and Applied Linguistics, Charles University in Prague.

Chafe, Wallace L. 1968. Idiomaticity as an anomaly in the chomskyan paradigm. *Foundations of Language* 4(2). 109–127.

Chen, Wei-Te, Claire Bonial & Martha Palmer. 2015. English light verb construction identification using lexical knowledge. In *Proceedings of the 29th AAAI Conference on Artificial Intelligence (AAAI’15)*, 2375–2381. AAAI Press.

Cook, Paul, Afsaneh Fazly & Suzanne Stevenson. 2007. Pulling their weight: Exploiting syntactic forms for the automatic identification of idiomatic expressions in context. In *Proceedings of the Workshop on a Broader Perspective on*
*Multiword Expressions* (MWE '07), 41–48. Association for Computational Linguistics.

Cowie, Anthony (ed.). 2001. *Phraseology: Theory, analysis, and applications*. Oxford, UK: Oxford University Press.

de Medeiros Caseli, Helena, Carlos Ramisch, Maria das Graças Volpe Nunes & Aline Villavicencio. 2010. Alignment-based extraction of multiword expressions. *Language Resources and Evaluation* 44(1-2). 59–77.

Everaert, Martin, Erik-Jan van der Linden, André Schenk & Rob Schreuder. 2014. *Idioms: Structural and psychological perspectives*. New York, USA & East Sussex, UK: Psychology Press.

Fazly, Afsaneh, Ryan North & Suzanne Stevenson. 2005. Automatically distinguishing literal and figurative usages of highly polysemous verbs. In *Proceedings of the ACL-SIGLEX Workshop on Deep Lexical Acquisition* (DeepLA '05), 38–47. Association for Computational Linguistics.

Fillmore, Charles J., Paul Kay & Mary Catherine O'Connore. 1988. Regularity and idiomaticity in grammatical constructions: The case of *Let Alone*. *Language* 64(3). 501–538.

Fraser, Bruce. 1970. Idioms within a transformational grammar. *Foundations of Language* 6(1). 22–42. http://www.jstor.org/stable/25000426.

Fujita, Atsushi, Kentaro Furihata, Kentaro Inui, Yuji Matsumoto & Koichi Takeuchi. 2004. Paraphrasing of Japanese light-verb constructions based on lexical conceptual structure. In *Proceedings of the Workshop on Multiword Expressions: Integrating Processing (MWE 2004)* (MWE '04), 9–16. Association for Computational Linguistics.

Ganitkevitch, Juri & Chris Callison-Burch. 2014. The multilingual paraphrase database. In Nicoletta Calzolari, Khalid Choukri, Thierry Declerck, Hrafn Loftson, Joseph Mariani Bente Maegaard, Asuncion Moreno, Jan Odijk & Stelios Piperidis (eds.), *Proceedings of the 9th International Conference on Language Resources and Evaluation (LREC 2014)*. European Language Resources Association (ELRA).

Granger, Sylviane & Fanny Meunier (eds.). 2008. *Phraseology: An interdisciplinary perspective*. Amsterdam / Philadelphia: John Benjamins.

Harris, Zellig. 1954. Distributional structure. *Word* 10(23). 146–162.

Healy, Adam. 1968. English idioms. *Kivung (Journal of the Linguistic Society of the University of Papua New Guinea)* 1(2). 71–108.

Katz, Graham. 2006. Automatic identification of non-compositional multi-word expressions using latent semantic analysis. In *Proceedings of the ACL/COLING-
06 Workshop on Multiword Expressions: Identifying and Exploiting Underlying Properties (MWE '06), 12–19. Association for Computational Linguistics.

Kauchak, David & Regina Barzilay. 2006. Paraphrasing for automatic evaluation. In *Proceedings of the Main Conference on Human Language Technology Conference of the North American Chapter of the Association of Computational Linguistics* (HLT-NAACL '06), 455–462. Association for Computational Linguistics.

Kettnerová, Václava, Petra Barančíková & Markéta Lopatková. 2016. Lexico-graphic description of Czech complex predicates: Between lexicon and grammar. In George Meladze Tinatin Margalitadze (ed.), *Proceedings of the 17th EU-RALEX international congress*. Tbilisi, Georgia: Ivane Javakhishvili Tbilisi University Press. September 6-10, 2016.

Kettnerová, Václava & Eduard Bejček. 2016. Distribution of valency complements in Czech complex predicates: Between verb and noun. In Nicoletta Calzolari, Khalid Choukri, Thierry Declerck, Marko Grobelnik, Bente Maegaard, Joseph Mariani, Asunción Moreno, Jan Odijk & Stelios Piperidis (eds.), *Proceedings of the 10th International Conference on Language Resources and Evaluation (LREC 2016)*, 515–521. Paris, France: European Language Resources Association.

Kettnerová, Václava & Markéta Lopatková. 2015. At the lexicon-grammar interface: The case of complex predicates in the functional generative description. In Eva Hajičová & Joakim Nivre (eds.), *Proceedings of depling 2015*, 191–200. Uppsala, Sweden: Uppsala University.

Kettnerová, Václava, Markéta Lopatková, Eduard Bejček, Anna Vernerová & Marie Podobová. 2013. Corpus based identification of Czech light verbs. In Katarína Gajdošová & Adriána Žáková (eds.), *Proceedings of the 7th International Conference Slovko 2013; Natural Language Processing, Corpus Linguistics, E-learning*, 118–128. Lüdenscheid, Germany: RAM-Verlag.

Křen, Michal, Tomáš Bartoň, Václav Cvrček, Milena Hnátková, Tomáš Jelínek, Jan Kocek, Renata Novotná, Vladimír Petkevič, Pavel Procházka, Věra Schmiedtová & Hana Skoumalová. 2010. *SYN2010: Balanced corpus of written Czech*. LINDAT/CLARIN digital library at Institute of Formal and Applied Linguistics, Charles University in Prague.

Krippendorff, Klaus. 2007. *Computing krippendorff's alpha reliability*. Tech. rep. University of Pennsylvania, Annenberg School for Communication. http://repository.upenn.edu/asc_papers/43.

Li, Linlin & Caroline Sporleder. 2009. Classifier combination for contextual idiom detection without labelled data. In *Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing (EMNLP '09)*, 315–323. Singapore: Association for Computational Linguistics.
Liu, Changsheng & Rebecca Hwa. 2016. Phrasal substitution of idiomatic expressions. In *HLT-NAACL*, 363–373.

Madnani, Nitin & Bonnie J. Dorr. 2013. Generating targeted paraphrases for improved translation. In *ACM Transactions on Intelligent Systems and Technology* 4(3). 40:1–40:25.

Marton, Yuval, Chris Callison-Burch & Philip Resnik. 2009. Improved statistical machine translation using monolingually-derived paraphrases. In *Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing: Volume 1 - Volume 1* (EMNLP ’09), 381–390. Association for Computational Linguistics.

Mikolov, Tomas, Kai Chen, Greg Corrado & Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. In *Workshop Proceedings of the International Conference on Learning Representations (ICLR) 2013*. arXiv preprint arXiv:1301.3781.

Muzny, Grace & Luke S. Zettlemoyer. 2013. Automatic idiom identification in Wiktionary. In *Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing* (EMNLP 2013), 1417–1421. Association for Computational Linguistics.

Nunberg, Geoffrey, Ivan A. Sag & Thomas Wasow. 1994. Idioms. *Language* 70(3). 491–538.

Papineni, Kishore, Salim Roukos, Todd Ward & Wei-Jing Zhu. 2002. BLEU: A method for automatic evaluation of machine translation. In *Proceedings of the 40th annual meeting on association for computational linguistics (ACL ’02)*, 311–318. Stroudsburg, PA, USA: Association for Computational Linguistics.
DOI:10.3115/1073083.1073135

Peng, Jing, Anna Feldman & Hamza Jazmati. 2015. Classifying idiomatic and literal expressions using vector space representations. In *Proceedings of recent advances in natural language processing*, 507–511. Association for Computational Linguistics.

Pershina, Maria, Yifan He & Ralph Grishman. 2015. Idiom paraphrases: Seventh heaven vs cloud nine. In *Workshop on linking models of lexical, sentential and discourse-level semantics* (LSDSem), 76–82. Association for Computational Linguistics.

Radimský, Jan. 2010. *Verbonominální predikáty s kategoriálním slovesem*. České Budějovice: Editio Universitatis Bohemiae Meridionalis.

Řehůřek, Radim & Petr Sojka. 2010. Software framework for topic modelling with large corpora. In *Proceedings of the LREC 2010 Workshop on New Challenges for*
NLP Frameworks, 45–50. European Language Resources Association (ELRA). May 22, 2010.

Sag, Ivan A., Timothy Baldwin, Francis Bond, Ann A. Copestake & Dan Flickinger. 2002. Multiword expressions: A pain in the neck for NLP. In *Proceedings of the 3rd International Conference on Computational Linguistics and Intelligent Text Processing*, vol. 2276/2010 (CICLing ’02), 1–15. Springer-Verlag.

Salton, Giancarlo, Robert J. Ross & John D. Kelleher. 2014. An Empirical Study of the Impact of Idioms on Phrase Based Statistical Machine Translation of English to Brazilian-Portuguese. In *Proceedings of the 3rd Workshop on Hybrid Approaches to Machine Translation, HyTra@EACL 2014, April 27, 2014, Gothenburg, Sweden*, 36–41. http://aclweb.org/anthology/W/W14/W14-1007.pdf.

Sinha, R. Mahesh K. 2009. Mining complex predicates in Hindi using a parallel Hindi-English corpus. In *Proceedings of the Workshop on Multiword Expressions: Identification, Interpretation, Disambiguation and Applications* (MWE ’09), 40–46. Association for Computational Linguistics.

van der Linden, Erik-Jan. 1992. Incremental processing and the hierarchical lexicon. *Computational Linguistics* 18(2). 219–238.

Wallis, Peter. 1993. Information retrieval based on paraphrase. In *PACLING '93, 1st Pacific Association for Computational Linguistics Conference*(formerly JAJSNLP, the Japan-Australia Joint Symposia on Natural Language Processing).

Zarrieß, Sina & Jonas Kuhn. 2009. Exploiting translational correspondences for pattern-independent MWE identification. In *Proceedings of the Workshop on Multiword Expressions: Identification, Interpretation, Disambiguation and Applications* (MWE ’09). Association for Computational Linguistics.

Zhou, Liang, Chin-Yew Lin & Eduard Hovy. 2006. Re-evaluating Machine Translation results with paraphrase support. In *Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing (EMNLP '06)*, 77–84. Association for Computational Linguistics.

Žolkovskij, Alexander K. & Igor A. Mel'čuk. 1965. O vozmožnom metode i instrumentax semantičeskogo sinteza. *Naučno-texničeskaja informacija* 6. 23–28.
<figure><img src="image_12.png" /></figure>
Chapter 3

Identifying senses of particles in
verb-particle constructions

Archna Bhatia

Florida Institute for Human and Machine Cognition, Ocala, FL, USA

Choh Man Teng

Florida Institute for Human and Machine Cognition, Pensacola, FL, USA

James F. Allen

University of Rochester, Rochester, NY, USA

Florida Institute for Human and Machine Cognition, Pensacola, FL, USA

In order to attain broad coverage understanding, a system need not only identify multiword expressions such as verb-particle constructions (VPCs), but must compute their meaning. It is not plausible to hand enumerate all possible combinations, although WordNet is an admirable start. This chapter focuses on the identification of senses of particles in VPCs in order to compute the meanings of VPCs – using information obtained from existing lexical resources such as WordNet, and augmenting it with additional knowledge based on linguistic investigation of VPCs identified in terms of generalizations encoded in the TRIPS ontology. The approach consists of first determining compositionality of a VPC based on the information present in WordNet, and then assigning a relevant sense to the particle in a compositional VPC based on the sense classes we have identified and encoded in the TRIPS’ computational lexicon. Contributions of the described work are twofold: (1) A discussion of senses of particles in VPCs and corresponding generalizations makes a linguistic contribution. (2) We show how linguistic knowledge can be used to automatically parse sentences containing VPCs and obtain a semantic representation of them. An advantage of the described approach is that VPCs not explicitly found in lexica can be identified and semantically interpreted.

<figure><img src="image_13.png" /></figure>
# 1 Introduction

To compute deep semantic representations of sentences, we need to pay attention to the richness of lexical meaning. Multiword expressions (MWEs) constitute a significant proportion of the lexicon in any natural language (Moreno-Ortiz et al. 2013). In fact, Jackendoff (1997) estimated the number of MWEs in a speaker's lexicon to be of the same order of magnitude as the number of single words. Thus, it is important to get a good interpretation of MWEs.

This chapter builds on and extends the work reported in Bhatia et al. (2017) with focus on a specific type of MWEs, namely verb-particle constructions (VPCs). VPCs consist of a verb and an adverbial or prepositional particle, e.g., *eat up*, *fade out*, *go on*, *show off*, and *walk down*.<fnref n="1" /> Adding every single occurrence of such verb-particle combinations to a lexicon is not efficient nor ideal since knowledge about individual parts (verb and particle) can be leveraged for many of these VPCs as they are interpretable compositionally, e.g., *fly up*.

Other VPCs that indeed are noncompositional require special interpretation, and hence need to be added into the lexicon, e.g., *bring off* 'achieve a goal' and *egg on* 'urge someone for an action that might not be a good idea'. Our work on compositionality of VPCs, described in Bhatia et al. (2017) and developed further here, helps identify VPCs of each type for a proper treatment.

For an inventory of senses for verbs, many lexical resources, such as WordNet (Miller 1995; Fellbaum 1998) and the TRIPS lexicon (Allen & Teng 2017), are available that can be leveraged for interpreting compositional VPC types. In contrast, there is not much for particles except for a few attempts at the semantics of a few particles, such as *up* (Cook & Stevenson 2006) and *out* (Tyler & Evans 2003). However, particles seem to add their own semantics in compositional VPCs and are found to be regular when occurring with verbs in specific verb classes. For example, the particle *up* has a DIRECTION sense when it appears in resultative VPCs with verbs of motion, such as *wander/stroll/go/run up* (Villavicencio 2006). In this chapter, we provide a refined set of senses for particles in VPCs originally presented in Bhatia et al. (2017). We discuss and demonstrate how these senses are identified in compositional VPCs in order to compute meanings of sentences containing VPCs.

For computation of meaning, we use a broad coverage deep semantic parser, TRIPS (Allen et al. 2007), which combines semantic, ontological, and grammati-

<footnote n="1">¹Note that we focus on the particle usage here, not on the prepositional usage, i.e., a verb followed by a particle not a prepositional phrase. However, there may be an overlap in lexical semantic content (i.e., senses) of the homophonous particles and prepositions, see §4.1.</footnote>
cal information to produce semantic representations.<fnref n="2" /> We encode the semantics of particles mentioned above in the TRIPS ontology.<fnref n="3" /> The ontology encodes semantic types, the set of word senses and semantic relations that can be used in logical form (LF) graphs. Word senses are defined based on subcategorization patterns and selectional restrictions driven by linguistic considerations. The semantic types in the ontology are, to a large extent, compatible with FrameNet (Johnson & Fillmore 2000). The ontology uses a rich set of semantic features. Unlike WordNet, our ontology does not attempt to capture all possible word senses, but rather focuses on the level of abstraction that affects linguistic processing.

This chapter builds on the work described in Bhatia et al. (2017) in the following ways: Improvements have been made in the classification of compositionality and sense types as well as in the heuristics to automatically identify the types. The evaluation is made more robust with a larger test data set with full coverage of the heuristics. A more thorough analysis has led to the identification of generalizations regarding sense types.

The chapter is organized as follows: Previous work on VPCs is discussed in §2. A classification of VPCs based on their compositionality is discussed in §3 with its feasibility using inter-annotator agreement in §3.1. A set of heuristics to identify different classes of VPCs are presented in §3.2 and their evaluation in §3.3. In §4, we discuss the semantics of particles in VPCs. An inventory of general sense classes for particles used in VPCs is provided in §4.1, which is followed by a brief discussion of manual sense annotations as well as the use of compositionality heuristics for sense identification. In §5, we present various generalizations corresponding to the identified sense classes for the particles, and briefly discuss how a computational lexicon (including a lexicon for particles) is built for the computation of meaning for VPCs. Through examples, we demonstrate the general procedure to compute the meaning of sentences involving compositional VPCs and that the linguistic generalizations are reasonably helpful in the accurate identification of particle senses in VPCs. §6 concludes the chapter.

<footnote n="2">²For a more detailed overview of the TRIPS system, refer to Allen & Teng (2017) and Allen et al. (2008).
The TRIPS parser can be accessed at: http://trips.ihmc.us/parser/cgi/parse</footnote>

<footnote n="3">³The TRIPS ontology can be accessed at: http://www.cs.rochester.edu/research/cisd/projects/trips/lexicon/browse-ont-lex-jax.html</footnote>
## 2 Related work

A lot of computational literature on VPCs focuses on the identification or extraction of VPCs, or on the compositionality of VPCs, as discussed below. There are a few articles dealing with different senses of particles, but they usually focus on only one or two specific particles rather than on a broader coverage of particles. For example, Niinuma (2014) discusses grammaticalization of the particle *away* in English, specifying directional, completive, and continuing or iterative usages of the particle. Ishizaki (2010; 2012) also studies grammaticalization of the particle *away* together with the particle *out*, presenting a classification of VPCs into three categories of fully compositional, partially idiomatic, and fully (or highly) idiomatic.

Vincze et al. (2011) presents the Wiki50 corpus that has 446 VPCs (342 unique types) annotated. Bannard (2002) makes an attempt to identify different types of VPCs in terms of compositionality and builds a (decision tree) classifier to identify the four types. Bannard et al. (2003) also adopt a similar approach for compositionality. As an annotation experiment, they investigate various VPCs to see whether the sense is contributed by the verb and/or the particle. They build four classifiers for automatic semantic analysis of VPCs. Patrick & Fletcher (2004) also have a similar approach, but they focus on automatic classification of different types of compositionality. Unlike our work, in all these works, the focus is on compositionality only, not on the identification of actual senses of the particles.

Cook & Stevenson (2006) discuss various senses for the particle *up* in a cognitive grammar framework, annotate a dataset and perform some classification experiments to identify the senses of *up* in unseen data. As a linguistic study, Jackendoff (2002) provides a very nice discussion of various types of VPCs involving particles such as directional particles, aspectual particles, time-AWAY constructions, and some idiomatic constructions. Our work differs from theirs in having a broader coverage of particles and/or strong emphasis on ontology with respect to the sense classes of the particles and how different particle sense classes relate to verbal ontological classes.

Fraser (1976) mentions semantic properties of verbs affecting patterns of verb-particle combinations, for instance semantically similar verbs *bolt/cement/clam/glue/paste/nail* all can combine with the particle *down* and specify the objects that can be used to join material. Our approach is based on the similar assumption that there are generalizations, such as particles with certain sense classes combine with specific verb classes or ontological classes. Villavicencio (2003) also adopts
the same approach where she tries to encode the information in terms of lexical
rules and restrictions, etc. However, her focus is on obtaining productive patterns
in VPCs rather than on their interpretation.

Our work also differs from the previous work mentioned above in the follow-
ing respect: We emphasize the building of complete semantic representations of
the sentences, not just the particles’ semantics or just the classification of VPCs.
Similar to our criteria for compositionality, McCarthy et al. (2003), Baldwin et
al. (2003), and Bannard et al. (2003) have looked at distributional similarity as a
measure of compositionality of VPCs. In contrast to the approaches focusing on
statistical classification based on word/syntax features, our approach (both sym-
bolic and statistical) uses information obtained from existing lexical resources,
such as WordNet, for the classification of VPCs. We augment it with additional
knowledge based on the linguistic investigation of VPCs identified in terms of
generalizations, which we encode into an ontology, in order to compute the se-
mantics of the compositional classes.

3 Classification of VPCs

VPCs have often been classified in terms of their compositionality (i.e., whether
all constituents of a VPC, the verb and the particle, contribute their simplex mean-
ings to the overall semantic content of the VPC). The classes fall somewhere be-
tween fully compositional VPCs, e.g., *fly up*, and fully idiomatic VPCs, e.g., *egg
on*. For example, see Fraser (1976), Chen (1986), O’Dowd (1998), Dehé (2002), and
Jackendoff (2002).

We also classify VPCs into two types, compositional VPCs and noncomposi-
tional VPCs. The difference between the two classes is that the meaning of a
compositional VPC is the sum of the meanings of its parts (the verb and the
particle) whereas a noncompositional VPC has semantic content which is not
contributed by the individual constituents (i.e., the verb and the particle).

The compositional VPCs can be further classified into three subtypes: symmet-
rically compositional VPCs, light particle compositional VPCs (LP-compositional
VPCs), and light verb compositional VPCs (LV-compositional VPCs), based on
the type of semantic content contributed by each of the constituents. Symmetri-
cally compositional VPCs refer to VPCs in which both constituents, the verb and
the particle, contribute their simplex meanings (their lexical-semantic content).
For example, in *Debris flew up and hit the window in the furthest unit*, the senses
for the verb *fly* (e.g., in WordNet, sense fly%2:38:00) as well as the particle *up*
(e.g., in WordNet, sense up%4:02:00) combine together to provide the meaning
of the VPC *fly up*.<fnref n="4" /> We distinguish the other two compositional VPC types from the symmetrically compositional VPCs only in the aspect that in the other two types, the particle or the verb have a relatively lighter contribution than the other constituent which adds its regular lexical-semantic content.<fnref n="5" />

LP-compositional VPCs involve particles which, instead of contributing a preposition like lexical-semantic content, contribute aspectual information to the VPCs. Verbs contribute most of the lexical-semantic content in such VPCs. For example, in *Susan finished up her paper* (Bannard & Baldwin 2003), the verb *finish* contributes its regular lexical content (e.g., in WordNet, sense finish%2:30:02). However, the particle *up*, instead of contributing its regular lexical-semantic content, adds aspectual information that the action was completed (i.e., the COMPLEELY sense in our sense inventory). See §4.1 for the specific senses of particles.

LV-compositional VPCs involve light verbs which carry bleached meaning compared to regular verbs, e.g., CAUSE and BECOME. The particles contribute their regular lexical-semantic content to the VPCs's semantics. For example, in *The thief made away with the cash*, the particle *away* contributes its regular meaning (e.g., WordNet sense away%4:02:00), but the verb *make*, instead of contributing its regular meaning (e.g., WordNet sense make%2:36:01), adds a bleached meaning (e.g., cause to be). For details on the procedure to compute meanings of sentences with compositional VPCs, see §5.

In a noncompositional VPC, the sum of meanings of individual parts (the verb and the particle) may not completely account for the meaning of the VPC or may not account for it at all. Let's consider a few examples. In *They turned away hundreds of fans*, the VPC *turned away* is noncompositional despite the fact that the individual constituents' semantic content is reflected in the semantics of the VPC, since the VPC has additional content ('refuse entrance or membership') besides those contributed by the constituents. This additional content is not inferable from the individual parts and needs to be included in the lexical entry for the VPC. Another example of a noncompositional VPC is idiomatic usages. For example, the VPC *egged on* in *John wouldn't have done the dangerous experiment if his brother hadn't egged him on* involves idiosyncratic content that is not inferable from the parts and hence needs to be encoded in the lexicon. Some idiomatic usages may involve certain generalizations which may aid in interpre-

<footnote n="4">⁴For this study, we have used WordNet version 3.0. The numbers appearing after the symbol % in the WordNet senses represent the sense keys in WordNet.</footnote>

<footnote n="5">⁵The term “light particle” is used in analogy with the term “light verb” which is commonly used in the literature for verbs with bleached content.</footnote>
tation of the VPC. For example, two generalizations regarding the interpretation
of the noncompositional VPC **take up** are (i) it takes the sense 'starting to do
an activity' when it appears with activities as direct objects, e.g., *She took up
photography/swimming* [activities] and (ii) it takes the sense 'assume a responsi-
bility' when it appears with positions/responsibilities as direct objects, e.g., *She
took up her position* [responsibility/position].

For identification of the compositionality type for a given VPC, one may adopt
tests as in Figure 1.

<figure><img src="image_14.png" /><figcaption>Figure 1: Tests to identify compositionality type</figcaption></figure>

## 3.1 Human agreement on coarse-grained classification of VPCs

From among all the VPCs for which WordNet has an entry, we automatically
extracted 50 random VPCs such that four particles, namely *up*, *down*, *out*, and
*away*, were represented in the extracted VPCs. Since a VPC may have both com-
positional and noncompositional usages in different contexts (represented by dif-
ferent word senses or synsets in WordNet), we restricted the assignment of an-
notation label for a specific VPC to only one label by considering a single synset
from WordNet for annotations. Some of the WordNet synsets do not have an ex-
ample with the exact VPCs.<fnref n="6" /> We restricted the automatic extraction of VPCs to only those VPCs for which WordNet had a synset which included the VPC in its example. This synset was presented to the annotators together with the VPC to be annotated and the example usage.

These test VPCs were manually annotated by three annotators for compositionality labels. Fleiss’ kappa score (Fleiss 1971) was used to test inter-annotator reliability. The three sets of annotations achieved a score of 0.651, an intermediate-good score.<fnref n="7" /> We also created the gold annotations from the three sets of manual annotations. In cases of disagreement, the three annotators discussed reasons for their decisions and arrived at a consensus to create the gold annotations for the VPCs. The distribution of compositional vs. noncompositional VPCs in the automatically extracted 50 test VPCs was 60% vs. 40%. In terms of the specific compositionality types, the distribution was 30% for symmetrically compositional cases, 30% for LP-compositional, and 40% for noncompositional cases. Note that the 50 randomly extracted test VPCs did not have an instance of an LV-compositional VPC.

In the manual annotations, there were 37 VPCs out of 50 for which there was full agreement among all three annotators on the coarse-grained labels for compositionality type. Since there were only two coarse-grained labels (compositional and noncompositional), for the rest of the 13 VPCs, there was agreement between two out of three annotators for the compositionality label. The annotators disagreed more on the compositional cases (76.92% of the disagreements were on the compositional VPCs) than on the noncompositional ones (23.08%).

In terms of VPCs with certain particles, VPCs with the particle *down* were found to be the most challenging for the annotators. For 42.86% of the usages of VPCs with *down* (3 out of 7 usages) among the 50 test cases, the annotators did not agree on the annotation label. On the other hand, VPCs with the particle *up* were found to be the least challenging. For only 14.29% of the cases (3 out of 21 usages), the annotators disagreed on the compositionality type. This may be due to the fact that VPCs with the particle *up* have a higher frequency than VPCs with other particles (Villavicencio 2006), such as *down*, and hence users have relatively better intuitions about VPCs with the particle *up* compared to VPCs with the particle *down*.

<footnote n="6">⁶It may have examples which use related verbs or related VPCs instead, or it may not have an example at all. For example, the WordNet entry *turn_away%2:38:02* for the VPC *turn away* does not have any example with the VPC itself.</footnote>

<footnote n="7">⁷NLTK’s agreement package was used to calculate the Fleiss’ kappa score which showed intermediate-good agreement. Cohen’s kappa score (Cohen 1960) was also calculated using the same package showing substantial agreement, also with a score of 0.651.</footnote>
3.2 Heuristics for compositionality of VPCs

As a first step toward an interpretation of VPCs, we need to determine whether a given VPC is compositional or not. To perform this task automatically, we employ a number of heuristics that make use of the rich inventory of hierarchically organized word senses (i.e., synsets) in WordNet which contains over 100,000 words including 64,188 multiwords. Heuristics 1–6 below are used to identify compositional VPCs, whereas heuristic 7 indicates non-compositionality.

1. If the verb is among the list of light verbs, and WordNet does not have an entry for the VPC, it is LV-compositional. For example, the VPC *make away* uses the light verb *make* and the VPC does not have an entry in WordNet.

2. Given a VPC, if heuristic 1 does not apply and WordNet has an entry for the verb as well as for the particle, but no entry for the VPC, VPC is compositional (LP-compositional or symmetrically compositional). For example, *fly* with the sense key fly%2:38:01 as well as *up* with the sense key up%4:02:00 appears in WordNet, but *fly up* does not appear in any synset in WordNet.

3. If WordNet has the VPC as well as the verb in the same synset, VPC is LP-compositional. For example, the VPC *sort out* (*sort_out%2:31:00*) and the verb *sort* (*sort%2:31:00*) both appear in the same synset in WordNet.

4. If WordNet has the verb in the VPC as a hypernym for the VPC, VPC is either symmetrically compositional or LP-compositional. For example, compositional VPC *push up* (*push_up%2:38:00*) has the verb *push* (*push%2:38:00*) as its direct hypernym.

5. If WordNet has the verb in the definition (in its base or inflected form) of the synset where the VPC appears, the VPC is either symmetrically compositional or LP-compositional. For example, the compositional VPC *move up* (*move_up%2:38:00*) has the verb *move* in its definition *move upwards*.

6. If WordNet has the relevant VPC as well as another VPC with the verb replaced with another verb in the same synset, the VPC is compositional (either symmetrically compositional or LP-compositional or LV-compositional). For example, the compositional VPCs *pull out* (*pull_out%2:35:00*) and *rip out* (*rip_out%2:35:00*) appear in the same WordNet synset.
7. If none of the above heuristics apply, the VPC is noncompositional. For example, none of the above heuristics apply to the idiomatic VPC *catch up* (*catch_up%2:38:0*).

## 3.3 An evaluation of the heuristics for compositionality of VPCs

For an evaluation of the heuristics, we used two test sets: (i) Test Set 1: the test set consisting of the same 50 randomly extracted VPCs that were used to calculate inter-annotator agreement scores mentioned in §3.1, and (ii) Test Set 2: a test set consisting of 653 VPCs created using the VPCs on the first page of each of the four English wiktionary entries with the title "Category: English phrasal verbs with particle" for particles *up*, *out*, *down*, and *away* respectively.

A Python implementation of the heuristics was applied to the test VPCs in both test sets to assign them a compositionality label. Heuristics 1–6 identify compositional VPCs, whereas heuristic 7 identifies noncompositional VPCs. Note that together these heuristics have full coverage, i.e., a prediction is made for each of the VPCs in the test sets. Heuristics 1 and 2 apply to VPCs for which WordNet does not have an entry. Heuristics 3-6 apply to VPCs for which WordNet has an entry. Heuristic 7 applies when none of the heuristics 1-6 apply.

Since Test Set 1 consisted of VPCs that were randomly extracted from WordNet, only heuristics 3–7 could be evaluated on Test Set 1. Heuristics 1 and 2 could not be evaluated using this test set since these two heuristics apply to VPCs that are not included in WordNet. In order to test heuristics 1 and 2, we used Test Set 2 which has VPCs that may or may not be included in WordNet.

Test Set 1 has manually created gold annotations. Hence, the labels assigned by the heuristics were tested against them for the VPCs in Test Set 1. For Test Set 2, on the other hand, we examined the heuristics-assigned labels corresponding to heuristics 1 and 2 manually.

An evaluation of the heuristics on Test Set 1 is presented in Table 1. Heuristics 3, 4, and 5 (used to identify compositional VPCs) performed perfectly in identifying compositionality of VPCs whereas heuristic 6 (also intended to identify compositional VPCs) did not have as high precision. Two out of the three cases where heuristic 6 made an incorrect prediction, however, involved noncompositional VPCs for which the individual constituents' meaning was also reflected in the semantics of the VPCs even though they also had additional content that was not inferable based on the constituents alone. The fact that the constituents' semantics was reflected in the VPCs' semantics was what heuristic 6 had captured. Heuristic 7 (the only heuristic used to identify noncompositional VPCs)
also had a lower but better than chance performance in comparison to the other heuristics.

<table><caption>Table 1: Evaluation of heuristics 3–7 using Test Set 1 (50 test cases)ª</caption><thead><tr><th>Heuristic #</th><th>Precision for the heuristic (%)</th><th>Coverage of the heuristic (%)</th></tr></thead><tbody><tr><td>3</td><td>100</td><td>16</td></tr><tr><td>4</td><td>100</td><td>6</td></tr><tr><td>5</td><td>100</td><td>12</td></tr><tr><td>6</td><td>76.92</td><td>26</td></tr><tr><td>7</td><td>58.62</td><td>58</td></tr><tr><td>Overall (heuristics 3–7)</td><td>70</td><td>100</td></tr></tbody></table>

Overall, heuristics 3–7 had a precision of 70% in assigning a label (compositional or noncompositional) to a VPC. Whenever multiple heuristics applied to a VPC (14% of the test cases), they were always correct in their prediction. This suggests that presence of multiple characteristics of compositionality (represented by the heuristics) may be a reliable indicator of compositionality. In 86% of the cases, a single heuristic identified the VPC as representative of its own category (compositional or noncompositional). Regarding the relatively lower performance of heuristic 7, since noncompositional VPCs carry additional information not contributed by individual constituents (common sense knowledge or idiosyncratic information), it may not be observable and hence is not captured using a heuristic as easily or directly.

More investigation is required into heuristics 6 and 7 to see how they can be modified to make them better usable for identification of the compositionality type of a VPC. For example, a future step may be to examine if certain generalizations exist in terms of verb types and particle types that can help us further determine when the cases identified by these heuristics are compositional or noncompositional.

As a result of the examination of the heuristics' output, we noticed a few more indicators which could also be incorporated into the heuristics to improve their performance in the future. For example, if the word *completely* or *thoroughly*

<footnote>ªWe define precision as Cn/Tn and coverage as Tn/N, where N is the corpus size, Tn is the sample size that heuristic n is applicable to, and Cn is the number of correct assignments it makes.</footnote>
appears in the definition of a synset of a VPC, the particle in the VPC may carry
the aspectual sense COMPLETELY. Hence, it could be labelled as a compositional
VPC (specifically, as an LP-compositional VPC).

As mentioned above, for an evaluation of the heuristics 1 and 2, we used Test
Set 2 that consisted of 653 VPCs, a subset of the VPCs mentioned in the English
wiktionary. The Python implementation of the heuristics mentioned above was
used to assign compositionality labels to the VPCs in Test Set 2. Out of the 653
VPCs, heuristic 1 applied to only 2 VPCs (0.3% of the test items) and heuristic 2
applied to 280 VPCs (42.88% of the test items). Heuristic 7 applied to 9 other VPCs
that did not have a WordNet entry. Overall, 44.56% of the test cases did not have
an entry in WordNet. Heuristic 2 covered 96.22% of these cases with reasonable
precision (80%, evaluated on 15 randomly selected test VPCs to which heuristic
2 applied).

Next, we move on to the semantics of particles in VPCs. We will focus mostly
on the compositional VPCs for the rest of this chapter.

4 Semantics of particles in VPCs

As mentioned in §3, particles contribute to the overall semantics of compositional VPCs. In order to study the contribution of particles in VPCs, in our prior work (Bhatia et al. 2017), we conducted an investigation of VPCs consisting of verbs in the ontology class ONT::EVENT-OF-CAUSATION in the TRIPS ontology. This class consisted of 1383 words with verb senses (and a total of 1784 verb senses of those words). Our investigation consisted of combinations of these verbs with the following particles (wherever the combinations appeared as VPCs): *across*, *away*, *by*, *down*, *in*, *into*, *off*, *on*, *out*, *over*, *through*, and *up*.<fnref n="8" /> We searched for examples for each of the combinations using Google and manually went through each of the examples to check for a number of properties. For example, we checked if any of the verb or the particle contributed to the overall meaning of the VPC. We identified the senses particles had in the VPCs if any. We checked: (i) if the particle could be taken out without a major change in the meaning, (ii) if the particle expressed RESULT or could be replaced with a RESULT-Prepositional Phrase,<fnref n="9" /> (iii) if a corresponding VPC consisting of the particle with the opposite

<footnote n="8">⁸While we do find prepositional phrasal verb constructions with *into*, we did not find any VPCs involving intransitive usages of *into*.</footnote>

<footnote n="9">⁹RESULT is one of the argument roles identified in the TRIPS ontology. The argument roles signal different argument positions for predicates as well as have their own inferential import, some other examples are AGENT, AFFECTED, MANNER, LOCATION, and FIGURE.</footnote>