<figure><img src="image_25.png" /><figcaption>Figure 5: C-structure for example (28)</figcaption></figure>

<figure><img src="image_26.png" /><figcaption>Figure 6: F-structure for example (28)</figcaption></figure>
In the c-structure the particle PRT is a separate constituent which may also
occur after the NP under VPmain. In the f-structure the verb and the particle are
analyzed as forming one predicate skrive*opp, and the particle also provides a
value to the feature PRT-FORM.

As in the case of selected prepositions, the lexical entry for skrive is associated
with the relevant frame through an invocation of the template describing this class of constructions. Part of the lexical entry for skrive is shown in (29).

(29) skrive V { [...] | @ (V-SUBJ-PRT-OBJ skrive opp) | [...] }

Part of the invoked template V-SUBJ-PRT-OBJ is shown in (30).

(30) V-SUBJ-PRT-OBJ (P prt) =
@(CONCAT P '* prt %FN)
(↑ PRED) = '%FN((↑ SUBJ)(↑ OBJ))'
(↑ CHECK PRT-VERB) = +
(↑ PRT-FORM) = c prt

The CONCAT template functions as in the template (27), yielding the predicate name skrive*opp as the value of PRED. The second last line assigns the value “+” to the path CHECK PRT-VERB, a feature which is checked by the syntactic rule introducing the particle PRT; see the VPmain rule in (43) below. The last line is a constraining equation⁶ which checks that the value of the feature PRT-FORM, which is introduced in the sentence by the particle, is the value of prt, i.e. *opp* in the template invocation in (29).

3.3.3 Verb-particle constructions with selected prepositions

The preceding sections have shown how prepositional verbs and verb-particle
constructions are analyzed. Phrasal verbs also allow both selected prepositions
and particles in the same MWE. An example involving both, in addition to a
reflexive object, is provided in the treebank example in (31). The analysis of (31)
is shown in Figures 7 and 8.

(31) Vi har et så enormt stort område å **sette oss inn i**.
we have a such enormously large area to set us in into
‘We have such an enormously large area to immerse ourselves in.’
<figure><img src="image_27.png" /><figcaption>Figure 7: The c-structure of sentence (31)</figcaption></figure>

<figure><img src="image_28.png" /><figcaption>Figure 8: Part of the simplified f-structure of sentence (31)</figcaption></figure>
In this example the MWE *sette oss inn i* occurs in an infinitival relative (CPinf) in an NP with the head *område* ‘area’. In the f-structure the infinitival relative occurs as a member of the set of adjuncts to the predicate *område*, also occurring as the second argument of *sette*\*seg\*inn\*i as its relativized argument (see Section 4.2 for the analysis of long-distance dependencies like relativization and topicalization). The template invoked by the verb *sette*, V-SUBJ-OBJrefl-PRT-POBJ, provides an analysis along the lines of the templates in (27) and (30).

## 3.4 Verbal idioms

A VP idiom is a flexible MWE in which at least one predicate-bearing lexeme (such as a noun or an adjective) is selected, with possible restrictions as to number, definiteness or other morphological properties applying. VP idioms are handled by a specific set of templates. For example, an idiom like *holde øye med* ‘keep an eye on’ is analyzed by means of a lexical template covering idioms consisting of a selected indefinite object plus a selected prepositional phrase. The treebank sentence in (32) is analyzed as shown in Figures 9 and 10.

(32) Samtidig **holdt** han umerkelig et skarpt **øye med** jentungen. simultaneously kept he unnoticeably a sharp eye with the girl child ‘At the same time he furtively kept a close eye on the girl.’

The analysis of the selected prepositional phrase *med jentungen* is as described in Section 3.3.1 for example (25). The selected lexeme in (32) is *øye*. In the f-structure in Figure 10 the idiomatic meaning is represented by incorporating *øye* in the predicate name, deriving the predicate name *holde#øye*\*med. The phrase *et skarpt øye* fills the function of OBJ, but is not analyzed as a semantic argument of the sentence predicate, which appears from its position outside the angled brackets (<...>) surrounding the argument list. This position signals that the constituent is syntactically subcategorized for without being a semantic argument.

The lexical entry for *holde* is associated with the VP idiom through an invocation of the idiom template describing the relevant class of idioms. Part of the lexical entry for *holde* is shown in (33). The template invocation has three parameters, the predicate name for the verb *holde*, the predicate of the selected noun *øye*, and the form of the selected preposition *med*. Part of the template is shown in (34); the full template is discussed in Section 4.3.

<footnote n="6">⁶This concept is explained in connection with example (20) above.</footnote>
<figure><img src="image_29.png" /><figcaption>Figure 9: The c-structure of sentence (32)</figcaption></figure>

<figure><img src="image_30.png" /><figcaption>Figure 10: The simplified f-structure of sentence (32)</figcaption></figure>
(33) holde V { [...] | @ (VPIDIOM-INDEFOBJ-POBJ holde øye med) (↑ OBJ NUM) = c sg | [...] }

(34) VPIDIOM-INDEFOBJ-POBJ (P OP prp) = @ (CONCAT P ‘# OP ‘* prp %FN) (↑ PRED) = ‘%FN ( (↑ SUBJ) (↑ OBL-TH) ) (↑ OBJ) (↑ OBL-TH CHECK P-SELFORM) = prp (↑ OBJ PRED FN) = c OP ~ (↑ OBJ DEF) = +

In addition to the template call, the lexical entry in (33) also specifies that the
selected object should be singular. It is a matter of choice whether such informa-
tion should be included in the individual lexical entry or give rise to a distinc-
tion between more fine-grained templates. In the template definition in (34), OP
(object predicate) is the variable for the selected noun predicate and prp for the
selected preposition. As in the case of the template in (30), the template invokes
the CONCAT template which builds the predicate name. The second last equation
requires the object to have the value of OP as its predicate (in this case ‘øye’),
and the final equation requires the object not to be definite. As for the equation
mentioning P-SELFORM, see the explanation of the template in (27).

3.5 Nonverbal flexible expressions

3.5.1 Nouns with selected prepositions

Nouns may also form MWEs by selecting prepositional phrases as their argu-
ments. For example, the noun *ansvar* ‘responsibility’ may select the preposition
*for* ‘for’, which can take a nominal phrase, an infinitival, or a nominal subclause
as complement, as in the treebank examples in (35–37).

(35) Hadde jeg **ansvar** **for** gutten?
    had I responsibility for the boy
    ‘Did I have responsibility for the boy?’

(36) Han fikk **ansvar** **for** å overta søket.
he got responsibility for to take over the search
‘He got the responsibility for taking over the search.’
(37) Jeg kan ikke ta **ansvar** for at det ble lekket.
I can not take responsibility for that it became leaked
'I cannot take responsibility for its having been leaked.'

<figure><img src="image_31.png" /><figcaption>Figure 11: C- and f-structure for the sentence (35)</figcaption></figure>

Example (35) is analyzed as in Figure 11. As in the case of prepositional verbs, the selected preposition does not contribute a PRED of its own, but is analyzed as forming a single predicate ansvar*for with the noun, taking the complement of the preposition as an argument with the function OBL-TH (an oblique argument expressing a theta role). With an infinitival or a clausal complement the syntactic function is COMP. The lexical entry for *ansvar* in (38) invokes three alternative templates for the three possible kinds of complements, in addition to its basic template as a mass noun.

(38) ansvar N { @*(MASSNOUN ansvar)
| @*(N-POBJ ansvar for)
| @*(N-PINFCOMP ansvar for)
| @*(N-PCOMP ansvar for) }
### 3.5.2 Adjectives with selected prepositions

Similarly, adjectives may select prepositional phrases as complements, for instance *flink til* 'clever at', as in the treebank example in (39).

(39) Hva er egentlig du flink til?
what are after all you clever at
‘What are you clever at, after all?’

<figure><img src="image_32.png" /><figcaption>Figure 12: C- and f-structure for the sentence (39)</figcaption></figure>

Example (39) is analyzed as in Figure 12. In this example the complement of the selected preposition has been questioned and occurs in the f-structure as the value of FOCUS-INT, i.e., interrogative focus. The predicative complement (PREDLINK) has the predicate *flink*\*til, taking the prepositional complement as its OBL-TH. The value of OBL-TH is identical with the value of FOCUS-INT, which is indicated by the shared index 8, resulting from the general analysis of wh-questions in the grammar.

## 4 Representing flexibility

Flexible MWEs must be recognizable across different types of syntactic modifications which separate their parts from each other in the sentence. Such modi-
fications include the simple occurrence of other words between the MWE parts, long-distance dependencies like topicalization, relativization and wh-question formation, presentative constructions, and various types of passive constructions. When flexible MWEs are treated by means of LFG templates, such modifications are automatically taken care of within the regular grammar. Having both a c-structure and an f-structure representation allows us to capture both the close semantic and functional association between the selecting and the selected words (in the f-structure) and their syntactic independence as different constituents (in the c-structure). We will present the analyses of some cases.

## 4.1 Intervening words

The simplest case of syntactic modification of an MWE is when other words, typically adverbs, occur between the MWE components. In a verb-second language like Norwegian the sentence subject also frequently breaks up a verb phrase MWE. The treebank example in (40) illustrates the recognition of the predicate trekke*seg*tilbake ('withdraw', literally: 'draw oneself back') across several intervening words. The analysis is shown in Figures 13 and 14.

(40) Da **trekker** jeg **meg** bare stille og rolig **tilbake**.
then draw I me only silently and calmly back

‘Then I simply withdraw silently and calmly.’

<figure><img src="image_33.png" /><figcaption>Figure 13: C-structure of sentence (40)</figcaption></figure>
<figure><img src="image_34.png" /><figcaption>Figure 14: Simplified f-structure of sentence (40)</figcaption></figure>

The mechanism for achieving this lies in the projection architecture of LFG, in which different constituents in c-structure may project the same f-structure, within which dependencies may be formulated. To illustrate we may consider the relevant fragments of the c-structure rules for I', S and VPmain in (41–43).

(41) I' → Vfin: ↑ = ↓
(S: ↑ = ↓)

(42) S → (PRONP: (↑ SUBJ) = ↓
@SUBJCASE)
[...]
(PRONrfl: { (↑ OBJ-BEN) = ↓
| (↑ OBJ) = ↓ })
[...]
(ADVPs+: ↓ ∈ (↑ ADJUNCT))
[...]
(APsmpl: ↓ ∈ (↑ ADJUNCT))
[...]
(VPmain: ↑ = ↓)
[...]
[...]

(43) VPmain → [...] (PRTP: ↑ = ↓ (↑ CHECK PRT-VERB) = c +) [...]
As explained in Section 2, the equation $↑ = ↓$ annotated to a rule daughter means that the rule daughter and its mother will project the same f-structure. Thus it can be seen that the Vfin daughter of I' (the verb) and the PRTP daughter of VPmain (the particle) will project the same f-structure.

A particle verb presupposes the presence of the required particle in the sentence, and a particle presupposes the presence of a particle verb. This mutual dependency is captured through two features, one feature PRT-VERB = +, carried by the verb and required by the rule introducing the particle, and, conversely, one feature PRT-FORM, carried by the particle and required by the verb to have the appropriate value. Thus, the constraint equation annotated to PRTP, ($↑$ CHECK PRT-VERB) = c +, demanding that its f-structure should have a feature PRT-VERB = + (i.e., that the verb should be a particle verb), will be satisfied if the finite verb has contributed such a feature to this common f-structure. A similar constraint equation associated with the verb, (($↑$ PRT-FORM) = c prt in (45) below), ensures that the particle has the form required by the verb.

The lexical entry for *trekke* is associated with the relevant frame through an invocation of the template for reflexive verb-particle constructions. Part of the lexical entry for *trekke* is shown in (44). The template has the form shown in (45).

(44) trekke V { [...] | @ (V-SUBJ-OBJrefl-PRT trekke tilbake) | [...] }

(45) V-SUBJ-OBJrefl-PRT (P prt) =
@ (CONCAT P * seg * prt %FN)
{ ($↑$ PRED) = '%FN<$↑$ SUBJ$>$($↑$ OBJ-BEN)' | ($↑$ PRED) = '%FN<$↑$ OBJ$>$($↑$ OBJ-BEN)($↑$ SUBJ)' | ($↑$ PRESENTATIVE) = + | ($↑$ SUBJ PRON-TYPE) = c expl | ~($↑$ OBJ DEF) = + }
@ (REFLEXIVE OBJ-BEN)
($↑$ CHECK PRT-VRB) = + | ($↑$ PRT-FORM) = c prt | ~($↑$ PASSIVE) = +

The template CONCAT constructs the predicate name trekke*seg*tilbake as the value of PRED. The reflexive occurring with reflexive verbs is analyzed as OBJ-BEN (indirect object). The reason for this is that there will be a direct object in the alternative presentative construction with an expletive *det* subject, such as
Det trekker seg tilbake store styrker 'There are big forces withdrawing', in which store styrker occurs as a syntactic object (OBJ); there can only be one OBJ. The presentative construction is described as the second alternative in the disjunction {...|...} in the template. The reflexive, like the expletive subject, is analyzed as a non-argument, which appears from the fact that it is placed outside the argument list enclosed by <...> in the value of PRED. The features PRT-VRB and PRT-FORM are explained in the discussion of the template in (30).

## 4.2 Long-distance dependencies

Long-distance dependencies involve syntactic dependencies across an arbitrary number of clause boundaries and comprise topicalization by fronting, relative clauses and wh-questions. Such dependencies are handled in the f-structure by means of a special type of equations using regular expressions to specify a set of alternative attribute paths into the f-structure. The term for this mechanism is FUNCTIONAL UNCERTAINTY. The rule in (46) shows a simplified version of the functional uncertainty equation handling the dependency between the topic and some embedded gap further down in the structure.

$$ (46) \quad \text{IP} \rightarrow \text{XP:} \quad \begin{array}{l} (\uparrow \text{TOPIC}) = \downarrow \\ (\uparrow \{\text{COMP} | \text{XCOMP}\} * \{\text{SUBJ} | \text{OBJ} | \text{OBL-TH}\}) = \downarrow \\ \text{[...]} \end{array} $$

The first equation annotated to XP in (46) specifies that the f-structure of the XP daughter (↓) is the value of the attribute TOPIC of the f-structure of the IP mother (↑). The second equation specifies that the daughter f-structure is also the value of one of a set of alternative attribute paths. COMP and XCOMP are the attributes of embedded finite and non-finite clauses. The regular expression {COMP | XCOMP} * describes all possible strings over the elements COMP and XCOMP (with repetitions), and the final disjunction specifies the last attribute of the string, enabling the TOPIC to be identical with an embedded SUBJ, OBJ or OBL-TH. We may illustrate with the treebank example in (47) of a prepositional verb fortelle om 'tell about' in a sentence where the OBL-TH, i.e., the selected prepositional phrase, has been topicalized.

(47) **Om** dette skal jeg **fortelle** nå.
about this shall I tell now
'This I will now tell about.'
<figure><img src="image_35.png" /><figcaption>Figure 15: C- and f-structure for example (47)</figcaption></figure>

The analysis of (47) is shown in Figure 15. In the f-structure the value of TOPIC, indexed 6, is also found as the value of OBL-TH in the embedded XCOMP with fortelle*om as predicate. Thus the attribute string from the set specified by the functional uncertainty equation in (46) for this example is (↑ XCOMP OBL-TH).

## 4.3 Passive alternations

Passive is another source of verbal MWE modifications, changing the syntactic functions of selected constituents. In LFG passive is analyzed as a lexical phenomenon modifying the value of PRED in a lexical entry for a verb, changing the mapping between argument positions and syntactic functions. In NorGram this is handled by passive templates invoked by the verb templates. The full version of the VP idiom template in (34) for idioms like holde øye med 'keep an eye on' is shown in (48), where different types of passive alternations are handled.

$$ (48) \quad \begin{aligned} \text{VPIDIOM-INDEFOBJ-POBJ} (\text{P OP prp}) = & \\ & \text{@}(\text{CONCAT P '# OP '* prp %FN}) \\ & \{ \text{@(PASS-OBL-TH} [(\uparrow \text{PRED}) = '%FN<((\uparrow \text{SUBJ})(\uparrow \text{OBL-TH}))>(\uparrow \text{OBJ})] \\ & | \{ (\uparrow \text{PRED}) = '%FN<NULL((\uparrow \text{OBL-TH}))>(\uparrow \text{SUBJ})(\uparrow \text{OBJ})' \end{aligned} $$
| (↑ PRED) = '%FN(↑ OBL-AG)(↑ OBL-TH)/(↑ SUBJ)(↑ OBJ)' } 
(↑ PASSIVE) = c + 
(↑ PRESENTATIVE-TYPE) = passive 
(↑ SUBJ PRON-TYPE) = c expl } 
(↑ OBL-TH CHECK P-SELFORM) = prp 
(↑ OBJ PRED FN) = c OP 
~(↑ OBJ DEF) = +

After the second line there follows a disjunction of two alternatives. The first alternative invokes the template PASS-OBL-TH, taking the predicate-argument structure as a parameter. This template allows the active/passive alternation whereby the OBL-TH, i.e., the complement of the selected preposition (see the discussion of example 25), may be the subject in a passive construction, as in the treebank example in (49).

(49) De var derimot ikke klar over at de ble holdt øye they were on the other hand not clear over that they became held eye med.
with

'On the other hand, they weren't aware that someone was keeping an eye on them.'

The second alternative in the main disjunction describes the impersonal (pre-
sentative) passive option with an expletive subject, as in the example in (50).

(50) Det ble holdt øye med dem.
it became held eye with them
'Someone was keeping an eye on them.'

The embedded disjunction of two predicate-argument structures in the fourth and fifth lines of the template describes the possibility of including an OBL-AG, i.e., an oblique agent in a prepositional phrase with *av* 'by'. The remaining equations require the passive form of the verb and expletive type of the subject pronoun.

5 Complementation patterns

Verbal MWEs in Norwegian show considerable variation in terms of subcatego-
rizational properties. Like simple verbs, MWEs can have transitivity shifts, take
different types of arguments, and take different combinations of arguments. The
verb-particle construction *si opp*, for instance, has both an intransitive reading, as in (51), and a transitive reading, as in (52) and (53). While the shift in transitivity does not significantly affect the semantics of the expression in (52), the shift in (53) leads to a change in meaning.

(51) 150 befal **sier opp**.
150 officers say up
‘150 officers resign.’

(52) Hun **sa opp** jobben.
she said up the job
‘She resigned from her job.’

(53) Man må **si opp** sjefen for Statkraft.
one must say up the boss for Statkraft
‘The head of Statkraft must be fired.’

More precisely, the theme object that is implicit in the intransitive usage in (51) is explicit in (52), while in (53) the object has the semantic role of experimenter instead of theme. The frames V-SUBJ-PRT and V-SUBJ-PRT-OBJ represent the intransitive and the transitive usages of *si opp*. NorGram, being mainly a syntactic framework, has one frame for both transitive readings, leaving semantic roles underspecified.

Most of the verbal MWEs in NorGram are phrasal verbs or VP idioms. Such MWEs have free subjects, so that any argument variation is in the complements.<fnref n="7" /> The lexical entries display a wide range of complementation patterns, one type being MWEs where the verb selects all of its complements. Table 2 presents types of VP idioms in NorGram with only selected complements. In idioms where the verb subcategorizes for only one selected complement, the selected element is either a nominal (O), a prepositional ([P + O]) or a predicative (PC) complement. There is also a type of VP idiom with two selected complements (O + PRT).

Most verbal MWEs in NorGram have free complements in addition to their selected complements. In the VP idiom *legge merke til* ‘notice’, the verb *legge* ‘lay’ selects the object *merke* ‘mark’ in the indefinite form and a prepositional complement which is either nominal, as in (54), clausal, as in (55), or an interrogative clausal complement, as in (56), all headed by the selected preposition *til* ‘to’.

<footnote n="7">⁷The exception to free subjects in VP idioms is expressions with the expletive subject *det* ‘it’. However, this type of argument variation is analyzed as a grammatical rather than a lexical selection of the subject and is thus not considered here.</footnote>
<table><caption>Table 2: Verbal MWEs with only selected complements</caption><thead><tr><th>Pattern</th><th>Example</th><th>Lit. translation</th><th>Id. translation</th></tr></thead><tbody><tr><td rowspan="4">V + O</td><td>slå følge</td><td>'beat company'</td><td>'accompany'</td></tr><tr><td>slå leir</td><td>'beat camp'</td><td>'camp'</td></tr><tr><td>ta feil</td><td>'take wrong'</td><td>'be wrong'</td></tr><tr><td>ta fyr</td><td>'take fire'</td><td>'catch fire'</td></tr><tr><td rowspan="4">V + [P + O]</td><td>gå i oppløsning</td><td>'go in dissolution'</td><td>'dissolve'</td></tr><tr><td>komme for en dag</td><td>'come for a day'</td><td>'be revealed'</td></tr><tr><td>løfte i flokk</td><td>'lift in flock'</td><td>'join forces'</td></tr><tr><td>legge på svøm</td><td>'lay on swim'</td><td>'start swimming'</td></tr><tr><td>V + PC</td><td>stå brud</td><td>'stand bride'</td><td>'get married'</td></tr><tr><td>V + O + PRT</td><td>sette livet til</td><td>'put the life to'</td><td>'lose one's life'</td></tr></tbody></table>

(54) Ingen legger merke til mannen som står urørlig og venter. no one lays mark to the man who stands motionless and waits 'No one notices the man who is standing waiting motionlessly.'

(55) Ingen legger merke til at mannen står urørlig og venter. no one lays mark to that the man stands motionless and waits 'No one notices that the man is standing waiting motionlessly.'

(56) Ingen legger merke til om mannen står urørlig og venter. no one lays mark to if the man stands motionless and waits 'No one notices whether the man is standing waiting motionlessly.'

MWEs that subcategorize for different types of complements are represented in the lexicon with one frame for each subcategorization pattern. In the template invocations in (57), POBJ, PCOMP and PCOMPint represent the different types of prepositional complements that occur with legge merke til in (54), (55), and (56), respectively.

(57) a. @<sub>(VPIDIOM-INDEFOBJ-POBJ legge merke til)</sub>

b. @<sub>(VPIDIOM-INDEFOBJ-PCOMP legge merke til)</sub>

c. @<sub>(VPIDIOM-INDEFOBJ-PCOMPint legge merke til)</sub>
While Table 2 shows different types of selected complements, examples (54–56) illustrate how one MWE may take different types of free complements. In both cases, we see that variation in the complementation is limited for individual MWEs. While *slå følge* ‘accompany’, *gå i oppløsning* ‘dissolve’ and the other examples in Table 2 all have fixed complement structures, *legge merke til* ‘notice’ has three different frames in which only one of the complements varies. To give an impression of the variety of complementation patterns in the lexicon it is thus necessary to turn to the inventory of unique frames, reflected in the number of templates. For instance, NorGram has more than 80 templates for phrasal verbs; these may be grouped into seven main classes according to the types and number of complements (Table 3).

<table><caption>Table 3: Main types of complementation patterns in phrasal verbs in NorGram</caption><thead><tr><td>Type</td><td>Example frame</td><td>Example MWE</td></tr></thead><tbody><tr><td>V + PRT</td><td>V-SUBJ-PRT</td><td>stryke med</td></tr><tr><td>V + PRT + 1 complement</td><td>V-SUBJ-PRT-XCOMP</td><td>få til</td></tr><tr><td>V + PRT + 2 complements</td><td>V-SUBJ-PRT-OBJ-OBJ</td><td>gjøre etter</td></tr><tr><td>V + PPsel</td><td>V-SUBJ-POBJ</td><td>advare mot</td></tr><tr><td>V + PPsel + 1 complement</td><td>V-SUBJ-OBJ-PACOMP</td><td>erklære for</td></tr><tr><td>V + PPsel + 2 complements</td><td>V-SUBJ-OBJ-POBJ-PCOMP</td><td>vedde med på</td></tr><tr><td>V + PRT + PPsel</td><td>V-SUBJ-PRT-POBJ</td><td>gå med på</td></tr><tr><td>V + PRT + PPsel + 1 complement</td><td>V-SUBJ-PRT-OBJ-POBJ</td><td>venne av med</td></tr></tbody></table>

Table 3 presents the different types of complementation patterns for verb-particle constructions, prepositional verbs, and verb-particle constructions with selected prepositions. The first column in the table is the pattern type, represented in terms of the main complement(s), which may be a particle (PRT), a selected prepositional phrase (PPsel), or both, and the number of additional complements.<fnref n="8" /> Examples of subcategorization frames for each type are given in the second column using template names. The example MWEs, represented in the table with only their fixed components, are instances of the example frames and are discussed in more detail in (58–70).

<footnote n="8">⁸"Main complement" in this context refers to the selected complement which determines the type of the overall construction, such as PRT in verb-particle constructions.</footnote>
As Table 3 shows, verb-particle constructions in NorGram may either be in-
transitive, such as *stryke med* ‘die’ in (58), or have one or two free complements,
such as *få noe til* ‘accomplish something’ in (59) and *gjøre noen noe etter* ‘repeat
something after someone’ in (60).

(58) Og vi fortsetter å banke deg til du **stryker med!**
and we continue to beat you until you stroke with
‘And we will continue to beat you until you’re dead!’

(59) Nå **fikk** han **til** å tenke igjen.
now got he to to think again
‘Now he managed to think again.’

(60) Ikke mange kunne ha **gjort** ham noe **slikt** etter!
not many could have done him something like that after
‘Not many people could have done what he did!’

The example *få noe til* in (59) is an instantiation of the frame V-SUBJ-PRT-XCOMP, with one free complement in the form of the infinitival complement *å tenke igjen* ‘to think again’. There is one other frame for this particular MWE in the lexicon, with a nominal object instead of the infinitival complement (V-SUBJ-PRT-OBJ). Example (61) illustrates this complement structure.

(61) Dette er hva du **fikk til**.
this is what you got to
‘This is what you accomplished.’

The lexicon also has a frame for *få til* with two free complements, in the form
of an object and an infinitival complement. The difference in the number of com-
plements also yields a difference in meaning, as shown in (62). These should thus
be considered different MWEs.

(62) Hvorfor **får** vi ikke dem **til** å bli?
why get we not them to to stay
‘Why can’t we make them stay?’

The last type of verb-particle construction in Table 3, with two free complements in addition to the particle, is exemplified with the frame V-SUBJ-PRT-OBJ-OBJ. This argument structure, illustrated in (60) for *gjøre noen noe etter*, involves
both an indirect object (OBJ-BEN), *ham* 'him', and a direct object (OBJ), *noe slikt* 'something like that' (OBJ-BEN is shortened to OBJ in the name of the template). A second frame of this type, which is slightly more complex with a nominal object and a clausal complement (COMP) as well as an expletive subject, is V-SUBJexpl-PRT-OBJ-COMP. The MWE *det faller noen noe inn* 'something occurs to someone' in (63) is an example of this frame, literally translating into 'it falls someone something in'. Except for the expletive subject and the particle, the frame has the same arguments as V-SUBJ-OBJ-COMP for single verbs such as *forklare* 'explain'. The frame is thus regular in terms of argument structure.

(63) Det *falt* ikke britene **inn** at særlig mange hadde lyst.
It fell not the Brits in that particularly many had desire
'It did not occur to the Brits that more than a few should want to.'

In contrast to verb-particle constructions which may be intransitive, preposi-
sitional verbs will always have a free complement, introduced by the selected
preposition. Prepositional verbs can subcategorize for exactly one prepositional
phrase, as in *advare mot noe* 'warn against something' in (64), where *mot seg-
regering* 'against segregation' is a PPsel.

(64) Han **advarer mot** segregering.
he warns against segregation
'He warns against segregation.'

Similar to verb-particle constructions, the prepositional verbs in NorGram can
take one or two complements in addition to the selected complement. In (65),
*erklære noen for noe* 'declare someone something' has one complement, the free
object *marken* 'the mark', in addition to the selected prepositional phrase *for død*
'for dead'.

(65) Der måtte myndighetene **erklære** marken **for** død.
there had to the government declare the mark for dead
‘There the authorities had to declare the (German) mark dead.’

The relevant frame in (65) is V-SUBJ-OBJ-PACOMP, where PACOMP is the se-
lected prepositional phrase. In this case, the preposition *for* takes the adjecti-
val predicative complement *død* ‘dead’. While PPsel is the c-structure category
for constituents headed by selected prepositions and may refer to any type of
prepositional complement, PACOMP is a syntactic variable that reflects the type
of complement.
The final type of prepositional verb in Table 3 is illustrated in (66) with the MWE vedde noe med noen på noe 'bet something with someone on something'.

(66) Abrams veddet en sigarett med Browne på at det regnet. Abrams bet a cigarette with Brown on that it rained 'Abrams bets a cigarette with Brown that it was raining.'

This example is an instance of the frame V-SUBJ-OBJ-POBJ-PCOMP, which has two complements in addition to a PPsel, in this case a free object and a second PPsel. The free object is en sigarett 'a cigarette'. In the first PPsel, which corresponds to POBJ in the subcategorization frame, the selected preposition med 'with' takes the nominal object Brown. In the second PPsel, corresponding to PCOMP, the preposition på 'on' takes the clausal complement at det regnet 'that it was raining'.

Like prepositional verbs, verb-particle constructions with selected prepositions always subcategorize for at least one free complement. Such constructions can have one complement, as in gå med på noe 'go along with something' in (67), which is an instance of the frame V-SUBJ-PRT-POBJ.

(67) I land som Sverige gikk fagbevegelsen med på de nye tankene in countries like Sweden went the unions with on the new thoughts 'In countries like Sweden the unions went along with the new ideas.'

In (67), the particle is med 'with', and the free argument de nye tankene 'the new thoughts' is the complement of the selected preposition på 'on'. The prepositional complement could, however, also be clausal, as in (68), or infinitival, as in (69). With the infinitival complement, there is a shift in meaning from 'go along with'/'admit' to 'agree'.

(68) Han vil ikke gå med på at hun er utpreget modig. he will not go with on that she is exceptionally brave 'He will not admit that she is exceptionally brave.'

(69) Til Libbys forbauselse hadde Jerry gått med på å prøve. to Libby's surprise had Jerry gone with on to try 'To Libby's surprise, Jerry had agreed to try.'

Verb-particle constructions with selected prepositions may also have two free complements. This is the case for venne noen av med noe 'wean someone off something' in (70). This example, instantiating the frame V-SUBJ-PRT-OBJ-POBJ, has
the particle *av* ‘off’, the free pronominal object *meg* ‘me’, and the selected prepo-
sitional object *med det* ‘with that’. Also here, the prepositional complement may
vary. The alternative frame is V-SUBJ-PRT-OBJ-PXCOMP, allowing an infinitival
prepositional complement, but in this case yielding no difference in meaning.

(70) Mor klarte aldri å venne meg av med det.
mother managed never to accustom me off with that
‘Mother never managed to wean me off that habit.’

The examples of complementation patterns for phrasal verbs in NorGram show that the subcategorizational properties of MWEs can be the source of variation both at the syntactic and the semantic levels. We have seen that the main types of complementation patterns in Table 3 are shared by a number of subcategorization frames. Table 4 presents some of the frames that are variants of the type V + PPsel + 1 complement in Table 3 (prepositional verbs with one free complement). The frames are divided into groups of MWEs that share the same or similar types of arguments, resulting in five categories of argument patterning for this type.<fnref n="9" /> While the current section provides only superficial observations about the types of MWE argument patterns in the NorGram lexicon, it seems that a more systematic study of their subcategorizational properties could provide useful information about MWE types and tokens and perhaps also new insights into the relationship between argument patterns and the semantics of MWEs.

6 Conclusion

In this chapter we have shown how the modularization of NorGram makes it
possible to integrate MWEs into the LFG analyses in a way that does justice to
the proper division of labor between the lexicon and the grammar. On the one
hand, each MWE is entered into the lexicon with the information necessary for
its idiomatic meaning. On the other hand, the syntactic treatment uses ordinary
syntactic rules to the extent that the flexibility of the individual MWE allows.

Up until now MWEs have been severely underrepresented in lexical resources
for Norwegian, as they have been for many other languages. The main strat-
egy for NorGram has been to incorporate them into the lexicon and grammar
when they are encountered during the construction of NorGramBank. MWEs
have thus been added to NorGram in tandem with the development of the tree-
bank. As a natural consequence of the way in which the MWEs are represented

<footnote n="9">⁹Several frames of this type are not listed here, including frames with expletive subjects and objects and subtypes of clausal complements.</footnote>
<table><caption>Table 4: Some variants of V + PPsel + 1 complement</caption><thead><tr><th>Complementation type</th><th>Subcategorization frame</th></tr></thead><tbody><tr><td rowspan="10">Free object<br>and PPsel</td><td>V-SUBJ-OBJ-PACOMP</td></tr><tr><td>V-SUBJ-OBJ-POBJACOMP</td></tr><tr><td>V-SUBJ-OBJ-POBJNCOMP</td></tr><tr><td>V-SUBJ-OBJ-POBJ</td></tr><tr><td>V-SUBJ-OBJ-PCOMP</td></tr><tr><td>V-SUBJ-OBJ-PCOMPinf</td></tr><tr><td>V-SUBJ-OBJ-PCOMPint</td></tr><tr><td>V-SUBJ-OBJ-PXCOMP</td></tr><tr><td>V-SUBJ-INDOBJ-POBJ</td></tr><tr><td>V-SUBJexpl-OBJ-POBJ</td></tr><tr><td rowspan="5">Reflexive object<br>and PPsel</td><td>V-SUBJ-OBJrefl-POBJ</td></tr><tr><td>V-SUBJ-OBJrefl-PCOMP</td></tr><tr><td>V-SUBJ-OBJrefl-PCOMPat</td></tr><tr><td>V-SUBJ-OBJrefl-PCOMPint</td></tr><tr><td>V-SUBJ-OBJrefl-PXCOMP</td></tr><tr><td rowspan="4">PPsel<br>and free<br>nominal<br>complement</td><td>V-SUBJ-POBJ-COMP</td></tr><tr><td>V-SUBJ-POBJ-XCOMP</td></tr><tr><td>V-SUBJ-POBJ-OBL</td></tr><tr><td>V-SUBJ-POBJ-OBLBEN</td></tr><tr><td rowspan="2">Prepositional reflexive object<br>and free nominal complement</td><td>V-SUBJ-POBJrefl-OBJ</td></tr><tr><td>V-SUBJ-POBJrefl-COMP</td></tr><tr><td rowspan="2">PPsel<br>and PPsel</td><td>V-SUBJ-POBJ-PXCOMP</td></tr><tr><td>V-SUBJ-POBJrefl-POBJ</td></tr></tbody></table>
in the grammar and lexicon, it is possible to search for the various MWE types in the treebank. The wealth of information provided by the LFG representations enables searching for many different properties of the MWEs, and the MWEs may be recovered in all the syntactic variations they occur in. As a result, NorGram-Bank is now an important resource for studying Norwegian MWEs in context.

## Acknowledgments

We thank Koenraad De Smedt and two anonymous reviewers for valuable comments and suggestions for improvements. This work was partially financed by the Norwegian Research Council and the University of Bergen through the INESS project.

## Abbreviations

=c symbol in a constraint equation: constrained to be equal to

[P + O] selected prepositional complement

<...>

* element in the name of the predicate of a lexeme with a selected semantically light element (e.g., a preposition)

\# element in the name of the predicate of a lexeme with a selected semantically heavy element (e.g., a noun, forming an idiom)

%FN variable over predicate names in a lexical template

→ phrase structure rule expansion

↑ variable in an equation, referring to the f-structure of the node immediately dominating the node to which the equation is annotated

↓ variable in an equation, referring to the f-structure of the node to which the equation is annotated

~ negation in an equation defining f-structure

INESS Infrastructure for the exploration of syntax and semantics

LFG Lexical-Functional Grammar

XLE the development platform Xerox Linguistic Environment

## References

Baldwin, Timothy & Su Nam Kim. 2010. Multiword expressions. In Nitin Indurkhya & Fred J. Damerau (eds.), *Handbook of natural language processing*, Second edition, chap. 12, 267–292. Boca Raton, FL, USA: Chapman & Hall/CRC.
Bresnan, Joan. 2001. *Lexical-functional syntax*. Malden, MA: Blackwell.

Butt, Miriam, Helge Dyvik, Tracy Holloway King, Hiroshi Masuichi & Christian Rohrer. 2002. The parallel grammar project. In John Carroll, Nelleke Oostdijk & Richard Sutcliffe (eds.), *Proceedings of the workshop on Grammar Engineering and Evaluation at the 19th international conference on Computational Linguistics (COLING)*, 1–7. Taipei, Taiwan: Association for Computational Linguistics.

Dalrymple, Mary. 2001. *Lexical functional grammar*. Vol. 34 (Syntax and Semantics). San Diego, CA: Academic Press.

Dalrymple, Mary, Ronald M. Kaplan & Tracy Holloway King. 2004. Linguistic generalizations over descriptions. In Miriam Butt & Tracy H. King (eds.), *Proceedings of the LFG’04 conference, 200–208*. University of Canterbury, Christchurch, New Zealand: CSLI Publications.

Dyvik, Helge. 2000. Nødvendige noder i norsk: Grunntrekk i en leksikalsk-funksjonell beskrivelse av norsk syntaks [Necessary nodes in Norwegian: Basic properties of a lexical-functional description of Norwegian syntax]. In Øivin Andersen, Kjersti Fløttum & Torodd Kinn (eds.), *Menneske, språk og felleskap*, 25–45. Oslo: Novus forlag.

Dyvik, Helge, Paul Meurer, Victoria Rosén, Koenraad De Smedt, Petter Haugereid, Gyri Smørdal Losnegaard, Gunn Inger Lyse & Martha Thunes. 2016. NorGramBank: A ‘deep’ treebank for Norwegian. In Nicoletta Calzolari, Khalid Choukri, Thierry Declerck, Marko Grobelnik, Bente Maegaard, Joseph Mariani, Asunción Moreno, Jan Odijk & Stelios Piperidis (eds.), *Proceedings of the tenth international conference on Language Resources and Evaluation (LREC 2016)*, 3555–3562. Portorož, Slovenia. http://www.lrec-conf.org/proceedings/lrec2016/summaries/943.html.

Kaplan, Ronald M. & Joan Bresnan. 1982. Lexical-Functional Grammar: A formal system for grammatical representation. In Joan Bresnan (ed.), *The mental representation of grammatical relations*, 173–281. Cambridge, MA: The MIT Press.

Lønning, Jan Tore, Stephan Oepen, Dorothee Beermann, Lars Hellan, John Carroll, Helge Dyvik, Dan Flickinger, Janne Bondi Johannessen, Paul Meurer, Torbjørn Nordgård, Victoria Rosén & Erik Velldal. 2004. LOGON: A Norwegian MT effort. In *Proceedings of the workshop in Recent Advances in Scandinavian Machine Translation*. Uppsala, Sweden.

Losnegaard, Gyri Smørdal, Gunn Inger Lyse, Martha Thunes, Victoria Rosén, Koenraad De Smedt, Helge Dyvik & Paul Meurer. 2012. What we have learned from Sofie: Extending lexical and grammatical coverage in an LFG parsebank. In Jan Hajič, Koenraad De Smedt, Marko Tadić & António Branco (eds.), *META-RESEARCH workshop on advanced treebanking at LREC’2012*, 69–76. Istanbul, Turkey.
Maxwell, John & Ronald M. Kaplan. 1993. The interface between phrasal and functional constraints. *Computational Linguistics* 19(4). 571–589.

Meurer, Paul. 2012. INESS-Search: A search system for LFG (and other) treebanks. In Miriam Butt & Tracy Holloway King (eds.), *Proceedings of the LFG'12 conference* (LFG Online Proceedings), 404–421. Stanford, CA: CSLI Publications.

Nordgård, Torbjørn. 2000. Nordkompleks: A Norwegian computational lexicon. In *COMLEX 2000 workshop on Computational Lexicography and Multimedia Dictionaries*, 89–92. Patras, Greece: University of Patras.

Nunberg, Geoffrey, Ivan Sag & Thomas Wasow. 1994. Idioms. *Language* 70(3). 491–538.

Rosén, Victoria, Koenraad De Smedt, Paul Meurer & Helge Dyvik. 2012. An open infrastructure for advanced treebanking. In Jan Hajič, Koenraad De Smedt, Marko Tadić & António Branco (eds.), *META-RESEARCH workshop on advanced treebanking at LREC'2012*, 22–29. Istanbul, Turkey.

Rosén, Victoria, Paul Meurer & Koenraad De Smedt. 2007. Designing and implementing discriminants for LFG grammars. In Tracy Holloway King & Miriam Butt (eds.), *The proceedings of the LFG'07 conference*, 397–417. Stanford: CSLI Publications.

Rosén, Victoria, Martha Thunes, Petter Haugereid, Gyri Smørdal Losnegaard, Helge Dyvik, Paul Meurer, Gunn Inger Lyse & Koenraad De Smedt. 2016. The enrichment of lexical resources through incremental parsebanking. *Language Resources and Evaluation* 50(2). 291–319.

Sag, Ivan, Timothy Baldwin, Francis Bond, Ann Copestake & Dan Flickinger. 2002. Multiword expressions: A pain in the neck for NLP. In *Proceedings of the 3rd international conference on Computational Linguistics and Intelligent Text Processing* (Lecture Notes in Computer Science 2276), 1–15. Springer.
# Chapter 4

## Issues in parsing MWEs in an LFG/XLE framework

### Stella Markantonatou

Institute for Language and Speech Processing/Athena RIC

### Niki Samaridi

Institute for Language and Speech Processing/Athena RIC

### Panagiotis Minos

Institute for Language and Speech Processing/Athena RIC

We present an LFG/XLE system coupled with an independent lexicographic environment for encoding and parsing Modern Greek MWEs. The system assigns a flat structure to the fixed sequences of words within MWEs, the so-called “words with spaces” (WWSs) with the help of a preprocessing module that receives the morphologically analysed string from a tagger external to XLE. We describe the overall system and discuss certain implications of the designing choices.

## 1 Introduction

This paper presents the system for parsing Modern Greek (MG) Multiword Expressions (MWEs) with LFG/XLE grammars that is schematically depicted in Figure 1 and discusses the issues encountered with the LFG/XLE representations. The main idea of the adopted parsing strategy is that the parser treats the sequential fixed parts of the MWEs as a type of “words with spaces” (WWS) (Sag et al. 2002). Our WWSs are fixed sequences of fixed words that may contain one word that declines (for instance, see example 7 in Table 1). The rigid word order is an important criterion of fixedness in the case of MG that has a relatively free word order. Morphological fixedness is also important in a language with rich

<figure><img src="image_36.png" /></figure>
morphology but, exactly for the same reason, the existence of an inflected word within an otherwise rigid structure is not a surprise. The usage of (this type of) WWSs has practical and theoretical implications.

WWSs have been used by Copestake et al. (2002), by Attia (2006) for parsing Arabic MWEs with LFG grammars, by Korkontzelos & Manandhar (2010) for shallow parsing and was recently shown to be beneficial for a transition-based dependency grammar parser of Modern Greek (Apidianaki et al. 2018). We have adopted the WWS approach in an effort to move as much as possible of the parsing burden from the LFG/XLE component to an external MWE recognizer (the “filter” from now on). At the same time, we have tried to allow for natural LFG analyses. The system depicted in Figure 1 consists of:

1. The ILSP FBT Tagger

2. IDION: A lexicographic tool that allows for formal descriptions of the MWEs

3. The filter

4. The XLE/LFG grammars

<figure><img src="image_37.png" /><figcaption>Figure 1: The overall structure of the parsing system</figcaption></figure>

The ILSP FBT Tagger and IDION are independent pieces of NLP software; they are compatible with the “core” parsing system that consists of the filter and the grammars (Samaridi & Markantonatou 2014). In what follows, we describe the parts 1–4 in separate sections in this order. We will use (1) as a working example. (1) is a verb MWE that contains a fixed NP *mavra matia* ‘black eyes’ and an obligatory sentential complement that is controlled by the MWE subject. The subject is free and fully agrees with the verb of the MWE (MG is a pro-drop language therefore in (1) no explicit subject is present):
(3) perno [tous dromous]
take the roads.ACC
'I wander'

(4) a. leo [to psomi psomaki]
call the bread.ACC little-bread.ACC
'to starve'

b. to psomi psomaki leo (emphatic)

c. * to psomi leo psomaki (emphatic)

d. * psomaki leo to psomi (emphatic)

In addition, the identification of the syntactic function of the fixed parts of verb MWEs is not straightforward in LFG. This is so because the governable grammatical functions (GFs) of LFG<fnref n="1" /> are defined on the basis of particular semantic and syntactic properties (Dalrymple 2001). Alas it is very often the case that the fixed parts of MG MWEs are not characterized by these particular properties. And still, one cannot avoid using a large choice of grammatical functions to model MG MWE phenomena because the language allows for some word order flexibility within verbal MWEs (4a,b) and often there are control (1) and binding phenomena (5) that have to be accounted for. LFG models these phenomena on the f-structure with the use of syntactic functions. (In (5) the WWS *to ksilo tis chronias tis* 'the beating the.GEN year.GEN hers' can be thought to have a noun head *ksilo* 'beating'; the structure contains a possessive pronoun that is bound by the free subject of the MWE.)

(5) I Maria efage [to ksilo tis chronias tis / *tou]
the Maria.FEM ate the beating the year.GEN hers / *his
'Maria has been beaten up.'

The OBJ function makes a good example of a GF that does not fit well to the MWE data. The WWS *tous_dromous* 'the roads' in (3) is a fixed simple NP; one would be tempted to assign the OBJ function to it but, on the other hand, the fixed NP never turns up as the subject of a passive form although the verb *perno* 'take' passivises. Furthermore, the WWS in (3) presents an idiosyncratic behavior with clitics; normally it cannot be replaced by a clitic, while this is absolutely possible in a compositional structure; the fixed NP can be replaced only in a very

<footnote n="1">¹The governable GFs of LFG are: SUBJ, OBJ, OBJ2, POSS, COMP, XCOMP.</footnote>
restricted context, namely when the same MWE precedes the structure with the clitic (Markantonatou & Samaridi 2018) producing an ironic or emphatic effect. Passivisation is a defining property of the OBJ GF in LFG (Dalrymple 2001) and free replacement by a clitic is definitely a defining property of objects in MG. On the other hand, the WWS in (4) behaves just as the WWS in (3) with respect to passivisation and cliticisation and all the other flexibility diagnostics; evidence mandates that the two WWSs are assigned the same GF and the question is whether they should be assigned the OBJ GF or some other GF. It is possible that the idea that MWEs use exactly the syntax employed in the analysis of compositional structures (Gross 1988a,b; Kay & Sag 2012; Bargmann & Sailer 2018) could be imported in LFG and the classical GFs could be assigned to fixed constituents along with a tree-like structure and constraints on inflection, passivisation, modifiability, cliticisation and linear precedence that do the job (Waszczuk & Savary 2015). The problem with the “compositional structure” approach is that it questions the notion of syntactic functions and the generalizations expressed with them: for instance, the OBJs of MG MWEs will be peculiar in that they hardly passivise and they are not replaced by clitics freely unless they occur in highly constrained contexts.

The system we present here uses the classical LFG GFs. This means that *zachari* 'sugar' in (2) is treated as a noun and the phrasal projection is assigned the OBL(ique) GF; on the same par, the bracketed strings in (3), (4) and (5) are assigned the PoS "No"(un) and project NPs that are assigned the OBJ GF. So far we have not used a set of GFs different from the one established in the literature because linear precedence phenomena in the fixed parts are captured with the use of WWSs and modifiability and cliticisation seem to require a more careful modeling than simply allowing or prohibiting them: cliticisation heavily depends on the context and modifiability seems to be rather restricted in MG. A concrete, corpus-based, analysis of both the phenomena has not been made available yet, to the best of our knowledge. This set-up demands that passivisation is blocked with a feature (and not with the absence of an OBJ GF as it would be the case if some other GF was used in the place of the OBJ GF). Of course, a similar blocking feature would be used in the grammar anyway for several non-passivisable transitive verbs of MG MWEs; this fact definitely emphasizes the problematic situation with the OBJ GF and passivisation. In a nutshell, we have used the OBJ GF not because it served our purposes well but because the in-depth exploration of the alternatives is considered a future challenge.

In the remainder of this document we will present and discuss the parts of the system as they are depicted in Figure 1.
## 3 ILSP FBT Tagger

The mature ILSP FBT Tagger (Papageorgiou et al. 2000) is an adaptation of the Brill tagger trained on MG text. It uses a PAROLE compatible tagset (Bilgram & Keson 1998) of 584 different tags that capture the morphological particularities of MG. The tagger works on the output of a sentence detection and tokenisation tool and assigns both a lemma and a set of tags corresponding to an exhaustive morphological analysis of each token. Figure 2 shows the output of the ILSP FBT Tagger for (1). We decided to use the ILSP FBT Tagger because the effort to develop an XFST morphological component is a project on its own. In the set-up of Figure 1, the tagger is a black box that allows for no identification of the fixed parts of MWEs at the level of morphological analysis, as it would be possible if, for instance, the XFST/XLE component was used as in Attia (2006). For this reason, the morphologically analysed output of the ILSP tagger that offers information only about tokens, is processed with a filter (Samaridi & Markantonatou 2014) that scans the output of the tagger for strings containing MWEs and feeds a script ("formatter") that transforms the output to a format readable by an LFG/XLE grammar; the filter informs the XLE parser whether an MWE exists, whether it contains any WWSs - if so, the WWSs are marked on the output string that feeds the parser - and whether the input string can receive both a compositional and a MWE interpretation.

## 4 IDION

The XLE parser receives lexical knowledge on MWEs from IDION<fnref n="2" />, an open source lexicographic environment for MWEs that is addressed both to the human user and to NLP applications and encodes, among others, morphosyntactic properties of MWEs in a, as much as possible, theory-neutral formalism. IDION is connected to the parsing system with an application that transcribes the IDION formalism to the XLE formalism (Minos et al. 2016). As opposed to other MWE DBs, such as DUELME (Grégoire 2010), that use a simplified formal language for encoding morphological features, IDION exhaustively describes morphological features with the ILSP-PAROLE compatible tagset that is also used by the ILSP FBT Tagger.

It is important to note that syntactic functions are assigned to phrasal constituents in Modern Greek (and not to parts of a word); therefore, diagnostics for constituent identification are also required along with diagnostics for the

<footnote n="2">²http://idion.ilsp.gr/</footnote>
<figure><img src="image_38.png" /><figcaption>Figure 2: The output of the ILSP FBT tagger for the verb MWE in (1)</figcaption></figure>

identification of WWSs. In IDION the following diagnostics are used for these
purposes (Markantonatou & Samaridi 2018): possible word order permutations,
the ability of XPs (modifiers included) to intervene between two words thus pos-
sibly indicating the border between two constituents, passivisability, clitic re-
placement, wh-questioning and causative-inchoative alternations. Grammatical
functions are identified with diagnostics that apply to compositional expressions
such as morphological marking and wh-questions (in MG subjects are always in
the nominative case and objects almost always in the accusative case, verbs agree
with their subjects and objects can be replaced by clitics).

The IDION encoding of the MWE structure corresponds to a rather flat tree
and does not make use of powerful expressive means, such as inheritance, that
in the literature have been combined with tree-based formalisms (Pollard & Sag
1987; Crabbé et al. 2013). The reason for choosing a perhaps redundant but rather
simple encoding is that we aim at ensuring IDION’s reusability. For this purpose,
we try to make sure that we use expressive means that are shared by or can be
easily transcribed to many formalisms and that the encoding does not rely on
implicit assumptions concerning the overall grammar of the language.<fnref n="3" /> To this
end, the IDION representation of verbal MWEs defines the following nodes: (i)

<footnote n="3">³For instance, in MG possession is expressed with the sequence “DET noun Possessive”. In IDION the whole sequence is encoded as fixed rather than encoding only the noun as fixed.</footnote>
the root category (default) (ii) the phrasal categories shown in (6) that are used to denote free nominal constituents of the MWE (iii) leaf nodes (words). Phrasal categories and words are directly linked to the root category. IDION only indexes the fixed contiguous parts of an MWE (the WWSs of our implementation) and does not assign them a phrasal structure.

(6) NP-NOM/NP-NOM-anim/NP-NOM-nonanim;
NP-GEN/NP-GEN-anim/NP-GEN-nonanim;
NP-ACC/NP-ACC-anim/NP-ACC-nonanim;

The Java-based transcription application provides for the remaining phrasal
categories needed for an LFG representation that requires the definition of con-
stituents and typically involves trees deeper than the ones defined in IDION. All
in all, IDION only specifies the phrasal categories shown in (6) and it is on the
transcription applications to specify the categories that are necessary for any
given formalism.

The IDION encoding of the MWE in (1) is given in Figure 3. On the first col-
umn it is specified whether the annotated part of the MWE is a phrasal category
(phrasal categories are shown in 6) or a word and whether it is optional or not (for
instance, the MWE of example (1) that is depicted in Figure 3 has only obligatory
parts). Words are encoded as lemmas and only complementisers are encoded as
such (in Figure 3, the depicted MWE contains a complementiser). On the second
column, the lemmas of the parts of the expression are listed, namely the verb
head *kano* ‘make’, the lemmatized parts of the WWS *mavros mati* ‘black eye’, the
complementizer *na* ‘to’ that always introduces a sentential complement and the
lemma form of the irregular verb head *vlepo* ‘see’ of the sentential complement.
On the third column are encoded the actual form of the WWS and the control
facts; in the case depicted in Figure 3, the sentential complement is controlled by
the NP-NOM-anim. The fourth column provides the full morphological analysis
of the fixed or semi-fixed parts of the MWE, for instance it is specified that the
head verb of the controlled sentential complement is always in the active voice
and in a form denoting perfect aspect; person and number of the controlled verb
are not specified as they are determined by the free subject of the MWE. On the
last column the parts of the WWS are indexed.

We developed a Java transcription application that generates XLE entries from
the IDION specifications.

The LFG/XLE entries listed below are developed out of the IDION representa-
tion of (1) shown in Figure 3. As a first step, the transcription application gener-
ates lexical entries for the WWSs that are indexed in the IDION representation of
<figure><img src="image_39.png" /><figcaption>Figure 3: The IDION encoding of the MWE in (1)</figcaption></figure>

the MWE; if one or more WWSs have been indexed in the IDION representation of the MWE, a corresponding number of XLE entries are produced and stored in the XLE lexicon. Morphological information about the entries, here the WWS and the verb head of the controlled sentential complement, is received from the annotation encoded on the fourth column. Next, the application generates the entry for the head verb of the MWE as follows: the NP-NOM-anim slot in the first column shows that the verb selects a free subject NP, the WWS that contains a noun and an adjective both in the accusative case shows that the head verb selects a fixed object and finally, the existence of a COMPL(ementiser) slot in the first column coupled with the control information on the third column shows that the head verb subcategorises for an XCOMP controlled by the subject of the main verb. This information generates the entry of the head verb *kano*. Finally, the head verb of the sentential complement is retrieved from the second column as it immediately follows COMPL. The application knows that the verb *vlepo* is transitive because it has a controlled subject and it is followed by an NP-ACC.

The WWS in MWE (1): mavra\_matia, NoCmPlAc

The verb head of MWE (1): kano<SUBJ,OBJ,XCOMP>
↑ OBJ PRED = mavra matia
↑ XCOMP PRED = vlepo<SUBJ,OBJ>
↑ XCOMP PRED FINITE = +
↑ XCOMP SUBJ = ↑SUBJ
# 5 The filter

The filter consists of two parts: the filter lexicon and the filtering part proper.

## 5.1 The filter lexicon

The filter consults the filter lexicon where each MWE entry is specified for the following:

1. **Compositionality:** Certain MWEs can take a compositional interpretation. For instance, the free subject verbal MWE in (1) has no compositional interpretation while the semi-fixed MWE in (7) can also take the compositional interpretation 'I grab them.'

(7) tis arpazo
them.FEM grab.1SG
'I am beaten up.'

2. The "signifier": the lemma of the substring of an MWE that instructs the filter to look at the appropriate filter lexicon entries. For the MWE in (1), the signifier is the lemma *kano* 'make, do'. If the expression is fixed as in (8) the symbol "~" is used as a signifier. (8) has no translation, it is a kind of swearing (often accompanied with an offensive gesture) meaning that someone has made a serious mistake or is totally idiot:

(8) pare pente
take.2SG.IMP five

3. The lemmatised form of "words with spaces" (WWSs) whether they are independent fixed MWEs as in (8) or substrings of an MWE as in (1). In the case of (8) the lemmatised WWS would be *perno pente* 'take five'. In the case of (1) the fixed part is *mavra matia* 'black eyes' and the corresponding lemmatised form is *mavros mati* 'black eye'.

4. PoS and morphological constraints on the parts of the WWS. For the fixed part of (1) *mavra matia* the constraints would be: *mavros*: adjective, plural, accusative, basic; *mati*: noun, common, plural, accusative.
## 5.2 The filtering part

The filter proper, implemented in Perl, reads the tagged sentence from an XML file (the output of the tagger) and stores it. Then, it checks whether a signifier exists and,

A1. If no signifier is found, the string is copied as it is on the formatter.

A2. If a signifier is found, the filter lexicon is scanned for some WWS entry. The filter checks whether the morphological constraints on the filter lexicon entries (headword and remaining words) match the lemma and the tags on the input string and:

B1. If they do not match, the input string is copied as it is on the formatter.

B2. If they match, the filter consults the filter lexicon whether the MWE can take a compositional reading and,

C1. if it can, it sends to the formatter the input string and goes to step C2

C2. if it cannot, the part of the string that has been recognized is replaced with the corresponding WWS and morphological constraints and the resulting new string is sent to the formatter.

# 6 The LFG analysis (implemented with XLE grammars)

The output of the formatter is processed with an LFG grammar of Modern Greek with sub-lexical rules that can parse the output of the tagger. The grammar runs on XLE, a parsing environment dedicated to writing, running and debugging LFG grammars.<fnref n="4" /> The trees generated by the sub-lexical rules can be seen in the c-structure of Figure 5.

Modern Greek verbal MWEs are rich in syntactic structure despite any simplifications that might result from the usage of WWSs. In Section 2 we discussed why we have adopted an LFG analysis that applies the classical LFG Grammatical Functions on MWEs despite the obvious problems. Thus, so far we have manipulated the lexicon by introducing the idiomatic lexical entries but we have not manipulated the grammar rules.

<footnote n="4">⁴XLE is the basis for the Parallel Grammar Project, which is developing industrial-strength grammars for English, French, German, Norwegian, Japanese, and Urdu. XLE is written in C and uses Tcl/Tk for the user interface. It currently runs on Solaris Unix, Linux, and Mac OS X.</footnote>
With the reservations discussed in Section 2 in mind, we proceed to present Table 1 where the various types of parsed MWE structures are listed. In all, simple sentences containing 850 verb MWEs have been parsed. In Table 1 we give the basic form of the MWEs: the reader should keep in mind that MG is a pro-drop language with no infinitives, therefore the 1st person singular present indicative (or the 3rd person present indicative if the verb is an impersonal one) are used as the verb's lemma. Our system parses strings in the Greek alphabet but in Table 1 we have used Latin characters for reasons of readability. We represent WWSs as sequences of words joined with underscores, e.g. *pare_pente* (1 in Table 1 and example 8). The column headed with “C” indicates whether the MWE receives a compositional interpretation (Y) or not (N). Lastly, the column headed with “FX” shows whether the MWE is flexible (FL), semi-flexible (SF) or fixed (F). We have marked as SF the MWEs that allow for no word order permutations but their head verb declines fully. MWEs that allow for word order permutations and their head verb declines fully are marked as FL.

With the approach described here, the lexicon has to be enriched with verb-
like predicates such as *ego_arpazo* (2 in Table 1) and *piano_gematos* (9 in Table 1),
noun-like predicates such as *mavros_mati* (10 in Table 1) and adjective-like pred-
icates such as *tapi_ke_psichremos* (7 in Table 1) and their morphological para-
digms. Therefore, the morphological paradigm of the verb *arpazo* has to be du-
plicated in order to develop the paradigm of *tis_arpazo*. Similarly, (7 in Table 1)
*meno tapi_ke_psichremos* contains a WWS that consists of the cranberry word
*tapi*, the conjunction *ke* ‘and’ and a fully declinable adjective *psichremos* ‘cool’
that occurs freely in compositional structures. However, the overall amount of
new lexical entries is not more than the entries required when MWEs are parsed
like compositional structures (that is, without assuming WWSs) because in a
“compositional approach” the same number of entries (or more) would be listed
as “idiomatic”. We have already pointed out that if the presented system is pro-
vided with the appropriate lexical entries and their morphological paradigms,
it uses the grammar developed for compositional structures to parse sentences
containing verb MWEs.

A wide variety of structures is shown in Table 1. 1 is a sentence but functions as
an adverb, the MWE in 2 and 3 function as intransitive verbs, 4 and 5 function as
transitive verbs with 5 featuring a case of where the subject binds a possessive
selected by the fixed object. 6 and 7 are predicative structures that contain a
controlled adjectival constituent normally modeled as an XCOMP in LFG. 8, 9
and 10 are MWEs that contain sentential complements, either free (8) or subject
to constraints such as control 9, 10 and strong selection requirements on the
form of the subordinated verb. These structures capture the typology of the 850
<table><caption>Table 1: Types of MG verb MWEs</caption><thead><tr><th>MWE</th><th>LFG representation</th><th>WWS lemma</th><th>C</th><th>FX</th></tr></thead><tbody><tr><td>1 perno pente take five :a type of swearing</td><td>PRED pare_pente</td><td>ADV: perno_pente</td><td>Y</td><td>F</td></tr><tr><td>2 tis arpazo them.CL.ACC grab 'I am beaten up'</td><td>PRED ego_arpazo &lt;SUBJ&gt;</td><td>V: ego_arpazo</td><td>Y</td><td>SF</td></tr><tr><td>3 tin pernao zachari her.ACC pass sugar.ACC 'I have an easy time'</td><td>PRED ego_pernao &lt;SUBJ,OBL&gt; ↑OBL PRED =zachari</td><td>V: ego_pernao</td><td>N</td><td>FL</td></tr><tr><td>4 richni touloumia nero pours bags.ACC water.ACC 'It rains cats and dogs'</td><td>PRED richno &lt;SUBJ,OBJ&gt; ↑OBJ PRED=touloumia_nero</td><td>N: touloumia_nero</td><td>N</td><td>FL</td></tr><tr><td>5 troo to ksilo tis chronias mou eat the beating the year.GEN mine 'I am beaten up'</td><td>PRED troo &lt;SUBJ,OBJ&gt; OBJ PRED=O_ksilo_tis_chronias&lt;POSS&gt; ↑OBJ POSS PRED= ego ↑OBJ POSS PERS =↑SUBJ PERS ↑OBJ POSS NUM =↑SUBJ NUM ↑OBJ POSS GEN =↑SUBJ GEN</td><td>N: o_ksilo_o_chronia</td><td>N</td><td>FL</td></tr><tr><td>6 meno stili alas to remain stele.ACC salt.GEN 'I am left speechless'</td><td>PRED meno &lt;SUBJ,XCOMP&gt; ↑XCOMP PRED=stili_alas&lt;SUBJ&gt; ↑XCOMP SUBJ=↑SUBJ</td><td>N: stili_alas</td><td>N</td><td>FL</td></tr><tr><td>7 meno tapi ke psichremos remain tapi and cool 'I lose all my money'</td><td>PRED meno &lt;SUBJ,XCOMP&gt; ↑XCOMP PRED=tapi_ke_psichremos &lt;SUBJ&gt; ↑XCOMP SUBJ=↑SUBJ</td><td>ADJ: tapi_ke_psichremos</td><td>N</td><td>FL</td></tr><tr><td>8 echi yousto na S has preference to S 'don't tell me that S'</td><td>PRED echi_yousto &lt;COMP&gt; ↑COMP COMPL=vα (impersonal)</td><td>V: echo_yousto</td><td>N</td><td>SF</td></tr><tr><td>9 richno adia na piaso yemata throw empty to catch full 'I fish out of/from'</td><td>PRED richno &lt;SUBJ,OBJ,XCOMP&gt; ↑XCOMP COMPL=na ↑OBJ PRED=adios ↑XCOMP PRED=piano_yematos&lt;SUBJ&gt; ↑XCOMP SUBJ=↑SUBJ ↑XCOMP PERF=+, ¬(↑XCOMP TENSE)</td><td>V: piano_yematos</td><td>N</td><td>FL</td></tr><tr><td>10 kano mavra matia na do NP make black eyes to see NP 'I have not met NP for a long time'</td><td>PRED kano &lt;SUBJ,OBJ,XCOMP&gt; ↑XCOMP COMPL=na ↑OBJ PRED=mavros_mati ↑XCOMP PRED=vlepo &lt;SUBJ, OBJ&gt; ↑OBJ PRED=ego ↑XCOMP SUBJ=↑SUBJ ↑XCOMP PERF=+, ¬(↑XCOMP TENSE)</td><td>N: mavros_mati</td><td>N</td><td>FL</td></tr></tbody></table>
verb MWEs that we parsed. Below we give selected parse-outs of the material in Table 1. Please notice that all f-structures contain a sentential feature IDIOM that is of semantic nature and conveys the meaning of the MWE. Figure 4 shows the f-structure of (9) that features the verb MWE 5 in Table 1. This MWE contains an OBJ GF headed by a WWS and a possessive anaphor that is analysed as a specifier of the projection of the WWS and is bound by the free subject; as a result the free subject and the anaphor are of the same gender and number.

(9) I Maria efage to ksilo tis chronias tis
the Maria.3SG.FEM ate the beating the year hers.3SG.FEM
'Maria was beaten up.'

<figure><img src="image_40.png" /><figcaption>Figure 4: f-structure for *I Maria efage to ksilo tis chronias tis*. ‘Maria was beaten up.’, example (9), MWE 5 in Table 1</figcaption></figure>

Figure 5 shows the c- and the f-structure of (10) that features an example of use of the verb MWE 10 in Table 1 and of example (1) that contains an OBJ GF headed by a WWS and a controlled sentential complement, an XCOMP in LFG terms. The result of the application of the sub-lexical rules is shown on the c-structure.

(10) Ekana mavra matia na tin do.
made.1SG black eyes to her see.1SG
'I have not seen her for a very long time.'
4 Issues in parsing MWEs in an LFG/XLE framework

<figure><img src="image_41.png" /></figure>

<figure><img src="image_42.png" /><figcaption>Figure 5: c- and f-structure for *Ekana mavra matia na tin do*. ‘I have not seen her for a long time’, example (10), MWE 10 in Table 1</figcaption></figure>
## 7 Discussion

We have presented a symbolic system for parsing MWEs that uses XLE and LFG grammars as its main components. MWEs are recognized as such before entering the XLE component and their sequential fixed parts are processed to form words with spaces (WWS). WWSs are processed as words by the XLE component. This system definitely reduces ambiguity since fewer parsings are available by definition; furthermore, the system does not require a lexicon more elaborate than the one required by a “compositional” approach. However, we have no way to measure whether the system (with the components that have been implemented so far) performs faster as there is no base system that we can use for a comparison – for instance, it would be interesting to evaluate the effect of the ambiguity that occurs in the filter.

An interesting feature of the system presented here is that it receives lexical knowledge from a lexicographic resource (IDION) that has been developed independently. The embedding of IDION into the LFG/XLE parsing system is a way of evaluating it. IDION has been designed with reusability issues in mind. However, the development of the transcription software indicated that some additional structural information would be beneficial, such as the marking of the head verbs and the marking of PPs (at the moment PPs are constructed by the transcription application that reads the IDION encoding and generates XLE entries). In the future, we aim to expand and improve the system in several ways, including an enrichment of IDION with other types of MWEs (nominal, adversial), a more sufficient implementation of the filter and, of course, a grammar capturing a wider range of MG structures.

## Abbreviations

<table><tr><td>GF</td><td>grammatical function</td><td>PoS</td><td>part of speech</td></tr><tr><td>LFG</td><td>lexical functional grammar</td><td>PP</td><td>prepositional phrase</td></tr><tr><td>MG</td><td>Modern Greek</td><td>SUBJ</td><td>subject</td></tr><tr><td>NP</td><td>noun phrase</td><td>WWSs</td><td>words with spaces</td></tr><tr><td>OBJ</td><td>object</td><td></td><td></td></tr></table>

## References

Apidianaki, Marianna, Prokopis Prokopidis & Haris Papageorgiou. 2018. Combining cross-lingual and syntactic evidence for Greek multiword expression
identification. *Bulletin of Scientific Terminology and Neologisms of the Academy of Athens, Special issue on MWEs in Greek and other languages: From theory to implementation.*

Attia, Mohammed A. 2006. Accommodating multiword expressions in an Arabic LFG grammar. In Tapio Salakoski, Filip Ginter, Tapio Pahikkala & Tampo Pyysalo (eds.), *Advances in natural language processing: 5th international conference, FinTAL, proceedings* (Lecture Notes in Computer Science 4139), 87–98. Berlin & Heidelberg: Springer.

Bargmann, Sascha & Manfred Sailer. 2018. The syntactic flexibility of semantically non-decomposable idioms. In Manfred Sailer & Stella Markantonatou (eds.), *Mutliword expressions: Insights from a multi-lingual perspective*, 1–40. Language Science Press. DOI:10.5281/zenodo.1182587

Bilgram, Thomas & Britt Keson. 1998. The construction of a tagged Danish corpus. In *Proceedings of the 11th Nordic conference of Computational Linguistics (NODALIDA 1998)*, 129–139. Copenhagen, Denmark: Center for Sprogteknologi, University of Copenhagen, Denmark. http://www.aclweb.org/anthology/W98-1614.

Copestake, Ann, Fabre Lambeau, Aline Villavicencio, Francis Bond, Timothy Baldwin, Ivan Sag & Dan Flickinger. 2002. Multiword expressions: Linguistic precision and reusability. In *Proceedings of the third international conference on Language Resources and Evaluation (LREC 2002)*, 1941–1947. Las Palmas, Canary Islands, Spain.

Crabbé, Benoît, Denys Duchier, Claire Gardent, Joseph Le Roux & Yannick Parmentier. 2013. XMG: eXtensible MetaGrammar. *Computational Linguistics* 39(3). 591–629. DOI:10.1162/COLI_a_00144

Dalrymple, Mary. 2001. *Lexical functional grammar*. Vol. 34 (Syntax and Semantics). San Diego, CA: Academic Press.

Grégoire, Nicole. 2010. DuELME: A Dutch electronic lexicon of multiword expressions. *Language Resources and Evaluation* 44(1–2). 23–39.

Gross, Maurice. 1988a. Les limites de la phrase figée. *Langage* 90. 7–23.

Gross, Maurice. 1988b. Sur les phrases figées complexes du français. *Langue française* 77. 47–70.

Kay, Paul & Ivan Sag. 2012. *A lexical theory of phrasal idioms*. Stanford, CA. http://www1.icsi.berkeley.edu/~kay/idiom-pdflatex.11-13-15.pdf.

Korkontzelos, Ioannis & Suresh Manandhar. 2010. Can recognising multiword expressions improve shallow parsing? In *Proc. of the 11th annual conference of the North American chapter of the Association for Computational Linguistics: Human Language Technologies NAACL/HLT 2010*, 636–644. Los Angeles, California.
Markantonatou, Stella & Niki Samaridi. 2018. Revisiting the grammatical function “object” (OBJ and OBJ₀). In Manfred Sailer & Stella Markantonatou (eds.), *Mutliword expressions: Insights from a multi-lingual perspective*, 187–213. Language Science Press. DOI:10.5281/zenodo.1182599

Minos, Panagiotis, Stella Markantonatou, Georgios Zakis & Elpiniki Margariti. 2016. *Generating LFG/XLE MWE entries from IDION (a theory neutral lexical DB)*. Struga, fYROM. http://typo.uni-konstanz.de/parseme/index.php/2-general/156-selected-posters-struga-7-8-april-2016. 6th PARSEME general meeting.

Papageorgiou, Haris, Prokopis Prokopidis, Voula Giouli & Stelios Piperidis. 2000. A unified POS tagging architecture and its application to Greek. In M. Gavrili-dou, G. Carayannis, S. Markantonatou, S. Piperidis & G. Stainhauer (eds.), *Proceedings of the 2nd Language Resources and Evaluation Conference (LREC 2000)*, 1455–1462. Athens, Greece.

Pollard, Carl & Ivan Sag. 1987. *Information-based syntax and semantics* (CSLI Lecture Notes 13). Stanford: CSLI.

Sag, Ivan, Timothy Baldwin, Francis Bond, Ann Copestake & Dan Flickinger. 2002. Multiword expressions: A pain in the neck for NLP. In *Proceedings of the 3rd international conference on Computational Linguistics and Intelligent Text Processing* (Lecture Notes in Computer Science 2276), 1–15. Springer.

Samaridi, Niki & Stella Markantonatou. 2014. Parsing Modern Greek verb MWEs with LFG/XLE grammars. In *10th workshop on Multiword Expressions (MWE 2014), European chapter of the Association for Computational Linguistics 2014*, 33–37. Gothenburg, Sweden.

Waszczuk, Jakub & Agata Savary. 2015. *Modeling syntactic properties of MWEs in LFG*. La Valletta. 4th PARSEME general meeting.
# Chapter 5

## Multiword expressions in multilingual applications within the Grammatical Framework

Krasimir Angelov

University of Gothenburg

The main focus of Grammatical Framework (GF) is in multilingual applications where the same type of content is produced and analyzed in several languages at once. This is achieved by joining the grammars for all languages with a shared interlingual representation. In designing the interlingua, multiword expressions are an important factor that must be considered. Here, I adopt the broader definition where everything that translates non-compositionally across languages is considered an expression. In this chapter I present multiword expressions from a cross-lingual perspective in relation to an interlingual grammar.

### 1 Introduction

Grammatical Framework (GF, Ranta 2011) is a programming language for developing multilingual applications. The typical applications are in natural language generation, dialogue systems, machine translation or in question answering systems where it is feasible to assume a limited language domain. In these scenarios it is possible to design a controlled language which can be completely covered with a formal grammar. On the other hand, these applications are typically highly multilingual. It is not uncommon to have a single grammar which supports simultaneously more than twenty languages. There are a number of challenges in this kind of application.

First of all, in order to scale to a high number of languages, GF is designed to work with an interlingua. Every grammar is divided into an abstract syntax and one or more concrete syntaxes. The abstract syntax is a language-independent

<figure><img src="image_43.png" /></figure>
interlingual representation of the application domain, while each of the concrete syntaxes renders an abstract syntax tree into a string in the corresponding natural language. In that setting, translation, for instance, is reduced to parsing the input sentence into an abstract tree and then rendering of the same tree into another concrete language.

Furthermore, developing even a small language fragment would normally require several low-level details, such as word order and gender/number agreement, to be reimplemented from scratch for every language and for every application. This would be highly ineffective if it was not aided by the development of the Resource Grammars Library (RGL, Ranta 2009) in GF. RGL is a library of wide coverage grammars for more than thirty languages developed by a community of linguists and computer scientists. By reusing the library, new applications can be built in short time by people who do not even have to be linguistically trained and who may not be experts in the target languages.

Working on the level of the RGL is still too low-level though. The library is trying to hide syntactic differences across languages but this is still not what we ultimately want in an application. What is needed is a model which can abstract over the language-independent semantics of the sentence. Phenomena like constructions and multiword expressions translate non-compositionally across languages, and thus are recurring obstacles that have to be resolved in every application. For that purpose there is a different grammar for each application. Application grammars, for example, are more semantically oriented. On the other hand, resource grammars are syntactic. Another difference between these two grammars is that resource grammars are highly lexicalized, but lexical entries often become semantic functions in application grammars. This is a key design decision which allows us to have an abstract language-independent representation. For example, such a representation lets us hide the language-specific multiword expressions in the modules for the concrete languages, without affecting the abstract syntax.

This strategy has been proven efficient in limited domains, and most of this chapter will be about how language-specific multiword expressions and constructions are represented in GF.

We have recently started to scale up from limited-domain applications to wide coverage parsing and translation. For this to be successful, it is important to have a library of commonly used constructions across different languages. Although this is still a moving target, I will report on the current efforts to build such a library by either reusing existing resources, or by creating those using automatic methods. This also shows that the strategy used for limited domains can scale to
an open domain, when there is a wide-coverage resource of raw data that can be
ported to the platform.

Please note that moving from lexicalized syntactic grammars to unlexicalized semantic grammars requires, for many languages, syntax to be represented in a discontinuous way. Just to give a simple example, forming questions in English requires that we move or add auxiliary verbs in front of the sentence, while the rest of the verb phrase is left somewhere in the middle. Other languages might not use auxiliaries at all or they might just form questions differently. This means that the verb phrase in English must be modelled as a single phrase with two discontinuous parts. The implications from this for the implementation of the framework will be discussed as well.

## 2 The basic principles of GF

GF is designed as a multilingual framework from the ground up. A typical application starts by identifying the relevant domain and then describing the desired phrases within that domain in multiple languages. In order to accommodate and link several diverse languages, the framework separates the grammar into two distinct conceptual layers: abstract and concrete syntax.

The **ABSTRACT SYNTAX** is a logical framework which acts as a language independent interlingua. It defines a collection of types and functions which can be used to build abstract syntax trees. Each abstract tree represents a phrase which is realized by using one of the available **CONCRETE SYNTAXES**. In this section, I will informally introduce the abstract and the concrete syntax in GF by example. For a more detailed introduction to GF we refer to Ranta (2011).

We start with the lexicon. On an abstract level, the lexicon consists of a simple inventory of word senses. For example, we might have:

cat N
fun horse_N : N

Here the first line declares that there is a category *N*, which will denote the type of all nouns. The second line defines a function with no arguments, a.k.a. a constant of type *N*. These abstract constants serve as cross-lingual lemmas. By convention we use names composed of an English lemma followed by a part of speech tag. When these are not sufficient to disambiguate the meaning of the word, then we can add more elements. For example, we could use WordNet's sense numbers for disambiguation:
fun arm_1_N : N (body part)

fun arm_3_N : N (weapon)

The lexicon starts to get interesting only when we move to the concrete syntax.
The concrete syntax for English looks something like:

lincat N = Number => Str
lin horse_N = table {Sg => "horse" ; Pl => "horses"}

Here the keyword lincat introduces the linearization category for nouns, i.e. for the type N, and lin introduces the linearization of the function horse_N itself.

In programming language parlance, the abstract category N is like an abstract data type, i.e. a mere name with a hidden implementation, while the linearization category in the concrete syntax is its actual implementation. In GF, unlike in other programming languages, a single type or a single function might have several different implementations – one for every concrete syntax. In this case, the implementation in English says that N is a table or an array of strings (Str) indexed by a Number. The number itself is another data type defined as an enumeration with two possible values – singular (Sg) and plural (Pl):

param Number = Sg | Pl

The linearization of horse_N, on the other hand, gives the actual values in the table. In English these would be the word forms horse and horses, and in French cheval, chevaux. In French, however, we also need to know the gender of the noun in order to take care of the word agreement in the syntax. Because of that the corresponding definition in the concrete syntax for French is slightly more complicated:

lincat N = {s : Number => Str ; g : Gender}
lin horse_N =
  {s = table {Sg => "cheval"; Pl => "chevaux"};
   g = Masc
  }

param Gender = Masc | Fem

Here the linearization category for N is not a simple table of word forms but a record with two fields – s and g. The field s is still an inflection table like in English, but there is also the field g of type Gender with two possible values, Masc and Fem. The linearization for horse_N assigns to the field s the inflection table for French and sets the field g to Masc.
It is also possible to have records which combine together more than one string field. This is used for instance in English where phrasal verbs consist of a main verb and a particle. Those verbs are modelled as records:

```
lincat V2 = {s : VForm => Str; part : Str; prep : Str}
lin swith_off_V2 = {s = table {VInf =>"switch";
                                  VPres=>"switches";
                                  ...};
  part = "off"
  prep = ""}
```

The field part keeps the particle while the s field is the inflection table of the main verb. There is also a third field, prep, which stores the potential preposition for transitive verbs. Since there is no preposition in this case, an empty string is added. In prepositional verbs, however, this field will be non-empty. It is even possible to have verbs with both a particle and a preposition.

It is possible to have multiple string fields in nouns as well. This happens for instance in Chinese where a noun is characterized by its lemma and its classifier. Both are string fields and they could be arbitrarily far apart in the final sentence. For that reason they are stored as two different fields in the record:

```
lincat N = {s : Str; c : Str}
lin horse_N = {s = "ma"; c = "pi"}
```

The structure of the lexicon in all languages is conceptually very similar. There might be more numbers and genders, or there might be grammatical cases, but in general a lexical entry in GF is an inflection table indexed by one or more parameters, and there might be additional fields for features such as gender, word class, classifier, or a particle.

The records shown above are rarely what the GF grammarian actually writes. Instead it is possible to isolate common patterns into reusable operations which allow us to have succinct definitions like:

```
lin horse N = mkN "horse" ;
lin switch_off_V2 = mkV2 (partV (mkV "switch") "off") ;
```

Here the smart paradigm (Détrez & Ranta 2012) operates mkN and mkV are responsible for predicting the inflection tables of nouns and verbs from the lemma. When the inflection is not predictable from the lemma alone then it is possible to specify extra arguments, i.e.:

```
lin mouse_N = mkN "mouse" "mice";
```
In this case the second argument of mkN is the irregular plural form of *mouse*. Auxiliary operations like partV and mkV2 are used to set the particle or the transitivity of the verb.

Having set the basics of the lexicon we can move on to the syntax. In the abstract syntax, the syntax is represented as a collection of n-ary functions. For example, adjectival modification requires two functions, AdjCN and UseN:

```
cat AP; CN
fun AdjCN : AP -> CN -> CN
fun UseN : N -> CN
```

This yields to two syntactic categories: adjectival phrases (AP) and common nouns (CN). The simplest common noun consists of just a single noun (N) and is produced by the function UseN. The function AdjCN lets us to modify the noun with one or more adjectival phrases. How exactly the adjectival phrases are attached is language specific.

In English, there is no gender and the adjective is always before the noun. The linearizations for AdjCN and UseN are simply:

```
lincat AP = Str
lincat CN = Number => Str

lin UseN n = n
lin AdjCN ap cn = table {Sg => ap ++ cn ! Sg;
                         Pl => ap ++ cn ! Pl}
```

Note that when building common noun phrases it is still not known whether the phrase should be used in singular or in plural. It will remain unknown until a determiner is fixed and a complete noun phrase built. For that purpose, the linearization category for CN is an inflection table indexed by number just like for the N category. Since the linearizations for CN and N are the same, the linearization rule for UseN is just the identity function. Since I have defined the linearization for adjectives to be a plain string, the linearization for AdjCN simply concatenates the adjective phrase in front of the common noun. Here the (++) operator indicates concatenation of token sequences, and the exclamation mark (!) is used to fetch the element from the table that corresponds to a given parameter.

Note that the two elements in the table of the last example are identical except that they select different numbers. There is a handy shorthand notation for this case:

```
lin AdjCN ap cn = \
    => ap ++ cn ! n
```
Here the operator (\\) creates a table whose index is the variable n. After the double arrow (=>) is the value itself, which is defined by using the variable n. When I substitute n with Sg and Pl I get the same values as in the previous example.

In French, the adjectival modification requires gender and number agreement. In addition, the adjective is sometimes put before and sometimes after the noun. This means that we need a more complex linearization type for AP:

lincat AP = {s : Gender => Number => Str;
isPrefix : Bool}

This type consists of an inflection table for the adjective and a Boolean parameter which determines whether the adjective should be placed before or after the noun. The linearization rule for AdjCN now is:

lincat CN = {s : Number => Str; g : Gender}
lin AdjCN ap cn = {
  s = \
 aps = ap.s ! cn.g ! n;
  cns = cn.s ! n
  in case ap.isPrefix of {
    True => aps ++ cns;
    False => cns ++ aps
  }
  g = cn.g
}

Here, in the let expression I first compute the right forms of the adjective and of the basic common noun. After that, I concatenate them in the right order depending on the parameter isPrefix. Note that cn.g is used in two different places. First it gives the right gender to use for the adjective, and second it is used to propagate the gender from the smaller common noun which is an argument of AdjCN to the bigger phrase. The rest of the syntax is built in a similar fashion by adding more and more syntactic combinators.

This section had the goal to demonstrate the essential features of GF and how these make it possible to hide language-specific details. In the abstract syntax I merely say that there are adjectives and nouns and that those can be combined together. How exactly this happens is determined by the concrete syntax. In this way, the abstract syntax can stay language-independent while all language-specific features can still be handled. It could be rightfully argued that the level
of abstractness as it is presented so far is still not sufficiently high. For example, I still assume that all languages have adjectives and nouns, which might be questioned for some languages. It did, however, work for the 30+ languages that are already supported in the framework. The most important problem that I will address in the next section, however, is that what is an adjective, noun, or verb in one language might not belong to the same part of speech in another language. This is a source of non-compositional constructions and multiword expressions that need to be handled on a different level in the framework.

## 3 Constructions and multiword expressions in GF

I shall divide expressions in two non-overlapping classes since they are handled differently in GF. The first class are expressions that have meaning only as a whole and that cannot be understood by interpreting their parts compositionally. Examples for those are *by and large*, *after all*, *long time no see*, *instead of*, *because of*, etc. Such expressions are composed of smaller units which have in general their own semantic and syntactic uses, but inside the expressions they are just tokens constituting a larger unit. MWEs cannot be parsed by using meaningful grammatical rules. For instance, in order to parse *instead of* compositionally, a syntactic rule could be added, which combines an adverb and a preposition to form another preposition:

fun foo : Adv -> Prep -> Prep

A rule like this would have no other use but to cover controversial syntactic sequences which do not have any compositional meaning anyway. This makes even less sense in a multilingual setting, since the internal structure of those expressions in English does not persist in other languages. In Swedish, for instance, *because of* translates as *på grund av*, and in Bulgarian, *instead of* translates as *vmesto*. In both cases the translation is another prepositional expression, but its internal composition is very different. The solution is very simple: to ignore the bogus internal composition of those expressions and to add them as multiword units in the lexicon:

fun instead_of_Prep, because_of_Prep : Prep

lin instead_of_Prep = mkPrep "instead of"

lin because_of_Prep = mkPrep "because of"

The implication of this choice is that the parser in GF (Angelov 2011) has to work, not on the level of words, but on a different, more semantic level. In the
case of multiword expressions, this semantic level is a cross-words level, and, in
agglutinative languages, it is often a sub-word level (Angelov 2015). This com-
plication means, for instance, that unlike in most other statistical parsers, GF
parsing is not done on top of a part of speech tagged input. Instead, the parser
performs both parsing and tagging, where a single tag might span several tokens
or conversely only a part of a token.

A subclass of non-compositional expressions is the class of phrasal and prepo-
sitional verbs. Examples of those were shown in the previous section. The com-
plication in this case is that they are not only composed of multiple words but
the words are not even consecutive. Unlike in frameworks based on context-free
grammars, in GF this is a trivial matter. Discontinuous expressions are modelled
by simply using more than one string fields inside a record. On a low-level both
tables and records in GF are modelled as tuples of strings which reduces the for-
malism to a Parallel Multiple Context-Free Grammar (PMCFG, Seki et al. 1991)
which is beyond context-free grammars. When an expression is embedded in a
sentence, then the syntactic rules know where to put each of the constituents.
The assumption, however, is that all lexical units of the same type have the same
types of discontinuities. For instance, the linearization type for all two-argument
verbs in English is:

lincat V2 = {s : VForm => Str; part : Str; prep : Str}

However, only some verbs have particles and only some others have prepositions.
In a monolingual grammar it is possible to split the category into a category for
simple verbs and a category for phrasal/prepositional verbs but this does not
scale across languages. Phrasal verbs in English, for example, are often translated
to simple verbs in Slavic languages, where the information from the particle is
encoded as a prefix attached to the root. Conversely, simple verbs in English
might become prepositional verbs in other languages or vice versa.

The second class of expressions is those that have both a compositional and
a non-compositional meaning. It is often the case that the second is the most
frequent meaning but the former cannot be excluded either. Since GF is a multi-
lingual framework, the most natural way of identifying multiword expressions is
cross-lingual. If an expression has a non-compositional meaning then it is quite
likely that it will be expressed in a very different way in another language. This
is a very empirical criterion which makes it easier to detect multiword expres-
sions, but on the other hand, it fuses multiword expressions with constructions.
Basically anything with a non-compositional abstract syntax across languages
is considered a multiword expression. This kind of expressions is obviously a
problem in an interlingua-based system.
The solution is to identify and factorize expressions. Figure 1 shows the abstract syntax trees for the sentences *My name is John* in English and the equivalent *Ich heiße John* in German. The translation is non-compositional because English has no equivalent for the German verb *heißen*. In a transfer-based translation system, I would have to explicitly manipulate the trees to get the one from the other. In an interlingual system I can factorize.

We add in the abstract syntax a new function which takes as input all fragments from the individual trees that stay invariable. In each of the concrete syntaxes we define that the function produces the corresponding language specific trees where the invariable subtrees are just plugged in the right places. In the particular case we would get:

Abstract:

fun have_name_Cl : NP -> PN -> Cl

English:

lin have_name_Cl p n = PredVP (DetCN (PossNP p) (UseN name_N))
(UseComp (CompNP (UsePN n)))

German:

lin have_name_Cl p n = PredVP p (CompV2 (mkV "heissen") (UsePN n))

<figure><img src="image_44.png" /><figcaption>Figure 1: An example for non-compositional abstract syntax</figcaption></figure>
The new function takes as arguments the subject (NP) and the proper name (PN) and produces a clause (C1). In the German example the subject is actually the pro-noun *ich* with an abstract syntax UsePron *i_Pron*. In English, on the other hand, the syntactic subject is *my name* but we are only interested in varying *my* so the argument UsePron *i_Pron* is wrapped with PossNP which in English gener-ates a possessive determiner from an NP, i.e. from *I* we get *my*. The determiner is then applied to the noun *name*. The result is of category clause which is the same as a sentence except that it has variable tense and word order. This makes it possible to reuse it for building relative clauses, questions and sentences. We can also inflect it in tense and polarity. This means that it is enough to factor-ize the construction only once and then it automatically becomes available in all possible forms. Once we have the new abstract function then we can use a language-independent tree as shown on Figure 1c.

Note that in the linearization rules, unlike in the lexicon and in the syntax of the grammar, tables and records were not used. Instead we are free to reuse the already existing syntactic functions that are available in the grammar. In the pre-vious section, how to define functions, such as AdjCN and UseN, was introduced. These functions can be used not only for parsing/generating sentences but also inside the definitions of new functions. This is exactly what is done here and thus, a lot of low-level details can be avoided.

For lexical units we can either reuse existing lexical definitions like *name_N* or define locally new ones like *mkV "heissen"*. This is handy since nouns like *name_N* are more common across languages and thus we would probably want them in the general lexicon anyway. On the other hand, verbs equivalent to *heißen* can be found in only some languages.

The previous example can be explained as a construction which differs across languages because of a lexical gap, i.e. the missing *heißen* verb in English. However, exactly the same solution can be also used for pure idioms. For example, a prototypical multiword expression like *kick the bucket* in English can be defined as a lexical verb phrase:

fun kick_theadhoc_VP : VP
lin kick_theadhoc_VP = ComplSlash (SlashV2a kick_V2)
          (DetCN (DetQuant DefArt NumSg)
           (UseN bucket_N))

A translation to another language could be realized either as a single verb equiv-alent to *die* or as another idiom. In either case the translation should still func-tion as a verb phrase. Note that the verb phrase above is not just a complicated
way to encode the string *kick the bucket*. When the expression in the example is evaluated it is reduced to a complex data structure which, among other things, contains all inflection forms of *kick* as well as all auxiliary verbs that must be used for forming the different tenses in English.

The common feature between the last two examples is that in both cases we have to move from lexical categories such as noun and verb to a higher-level syntactic categories. For example instead of assuming the existence of a specific verb we just assume that there is a specific verb phrase or a sentence that conveys the same meaning. Similarly instead of nouns we use noun phrases and instead of adjectives – adjective phrases. Basically we move upwards in the hierarchy of syntactic categories until we reach a level where the differences across languages are entirely contained within the selected category.

If the multiword expression contains variable parts then they become arguments of the abstract syntax function. The order in which the arguments are listed in the type of the function is completely irrelevant since in the concrete syntax we are free to use the arguments in an arbitrary order regardless of the order in which they are declared. It is just by convention that we usually choose to use the order in which they are used in English. Note, however, that this freedom does not come for free. For instance, most statistical PMCFG parsers assume that the arguments to a function are used in the order in which they are defined. This assumption is always satisfiable if the grammar is monolingual but in a multilingual setting there is simply no natural order. Moreover, the grammar in a typical statistical parser is learned from corpora and is generally not intended to be interpreted, so any argument order is just as good. In contrast the typical GF grammar is developed by a grammarian who might have his/her own aesthetic preferences.

Using functions with arguments is just one of the ways to make a multiword expression variable. Sometimes general modifiers are admitted in the middle of an expression. Typical examples are light verb constructions such as *I am back* which also admit modifications like *I am already back*. It is not difficult to model the verb phrase *copula+back*:

lin am_back_VP = UseComp (CompAdv back_Adv)

What is not visible here, however, is that the computed verb phrase is discontinuous. The two important parts are an inflection table with all forms of the copula and a second field which contains the argument of the copula, i.e. the adverb *back*. Now if we modify the new lexical verb phrase:

AdVVP already_AdV am_back_VP
then the Resource Grammar automatically knows that the adverb *already* should be inserted between the copula and the argument. The insertion is possible only because of the discontinuity of the verb phrase. Note also that the same adverbial modification in another language may not require discontinuity. For example the equivalent in Bulgarian for *I am back* consists of a single verb and then the adverb is placed before the verb. None of this, however, is visible in the abstract syntax.

In general the ability of the framework to deal with discontinuous phrases is heavily exploited in the resource grammar. It is one of the most powerful features that allows us to hide language specific details and it helps in the implementation of some constructions.

## 4 Libraries of constructions in GF

Constructions and multiword expressions are really abundant in any natural language, and it is part of our mission to collect and organize GF resources for as many languages as possible. The main realization of that mission, so far, is the RGL. In the recent years we have also started to collect general lexical resources. Ultimately we would like to have a Resource Lexicons Library with a multilingual translation lexicon for many languages. Even that is not the end and we should also consider collecting libraries of constructions. There were two pilot projects in that direction: Gruzitis et al. (2015) and Enache et al. (2014).

In Gruzitis et al. (2015) the goal is to formalize the Swedish Constructicon (Lyngfelt et al. 2012). The original constructicon is a semi-formal database which covers common constructions in Swedish relevant for second language learners. There is also an ongoing work to link the resource with the Berkeley Constructicon for English (Bäckström et al. 2014). The focus, however, is in language learning rather than parsing or translation. As such it was not the primary goal to organize the constructicon as a formal grammar usable for automatic processing. Instead each entry in the resource combines an informal textual description with a syntactic pattern written in a semi-formal style. The syntactic patterns were parsed and converted to GF rules which extend the Swedish Resource Grammar.

The original constructicon contains 374 entries of which the project focused on the 105 constructions for verb phrases. Due to inconsistencies in the original resource in the first round only 43 out of the 105 constructions were successfully converted. After several iterations of manual inspection and correction, the number of successful constructions increased to 93. The remaining cases were consistently annotated but are corner cases that are currently not supported by the conversion algorithm. The necessary corrections and inconsistencies were
sent back to the developers of the constructicon and are fixed by now. The ex-
periment, however, clearly showed the advantage of using a formal system that
can guard against accidental errors that are imminent in a free text format.

At the end each of the constructions was converted to one or more GF func-
tions which in total resulted in 127 abstract functions. For 98 out of these 127
abstract functions, the corresponding concrete syntax was also successfully con-
structed automatically. A logical continuation of the project would be to also
convert the aligned entries from the Berkeley Constructicon and later to add
other languages.

Enache et al. (2014) started from a much lower level and tried to find candidates for multiword expressions from the Wikitravel phrase collection in English, German, French and Swedish. The general idea is that, given a pair of parallel sentences, the algorithm extracts all possible abstract syntax trees for each sentence and if there is no common abstract tree for both sentences, then the pair must contain a non-compositional expression. The candidates are then manually examined and the new constructions are added in a library of constructions. The majority of constructions found in this way span over larger syntactic structures and are thus above the level of a simple lexicon. For example out of 171 candidates 142 expressions were syntactic. They can be roughly classified as: greetings, weather reports, time expressions, money, units of measurement and spatial deixis. The remaining 29 expressions are lexical. For example locker in English translates as låsbart skåp ('lockable closet') in Swedish.

Another experiment in Enache et al. (2014) is to learn a lexicon of compound nouns between English and German. The method uses automatic word alignment in a parallel corpus. The candidates for compounds are pairs of phrases where: the English side must be parsable as a noun phrase with the GF grammar, the German side must consist of a single word, and finally the overall probability for the pair must be above a fixed threshold level. The compound nouns extracted in this way were added to the lexicon of a statistical machine translation system and the evaluation showed a noticeable improvement in the BLEU score.

5 Application grammars

The discussions so far were on the level of the Resource Grammars. The typi-
cal GF applications, however, never use the resource grammars directly. Instead
they are used as libraries to build application grammars. The main difference
is that while the abstract syntax of a resource grammar describes some kind of
abstracted syntactic level, the application grammar describes an abstracted do-
main semantics. Another way to see the difference is to think about the abstract
syntax of the application grammar as an ontological language for describing the
application domain. The abstract syntax of the resource grammar, on the other
hand, is an ontology which describes the syntactic constructions that someone
would expect to find in a natural language.

While in the resource grammar we work with categories like noun phrase
and verb phrase, in the application grammar we switch to semantic categories
like person, agent, food, drink, etc. The abstract syntax functions, on the other
hand, are semantic predicates which take, for instance, an agent and a drink and
produce a statement like:

someone(person) drinks something(drink)

The main role of these new semantic categories is to provide sortal restrictions on
the types of nouns that can be used for the different arguments of the predicates.
Otherwise the predicates are implemented in a fashion that is very similar to the
one for multiword expressions presented in Section 3. In particular most of the
predicates are de-lexicalized which gives us more freedom to keep the abstract
syntax language-independent while hiding all differences in the concrete syntax.

The sortal restrictions might be relevant for general multiword expressions
as well. For example part of the annotations in the Swedish Constructicon are
about semantic roles such as Actor, Theme, Result, etc. Those were ignored while
converting the resource to GF, but it is possible that some of these constructions
are valid only when the constraints are satisfied.

There are several advantages in working with application grammars. First,
they are typically much smaller than the resource grammars, which also makes
them computationally much more efficient. Second, since the application gram-
mars cover only a specific domain, they can guarantee translation with publish-
ing quality. However, when the resource grammars are used directly in trans-
lation then the quality is much worse. Most of the problems can be attributed
to multiword expressions which are simply not covered by the vanilla resources.
Having a comprehensive grammar of multiword expressions should improve the
quality a lot, but since building a general and comprehensive resource is very ex-
pensive, we currently do it on application by application basis.

The main disadvantage of the application grammars is that they lack robust-
ness. They can analyse input conforming to the grammar but fail completely if
there is even a minor violation. For that reason they are mostly used for con-
trolled languages (Angelov & Ranta 2010) where the users must use authoring
tools that help them to stay within the scope of the grammar. A screenshot of
one of those tools (Ranta et al. 2010) is shown on Figure 2. With this interface the
users are not allowed to enter free text but instead they compose a sentence by choosing words from a list of options. The sentence is built incrementally and at each step the list contains only words that are permitted as a possible next word in the sentence.

<figure><img src="image_45.png" /><figcaption>Figure 2: An authoring interface for writing Controlled Languages</figcaption></figure>

The controlled language authoring is useful only when the grammar is re-
strictive. If the same interface is used with the resource grammar, then since
there are very little restrictions, almost every word can appear almost every-
where. The analysis of a strange combination of words, however, could be equally
strange. The other disadvantage of that interface is that it is not possible to get
an overview of all constructions that are available in the grammar. In a sense,
that interface gives us the ant's point of view which sees each word one by one.
What we sometimes want is the bird's view which sees the grammar from the
top.

One such interface was developed in Hedström et al. (2016). With that interface
the user is first presented with a list of all possible constructions. When a particu-
lar construction is chosen then he/she is guided to a customization interface like
the one on Figure 3. There the user sees an example of the construction rendered
in two languages. Below the example, there is a list of options that can be used
to customize the construction. On the figure, the example is the construction is have_name_cl from Section 3 rendered in Swedish and Bulgarian. The possible customizations are to turn the construction from a statement to a question or to change the subject, i.e. Who are we talking about?.

<figure><img src="image_46.png" /><figcaption>Figure 3: A browsing interface for an application grammar</figcaption></figure>

This particular interface is not restricted to controlled languages. It can be configured to work with any grammar where the configuration describes which phrases should be included in the browser. For example, if it is coupled with the resource grammar, then it is not necessary to make the whole of the grammar visible. Instead the browser can only include phrases that are relevant for a particular purpose. For example, the interface is currently used in an offline mobile translation application (Angelov et al. 2014) which can translate free text. The browsing interface, however, does not expose the entire grammar, and instead it only covers common tourist phrases for which we can guarantee publishing quality.

# 6 Wide coverage grammars

The resource grammars and the application grammars are the two main types of grammars that we usually deal with in GF. Just in the last few years, however, we have started scaling up the framework to an open domain. The milestone that made that possible is the numerous improvements in the compiler and the interpreter for bigger grammars, and in particular the improvements in the GF parser (Angelov 2011).
There are two challenges that we have to deal with in the open domain. The first is robustness and the second disambiguation. We get the robustness by using a wide coverage grammar which basically consists of the resource grammar plus a large lexicon. On top of that we added minor extensions that deal with ungrammatical input. The disambiguation relies on a statistical ranking trained on the Penn Treebank (Angelov 2011).

As we mentioned earlier, translation via the vanilla resource grammar is far from perfect. We compensate, however, by plugging a high-quality application grammar for a particular domain. By combining the two we get decent quality as long as we stay close to the target domain. For example, Ranta et al. (2015) reports BLEU scores above 70% for technical descriptions of places and objects related to accessibility by disabled people. Translations outside of the domain are still possible thanks to the resource grammar.

Again, one of the major roles of the application module in the wide-coverage translator is to provide proper translations for non-compositional expressions. We expect that scaling further the quality of the generic translator will also critically depend on the availability of a wide-coverage resource of constructions.

## 7 Conclusion

In general we have no doubt that GF can cope with multiword expressions. Almost every application grammar in GF must deal with some of them. Moreover, we often have to deal with constructions across languages. The key enabling device to allow variability in the constructions is the fact that the framework allows for discontinuities. The interesting challenge that we see, however, is how to collect a good inventory of constructions. Our current case by case solution does not scale well for open-domain applications.

## References

* Angelov, Krasimir. 2011. *The mechanics of the Grammatical Framework*. Chalmers University of Technology dissertation.

* Angelov, Krasimir. 2015. Orthography engineering in Grammatical Framework. In *Proceedings of the Grammar Engineering Across Frameworks (GEAF) workshop*, 53rd annual meeting of the ACL and 7th IJCNLP, 33–40. Beijing, China.
Angelov, Krasimir, Björn Bringert & Aarne Ranta. 2014. Speech-enabled hybrid multilingual translation for mobile devices. In *Proceedings of the demonstrations at the 14th conference of the European chapter of the Association for Computational Linguistics*, 41–44. Gothenburg, Sweden: Association for Computational Linguistics. http://aclweb.org/anthology/E14-2011.

Angelov, Krasimir & Aarne Ranta. 2010. Implementing controlled languages in GF. In *Proceedings of the workshop on Controlled Natural Language* (CNL’09), 82–101. Marettimo Island, Italy: Springer-Verlag.

Bäckström, Linnéa, Benjamin Lyngfelt & Emma Sköldberg. 2014. Towards interlingual constructicography: On correspondence between constructicon resources for English and Swedish. *Constructions and Frames* 6. 9–33.

Détrez, Grégoire & Aarne Ranta. 2012. Smart paradigms and the predictability and complexity of inflectional morphology. In *Proceedings of the 13th conference of the European chapter of the Association for Computational Linguistics (EACL 2012)*, 645–653. Avignon, France: Association for Computational Linguistics. http://aclweb.org/anthology/E12-1066.

Enache, Ramona, Inari Listenmaa & Prasanth Kolachina. 2014. Handling non-compositionality in multilingual CNLs. In *Controlled Natural Language*. CNL 2014 (Lecture Notes in Computer Science 8625), 147–154. Cham: Springer.

Gruzitis, Normunds, Dana Dannells, Benjamin Lyngfelt & Aarne Ranta. 2015. Formalising the Swedish constructicon in Grammatical Framework. In *Proceedings of the Grammar Engineering Across Frameworks (GEAF) 2015 workshop*, 49–56. Beijing, China: Association for Computational Linguistics. http://aclweb.org/anthology/W15-3307.

Hedström, Björn, Matilda Horppu & David Michaëlsson. 2016. *Parlira: An interactive phrasebook for Android devices*. Tech. rep. Chalmers University of Technology.

Lyngfelt, Benjamin, Lars Borin, Markus Forsberg, Julia Prentice, Rudolf Rydstedt, Emma Sköldberg & Sofia Tingsell. 2012. Adding a constructicon to the Swedish resource network of Språkbanken. In Jeremy Jancsary (ed.), *Proceedings of KONVENS 2012*, 452–461. ÖGAI. LexSem 2012 workshop.

Ranta, Aarne. 2009. The GF Resource Grammar Library. *Linguistic Issues in Language Technology*.

Ranta, Aarne. 2011. *Grammatical Framework: Programming with multilingual grammars*. Stanford: CSLI Publications.

Ranta, Aarne, Krasimir Angelov & Thomas Hallgren. 2010. Tools for multilingual grammar-based translation on the web. In *Proceedings of the 48th annual meeting of the Association for Computational Linguistics (ACL 2010) system demonstrations*, 66–71. Uppsala, Sweden.
Ranta, Aarne, Christina Unger & Daniel Vidal Hussey. 2015. Grammar engineering for a customer: A case study with five languages. In *Proceedings of the Grammar Engineering Across Frameworks (GEAF) 2015 workshop*, 1–8. Beijing, China: Association for Computational Linguistics. http://aclweb.org/anthology/W15-3301.

Seki, Hiroyuki, Takashi Matsumura, Mamoru Fujii & Tadao Kasami. 1991. On multiple context-free grammars. *Theoretical Computer Science* 88(2). 191–229.
# Chapter 6

## Statistical MWE-aware parsing

Mathieu Constant

ATILF UMR 7118, Université de Lorraine/CNRS

Gülşen Eryiğit

Istanbul Technical University

Carlos Ramisch

Aix-Marseille Université

Mike Rosner

University of Malta

Gerold Schneider

University of Konstanz and University of Zurich

This chapter aims at presenting different strategies that have been designed to incorporate multiword expression (MWE) identification in the process of syntactic parsing using statistical approaches. We discuss MWE representation in treebanks, pipeline and joint orchestrations, the integration of external lexicons and the evaluation of MWE-aware parsers, concluding with our suggestions for future research.

### 1 Introduction

Supervised STATISTICAL PARSING is nowadays an important and challenging field of natural language processing (NLP). It consists in predicting the most probable syntactic structure of a new sentence, given a statistical model that has been trained on a TREEBANK, that is, a syntactically annotated corpus. Since the semi-final works of Nivre & Nilsson (2004) for dependency parsing and Arun & Keller

<figure><img src="image_47.png" /></figure>
(2005) for constituency parsing, a new research line has emerged: incorporating
the analysis of multiword expressions (MWEs) in such parsers. The main objec-
tive of this chapter is to present different approaches that have been developed
and evaluated for statistical MWE-aware parsing systems.

The design of MWE-aware parsers must address the following questions: How
are MWEs represented in combination with syntactic trees? When is MWE iden-
tification performed with respect to parsing? What algorithms and machine
learning techniques are to be used for the two tasks? How can external lexical
resources be integrated to improve MWE coverage? How are systems evaluated?

Answering the question about MWE REPRESENTATION is fundamental as it en-
ables the definition of a system’s output. Hence, it influences the design of data-
sets used for training and testing, including treebanks, as shown in Section 3.

The ORCHESTRATION issue is also crucial in order to position MWE identifica-
tion with respect to parsing: should it be performed before, during, or after it? The
answer is not straightforward as it might depend on the type of MWE (Eryiğit
et al. 2011). Orchestration also implies determining how the two components in-
teract. For instance, in pipeline strategies (before or after) discussed in Section 4,
should the intermediate input/output be computed using MWE concatenation
strategies or MWE substitution ones? Joint strategies (during) discussed in Sec-
tion 5 alongside *n*-best strategies, might involve different methods like adapting
a grammatical formalism for constituency parsing (Green et al. 2013) or concate-
nating arc labels in dependency parsing (Vincze et al. 2013).

Concerning ALGORITHMS and machine learning, most techniques use worka-
round approaches by adapting the MWE-aware representation to existing repre-
sentations directly exploitable by off-the-shelf tools (Nasr et al. 2015). Nonethe-
less, new parsing algorithms have been recently proposed that include specific
handling of MWEs, notably when using joint strategies (Nivre 2014).

The integration of EXOGENOUS LEXICAL KNOWLEDGE in the system, discussed
in Section 6, is non-trivial but potentially helpful. Indeed, supervised systems are
trained on datasets of limited size. Therefore, one drawback of such systems is the
limited coverage in terms of MWEs. One possible solution consists in integrating
knowledge coming from large-scale MWE lexicons, either manually built and/or
validated (Candito & Constant 2014) or automatically acquired (Schneider 2012).

The last issue concerns EVALUATION: what is the impact of MWE identifica-
tion on syntactic parsing and vice-versa? What types of measure are adequate to
quantify this impact? We try to answer these questions in Section 7.

The outline of this chapter is as follows. First, we briefly explain some basic
concepts and terms in statistical parsing in Section 2. Then, each section ad-
dresses the questions above. We conclude in Section 8 by providing a summary of
the current research in statistical MWE-aware parsing and presenting pointers
that, in our opinion, may lead to significant advances in the field in the future.

## 2 Statistical parsing

Parsing, also referred as syntactic analysis, is the process of assigning a syn-
tactic structure to a given input sentence. The analysis is aimed at producing a
valid syntactic tree conforming to a hand-written or automatically induced lan-
guage grammar. With the emergence of manually annotated datasets (i.e. tree-
banks) and machine learning techniques, statistical parsing (Collins 1996; Char-
niak 2000) has become the dominant approach in the parsing literature.

Statistical parsing aims at selecting the most probable parse tree from the
set of all possible parse trees for a given sentence. These data-driven parsing
models may be basically grouped under generative or discriminative approaches.
GENERATIVE parsing models generally rely on a grammatical formalism whereas
DISCRIMINATIVE ones are usually performed without any underlying grammar.
There exist also joint approaches where a discriminative model is used to rerank
the top *n* candidates of a generative parser.

Constituency and dependency formalisms are the two most common parsing
formalisms used in statistical parsing. Figure 1 and Figure 4 each provide con-
stituency and dependency parse tree samples for the sentence *The prime minister
made a few good decisions.*

In the **CONSTITUENCY FORMALISM**, a sentence is regarded as being composed of
phrases and parsing is the task of determining the underlying phrase structure.
For example, a statistical generative constituency parser aims to assign proba-
bilities to a parse tree by combining the probabilities of each of its sub-phrases.
In the **DEPENDENCY FORMALISM**, parsing is defined as correctly determining the
depending relations between words of an input sentence. More precisely, the
aim of dependency parsing is to correctly determine the dependent-head rela-
tionships between words and also the type of these relationships such as subject,
object, predicate. Dependency parsing is nowadays strikingly more popular than
constituency parsing and attracts the attention of an ever-growing community
in NLP. Furthermore, most existing MWE-aware parsers are developed in the
depending framework. Therefore, in this chapter, we focus mainly on differ-
ent orchestration scenarios applied for different statistical dependency parsing
approaches.

The two commonly used approaches for statistical dependency parsing in the
literature are transition-based (Yamada & Matsumoto 2003; Nivre et al. 2007) and
graph-based (Eisner 1996; McDonald et al. 2006; Nakagawa 2007). TRANSITION-
BASED approaches treat the dependency parsing task as the determination of
parsing actions (such as push/pop operations in a shift-reduce parser) by the
use of a machine learning classifier. GRAPH-BASED approaches treat parsing as
finding the most likely path within a graph, such as the highest-scoring directed
spanning tree in a complete graph. Most MWE-aware parsing strategies are adap-
tations of standard parsers experimenting with various models of orchestration
concerning the scheduling of MWE identification with respect to syntactic anal-
ysis.

MWEs pose *challenges* for all areas of NLP, and statistical parsing is not an
exception. An MWE may be *ambiguous* among accidental co-occurrence, literal,
and idiomatic uses. The possible surface forms of an MWE *vary*, especially due
to morphological variations which may become radical in morphologically rich
languages. MWE components do not have to appear in consecutive locations
within a sentence and it is hard to correctly identify a *discontinuous* MWE by ig-
noring the intervening words. The syntactic *non-compositionality* of MWEs may
result in irregular parse trees. The ambiguous, discontinuous, non-compositional
and variable nature of MWEs needs to be carefully handled during parsing in or-
der to produce a valid syntactic structure. Additionally, annotated datasets (tree-
banks) are crucial resources for the training of data-driven statistical parsers. The
*scarcity and limited size* of MWE-annotated treebanks is a great challenge faced
by MWE-aware parsing.

3 MWE representations in treebanks

The choice of an appropriate MWE representation is crucial, with strong consequences on the format of treebanks. Representational choices that have affected existing treebanks in this way range from words-with-spaces – e.g., the French treebank (Candito & Crabbé 2009) – to the use of special MWE syntactic relations – e.g., the Universal Dependencies project (Nivre et al. 2016). Some treebanks may not even contain MWE representations at all, while others may have sophisticated multi-layer representations (Bejček et al. 2012).

The number and variety of available MWE-aware treebanks is growing (Rosén et al. 2015). They do not necessarily cover the same kinds of MWEs. They often belong to the constituency or the dependency frameworks, but some can also be compatible with different types of grammatical formalisms, like lexical functional grammar (Dyvik et al. 2016). To narrow down the scope of this section, we focus on MWE representations in relation to treebanks *that are useful to or that have been used in statistical MWE-aware parsing*.
<figure><img src="image_48.png" /><figcaption>Figure 1: Constituency MWE-aware tree with words-with-spaces representation</figcaption></figure>

## 3.1 No representation at all

The simplest and most obvious MWE representation is not to consider MWEs at all, only considering separate word tokens. While such a treatment is simplistic, it also has a number of advantages. First and foremost, it is easy to operationalize: no distinction is necessary between single words in combination and MWEs. MWEs include a variety of phenomena: compound nouns, technical terms, multi-word entities, light-verb constructions, phrasal verbs, idioms, and proverbs. In general they are partly non-compositional, but due to this characteristic they also border on or overlap with collocations, which are an inherently gradient phenomenon. Not representing MWEs can thus be seen as a tacit assumption that all forms of MWEs are gradient.

Statistical parsers were conceived to improve parsing performance by modeling lexical interactions (Gross 1984; Sinclair 1991; Collins 1999). As MWEs are a subclass of collocations, the statistical attraction between the participating words is typically very strong and errors are therefore much rarer. Statistical parsers generally perform better on relations that are semantically expected (as e.g., in selectional preferences), so performance on verb complements for example is much higher than on verb adjuncts.

## 3.2 Words-with-spaces representation

A simple representation consists in considering MWEs as single nodes of the syntactic tree (Sag et al. 2002), such as in the strategy adopted in the LFG/XLE parser described by Angelov (2019 [this volume]). This “words-with-spaces” representation implies that MWEs have an atomic interpretation. In the constituency
framework, the MWE forms are leafs. Their parent nodes correspond to their parts-of-speech (POS) category, as shown in Figure 1. For instance, *prime minister* has a noun parent node and *a few* has a determiner parent node. A concrete example where MWEs are represented this way is the first version of the French treebank distributed for parsing (Candito & Crabbé 2009). In the dependency framework, the MWE node has the same linguistic attributes as a single word token: POS tag, lemma and morphological features. For instance, *hot dogs* would be a noun in plural, whose lemma is *hot dog*. Such representations imply that MWEs have been pre-identified and represented as word-with-space tokens before parsing. Moreover, they have several drawbacks in terms of linguistic expressiveness. First, discontinuous MWEs like the light-verb construction *make decisions* in Figure 1 cannot be represented this way. Then, the semantic processing of semi-compositional MWEs might be problematic as the internal syntactic structure is impossible to retrieve.

## 3.3 Chunking representations

Another way of representing MWEs uses **CHUNKING**. Chunks are a polysemous concept, but its two meanings are related. On the one hand, chunks are seen as psycholinguistic units that are partly or fully lexicalized, that is, stored as one entity in the mental lexicon (Miller 1956; Pawley & Syder 1983; Tomasello 1998; Wray 2008). On the other hand, they are the concrete output of applying finite-state technology to obtain base-NPs and verb groups deterministically. While the psycholinguistic and the computational concepts are related, the latter has the drawback that chunks need to be continuous.

Black et al. (1991) pointed out that dependency grammars are particularly suited to model chunks and parse between heads of chunks. In fact, chunks are close to Tesnière’s original conception of nucleus, which is typically not a single word (Tesnière 1959). Some dependency parsers following this scheme exist, for example Schneider (2008). Nivre (2014) has proposed a transition-based parser that performs MWE merging as it syntactically parses a sentence. This operation can be seen as MWE chunking.

A standard way of representing chunks in tagging systems is the IOB annotation scheme (Ramshaw & Marcus 1995).<fnref n="1" /> Such representations have been successfully adapted to named entity recognition (Tjong Kim Sang 2002) and MWE identification (Vincze et al. 2011; Constant et al. 2012). For MWEs, there are variants covering continuous MWEs (Blunsom & Baldwin 2006) and gappy

<footnote n="1">¹Tokens are tagged as “B” for *begin*, “I” for *inside* and “O” for *outside* a chunk.</footnote>
<figure><img src="image_49.png" /><figcaption>Figure 2: Chunking-based representation with IOB tags (Schneider et al. 2014)</figcaption></figure>

<figure><img src="image_50.png" /><figcaption>Figure 3: Flat constituency subtree representation (Green et al. 2011)</figcaption></figure>

ones (Schneider et al. 2014). For instance, Schneider et al. (2014) use a 6-tag set (with additional lowercased tags in order to emphasize nested MWE structures) to represent MWEs enabling 1-level nesting, as shown in Figure 2. Such representations can be used in treebanks for training pipeline MWE-aware systems (Section 4) and joint MWE-aware parsers (Section 5).

## 3.4 Subtree representations

Another way of representing MWEs is to annotate them as SUBTREES made of several nodes of the syntactic tree. Many treebanks using such representations can be found in Rosén et al. (2015). Several types of subtree MWE representations were proposed in treebanks, according to the language, MWE type and syntactic formalism.

For processing purposes, words-with-spaces representations have often been automatically converted into flat subtrees. In the constituency framework, an
<figure><img src="image_51.png" /><figcaption>Figure 4: Flat head-initial dependency subtree representation</figcaption></figure>

<figure><img src="image_52.png" /><figcaption>Figure 5: Structured dependency subtree representation with extended labels</figcaption></figure>

MWE is considered as a special constituent with a given POS tag. MWE components are leaves of the MWE subtree, as shown in Figure 3.<fnref n="2" /> There exist different variants for constituency treebanks (Głowińska & Przepiórkowski 2010). This representation has been used by Arun & Keller (2005) and Green et al. (2011), especially for compounds. In the dependency framework, flat subtrees can be either head-initial, that is, the root of the subtree is the first token (Nivre et al. 2004; Seddah et al. 2013), or head-final, with the root being the last token of the MWE (Eryiğit et al. 2011). All other MWE component tokens depend on this arbitrarily defined head, as shown in Figure 4. This representation is used, for example, in the Universal Dependencies treebanks (Nivre et al. 2016).

Flat subtree representations have a disadvantage: the internal syntactic structure of MWEs, required for semi-fixed MWEs in particular, is lost, like for words-with-spaces representation. To retain the internal syntactic structure as well as the MWE status, some authors propose representing an MWE with its syntactic subtree, where arc labels are extended with MWE tags, as shown in Figure 5. This kind of representation has been used, for instance, for annotating light-verb constructions (Vincze et al. 2013) and continuous MWEs (Candito & Constant 2014).

Candito & Constant (2014) adopt a hybrid representation scheme to distinguish regular from irregular MWEs. Regular MWEs have a regular syntactic

<footnote n="2">²MWE-related symbols MWN and MWD respectively stand for *multiword noun* and *determiner*.</footnote>
<figure><img src="image_53.png" /><figcaption>Figure 6: Representation on two distinct layers (Constant et al. 2016)</figcaption></figure>

<figure><img src="image_54.png" /><figcaption>Figure 7: Representation on factorized lexical and syntactic layers (Constant & Nivre 2016)</figcaption></figure>

structure<fnref n="3" /> whereas they display semantic irregularity. They are represented with structured MWE subtrees, as in Figure 5. Irregular MWEs display an irregular syntactic structure (e.g., *by and large* is the coordination of a preposition and an adjective) and therefore cannot be analysed syntactically in a compositional way. They are represented with flat subtrees, as in Figure 4.

## 3.5 Multilayer representations

One of the most interesting MWE representations combined with (deep) syntactic analysis is the one used in the Prague Dependency Treebank (Bejček et al. 2012). It combines three different analysis layers in the form of trees: morphological (*m*-layer), syntactic (*a*-layer) and “semantic” ones (*t*-layer). Nodes of one layer can be linked to nodes of another layer to model the interleaving of the different types of analysis. MWEs are represented on the *t*-layer and are associated with MWE entries of a lexicon. To our knowledge, there is unfortunately no statistical parser outputting such combined structures.

Though less linguistically expressive, other multilayer representations have been proposed on top of a combined lexical and syntactic parser. The proposal of Constant et al. (2016) is to have two distinct layers for representing lexical and syntactic analysis in the form of dependency trees. The two layers share the same

<footnote n="3">³The distinction between irregular and regular MWEs is arbitrary, being defined by a manually-built set of POS patterns.</footnote>
nodes, that correspond to the tokens, as shown in Figure 6. The syntactic layer represents the syntactic structure in the dependency framework. The lexical layer represents the lexical segmentation in the form of a tree. Arcs in MWE subtrees have a special label “mwe”. For instance, the MWE *prime minister* corresponds to a subtree whose root is *prime* and which is composed of an “mwe” arc from *prime* to *minister*. In order to form a unique tree for the lexical layer, lexical units are sequentially related via arcs labeled “lex”. For instance, the MWE *prime minister* is linked to the following lexical unit *made decisions*. This dual representation has several advantages. First, syntactic and lexical analyses are explicitly separated. In the case of regular MWEs, there is a clear distinction between the syntactic and the semantic status (regular syntactic structure vs. irregular semantics). In addition, the representation enables not only nested MWEs to be annotated (e.g., *a few* in *made a few good decisions*) but also fully overlapping expressions (e.g., the noun compound *rain check* inside the light verb construction *to take a rain check*). On the down side, irregular MWEs are duplicated on the two layers because there is no possible compositional syntactic analysis (e.g., *a few*). Additionally, arcs linking lexical units could be made implicit, as they can straightforwardly be computed from their positions in the sequence.

Constant & Nivre (2016) correct the main drawbacks of the previous two-layer representation by making it more compact and more factorized. The representation is still composed of two layers, but the lexical layer is a forest of constituent-like trees representing complex lexical units like MWEs, as shown in Figure 7. Here, the discontinuous MWE *made decisions* is represented by a tree whose root corresponds to a new lexical node having linguistic attributes like any token: a form (*made decisions*), a lemma (*make decision*), a POS tag (verb) and morphological features (past tense). It is straightforward to elegantly represent embedded and fully overlapping MWEs, as lexical units are trees. Irregular MWEs like *a few* and simple words are called **SYNTACTIC NODES**. The syntactic layer is a dependency tree over such nodes. Therefore, irregular MWE nodes and simple word nodes are shared by the two layers. For example, there is a “det” arc from *decisions* to *a few*, as it is compositionally modified by the complex determiner. This representation is not without some limitations: the lexical layer cannot represent an MWE that strictly requires a graph (and not a tree). For instance, it is impossible to represent the coordinated MWEs *had shower* and *had bath* in the sentence *John had<sub>1,2</sub> a shower<sub>1</sub> then a bath<sub>2</sub>*.
# 4 Pipeline approaches

A minimal processing pipeline consists of a collection of two processes arranged in a chain so that the output of the first process is the input of the other. Thus, a processing pipeline for statistical MWE parsing involves two processes, one to identify the MWEs in the input sentence, and another for parsing the sentence into one or more structures that include the MWEs. The question that we address in this section concerns the order in which these two processes are arranged, and there are clearly two possibilities referred to as preprocessing (Section 4.1), and postprocessing (Section 4.2).

## 4.1 Preprocessing approaches

Preprocessing means that the MWE identification task takes place before parsing. For the parser to benefit from this, a decision must be made about how to represent MWEs in the input. As discussed earlier, there are different approaches, the most important of which employ concatenation (Section 4.1.1), or substitution (Section 4.1.2) operations, as discussed in the following sections.

### 4.1.1 Concatenation approach

A widely used pipeline approach to statistical MWE-aware parsing is to have a RETokenization phase before parsing. It consists in first pre-identifying MWEs, then concatenating their components in one single token, and finally applying a syntactic parser trained on a treebank where MWEs have a words-with-spaces representation (Section 3.2). Note that this approach is limited to continuous MWEs.

For example, given the input token sequence *The prime minister made a few good decisions*, the MWEs *prime minister* and *a few* are first pre-identified. Each of them is then merged by concatenating its components into a single token. The sequence is retokenized as *The prime_minister made aFew good decisions* and is then parsed. This approach has the advantage of reducing the token-count of the sentence and hence reducing the search space of the parser. However, it may not be realistic to recognize some types of MWEs without access to morpho-syntactic information.

Seminal studies on gold MWE identification performed before either consti-
tuency parsing (Arun & Keller 2005) or dependency parsing (Nivre et al. 2004;
Eryiğit et al. 2011) showed that it may have a great impact on parsing accu-
racy. Other studies confirmed that more realistic MWE pre-identification actu-
ally helps parsing. Korkontzelos & Manandhar (2010) evaluated MWE pre-identification using Wordnet 3.0 for lexicon lookup before shallow parsing. The set of MWEs was limited to two-word continuous compound nominals, proper names, and adjective-noun constructions. The authors showed that the approach improves shallow parsing accuracy. For instance, without MWE pre-identification, *he threw the fire wheel up into the air* is erroneously parsed as: *(he) (threw) (the fire) (wheel up) (into) (the air)*, whereas with MWE pre-identification the result is: *(he) (threw) (the fire wheel) (up) (into) (the air)*. Cafferkey et al. (2007) carried out similar experiments with a probabilistic constituency parser. MWEs were automatically identified by applying a named entity recognizer and list of prepositional MWEs. A slight but statistically significant improvement was observed. We should note that in the above studies, MWE identification itself was not evaluated.

The SPMRL shared task (Seddah et al. 2013) had a special track dedicated to MWE-aware parsing in French. The provided treebank included continuous MWE annotations represented as flat subtrees (Figure 4). All but one competing team did not develop special treatments for MWEs. The winning team was the only one to have a preprocessing stage to identify MWEs using a tagger based on linear conditional random fields (Constant, Candito, et al. 2013). The tagger model also incorporated features based on an MWE lexicon (Section 6.3).

### 4.1.2 Substitution approach

Another approach is to use substitution: whenever an MWE from the lexicon matches, it is replaced by its head word. Such approach is employed by Weeds et al. (2007) for technical terms (Section 6.2), and by Schneider (2008) on all chunks. In a typical substitution approach, for example, the term *natural language processing* would be replaced by *processing* before parsing.

The advantage of keeping the lexical head is that resources taking lexical relations into account, such as bi-lexical disambiguation (Collins 1999), can use the lexical information. Thus, potential sparsity problems are reduced in comparison to the concatenation approach. For example, the prepositional phrase attachment ambiguity in *We help users with natural language processing* can be resolved properly, even if *natural language processing* is unseen in the training data. As long as *processing* exists in the training corpus, the ambiguity can be solved because the combination *help-with-processing* is more likely than *user-with-processing*.

The potential drawbacks of this approach are that, on the one hand, strings may be ambiguous, and on the other hand non-compositionality may affect the results. Ambiguous strings are illustrated below: while the first sentence of each
example is an MWE, the second is accidental cooccurrence. The last example
involves light verbs, for which Tu & Roth (2011) use token-wise disambiguation,
as ambiguity is relatively frequent.

(1) a. I saw her, and *by the way* she went there on foot.
b. I recognized her *by the way* she walks.

(2) a. In *natural language processing*, humans are also challenged.
b. In *natural language processing* can be difficult.

(3) a. The politician *took* a strong *position* on the issue.
b. The soldier *took* a vanguard *position* on the mountain top.

Non-compositionality may lead to situations in which the head is semantically
so different that attachment preferences are also affected.

(4) a. I saw the road with the *torch light*.
b. I saw the road with the *traffic light*.

If the MWE *traffic light* is reduced to *light*, the chances are that the prepo-
sitional phrase is erroneously attached to the verb, as *see-with-light* is likely. If
*traffic light* is treated as an MWE, bi-lexical disambiguation can only profit if very
large annotated resources exist. Unless a backoff method to treat MWE compo-
nents is included, the increased data sparseness may easily lead to worse results.

## 4.2 Postprocessing approach

In this section, we present approaches where parsing precedes MWE processing.
We make a distinction between MWE identification and discovery. We define
**IDENTIFICATION** as the process of recognizing MWEs in context, that is, as tokens
inside running text. On the other hand, **DISCOVERY** aims at creating a lexicon of
MWE types from the corpus. This lexicon can later be used to guide MWE identifi-
cation and parsing. In this section, we describe approaches for identification after
parsing (Section 4.2.1) and for discovery after parsing (Section 4.2.2), focusing on
works in which the result of discovery was later employed for identification.

4.2.1 Post-parsing MWE identification

Identifying MWEs after syntactic parsing is a natural approach to MWE-aware
parsing as an MWE generally constitutes a syntactic constituent. In the depen-
dency framework, there is usually a path continuously linking the MWE compo-
nents in the syntactic tree. As a consequence, pre-parsing is particularly relevant
for detecting discontinuous MWEs, that is, MWEs that include alien elements, by employing adapted lexicon lookup methods. In Figure 7, the MWE *made decisions* is discontinuous. As there is an object arc from *made* to *decisions*, the two words are *syntactically* adjacent. A matching procedure taking the syntactic structure into account can therefore be beneficial for MWE identification. Furthermore, MWEs can have different syntactic variants. For instance, *a decision was made by John* is the passive voice variant of *John made a decision*. The detection of such syntactic variants obviously benefits from the result of syntactic parsing.

Fazly et al. (2009) identify verb-noun expressions in a parsed text based on a list of 60 candidate expressions. First, they identify candidate occurrences of the expressions using rules based on syntactic annotations and lexical values. Then, they discriminate MWEs from literal expressions using different methods. One is based on the assumption that a verbal MWE expression has *fewer syntactic variants* than its literal counterparts, giving rise to the heuristic that canonical forms are idiomatic (e.g., *pull one's weight*) and non-canonical variants are literal (e.g., *pull a weight, pull the weights*). Another method compared the distributional contexts of co-occurring verb-object pairs to two sets of gold-standard contexts: one for idiomatic readings and another one for literal readings.

Nagy T. & Vincze (2014) compare the use of parsers and of a syntax-based pipeline approach to identify verb particle constructions in English. English off-the-shelf parsers usually have a specific syntactic arc label to identify occurrences of verb-particle constructions. Nonetheless, such parsers tend to get good precision but low recall, as they do not use dedicated features for this task. The pipeline method developed in this paper uses a standard parser to identify a first set of candidates. This set is subsequently enlarged using other syntactic relations. A classifier is then applied in order to decide whether they are verb-particle constructions or not. They show a significant gain in terms of recall and F-score with respect to standard parsers on the Wiki50 corpus (Vincze et al. 2011).

### 4.2.2 Post-parsing MWE discovery

This section discusses the discovery of new MWEs after parsing. This is particularly useful for the creation of resources that can be used for MWE-aware parsing (Section 6). For instance, such lexicon of newly discovered MWEs can be subsequently used for MWE pre-identification at the next cycle of processing. Seretan (2011) has shown that discovery based on parsed corpora provides considerably cleaner results than those relying on shallow analysis (e.g., POS-tagged corpora). Foufi et al. (2019 [this volume]) discuss the integration of resources built with the help of MWE discovery into a language-independent symbolic parser.
Since the literature in MWE discovery is huge, we focus on two studies that represent a sample of this type of approach. Lehmann & Schneider (2011) and Ronan & Schneider (2015) used automatically parsed data for discovering MWEs of different types, including idiomatic verb + prepositional phrase (PP) combinations and light-verb constructions in English. These cases involved the use of different collocation extraction scores.

For discovering Verb-PP idioms the O/E score was used, combined with filters including T-score and Yule's K (which estimates the degree of non-modifiability of a candidate). Table 1 reproduces the results of discovery, sorting the candidates by descending O/E score. Among the top-ranked candidates, many are genuine idioms (e.g., *to kill two birds with one stone*).

<table><caption>Table 1: Top-ranked verb-object + preposition-noun tuples, using the O/E score (Lehmann & Schneider 2011)</caption><thead><tr><th>verb</th><th>object</th><th>prep</th><th>desc. noun</th><th>T-score</th><th>O/E</th></tr></thead><tbody><tr><td>send</td><td>shiver</td><td>down</td><td>spine</td><td>5.74456</td><td>2.21477 × 10<sup>8</sup></td></tr><tr><td>tap</td><td>esc</td><td>for</td><td>escape</td><td>6.40312</td><td>2.1134 × 10<sup>8</sup></td></tr><tr><td>separate</td><td>shield</td><td>from</td><td>plate</td><td>6.78233</td><td>2.33384 × 10<sup>7</sup></td></tr><tr><td>refer</td><td>gentleman</td><td>to</td><td>reply</td><td>8.24621</td><td>7.8143 × 10<sup>6</sup></td></tr><tr><td>obtain</td><td>property</td><td>by</td><td>deception</td><td>5.2915</td><td>7.60043 × 10<sup>6</sup></td></tr><tr><td>ask</td><td>secretary</td><td>for</td><td>affairs</td><td>6.40312</td><td>5.01529 × 10<sup>6</sup></td></tr><tr><td>kill</td><td>bird</td><td>with</td><td>stone</td><td>5.38516</td><td>3.37917 × 10<sup>6</sup></td></tr><tr><td>add</td><td>insult</td><td>to</td><td>injury</td><td>6.08276</td><td>2.21769 × 10<sup>6</sup></td></tr><tr><td>throw</td><td>caution</td><td>to</td><td>wind</td><td>5.09902</td><td>2.03157 × 10<sup>6</sup></td></tr><tr><td>refer</td><td>friend</td><td>to</td><td>reply</td><td>7.54983</td><td>1.36298 × 10<sup>6</sup></td></tr><tr><td>report</td><td>loss</td><td>on</td><td>turnover</td><td>7.14142</td><td>1.34742 × 10<sup>6</sup></td></tr></tbody></table>

For discovering light-verb constructions, the t-score was used together with a number of filters including WordNet and NomBank lookup (Ronan & Schneider 2015). An example of analysis is shown in Figure 8, showing a precision and recall plot by candidate list length. The vertical axis shows precision and recall, respectively, the horizontal axis (which is logarithmic) gives the cutoff in the ranked list of candidates to be included in the evaluation. For the cutoff at 20, the reported candidates for *give+object*, precision is 100%, while recall is 10%. At rank 2560, about 88% of all instances in the gold standard were found.
<figure><img src="image_55.png" /><figcaption>Figure 8: Precision vs. recall curve of the light verb *give* in the British National Corpus, using t-score (Ronan & Schneider 2015)</figcaption></figure>

# 5 Joint approaches

Joint approaches perform parsing and MWE identification simultaneously. Since syntactic and lexical-semantic information are complementary, both processes can help each other if performed together. In such systems, MWE lexical-semantic segmentation is often seen as a by-product of syntactic analysis, or vice-versa.

Some MWEs require quite sophisticated syntactic information to be recognized, such as subcategorization frames and phrase structure. Joint approaches favor delaying the decision as to whether a given combination is an MWE to the parser, where this information is available. In other words, the system has access to the right information at the right moment.

Parser evaluation scores are often reported on standard test sets, where MWEs have been manually pre-identified (gold). Jointly performing MWE identification and parsing is more realistic than parsing pre-annotated test sets, where MWEs are often represented as words with spaces (Figure 1). Indeed, when moving from standard test sets to real texts, gold MWE identification is not necessarily available. It may be hard to use a pipeline approach (Section 4) if the target MWEs are ambiguous or discontinuous.

On the downside, parsers that perform both syntactic analysis and MWE identification simultaneously are harder to design. First, ambiguity is increased, often
by a larger number of labels and/or parsing decisions that are possible at a given
moment. It is crucial, for such systems, to have coherent MWE annotations in
treebanks, datasets that are large enough, and features that generalize well.

We classify such approaches according to the degree of “MWE-awareness” of
the parser. In shallow approaches, the parser generates *n*-best solutions with-
out putting any particular emphasis on MWEs, then uses MWE information for
reranking (Section 5.1). The majority of joint approaches add MWE information
to training and test treebanks, and then use off-the-shelf parsers enriched with
dedicated MWE features (Section 5.2). We also present fully MWE-aware parsers
that take them into account in the parsing algorithm itself (Section 5.3).

## 5.1 *n*-best and reranking approaches

One possible orchestration solution is to consider MWE identification as a reto-
kenization problem, as described in Section 4.1.1. In *n*-best approaches, however,
the text is first segmented into tokens in a non-deterministic way, considering
*several possible segmentations*. Usually, the output of such non-deterministic to-
kenizer is a lattice containing all possible segmentation paths for a sentence
(Sagot & Boullier 2005). This representation is particularly suited for ambigu-
ous irregular constructions, that could be considered as MWEs or as accidental
co-occurrence, depending on the context. The parser then must take this ambigu-
ous segmentation and uses simple parsing models to disambiguate the input and
generate a parse tree (Nasr et al. 2011).

An *n*-best MWE identifier is used by Constant, Le Roux & Sigogne (2013), pro-
ducing a lattice of possible segmentations. Then, a PCFG-LA parser is used to
disambiguate the possible readings. The authors test two variants. First, they
consider that MWEs in the lattice are single nodes (words with spaces). Thus,
different segmentation possibilities in the lattice are represented by paths with
different lengths. Second, they consider that MWE components are individual
nodes tagged using an IOB scheme, like in Figure 2. The latter obtains better per-
formance because all possible paths in the lattice have the same length, resulting
in more accurate parsing scores.

Conversely, the parser can use the same kind of approach and also generate
*n*-best parsing trees. A reranker can then use MWE-aware features, among oth-
ers, to choose the highest scoring tree. Constant et al. (2012), for instance, use a
deterministic tokenizer but output *n*-best MWE-aware syntactic trees using the
Berkeley constituency parser. Then, they use a discriminative reranker to choose
the correct parse tree that includes MWE features.
These are considered joint approaches because, even though MWE segmentation and parsing are independent processes, one needs to be aware of the format of the input/output of the other. For example, the parser has to be able to process lattices as input, provided by the non-deterministic MWE identifier.

## 5.2 Treebank modification approaches

In Section 3, we discussed several ways to represent MWEs in treebanks. Standard statistical parsers trained on such treebanks will be inherently aware of MWEs, provided that they can handle the particular MWE representation in that treebank. For example, if MWEs are represented as subtrees (Figure 5), then there is no need to explicitly handle MWEs (Nivre et al. 2016). This subsection covers MWE-aware parsing studies in which the learning and parsing algorithms *reMAIN unchanged* with respect to their standard version.

Approaches discussed in this section face several challenges. First, most of the time MWEs are either *absent* from treebanks, or the available representation requires *adaptations* in order to be usable by the parser. Second, parsers learned from MWE-annotated treebanks often require *extra features* to take MWEs into account properly. Third, these features may suffer from data *sparseness*, as individual MWEs may not occur often enough in limited-size treebanks.<fnref n="4" />

In this subsection, we present approaches that tackle the challenges posed by MWEs by:

* adding or modifying the MWE representation in the treebanks, and/or

* adding MWE-dedicated features to the parsing model.

The last challenge, related to data sparseness and domain adaptation, is tackled by integrating external resources in the parser, as discussed in Section 6.

In constituency parsing, several parsers, MWE representations and feature sets have been tested, especially on continuous MWEs in the French treebank. Constant, Le Roux & Sigogne (2013) experiment with two implementations of a PCFG-LA parser, using a representation similar to the one of Green et al. (2011) and a variant similar to IOB encoding.

When MWE annotation is absent, a reasonably straightforward solution is to automatically project an MWE lexicon on the treebank before training the parser. For instance, Kato et al. (2016) project a lexicon of compound function words (e.g., *a number of*) onto the English Ontonotes constituency treebank. Syntactic trees

<footnote n="4">⁴Some MWE categories may never occur (e.g., colloquial idioms) because many existing treebanks cover a single register (e.g., newspapers).</footnote>
are modified to take MWEs into account. Constituents are then automatically
transformed into dependencies and a standard first-order graph-based parser is
learned. While the training data is modified, no MWE features are added to the
model.

Early experiments on MWE-aware dependency parsing compared two repre-
sentation variants: MWEs as subtrees or as words with spaces (Nivre & Nilsson
2004). The results indicated that the subtree representation (joint approach) is
worse than parsing MWEs as words with spaces (pipeline approach). However,
these results were obtained assuming gold MWE segmentation.

Vincze et al. (2013) were among the first to use a dependency parser to perform realistic MWE identification. They focus on light-verb constructions (LVCs) in Hungarian. They first perform an automatic matching of two annotation layers in the Szeged treebank: syntactic dependencies and LVCs. As a result, the dependency link between a light verb and a predicative noun (e.g., OBJ) is suffixed with a LVC tag, whereas regular verb-argument links remain unchanged, like in Figure 5. An off-the-shelf parser is used to predict the syntactic structure of sentences, including LVC links. Given that Hungarian is a relatively free word-order language, LVCs often involve long-distance dependencies. When compared with a classifier baseline, the parser performs slightly worse on continuous LVC instances (F1 = 81% vs. 82.8%) but considerably better on discontinuous LVCs (F1 = 64% vs. 60%).

Treebanks containing MWEs as words with spaces pose problems when con-
verted into subtrees. When splitting an MWE, one needs to manually or semi-
automatically assign POS tags, lemmas and morphological features to the indi-
vidual MWE components. Additionally, the internal syntactic structure must be
inferred. Since it is difficult to automate this task, the internal syntactic struc-
ture of decomposed MWEs is often underspecified using flat head-initial subtrees
(Seddah et al. 2013), head-initial (Nivre et al. 2016) or head-final chained subtrees
(Eryiğit et al. 2011), as detailed in Section 3.4. Eryiğit et al. (2011) compare parsing
and MWE identification accuracy on different treebank representations for differ-
ent MWE types. Their original treebank includes MWEs as words with spaces,
which are semi-automatically transformed into subtrees. Contrary to previous
conclusions (Nivre & Nilsson 2004), results indicate that subtrees may be a more
suitable solution for some MWE types, specially when looking at MWE-aware
parsing evaluation metrics (Section 7). In this study, the words-with-spaces rep-
resentation is shown to have a harming effect on the types where it increases
lexical sparsity, such as in Turkish light-verb constructions.

Candito & Constant (2014) explore several orchestrations for combining syn-
tactic parsing and continuous MWE identification in French, distinguishing syn-
tactically regular from irregular multiword constructions. In particular, they experimented with an off-the-shelf graph-based parser that was learned from an MWE-aware treebank where the subtrees representing regular and irregular expressions have their usual labels suffixed by the POS of the MWE, as shown in Figure 5. They showed on-par results with different pipeline variants.

Nasr et al. (2015) focus on ambiguous compound grammatical words in French of the form ADV+*que* and de+DET. While these represent a limited scope, such constructions are pervasive and hard to identify without access to syntactic information, because its component words can co-occur by chance. For instance, the two sentences below have the same sequences of POS and similar lexical units, but the first one contains an MWE whereas the second one does not:

(5) Je chante *bien que* je sois triste.
I sing well that I am sad
'I sing even though I am sad'

(6) Je pense *bien que* je suis triste.
I think well that I am sad
'Indeed, I think that I am sad.'

In order to deal with these constructions, the training treebank is modified similarly to Candito & Constant (2014), splitting MWEs originally represented as words with spaces into two tokens linked by a special dependency. For example, since *bien que* functions as a conjunction, the conjunction *que* becomes the head, modified by the adverb *bien*. Using a standard graph-based dependency parser, the authors evaluate the identification of the target MWEs on a dedicated dataset. As described in Section 6.3, the use of subcategorization frame information for verbs, coming from an external lexicon, improves the results.

## 5.3 MWE-aware parsing models

The models discussed up to now have the advantage of being simple and fast to deploy. Provided that the training treebank contains MWEs in a suitable representation (which can be manually or automatically converted), the parsing algorithm itself does not need to be changed to accommodate MWEs. These approaches achieve reasonably good results, specially if compared to MWE systems based on purely sequential models. However, they often use language-specific or treebank-specific workarounds and are not always generalizable. Therefore,
some recent contributions focus on designing parsing models that are truly awa-
re of MWEs in the model, with promising results.

In the framework of constituency parsing, Green et al. (2011) propose and eval-
uate an MWE-aware parser based on tree substitution grammars (TSGs). This
work was latter extended, comparing the TSG with a PCFG model enriched with
a factorized lexicon (Green et al. 2013). The authors apply these models to MWE-
rich treebanks for French and Arabic, showing gains for both parsing and MWE
identification. The authors state that TSGs are more powerful than PCFGs, be-
ing able to store lexicalized tree fragments. They are therefore more suitable for
idiomatic MWEs, whose particular syntactic analysis requires larger contexts to
be predicted.

Along the same lines, Le Roux et al. (2014) design a joint parsing and MWE
identification model based on dual decomposition. In this work, however, a spe-
cialized sequence model performs lexical segmentation of MWEs. The MWE iden-
tification module uses conditional random fields, while the parsing module uses
a PCFG-LA also including MWE identification, using the approach of Green et al.
(2013). Both models are combined using penalty vectors that are updated in an
iterative way. In other words, until reaching consensus on MWE identification,
the MWE identifier and parser analyse the input sentence. If the systems do not
agree, they are penalized in proportion to the difference between the given so-
lution and the average solution. This model reaches impressive performance on
the French treebank, reaching an MWE identification F-score of up to 82.4% on
the test set.

Constant & Nivre (2016) propose a new dependency parsing system that jointly
performs syntactic analysis and lexical segmentation (including MWE identifica-
tion). The authors design and evaluate a transition-based parser using two syn-
chronized stacks: one for syntactic parsing and another for lexical segmentation.
The synchronization of both stacks is guaranteed by a unique PUSH transition
which pushes the first element of the buffer on both stacks. The parser mod-
els MWE-dedicated transitions MERGEN and MERGEF, which respectively create
new merged lexical nodes for regular MWEs and lexico-syntactic nodes for fixed
MWEs. An additional COMPLETE transition marks that a given lexical node has
been fully parsed (while being potentially implicit). This approach obtains re-
sults that compare with or exceed state-of-the-art performance on French and
English MWE-rich treebanks. Finally, the authors show that lexical information
can guide parsing, leading to slightly better syntactic trees. The converse assump-
tion does not seem to hold, though, as adding syntactic information to a purely
lexical parser tends to slightly degrade its performance.
# 6 Integration of lexical resources

Lexical resources are large-scale repositories of information typically about simple words, more rarely about MWEs. They can play different roles with respect to statistical MWE-aware parsing, and in this section we discuss three of them. We show how lexical information can help in general to resolve parsing ambiguities (Section 6.1). Then, we focus on the availability of lexical information within pipeline approaches (Section 6.2). Finally, we shift the emphasis to the effect of lexical resources on MWE identification rather than on parsing itself (Section 6.3).

## 6.1 General integration of lexical resources in statistical parsers

Statistical parsers have several drawbacks due to the limited size of available gold standard treebanks used for training. Many words in the datasets are infrequent, which makes it very difficult to learn relevant (lexical) regularities. In addition, when parsing an unseen text, some words are simply absent from the training dataset, which negatively impacts parsing accuracy. Experiments with different solutions have been undertaken within the parsing community, notably by incorporating external resources mostly (but not only) learned automatically from large raw corpora.

The use of word clusters is one method to deal with the lexical sparsity issue. Clusters (e.g., Brown clusters), consist of groups of words occurring in the same context. Replacing words by clusters or using clusters as features has each been shown to improve parsing accuracy (Koo et al. 2008; Candito & Seddah 2010). Pairs of words that co-occur frequently in large corpora tend to be related syntactically. The provision of information about such lexical affinities to the parser has been shown to usefully support syntactic attachment decisions. Lexical affinities might be integrated using either soft constraints (Bansal & Klein 2011; Mirroshandel et al. 2012) or hard ones (Mirroshandel & Nasr 2016). The deep learning revolution has opened new perspectives to help handle lexical sparsity, as words are represented as continuous space vectors (i.e., word embeddings) learned from large corpora. Words having similar syntactic behaviors have vectors that are geometrically close to each other (Durrett & Klein 2015; Dyer et al. 2015).

The use of external lexicons has also turned out to be of great interest, notably for dependency parsing. For instance, Candito et al. (2010) successfully use the MElt tagger (Denis & Sagot 2012), thereby incorporating features based on a large-scale morphological lexicon. The integration of hard constraints based on syntactic lexicons was also shown to have a positive impact (Mirroshandel et al. 2013).
## 6.2 MWE resources help parsing

We now give examples of MWE lexical resource integration using a pipeline approach (Section 4) in which MWEs are replaced by their syntactic heads. We do so on two levels: general NP chunking and technical terms.

On the chunking level, replacing chunks with their head words reduces parsing complexity considerably. According to experiments carried out by Prins (2005), parsing performance also increases slightly. However, experiments on technical terms have not confirmed this hypothesis. In other words, replacing chunks with their heads does not necessarily lead to improved results in other settings.

Weeds et al. (2007) used a substitution approach (Section 4.1.2) for term identification in the domain of biomedical research, where gene and protein names in particular are often MWEs. Because taggers, unless they are trained on the domain, perform very poorly, they report better results when replacing technical terms with their head, using a large lexicon of domain terms.<fnref n="5" />

A comparable example is the situation in which a sentence such as ... he did not see the traffic_N light_V is POS-tagged incorrectly (light_V instead of light_N). Here, a pipeline substitution approach relying on an MWE lexicon can clearly improve results. This improvement is passed on to the subsequent parsing step. When domain-adapted taggers are available, though, the advantages of the substitution approach tend to disappear. The performance of adapted taggers is often comparable or slightly higher than that of the substitution approach, as tagging accuracy of technical terms increases. In short, sometimes it is better to adapt statistical models (in this case, a domain-adapted tagger) rather than using lexical resources (in this case, an MWE gazetteer of the domain).

Schneider (2014) conducted an experiment using LT-TTT2, an off-the-shelf rule-based named entity recognizer (Grover 2008) on the standard evaluation suite GREVAL (Carroll et al. 2003) with the same approach of replacing multi-word named entities by the head of the MWE. The performance of the substitution approach was slightly worse than when leaving the MWE unchanged. Also this experiment did confirm that statistically motivated resources are usually better than purely lexical resources.

## 6.3 Lexical resources help MWE-aware parsing

Having discussed the effect of lexical resources on parsing accuracy, we now turn to two different ways to use them as a source of features for dependency parsers, to help MWE identification as well as parsing accuracy.

<footnote n="5">⁵Such a lexicon is often referred to as “gazetteer”.</footnote>
The first is to use MWE lexicons to alleviate the low coverage of MWEs in the training dataset. The idea is to perform an MWE pre-segmentation of the input text by lexicon lookup. The pre-segmentation, encoded in an IOB-like format, is then used as source of features during MWE-aware parsing, either in the parser itself for joint approaches (Candito & Constant 2014), or in the MWE tagger applied before parsing in pipeline approaches (Constant et al. 2012; Constant, Candito, et al. 2013).

One advantage of using soft constraints like features is their ability to handle ambiguous MWEs. Let us take the sequence *up to*, which can be either a complex preposition (*no more than*) or an accidental co-occurrence (*look up to the sky*). A naive segmentation will systematically consider it to be an MWE, independently of the context. However, a better decision can be made taking the context (i.e., the set of other features) into account. Using a joint approach on the French treebank, Candito & Constant (2014) managed to gain around 4 points in terms of tagged MWE identification F-score using such lexicon-based features: F1 = 74.5 (with) vs. F1 = 70.7 (without). We should recall, however, that their approach is limited to continuous MWEs.

A second method proposed by Nasr et al. (2015) is to incorporate subcategorization frame information, derived from a syntactic lexicon, as features in a joint parser. This was used to improve the resolution of ambiguities between grammatical compound MWEs and accidental co-occurrences. An example is the French sequence *bien que* which is either a multiword conjunction ('although') or an adverb ('well') followed by a relative conjunction ('that'), as exemplified in Section 5.2. This ambiguity may be resolved using information about the verb in the syntactic neighborhood. The authors included specific features indicating whether a given verb accepts a given complement: *manger* ('to eat') -QUE -DE, *penser* ('to think') +QUE -DE, *boire* ('to drink') -QUE -DE, *parler* ('to speak') -QUE +DE. In particular, they show for French that there is a 1-point gain in F-score, 85.24 (without) vs. 86.41 (with), for MWEs of the form *ADV+que* (ADV+*that*). The effect is spectacular for compounds of the form *de+DET*, that display a 15-point gain: 75.00 (without) vs. 84.67 (with).

## 7 Evaluation

Evaluating a syntactic parser generally consists in comparing the output to reference (gold-standard) parses from a manually labeled treebank. In the case of consistency parsing, a constituent is treated as correct if there exists a constituent in the gold standard parse with the same labels, starting and ending points. These
parsers are traditionally evaluated through precision, recall and F-score (Black et al. 1991; Sekine & Collins 1997).

In standard dependency parsing with single-head constraint<sup>6</sup>, the number of dependencies produced by a parser is equal to the number of total dependencies in the gold-standard parse tree. Common metrics to evaluate these parsers include the percentage of tokens with correct head, called **UNLABELLED ATTACHMENT SCORE (UAS)**, and the percentage of tokens with correct head *and* dependency label, called **LABELED ATTACHMENT SCORE (LAS)** (Buchholz & Marsi 2006; Nilsson et al. 2007).

The evaluation of MWE-aware parsers and the evaluation of whether or not MWE pre-identification helps improving the parsing quality should be carefully carried out. As stated in previous sections, in most works where MWE identification is realized before parsing, the MWEs are merged into single tokens. As a result, the common metrics for parsing evaluation given above become problematic for measuring the impact of MWE identification on parsing performance (Eryiğit et al. 2011). For example, in dependency parsing, the concatenation of MWEs into single units decrements the total number of evaluated dependencies. It is thus possible to obtain different scores without actually changing the quality of the parser, but simply the representation of the results. Instead of UAS and LAS metrics, the attachment scores on the surrounding structures, namely $UAS_{surr}$ and $LAS_{surr}$ (i.e., the accuracy on the dependency relations excluding the ones between MWE elements) are more appropriate for extrinsic evaluation of the impact of MWE identification on parsing. Similar considerations apply to constituency parsing.

Figure 9 provides two example sequences for the phenomena discussed above; one containing a continuous MWE (on the left side) and another one containing a non-continuous MWE (on the right side). The dependency trees in this figure provide the gold standard unlabeled dependency relations for both examples. Correctly predicted dependencies are presented with check marks (✔) over the relations, whereas the wrongly predicted dependencies are presented with a cross mark (X). The continuous MWE of the left side sequence consists of three tokens ($w_4$, $w_5$ and $w_6$). In other words, the two dependency relations of the overall sequence belong to the relations between MWE elements. The non-continuous MWE of the right side sequence consists of two tokens ($w_3$ and $w_6$).

The first examples of each column (A and E) show the success of a dependency parser without any prior MWE identification process. In the remaining settings, an MWE identifier is run over the given sequence before parsing. Both

<footnote n="6">⁶Each dependent node has at most one head in the produced dependency tree.</footnote>
<figure><img src="image_56.png" /><figcaption>Figure 9: Extrinsic evaluation examples of the impact of MWE identification on dependency parsing performances</figcaption></figure>

the overall unlabeled accuracy $UAS_{OA}$ and the accuracy of the surrounding structures $UAS_{surr}$ are provided next to the trees. Examples (B), (C) and (D) show the correctly detected relations by applying an MWE identifier prior to the syntactic parsing. In (C) and (D), the detected MWE is combined into a single unit ($w_4w_5w_6$) whereas in (B), the detected MWE is represented as a subtree.

In (A), (B) and (C), although the parser success does not change on detecting the syntactic dependencies, $UAS_{OA}$ is affected by the total number of evaluated dependencies, whereas $UAS_{surr}$ remains stable, as expected. In (D), MWE identification helps the parser to detect one more dependency relation, which is reflected in $UAS_{surr}$. Similarly, in (F), the pre-identification of “$w_3$ - $w_6$” MWE has no impact on the parser’s performance. Although this can be directly observed by $UAS_{surr}$ (60%), $UAS_{OA}$ mistakenly gives the impression of an improvement in parsing performance (50% ⇒ 66.6%). This is because in this setting (second column of Figure 9) $UAS_{OA}$ evaluates the performance of MWE pre-identification and dependency parsing as a whole. In (G), the parser performs better after MWE identification, which is again reflected in the surrounding structure evaluation.

Although $UAS_{surr}$ and $LAS_{surr}$ are valuable scores for measuring the impact of identifying different MWE types on parsing performance, they are troublesome with automatic MWE identification, when gold-standard MWE segmentation is not available. Then, erroneous MWE identification would degrade parsing scores
on the surrounding dependencies. An alternative solution is to detach the con-
catened MWE components (if any) into a dependency or constituency subtree
(Candito & Constant 2014; Eryiğit et al. 2011). This way, the standard evaluation
scores UAS and LAS are still applicable in all different orchestration scenarios,
for both continuous and non-continuous MWEs, successfully assessing the per-
formance of joint syntactic parsing and MWE identification as a whole.

8 Conclusions

In this chapter, we elaborated upon several approaches for combining MWE pro-
cessing with statistical parsing to yield statistical MWE-aware parsing. These ap-
proaches depend on different parameters such as MWE representation, orches-
tration and external resource integration. First of all, the selected MWE repre-
sentation combined with syntactic analysis have a strong impact on the system
implementation, since the more elaborated and hence more linguistically expres-
sive the representation is, the more complex the computational system has to be.
Representations vary from simple words with spaces to multilayer structures.
The timing of MWE identification with respect to syntactic parsing, namely or-
chestration, is a crucial feature that needs to be carefully taken into account when
designing a statistical MWE-aware parser, as the best choice partly depends on
MWE type under consideration. MWE identification may be performed before,
after, or during parsing. The first two were discussed under the rubric "pipeline"
approaches in Section 4; the third, under "joint" approaches, in Section 5. Last,
we showed that the use of external resources is another important feature that
is required to handle the sparsity problem, not only to support syntactic attach-
ment decisions, but also MWE identification.

Although it is difficult to draw hard and fast conclusions, it seems that further
investigation of dedicated MWE-aware parsing models is called for. Such models
can benefit from joint modeling of closely related tasks, with information from
one layer helping to disambiguate the other. Joint approaches seem to offer a
very promising line of research, as has been shown for other NLP tasks: e.g.,
joint POS tagging and parsing (Bohnet et al. 2013), joint syntactic and semantic
parsing (Henderson et al. 2013). Such approaches are now becoming prominent
in NLP alongside the deep learning revolution. In fact, most joint approaches to
statistical MWE-aware parsing are not truly joint, as they consist of workaround
solutions. We saw how many studies investigated the use of off-the-shelf parsers
by modifying training data, thus making the datasets MWE-aware. Truly joint
systems are rarer, requiring the use of specific grammatical formalisms for con-
stituency parsing or the development of new dependency parsing mechanisms
dedicated to MWE identification.

As a consequence, there is much ground for future work. However, special
emphasis should be given to the development of MWE-rich treebanks. Not only
are these resources lacking for many languages, but also the representation and
covered MWE types vary considerably among different resources. We believe
that the development of new MWE-aware parsing models and resources would
enable satisfactory solutions for this hard problem. Such solutions could then
be further integrated into downstream applications, taking a significant step to-
wards semantic processing of MWEs, and thus of a key element of language itself.

## Acknowledgements

This work was supported by the PARSEME action (COST IC1207). It was also
partially funded by the French National Research Agency (ANR) through the
PARSEME-FR project (ANR-14-CERA-0001) and by a TUBITAK 1001 grant (no:
112E276).

## References

Angelov, Krasimir. 2019. Multiword expressions in multilingual applications within the Grammatical Framework. In Yannick Parmentier & Jakub Waszczuk (eds.), *Representation and parsing of multiword expressions: Current trends*, 127–146. Berlin: Language Science Press. DOI:10.5281/zenodo.2579041

Arun, Abhishek & Frank Keller. 2005. Lexicalization in crosslinguistic probabilistic parsing: The case of French. In *Proceedings of the 43rd annual meeting of the Association for Computational Linguistics (ACL’05)*, 306–313. Ann Arbor, Michigan: Association for Computational Linguistics. http://aclweb.org/anthology/P05-1038.

Bansal, Mohit & Dan Klein. 2011. Web-scale features for full-scale parsing. In *Proceedings of the 49th annual meeting of the Association for Computational Linguistics: Human Language Technologies (ACL-HLT’11)*, 693–702. Portland, Oregon. http://www.aclweb.org/anthology/P11-1070.

Bejček, Eduard, Jarmila Panevová, Jan Popelka, Pavel Straňák, Magda Ševčíková, Jan Štěpánek & Zdeněk Žabokrtský. 2012. Prague Dependency Treebank 2.5 – A revisited version of PDT 2.0. In *Proc. of COLING 2012*, 231–246. Bombay, India.
Black, E., S. Abney, S. Flickenger, C. Gdaniec, C. Grishman, P. Harrison, D. Hindle, R. Ingria, F. Jelinek, J. Klavans, M. Liberman, M. Marcus, S. Roukos, B. Santorini & T. Strzalkowski. 1991. Procedure for quantitatively comparing the syntactic coverage of English grammars. In *Proceedings of the workshop on Speech and Natural Language*, 306–311. Pacific Grove, California: Association for Computational Linguistics. DOI:10.3115/112405.112467

Blunsom, Phil & Timothy Baldwin. 2006. Multilingual deep lexical acquisition for HPSGs via supertagging. In *Proceedings of the 2006 conference on Empirical Methods in Natural Language Processing (EMNLP 2006)*, 164–171. Sydney.

Bohnet, Bernd, Joakim Nivre, Igor Boguslavsky, Richard Farkas, Filip Ginter & Jan Hajic. 2013. Joint morphological and syntactic analysis for richly inflected languages. *Transactions of the Association for Computational Linguistics (TACL)* 1. 415–428.

Buchholz, Sabine & Erwin Marsi. 2006. CoNLL-X shared task on multilingual dependency parsing. In *Proceedings of the tenth conference on Computational Natural Language Learning (CoNLL-X 2006)*, 149–164. New York, NY.

Cafferkey, Conor, Deirdre Hogan & Josef van Genabith. 2007. Multi-word units in treebank-based probabilistic parsing and generation. In *Proceedings of the international conference on Recent Advances in Natural Language Processing (RANLP 2007)*. Borovets, Bulgaria.

Candito, Marie & Mathieu Constant. 2014. Strategies for contiguous multiword expression analysis and dependency parsing. In *Proceedings of the 52nd annual meeting of the Association for Computational Linguistics (volume 1: long papers)*, 743–753. Baltimore, Maryland: Association for Computational Linguistics. http://aclweb.org/anthology/P14-1070.

Candito, Marie & Benoît Crabbé. 2009. Improving generative statistical parsing with semi-supervised word clustering. In *Proc. of the 11th International Conference on Parsing Technologies (IWPT’09)*, 138–141. Paris, France: Association for Computational Linguistics. http://www.aclweb.org/anthology/W09-3821.

Candito, Marie, Joakim Nivre, Pascal Denis & Enrique Henestroza Anguiano. 2010. Benchmarking of statistical dependency parsers for French. In *Proceedings of COLING 2010, 23rd international conference on Computational Linguistics*, *posters volume*, 108–116. Beijing, China.

Candito, Marie & Djamé Seddah. 2010. Parsing word clusters. In *Proceedings of the NAACL HLT 2010 first workshop on Statistical Parsing of Morphologically-Rich Languages*, 76–84. Los Angeles, California.

Carroll, John, Guido Minnen & Edward Briscoe. 2003. Parser evaluation: Using a grammatical relation annotation scheme. In Anne Abeillé (ed.), *Treebanks: Building and using parsed corpora*, 299–316. Dordrecht: Kluwer.
Charniak, Eugene. 2000. A maximum-entropy-inspired parser. In *Proceedings of the 1st North American chapter of the Association for Computational Linguistics conference*, 132–139. Seattle, Washington.

Collins, Michael. 1996. A new statistical parser based on bigram lexical dependencies. In *Proceedings of the 34th annual meeting of the Association for Computational Linguistics (ACL 1996)*, 184–191. Santa Cruz, California.

Collins, Michael. 1999. *Head-driven statistical models for natural language parsing*. Philadelphia, PA: University of Pennsylvania Ph.D. thesis.

Constant, Mathieu, Marie Candito & Djamé Seddah. 2013. The LIGM-Alpage architecture for the SPMRL 2013 shared task: Multiword expression analysis and dependency parsing. In *Proceedings of the fourth workshop on Statistical Parsing of Morphologically-Rich Languages*, 46–52. Seattle, Washington, USA: Association for Computational Linguistics. http://aclweb.org/anthology/W13-4905.

Constant, Mathieu, Joseph Le Roux & Anthony Sigogne. 2013. Combining compound recognition and PCFG-LA parsing with word lattices and conditional random fields. *ACM Transaction on Speech and Language Processing (TSLP), Special Issue on MWEs* 10(3).

Constant, Mathieu, Joseph Le Roux & Nadi Tomeh. 2016. Deep lexical segmentation and syntactic parsing in the easy-first dependency framework. In *Proceedings of the 15th annual conference of the North American chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL HLT 2016)*, 1095–1101. San Diego, California.

Constant, Mathieu & Joakim Nivre. 2016. A transition-based system for joint lexical and syntactic analysis. In *Proceedings of the 54th annual meeting of the Association for Computational Linguistics*, vol. 1: *Long papers*, 161–171. Berlin, Germany: Association for Computational Linguistics. http://www.aclweb.org/anthology/P16-1016.

Constant, Mathieu, Anthony Sigogne & Patrick Watrin. 2012. Discriminative strategies to integrate multiword expression recognition and parsing. In *Proceedings of the 50th annual meeting of the Association for Computational Linguistics: Long papers*, vol. 1 (ACL ’12), 204–212. Jeju Island, Korea: Association for Computational Linguistics. http://dl.acm.org/citation.cfm?id=2390524.2390554.

Denis, Pascal & Benoît Sagot. 2012. Coupling an annotated corpus and a lexicon for state-of-the-art POS tagging. *Language Resources and Evaluation* 46(4). 721–736. DOI:10.1007/s10579-012-9193-0

Durrett, Greg & Dan Klein. 2015. Neural CRF parsing. In *Proceedings of the 53rd annual meeting of the Association for Computational Linguistics and the 7th*
*International Joint Conference on Natural Language Processing*, vol. 1: Long Papers, 302–312. Beijing.

Dyer, Chris, Miguel Ballesteros, Wang Ling, Austin Matthews & Noah A. Smith. 2015. Transition-based dependency parsing with stack long short-term memory. In *Proc. of ACL 2015*, 334–343. Beijing.

Dyvik, Helge, Paul Meurer, Victoria Rosén, Koenraad De Smedt, Petter Haugereid, Gyri Smørdal Losnegaard, Gunn Inger Lyse & Martha Thunes. 2016. NorGramBank: A ‘deep’ treebank for Norwegian. In Nicoletta Calzolari, Khalid Choukri, Thierry Declerck, Marko Grobelnik, Bente Maegaard, Joseph Mariani, Asunción Moreno, Jan Odijk & Stelios Piperidis (eds.), *Proceedings of the tenth international conference on Language Resources and Evaluation (LREC 2016)*, 3555–3562. Portorož, Slovenia. http://www.lrec-conf.org/proceedings/lrec2016/summaries/943.html.

Eisner, Jason M. 1996. Three new probabilistic models for dependency parsing: An exploration. In *Proceedings of the 16th conference on Computational Linguistics (ACL 1996)*, 340–345. Santa Cruz, California.

Eryiğit, Gülşen, Tugay İlbay & Ozan Arkan Can. 2011. Multiword expressions in statistical dependency parsing. In *Proceedings of the second workshop on Statistical Parsing of Morphologically Rich Languages*, 45–55. Dublin, Ireland.

Fazly, Afsaneh, Paul Cook & Suzanne Stevenson. 2009. Unsupervised type and token identification of idiomatic expressions. *Computational Linguistics* 35(1). 61–103.

Foufi, Vasiliki, Luka Nerima & Eric Wehrli. 2019. Multilingual parsing and MWE detection. In Yannick Parmentier & Jakub Waszczuk (eds.), *Representation and parsing of multiword expressions: Current trends*, 217–237. Berlin: Language Science Press. DOI:10.5281/zenodo.2579047

Głowińska, Katarzyna & Adam Przepiórkowski. 2010. The design of syntactic annotation levels in the National Corpus of Polish. In *Proc. of the seventh international conference on Language Resources and Evaluation (LREC 2010)*. Valletta, Malta.

Green, Spence, Marie-Catherine de Marneffe, John Bauer & Christopher D. Manning. 2011. Multiword expression identification with tree substitution grammars: A parsing tour de force with French. In *Proc. of EMNLP 2011*, 725–735. Edinburgh.

Green, Spence, Marie-Catherine de Marneffe & Christopher D. Manning. 2013. Parsing models for identifying multiword expressions. *Computational Linguistics* 39(1). 195–227.
Gross, Maurice. 1984. Lexicon-grammar and the syntactic analysis of French. In *Proceedings of the 10th international conference on Computational Linguistics and 22nd annual meeting of the Association for Computational Linguistics*, 275–282. Stanford, California, USA: Association for Computational Linguistics.

Grover, Claire. 2008. *LT-TTT2: Example pipelines documentation*. Tech. rep. Edinburgh Language Technology Group.

Henderson, James, Paola Merlo, Ivan Titov & Gabriele Musillo. 2013. Multilingual joint parsing of syntactic and semantic dependencies with a latent variable model. *Computational Linguistics* 39(4). 949–998.

Kato, Akihiko, Hiroyuki Shindo & Yuji Matsumoto. 2016. Construction of an English dependency corpus incorporating compound function words. In *Proceedings of the tenth international conference on Language Resources and Evaluation (LREC 2016)*. Portorož, Slovenia: European Language Resources Association (ELRA).

Koo, Terry, Xavier Carreras & Michael Collins. 2008. Simple semi-supervised dependency parsing. In *Proceedings of ACL-08: HLT*, 595–603. Columbus, Ohio.

Korkontzelos, Ioannis & Suresh Manandhar. 2010. Can recognising multiword expressions improve shallow parsing? In *Proc. of the 11th annual conference of the North American chapter of the Association for Computational Linguistics: Human Language Technologies NAACL/HLT 2010*, 636–644. Los Angeles, California.

Le Roux, Joseph, Antoine Rozenknop & Mathieu Constant. 2014. Syntactic parsing and compound recognition via dual decomposition: Application to French. In *Proc. of COLING 2014*. Dublin, Ireland.

Lehmann, Hans Martin & Gerold Schneider. 2011. A large-scale investigation of verb-attached prepositional phrases. In S. Hoffmann, P. Rayson & G. Leech (eds.), *Studies in variation, contacts and change in English, volume 6: Methodological and historical dimensions of corpus linguistics*. Helsinki: Varieng.

McDonald, Ryan, Kevin Lerman & Fernando Pereira. 2006. Multilingual dependency analysis with a two-stage discriminative parser. In *Proceedings of the tenth conference on Computational Natural Language Learning (CoNLL-X)*, 216–220. New York City: Association for Computational Linguistics. http://www.aclweb.org/anthology/W/W06/W06-2932.

Miller, George Armitage. 1956. The magical number seven, plus or minus two: Some limits on our capacity for processing information. *Psychological Review* 63. 81–97.

Mirroshandel, Seyed Abolghasem & Alexis Nasr. 2016. Integrating selectional constraints and subcategorization frames in a dependency parser. *Computational Linguistics* 42(1). 55–90.