# Extracting and Analyzing English Multi-word Expressions with Slots: A Case Study of 'take'

Tomoyuki Tsuchiya
Kyushu University
tsuchiya@flc.kyushu-u.ac.jp

## 1 Introduction

Multi-word expression (MWE) has been argued as one of the significant issues for large-scale natural language processing [1], and similar issues have also been coming into discussions in corpus linguistics and cognitive linguistics areas. Conventionalized expressions have been investigated from various aspects- syntactically, semantically and sociolinguistically for instance- and many terms have been applied to this linguistic phenomenon (ex. idioms, formulaic language, construction). One of these terms called 'construction' refers to a type of MWEs whose parts are not fully fixed. This paper discusses the theoretical issues of construction and reports the results of a preliminary investigation using a large-scale web corpus.

Construction Grammar $(\mathrm{CxG})$ is a highly explanatory linguistic theory that aims to reveal the knowledge and productivity of human language based on the speakers' actual usage. CxG assumes conventionalized sequences of linguistic elements called construction and one or more empty slots in each construction to accept certain lexical items to fit in and generate speech with a certain level of productivity. The linguistic unit of construction can vary into several levels; word, morpheme, or other smaller units, and the level of abstraction of a slot can also vary; partof-speech, semantics, morphology, and grammatical level. Moreover, many constructions are conventionalized and shared in the language community with nonliteral meaning (ex. let alone in "Fred will not eat shrimp, let alone squid" from [2]).

This paper investigates a method to extract constructions with slots with certain selection restrictions and word range that are linguistically objective from large-scale corpus data.

## 1.1 Constructions in Linguistics: A Frame and slots

$\mathrm{CxG}$ aims to explain the productivity of language while maintaining the usage-based aspect of language.

The collostructional analysis established by [3] uses a statistical method to investigate each lexical item's productivity that fits in the slot. The term 'collostruction' designates relatively more concrete expressions closer to collocation.

## 1.2 Remaining Issues of CxG and collostructional analysis

## 1.2.1 Frame Range of Constructions

Methodologically, collostructional analysis focuses mainly on the type and collocational strength of collexemes but the definition of the starting and end points of each collocation is arbitrary. For instance, [3] conducted a case study of [ $X$ think nothing of $\left.V_{\text {gerund }}\right]$ and calculated the collocational strength of each collexeme. The word sequence of this study starts from the subject noun phrase as $\mathrm{X}$ and ends with gerund verb. [3] investigated this sequence since it appears in the dictionary and $\mathrm{V}_{\text {gerund }}$ slot has productivity to a certain extent, so designating $\mathrm{X}$ as the starting point and $\mathrm{V}$ slot as the end point of the sequence should be taken into consideration.

## 1.2.2 Slots of Constructions

Moreover, the collocational strength indicates the productivity of the slots of a collostruction, however, the slots of the construction are also designated arbitrarily by the researchers themselves. For instance, $[X$ think nothing of $\left.V_{\text {gerund }}\right]$ has two empty slots; $\mathrm{X}$ and $\mathrm{V}_{\text {gerund }}$, but it is more or less possible that the word nothing can become a slot to form a more abstract construction $\left[X\right.$ think $Y$ of $\left.V_{\text {gerund }}\right]$.

Figure 1 The concordance list of sequences start with take

## 1.2.3 Towards objective designation of the frame range and the slots

By establishing an objective and simple method to extract constructions with productive slots, it would be possible to offer an index that contains useful constructions for language learners.

## 2 Data and Method

## 2.1 Data: EnTenTen Corpus

[1] argue the lexical proliferation problem caused by light verb construction with examples; take a walk, take a hike, take a trip, and take a flight. In order to assess the plausibility of light verb constructions as MWE, this paper focuses on MWEs which starts from a light verb take.

For collecting data, the EnTenTen 15 corpus was used in this study. EnTenTen15 corpus is an English web corpus, which collects text from the Internet. It contains 13 billion words, and the part-of-speech was tagged on each word by TreeTagger (version 3) using Penn Treebank tagset.

## 2.2 Method

Using Corpus Query Language (CQL) in the Sketch Engine, expressions consist of 7 words and start from the light verb take was collected from the corpus. Any punctuations and symbols are not included in this word sequence. The first 10 million hits were used for the observation due to the Sketch Engine's system default. In most cases, this number is sufficiently representative, so random sampling was not applied in this data collection process.

Subsequently, by counting the frequency of tokens with the Sketch Engine's frequency function, 8,950,602 tokens were found in total. In the counting process, any inflection of words (e.g. declension of nouns and conjugation of verbs) was ignored. From this frequency data, 57 patterns that have at least one slot were made for each token as

Figure 2 The frequency list of tokens from the concordance

table 1. Two or more adjacent slots were combined into one slot. For each pattern, type frequency ( $t \_f r e q$ ), total token frequencies (freq_sum), the mean of type frequencies (mean_freq), and the percentage of the most frequent token in the total token frequency (max_freq_perc) were calculated.

Table 1 Slot generation patterns

Finally, patterns that match the following three conditions were extracted from the pattern data; more than three types, more than 1,000 total token frequency, and the most frequent token accounts for more than 10 percent of the total token frequency. Though the constant of each condition above is provisional, it reflects the common characteristic of constructions. The first condition was set in order to extract constructions that have appropriate level of productivity. The second condition was set because constructions with a certain productivity should occur in a certain amount in language activities. The third condition was set since collocational analyses by [3] indicate the existence of a prototypical lexical item, which commonly fits in the slot.

Regarding the procedue above, three types of data have been generated; the raw data which was collected from the actual corpus, the frequency tokens of the raw data, and the abstract patterns that generated from each token. The examples of each data type are as follows.

1. Raw data: ... film and exhibition will take place at The Lights in Andover on Tuesday 26. . .

2. Frequency token: take place at The Lights in An-
dover, take place during break and over lunch, take time out of his busy schedule

## 3. Abstract pattern:

patterns from take place at The Lights in Andover...
- take ... at ... Andover,
- take ... at ... in,
- take ... at ... in Andover

-...
- take ... Andover

patterns from take time out of his busy schedule...
- take time ... of his busy schedule
- take time out ... his busy schedule
- take time out of ... busy schedule

-...
- take ... schedule

## 3 Results and their observation

Consequently, there were 1,693 patterns that meet all of the conditions above. The examples of the extracted patterns are listed in table 3.

Table 2 Basic statistics of the extracted patterns

As shown in table 4, "take a ... of" has the most types among all patterns and appears the most in the corpus. Most patterns have a highly prototypical token, which account for more than $90 \%$ of the tokens appear with the pattern (ex. take a look ... some of the, take ... life ... its), but there are patterns whose prototypical token accounts for less than $90 \%$ (ex. on a life ... own, take ... law into ... own hand). The tokens appear in the pattern on a life ... own were take on a life fof its/their/his/her/all its/on its/on their\} own, and the tokens appear in the pattern take ... law into ... own hand were take \{the/immigration/international/federal/religious\} law into \{their/his/your/its/our/her/one/my/\} own hand.

## 4 Discussion and Conclusion

As in table 4, the raw frequency of abstract patterns is higher compared to less abstract patterns. Abstract constructions appear in the corpus more frequently, but the selection restriction of their slots is loose, so various lexi- cal items are acceptable. Therefore, the type frequency of actual expression is higher and do not have a prototypical token.

We can also estimate the end point of the construction by analyzing pattenrs that shares a common prototype token but have different endpoints. For example, among the patterns that share the prototypical token take away the sin of the world, the patterns which end with "world" have fewer types than ones that end with "the" in table 6. In terms of the conventionality of language, the patterns that end with "world" are more conventionalized and restrict the repertory of lexical items that fit in the remaining slots.

The slot-ness of each pattern can be estimated by the number of types that appear in the token list. In table 5, for instance, "your ... to ... level" has 904 types in the token list, whereas "your ... the ... level" has approximately 200 types fewer than the former pattern. Though it is not statistically tested, we can estimate that the first pattern has more productive slots than the second one.

This paper pointed out the theoretical difficulties of $\mathrm{CxG}$ in terms of the arbitrariness of the designation of the word range and slots and investigated an objective method to designate them by analysing the frequencies of word sequences which start from a light verb take. To extract constructions that are beneficial for English learning and create an index of it, the data shown in this paper should be observed in terms of their productivity.

## Acknowledgment

This work was supported by JSPS KAKENHI Grant Numbers JP17KT0061 and JP17K17943.

## References

[1] Ivan A Sag, Timothy Baldwin, Francis Bond, Ann Copestake, and Dan Flickinger. Multiword expressions: A pain in the neck for nlp. In International Conference on Intelligent Text Processing and Computational Linguistics, pp. 1-15. Springer, 2002.

[2] Charles J Fillmore, Paul Kay, and Mary C. O’onnor. Regularity and idiomaticity in grammatical constructions: The case of let alone. Language, Vol. 64, No. 3, pp. 501-538, 1988.

[3] Anatol Stefanowitsch and Stefan Th Gries. Collostructions: Investigating the interaction of words and constructions. International journal of corpus linguistics, Vol. 8, No. 2, pp. 209-243, 2003.

Table 3 Pattern examples and basic statistics ( $15 \leq \mathrm{t}_{-}$freq $\left.\leq 17\right)$

Table 4 The list of patterns with the highest $t_{-}$freq frequency $\left(5,000 \leq t_{-} f r e q\right)$

Table 5 A group of patterns that share a common prototype

Table 6 A group of patterns that share take away the sin of the world as the prototype ( $t_{-}$freq $\geq 30$ )