This article shows that the Web can be employed to obtain frequencies for bigrams that are unseen in a given corpus. We describe a method for retrieving counts for adjective-noun, noun-noun, and verb-object bigrams from the Web by querying a search engine. We evaluate this method by demonstrating: (a) a high correlation between Web frequencies and corpus frequencies; (b) a reliable correlation between Web frequencies and plausibility judgments; (c) a reliable correlation between Web frequencies and frequencies recreated using class-based smoothing; (d) a good performance of Web frequencies in a pseudodisambiguation task. 1 A reviewer points out that information providers such as Lexis Nexis http://www.lexisnexis.com/ might have databases that are even larger than the Web. Lexis Nexis provides full-text access to news sources (including newspapers, wire services, and broadcast transcripts) and legal data (including case law, codes, regulations, legal news, and law reviews). 2 This is the number of pages indexed by Google in December 2002, as estimated by Search Engine Showdown http://www.searchengineshowdown.com/ . Computational Linguistics Volume 29, Number 3 Keller and Lapata Web Frequencies for Unseen Bigrams Obtaining Frequencies from the Web The data sets used in the present experiment were obtained from the British National Corpus (BNC) (see Burnard [1995]). The BNC is a large, synchronic corpus, consisting of 90 million words of text and 10 million words of speech. The BNC is a balanced corpus (i.e., it was compiled so as to represent a wide range of present day British English). The written part includes samples from newspapers, magazines, books (both academic and fiction), letters, and school and university essays, among other kinds of text. The spoken part consists of spontaneous conversations, recorded from volunteers balanced by age, region, and social class. Other samples of spoken language are also included, ranging from business or government meetings to radio shows and phoneins. The corpus represents many different styles and varieties and is not limited to any particular subject field, genre, or register. For the present study, the BNC was used to extract data for three types of predicateargument relations. The first type is adjective-noun bigrams, in which we assume that the noun is the predicate that takes the adjective as its argument. 3 The second predicate-argument type we investigated is noun-noun compounds. For these, we assume that the rightmost noun is the predicate that selects the leftmost noun as its argument (as compound nouns are generally right-headed in English). Third, we included verb-object bigrams, in which the verb is the predicate that selects the object as its argument. We considered only direct NP objects; the bigram consists of the verb and the head noun of the object. For each of the three predicate-argument relations, we gathered two data sets, one containing seen bigrams (i.e., bigrams that occur in the BNC) and one with unseen bigrams (i.e., bigrams that do not occur in the BNC). For the seen adjective-noun bigrams, we used the data of Lapata, McDonald, and Keller (1999) , who compiled a set of 90 bigrams as follows. First, 30 adjectives were randomly chosen from a part-of-speech-tagged and lemmatized version of the BNC so that each adjective had exactly two senses according to WordNet (Miller et al. 1990) and was unambiguously tagged as "adjective" 98.6% of the time. Lapata, McDonald, and Keller used the part-of-speech-tagged version that is made available with the BNC and was tagged using CLAWS4 (Leech, Garside, and Bryant 1994), a probabilistic partof-speech tagger, with error rate ranging from 3% to 4%. The lemmatized version of the corpus was obtained using Karp et al.'s (1992) morphological analyzer. The 30 adjectives ranged in BNC frequency from 1.9 to 49.1 per million words; that is, they covered the whole range from fairly infrequent to highly frequent items. Gsearch (Corley et al. 2001), a chart parser that detects syntactic patterns in a tagged corpus by exploiting a user-specified context-free grammar and a syntactic query, was used to extract all nouns occurring in a head-modifier relationship with one of the 30 adjectives. Examples of the syntactic patterns the parser identified are given in Table 1. In the case of adjectives modifying compound nouns, only sequences of two nouns were included, and the rightmost-occurring noun was considered the head. Bigrams involving proper nouns or low-frequency nouns (less than 10 per million words) were discarded. This was necessary because the bigrams were used in experiments involving native speakers (see Section 3.2), and we wanted to reduce the risk of including words unfamiliar to the experimental subjects. For each adjective, the set of bigrams was divided into three frequency bands based on an equal division of the
REF evaluated the utility of using web search engine statistics for unseen bigram.
Using the Web to Obtain Frequencies for Unseen Bigrams
Computational Linguistics
Computational Linguistics
Computer Science