Previous work demonstrated that web counts can be used to approximate bigram frequencies, and thus should be useful for a wide variety of NLP tasks. So far, only two generation tasks (candidate selection for machine translation and confusion-set disambiguation) have been tested using web-scale data sets. The present paper investigates if these results generalize to tasks covering both syntax and semantics, both generation and analysis, and a larger range of n-grams. For the majority of tasks, we find that simple, unsupervised models perform better when n-gram frequencies are obtained from the web rather than from a large corpus. However, in most cases, web-based models fail to outperform more sophisticated state-of-theart models trained on small corpora. We argue that web-based models should therefore be used as a baseline for, rather than an alternative to, standard models.
More recently, REF showed that simple unsupervised models perform significantly better when the frequencies are obtained from the web, rather than from a large standard corpus.
The Web As A Baseline: Evaluating The Performance Of Unsupervised Web-Based Models For A Range Of NLP Tasks
Human Language Technology Conference And Meeting Of The North American Association For Computational Linguistics
Computer Science