Constructing Typing-Time Corpora: A New Way to Answer Old Questions

Constructing Typing-Time Corpora: A New Way to Answer Old Questions Uriel Cohen Priva (urielc@stanford.edu) Department of Linguistics, 450 Serra Mall Stanford, CA 94305 USA questions at hand. The creation of even a small-scale corpus is an expensive and time-consuming project, and therefore, there is much gain in finding a simpler alter- native. In this paper, we propose a solution to this problem. We show that by wedding two methodologi- cal advancements, tracking typing speed and collection of data over the web, we can create an alternative both to experiments which require phonetic labeling and to phonetically-labeled corpora, typing-time corpora — cor- pora of typed data in which each letter and word is an- notated with the time it took to type. Abstract Many current studies in linguistics and psycholinguistics require the use of phonetically labeled speech data. Col- lecting and annotating such data is expensive and slow. An alternative approach makes use of pre-labeled speech corpora, but these are available for very few languages, might not contain the desired linguistic environment, and the construction of new ones is still expensive and time-consuming. We present a fast and cost-efficient method for constructing a new type of corpus which retains many of the advantages of phonetically labeled speech, typing-time corpora. In this paper we show that an English typing-time corpus collected over the web is sufficient to replicate word frequency and neighborhood density effects. We then demonstrate the transferability of this method to less studied languages and to different orthographies. We show that a smaller Hebrew typing corpus collected over the web can be used to find length- ening effects in infrequent Hebrew words. A number of studies (Weingarten, Nottbusch, & Will, 2004; Zesiger, Orliaguet, Bo¨e, & Mounoud, 1994 among others) demonstrate that typing is sensitive to language- based effects. Weingarten et al. (2004) show that typing is sensitive to phonological and morphological proper- ties of the words being typed. Zesiger et al. (1994) show that actual words are typed faster than pseudo-words and that frequent words are typed faster than infrequent words. These effects demonstrate that even though a typing task is different from spoken speech production, it does exhibit linguistic effects that are normally asso- ciated with speech. Keywords: Typing-time; Corpora; Frequency; Neigh- borhood density; Amazon Mechanical Turk Introduction Many studies in linguistics and psycholinguistic require either the precise annotation of durations and latencies for speech data gathered in carefully controlled exper- iments, or the availability of phonetically labeled cor- pora. For example, evidence for neighborhood density in production depends on measuring speech production latencies (Vitevitch, 2002). The study of production dif- ficulties relies on measuring the lengthening of words in a difficult context (Fox Tree & Clark, 1997). Stud- ies of frequency and predictability-dependent phonetic reduction (Van Son & Van Santen, 2005; Pluymaek- ers, Ernestus, & Baayen, 2005; Aylett & Turk, 2006; Bell, Brenier, Gregory, Girand, & Jurafsky, 2009) make use of corpora containing exact word and phone du- rations, such as the Switchboard Corpus (Godfrey & Holliman, 1997) and Buckeye Corpus of Conversational Speech (Pitt et al., 2007) for English, the Spoken Dutch Corpus (Oostdijk, 2000) and the Kiel Corpus of Spon- taneous Speech (Kohler, P¨ atzold, & Simpson, 1995) for German. However, neither the experimental approach nor the corpus-based one may be a feasible option when trying to address the problem of data availability in less stud- ied languages. Subjects may not always be available on the one hand, and Switchboard-like corpora do not ex- ist for most languages on the other hand. In addition, even when a corpus is accessible, it might not contain the relevant linguistic environments for addressing the Not only does typing time provides a window to lin- guistic performance, but it also holds a big advantage, as it allows the automatic gathering of large amounts of data through the web. A simple way to utilize this is by using Amazon Mechanical Turk (AMT), a vir- tual work marketplace created by Amazon.com. On AMT, requesters can upload work requests in the form of HTML pages, which workers can access online. Sev- eral researchers in the natural language processing com- munity (Callison-Burch, 2009; Colowick & Pool, 2007; Snow, O’Connor, Jurafsky, & Ng, 2008 among others) make use of AMT to construct corpora for which human- labeled data is not available, or to annotate new data sets. In this paper, we demonstrate that extending the use of AMT to the construction of typing-time corpora provides an easy and cost-efficient alternative to labora- tory experiments and extant corpora. We show evidence that supports the applicability of this methodology by constructing a typing time corpus for English, and using it to replicate two well known effects on language pro- duction: word frequency (Bell et al., 2009) and neighbor- hood density (Coltheart, Davelaar, Jonasson, & Besner, 1977; Vitevitch, 2002; Adelman & Brown, 2007). We then extend these results to a less studied language. We