Searching Spontaneous Conversational Speech held in conjunction with the 30 th Annual International

In this presentation we begin by summarizing the challenges posed by searching spontaneous conversational speech. We then summarize two MITRE efforts to address this challenge. We first describe Audio Hot Spotting (AHS) and then Cross Language Automated Speech Recognition (CLASR). We conclude by outlining some promising opportunities for future research. THE CHALLENGE OF CONVERSATIONAL SPEECH Searching conversational speech is a grand challenge. Telephone conversations alone illustrate the scale of the challenge with over a billion fixed lines worldwide creating 3,785 billion minutes (63B hours) of conversations annually, equivalent to about 15 exabytes of data (ITU 2002). Add to this voluminous and highly variable mobile and wireless communication. In addition, 47,776 radio stations add 70 million hours of original radio programming per year, some of which are conversations. Add to this 31 million hours of original television programming/year from 21,264 stations (CIA World Factbook 2002). To further complicate this, we are faced with thousands (approximately 6,800) of languages and as much as ten thousand dialects globally. While the potential human need to search audio is unknown, even if only a small portion of the roughly 5 billion searches per month on the Internet (searchengine.com) could be satisfied from spoken language collections, this still remains a large requirement. Figure 1. Foreign Language Spoken Language Needs As Figure 1 illustrates, there are over 300 spoken languages with more than one million speakers but only 66 of these are written and for which we have a translation dictionary. Of these, we have ASR and MT for only 44, and only 20 of these are considered “done” in the sense that systems exists for automated transcription and translation. In addition to the challenge of lack of written materials, which we will return to subsequently, there are many challenges beyond scale. These include challenges with language in general, such as polysemy, ambiguity, imprecision, malformedness, intention, and emotion. And in addition to the traditional set of challenges with automated speech recognition such as noise, microphone variability, and speaker disfluencies, the kind of conversational speech that occurs in telephone calls, meetings, interviews has additional challenges including: Multiparty – multiple, interacting speakers Human and Machine – can include human-human conversations and human-machine conversation Talkover – multiple simultaneous speakers talk over speaker turns Spontaneity – unpredictable shifts in speakers, topics, and acoustic environments. Diverse genre – Conversation is found in many venues including meetings, radio/TV talk shows, interviews, town halls, debates, presentations which vary in degree of structure, roles of participants, lengths, degree of formality, and so on. Multiple media – conversational speech is found in audio, video Real time and retrospective – access during the speech event or after Tasks: document routing, (doc/passage/fact) retrieval or question/answering, browsing, tracking entities and events , summarization (e.g., speakers, topics) Multilingual – multiple languages, sometimes from the same speaker Acoustic challenges – spoken conversations often occur over cell phones or hand held radios which come in and out of range and have highly variable signal to noise ratios.