Speech retrieval using phonemes with error correction

In speech retrieval, text queries are matched to spoken-word recordings. We undertook a series of retrieval experiments on such speech documents, using both phonemes and words for matching text to speech. We found that, given a reliable automatic word-based process for speech recognition, phoneme-based retrieval can be as effective as word-based retrieval. However, retrieval is v. orse with phoneme-based speech recognition. Error correction techniques, such as manual error correction of phoneme sequences and use of a standard string edit distance, did not improve the effectiveness of phoneme-based retrieval. In speech retrieval, textual queries are used to retrieve speech documents from a corpus. An approach to speech retrieval is to preprocess the speech documents with a speech recognition system, thus converting the audio signals into indexable tokens such as words or phonemes. The word-based approach to speech recognition has two key problems. First, it requires a dictionary. Unknown words, those not present in the dictionary, will not be recognised, a problem that is likely to be acute for names. Second, the detection of word boundaries can be difficult because there is no explicit delimiting of words in continuous speech. These problems are likely to decrease retrieval effectiveness, particularly because more discriminating query terms are rare and therefore less likely to be recognised. We investigated a phoneme-based approach to both the recognition and retrieval processes. Prior to retrieval, a recogniser is used to extract phonemes from the audio documents. A phoneme is the smallest context-free unit of speech that has meaning; the physical sound of a phoneme is called a phone. A small, fixed set of phones is used for phoneme recognition. The two difficulties with wordbased recognition do not arise with phonemes. First, words are represented as a sequence of phonemes, and words known or unknown are represented by their component phones. Second, recognition of word boundaries is unnecessary. However, phoneme recognition is more error-prone than word recognition. Phoneme boundaries are difficult to detect, and, in word recognition, semantic context can be used to aid the recognition process; no such context is available for phoneme recognition. Once the recognition is complete, index features must be extracted from the documents, to allow efficient retrieval on a large corpus. In text documents, the features exPermission to make digital/hard copy of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage, the copyright notice, the title of the publication and its date appear, and notice is given that copying is by permission of ACM, Inc. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or fee. SIGIR’98, Melbourne, Australia @ 1998 ACM l-58113-015-5 8198 $5.00. tracted are usually words. In speech documents, individual phonemes are not sufficiently discriminating; we chose to index all phoneme strings of length n, or n-grams, where individual n-grams were allowed to overlap. An example of 4-grams, or quadgrams, is shown in Figure 1 for the phrase “a question about”, where word boundary information has been discarded. N-gram indexing has been shown to be language independent [l]. As for word-based retrieval, n-gram-based retrieval has potential limitations. An n-gram created from a query term can match any word containing that n-gram, and in the document collection the n-gram may be formed across a word boundary, that is, by composing the end of one word with the start of another. These false matches can degrade effectiveness. The document collection used in our experiments consists of 1500 speech documents and 49 queries, provided by Text Retrieval Conference (TREC) sponsored by NIST and DARPA to encourage research in information retrieval. For our experiments, three transcriptions of the speech documents were used. These are a manual transcription in words, an automatic transcription in words, and an automatic transcription in phonemes. The queries and documents were neither stopped nor stemmed. There is also a training set of about 1500 documents. This set of documents was used to train the phoneme recognition system. The Carnegie-Mellon Pronouncing Dictionary (CMU) was used to translate words to phonemes. In this speech document collection, there are approximately 18,000 unique words. Of these, 2000 are not in the dictionary. Most of these words are names and plurals. We manually added these pronunciations, thus mimicking the behaviour of a word-based speech recogniser that outputs phonemes. The text retrieval engine MG [4] was used for the retrieval experiments. For each query, 1000 documents are retrieved. A standard cosine formulation was used to estimate similarity. Average precision was used to measure retrieval performance. For most of the queries for this collection there is only one relevant document per query. Hence, precision can be computed as the mean reciprocal of the rank. Average precision of the set of queries & is calculated as: Average precision = C,‘oEy)-l where rank, is the rank at which the relevant document for query Q is found. To establish baselines we used the word-based manual and automatic transcriptions of the speech documents; results are shown in Table 1. These transcriptions were also translated to phonemes, to simulate the best possible retrieval results given minimum transcription errors. For these experiments, trigrams and quadgrams are used. The first two columns (unbounded) of Table 2 show effectiveness using queries where n-grams were created from