Abstract:
A method for comparing a first audio data source with a plurality of audio data sources, wherein the first audio data source has an utterance spoken by a first person and the plurality of audio data sources have the same utterance spoken by a second person. The method includes performing a speech recognition function on the first audio data source to isolate at least one element of the first audio data source. The method also includes comparing the isolated element with a corresponding element in the plurality of audio data sources and determining whether the utterance spoken by the first person contained an error based on the comparison.

Description:
BACKGROUND  
       [0001]     Various computer-assisted products exist that enable a person to learn a non-native language. In such products, a person speaks an utterance into a microphone and the products teach lexicon and syntax by giving the person a general idea of errors in pronunciation. Such products do not immediately pinpoint exact errors in pronunciation and thus do not typically provide useful feedback that the speaker needs to master a non-native language.  
       SUMMARY  
       [0002]     In one embodiment, the present invention is directed to a method for comparing a first audio data source with a plurality of audio data sources, wherein the first audio data source has an utterance spoken by a first person and the plurality of audio data sources have the same utterance spoken by a second person. The method includes performing a speech recognition function on the first audio data source to isolate at least one element of the first audio data source. The method also includes comparing the isolated element with a corresponding element in the plurality of audio data sources and determining whether the utterance spoken by the first person contained an error based on the comparison.  
         [0003]     In one embodiment, the present invention is directed to an apparatus. The apparatus includes an audio capture module for receiving a first spoken utterance, wherein the first spoken utterance is spoken in a language that is not native to a speaker of the first spoken utterance, and a comparison module for comparing an element of the first spoken utterance with a corresponding element in audio data from a plurality of audio data sources, wherein the audio data from a plurality of audio data sources are spoken in the language, wherein the language is native to each speaker of the audio data from a plurality of audio data sources, and wherein the first spoken utterance and the audio data from a plurality of audio data sources are substantially similar in content.  
         [0004]     In one embodiment, the present invention is directed to a system. The system includes an audio input device and a comparison device in communication with the audio input device. The comparison device includes an audio capture module for receiving a first spoken utterance, wherein the first spoken utterance is spoken in a language that is not native to a speaker of the first spoken utterance and a comparison module for comparing an element of the first spoken utterance with a corresponding element in audio data from a plurality of audio data sources, wherein the audio data from a plurality of audio data sources are spoken in the language, wherein the language is native to each speaker of the audio data from a plurality of audio data sources, and wherein the first spoken utterance and the audio data from a plurality of audio data sources are substantially similar in content.  
         [0005]     In one embodiment, the present invention is directed to an apparatus. The apparatus includes means for performing a speech recognition function on a first audio data source to isolate at least one element of the first audio data source, wherein the first audio data source has an utterance spoken by a first person. The apparatus also includes means for comparing the isolated element with a corresponding element in a plurality of audio data sources, wherein the plurality of audio data sources have the same utterance as that spoken by the first person that is spoken by a second person and means for determining whether the utterance spoken by the first person contained an error based on the comparison.  
         [0006]     In one embodiment, the present invention is directed to a computer readable medium having stored thereon instructions which, when executed by a processor, cause the processor to perform a speech recognition function on a first audio data source to isolate at least one element of the first audio data source, wherein the first audio data source has an utterance spoken by a first person, compare the isolated element with a corresponding element in a plurality of audio data sources, wherein the plurality of audio data sources have the same utterance as that spoken by the first person that is spoken by a second person, and determine whether the utterance spoken by the first person contained an error based on the comparison. 
     
    
     BRIEF DESCRIPTION OF THE DRAWINGS  
       [0007]     Further advantages of the present invention may be better understood by referring to the following description taken in conjunction with the accompanying drawings, in which:  
         [0008]      FIG. 1  is a diagram illustrating a system for comparing speech elements according to one embodiment of the present invention; and  
         [0009]      FIG. 2  is a flow diagram illustrating a process of comparing speech elements according to one embodiment of the present invention.  
     
    
     DESCRIPTION  
       [0010]     It is to be understood that the figures and descriptions of the present invention have been simplified to illustrate elements that are relevant for a clear understanding of the present invention, while eliminating, for purposes of clarity, other elements such as, for example, various operating systems details and components. Those of ordinary skill in the art will recognize, however, that these and other elements may be desirable. However, because such elements are well known in the art, and because they do not facilitate a better understanding of the present invention, a discussion of such elements is not provided herein.  
         [0011]     The present invention is directed generally and in various embodiments to systems and methods for comparing a first audio file with a second audio file to ascertain, or pinpoint, features of the first audio file. In one embodiment, the first audio file may be a word, phrase, sentence, etc. that is spoken by a first person in a language that is non-native to that person and the second audio file may be the same word, phrase, sentence, etc. that is spoken by a second person for whom the language is native.  
         [0012]      FIG. 1  is a diagram illustrating a system  10  for comparing speech elements according to one embodiment of the present invention. The system includes an audio input device  12 , such as a microphone, into which a user of the system  10  can speak or otherwise input voice data to a comparison device  14 . The voice data may be, for example, utterances such as, for example, spoken words, phrases, sentences, etc. The system  10  also includes an input device  16  such as, for example, a keyboard or a mouse, and a display  18  such as, for example, a computer or video screen or monitor. The audio input device  12 , the input device  16 , and the display  18  are in communication with the comparison device  14  via an input/output interface  20 .  
         [0013]     An audio capture module  22  captures audio data input from the audio input device  12  and stores the data in an audio database  24 . The audio data may be stored in the database  24  in, for example, a digital audio format such as, for example, in .wav format. A speech recognition interface  26  transfers the audio data to a speech recognizer  28 . The speech recognizer  28  may be any type of suitable speech recognizer such as, for example, the CMU Sphinx Open Source speech recognizer. The speech recognizer  28  translates the audio data signal into its component sounds (i.e. its phones) to determine the duration of each sound and to attach a score to each sound.  
         [0014]     As described further hereinbelow in conjunction with  FIG. 2 , a comparison module  30  compares the sounds from the speech recognizer with those that are stored in a native speaker database  32 . The native speaker database  32  contains speech data that correspond to words, phrases, sentences, etc. that are spoken in a language by a speaker whose native tongue is the language in which the words, phrases, sentences, etc. are spoken. The words, phrases, sentences, etc. correspond to those that were spoken by a user of the system  10  and thus input via the audio input device  12 .  
         [0015]     The comparison module  30  directs a display module  34  to output the results of the comparison to the display  18  so that the user of the system may view the results of the comparison.  
         [0016]      FIG. 2  is a flow diagram illustrating a process of comparing speech elements according to one embodiment of the present invention. At step  40 , an audio signal representing the spoken voice of, for example, the user of the system  10  is input via, for example, the audio input device  12 . For example, the user could signal that input is about to begin by pressing a key or button on the input device  16 , speaking into the audio input device  12 , and then signal when input is done by pressing a key or button on the input device  16 . At step  42 , the audio capture module  22  creates an audio data source, such as, for example, an audio file, an in-memory data structure, or other encoding, of the input signal to be stored in the audio database  24 .  
         [0017]     At step  44 , the speech recognizer  28  performs speech recognition functions on each phone in the input audio data source. As an example, the speech recognizer could be operated in a forced alignment mode at step  44 . In such a case, the orthographic text corresponding to the input audio signal is sent to the speech recognizer  28  along with the audio data source. The speech recognizer  28  outputs duration elements and recognition elements for each phone in the input audio data source.  
         [0018]     At step  46 , the comparison module  30  isolates each of the duration and recognition elements. At step  48 , the comparison module  30  individually compares the duration and recognition elements that were isolated at step  46  with native speaker duration and recognition elements from a plurality of audio data sources, where the elements corresponding to the same phone that are stored in the native speaker database  32 .  
         [0019]     At step  50 , the comparison module  30  determines whether the differences between both the duration and the recognition elements are within a threshold. Such a threshold could be, for example, based on the auditory confusability of this sound compared to the most proximate other sound in the language. If both of the values of the duration and recognition elements compare within their respective thresholds, the pronunciation of the phone is marked as “good” at step  52 . If the value of either the duration or the recognition elements does not compare within their respective thresholds, the pronunciation of the phone is marked as “bad” at step  54 .  
         [0020]     At step  56 , the display module  34  displays the performance of the user via the display  18  to the user. The performance of the user represents how well, in terms of duration and recognition, the user spoke the phone that was compared with that of at least one native speaker. The performance may be displayed by showing the correct utterance and the incorrect utterance to show the contrast in meaning between the utterances. At step  58 , the process repeats for each phone in the audio input data source that was input to the speech recognizer  28 .  
         [0021]     In one embodiment of the present invention, the methods described herein are embodied in, for example, computer software code that is coded in any suitable programming language such as, for example, C, C++, or microcode. Such computer software code may be embodied in a computer readable medium or media such as, for example, a magnetic storage medium such as a floppy disk or an optical storage medium such as a CD-ROM. Furthermore, in one embodiment of the invention, the system  10  is implemented on any suitable computer such as, for example, a personal computer.  
         [0022]     While several embodiments of the invention have been described, it should be apparent, however, that various modifications, alterations and adaptations to those embodiments may occur to persons skilled in the art with the attainment of some or all of the advantages of the present invention. It is therefore intended to cover all such modifications, alterations and adaptations without departing from the scope and spirit of the present invention as defined by the appended claims.