METHOD OF RECOGNIZING SPEECH AND ELECTRONIC DEVICE THEREOF

A method of recognizing a speech and an electronic device thereof are provided. The method includes: segmenting a speech signal into a plurality of sections at preset time intervals; performing a phoneme recognition with respect to one of the plurality of sections of the speech signal by using a first acoustic model; extracting a candidate word of the one of the plurality of sections of the speech signal by using the phoneme recognition result; and performing a speech recognition with respect to the one the plurality of sections the speech signal by using the candidate word.

DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS

Exemplary embodiments are described in greater detail with reference to the accompanying drawings.

FIG. 1is a schematic block diagram illustrating a structure of an electronic device100for performing speech recognition according to an exemplary embodiment. Referring toFIG. 1, the electronic device100includes a speech signal input part110, a speech signal segmenter120, a phoneme recognizer130, a candidate word extractor140, and a speech recognizer150. The electronic device100according to the present exemplary embodiment may be realized as various types of electronic devices such as a smart phone, a smart television (TV), a desktop personal computer (PC), a tablet PC, etc. Accordingly, the above-noted elements of the electronic device may take the form of an entirely hardware embodiment such as a processor or circuit(s), an entirely software embodiment (including firmware, resident software, micro-code, etc.) or an embodiment combining software and hardware.

The speech signal input part110receives a speech signal corresponding to a speech uttered by a user. The speech signal input part110may include a microphone and an amplifier for amplifying the received speech. However, receiving of the speech signal in real time by using the microphone is only an exemplary embodiment, and thus the speech signal input part110may receive the speech signal through a pre-stored file.

The speech signal segmenter120segments the speech signal into a plurality of sections. In detail, the speech signal segmenter120may segment the speech signal into the plurality of sections at preset time intervals (e.g., 0.1 second).

The phoneme recognizer130recognizes a phoneme of a speech signal of one of the plurality of sections segmented by the speech signal segmenter120. In detail, the phoneme recognizer130may calculate a Gaussian probability distribution of a characteristic vector corresponding to the speech signal of the one section by using a first acoustic model for phoneme recognition and selects an optimum phoneme.

The phoneme recognizer130may delete at least a last one of a plurality of phonemes of the speech signal of the one section by using a segmented viterbi algorithm. In detail, since the speech signal segmenter120segments the speech signals in units of time, and not in the units of phoneme, the phoneme recognizer130may not properly recognize a phoneme positioned in an end part of the one section. Therefore, the phoneme recognizer130deletes at least one phoneme positioned in the end part of the one section and outputs the deleted at least one phoneme to the speech signal segmenter120to use the deleted at least one phoneme for recognizing a phoneme of a next section.

The candidate word extractor140extracts a candidate word of the recognized phoneme by using a phoneme recognition result output from the phoneme recognizer130. In detail, the candidate word extractor140extracts a similar phoneme pronounced similarly to the phoneme output from the phoneme recognizer130and generates a word graph of the speech signal of the one section for extracting the candidate word by using the similar phoneme. However, the generation of the word graph of the speech signal of the one section to extract the candidate word is only exemplary, and thus a candidate word list of the speech signal of the one section may be generated. The candidate word extractor130outputs the word graph of the speech signal of the one section to the speech recognizer150.

The speech recognizer150performs speech recognition with respect to the speech signal of the one section by using the candidate word extracted by the candidate word extractor140. In detail, the speech recognizer150may search the word graph output from the candidate word extractor140for an optimum path of the speech signal of the one section output from the speech signal segmenter120to perform the speech recognition.

The phoneme recognizer130, the candidate word extractor140, and the speech recognizer150may operate in parallel in different cores of a processor or different processors. In other words, if the phoneme recognizer130performs phoneme recognition with respect to a speech signal of a first section, the speech recognizer150transmits a result of the recognition of the first section to the candidate word extractor140and performs phoneme recognition with respect to a speech signal of a second section. The candidate word extractor140extracts the candidate word based on the phoneme recognition result of the first section, outputs the extracted candidate word to the speech recognizer150, and extracts a candidate word by using the phoneme recognition result of the speech recognition of the second section output from the phoneme recognizer130. The speech recognizer150performs the speech recognition with respect to the speech signal of the first section by using the candidate word of the speech signal of the first section extracted by the candidate word extractor140and performs the speech recognition with respect to the speech signal of the second section by using the candidate word of the speech signal of the second section extracted by the candidate word extractor140.

The electronic device100described above rapidly performs phoneme recognition with a relatively small number of calculations, extracts a small number of candidate words based on the result of the phoneme recognition, and performs speech recognition by using a noticeably smaller number of candidate words than an existing method of recognizing a plurality of continuous words. Also, the electronic device100performs the phoneme recognition, the extraction of the candidate word, and the speech recognition in parallel to allow a user to further rapidly perform speech recognition.

FIG. 2is a block diagram illustrating a detailed structure of the electronic device100for recognizing a speech according to an exemplary embodiment. Referring toFIG. 2, the electronic device100includes the speech signal input part110, the speech signal segmenter120, the phoneme recognizer130, the candidate word extractor140, and the speech recognizer150.

The speech signal input part110receives a speech signal corresponding to a user's speech. The speech signal input part110may receive the speech signal in real time from a speech input device such as a microphone. However, this is only an example, and the speech signal input part110may receive the speech signal from a file stored in a storage (not shown) of the electronic device100.

The speech signal segmenter120segments the speech signal into a plurality of sections at preset time intervals. Here, the speech signal segmenter120includes a section segmenter121, a preprocessor122, and a characteristic vector extractor123.

The section segmenter121segments the speech signal output from the speech signal input part110at the preset time intervals (e.g., 0.1 seconds).

The preprocessor122performs signal-processing, such as noise removal, with respect to a speech signal of one of the plurality of sections.

The characteristic vector extractor123extracts a characteristic vector from the speech signal of the one section which is preprocessed. The characteristic vector extractor123outputs the characteristic vector of the speech signal of the one section to the phoneme recognizer130and the speech recognizer150.

The phoneme recognizer130performs speech recognition by using the characteristic vector extracted by the characteristic vector extractor123. Here, the phoneme recognizer130includes a first Gaussian probability calculator131, a first acoustic model132, an optimum candidate searcher133, and a section segmentation error corrector134.

The first Gaussian probability calculator131calculates a Gaussian probability of the characteristic vector of the speech signal of the one section by using the first acoustic model132.

The first acoustic model132is an acoustic model for phoneme recognition and stores information about 40 to 50 phonemes in the case of Korean language. The first acoustic model132may be a hidden Markov model (HMM) acoustic model. In particular, the first acoustic model132may be more simply realized than an acoustic model applied to an existing method of recognizing a plurality of continuous words to enable rapid speech recognition.

The optimum candidate searcher133selects optimum phonemes included in the speech signal of the one section based on the calculation results of the first acoustic model132and the first Gaussian probability calculator131.

The section segmentation error corrector134deletes at least a last one of the plurality of phonemes selected by the optimum candidate searcher133. In detail, the speech signal segmenter120according to the present exemplary embodiment segments the speech signal based on time, and not based on a phoneme. Therefore, all data of the last phonemes of the speech signal of the one section input into the phoneme recognizer130may not be input, and thus the at least last one of the plurality of phonemes selected by the optimum candidate searcher133may be an incorrectly selected phoneme. Therefore, the section segmentation error corrector134deletes the at least last one of the plurality of phonemes selected by the optimum candidate searcher133and outputs the phonemes, which are not deleted, to the candidate word extractor140. The section segmentation error corrector134outputs the at least one deleted phoneme to the section segmenter121to recognize the at least one deleted phoneme in a next section.

The phoneme recognizer130according to the present exemplary embodiment deletes the at least last one of the plurality of phonemes selected by the optimum candidate searcher133to correct a section segmentation error through the second segmentation error corrector134. However, this is only an example, and the phoneme recognizer130may search an end part of a phoneme by using a HMM state position check or a signal processing technique to minimize a section segmentation error.

The candidate word extractor140extracts a candidate word based on the phoneme of the speech signal of the one section recognized by the phoneme recognizer130. The candidate word extractor140includes a similarity calculator141and a section word graph generator142.

The similarity calculator141calculates a pronunciation similarity between the phoneme of the speech signal of the one section and other phonemes by using a pronunciation dictionary to extract a similar phoneme pronounced similarly to the phoneme of the speech signal of the one section.

The section word graph generator142generates a section word graph for generating a candidate word based on extracted similar phonemes. Here, the section word graph may be a network type graph on which recognized phonemes are connected to the similar phonemes. The section word graph generator142outputs the section word graph for extracting the candidate word of the speech signal of the one section to an optimum word graph path searcher153.

In the above-described exemplary embodiment, the candidate word extractor140generates the section word graph, but this is only exemplary. Therefore, the candidate word extractor140may extract candidate words to generate a candidate word list.

The speech recognizer150performs speech recognition with respect to one section by using the candidate words output from the candidate word extractor140. The speech recognizer150includes a second Gaussian probability calculator151, a second acoustic model152, the optimum word graph path searcher153, a language model154, and a speech recognition output part155.

The second Gaussian probability calculator151calculates a Gaussian probability distribution of the speech signal of the one section by using the second acoustic model152.

Here, the second acoustic model152is an acoustic model used in a general method of recognizing a plurality of continuous words and may be an acoustic model using a triphone. In particular, in order to perform a complicated speech recognition, the second acoustic model152stores a larger amount of data than the first acoustic model132.

The optimum word graph path searcher153searches for an optimum path corresponding to the speech signal of the section word graph output from the section word graph generator142based on the calculation result of the second Gaussian probability calculator151. Here, the optimum word graph path searcher153may perform the speech recognition by using the language model154storing a grammar and a sentence structure in order to further accurately recognize a sentence. In other words, the first acoustic model132may be an acoustic mode specialized for high-speed speech recognition, and the second acoustic model152may be an elaborate acoustic model for improving the performance of a continuous word speech recognition.

The speech recognition output part155outputs a word string (a sentence) generated by the optimum path searched by the optimum word graph path searcher153.

The phoneme recognizer130, the candidate word extractor140, and the speech recognizer150may be formed in pipeline shapes which operate through different cores in parallel. In detail, as shown inFIG. 3, the speech signal segmenter120segments a speech signal into N sections and transmits the speech signals of the N sections to the phoneme recognizer130. The phoneme recognizer130performs phoneme recognition with respect to a first section at a time t1. At a time t2, the phoneme recognizer130performs phoneme recognition with respect to a second section, and the candidate word extractor140extracts a candidate word of the first section. At a time t3, the phoneme recognizer130performs a phoneme recognition with respect to a third section, the candidate word extractor140extract a candidate word of the second section, and the speech recognizer150performs a speech recognition with respect to the first section. According to this method, the phoneme recognizer130, the candidate word extractor140, and the speech recognizer150operate in parallel at each time. The speech recognizer150performs and outputs speech recognitions with respect to speech signals of all sections after a short time tn+2−tn from a time when a user ends uttering.

As described above, the electronic device100performs a phoneme recognition operation, a candidate word extracting operation using phoneme recognition, and a speech recognition operation using a candidate word in parallel. Therefore, the electronic device100performs speech recognition more rapidly than an existing method of recognizing a plurality of continuous words.

A speech recognition method of the electronic device100according to an exemplary embodiment will now be described with reference toFIG. 4.

Referring toFIG. 4, in operation S410, the electronic device100determines whether a speech signal is input. The speech signal may be input in real time through a speech input device such as a microphone or through a pre-stored file.

If it is determined in operation S410that the speech signal is input, the electronic device100segments the speech signal into a plurality of sections at preset time intervals in operation S420. In detail, the electronic device100segments the input speech signal into the plurality of sections at the preset time intervals (e.g., 0.1 seconds) and performs signal-processing with respect to a speech signal of one of the plurality of sections to extract a characteristic vector.

In operation S430, the electronic device100recognizes a phoneme of the speech signal of the one section. In detail, the electronic device100recognizes the phoneme of the speech signal of the one section by using a first acoustic model. In order to further accurately recognize the phoneme, the electronic device100deletes at least one last phoneme of a plurality of recognized phonemes and uses the at least one deleted phoneme to recognize a phoneme of a speech signal of a next section.

In operation S440, the electronic device100extracts a candidate word of the speech signal of the one section by using the phoneme recognition result. In detail, the electronic device100extracts similar phonemes of the plurality of recognized phonemes and generates a word graph for extracting the candidate word. Here, the word graph is a network type graph on which the recognized phonemes are respectively connected to the similar phonemes.

In operation S450, the electronic device100performs speech recognition with respect to the speech signal of the one section by using the candidate word. In detail, the electronic device100performs speech recognition with respect to the speech signal of the one section by using a second acoustic model and a language model of the candidate word (the word graph) extracted in operation S440.

The electronic device100may repeatedly perform operations S430through S450with respect to speech signals of next sections. The electronic device100may repeatedly perform operations S430through S450in parallel by using different cores of a processor.

As described above, according to the speech recognition method, an electronic device may more rapidly and accurately perform speech recognition than an existing method of recognizing a plurality of continuous words.