Patent Publication Number: US-2022238102-A1

Title: Wake-up word recognition training system and method

Description:
BACKGROUND 
     Technical Field 
     The present invention relates to a wake-up word recognition training system and method for performing wake-up word recognition training by using a speech recognition model. 
     Related Art 
     In recent years, the development of automatic speech recognition (ASR) has gradually matured (for example: speech recognition of Google or Siri). In operating an electronic product such as a mobile device or a personal computer, users more frequently use the function of speech input or speech control. However, in a previous speech recognition technology, a neural network is usually trained by using marked data or key information, usually having problems of excessive time consumption and low accuracy in preparing training data. Therefore, how to increase the accuracy of a wake-up word recognition model without spending too much processing time is one of the problems to be alleviated in this field. 
     For a conventional method, the improvement in the hit rate of a “wake-up word” requires a large number of people to record speech data of the “wake-up word”, so as to fine-tune a deep learning model. In addition, because speech data of each “wake-up word” is independent, a “wake-up word” needs to be changed by repeating the above operation, usually being time-consuming and laborious. In view of this, it is an important issue to provide a type of process automation and reduce the occurrence of human errors. 
     SUMMARY 
     The present invention provides a wake-up word recognition training system and method, capable of saving the trouble of recording speech data, automating the process, and reducing the probability of occurrence of human errors. 
     The present invention provides a wake-up word recognition training system, including a sentence database, a phoneme disassembly module, a phoneme analysis module, and a wake-up word recognition module, where the sentence database stores a plurality of sentences and a phoneme sequence and a speech signal corresponding to each of the sentences; the phoneme disassembly module disassembles a wake-up word inputted from the outside to obtain a wake-up word phoneme sequence; the phoneme analysis module matches the wake-up word phoneme sequence to the sentences and/or phoneme sequences thereof, to obtain wake-up word part-of-speech sentences and non-wake-up word part-of-speech sentences; and the wake-up word recognition module filters speech signal fragments and constructs a deep neural network model according to a phoneme combination of the wake-up word part-of-speech sentences and the non-wake-up word part-of-speech sentences, and trains the deep neural network model according to phoneme sequences and signal fragments thereof of the wake-up word part-of-speech sentences and the non-wake-up word part-of-speech sentences, to establish a wake-up word recognition model. 
     In an embodiment of the present invention, the foregoing phoneme analysis module comprises a phoneme comparison module and a sentence classification module; the phoneme comparison module compares the sentences and the phoneme sequences in the sentence database according to the wake-up word phoneme sequence; and the sentence classification module divides the sentences in the sentence database into the wake-up word part-of-speech sentences and the non-wake-up word part-of-speech sentences according to a comparison result of the phoneme comparison module. 
     In an embodiment of the present invention, constituent units of the foregoing phoneme sequence comprise a sequence of three consecutive phonemes, a sequence of two consecutive phonemes, and/or a single phoneme. 
     In an embodiment of the present invention, the foregoing wake-up word recognition module includes a generic speech recognition module and a dynamic programming algorithm, and speech signal fragments of phoneme sequences corresponding to the sentences of the wake-up word in the sentence database are found out to serve as training data about the wake-up word for the deep neural network model by the generic speech recognition module cooperating with the dynamic programming algorithm. 
     In an embodiment of the present invention, sentences corresponding to a non-wake-up word and/or speech signal fragments of phoneme sequences thereof in the sentence database are found out to serve as training data about the non-wake-up word for the deep neural network model by the foregoing wake-up word recognition module using a method of text comparison. 
     The present invention provides a wake-up word recognition training method, comprising: disassembling, by using a phoneme disassembly module, a wake-up word inputted from the outside to obtain a wake-up word phoneme sequence; matching, by using a phoneme analysis module, the wake-up word phoneme sequence to each of a plurality of sentences and/or phoneme sequences corresponding to the sentences, to obtain wake-up word part-of-speech sentences and non-wake-up word part-of-speech sentences; constructing, by using a wake-up word recognition module, a deep neural network model according to a phoneme combination of the wake-up word part-of-speech sentences and the non-wake-up word part-of-speech sentences, and training the deep neural network model according to phoneme sequences and signal fragments thereof of the wake-up word part-of-speech sentences and the non-wake-up word part-of-speech sentences, to establish a wake-up word recognition model. 
     In an embodiment of the present invention, constituent units of the foregoing phoneme sequence comprise a sequence of three consecutive phonemes, a sequence of two consecutive phonemes, and/or a single phoneme. 
     In an embodiment of the present invention, the foregoing wake-up word recognition module includes a generic speech recognition module and a dynamic programming algorithm, and phoneme sequence fragments corresponding to the sentences of the wake-up word in the sentence database are found out to serve as training data for the deep neural network model by the generic speech recognition module cooperating with the dynamic programming algorithm. 
     In an embodiment of the present invention, sentences corresponding to a non-wake-up word and/or phoneme sequence fragments thereof in the sentence database are found out to serve as training data about the non-wake-up word for the deep neural network model by the foregoing wake-up word recognition module using a method of text comparison. 
     In an embodiment of the present invention, the foregoing matching, by using a phoneme analysis module, the wake-up word phoneme sequence to each of a plurality of sentences and/or phoneme sequences corresponding to the sentences, to obtain wake-up word part-of-speech sentences and non-wake-up word part-of-speech sentences comprises: comparing, by using a phoneme comparison module, the sentences and the phoneme sequences in the sentence database according to the wake-up word phoneme sequence; and obtaining, by using a sentence classification module, the wake-up word part-of-speech sentences and the non-wake-up word part-of-speech sentences from the sentences according to a comparison result of the phoneme comparison module. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         FIG. 1  is a schematic block diagram of elements of a system according to the present invention. 
         FIG. 2  is a flowchart of steps according to an embodiment of the present invention. 
         FIG. 3  is a schematic diagram of the classification in a sentence database by a processing module of the present invention. 
         FIG. 4A  and  FIG. 4B  are schematic diagrams of relationships between wake-up words and phoneme sequences of the present invention. 
     
    
    
     DETAILED DESCRIPTION 
     To make the features and advantages of the present invention clear and easy to understand, the following gives a detailed description of embodiments with reference to accompanying drawings. 
       FIG. 1  is a schematic block diagram of elements of a system according to the present invention. In  FIG. 1 , a wake-up word recognition training system includes an input module  110 , a sentence database  120 , and a processing module  130 , where the processing module  130  includes a phoneme disassembly module  140 , a phoneme comparison module  150 , a sentence classification module  160 , a wake-up word recognition module  170  and a generic speech recognition module  180 . 
     The input module  110  obtains a wake-up word; the sentence database  120  stores a plurality of sentences  121  and corresponding phoneme sequences  122 ; the phoneme disassembly module  140  disassembles the wake-up word to obtain a wake-up word phoneme sequence; the phoneme comparison module  150  compares the sentences  121  and phoneme sequences  122  thereof according to the wake-up word phoneme sequence; the sentence classification module  160  divides the sentences  121  in the sentence database into wake-up word part-of-speech sentences and non-wake-up word part-of-speech sentences according to a comparison result of the phoneme comparison module  150 ; and the wake-up word recognition module  170  extracts, by using the generic speech recognition module  180  and a dynamic programming algorithm, speech signal fragments of the wake-up word phoneme sequence from the wake-up word part-of-speech sentences according to a classification result of the sentence classification module  160 , and obtains signal fragments of a non-wake-up word phoneme sequence from the non-wake-up word part-of-speech sentences by using a method of text comparison; and further imports a deep neural network model, and trains the deep neural network model and establishes a wake-up word recognition model by using the two types of speech signal fragments according to a result of the wake-up word recognition module  170 . 
     In this embodiment, the wake-up word recognition model further includes a generic speech recognition module  180 , through which the speech signal fragments of the wake-up word phoneme sequence are extracted from the wake-up word part-of-speech sentences in cooperation with the dynamic programming algorithm, to serve as training data about the wake-up word. 
       FIG. 2  is a flowchart of steps according to an embodiment of the present invention.  FIG. 3  is a schematic diagram of the classification in a sentence database by a processing module of the present invention.  FIG. 4A  and  FIG. 4B  are schematic diagrams of relationships between wake-up words and phoneme sequences of the present invention. For better understanding, reference may be made to  FIG. 2  to  FIG. 4B  together. 
     In  FIG. 2 , main steps of a wake-up word recognition training method are as follows: 
     Step S 210 : Obtain a wake-up word. 
     In this embodiment, text input or using a speech input device, such as a microphone, to input a personalized wake-up word may be a main way to obtain a wake-up word. In an embodiment of the present invention, “nihao” is used as an example for description. 
     In the present invention, the wake-up word is not limited to greetings, and the wake-up word may alternatively be replaced with a similar concept, including: a keyword with a specific purpose, a speech command, a trigger word, a human name or a title. 
     Step S 220 : Obtain a wake-up word phoneme sequence. 
     In this embodiment, after a wake-up word  410  “nihao” is obtained, a phoneme disassembly module of a processing module is used to disassemble the wake-up word  410  “nihao” to obtain a wake-up word phoneme sequence  420 , and SIL-n-i-h-au-SIL can be obtained by analyzing the wake-up word phoneme sequence  420 , where SIL is a silent phoneme, and the phoneme sequence may be three consecutive phonemes (triphone), two consecutive phonemes (right-context dependent or left-context dependent), or a single phoneme. In this embodiment, a sequence of three phonemes is adopted, so that a phoneme combination  430  including  4  groups of phonemes “SIL-n-i”, “n-i-h”, “i-h-au”, and “h-au-SIL” can be obtained (reference is made to  FIG. 4A ). 
     Step S 230 : Compare the wake-up word phoneme sequence with phoneme sequences of sentences. 
     In this embodiment, a phoneme comparison module is used to compare a plurality of sentences in the sentence database and a phoneme sequence corresponding to each of the sentences according to the wake-up word phoneme sequence. In this step, the sequence of three consecutive phonemes in SIL-n-i-h-au-SIL obtained by analyzing the wake-up word phoneme sequence  420  is mainly used, that is, the 4 groups of phonemes “SIL-n-i”, “n-i-h”, “i-h-au”, and “h-au-SIL” described above. 
     Step S 240 : Classify the sentences. 
     Following the previous step, a sentence classification module, according to a comparison result of the phoneme comparison module, marks a sentence as a wake-up word part-of-speech sentence  1211  if the sentence has the same wake-up word phoneme sequence, and the wake-up word part-of-speech sentence corresponds to a wake-up word part-of-speech sentence phoneme sequence  1221 ; otherwise, marks a sentence as a non-wake-up word part-of-speech sentence  1212  if the sentence does not belong to a wake-up word category after comparison, and the non-wake-up word part-of-speech sentence corresponds to a non-wake-up word part-of-speech sentence phoneme sequence  1222 . 
     The non-wake-up word part-of-speech sentence phoneme sequence  1222  may alternatively be another phoneme combination, a filling model, or a wake-up word adversarial model. 
     Step S 250 : Establish a wake-up word recognition model. 
     The wake-up word recognition module imports a deep neural network model, and according to the wake-up word part-of-speech sentence phoneme sequence of the wake-up word part-of-speech sentence, the non-wake-up word part-of-speech sentence phoneme sequence of the non-wake-up word part-of-speech sentence, and the signal fragments of the wake-up word part-of-speech sentence and the non-wake-up word part-of-speech sentence, trains the deep neural network model and establishes a wake-up word recognition model. 
     In this embodiment, the speech signal fragments of the phoneme sequences corresponding to the sentences of the wake-up word in the wake-up word part-of-speech sentence phoneme sequence  1221  in the sentence database are found out to serve as the training data for the deep neural network model by the wake-up word recognition module using a dynamic programming algorithm in combination with the non-wake-up word part-of-speech sentence phoneme sequence  1222  and a speech signal thereof. 
     Referring to  FIG. 4B , by using “SIL-n-i” of a phoneme combination  430 A, a corresponding sentence combination  440 A in the sentence database can be obtained; and by further using “n-i-h” of a phoneme combination  430 B, a corresponding sentence combination  440 B in the sentence database can be further obtained. 
     In this embodiment, the corresponding phoneme sequence and signal fragment in the sentence database are obtained by mainly using a Viterbi algorithm. 
     The deep neural network model is trained by further using a stochastic gradient descent algorithm. 
     The dynamic programming algorithm includes but is not limited to a Viterbi algorithm or dynamic time warping. The Viterbi algorithm is a dynamic programming algorithm used to find a Viterbi path most likely to produce a sequence of observed events. The Viterbi algorithm is a dynamic programming algorithm used to find an observation result most likely to explain a correlation, while the dynamic time warping is a pragmatic method for classifying two signals of different time lengths. Alignments provided by the dynamic time warping can effectively match the two signals, thereby calculating a similarity between the two signals. 
     In summary, in the present invention, recognition training on the structure of the deep neural network model has the following advantages: 
     1. Require a relatively small number of people to record speech data. 
     2. Customize an exclusive wake-up word. 
     3. Reduce the production cost and time of a “wake-up word” recognition system. 
     4. Improve a hit rate of the “wake-up word” recognition system. 
     Although the present invention is disclosed above with the foregoing embodiments, the embodiments are not intended to limit the present invention. Changes and modifications equivalent to replacements made by anyone familiar with related arts without departing from the spirit and scope of the present invention shall still fall within the patent protection scope of the present invention.