Normalization of speech signals

Abstract: It is an object of the invention to eliminate an influence by line characteristics in a real-time manner in order to raise recognizing precision of an input speech and to enable the speech to be recognized in a real-time manner. For this purpose, an estimate value of a long-time mean of a parameter is obtained from speech feature parameters which are sequentially inputted by using the speech feature parameters which have already been inputted, and the speech feature parameter inputted at that time point is normalized by using the obtained estimate value. Each time the speech feature parameter is inputted, the latest estimate value is obtained by using the already inputted parameters including the inputted speech feature parameter, and the latest input speech feature parameter is normalized by using the updated estimate value. Since the reliability of the estimate value is higher as the number of speech feature parameters used when the estimate value is obtained is larger, the estimate value is normalized by adding a weight in accordance with the reliability.

Description: The invention relates to a technique for performing a speech recognition by using a feature of a speech time series such as a cepstrum or the like. The invention also relates to a removal of a multiplicational noise such as line characteristics or the like. The invention further relates to a technique for enabling an instantaneous or successive adaptation to a noise. In case of performing a speech recognition in a real environment, particularly, a multiplicational noise due to a distortion of line characteristics due to an influence by a microphone, telephone line characteristics, or the like and an additive noise such as an internal noise or the like cause problems. As a method of coping with the distortion of the line characteristics among them, a Cepstrum Mean Subtraction (CMS) method has been proposed. The CMS method has been disclosed in detail in Rahim, et al., "Signal Bias Removal for Robust Telephone Based Speech Recognition in Adverse Environments", Proc. of ICASSP'94, 1994 or the like. The CMS method is a method of compensating the distortion of the line characteristics. According to such a method, on the basis of information extracted from an input speech, the line distortion is corrected on the input time series side or the model side such as HMM or the like, thereby making it adaptive to the input environment. Thus, even if the line characteristics fluctuate, it is possible to flexibly cope with such a situation. The CMS method is a method of compensating multiplicational noise (line distortion) which acts due to a convolution of an impulse response. A long-time spectrum of an input speech is subtracted from the input speech and a long-time spectrum of a speech used in the model formation is subtracted from a model, thereby normalizing a difference of the line characteristics. The normalizing process is generally performed in a logarithm spectrum region or a cepstrum region. Since the multiplicational noise appears as an additive distortion in those two regions, the multiplicational noise can be compensated by a subtraction. A method of performing such a process in the cepstrum region is called a CMS method. By using the CMS method as mentioned above, it is possible to cope with the distortion of the line characteristics due to the influence by the microphone, telephone line characteristics, or the like. In case of using the CMS method, however, in order to compute a cepstrum long-time mean (CM) from the speech inputted as a recognition target, the user has to wait for completion of the input of the speech as a recognition target. The recognizing process is performed after the CM was obtained, namely, after the end of the speech input. Therefore, a recognition algorithm cannot be made operative synchronously with the speech input. It is, consequently, impossible to perform a real-time process according to the conventional method. It is known from US-A-5,583,961 to process speech to recognise the speaker using processing in which cepstral vectors are normalised by subtracting a running average calculated from cepstral coefficients for a number of frames, the running average being updated on a frame by frame basis. Aspects of the present invention are set out in the appended claims. According to an embodiment of the invention, since a distortion of line characteristics which can fluctuate can be compensated at a high speed in a semi-real-time manner, a speech recognition can be performed in a real-time manner at a high precision after a normalization of the line characteristics was performed. BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1 is a functional block constructional diagram of an apparatus according to the invention; Fig. 2 is a flowchart for a speech recognizing method using a computing method of CM by a weighting process of an input length; Fig. 3 is a flowchart for a speech recognizing method using a CM computing method at the time of a backward search using a multi-pass search; Fig. 4 is a flowchart for a speech recognizing method which uses the CM computing method by the weighting process of the input length at the time of a forward search and, further, performs a re-computation of a CM at the time of the backward search; Fig. 5 is a comparison diagram of a recognition processing time of each method; and Fig. 6 is a block constructional diagram of the apparatus according to the invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT An embodiment according to the invention will now be described in detail hereinbelow with reference to the drawings. Fig. 6 is a block diagram showing a construction of a speech recognizing apparatus according to the invention. Reference numeral 101 denotes a display unit such as CRT, liquid crystal display, or the like for displaying a character train obtained as a result of a speech recognition; and 102 indicates a print unit such as LBP, ink jet printer, or the like for printing the character train obtained as a result of the speech recognition. The character train which is outputted from the display unit 101 or print unit 102 is a train of character fonts corresponding to candidate character codes of recognition results. Reference numeral 103 denotes a speech input unit such as a microphone or the like. A speech can be also inputted via communicating means such as public line, LAN, or the like as a speech input unit 103. Reference numeral 104 denotes a CPU for controlling so as to execute various processes, which will be explained hereinlater, in accordance with control programs stored in a memory unit 105 or a detachable storing medium 107 such as a CD-ROM or the like. Although the CPU 104 controls the execution of various applications in addition to processes, which will be explained hereinlater, the CPU 104 also controls (for example, "print", "end", and the like) so as to execute various processes regarding the applications in accordance with a command corresponding to the speech recognized by the method of the invention. Reference numeral 105 denotes the memory unit provided in the apparatus. The memory unit 105 stores control programs for various processes which are executed by the CPU 104 and will be explained hereinlater, various parameters (for example, data corresponding to a dictionary which is used for the speech recognition, and the like) which are necessary for the control programs, speech data to be recognized, and the like. The control programs and various parameters which are stored in the memory unit 105 can be inputted from the storing medium 107 or can be also inputted and stored through a communication line. Reference numeral 106 denotes a communication I/F for controlling a communication so as to transmit and receive data through communicating means such as public line, LAN, or the like. It is also possible to construct in a manner such that speech inputted by another apparatus or control programs and various parameters stored in the other apparatuses are fetched into the apparatus via the communication I/F 106 and are stored into the memory unit 105, and after that, each process which will be explained hereinlater is started. Reference numeral 107 denotes the storing medium such as CD-ROM, FD, or the like which can be attached and detached to/from the apparatus main body, namely, the storing medium which can store the control programs, various parameters, and speech data which have already been described as being stored in the memory unit 105 and can be attached and detached to/from the apparatus main body. The data can be downloaded from the storing medium 107 to the memory unit 105 before starting various processes, which will be explained hereinlater, or the CPU 104 can also directly access to the storing medium 107. Control programs for various processes are stored in memory unit 105. and a parameter storing portion, a speech data storing portion, a working area, and the like besides them are also provided. Fig. 1 is a functional block constructional diagram of a speech recognizing apparatus of the embodiment according to the invention. In the diagram, reference numeral 1 denotes a microphone for inputting a speech. The microphone 1 is realized by the speech input unit 103. Reference numeral 2 denotes a speech analyzer for converting the inputted speech to a feature parameter time series; 3 a searcher for searching a correct recognition series by using information of feature parameters, grammar, and phoneme models. The speech analyzer 2 and searcher 3 are realized by the CPU 104 in Fig. 6. Reference numeral 4 denotes a recognition grammar and 5 indicates a phoneme model for speech recognition. The recognition grammar 4 and phoneme model 5 for speech recognition use the data stored in the memory unit 105 or storing medium 107. First, 1) a computation of a cepstrum long-time mean (CM) by a weighting process of an input length will now be described. This method is a method of updating an estimate value of the CM each time a frame of speech is inputted and subtracting the latest estimate value of the CM from the latest speech frame. However, since the number of speech frames which are used for the computation of the estimate value of the CM is small just after the start of the speech, its value includes many errors, so that there is a fear of deterioration of the recognizing performance. To prevent such a drawback, a weight according to the number of frames used in the computation of the CM estimate value, namely, the number of input frames at that time point is added to the CM. When the number of input frames is small, the weight to be added is reduced, thereby decreasing an amount of CM which is subtracted. On the contrary, when the number of input frames rises, the weight is increased, thereby enlarging an amount of CM which is subtracted. The weight is increased or decreased in a range of 0 to 1 in accordance with the number of frames (input time length). The speech analyzer 2 performs a general cepstrum analysis of the speech inputted from the microphone 1. A normalization of a parameter x n of the n-th frame in one input speech which was cut-out after the analysis is executed by the following equation (1), thereby obtaining a normalized parameter (x n )' of the n-th frame. where, µ d : CM of a database τ: constant The weight n/(n+τ) by the frame number of the second term is not limited to the above equation but can be also substituted by a function in which a value is set to a value within a range from 0 to 1 with an increase in n. The parameter (x n )' obtained by normalizing as mentioned above is recognized as a feature parameter. Subsequently, 2) a subtracting process of the CM using the multi-pass search will now be described. The multi-pass method is a method of narrowing correct answer candidates by repeating the search a plurality of times (at least, twice). In the search of the first time, the search is performed while computing the CM of the input speech. In the first search, the CM of the input speech is not reflected to the speech recognition. At the time of the search of the second time, the CM obtained from the whole input speech in which the speech input has already been completed and the computation has been finished is subtracted from the input speech cepstrum, thereby removing the multiplicational noise as a fluctuation in line characteristics. The multi-pass search is used in the searcher 3 in Fig. 1. There is a Tree-trellis based search or a Forward-backward search as a method of recognizing by the multi-pass search. The Tree-trellis based search has been disclosed in detail in the literature by F. K. Soong and E. F. Huang, "A Tree-Trellis Based Fast Search for Finding the N Best Sentence Hypoteses in Continuous Speech Recognition", Proc. of ICASSP91, pp.705 to 708, May, 1991. The Forward-backward search has been described in detail in the literature by S. Austin, R. Schwartz, and P. Placeway, "The Forward-Backward Search Algorithm", Proc. of ICASSP91, pp. 697 to 700, May, 1991. Although an example using the Tree-trellis based search will be described in the embodiment, the searching method of a searcher 3 is not limited to such an example. The Tree-trellis based search method is an algorithm for realizing a strict N-best search and is constructed by searches of two stages of the forward search and the backward search. In the forward search (the first search), a Viterbi search synchronized with a frame is performed and an upper predetermined number of results among the results obtained by the forward search are selected as recognition result candidates of the input speech. In the backward search (the second search), an A* search in which the results of the forward search are heuristic is performed with respect to each of the selected candidates. In the searches of two stages, the CM of the whole input speech is computed in parallel with the search at the time of the forward search and the resultant CM is subtracted and the searching process is executed at the time of the backward search, thereby normalizing the line characteristics. The normalization at the time of the backward search is executed by either one of the following equations (2) and (3). or where, N: frame length of the input speech According to the equation (2), since the amount of CM which is subtracted is reduced particularly in the case where the frame length of the input speech is short, errors of the CM which occur because of the short input time length can be reduced and it is effective to improve the recognizing performance. By changing the normalizing method of the input speech in accordance with the forward search and the backward search as mentioned above, although the conditions of the A* search are not satisfied, by sufficiently enlarging a stack of N-best and sorting the recognition result candidates in accordance with the likelihood, sufficient precision can be maintained. Further, the methods of 1) and 2) are combined, in the forward search, the search is performed by the method of 1) while executing the normalization by the CM obtained by the weighting process of the input length and, in the backward search, the normalization is executed by the method of 2). A flow of the computation mentioned above is shown in flowcharts of Figs. 2 to 4. Fig. 2 is the flowchart for a speech recognizing method using the computing method of the CM by the weighting process of the input length of 1). The input parameter is normalized (22) by the equation (1) before the forward search (23). At the time of the backward search, a logarithm likelihood of each parameter calculated at the time of the forward search (24) is used as it is and added. Fig. 3 is the flowchart for a speech recognizing method using the CM computing method at the time of the backward search using the multipass search in 2). According to this method, at the time of the forward search (32), the search using the cepstrum parameter by the ordinary speech analysis which is not normalized is performed. After completion of the speech input (33), each candidate selected by the forward search is subjected to the normalization of the input parameter by the equation (2) or (3), thereby searching (34). Thus, the likelihood at the time of the forward search and that at the time of the backward search are different and the A* condition is not satisfied. Therefore, the recognition result candidates are sorted in accordance with the likelihood (35) and the result is outputted (36). Fig. 4 relates to the combination method of 1) and 2). At the time of the forward search (43), the search is performed (42) by using the parameter normalized by the equation (1). At the time of the backward search (45), the search is executed by using the parameter (44) normalized by the equation (2) or (3). In this case as well, since the likelihoods in the forward search and the backward search are obviously different, the recognition result candidates are sorted in accordance with the likelihood (46) and the recognition result is outputted (47). The above processes are executed every speech input. The embodiment has been described above with respect to the example of normalizing the line characteristics every speech input. However, when it is discriminated that there is no fluctuation of the line characteristics, by normalizing the line characteristics with respect to a plurality of speech inputs, the more stable normalization of the line characteristics can be performed. In the above description, each time the input speech changes, the value of n in the equation (1) or the value of N in the equations (2) and (3) is cleared. However, in the method of performing the normalization for a plurality of input speeches, such a value is not cleared every time but the value is held for a plurality of speech inputs and the normalization is executed. Although the embodiment has been described above with respect to the example of presuming the cepstrum as a parameter, a logarithm spectrum can be also used in place of the cepstrum. Fig. 5 is a diagram showing an effect of the invention and shows a recognition processing time of each method. Experimental conditions are as follows. First, the evaluation is made by an unspecified speaker sentence recognition. The number of words is set to 1004. The word is perplexity 30.2. The analyzing conditions are set to a sampling rate of 8 kHz, a frame period of 10 msec, a window width of 25.6 msec, and a preemphasis of 0.97. As for the feature amounts, an LPC melcepstrum of 12-th degree, a ΔLPC melcepstrum of 12-th degree, and a Δ logarithm power are used. In the speech recognition, an HMM of 3 states and 6 mixture is used in a recognition system in which the HMM is used as a base and 243 kinds of right environment dependency phonemes HMM are used in a diagonalization covariance matrix. Recognition targets are 50 sentences uttered by ten speakers. By transmitting the speeches through a band pass filter of 300 to 3200 Hz, a distortion of line characteristics are given. According to the conventional method, the normalizing process can be performed only at a time point when the speech utterance is finished, so that it takes 1 sec or more as a recognizing time after the end of the utterance. On the other hand, according to the embodiment, in any case, the recognition is finished for a time interval from about 0.05 sec to about 0.26 sec after the end of the utterance. A numerical value shown in the lower portion of the diagram indicates a recognition rate of each method in case of adjusting τ. For comparison, a recognition rate in case of not performing the normalization of the line characteristics is equal to 65.0 %. According to the combination method of 1) + 2), it will be understood that the reduction in recognition rate is very small and a computation amount can be remarkably reduced. According to the method as mentioned above, by processing the normalization of the distortion of the line characteristics synchronously with the speech input, the line characteristics can be normalized at a high speed.

Claims: A speech processing method comprising the steps of: receiving an input (21) of a sequence of feature parameters; each time a new feature parameter is inputted, updating (22) a long-term mean value of the feature parameters by using the feature parameters inputted so far; and performing (22) a normalization of each new feature parameter using the long-term mean value to obtain a sequence of normalised parameters, wherein, when performing normalization of each new feature parameter, the long-term mean value is weighted using a weight determined according to the number of feature parameters used to derive the long-term mean value of said parameter; characterised in that said normalization process is applied to a multi-pass search speech recognition process of an input speech, wherein during a first search a number of recognition candidates is determined while said long-term mean value is calculated, and a second search is carried out only on said number of recognition candidates by applying said normalization process using said long-term mean value. A method according to claim 1, wherein the long-term mean value of said feature parameter is a value representative of noise characteristics. A method according to claim 1, wherein said feature parameter is a cepstrum parameter. A method according to claim 1, wherein said feature parameter is a logarithm spectrum time series. A method according to claim 1, wherein said weight is increased as the number of said feature parameters increases. A method as claimed in claim 5 wherein the weight is calculated as n / n + τ where n is the number of feature parameters used to derive the long-term mean value and τ is a constant. A method according to claim 1, wherein the normalisation of said parameter is a process for subtracting said weighted long-term mean of said parameters from said inputted feature parameter. A method according to claim 1, wherein the normalization of said feature parameter is a process of compensating for line characteristics. A method according to claim 1, further comprising the steps of: cutting out (21) a speech interval from an input speech; and obtaining (22) the long-term mean of said feature parameter which continues in said cut-out speech interval. A method according to claim 1, comprising performing (23; 24, 25) speech recognition by using said normalized parameters. A method according to claim 10, comprising displaying (25) characters obtained as a result of said speech recognition on a display (101). A method according to claim 10, comprising printing (25) characters obtained as a result of said speech recognition by operation of a printing apparatus (102). A method according to claim 1, comprising analysing speech inputted from a microphone (103) to obtain said sequence of feature parameters. A method according to claim 1, wherein said sequence of parameters is inputted through a communication line (106). A method according to claim 10, wherein the operation of an application is controlled in accordance with a result of said speech recognition. A speech processing apparatus comprising: input means (2) for receiving an input of a sequence of feature parameters; long-term mean deriving means (104) for, each time a new feature parameter is inputted, updating (22) a long-term mean value of the feature parameters by using the feature parameters inputted so far; and normalising means (104) for performing (22) a normalization of each new feature parameter using the long-term mean value to obtain a sequence of normalised parameters, wherein, when performing normalization of each new feature parameter, the normalising means is operable to weight the long-term mean value using a weight determined according to the number of feature parameters used to derive the long-term mean value of said parameter; characterised in that said normalizing means is applicable to a multi-pass search speech recognition process of an input speech, the apparatus comprising means for determining during a first search a number of recognition candidates while said long-term mean value is calculated, and means for carrying out a second search only on said number of recognition candidates by applying said normalization process using said long-term mean value. An apparatus according to claim 16, wherein the long-term mean of said parameter is a value representative of noise characteristics. An apparatus according to claim 16, wherein said feature parameter is a cepstrum parameter. An apparatus according to claim 16, wherein said feature parameter is a logarithm spectrum time series. An apparatus according to claim 17, wherein the normalising means increases said weight as the number of said feature parameters increases. Apparatus as claimed in claim 20, wherein the normalising means calculates the weight as n / n + τ where n is the number of feature parameters used to derive the long-term mean and τ is a constant. An apparatus according to claim 16, wherein said normalising means performs the normalisation by subtracting said long-time mean of said parameter obtained from said inputted feature parameter. An apparatus according to claim 16, wherein the normalising means provides compensation for line characteristics of a transmission line via which the input speech is received. An apparatus according to claim 16, further comprising cut-out means for cutting out a speech interval from input speech, and wherein said long-term mean deriving means obtains the long-term mean of said feature parameter which continues in said speech interval cut out by said cut-out means. An apparatus according to claim 16, further having recognising means for recognising speech by using said normalised parameter. An apparatus according to claim 25, further having display means (101) for displaying characters as a result which was speech recognised by said recognising means. An apparatus according to claim 25, further having printing means (102) for printing characters as a result which was speech recognised by said recognising means. An apparatus according to claim 16, further having speech analysing means for obtaining said parameter by analysing (2) speech inputted from a microphone (1). An apparatus according to claim 16, further having a microphone (1) for inputting speech. An apparatus according to claim 16, further having communication line control means (106) for inputting said parameter through a communication line. An apparatus according to claim 25, further having control means (104) for controlling the operation of an application in accordance with a result which was speech recognised by said recognising means. A data carrier storing program instruction for carrying out the method according to any of claims 1 to 15 when run on a computer.

Inventors: KOMORI YASUHIRO, KOSAKA TETSUO, KOMORI, YASUHIRO, KOSAKA, TETSUO
Applicants: CANON KK, CANON KABUSHIKI KAISHA
Application year: 1998
Main classification: G10L15
IPC codes: G10L21, G10L15