Abstract:
An approach to scoring acoustically-based events, such as hypothesized instances of keywords, in a speech processing system make use of scores of individual components of the event. Data characterizing an instance of an event are first accepted. This data includes a score for the event. The event is associated with a number of component events from a set of component events, such as a set of phonemes. Probability models are also accepted for component scores associated with each of the set of component events in each of two of more possible classes of the event, such as a class of true occurrences of the event and a class of false detections of the event. The event is then scored. This scoring includes computing a probability of one of the two or more possible classes for the event using the accepted probability models.

Description:
CROSS-REFERENCE TO RELATED APPLICATIONS  
     This application claims the benefit of U.S. Provisional Application No. 60/489,390 filed Jul. 23, 2003, which is incorporated herein by reference. 
    
    
     BACKGROUND  
     This invention relates to scoring of acoustically-based events in a word spotting system. 
     Word spotting systems are used to detect the presence of specified keywords or phases or other linguistic events in an acoustically-based signal. Many word spotting systems provide a score associated with each detection. Such scores can be useful for characterizing which detections are more likely to correspond to a true events (“hits”) rather than misses, which are sometimes referred to as false alarms. 
     Some word spotting systems make use of statistical models, such as Hidden Markov Models (HMMs), which are trained based on a training corpus of speech. In such systems, probabilistically motivated scores have been used to characterize the detections. One such score is a posterior probability (or equivalently a logarithm of the posterior probability) that occurred (e.g., started, ended) at a particular time given acoustically-based signal and the HMM model for the keyword of interest and for other speech. 
     It has been observed that the probabilistically motivated scores can be variable, depending on factors such as the audio conditions and the specific word or phrase that is being detected. For example, scores obtained in different audio conditions or for different words and phrases are not necessarily comparable. 
     SUMMARY 
     In one aspect, in general, the invention features a method and corresponding software and a system for scoring acoustically-based events in a speech processing system. Data characterizing an instance of an event are first accepted. This data includes a score for the event. The event is associated with a number of component events from a set of component events. Probability models are also accepted for component scores associated with each of the set of component events in each of two of more possible classes of the event. The event is then scored. This scoring includes computing a probability of one of the two or more possible classes for the event using the accepted probability models. 
     Aspects of the invention can include one or more of the following features: 
     The two or more classes of the event can include true occurrence of the event, and the classes can include false detection of the event. 
     The acoustically-based event can include a linguistically-defined event, which can include one or more word events. The component events can include subword units, such as phonemes. 
     The probability models for the component scores can be Gaussian models. 
     The method can further include accepting data characterizing multiple instances of events, such that at least some of the events are known to belong to each of the two or more classes of events. The method can further include estimating parameters for the probability models for the component scores from the data characterizing the multiple instances of events. Estimating the parameters can include applying a Gibbs sampling approach. 
     Aspects of the invention can have one or more of the following advantages. 
     The approach can make scores for different events, which may have different phonetic content, more comparable. 
     The overall accuracy of a word spotting system can be improved using this approach. 
     Other features and advantages of the invention are apparent from the following description, and from the claims. 
    
    
     
       DESCRIPTION OF DRAWINGS 
         FIG. 1  is a block diagram of training components of a word spotting system. 
         FIG. 2  is a block diagram of runtime components of a word spotting system. 
         FIGS. 3-9  are pseudocode for a procedures executed in the training component of the word spotting system. 
     
    
    
     DESCRIPTION 
     Referring to  FIGS. 1 and 2 , a word spotting system includes a training subsystem  101  ( FIG. 1 ), which includes components that are used during a training or parameter estimation phase, and a runtime subsystem  102  ( FIG. 2 ), which includes components that are used during processing of unknown speech  126 . (The speech is “unknown” in that the locations of desired events are not known.) 
     Referring first to the runtime subsystem  102 , which is shown in  FIG. 2 , a word spotting engine  120  accepts the unknown speech  126  as input and produces putative detections  144  of one or more words, phrases, or other linguistic events which are specified by corresponding queries. Each putative detection of an event is associated with a score that is calculated by the word spotting engine  120 . The word spotting engine  120  is configured with models  122  that are computed by the training subsystem  101 , which is described further below. The models  122  includes statistically estimated parameters for analytic probabilistics models for linguistically-based subword units. In this version of the system, these units include approximately 40 English phonemes. The statistical models for these units are represented using Hidden Markov Models (HMMs). 
     The word spotting engine  120  processes the unknown speech  126  to detect instances of the events specified by the queries. These detections are termed putative events  144 . Each putative event is associate a score and the identity of the query that was detected, as well as an indication of when the putative event occurred in the unknown speech (e.g., a start time and/or an end time). In this version of the system, the score associated with a putative event is a probability that the event started at the indicated time conditioned on the entire unknown speech signal  126  and based on the models  122 . These scores that are output from the word spotting engine  120  are referred to below as “raw scores.” 
     The raw scores for the putative events  144  are processed by a score normalizer  140  to produce putative events with normalized scores  152 . The score normalizer  140  makes use of normalization parameters  142 , which are determined by the training subsystem  101 . Generally, the score normalizer  140  uses the phonetic content of a query and the normalization parameters that are associated with that phonetic content to map the raw score for the query to a normalized score. The normalized score can be interpreted as a probability that the putative event is a true detection of the query. The normalization score is a number between 0.0 and 1.0 with a larger number being associated with a greater certainty that the putative event is a true detection of the query. 
     Referring to  FIG. 1 , the models  122  that are used by the word spotting engine  120  are estimated by a word spotting trainer  110  from training speech (A)  112  using conventional HMM parameter estimation techniques, for example, using the well-known Forward-Backward algorithm. 
     The normalization parameters  142  are estimated by a normalization parameter estimator  130 . This parameter estimator takes as inputs a set of true instances of query events along with their associated raw scores  132 , as well as a set of false alarms and their scores  134 , that were produced by the word spotting engine  120  when run on training speech (B)  124 . These sets of true events and false alarms include instances associated with a number of different queries, which together provide a sampling of the subword units used to represent the queries. Preferably, training speech (A)  112 , which is used to estimate models  122 , and training speech (B)  124  are different, although the procedure can be carried out with the same training speech, optionally using one of a variety of statistical jackknifing techniques with the same speech. 
     The normalization parameter estimator  130  and the associated score normalizer  140  are based on a probabilistic model that treats each raw score, R (q) , for an instance of a putative detection of a query q expressed as a logarithm of a probability that the query q occurred, as having an additive form that includes terms each associated with a different subword (phonetic) unit of a query. That is, if the query q is represented as the sequence of N units s 1 , . . . , s N , (the dependence of the length N on the specific query q is omitted in the notation below to simplify the notation) then the raw score is represented as R (q) =Σ i=1   N r s     i   . The component scores r s     i    are modeled as being conditionally independent of one another give that the event is known to be either a true detection or a false alarm. The distribution of each term depends on the identity of the subword unit, s i , and on whether the event is a true detection or a false alarm. 
     The queries are all represented using a common limited set of subword units, in this version of the system, a set of approximately L=40 English phonemes. Normalization parameters  142  therefore include parameters for 2L distributions, two for each subword unit s, one for a true detection (“Hit”), P s (r|Hit), and one for a miss (false alarm), P s (r|Miss). 
     Each of these distributions that are associated with the subword units is modeled as a Gaussian (Normal) distribution, with the shared variances among the Hit distributions and among the Miss distributions. Specifically, the distributions take the form:
 
 P   S ( r |Hit)= N ( r;  μ H,s ,σ H   2 )
 
and
 
 P   S ( r |Miss)= N ( r;  μ M,s ,σ M   2 ).
 
     Therefore normalization parameters  142  include 2L means μ H,s  and μ M,s , and two variances σ H   2  and σ H   2 . 
     Because of the additive form R (q) =Σ i=1   N r s     i   , and the assumption of conditional independence of the component scores, the distribution of the raw score conditioned on the detection being either a hit of a miss is also Gaussian with a mean than is the sum of the means of the component scores and a variance that is a sum of the variance of the component scores. Specifically,
 
 P   (q) ( R   (q) |Hit)= N ( R   (q) ;Σ i=1   N μ H,s     i     , Nσ   H   2 )
 
and similarly
 
 P   (q) ( R   (q) |Miss)= N ( R   (q) ; Σ i=1   N μ M,s     i     , Nσ   M   2 ).
 
     The score normalizer  140  takes as input a raw score R (q)  for a query q, which is represented as the sequence of units s 1 , . . . , s N , and outputs a normalized score, which is computed as a probability Pr(Hit|R (q) ) based on the normalization parameters. Score normalizer  140  implements a computation based on Bayes&#39; Rule:
 
Pr(Hit| R   (q) )= P     (q)     (R     (q)     |Hit)Pr(Hit) / P   (q) ( R   (q) )
 
where
 
 P   (q) ( R   (q) )= P   (q) ( R   (q) |Hit)Pr(Hit)+ P   (q) ( R   (q) |Miss)(1−Pr(Hit))
 
     The a priori probability that a detection is a hit, Pr(Hit), is treated as independent of the query. This a priori probability is computed from the relative number of true query events  132  and false alarms  134  is also stored as one of the parameters of normalization parameters  142 . 
     Referring to  FIG. 1 , the normalization parameter estimator  130  takes as input a number of true hits and their associated raw scores, and a number of false alarms with their raw scores. To handle the unobserved nature of the component score, the normalization parameter estimator uses an interactive parameter estimation approach, which makes use of a Gibbs Sampling technique in the iteration. 
     Referring to  FIGS. 3-9 , the normalization parameter estimator  130  makes use of a number of procedures to estimate the parameters
 
Pr(Hit), {μ H,i , μ M,i } i=1,L , σ H   2 , σ M   2 :
 
     The normalization parameter estimator  130  estimates the parameter Pr(Hit) according to the fraction of the number of true hits to the total number of detections. Alternatively, this parameter is set to quantity that reflects the estimated fraction of events that will be later detected by the word spotting engine on the unknown speech, or set to some other constant according to other criteria, such as by optimizing the quantity to increase accuracy. 
     The normalization parameter estimator  130  estimates the parameters for the hits, {μ H,i } i=1,L , σ H   2  from the set to true hits  132  independently of the corresponding parameters that it estimates from the false alarms  134 . For notational simplicity, we drop the subscript H and M in the following discussion, and refer to the entire set of values for either the hits or the misses as μ≡{μ □,i } i=1,L . Similarly, the entire set of queries and their corresponding raw scores are denoted Q≡{q} and R≡{R (q) }, respectively. (In the discussion below, each element of the sets corresponds to a single instance of a query.) 
     Referring to  FIG. 3 , the overall parameter estimate procedure to determine ({circumflex over (μ)} (1) , {circumflex over (σ)} (1) ) makes use of a Gibbs Sampling approach that is implemented by the function Gibbs_sample() (line  300 ). (The Gibbs_sample() procedure is called twice, once for the hits, and once for the false alarms.) The first step of the procedure is to determine and estimate of the Maximum Likelihood (ML) estimate of the parameters, which optimally satisfies,
 
({circumflex over (μ)},{circumflex over (σ)})=arg max  P ( R|Q,  μ, σ) μ,σ
 
     A function em_estimate() is executed to yield an approximation ({circumflex over (μ)} (1) , {circumflex over (σ)} (1) ) of this ML estimate. The details of this procedure are discussed further below with reference to  FIGS. 4-6  that include the pseudocode for the function. 
     The Gibbs_sample() procedure continues with a three-step interation (lines  320 - 350 ). In the first step of the iteration (line  330 ), a function sample_factor() is used to generate a random sampling of the component scores based on the raw scores for the queries, and the current parameter values. This function yields a set {{tilde over (r)} (q) } with one vector element per query, where {tilde over (r)} (q) ≡({tilde over (r)} 1   (q) , . . . , {tilde over (r)} N   (q) ) is the vector of component scores for query q, and N is the length of the phonetic representation of q. For each of the queries, the component scores are drawn at random constrained to satisfy match the total raw score for the query, Σ i {tilde over (r)} i   (q) =R (q) . The sample_factor() function is described below with reference to  FIG. 7 . 
     In the next step of the iteration (line  340 ), the randomly drawn component scores are used in a function sample_mean() to reestimate the means of the component scores, {circumflex over (μ)} (i) =(μ 1   (i) , . . . μ L   (i) ) T . The sample_mean() is described below with reference to  FIG. 8 . 
     In the third and final step of the iteration (line  350 ), the randomly drawn component scores, and the newly updated means of the distributions of the component scores are used in a function sample_sig() to reestimate the shared standard deviation of the distributions, {circumflex over (σ)} (i) . 
     After the specified number of iterations (num_iter), the Gibbs_sample() procedure returns the current estimate of the parameters of the distributions for the component scores (line  360 ). 
     Referring to  FIG. 4 , the em_iterate() function (line  400 ) is called from the Gibbs_sample() function. Initial estimates for the parameters are first obtained using a initialize_iter() function (line  410 ). The procedure is relatively insensitive to this initial estimate, which can, for example, set all the mean parameters to a common shared value. 
     The em_iterate() makes use of the Estimate-Maximize (EM) algorithm, starting at the initial estimate ({circumflex over (μ)} (0) , {circumflex over (σ)} (0) ), and iterating until a stopping condition, in this case the maximum number of iterations num_iter, is reached. Each iteration involves two steps. First, a function expect_factor() (line  430 ) is used to determine expected values of sufficient statistics for updating the parameter values, and then a function maximize_like() (line  440 ) uses these expected values to reestimate the parameter values. After the maximum number of iterations is reached, the current estimates of the parameter values are returned as an estimate of the Maximum Likelihood estimate of the parameter values. 
     Referring to  FIG. 5 , the expect factor() function (line  500 ) iterates over each of the queries q (lines  510 - 530 ). For each query, the function first computes an expected value,  r 1     (q) , of the vector of component scores r (q) =(r 1   (q) ), . . . , r N   (q)  for the query, conditioned on the current estimates or the parameter values and on the value of the total raw score, R (q) , for the query (line  520 ). Then the function computes an expected value  r 2     (q)  of the (element wise) square of the component scores (line  530 ). 
     Referring to  FIG. 6 , the maximize_like() function (line  600 ) uses the expected values of the sufficient statistics by accumulating, for each phoneme k, a sum of the expected first and second order (squared) statistics corresponding to that phoneme into accum 1 [k], and accum 2 [k], respectively (line  620 - 630 ), as well as counting the total number of occurrences of that phoneme (line  640 ). The updated mean for each phoneme, {circumflex over (μ)} k , is computed as the average of the first order statistic (line  650 ). The updated standard deviation (square root of the variance), {circumflex over (σ)}, is computed based on the accumulated second order statistic and the updated means for the phonemes (line  670 ). The maximize_like() function then returns the updated mean and standard deviation estimates (line  680 ). 
     Referring to  FIG. 7 , the sample_factor() function (line  700 ) is used in the three-step iteration of the Gibbs_sample() function (see  FIG. 4 ). For each query, q, a vector of component scores {tilde over (r)} (q) ≡({tilde over (r)} 1   (q) , . . . , {tilde over (r)} N   (q)  ) is drawn at random from the distribution for those component scores conditioned on the total raw score for the query, R (q) , and the current estimates of the mean and standard deviation parameters of the component scores (line  730 - 740 ). The set of these random draws, {tilde over (r)}={{tilde over (r)} (q) } is returned by the function. 
     Referring to  FIG. 8 , the sample_mean() function takes the randomly drawn component scores and computes updated mean parameters for the phonemes by drawing from a normal distribution for each phoneme. For each phoneme, k, the mean of this distribution, {circumflex over (μ)} k , is computed as essentially the average of the corresponding randomly drawn component scores (lines  820 - 840 ). The standard deviation of the distribution, {circumflex over (σ)} k , is taken to be the current estimate of the standard deviation divided by the number of occurrences of the phoneme (line  850 ). The updated value of the mean parameter, {tilde over (μ)} k , is then drawn at random (line  860 ). The vector of all the randomly drawn mean parameters is then returned by the function (line  870 ). 
     Referring to  FIG. 9 , the sample_sig() function is used to update the standard deviation of the distributions of the component scores. The standard deviation is drawn from an Inverted Gamma (IG) distribution (line  930 ). The parameters of the IG function are one half the count of the total number of phonemes in all the queries (line  920 ), and one half the sum of the squared deviations of the of the randomly drawn component scores, r i   (q)  from the means for the corresponding phonemes s i   (q) . 
     In an optional mode, the normalization parameter estimator does not assume that the variances of the component score distributions are tied to a common value, and it independently estimates each variance using a variant of the procedures shown in  FIGS. 3-9  and discussed above. 
     In alternative embodiments, different forms of probability distributions, and different parameter estimation methods are used. These estimates can form Maximum Likelihood (ML), Maximum A Posteriori (MAP), Maximum Mutual Information (MMI), or other types of estimates of the parameter values. Various types of prior distributions of parameter values can be used for those estimation techniques that depend on such prior estimates. Various numerical techniques can also be use to optimize or calculate the parameter values. 
     In the discussion above, each putative instance of a query is associated with a particular phoneme sequence. In alternative forms of the approach, each query may allow multiple different phoneme sequences, for example to allow alternative pronunciations or alternative word sequences. In this alternative approach, the phoneme sequence associated with an instance of a query (hit or miss) can be treated as unknown or as a random variable, which can have a prior distribution based on the query. Also, as introduced above, the subword units are not necessarily phonemes. Larger linguistic units such as syllables or demi-syllables whole words can be use, as can arbitrary units derived from data. Also, other forms of models, both statistical and non-statistical, can be used by the word spotting engine to locate the putative events with their associated scores. 
     The system described above can be implemented in software, with instructions stored on a computer-readable medium, such as a magnetic or an optical disk. The software can be executed on different types of processors, including general purpose processors and signal processors. For example, the system can be hosted on a general purpose computer executing the Windows operating system. Some or all of the functional can also be implemented using hardware, such as using ASICs or custom integrated circuits. The system can be implemented on a single computer, or can be distributed over multiple computers. For example, the training subsystem can be hosted on one computer while the runtime component is hosted on another component. 
     It is to be understood that the foregoing description is intended to illustrate and not to limit the scope of the invention, which is defined by the scope of the appended claims. Other embodiments are within the scope of the following claims.