Pattern and speech recognition using accumulated partial scores from a posteriori odds, with pruning based on calculation amount

A speech recognition apparatus includes a data input portion to which input data which is a speech pattern is applied, a score calculation portion calculating a score indicating a possibility of recognition of a partial pattern of the speech pattern based on the estimate of a posteriori odds, an optimization design portion designing optimized parameters for calculating the estimate of the a posteriori odds in the score calculation portion and/or optimized parameters of pruning functions controlling calculation amount in the pruning processing portion, a pruning processing portion pruning the score for making operation efficient, an accumulated score calculating portion accumulating pruned scores to calculate an accumulated score, a recognition result decision portion classifying input data for every class based on the accumulated score and deciding a recognition result, and a recognition result output portion providing the recognition result.

BACKGROUND OF THE INVENTION 
1. Field of the Invention 
The present invention generally relates to a pattern recognition method, a 
speech recognition method, and a speech recognition apparatus, and more 
particularly, to a pattern recognition method, a speech recognition 
method, and a speech recognition apparatus using a posteriori odds. 
2. Description of the Background Art 
Natural and spontaneous speech utterance does not always follow a 
grammatical rule. In addition, natural speech often contains a variety of 
acoustic phenomena such as interjections and tongue-clickings. One desired 
method for using such natural speech as an input interface to a system is 
to detect only prescribed key words important to the system from uttered 
speech together with their locations (word spotting), and to decide the 
most probable word sequence through a high-level processing such as a 
syntactic analysis based on the detected result. 
One method for word spotting is to calculate any score indicating a degree 
of matching with respect to a target word for every partial pattern of 
observed speech and to compare the score with a threshold value to decide 
a word candidate. In this case, the threshold value was chosen 
heuristically. Further, since the score of an individual word candidate is 
normalized so as not to depend on time length of the word for comparison 
with the threshold value, possibilities of recognition of word sequences 
different in length cannot be directly compared by simply accumulating 
word scores. Mainly for this reason, only a conventional word sequence 
score in this framework based on heuristics was available. 
In another method for word spotting, a "garbage class", that is, a class 
containing every acoustic phenomenon other than target words, is prepared 
in addition to each class of the target words, and a word sequence is 
recognized regarding observed speech as uninterrupted continuation of 
acoustic phenomena of these classes. This method suffers from the same 
problem as that of a conventional continuous speech recognition. More 
specifically, an accumulated score must be stored and calculated for every 
grammatical node at each time. The number of grammatical nodes increases 
explosively as the grammatical complexity increases, making the 
calculation impractical. In order to prevent this, a method (beam search) 
is employed in which a grammatical node having a low accumulated score is 
rejected during calculation. This beam search is also just one of 
heuristics. 
A strategy common to these methods is to reduce a large amount of 
calculation for obtaining the most probable word sequence from observed 
speech by pruning candidates based on empirical knowledge at the sacrifice 
of theoretical optimality. In this case, even if a score calculation 
portion and a candidate pruning portion are optimally designed, the entire 
system will not be optimized, as far as different criteria are used for 
respective portions. The entire system including the score calculation 
portion and the candidate pruning portion must be directly optimized by 
using a single objective function. However, the score calculation portion 
and the candidate pruning portion were separately designed in a 
conventional design of a speech recognition apparatus based on word 
spotting. 
In brief, a continuous speech recognition apparatus based on word spotting 
was structured of two processing mechanisms; pruning of the number of 
partial hypotheses based on a score and decision of a word sequence 
according to combinations of the partial hypotheses. Because of the 
structural complexity, the design was only partially or heuristically 
optimized. An optimization method of the entire system having a 
theoretical background was not proposed. 
On the other hand, a generalized probabilistic descent method (GPD), which 
has recently been presented, proposes a framework of optimization 
according to gradient search of a system including a discontinuous 
processing such as selection of a minimum value and decision, by employing 
approximation using a 1st-differentiable continuous function. A learning 
method for minimizing an error rate in speech classification is proposed 
as a specific application of the GPD. Further, the GDP is also applied to 
spotting. 
SUMMARY OF THE INVENTION 
One object of the present invention is to provide a speech recognition 
method for, using a score based on the a posteriori odds, which has never 
been proposed, pruning the number of partial hypotheses based on the 
score, and deciding a word sequence according to combinations of the 
partial hypotheses to recognize speech, and a speech recognition apparatus 
therefore. 
Another object of the present invention is to provide a speech recognition 
method for optimizing the entire system including both a score decision 
portion and a candidate pruning portion based on the a posteriori odds, 
and a speech recognition apparatus therefore. 
Still another object of the present invention is to provide not only a 
speech recognition method and a speech recognition apparatus but also a 
pattern recognition method for, using a score based on the a posteriori 
odds, pruning the number of partial hypotheses based on the score, and 
deciding a continuous pattern according to combinations of the partial 
patterns to recognize the pattern. 
In one aspect of the present invention, a pattern recognition method for 
recognizing a plurality of partial patterns of a continuous pattern to 
recognize the continuous pattern as a combination of the partial patterns 
includes the first step of deciding based on an estimate of the a 
posteriori odds a plurality of physical quantities each indicating a 
possibility of recognition of each partial pattern and corresponding to 
each partial pattern, the second step of deciding an accumulated physical 
quantity indicating a possibility of recognition of a combination of the 
partial patterns based on each physical quantity, and the third step of 
classifying the continuous pattern into one of predetermined classes of 
combination based on the decided accumulated physical quantity. 
Therefore, according to the present invention, since possibilities of the 
plurality of partial patterns forming the continuous pattern are decided 
according to the physical quantities based on the estimate of the a 
posteriori odds, and the continuous pattern is recognized based on a 
possibility of the combination of the partial patterns decided by 
accumulation of the physical quantities, pattern recognition supported 
theoretically can be carried out. 
In the one aspect of the present invention, the first step includes the 
step of pruning partial pattern candidates for decision of the accumulated 
physical quantity or classification of the continuous pattern. Therefore, 
it is possible to make recognition of the continuous pattern efficient. 
In the one aspect of the present invention, the first step includes the 
step of finding the estimate of the a posteriori odds using a parameter 
maximizing likelihood of the estimated a posteriori odds for decision of 
the physical quantity. Therefore, it is possible to optimize recognition 
of the continuous pattern. 
In the one aspect of the present invention, the first step includes the 
step of representing a classification error rate as a predetermined 
function and finding the estimate of the a posteriori odds using a 
parameter minimizing a value of the function for decision of the physical 
quantity. Therefore, it is possible to optimize recognition of the 
continuous pattern. 
In the one aspect of the present invention, the first step includes the 
step of representing a classification error rate and a calculation amount 
as a predetermined function and finding the estimate of the a posteriori 
odds using a parameter minimizing a value of the function for decision of 
the physical quantity. Therefore, it is possible to optimize recognition 
of the continuous pattern. 
In the one aspect of the present invention, the first step includes the 
step of representing a classification error rate as a predetermined 
function and finding the estimate of the a posteriori odds using a 
parameter minimizing a value of the function for decision of the physical 
quantity and pruning of the partial patterns. Therefore, it is possible to 
optimize recognition of the continuous pattern. 
In the one aspect of the present invention, the first step includes the 
step of representing a classification error rate and a calculation amount 
as a predetermined function and finding the estimate of the a posteriori 
odds using a parameter minimizing a value of the function for decision of 
the physical quantity and pruning of the partial patterns. Therefore, it 
is possible to optimize recognition of the continuous pattern. 
In another aspect of the present invention, a speech recognition method for 
recognizing a plurality of partial patterns of a time series speech 
pattern to recognize the speech pattern as a combination of the partial 
patterns includes the first step of deciding based on an estimate of the a 
posteriori odds a plurality of scores each indicating a possibility of 
recognition of each partial pattern and corresponding to each partial 
pattern, the second step of deciding an accumulated score indicating a 
possibility of recognition of a combination of the partial patterns based 
on each score, and the third step of classifying the speech pattern into 
one of predetermined classes of the combination based on the decided 
accumulated score. 
Therefore, according to the present invention, since possibilities of the 
plurality of partial patterns forming the speech pattern are decided 
according to the scores based on the estimate of the a posteriori odds, 
and the speech pattern is recognized based on a possibility of a 
combination of the partial patterns decided by accumulation of the scores, 
speech recognition supported theoretically can be carried out. 
In the another aspect of the present invention, the first step includes the 
step of pruning partial pattern candidates for decision of the accumulated 
score or classification of the speech pattern. Therefore, it is possible 
to make recognition of the speech pattern efficient. 
In the another aspect of the present invention, the first step includes the 
step of finding the estimate of the a posteriori odds using a parameter 
maximizing likelihood of the estimated a posteriori odds for decision of 
the score. Therefore, it is possible to optimize recognition of the speech 
pattern. 
In the another aspect of the present invention, the first step includes the 
step of representing a classification error rate as a predetermined 
function and finding the estimate of the a posteriori odds using a 
parameter minimizing a value of the function for decision of the score. 
Therefore, it is possible to optimize recognition of the speech pattern. 
In the another aspect of the present invention, the first step includes the 
step of representing a classification error rate and a calculation amount 
as a predetermined function and finding the estimate of the a posteriori 
odds using a parameter minimizing a value of the function for decision of 
the score. Therefore, it is possible to optimize recognition of the speech 
pattern. 
In the another aspect of the present invention, the first step includes the 
step of representing a classification error rate as a predetermined 
function and finding the estimate of the a posteriori odds using a 
parameter minimizing a value of the function for decision of the score and 
pruning of the partial pattern candidates. Therefore, it is possible to 
optimize recognition of the speech pattern. 
In the another aspect of the present invention, the first step includes the 
step of representing a classification error rate and a calculation amount 
as a predetermined function and finding the estimate of the a posteriori 
odds using a parameter minimizing a value of the function for decision of 
the score and pruning of the partial pattern candidates. Therefore, it is 
possible to optimize recognition of the speech pattern. 
In still another aspect of the present invention, a speech recognition 
apparatus for recognizing a plurality of partial patterns of a time series 
speech pattern to recognize the speech pattern as a combination of the 
partial patterns includes a score decision portion for deciding based on 
an estimate of the a posteriori odds a plurality of scores each indicating 
a possibility of recognition of each partial pattern and corresponding to 
each partial pattern, an accumulated score decision portion for deciding 
an accumulated score indicating a possibility of recognition of a 
combination of the partial patterns based on each score, and a 
classification portion for classifying the speech pattern into one of 
predetermined classes of combination based on the decided accumulated 
score. 
Therefore, according to the present invention, since possibilities of 
recognition of the plurality of partial patterns included in the speech 
pattern are decided by the scores based on the estimate of the a 
posteriori odds, and the speech pattern is recognized based on a 
possibility of a combination of the partial patterns decided by 
accumulation of the scores, a speech recognition apparatus can be provided 
which can carry out speech recognition supported theoretically. 
In the still another aspect of the present invention, the speech 
recognition apparatus further includes a pruning portion for pruning 
partial pattern candidates for decision of the score or classification of 
the speech pattern. Therefore, it is possible to make recognition of the 
speech pattern efficient. 
In the still anther aspect of the present invention, the speech recognition 
apparatus further includes an optimization portion for optimizing the 
estimated a posteriori odds using a parameter maximizing likelihood of the 
estimate of the a posteriori odds for decision of the score. Therefore, it 
is possible to optimize recognition of the speech pattern. 
In the still another aspect of the present invention, the speech 
recognition apparatus further includes an optimization portion for 
representing a classification error rate as a predetermined function and 
optimizing the estimate of the a posteriori odds using a parameter 
minimizing a value of the function for decision of the score. Therefore, 
it is possible to optimize recognition of the speech pattern. 
In the still another aspect of the present invention, the speech 
recognition apparatus further includes an optimization portion for 
representing a classification error rate and a calculation amount as a 
predetermined function and optimizing the estimate of the a posteriori 
odds using a parameter minimizing a value of the function for decision of 
the score. Therefore, it is possible to optimize recognition of the speech 
pattern. 
In the still another aspect of the present invention, the speech 
recognition apparatus further includes an optimization portion for 
representing a classification error rate as a predetermined function and 
optimizing the estimate of the a posteriori odds using a parameter 
minimizing a value of the function for decision of the score and pruning 
of the partial pattern candidates. Therefore, it is possible to optimize 
recognition of the speech pattern. 
In the still another aspect of the present invention, the speech 
recognition apparatus further includes an optimization portion for 
representing a classification error rate and a calculation amount as a 
predetermined function and optimizing the estimate of the a posteriori 
odds using a parameter minimizing a value of the function for decision of 
the score and pruning of the partial pattern candidates. Therefore, it is 
possible to optimize recognition of the speech pattern. 
The foregoing and other objects, features, aspects and advantages of the 
present invention will become more apparent from the following detailed 
description of the present invention when taken in conjunction with the 
accompanying drawings.

DESCRIPTION OF THE PREFERRED EMBODIMENTS 
Before describing embodiments of the present invention with reference to 
the drawings, the present invention will be conceptually described. Since 
a speech recognition apparatus by word spotting is considered to recognize 
speech based on combination decisions of individual word candidates, 
formulation of a mechanism for solving a general combination decision 
problem will first be described. Secondly, a design criterion based on 
maximum likelihood estimation, a minimum decision combination error, and 
the minimum decision combination error to which a minimum calculating 
amount is added will be described as an optimization design method. 
Thirdly, a speech recognition apparatus will be described with reference 
to the drawings. 
Formulation of Decision Combination Mechanism 
1.1 Decision mechanism based on logarithmic a posteriori odds 
Description will be given of a decision mechanism in which a decision a is 
made based on an obtained evidence x. Assume that a set of decisions which 
may be selected is A.dagger. (.dagger. is to be attached to a set to be 
represented in a calligraphic character hereinafter)={.alpha..sub.j 
}.sub.j=1.sup.J. Since respective .alpha..sub.j are not always exclusive 
with each other in general, a plurality of decisions may be made from the 
evidence x, or no decision may be made from the evidence x. In such a 
case, not a.epsilon.A.dagger., but a.epsilon.P.dagger..OR 
right.p.dagger.(A.dagger.), wherein p.dagger.(A.dagger.) is a power set of 
A.dagger., indicating that a plurality of decisions are made in the case 
of .vertline.a.vertline.&gt;1, and that no decision is made in the case of 
a=.PHI. (.PHI. is a null set). 
Since decisions partially exclusive with each other may be made, P.dagger. 
is a subset of p.dagger. (A.dagger.) in general. 
When a posteriori probability Pr (a.vertline.x) of the decision a based on 
the obtained evidence x is given, if a* according to Bayesian decision 
rule, that is, according to the expression (1), is selected, a decision 
error is minimized. Assuming that a posteriori probabilities 
Pr(.alpha..sub.j .vertline.x) at which respective decisions .alpha..sub.j 
are made when the evidence x is obtained are independent from each other, 
the expression (2) holds. It should be noted that 1(.cndot.) is a 
two-valued function which takes 1 and 0 when a logical expression within 
parentheses is a true value and a false value, respectively. 
It should be noted also that 1(.cndot.) can also be a two-valued function 
represented by the equation (36) The two definitions of this function is 
distinguished by its argument; whether the argument is a logical 
expression or a numerical-valued expression. 
Here, a* can be obtained even in such a state as shown in the expression 
(5) using scores S.sub.a (x) by logarithm of a ratio of the a posteriori 
probabilities of the decision a to a decision .PHI. as represented by the 
expressions (3) and (4). In the expression (4), if Pr(.alpha..sub.j 
.vertline.x)/(1-Pr(.alpha..sub.j .vertline.x)) is defined as represented 
by the expression (6), O(.alpha..sub.j .vertline.x) is an amount, which is 
called a posteriori odds, indicating a possibility of recognition of the 
decision .alpha..sub.j based on the evidence x. The expression (5) 
indicates that a possibility of recognition of the decision a is estimated 
by a sum of the logarithmic a posteriori odds of individual decisions 
.alpha..sub.j .epsilon.a, and that the best decision a* can be made by 
searching for the maximum value. 
Based on the above, description will now be given of a decision mechanism 
in which a decision is made by comparison of the estimates of score 
S.sub.a (x) obtained by a sum of the logarithmic a posteriori odds. More 
specifically, when the estimates of the logarithmic a posteriori odds 
lnO(.alpha..sub.j .vertline.x) are obtained as .eta..sub.j (x;.LAMBDA.) by 
a parameter set .LAMBDA. of the entire system of the decision mechanism, a 
decision is made by search of the maximum value of the estimates S.sub.a 
(x;.LAMBDA.) of the score S.sub.a (x) shown in the expression (7). 
##EQU1## 
1.2 Decision combination mechanism based on accumulated score of 
logarithmic a posteriori odds 
The case is considered where a plurality of decision combination problems 
(a plurality of combined decision problems) are solved. Assume that 
decision on each element q.sub.i of a group Q={q.sub.i }.sub.i=1.sup.I of 
I problems {q.sub.i }.sub.i=1.sup.I is a.sub.i .epsilon.P.dagger.. When a 
group X={x.sub.i }.sub.i=1.sup.I of evidence x.sub.i of respective I 
problems {q.sub.i }.sub.i=1.sup.I is obtained, consider to find a group of 
the most probable decisions among a set .OMEGA..OR right.P.dagger..sup.I 
of decision groups which can be selected. When respective problems q.sub.i 
are independent from each other, the expression (8) holds for the a 
posteriori probability Pr(A.vertline.X) of a decision group A={a.sub.i 
}.sub.i=1.sup.I .epsilon..OMEGA.. Therefore, similar to the case of 
.sctn.1.1, if score S.sub.A (X) of a decision group A is defined as 
represented by the expressions (9) and (10), selection of a decision group 
A* shown in the expression (11) is equivalent to Bayesian decision rule. 
Since a true value of a posteriori probability Pr (A.vertline.X) is 
unknown actually, A* as represented by the expression (13) which maximizes 
score S.sub.A (X;.LAMBDA.) defined by a sum of the estimates of the 
logarithmic a posteriori odds represented by the expression (12) is 
regarded as an optimal decision group. 
##EQU2## 
1.3 Combination classification mechanism based on accumulated score of 
logarithmic a posteriori odds 
Then, consider the case where combinations are classified into any of C 
classes .OMEGA..sub.1, .OMEGA..sub.2, . . . , .OMEGA..sub.C exclusive with 
each other of a universal set .OMEGA. of a decision group A by an evidence 
group X. In the case of classification, c* shown in the expressions (14) 
and (15) based on Bayesian decision rule should be employed as a 
classification result. This decision rule may be replaced with a decision 
rule represented by the expressions (16) and (17), which are equivalent to 
the expressions (14) and (15). 
Since a true S.sub.A (X;.LAMBDA.) value is unknown actually, a score for 
every class may be decided as represented by the expression (18) based on 
the estimates S.sub.A (X;.LAMBDA.), and c* shown in the expression (19) 
may be employed as a classification result. Since it is difficult actually 
to find scores S.sub.A (X) for all elements A included in all classes, the 
score shown in the expression (20) may be alternatively used. A dynamic 
programming can be used for search of the maximum value, considerably 
reducing the amount of calculation. 
In the following, used is score S.sub.C (X;.LAMBDA.) represented by the 
expression (21) which is obtained by generalization of both scores 
represented by the expressions (18) and (20). It should be noted that 
.xi..sub.C is a positive constant. S.sub.C (X;.LAMBDA.) matches the score 
represented by the expression (18) when .xi..sub.C =1, and approaches the 
score represented by the expression (20) limitlessly when .xi..sub.C 
.fwdarw..infin.. 
##EQU3## 
1.4 Pruning of candidates for decision combination and decision combination 
classification 
In order to implement decision combination described in .sctn.1.2 and 
decision combination classification described in .sctn.1.3 on a computer, 
it is necessary to find scores S.sub.A (X;.LAMBDA.) for all possible 
decisions A. Since the total number of decision combinations A is 
O(2.sup.JI), however, the total number explosively increases with I and J 
increased. Although the total number is decreased to some extent in search 
of the maximum value by using a dynamic programming, a method which 
sacrifices optimality of a result must be employed, when it is desired to 
further decrease the amount of calculation. One method is pruning, in 
which, when it is heuristically determined for a certain partial decision 
combination B.epsilon.P.dagger..sup.I' (I'&lt;I) that all score values of 
decision combinations A including B as a part are negligibly small as 
compared to scores of the other decisions, the scores S.sub.A (X;.LAMBDA.) 
are excluded from computation without being computed to the end. If 
pruning is applied to all B.epsilon.P.dagger..sup.I' for a certain fixed 
I'&lt;I, and the number of the candidates is decreased to 1/N, for example, 
the total number can be decreased to 1/N.sup.I-I', resulting in dramatic 
reduction of the amount of calculation. 
Assume that .PHI.(A) denotes a universal set of partial decision 
combinations leading to a certain decision combination A which are used 
for pruning determination. If pruning determination according to partial 
decision combination B is represented by a function .omega..sub.B 
(X;.LAMBDA.), a pruning function .omega..sub.B (X;.LAMBDA.) takes 0 when 
it is determined that the score value of a decision combination including 
B as a part is negligibly small, and takes 1 otherwise. In decision 
combination and decision combination classification including a pruning 
processing, A.about.* and c.about.* represented by the expressions (24) 
and (25) are selected assuming that the score function is the expressions 
(22) and (23). In calculation of a score function S.about..sub.A 
(X;.LAMBDA.) represented by the expression (22), since the value of 
S.about..sub.A (X;.LAMBDA.) becomes negative infinite for decision 
combination candidates to be rejected, it cannot be an optimal solution. 
Thus calculation of the value of S.about..sub.2 (X;.LAMBDA.) can be 
omitted when it is known that the decision combination A is rejected, 
because in that case the value of exp(S.about..sub.2 (X;.LAMBDA.) in S in 
the expression (23) always equals to 0 in spite of the value of 
S.degree..sub.A (X;.LAMBDA.), which does not affect the value of 
S.degree.c(X;.LAMBDA.) in the expression (23). 
Although A.about.* and C.about.* do not match A* and C* in general, 
selection of pruning function .omega..sub.B (X;.LAMBDA.) considerably 
affects the frequency of mismatch and the amount of calculation. Taking 
that into consideration, a pruning function is appropriately selected 
based on empirical knowledge in general. 
##EQU4## 
II. Optimization Design Method for Decision Combination and Decision 
Combination Classification 
Description will be given here of a design method of a parameter set 
.LAMBDA. for a mechanism of deciding or classifying a decision group based 
on scores S.about..sub.A (X;.LAMBDA.), S.about..sub.C (X;.LAMBDA.) 
according to the expressions (22) and (23). 
2.1 Design based on maximum likelihood criterion 
The estimate .pi..sub.j (x;.LAMBDA.) of a posteriori probability 
Pr(.alpha..sub.j .vertline.x) of decision .alpha..sub.j based on a 
decision mechanism of a parameter set .LAMBDA. are given as represented by 
the expression (26) according to the expression (6). Assuming that p.sub.j 
(x) is a two-valued function which takes 1 when decision for evidence x is 
correct, and takes 0 otherwise, logarithmic likelihood l(x;.LAMBDA.) of 
the parameter set .LAMBDA. for evidence x is represented as in the 
expressions (27) and (28). At this time, a parameter set .LAMBDA.* as 
represented by the expression (30) maximizing a likelihood function 
L(.LAMBDA.) represented by the expression (29) is optimal under the 
maximum likelihood criterion. 
If l(x;.LAMBDA.) is a continuous function 1-st differentiable with respect 
to .LAMBDA., a suboptimal solution of .LAMBDA.* can be found numerically 
with a steepest descent method or a generalized probability descent 
method. 
This design method cannot consider a pruning processing. In addition, since 
a shape of distribution of a posteriori probability is generally unknown, 
such a parameter set based on maximum likelihood estimation obtained by 
assumption of a shape of distribution of a posteriori probability does not 
guarantee the minimum decision error for applied data. Unrobustness in the 
case where there is variation in the number of data for every class is 
also pointed out. Since the value of p.sub.j (x.sub.i) cannot be 
established for every evidence x.sub.i in decision combination 
classification, maximum likelihood estimation is difficult. An EM 
algorithm, for example, must be used. It is possible to give an 
appropriate initial value for a design method by a probabilistic gradient 
search shown hereinafter. 
##EQU5## 
2.2 Design based on minimum decision error criterion 
The most important object of a parameter set design is to minimize a 
decision error of a resultant decision mechanism. Therefore, a design 
method using direct minimization of a decision error as a criterion will 
be described in this section. 
When a correct decision group is A.sup.0, a cost function l(X;.LAMBDA.) is 
decided as represented by the expression (31). The cost function 
l(X;.LAMBDA.) is a function which takes 1 and 0 when the decision 
combination according to the expression (13) is incorrect and correct, 
respectively. Therefore, the expectation L(.LAMBDA.) represented by the 
expression (32) indicates a decision error rate. A parameter set 
.LAMBDA.** using the expectation L(.LAMBDA.) as a loss function and 
minimizing the same is a parameter set minimizing a decision combination 
error. 
In the case of classification, the cost function l(X;.LAMBDA.) represented 
by the expression (34) is used, assuming that a correct classification 
result is c.sup.0. 
Since a method for effectively finding .LAMBDA.** based on the above 
definition from a finite number of samples is unknown, the cost function 
is approximated by the continuous function l.about.(X;.LAMBDA.) 
represented by the expression (35). It should be noted that .zeta..sub.A 
is a positive constant, an operator .vertline..cndot..vertline. represents 
the total number of elements of a set, and that a function 
1.about.(.cndot.) is a 1-st differentiable continuous function which 
approximates a binary step as represented by the expression (36). A 
sigmoid function 1.about.(x)=1/{1+exp(-x)}, for example, is considered. 
The function 1.about.(.cndot.) is called a smooth step function 
hereinafter. 
A classification problem is approximated by the expression (37). It should 
be noted that (.zeta..sub.C is a positive constant. If 
l.about.(X;.LAMBDA.) is a continuous function 1-st differentiable with 
respect to .LAMBDA., a generalized probabilistic descent method can be 
used. By assuming that 1.about.(y).fwdarw.1(y), an approximation cost 
function can be limitlessly approached to a true cost function. 
##EQU6## 
2.3 Design based on criterion of amount of calculation added to minimum 
decision error 
A parameter set must be optimized in terms of not only minimization of a 
decision error rate, but also the amount of calculation. 
Define the cost function l(X;.LAMBDA.) of a decision error or 
classification error according to the expression (31) or (34), similar to 
the case of .sctn.2.2. In addition, define a cost function l'(X;.LAMBDA.) 
which directly reflects the amount of calculation, such as l'(X;.LAMBDA.) 
represented by the expression (38), wherein .psi.(X) is a set of partial 
decisions B used for pruning candidates when a decision group is obtained 
based on an obtained evidence group X. In this case, the expectation of 
l'(X;.LAMBDA.) represents insufficiency of pruning. It is expected that 
the more insufficient pruning is, the larger the amount of calculation is. 
When a loss function is defined, as represented by the expression (39), by 
the expectation L'(.LAMBDA.) of a weighted sum of two cost functions, 
either of which is multiplied by a positive constant .gamma., a parameter 
set .LAMBDA.'** which minimizes the L'(.LAMBDA.) as represented by the 
expression (40) can be regarded as an optimal parameter set in terms of 
both the number of decision errors and the amount of calculation. The 
degree of balance between these is controlled by the constant .gamma.. 
When both two cost functions l(X;.LAMBDA.) and l'(X;.LAMBDA.) are 
approximated by a continuous function 1-st differentiable with respect to 
.LAMBDA. using a smooth step function similar to the case of the 
expression (35), a suboptimal numerical solution of .LAMBDA.'** can be 
found with a generalized probabilistic descent method. 
##EQU7## 
It should be noted that the cost functions l (X;.LAMBDA.) and 
l'(X;.LAMBDA.) are not limited to the function decided according to the 
expression (31) or the like. 
III. Embodiment of Speech Recognition Apparatus 
FIG. 1 is a schematic block diagram showing a speech recognition apparatus 
according to one embodiment of the present invention. FIG. 2 is a flow 
chart for explaining operation of the speech recognition apparatus shown 
in FIG. 1. FIG. 3 is a schematic block diagram showing an internal 
configuration of an optimization design portion of FIG. 1. FIG. 4 is a 
flow chart for explaining operation of the optimization design portion of 
FIG. 3. 
In this section, the speech recognition apparatus will be described as a 
specific example of a decision combination mechanism. The speech 
recognition apparatus will first be described specifically with reference 
to FIGS. 1 to 4. Then, description will be given of a logarithmic a 
posteriori odds estimation function, recognition of a word sequence, 
pruning of a word and the like. 
Referring to FIG. 1, a speech recognition apparatus 1 includes a data input 
portion 5, a score calculation portion 7, a pruning processing portion 9, 
an accumulated score calculation portion 11, a recognition result decision 
portion 13, a recognition result output portion 15, and an optimization 
design portion 18. Input data 3 is applied to data input portion 5. Input 
data 3 is a time series speech pattern on speech, specifically. Based on 
input data 3 applied to data input portion 5, score calculation portion 7 
decides scores based on the a posteriori odds described in 5176 1.1. 
Pruning processing portion 9 prunes the scores calculated by score 
calculation portion 7 in order to facilitate the processing in accumulated 
score calculation portion 11 and recognition result decision portion 13. 
More specifically, pruning for decision combination and decision 
combination classification described in .sctn.1.4 is carried out, 
Accumulated score calculation portion 11 decides an accumulated score of 
the logarithmic a posteriori odds described in .sctn.1.2. The accumulated 
score is an accumulation of scores calculated by score calculation portion 
7. As a number of scores increases, the number of combinations for the 
accumulated score increases as described before. Therefore, pruning 
processing portion 9 restricts the number of scores as much as possible. 
Recognition result decision portion 13 decides a recognition result of 
speech based on the value of the accumulated score calculated by 
accumulated score calculation portion 11. Recognition result output 
portion 15 provides the result as a recognition result 17. 
Referring to FIGS. 3 and 4, description will now be given of optimization 
design portion 18 shown in FIG. 1. 
Optimization design portion 18 carries out optimization of only score 
decision by score calculation portion 7, or optimization of both score 
decision by score calculation portion 7 and pruning by pruning processing 
portion 9. Operation of optimization design portion 18 may be carried out 
sequentially, synchronously with or prior to operation of speech 
recognition apparatus 1. The optimization design includes a design based 
on the maximum likelihood criterion described in .sctn.2.1, a design based 
on the minimum decision error criterion described in .sctn.2.2, and a 
design based on pruning described in .sctn.2.3. Based on any of these 
designs, optimization design portion 18 includes an initial parameter 
input portion 21 to which an initial parameter 19 is applied, a learning 
data input portion 25 to which learning data 23 is applied, a parameter 
modification portion 27, and a modified parameter output portion 31 
providing a modified parameter 29. 
Initial parameter 19 is the above described parameter of system. Learning 
data 23 is data accompanied by a time series speech pattern and a correct 
recognition result of the speech pattern. Being accompanied by the correct 
recognition result, it is clear whether the recognition result of the time 
series speech pattern is correct or incorrect. Therefore, based on the 
correct recognition result, parameter modification portion 27 modifies a 
parameter, that is, the above described .LAMBDA.. If the maximum 
likelihood criterion described in .sctn.2.1, for example, is used as a 
criterion for modification, the parameter set is optimized under the 
maximum likelihood criterion. If parameter modification portion 27 
modifies the parameter based on the minimum decision error criterion in 
.sctn.2.2, the decision error rate of recognition result by recognition 
result decision portion 13 shown in FIG. 1 becomes minimized. If parameter 
modification portion 27 modifies the parameter based on the criterion of 
the amount of calculation added to the minimum decision error in 
.sctn.2.3, the parameter set is optimized under the criterion of the 
amount of calculation added to the decision error rate of recognition 
result by recognition result decision portion 13. 
As shown in the flow chart of FIG. 4, by repetition of the procedure from 
learning data input to judgement of completion of learning, optimization 
design portion 18 can carry out a more optimal parameter design. 
From .sctn.1.1 to .sctn.2.3, a set was used for description. As a 
generalized example, description will now be given of some cautions of the 
speech recognition apparatus hereinafter. First, assume that observed 
speech is X={x.sub.i .uparw.}.sub.i=1.sup.I, wherein each element is an 
S-dimensional real vector (x.sub.i .uparw..epsilon.R.sub.e.sup.S). When a 
word vocabulary is W.dagger.={w.sub.k }.sub.k=1.sup.K, it is assumed that 
the most probable word sequence W.sub.C* among all word sequences W.sub.C 
represented by the expression (41) is found when observed time series X is 
obtained with a set of grammatically allowable word sequences denoted as 
G. In the expression (41), lc is a length of a word sequence. 
Assuming that .OMEGA..sub.C represents a set of combinations resulting in 
word sequence W.sub.C among combinations for all s, e of decision 
a.sub.s.sup.e as to which word observed partial time series X.sub.s.sup.e 
represented by the expression (42) matches, optimization design portion 18 
can design parameter set .LAMBDA.'** using any of the three optimal 
criteria. 
EQU W.sub.c ={w.sub.k.sbsb.c (1), w.sub.k.sbsb.c (2), . . . , w.sub.k.sbsb.c 
(lc)}.epsilon.G (41) 
EQU X.sub.s.sup.e ={x.sub.s .uparw.,x.sub.s+1 .uparw., . . . , x.sub.e-1 
.uparw.,x.sub.e }(1.ltoreq.s.ltoreq.e.ltoreq.I) (42) 
3.1 Logarithmic a posteriori odds estimation function 
It is assumed that the logarithmic a posteriori odds of observed partial 
time series X.sub.s.sup.e being determined as a word w.sub.k is estimated 
to be Y.sub.k (X.sub.s.sup.e ;.LAMBDA.) by parameter set .LAMBDA.. It is 
assumed that each word w.sub.k is represented as concatenation of elements 
of a subword (phoneme or acoustic event, for example) set 
A.dagger.={.alpha..sub.j }.sub.j=1.sup.J, and that .LAMBDA. is 
(.LAMBDA.={.lambda..sub.J }.sub.j=1.sup.J) formed of model .lambda..sub.j 
for every subword. 
It is assumed that each subword model is (.lambda..sub.j ={R.sub.j, 
V.sub.j, .phi..sub.j .uparw.}) formed of a prototype R.sub.j, a 
variance-covariance matrix set V.sub.j, and a coefficient vector 
.phi..sub.j .uparw.. It is assumed that the prototype is N sets of M 
reference vectors concatenated continuously, and R.sub.j ={R.sub.jn 
={r.sub.jnm .uparw.}.sub.m=1.sup.M }.sub.n=1.sup.N, V.sub.j ={V.sub.jn 
={.SIGMA..sub.jnm }.sub.m=1.sup.M }.sub.n=1.sup.N, .phi..sub.j 
.uparw.={.phi..sub.j0, .phi..sub.j1 }, wherein r.sub.jnm 
.uparw..epsilon.R.sub.e.sup.S, .SIGMA..sub.jnm .epsilon.R.sub.e.sup.s*s 
(s*s means s.times.s). 
First, a model is assumed in which the logarithmic a posteriori odds of 
observed partial time series X.sub.s.sup.e being determined as subword 
.alpha..sub.j is estimated by a linear expression of a distance 
D(X.sub.s.sup.e, R.sub.j, V.sub.j) determined by variance-covariance 
matrix set V.sub.j of prototype R.sub.j, and .eta..sub.j (X.sub.s.sup.e 
;.LAMBDA.) is defined as represented by the expression (43). Here, 
distance D(X.sub.s.sup.e, R.sub.j, V.sub.j) is defined hierarchically as 
follows. 
First, define a distance .delta. between a vector x.sub.i .uparw. at a time 
i of observed speech and a reference vector r.sub.jnm .uparw. as 
represented by the expression (44) using a quadratic form determined by 
corresponding variance-covariance matrix .SIGMA..sub.jnm. This distance is 
called a local distance. 
Secondly, define a distance .DELTA. between the vector x.sub.i .uparw. at 
time i of observed speech and the n-th reference vector set R.sub.jn of 
prototype R.sub.j as represented by the expression (45), wherein 
.xi..sub.s is a positive constant. This distance is called a state 
distance. 
Thirdly, consider a distance in a matching path between observed partial 
time series X.sub.s.sup.e and prototype R.sub.j. Matching path .theta., 
given in a form as represented by the expression (46), is a set of two 
dimensional coordinates matching time i=s, s+1, . . . , e-1, e and index 
n=1, 2, . . . , N of a reference vector set of the prototype according to 
{i, m}=(.iota..sub.l, .nu..sub.l }. It is also assumed that matching path 
.theta. satisfies all the end point condition as represented by the 
expression (47) and the order condition as represented by the expression 
(48). At this time, a distance in each matching path .theta. is defined as 
represented by the expression (49), wherein .rho..sub.l (.theta.) is a 
weighted factor depending on matching path .theta., which satisfies 
##EQU8## 
and .THETA..sub.s.sup.e is a universal set of matching path .theta. which 
is possible in an interval s, e!. This distance D.sub..theta. (.cndot.) 
is called a path distance. 
Finally, define a distance D(.cndot.) of the expression (43) as represented 
by the expression (50). This distance is called a general distance. In the 
expression, .xi..sub.G is a positive constant. 
Then using logarithmic a posteriori odds .eta..sub.j (.cndot.) for every 
subword, logarithmic a posteriori odds Y.sub.k (X.sub.s.sup.e ;.LAMBDA.) 
of word w.sub.k as shown in the expression (51) represented by 
concatenation of subwords .alpha..sub.j is defined hierarchically as 
follows. 
First, consider the logarithmic a posteriori odds in one subword boundary 
sequence of observed partial time series X.sub.s.sup.e. It is assumed that 
a boundary sequence .beta. represented by the expression (52) matches the 
partial time series represented by (53) and the first subword represented 
by (54) of word w.sub.k, and that it satisfies the end point condition 
represented by the expression (55). At this time, the a posteriori odds of 
word w.sub.k in each subword boundary sequence is decided as represented 
by the expression (56). 
Secondly, define corresponding a posteriori odds Y.sub.k of word w.sub.k as 
represented by the expression (57), wherein .xi..sub.w is a positive 
constant, and B.dagger..sub.s.sup.e is a universal set of boundary 
sequence .beta. which is possible in X.sub.s.sup.e. 
##EQU9## 
3.2 Recognition of word sequence 
Because of a characteristic of a problem of continuous speech recognition, 
a universal set .OMEGA. of decision combination for obtaining a word 
sequence is a set of A satisfying conditions including condition 1 as 
represented by the expression (58) since, when there exists a word, the 
word must be decided to be one word, and condition 2 as represented by the 
expression (59) since there must be no time overlapping between adjacent 
words. According to the condition 1, a word is uniquely decided from a 
partial decision satisfying a.sub.s.sup.e .noteq..PHI.. Since a time 
sequence of words is clarified according to the condition 2, a word 
sequence is also uniquely decided. 
According to the above described parameter set, the score of combination A 
of partial decisions a.sub.s.sup.e is calculated as represented by the 
expression (60). Therefore, the score of word sequence W.sub.c is 
calculated using the expressions (60) and (21). As described above, when 
accumulation of the logarithmic a posteriori odds is used as a score of a 
word sequence, the score can be compared irrespective of the number of 
words included in the word sequence. 
##EQU10## 
3.3 Pruning at word level 
Since there are .vertline.P.dagger..vertline.=J+1 kinds of decisions for 
all partially observed time series X.sub.s.sup.e, the total number of 
decision combinations A is on the order represented by the expression 
(61), which is relatively large. In this section, the case is considered 
where pruning at a word level is introduced in order to decrease the total 
number. As an example, a simple and classical method to be described next 
is employed here. More specifically, only partially observed time series, 
which satisfies both a condition 1 that the value of the score takes the 
maximum value among scores for neighboring partially observed time series, 
and a condition 2 that the value of the score exceeds the threshold value, 
is left for each word as a candidate of the same. 
Let B(w.sub.k .vertline.X.sub.s.sup.e) denote a partial decision that 
partially observed time series X.sub.s.sup.e matches word w.sub.k, the 
score of a word sequence including pruning based on the above conditions 
is represented as shown in the expression (62). A pruning function defined 
in the right side of the expression (63) is used in the expression (62), 
wherein it is assumed that .omega..sub.lk (.cndot.) and .omega..sub.2k 
(.cndot.) correspond to the conditions 1 and 2, respectively, and that 
they are decided as represented by the expressions (64) and (65), 
respectively. In these expressions, .kappa..sub.k is a constant, h.sub.k 
is a threshold value, and S.dagger..sub.k (e) is a set of values which a 
beginning end point s of word w.sub.k can take with respect to an ending 
end point e, .LAMBDA.={{.lambda..sub.j }.sub.j=1.sup.J, {h.sub.k 
}.sub.k=1.sup.K }. 
In terms of efficiency of search, .omega..sub.lk (.cndot.) divides for 
representation the condition that "the score is maximum in neighboring 
partially observed time series" into two maximum conditions regarding the 
beginning end point and the ending end point. When the function 
represented by the expression (63) is approximated by a continuous 
function in order for application of optimization in .sctn.2.3, the 
function is decided as represented by the expression (66). The right side 
of the expression (66) is decided by the functions represented by the 
expressions (67) to (69). 
##EQU11## 
3.4 Implementation 
As described above, a speech recognition apparatus using an optimization 
method should be implemented theoretically. However, what is given in 
actuality is only limited computer resources and limited learning samples. 
It is considered that we must face various problems actually. Description 
will now be given of how to cope with these problems practically. 
First, consider implementation of recognition and learning with a limited 
amount of calculation. In definition of a score according to a logarithmic 
a posteriori odds estimation function, such a form as represented by the 
expression (21) is often used. As described above, a large amount of 
calculation is required in order to find the value of score. Therefore, a 
positive infinity limit of a constant such as .xi..sub.C, that is, a 
maximum value or a minimum value, is alternatively used, if necessary. As 
a result, it is possible to reduce the amount of calculation by a dynamic 
programming or the like. 
If, for example, a sigmoid function is used as a smooth step function 
1.about.(.cndot.) which appears on definition of a pruning function 
according to the expression (63), the value of the sigmoid function is 
always positive. Therefore, all the word hypotheses to be originally 
rejected in pruning must be left at the time of learning. Therefore, a 
piece-wise linear function represented by the expression (70) or a 
piece-wise parabola function represented by the expression (71) which 
always takes 0 is alternatively used at values far from the threshold 
value. 
Secondly, since an optimal parameter must be estimated from a limited 
number of learning samples in practice, the number of free parameters must 
be decreased as much as possible. For example, since variance-covariance 
matrix .SIGMA..sub.93 .sub.jnm in a subword model has a large number of 
free parameters, the variance-covariance must be fixed to an identity 
matrix or a diagonal matrix, so that only a diagonal element is a free 
parameter. In particular, to set parameters which seem to be less 
independent to a "tied" relation is effective, such as a common threshold 
value h.sub.k for pruning irrespective of a word, or representation of all 
word models by concatenation of a fewer kinds of subwords, as described 
above. 
Thirdly, since the method of this embodiment uses gradient search, care 
must be taken of a balance between adjustment quantities of respective 
parameters so that convergence will not be extremely delayed. Care must 
also be taken of a domain of variability of a parameter. Since a domain of 
variability of .phi..sub.j1 is (-.infin., 0), for example, .phi..sub.j1 
may sometimes go out of a domain of variability allowed by small 
modification based on definition of gradient search. In such a case, 
.phi..sub.jl may be replaced with .phi..sub.jl =-exp(.phi.'.sub.jl), so 
that .phi.'.sub.j1 is a free parameter of a domain of variability 
(-.infin., .infin.). Alternatively, the system may be configured without 
using parameters whose adjustment quantities are difficult to adjust. For 
example, in .eta..sub.j (X.sub.s.sup.e ; .LAMBDA.) of the expression (43) 
used in .sctn.3.1, coefficient vector .phi..sub.j .uparw. was used as a 
parameter. However, .eta..sub.j (X.sub.s.sup.e ; .LAMBDA.) is defined here 
by a function represented by the expression (72) or a function represented 
by the expression (73) as a possibility based on contention with a 
neighboring class or the above described "garbage class". In these 
expressions, .zeta..sub.D is a positive constant, and .lambda..sub.0 
={R.sub.0, V.sub.0 } is a model of "garbage class". 
##EQU12## 
IV. Conclusion 
As a framework for solving a decision combination problem involving 
uncertainty, description was made of a design method minimizing the number 
of decision errors and the amount of calculation in a framework including 
formulation of what is based on accumulation of logarithmic a posteriori 
odds and pruning for reducing the amount of calculation or saving the 
storage capacity. A framework of a speech recognition method and a speech 
recognition apparatus was described as this design method. However, this 
framework can be applied not only to speech recognition but also to an 
inference problem in general involving uncertainty. In only the field of 
speech recognition, there are many options in logarithmic a posteriori 
odds estimation functions and pruning criteria. It is necessary to decide 
the logarithmic a posteriori odds estimation function or the pruning 
criterion taking various conditions into consideration. 
The decision mechanism can be applied to a parallel distributed computer, 
since the mechanism uses scores individually evaluated for a plurality of 
decisions which can be made from one evidence. 
As described above, according to the embodiments of the present invention, 
since respective possibilities of recognition of a plurality of partial 
patterns forming a continuous pattern are decided by physical quantities 
based on the estimate of a posteriori odds, and the continuous pattern is 
recognized based on a possibility of a combination of the partial patterns 
decided by accumulation of the physical quantities, pattern recognition 
supported theoretically can be carried out. 
According to the embodiments of the present invention, by pruning partial 
pattern candidates based on physical quantities based on the estimate of 
the a posteriori odds, calculation of an accumulated physical quantity is 
facilitated, resulting in efficient pattern recognition. 
Further, according to the embodiments of the present invention, by using a 
parameter optimized based on a predetermined criterion, optimized pattern 
recognition can be carried out. 
Further, according to the embodiments of the present invention, since 
respective possibilities of recognition of a plurality of partial patterns 
forming a speech pattern are decided by scores based on the estimate of 
the a posteriori odds, and the speech pattern is recognized based on a 
possibility of a combination of the partial patterns decided by 
accumulation of the scores, speech recognition supported theoretically can 
be carried out. 
Further, according to the embodiments of the present invention, by pruning 
partial pattern candidates based on the score based on the estimate of the 
a posteriori odds, calculation of an accumulated score is facilitated, 
resulting in efficient speech recognition. 
Further, according to the embodiments of the present invention, by using a 
parameter optimized based on a predetermined criterion, optimized speech 
recognition can be carried out. 
Although the present invention has been described and illustrated in 
detail, it is clearly understood that the same is by way of illustration 
and example only and is not to be taken by way of limitation, the spirit 
and scope of the present invention being limited only by the terms of the 
appended claims.