Method and apparatus for language and speaker recognition

An initial learning phase creates histograms for each of the languages to be recognized. A first pass enters a number of samples of speech, and at each predetermined instant of time, each sample of speech is Fast Fourier Transformed (FFT) to create a spectrum showing frequency content of the speech at that instant of time (a spectral vector). The frequency content is compared with frequency contents which have been previously stored. If the current spectral vector is close enough to a previously stored spectral vector, a weighted average between the two is formed, and a weight indicating frequency of occurrence is incremented. If the current value is not similar to one which has been previously stored, it is stored with an initial weight of "1". The most common frequency spectra are determined for all of the languages grouped together to form a composite basis set. A second pass then puts a sample of sounds through the Fast Fourier Transform to again obtain frequency spectrums. The obtained frequency spectrums are compared against all of the prestored frequency spectra in the composite basis set, and a closest match is determined. A number of occurrences of each frequency spectra in the composite basis set is plotted as a histogram. This histogram is used during the recognition phase to determine a closest fit between an unknown language and one of the known languages.

FIELD OF THE INVENTION 
The present invention defines a method and apparatus for recognizing 
aspects of sound. More specifically, the invention allows recognition by 
prestoring a histogram of occurrences of spectral vectors of all the 
aspects and building an occurrence table of these spectral vectors for 
each known aspect. Pattern recognition is used to recognize the closest 
match to this occurrence table to recognize the aspect. 
BACKGROUND OF THE INVENTION 
There are many applications where it is desirable to determine an aspect of 
spoken sounds. This aspect may include identifying a language being 
spoken, identifying a particular speaker, identifying a device, such as a 
helicopter or airplane and a type of the device, and identifying a radar 
signature, for instance. For instance, a user may have a tape recording of 
information, which the user needs to understand. If this information is in 
a foreign language, it may be required to be translated. However, without 
knowing what language the information is in, it will be difficult for the 
user to choose a proper translator. 
Similarly, it may be useful, when processing tape recordings, to determine 
who is the speaker at any particular time. This will be especially useful 
in making transcripts of a recorded conversation, when it may be difficult 
to determine who is speaking and at what time. 
It is well known that all language is made up of certain phonetic sounds. 
The English language, for example, has thirty-eight phonetic sounds that 
make up every single word. In average English continuous speech, there are 
approximately ten phonetic sounds which are uttered every second. Other 
languages are composed of other phonetic sounds. 
Prior techniques for recognizing languages have attempted to identify a 
number of these phonetic sounds. When a determined number of phonetic 
sounds are identified, a match to the particular language which has these 
phonetic sounds is established. However, this technique takes a long time 
to determine the proper language, and may allow errors in the language 
determination. 
The inventor of the present invention has recognized that one reason for 
this is certain phonetic sounds are found in more than one language. 
Therefore, it would take a very long time to recognize any particular 
language, as many of the phonetic sounds, some of which are infrequently 
uttered, will have to be recognized before a positive language match can 
be determined. 
The present invention makes use of this property of languages in a new way 
which is independent of the actual phonetic sounds which are being 
uttered. 
SUMMARY OF THE INVENTION 
The present invention obviates all these problems which have existed in the 
prior art by providing a new technique for recognizing aspects of sound. 
According to the present invention, these aspects can include identifying 
a language being spoken, identifying a particular speaker, a device, a 
radar signature, or any other aspect. Identifying the language being 
spoken will be used herein as an example. One aspect of the invention 
creates energy distribution diagrams for known speech. In the preferred 
embodiment, this is done by using an initial learning phase, during which 
histograms for each of the languages to be recognized are formed. This 
learning phase uses a two pass process. 
The preferred embodiment uses a two pass learning technique described 
below. A first pass enters a number of samples of speech, and each of 
these samples of speech are continually processed. At each predetermined 
instant of time, each sample of speech is Fast Fourier Transformed (FFT) 
to create a spectrum showing frequency content of the speech at that 
instant of time (a spectral vector). This frequency content represents a 
sound at a particular instant. The frequency content is compared with 
frequency contents which have been stored. If the current spectral vector 
is close enough to a previously stored spectral vector, a weighted average 
between the two is formed, and a weight indicating frequency of occurrence 
is incremented. If the current value is not similar to one which has been 
previously stored, it is stored with an initial weight of "1". 
The end result of this first pass is a plurality of frequency spectrums for 
the language, for each of a plurality of instants of time, and numbers of 
occurrences of each of these frequency spectrum. The most common frequency 
spectrum, as determined from those with a highest number of occurrences, 
are determined for each language to form a basis set for the language. 
Each of these frequency spectrum for each of the languages are grouped 
together to form a composite basis set. This composite basis set therefore 
includes the most commonly occurring frequency spectrum for each of the 
many languages which can be recognized. 
A second pass then puts a sample of sounds, which may be the same sounds or 
different sounds than the previously obtained sounds, through the Fast 
Fourier Transform to again obtain frequency spectrums. The obtained 
frequency spectrums are compared against all of the pre-stored frequency 
spectra in the composite basis set, and a closest match is determined. A 
number of occurrences of each frequency spectra in the composite basis set 
is maintained. 
For each known language sent through the second pass, therefore, a number 
of occurrences of each of the frequency spectra for each of the languages 
is obtained. This information is used to form a histogram between the 
various spectra of the composite basis set and the number of occurrences 
of each of the frequency spectrum. 
This histogram is used during the recognition phase to determine a closest 
fit between an unknown language which is currently being spoken and one of 
the known languages which has been represented in terms of histograms 
during the learning phase. The unknown language is Fast Fourier 
Transformed at the instants of time as in the learning phase to form 
frequency spectrum information which is compared against the composite 
basis set used in the second pass of the learning phase. A histogram is 
formed from the number of occurrences of each element of the composite 
basis set. This histogram of the unknown language is compared against all 
of the histograms for all of the known languages, and a closest fit is 
determined. 
By using inter-language dependency in forming the known histograms and the 
unknown histogram, the possibility of error and the speed of convergence 
of a proper result is maximized. The inter-language dependencies come from 
the composite basis set including the most common spectrum distributions 
from each of the languages to be determined and not just from the one 
particular language. In addition, the use of spectral distributions at 
predetermined instants of time ensure that all phonetic sounds, and not 
just those which are the easiest to recognized using machine recognition, 
enter into the recognition process.

DESCRIPTION OF THE PREFERRED EMBODIMENT 
A preferred embodiment of the invention will now be described in detail 
with reference to the accompanying drawings. 
FIG. 1 shows an overview of the hardware configuration of the recognition 
system of the present invention. The initial data comes from an audio 
source 100 which can be a tape recorder, a radio, a radar device , a 
microphone or any other source of sound. The information is first 
amplified by amplifier 102, and then is band pass filtered by band pass 
filter 104. Band pass filter 104 limits the pass band of the filter to 
telephone bandwidths, approximately 90 Hz to 3800 Hz. This is necessary to 
prevent the so-called aliasing or frequency folding in the sampling 
process. It would be understood by those of skill in the art that the 
aliasing filter may not be necessary for other than speech applications. 
The band pass filtered signal 105 is coupled to first processor 106. First 
processor 106 includes an A-D converter 108 which digitizes the band pass 
filtered sounds 105 at 8 kHz to produce a 14 bit signal 110. The digitized 
signal 110 is coupled to a digital signal processor 112 which processes 
the language recognition according to the invention as will be described 
later with reference to the flowcharts. 
User interface is accomplished using a 82286-82287 microprocessor pair 
which is coupled to a user interface 116. 
The actual operation of the present invention is controlled by the signal 
processor 112, which in this embodiment is a TI TMS320C25. The code for 
the C25 in this embodiment was written in TI assembler language and 
assembled using a TI XASM25 assembler. This code will be described in 
detail herein. 
The first embodiment of the invention recognizes a language which is being 
spoken, from among a plurality of languages. The language recognition 
system of the present invention typically operates using pre-stored 
language recognition information. The general operation of the system is 
shown by the flowcharts of FIG. 2. 
FIG. 2A begins at step 200 with a learning mode which is done off line. In 
the learning mode, a known language is entered at step 202. This known 
language is converted into basis vectors or a set of representative sounds 
at step 204. The basis vectors are combined into a composite basis vector, 
and a histogram of occurances of the elements of the composite basis 
vector is created at step 206. 
FIG. 2B shows the recognition mode which is the mode normally operating in 
digital signal processor 112. FIG. 2B begins with step 220, in which the 
unknown language is entered at step 222. At step 223, the unknown language 
is compared with the basis vector to build a histogram. Euclidean distance 
to each of the basis vectors in the composite basis vector is determined 
at step 224 to recognize a language. 
This summary will be elucidated throughout this specification. 
The learning mode, summarized shown in FIG. 2A, is a mode in which the 
reference basis vectors and histograms, used to recognize the spoken 
language, is created. Once these vectors are created, they are 
user-transparent, and are stored in memory 122. Depending on the amount of 
memory available, many different basis vectors may be created and stored. 
For instance, different basis vectors can be created for all known 
languages, as well as all known dialects of all known languages. 
Alternately, only the most common ones may be created, if desired. The 
technique used to create the basis vectors will now be described in 
detail. This technique uses a two pass system of learning. In summary, the 
first pass determines all possible spectral contents of all languages, and 
the second pass determines the occurrences of each of these spectral 
contents. 
FIG. 3 shows the first pass of the learning mode of the present invention. 
The learning mode exposes the computer system to a known language such that 
the computer system, using the unique technique of the present invention, 
can produce the basis vectors used for later recognition of this known 
language. Using pattern recognition parlance, this is doing a "future 
selection". The technique of the present invention arranges these features 
in a sequence and uses them in a process called vector quantization which 
will be described herein. 
The first pass of the embodiment of the present invention creates a first 
bank of information for each language. The first pass uses at least five 
speakers, each of which speak for at least five minutes. A better 
distribution may be obtained by using five male and five female speakers. 
However, the actual number of speakers and time of speaking can obviously 
be changed without changing the present invention. 
The counter I is initialized in step 299. The data is entered into the 
system at step 300 where the A-D converter 108 digitizes the sounds every 
16 ms (8 kHz). A 128 point butterfly Fast Fourier Transform (FFT) is done 
at step 302 after 128 samples are taken. This equivalently creates 
information which represents the energy in each of a plurality of 
frequency cells. The 128 point FFT results in sixty-four indications of 
energy, each indioating an energy in one spectral range. Each of these 
numbers is represented in the computer by a word, and each of the 
sixty-four words represent the energy in one of the cells. The cells are 
evenly spaced from 0 to 3800 hertz, and therefore are each separated by 
approximately 60 Hz. Therefore, the sixty-four numbers represent energy in 
60 hertz cells over the spectral range extending from 0 to 3800 Hz. 
The 128 point FFT gives us 64 numbers representing these 64 cells. 
Therefore, for instance, cell 1 covers from 0 through approximately 60 
hertz (this should always be zero due to the bandpass filtering below 90 
hertz). Cell 2 covers approximately 60 through approximately 120 hertz. . 
. Cell 64 covers approximately 3740 through 3840 hertz. Each of the cells 
is represented by two 8-bit bytes or one computer word. The 64 word array 
therefore represents a spectral analysis of the entered sound at a 
snapshot of time. The 64 computer words, taken as a whole, are called the 
SPECTRA vector. At any given time, this vector represents the energy 
distribution of the spoken sound. 
Therefore, for each period of time, the process gives us 64 words of data. 
This data is then stored in an array called SPECTRA, which has 64 memory 
locations. Since this information is also obtained every period of time, 
the array in which it is stored must also have a second dimension for 
holding the information obtained at each period of time. 
If the amount of memory available was unlimited, all of the data could 
simply be stored as it is obtained, in the array SPECTRA at location i 
(where i has been initialized to 1 at step 299) and be incremented at each 
16.times.128 milliseconds to produce an array of data for later 
processing. However, for a five-minute processing sequence, this would 
produce i={(5 speakers) (5 min) (60 sec/min) (8000 samples/sec) (64 
spectra/sample-speaker) (1 word/location-spectra)}/ (128 
samples/location), which would require storage of about six million words 
of information (12 megabytes). While this is attainable, it would require 
expensive hardware. The preferred embodiment of the present invention 
processes the data as it is taken in to thereby minimize the amount of 
data storage which needs to be done. In order to do this, the present 
information must be compared with all previously stored information. 
This is done by setting up a loop at step 304 from 1 to the current point 
(i-1). During the first pass, no comparisons are made. Therefore, the 
information is stored in the array SPECTRA at position I (here 1) at step 
350. However, for all other passes besides the first pass, the loop set up 
at step 304 is executed. 
First, at step 306, the contents of the array SPECTRA at position n is 
obtained. While step 306 shows the value (N,64), it should be understood 
that this is shorthand for SPECTRA (1,1-64), and is intended to denote the 
contents of the entire SPECTRA vector from position 1 through position 64. 
Once SPECTRA (N,64) is obtained, the current values are compared with this 
stored SPECTRA (N,64) using a dot product technique at step 308. This dot 
product technique will be described in detail later on. To summarize, 
however, the dot product produces an angle indicative of a vector 
difference between the vector formed by the current values and the vector 
formed by SPECTRA (N,64), which is from 0.degree. to 90.degree.. This 
embodiment considers the two vectors to be similar if the angle of 
difference is than 2.5.degree.. 
If the angle is less than 2.5.degree., as determined at step 310, the 
vectors are considered similar, and a weighted average is calculated at 
step 312. An array of weights is stored as WEIGHT (N) in which the number 
of values which have been weighted in the array SPECTRA at position n is 
maintained. This value WEIGHT (N) is obtained and stored in a first 
temporary position T1. The value of the array SPECTRA (N,64) at position N 
is multiplied by T1 (the number of values making up the weighted value) 
and maintained at a second temporary position T2. A third temporary 
position T3 gets the value of the weighted SPECTRA value, added to the 
current values, to produce a new weighted value in position T3. The value 
WEIGHT (N) is then incremented to indicate one additional value stored in 
SPECTRA (N,64), and the new weighted average value of SPECTRA (N,64) is 
stored in the proper position by dividing the value of T3 by the 
incremented weight. 
A flag is also set to 0 indicating that the current value has been stored, 
and the loop is ended in any appropriate way, depending upon the 
programming language which is being used. 
If the result at step 310 is no, (the angle is not less than 2.5.degree.), 
the loop is incremented to the next N value at step 314. This is done 
until the last N value has been tested and therefore all of the values of 
SPECTRA array have been tested. 
If the angle is greater than 2.5.degree. for all values already stored at 
the end of the loop, this means that no previously stored value is 
sufficiently close to the current values to do a weighted average, and the 
current values therefore need to be stored as a new value. Therefore, step 
350 is executed in which the current values are stored in the array 
SPECTRA (I,64) at position I. Step 354 sets WEIGHT (I) of the weight 
matrix to 1, indicating that one value is stored in position i of SPECTRA. 
The value i (the pointer) is then incremented at step 356, and control 
then passes to position A in FIG. 3. Position A returns to step 300 where 
another sound is digitized. 
The loop is ended either by an external timer interrupt, or by the 
operator. A typical pass of information would be five minutes of 
information for five different speakers of each sex. This creates a set of 
features from the five speakers which indicates average spectral 
distributions of sound across these five people. 
The concept of dot product is well known in the field of pattern 
recognition, but will be described herein for convenience. Each set of 64 
values obtained from the FFT can be considered as a vector having 
magnitude and direction (in 64 dimensions). To multiply one vector by 
another vector, we obtain the following situation shown with reference to 
formula 1: 
EQU A.multidot.B615 =.vertline.A.vertline..multidot..vertline.B.vertline. cos 
.THETA. (1) 
where A and B are magnitudes of the vectors. The desired end result of the 
dot product is the value of the angle .THETA. which is the correlation 
angle between the two vectors. Conceptually, this angle indicates the 
similarity in directions between the two vectors. 
In order to calculate the dot product of the two vectors including the 64 
bits of information that we have for each, we must calculate the relation: 
##EQU1## 
Substituting between formula (1) and formula B allows us to solve for 
.THETA. as 
##EQU2## 
Therefore, if the two vectors are identical, the value .THETA. is equal to 
0.degree. and cos .THETA. is equal to 1. If the two vectors are completely 
opposite, the opposite identity is established. The dot product technique 
takes advantage of the fact that there are two ways of computing the dot 
product using formulas Nos. 1 and 2. This enables comparison between the 
two vectors. 
After pass 1 is completed, a number of basis vectors are obtained, and each 
one has a weight which indicates the number of the occurrences of that 
vector. The basis vectors created, along with the weights, are further 
processed in pass 2. 
It is understood that pass 1 should be processed in real time, to minimize 
the amount of memory used. However, with an unlimited storage, both pass 1 
and pass 2 could be performed as a single sample is taken. Alternately, 
with a sufficient amount of processor capability, both pass 1 and pass 2 
could simultaneously be processed while the data is being obtained. 
The pass 2 operation creates a histogram using information from the basis 
sets which have already been created in pass 1. This histogram represents 
the frequency of occurrence for each basis sound for each language or 
speaker. The key point of the present invention is that the histogram 
which is created, is an occurrence vector of each basis set among all 
basis sets for all languages to be recognized, and does not represent the 
basis sounds themselves. This will be described in detail with reference 
to FIG. 4 which represents the pass 2 technique. 
What is obtained at the end of pass 1 is an average of the spectral content 
of all occurrences of the sounds which have been detected in the language, 
and the weight (number of times of occurrence) for each spectrum. Each 
spectrum represents one basis vector, and each basis vector has a weight 
dependent on its frequency of occurrence. 
At the end of pass 1, we therefore have enough information to prepare a 
histogram between the different basis vectors in the language and the 
frequency of occurrence of each of these basis vectors. This would be 
sufficient to prepare a histogram which would enable the different 
languages to be recognized. However, pass 2 adds additional inter-language 
dependency to this technique which enables the recognition process to 
converge faster. 
Pass 2 can be conceptually explained as follows. Each language, as 
discussed above, consists of a number of phonetic sounds which are common 
to the language. By determining the frequency of occurrence of these 
phonetic sounds, the language could be recognized. However different 
languages share common phonetic sounds. 
To give an example, phonetic sound x may be common to English, French and 
German. It may even have a relatively high frequency of occurrence in all 
three languages. Phonetic sound y may also be common to English, French 
and German, but may have a high frequency of occurrence in the English 
language. In the other languages, phonetic sound y may have a low 
frequency of occurrence. Another problem with prior recognition systems is 
that some phonetic sounds are sub-vocalized, and therefore hard to 
recognize. The inventor of the present invention has recognized that the 
inter-language dependencies (that is, phonetic sounds which are common to 
multiple languages) enable ready recognition of the various languages. The 
inventor has also recognized that spectral distributions calculated at all 
times obviate the problem of difficulty of detecting sub-vocalized sounds. 
Pass 2 calculates the histograms by using all the values determined in pass 
1 for all languages, to add inter-language dependencies between the 
various 
languages. 
Pass 2 begins at step 399 which sets the histogram to zero, followed by 
step 400, where the composite basis set CBASIS is created. Step 400 gets 
the x most common SPECTRA values (those with the highest weights) for each 
of y languages to be recognized and stores this in the CBASIS array. In 
this embodiment, the preferred value of x is 15. If, for example, there 
are ten languages to be recognized, this yields 150.times.64 entries in 
the array CBASIS. 
Each of these 150 entries (x by y) represents a basis vector which has been 
found as having a high occurrence in one of the languages to be 
recognized. Each of these basis vectors which has a high frequency of 
occurrence in one language. By using each basis vector in each of the 
languages to be recognized, the inter-language dependencies of the various 
sounds (SPECTRA (64)) in each of the languages can be determined, not just 
those languages in which the values occur. 
Step 402 begins the second pass in which new sounds from the language to be 
recognized are obtained. These sounds are digitized and fast Fourier 
transformed in the same way as steps 300 and 302 of FIG. 3. 
The next step for each sound which is entered is to form the histogram for 
each known language. To do this, a for loop is set up between steps 404 
and 406, which increments between 1 and (x*y) (which is all of the various 
basis vectors). Within this loop, each element of the composite vector 
array CBASIS is compared with the current SPECTRA which has been obtained 
at step 410. The comparison is actually a comparison measure using 
euclidian distance, comparing the incoming SPECTRA (64) with each vector 
in the composite basis set CBASIS (n,64). Step 412 determines if this 
distance is less than 20,000. This value has been empirically determined 
as sufficiently close to represent a "hit". If the value is less than 
20,000, the value is compared against a previous lowest answer which has 
been previously stored. Those of ordinary skill in the art would 
understand that a very large "previous answer" is initially stored as an 
initial value. If the current answer is greater than the previous answer, 
flow passes to step 406 which increments the loop, without changing the 
current stored minimum. If the answer at step 412 is less than 20,000, and 
the answer at 414 is less than the previous answer, that means that this 
pass of the loop has received a lower answer than any previous pass of the 
loop. Accordingly, the current answer becomes the previous answer at step 
416, and the current count of the closest match, which is kept in a 
temporary location T1 becomes N (the current loop count). The loop is then 
incremented again at step 406. 
The temporary location T1 keeps the number of the lowest answer, and 
therefore the closest match. Accordingly, as long as any occurrences of 
the answer less than 20,000 have been determined at step 412, the 
histogram address of T1 is incremented at step 420. 
Therefore, the histogram array or vector is successively incremented 
through its different values as the loop is executed. Each of the 
different values of the histogram represent one specific sound or SPECTRA, 
among the set of sounds or SPECTRA making up each of the most common 
spectral distributions of each of the known languages. The effect is to 
find an average distribution for the particular language. This average 
distribution also includes the effect of interlanguage dependency. 
Pass 2 therefore provides us with a histogram in which each of a plurality 
of sounds or SPECTRA from each of the languages are plotted to show their 
number of occurrences. These reference histograms are used during the 
recognition phase, which will be described in detail with reference to 
FIG. 5. 
The FIG. 5 flowchart shows the steps used by the present invention to 
recognize one of the plurality of languages, and therefore is the one that 
is normally executed by the hardware assembly shown in FIG. 1. The 
learning modes will typically have been done prior to the final operation 
and are therefore are transparent to the user During the recognition mode, 
it is assumed that histograms for each of the languages of interest have 
therefore been previously produced. 
The objective of the recognition mode is to find the histogram vector, 
among the set of known histogram vectors, which is closest to the 
histogram vector created for the unknown language. This is done by 
determining the euclidian distances with the known language histogram 
vectors. If the nearest euclidian distance is sufficiently close, this is 
assumed to be a match, and therefore indicates a recognition. 
For purposes of explanation, the euclidian distance will now be described. 
So-called euclidian distance is the distance between two vector points in 
free space. Using the terminology that 
EQU A=A(64) B=B(64), then 
##EQU3## 
This is essentially the old c.sup.2 =a.sup.2 +b.sup.2 from euclidian 
geometry, expanded into 64 dimensions to meet the 64 SPECTRA values. The 
numbers for Ai and Bi can vary from 0 to 16383 (2.sup.14 for the 14 bit 
A-D). A distance match of 20,000 is empirically determined for these 
2.sup.14 bits. It is understood that for different numbers of bits, those 
of the ordinary skill in the art could and would be expected to find 
different empirical values. 
FIG. 5, showing the recognition phase, will now be explained. 
Step 500 is the initial step of the recognition phase, and could equally 
well be a part of the preformed data. Step 500 first loads the composite 
basis array CBASIS where the array has xy elements: the x most common 
SPECTRA values for each of the y languages to be recognized. Step 500 also 
loads y histograms, and using the CBASIS array and the y histograms forms 
a reference array. This reference array has a correlation between each of 
the y histograms, each of the xy SPECTRA in each of the histograms, and 
the values of the xy SPECTRA. 
Step 502 gets the sounds of the language to be analyzed and digitizes and 
FFTs these sounds, similar to the way this is done in steps 300 and 302 of 
FIG. 3. Step 504 compares the input sounds against silence. This is done 
according to the present invention by taking the sum of all of the SPECTRA 
cells, and adding these up. If all of these sounds add up to fourty or 
less, the SPECTRA is labeled as silence and is appropriately ignored. If 
the SPECTRA is determined not to be silence in step 504, a histogram for 
the language to be analyzed is created at step 506. This histogram is 
created in the same way as the histogram created in steps 404-420 of FIG. 
4, using all of the spectral categories for all of the languages i to be 
analyzed. This histogram is created for 3 seconds in order to form an 
initial histogram. 
Step 508 compares the histogram for the language to be analyzed to all 
elements of the reference array 1 through y where y is the number of 
languages being analyzed. This comparison yields a euclidian distance for 
each of the values 1 through y. Step 510 determines the minimum among 
these euclidian distances and determines if this minimum is less than 
20,000. If the minimum distance is not less than 20,000, step 512 updates 
the histogram for the language to be analyzed, and returns control to step 
508 to redo the test. At this point, we assume that the analysis has not 
"converged". However, if the result is positive at step 510, and the 
minimum distance is less than 20,000, then the minimum distance language 
is determined to be the proper one at step 513 thus ending the recognition 
phase. 
Therefore, it can be said that if the computed distance of the unknown 
versus the reference is the minimum between all the references and less 
than a user-chosen limit (here empirically determined to be 20,000), then 
we can say the unknown language has been recognized to be this minimum. 
Because of the inter-language dependencies which have been added into the 
histogram categories, the present invention enables a quicker 
determination of a proper language. Although a phonetic sound may be 
present in two or more languages, typically this phonetic sound will sound 
slightly different in different languages. By taking the SPECTRA 
distribution of this phonetic sound, and determining the minimum Euclidean 
distance, the closest possible fit is determined. Therefore, even if there 
are many similar sounds, the closest one will be chosen, thereby choosing 
the proper language even when the sounds are similar for different 
languages. This enables the recognition to converge faster. 
An additional nuance of the system averages all the language histograms and 
creates a null language. This null language is loaded as one of the y 
histograms. Whenever the system recognizes this null language as being the 
closest match, this is determined as a rejection of the language. 
A second embodiment of the invention operates similar to the first 
embodiment, but the aspect to be determined is optimized for speaker 
identification, as compared with language identification. Language 
identification identifies the language which is being spoken. Speaker 
identification identifies the specific speaker who is speaking the 
language. The techniques and concepts are much the same as the first 
embodiment. This second embodiment is shown in the flowchart of FIG. 6 in 
somewhat summary form. Step 600 executes pass 1 for each of the speakers 
to be recognized. Each speaker is executed for five minutes, or for some 
other user selectable amount of time. This creates a set of basis vectors 
for each of the z speakers to be recognized. The pass 2 system is executed 
at step 602 where the x most common SPECTRA values for each of the z 
speakers to be recognized is first determined to form CBASIS, or composite 
basis vector just as in pass 2 shown in FIG. 4. Step 604 then executes the 
rest of pass 2 (steps 402-420) with the only exception that step 412 in 
FIG. 4 is replaced with a comparison with 15,000 as the euclidian distance 
instead of the comparison with 20,000. This is because the match for 
speaker recognition is required to be closer than the necessary match for 
language recognition. At the end of step 604, the histograms for each of 
the speakers to be analyzed has been formed. Step 606 begins the recognize 
phase, and executes all elements of the recognize flowchart of FIG. 5 with 
the exception of step 510 in which the value to be compared with is 
15,000. 
The system is operated by use of a plurality of user friendly menus which 
enable the user to perform various functions. The main menu allows the 
user to choose between building new basis sets, looking at previously 
stored language histograms, or entering the recognized language's menu. 
Some sub-menus allow changing the rate of sampling, the number of points 
of FFT transformation, and the different ways in which the data is being 
distributed. 
A sample set of reference histograms for 
English, Chinese, and Russian are shown in FIGS. 7A-7C. These histograms 
show the sound indicated by numbers on the x axis, and show the number of 
occurrences on the y axis. These examples use only approximately 68 
different sounds as the possible sounds, but it is understood that many 
more than these are possible to be used. 
Many modifications in the above program and technique are possible. For 
instance, as stated above, it would be quite feasible to operate the 
entire learning phase in a single pass, assuming that sufficient 
processing speed and power and sufficient memory were available. This 
would obviate the need for two different entries of data. Of course, the 
various empirical values which have been described herein could be 
modified by users. In addition, any number of languages could be used by 
this system, and limited only by the amount of available memory space. 
In addition, other aspects of sound could be determined besides speaker 
identification and language identification including identification of 
dialects, possible area of origin of the speaker, and many other 
applications are possible. In addition, this technique could be used to 
identify a type of aircraft from its sound, or by converting a radar trace 
to sound, a radar signature could be identified. Of course, these examples 
are not limiting, and many other uses for the aspect recognition of the 
present invention. 
All of these modifications are intended to be encompassed within the 
following claims.