COMPUTER SYSTEM AND COMPUTER-IMPLEMENTED PROCESS FOR PHONOLOGY-BASED AUTOMATIC SPEECH RECOGNITION

Abstract: The present invention is based on the use of linguistic, especially phonological, knowledge to guide the speech recognition process. A speech signal containing an utterance is received and linguistic cues in the speech signal are detected. From these detected linguistic cues, a symbolic representation of the contents of the speech signal is generated. This symbolic representation comprises at least one word division, wherein each word division consists of an onset-rhyme pair and associated phonological elements. These phonological elements are univalent, may appear in all languages and are distinguishable from each other and directly interpretable in the speech signal. A lexicon of predetermined symbolic representations is provided for words in a particular language. A best match to the generated symbolic representation is found in the lexicon, thereby recognizing the spoken word.

Description: COMPUTER SYSTEM AND COMPUTER-IMPLEMENTED PROCESS FOR PHONOLOGY-BASED AUTOMATIC SPEECH RECOGNITION Claim to Copyright Ownership Portions of the disclosure of this patent document and the accompanying appendix contain material which is subject to copyright protection and for which copyright protection is expressly claimed. The copyright owner has no objection to the facsimile reproduction, e.g., photocopy, by anyone of the patent document at it a pears in files of any Patent Office, but otherwise reserves iii copyright rights whatsoever, for example, including but .-...t restricted to the right to load the software on a compuτer system. Field of the Invention The present invention is related to machine recognition of speech and spoken words. It is more particularly related to speech and word recognition based on a linguistic theory known as government phonology. Background of the Invention "Automatic speech recognition" signifies machine conversion of sounds, created by or simulating natural human speech, into a machine-recognizable representation indicative of a word or words actually spoken. Typically, sounds are converted to a speech signal, such as an analog or digital electrical signal, which the machine then processes. Automatic speech recognition involves recognizing a spoken word as such, not determining the meaning of the word. Automatic speech recognition may be either continuous or performed on isolated words. (Determining the meaning of a spoken word is a prcblem of speech understanding and may require, for example, that the contextual use of the word be analyzed) . To date, numerous machines and processes for automatic speech recornition have been proposed. Most currently commercially-available automatic speech recognition systems include computer programs which are used to process intensively a speech signal using statistical models of the speech signals generated from different spoken words. This technique is generally known as the hidden Markov model (HMM) method and is generally computationally intensive. Each word which can be recogr..zed by the machine typically must have a hidden Markov model derived for it based on the spectrum of one or more acousti: images of the word. Also, all of the HMMs for all of the words which the system is capable of recognizing typically must be derived together. Thus, adding a word to the set of words recognizable by the machine typically involves restructuring the whole lexicon. Some of these systems perform a type of segmentation of the speech signal to identify "syllables," which are then processed using HMMs. Another kind of automatic speech recognition system uses phoneme matrices. A phoneme matrix indicates mouth position over a pericd of time, according to binary, or bivalent, articulatory variables representing a vocal tract configuration used to create a sound. For example, there are about twenty binary features, recognized by a theory known as generative phonology, from which a phoneme matrix is constructed. A segmented phonetic sequence is extracted from a speech signal, and converted by a set of rules to obtain a phoneme matrix. The phoneme matrix is then compared to stored sample phoneme matrices for known words using time-warping techniques. This approach has been generally discredited because the rules used to generate phoneme matrices are generally arbitrary and are not likely to be adequate models of how humans process speech. Another kind of automatic speech recognition system is template-based. Each word has a template which represents the spectral evolution of the word over time. Such a system also uses time-varping techniques, but uses them to match the spectral change of an input speech signal with the stored templates. Summary of the Invention The most significant drawback of previous speech recognition systems is that either (l) they do not use any knowledge of linguistics or phonology to assist in the processing of t.-.e speech signal, or (2) they are based on a theory which rer.ders the speech recognition problem extremely difficult. For example, the statistical and template-based approaches perform pattern matching at the speech signal level. That is, a speech signal is matched directly with a representative speech signal or statistical model of a speech signal. In these systems, it is therefore more difficult to perform speaker-independent recognition because the effects of speaker variation can be found in speech signals used in the lexicon. The phoneme-based approaches, while attempting to use some linguistic knowledge, are based upon a generally discredited theory of phonology. It has been proven that the problems of extracting phonetic segments, and of determining and applying the rules by which a phonetic representation can be generated from these segments, are intractable. The rules which convert these segments to a phonemic representation are also language specific. Also, this kind of system relies on articulatory information for speech processing whereas a human must process speech using acoustic information. In th s invention, by applying and using knowledge of phonology and linguistics to guide the speech recognition process, an automatic speech recognition system has been developed that is less complex and less computationally intensive than prior systems. An appropriate theory of phonology cn which the invention can be based is government phonology. Some aspects of this theory are common with other theories of phonology which may also provide suitable bases for speech recognition systems. Because of the universality of this linruistic knowledge, a speech recognition system can be developei which is speaker-independent and is useful for multiple languages. The l.nguistic knowledge used in this invention includes tr.÷ fact that the speech signal includes acoustic cues to sic-.ificant linguistic information. These cues may be grouped together so as to define primitive phonological elements which are univalent. The elements are univalent because they are either present or not in a representation (and the speech signal); there is no definition of the absence of a phonological element. The use of these elements is based on the belief that speech processing uses the acoustic signal and not articulatory variables. This invention also utilizes the fact that each word includes one or more structural divisions each of which consist of two constituents, an onset; and a rhyme. According to government phonology, the structure of a word is constant, and is not affected by phonological processes. Therefore, each acoustically distinct word has only one unique phonological representation consisting of sequences of onset rhyme pairs, wherein each of the onset and rhyme constituents is associated with one or more phonological elements. Thus, with this system, the creation of lexical entries can be performed using textual or keyboard input, rather than by recording of actual speech. Also, the lexical representations are independent of speaker variation. Effects of speaker variation can be eliminated in the detection of the acoustic cues and in the matching process. Also, a word can be added to the lexicon without any need for adjusting the current lexicon. There are numerous other advantages of this system, as will be apparent from the following detailed description. Accordingly, one aspect of the invention is a machine-implemented process for recognition of a spoken word in an utterance. This process involves receiving an speech signal rontaining an utterance and detecting presence of linguistic cues in the speech signal. From detection of linguistic cues, a symbolic representation is generated. The symbol! : representation comprises at least one word division, each wcrd division consisting of an onset-rhyme pair and associated phonological elements. A best match to the symbolic representation is identified in a lexicon of symbolic representations of words, thereby recognizing the spoken word. In one embodiment, the step of generating involves detecting presence of univalent phonological elements in the speech signal according to detected linguistic cues, and associating each detected univalent phonological element with one of an onset or a rhyme constituent. In another embodiment, the step of detecting involves detecting voiced and voiceless regions in the speech signal, detecting sonorant and nucleus regions in the voiced regions of the speech signal, and detecting presence of phonological elements within each of any silence, fricative, sonorant and nucleus regions detected, and identifying each of the detected regions as defining either an onset or a rhyme. Another aspect of the invention an apparatus for recognition of a spoken word in an utterance. This apparatus includes 1) means for receiving an speech signal containing an utterance; 2) means for detecting presence of linguistic cues in the speech signal; 3) means for generating, from detection of linguistic cues, a symbolic representation comprising at least one word division, each word division consisting of an onset-rhyme pair and associated phonological elements; and 4) means for identifying a best match to the syrr-oolic representation in a lexicon of symbolic representations of words, thereby recognizing the spoken word. In one embodiment, the means for generating a symbolic representation comprises means for detecting presence of ur.i -alent phonological elements in the speech signal ac:.rding to detected linguistic cues, and means for asn ciating each detected univalent phonological element with one cf an onset or a rhyme constituent. In another emi;odiment, the means for detecting comprises means for detecting voiced and voiceless regions in the speech signal, means detecting sonorant and nucleus regions in the voiced regions of the speech signal, and means detecting presence of phonological elements within each of any silence, fricative, sonorant and nucleus regions detected, and identifying each of the detected regions as defining either an onset or a rhyme. Another aspect of the invention is an apparatus for f or recognition of a spoken word in an utterance. This apparatus includes a univalent element and structure detector having an input for receiving a speech signal and an output providing a symbolic representation of a word detected in the speech signal, wherein the symbolic representation comprises at least one word division, each word division consisting of an onset-rhyme pair and associated phonological elements. A lexicon of predetermined symbolic representations of words is also provided. A lexical matching system having a first input for receiving the symbolic representation from the output of the univalent element and structure detector, a second input for receiving predetermined symbolic representations from the lexicon and an output, provides an indication of the predetermined symbolic representation which best matches the symbolic representation output by the univalent element and structure detector. In one embodiment, the univalent element and structure detector includes a phonetic classifier and segmenter having an input for receiving features of the speech signal and an output, which provides a string of tokens indicative of presence of univalent phonological elements in the speech signal according to detected linguistic cues. A word parser has an input for receiving the string of tokens from the phonetic classifier and segmenter and associates each detected univalent phonological element with one of an onset or a rhyme constituent so as to provide the symbolic representation. Another aspect of the invention is an automatic speech recognition system comprising: means for detecting presence of at least one of a plurality of uniquely identifiable univalent phonological elements from a speech signal; means for associating each detected phonological element with either an onset or a rhyme; and means for comparing a combination of the detected phonological elements and associated onset and rhymes to a lexicon of possible combinations of such onset and rhyre divisions and phonological elements and for selecting from the lexicon a possible acceptable representation of a word most similar to the detected onset and rhyme divisions and phonological elements. Another aspect of the invention is a method for automatic speech recognition, in which presence of a plurality of uniquely identifiable phonological elements is detected in a speech signal. The detected phonological elements are associated with either an onset or a rhyme. The combination of the onset and rhyme divisions and the associated detected phonological c- ..e ents are compared to a lexicon of possible combinations of such onset and rhyme divisions and phonological elements from which is selected a possible acceptable representation of a word most similar to the detected onset and rhyme divisions and phonological elements. Brief Description of the Drawings In the drawing, Fig. 1 is a representation of the possible structural divisions, or constituents, of any language; Fig. 2 is a block diagram of a speech recognition system in accordance with the invention; Fig. 3 is a block diagram of a computer system suitable for practicing the present invention; Fig. 4 illustrates a typical plot of the short-time energy level (for the Japanese word "yozakura"); Fig. 5 is a typical zero crossing rate plot (for the English word "six") ; Fig. 6 is a block diagram illustrating the univalent element and structure detector of Fig. 1 in more detail; Fig. 7 is a block diagram illustrating the feature extractor and feature analyzer of Fig. 6 in more detail; Fig. 8 is a flow chart describing the operation of a phonetic classifier and segmenter 108 and its interaction with element detectors 104 and 106; Fig. 9 is a flow chart describing how a voiced region iε subclassified; Fig. 10 is a flow chart describing how initial silence is detected; Fig. 11 is a flow chart describing how nuclei and sonorants are found; Fig. 12 is a flow chart describing how a nucleus subregion is found; Fig. 13 is a flow chart describing how an unstressed region is processed; Fig. 14 is a flow chart describing how a sonorant subregion is found; Fig. 15 is a flow chart describing how a nucleus type is assigned; Fig. 16 is a flow chart describing how a nucleus subregion is processed to find phonological elements A, U and I; Fig. 17 is a flow chart describing how a branching type and an element code are processed for a nuclear region; Fig. 18 is a flow chart describing steps 310 and 312 of Fig. 9 in more detail; Fig. 19 is a flow chart describing how a portion of a voiced region is processed as a nucleus region; Fig. 20 is flow chart describing a portion of step 308 of Fig. 9 in more detail; Fig. 21 is a flow chart describing how an unvoiced region is processed; Fig. 22 is a flow chart describing how a fricative subregion is found; Fig. 23 is a flow chart describing how a fricative subregion is processed; Fig. 24 is a flow chart describing how a short fricative subregion is processed; Fig. 25 iε a flow chart describing how a silent region is processed; Fig. 26 is a flow chart describing how a noninitial unvoiced region is proceεsed; Fig. 27 is a flow chart describing how processing of an unvoiced region is completed; Fig. 28 is a flow chart deεcribing how the output file from the phonetic claεεifier and εegmenter iε parsed into a symbolic representation; Figs. 29A-B are flow charts deεcribing how onεetε are built; Figε. 30A-B are flow chartε deεcribing how rhymeε are built; Fig. 31 illustrates a sample lexicon for the numbers 1 through 10 in Japanese; Fig. 32 iε a flow chart deεcribing how a εymbolic representation iε matched to entries in a suitable lexicon; Fig. 33 is a block diagram of the lexical entry system; Fig. 34 iε a flow chart describing how the lexical entry system operates; Fig. 35 is a graph illustrating the contents of a sample feature file over a period of time. Detailed Description The present invention will be more completely understood through the following detailed description which should be read in conjunction with the attached drawing in which similar reference numbers indicate similar structures . Throughout this document some terms of phenology are used. Where not defined herein, the definitions tf these terms are intended to be the same as found in Introduction to Theoretical Linguistics, by J. Lyons, Cambridge University Press, 1969. This and all other references referred to herein are hereby expressly incorporated by reference. One problem to be solved by any speech recognition εystem, if it is to be speaker independent, is eliminating effects of variation between different speakerε on acoustic properties of a spoken word, in both time and frequency. Such variation is due to, among other things, regional accent, sex, age, pitch of voice, emotional state, and speaking rate. Another problem is distinguiεhing εpeech from background noiεe. Speaker variation and noise are nonlinguistic phenomena which humans, although senεitive to their presence, can adequately ignore or compensate for so that communication with previously unknown speakerε presents little difficulty. A human can compensate for variation and noise and perceive the intended word correctly most of the time. Given that recognition errors made by humans are leεε frequent when words are εpoken in the context of a sentence, it εee s reasonable to assume that corrections are baεed on higher-level, non-acouεtic information. A theory of phonology called government phonology is likely to be a suitable model of how a human processes speech using higher-level information and has been used as a basis for this invention. Accordingly, the invention will be better understood given the following brief description of major concepts in this theory. A better understanding of thiε theory may also be obtained from publications discussing this theory, of which the following are exemplary: "The Internal Structure of Phonological Elementε: A Theory of Charm and Government", by Jonathan Kaye, Jean Lδwenεtamm and J.R. Vergnaud in Phonology Yearbook 2, 1985, pp. 305-328. (Kaye et al . , 1985). "Conεtituent Structure and Government in Phonology," Phonology, by Jonathan Kaye et al . , Vol. 7, 1990, pp. 193-231. (Kaye et al . , 1990). "Coda Licensing," by Jonathan Kaye in Phonology, Vol 7, 1990, pp. 301-330. (Kaye 1990). "A Computational Learning Model for Metrical Phonology" by B. Elan Dresher and Jonathan Kaye, Cognition Vol. 34, 1990, pp. 137-195. (Dresher et al. , 1990). "Derivations and Interfaces," by Jonathan Kaye, in SPAS Working Papers in Linquiεticε and Phoneticε, Vol. 3, 1993, pageε 90-126. (Kaye, 1993). A. Theoretical Phonological Baεiε and Definitions In general, there are acoustic cues in any speech signal which are indicative of significant linguistic and especially phonological information. These cueε are found in εpeech in all human languages. In government phonology, these cues are related to the univalent atomic units of segmental repreεentation, known aε phonological elements, in εuch a way that one or more cues relate uniquely to a εingle element . The advantage of this relationship between acoustic cues and phonological elements is that it allowε a relatively direct mapping between linguistic units and their manifestation in the εpeech signal. This theory suggests that there are a small number, e.g., less than 15 of these univalent phonological elementε used in all spoken human natural languages. "Univalent" signifieε that the phonological element iε either present or not in a speech signal. This iε in - 1? contrast to some other phonological theories, such aε thoεe involving phonemes, which suggest that certain features (or articulatory variables) are always present and take one of two states. In other words, univalent phonological elements are different from binary articulatory variables in that there is no definition of the absence of a phonological element. Accordingly, a phonological element is a univalent universal phonological property, the presence of which can be detected using one or more distinct cueε in a context-free manner. That is, a cue or group of cueε indicateε the presence of a phonological element independently of the context in which it is found. Also, these phonological properties are not language specific. Examples of phonological elements and associated acoustic signatures which have been suggested by government phonology so far are listed in Table 1. This asεociation of acoustic signatureε with phonological elementε iε only an example of those currently suggested by government phonology. It is possible that other groupings may exist. Element Acoustic εignature Abrupt drop in energy (in range 0.5kHz-2.0kHz) for a minimum duration of 20ms Aperiodic energy characterized by, for example, a high zero-crossing rate 'Mass': high value of 1st formant frequency (FI), convergent with 2nd formant frequency (F2); broad distribution in energy between 100Hz and 2kHz 'Dip': convergence of F2 and 3rd formant frequency (F3) at about 2.5kHz U 'Rurp' : low F2, convergent with FI; Diffuse-falling burst spectrum; narrow band at low frequency with a rapid fall in the frequency domain v Neutral spectrum; compact R Diffuse, rising burεt spectrum N Similar to L H Raiεed fundamental frequency (f 0 ); additionally, in non- nuclear positions, FI cutback in transition to following vowel L Lowered f 0 ; additionally, in non-nuclear poεitionε, rapid riεe in frequency of FI at onεet of following vowel TABLE 1 Due to similarities, some cf these phonological elements may possibly be combined theoretically (conflated) aε one, such as h and H or N and L. That is, they may actually be orie phonological element manifested in two different ways . Since some of these phonological elements are present both in vowels and consonants, they cannot simply be discuεεed under theεe two categories. For example, although elements U and I are commonly found in vowels, they are also often present in conεonants and are related to what is commonly called the place of articulation of the consonant. Phonological elements ? and h are commonly found in consonants and are related to what is commonly called the manner of articulation of the consonant. It should be understood that, in government phonology, there is no true definition of a vowel or consonant. - 15 - Each univalent phonelogical element has a unique identifiable acoustic signature as well as an articulatory representation. Since a phonological element is also a cognitive object aεsociated to some sound pattern rather than a set of articulatory variables associated to some vocal tract configuration, a closer relationship is expected to exist between a phonological element and its acoustic properties than between a phonological element and its articulatory properties. This relationship reflects the primacy of sound patterns over vocal-tract configurations in speech processing. The use of articulatory properties in connection with the invention to describe phonological elements is really only a convenience to assist those not familiar with the theory in government phonology. Table 2 lists the phonological elements presently suggested by the theory of government phonology along with their associated sounds and salient articulatory features. Element Sound Salient Feature A roughly "a" aε in father: non-high I roughly "i" as in hit front/palatal U roughly "oo" aε in book labial/round N roughly "n" as in button; nasal R roughly "tt" in American coronal English pretty; h "h" aε in horse; noise - glottal catch as in "uh-oh" occluded or Cockney "pretty" H - high tone as in tone stiff vocal cords languages or "tenεe/fortis" stops, cf. Korean p' L - low tone as in tone slack vocal cords languages or truly "voiced" stops as in French b (not English b) v - cold vowel none TABLE 2 A phonological element is pronounceable in isolation and can appear in a speech signal by itself. Also, two or more phonological elerents may be combined to form a phonological expreεεicn. A phonological expression is defined as the combination of one or more phonological elements, called "operatorε," with at most one other phonological •-•lement, called a "head" element. The combination of phonological elementε into an expreεεion can be represented by the following equation: R,?,h.U (1) The head appears to the right of the " . " The phonological elements combined in an expreεεion are all manifeεted; however thoεe which are operatorε appear to be dependent upon, or dominated by, the head element in the εpeech signal. Generally speaking, the head element is more predominant in the speech signal. As a universal principle, i.e., for all languages, all tense vowels are generally full-headed, while lax vowels are generally empty-headed. Thus, the vowel i aε in "heat" may be repreεented by the phonological expreεεion (_.I), uεing the convention of placing the head element on the right of tne operator. The lax vowel in "hit" may be then (I._), i.e., an empty-headed phonological expression containing the phonological element I as an operator. There are also language-specific conditions, or licensing constraints, which limit the combinations of phonological elementε in phonological expreεεionε. In Engliεh for instance the following licensing constraints apply: non-branching nuclei (a concept defined below) are headlesε, and the U and I elementε cannot combine. Phonological expressions or elements constituting any word are grouped into structural divisionε of the word, called constituents, of which there are three types: the onset (0), the nucleus (N) and the rhyme (R), where a rhyme necessarily contains a nucleus. A well-formed phonological representation of a word thus consists of a sequence of O-R pairs, so that the right-most position within a given domain (i.e., the last position to occur in time) is always a nucleuε. ("Left" refers to the first to occur in time and "right" refers to the last to occur in time.) Each constituent (onset, rhyme or nucleus) may contain up to two skeletal positions, i.e., may branch. All possible structures of constituentε are repreεented by the illustration of Fig. 1, where 0=onset, N=nucleus, and R=rhyme. These structures are, in order from left to right in Fig. 1, the non-branching onset, the branching onset, the non-branching nucleuε (in a non-branching rhyme), the branching nucleus, and the branching rhyme. Each skeletal position iε aεεociated with a phonological expreεsion, thus defining a phonological segment. A skeletal position in an onset is referred to as an onset position; in a nucleus it is referred to as a nuclear position. The right branch of the branching rhyme is also called the "Coda position." One consequence of the foregoing conεtraintε is that a branching nucleus followed by a coda εegment iε disallowed; a phonological expression following a branching nucleus must be aεεociated with the following onser-rhyme pair, normally with the following onset A universal principle known as "Coda Licensing" prevents a phonological expresεion from being aεεociated with a coda poεition unless it is followed by an onset position which is associated to a phonological expression with appropriate properties. (This allows effectε of the "Maximal Onset Principle" to be derived in a principled way rather than by stipulation, as in many other phonological theories.) Becauεe conεtituents may branch, the constituents in languages may vary only with respect to three binary parameters. In other words, setting three parameters suffices to characterize the possible constituents that may be found in any language. These parameters are shown in Table 3. Parameter Posεible Valueε 1. Branching Onset yes/no 2. Branching Nucleus yes/no 3. Branching Rhyme yeε/no TABLE 3 The parameterε for a few languages are shown in Table 4 (the column numbers refer to the row numbers of the parameters above in Table 3). These parameterε have been determined empirically by phonological analyεiε. A phonological representation of a word in government phonology thus includes indications of (1) the constituentε (0, N and R) (2) skeletal positions (the x's in Fig. 1) and (3) phonological expresεionε aεεociated with the skeletal poεitionε. Therefore, the lexical repreεentation of a word from a phonological point of view (i.e., as a human mind may "store" a representation of a word) is its constituent structure plus a list of the phonological elementε which are associated with each skeletal position. There may be a one-many rr many-one relation between any of these levels. The structures above, according to government phonology, exhaust tne possibilities for all known human languages of the world. Government phonology allows for onset and nuclear poεitionε to be lexically empty, in which case they contain only the cold vowel as a null element. An onset position may or may not dominate a skeletal position, but a nuclear position always dominates a skeletal position. These empty positions, in particular empty nuclear positions, are subject to strict licensing restrictions defined by what is known as the Empty Category Principle. (Licensing deals with whether a vowel is pronounced or silent. See Kaye et al . 1990). Domain-final empty nuclear positions represent a special case; they can be licensed para etrically by the language. English and German, for example, license domain-final empty nuclear positions, while Japanese does net. A licensed empty nuclear position iε not realized phonetically; if it lε unlicenεed, it iε pronounced in most languages as the cold vowel, i.e., a high back unrounded vowel. This expla.ns why English and German have word-final conεonants; the "f.nal" consonant is followed by a licenεed empty nucleus, wh.ch iε, therefore, not audible. The phonological mechanism by which word-internal empty nuclear positions are licensed will not concern us here. Permitted sequences of branching conεtituents are determined by the properties of the phonological segments contained within these constituentε. The relationεhipε into which adjacent skeletal positions enter are called governing relations. Certain phonological segmentε have governing propertieε and may be associated to governing skeletal poεitionε; they are known aε governors. Other phonological segments are governable and aεsociate to governed positions; they are referred to as governees . From these governing relations one can derive the phonotactic constraints of natural languages, i.e., which define the well-formed sequences of phonological segments. Therefore, these conεtraintε are derived in a principled way, in contrast to some other theories in which these constraintε are arbitrary language-specific statements. Phonological representations are furthermore subject to the Phonological Projection Principle, similar to the projection principle of government and binding theory (a dominant theory of syntax that deals with the re tionships of adjacent objects) as described in Noam Chomsky s 1981 Lectures on Government and Binding (Dordrecht: Foris) . The projection principle aε applied to phonology εtates that the oonstituent structure of a phonological εtring is not affected by phonological proceεses. Relationships between skeletal poεitionε are fixed lexically and cannot be changed. In practice, thiε meanε that constituent structure and governing relationε are constant throughout ε phonological derivation. That iε, there is nothing corresponding to the notions of deletion, epenthesis or reεyllabification commonly found in other phonolirical theories (e.g., lexical phonology). Rather, during a phonological derivation, while some portions of tne phonological segments in a word may be lost, or a phonological segment may gain additional material through phonological events such as spreading, the constituent structure of the word as listed in the lexicon is preserved. A more detailed discussion of derivation as uεed in government phonology is found in Kaye, 1993. One consequence of the projection principle and derivations is that empty skeletal positions may occur in surface forms, subject to the above licensing restrictions. The constancy of the constituent structure of words has important consequences in speech recognition. First, words are always comprised of one or more structural divisionε, each conεiεting of an onset followed by a rhyme, whatever the apparent clusters of phonological segments in the speech signal seem to indicate. Thus, there is no need to store multiple versionε of wordε which can be realized differently, for example, due to vowel-zero alternations cr 'epenthetic' consonants. If an impossible sequence is detected, then a phonological parser may, for example, insert an empty nucleus at the appropriate place, so the correct constituent structure is used in lexical access, Second, related to the concept of derivation in other phonological theories iε the general asεumption that the phonological representation of a word is part of the lexical entry for that word in the human psychological lexicon. In addition, in government phonology a phonological representation of a word is alεo the addreεε or location of the lexical entry for the word. Thuε the huiran lexicon iε a form of content addressable or associative memory that responds to a phonological representation identified in a surface form. The psychological lexicon alεt has a collating sequence based on the phonological representations of its forms. Forms that are phonologicaliy εi ilar are lexically proximate (i.e. found near each other in the lexicon). Phonological similarity is based on, hierarchically, (1) the consonants, (2) the constituent structure, and (3) the vowels. Vowels provide very little information, whereas the consonants form a code. From a speech recognition point of view, consonantal identification and constituent structure are therefore considerably more important to humans than vowel identification. B. Implementation■ Recognizing a word in a εpeech signal involves addressing at least three problems (which are neither necessarily distinct nor performed separately): 1) detecting significant information from the speech signal, 2) generating a representation of the word from the εignificant information detected in the εpeech signal and 3) searching a lexicon to find the word which best matches the generated repreεentation. A εystem for addresεing theεe proble ε uεing the theory of government phonology will now be diεcussed in connection with Fig. 2. Not all aspects of this theory have been implemented in this recognition system. For example, the recognition of tonal patterns, common in tone languages such as Chinese and Thai, haε not been implemented. Not all of the phonological elements discussed above are expressly detected. Also, only limited principles and licensing constraintε are implemented. Nonetheless, suitable recognition over a variety of speakerε can be obtained with reasonable accuracy given a limited vocabulary, such as the numbers 1 through 10 in Japanese, English or Korean. Fig. 2 is a block diagram illustrating the inter¬ connection of major components of a speei.i recognition εyεtem. This εpeech recognition εyεtem -. . receiveε a speech signal 42, an indication of words in a anguage 44, and outputs an indication of a recognized word, as indicated at 46. An optional word boundary detector -i3 procesεeε the speech signal to isolate words from silence in the speech signal. The output of the word boundary detector iε the speech signal and a pair of markers indicating preliminary boundaries of the words, as indicated at 50. A univalent element and structure detector 52 then analyzes each of the individual words within the speech signal. The output of this detector 52 is a symbolic representation 54 of the structure and phonological elements within each word in the speech signal which a lexical matching system 55 useε to identify the word. A lexicon preparation εyεtem 56 receiveε indications of words which are input to the system and ensures that the input complies with the theory of government phonology, to provide a lexicon 58 in which each wcrd is represented and accessible according to its constituent structure and phonological elements. Each word in a given language in lexicon 58 εhould have, according to government phonology, a εingle representation which is independent of the representations of all other words in the lexicon 58. That is, the addition of one word to the lexicon 58 does not change the representation of other words in the lexicon. The lexical matching system 55 uses the εymbolic representation 54 to accesε the lexicon 58, and outputε the beεt match of the repreεentation 54 in the lexicon 58 aε the recognized word 46. Each of the word boundary detector 48, univalent element and structure detector 52, lexicon preparation system 56, lexicon 58 and lexical matching system 55, may be implemented using a computer program and a general-purpose, programmable, digital electronic coirtiter. The lexicon preparation εyεtem and lexicon 58 ma: be on a εeparate computer, perhaps in a remote locatiin, so aε to enable generation of a computer-readable databaεe of lexical entries accessible by many speech recognition systems. They may also be implemented using cuεto , semi-custom, or other specialized electronic circuits, including integrated circuits. Each of these components will be described in more detail below. A suitable computer generally has a structure illustrated as the computer 60 in Fig. 3. The computer 60 includes at least one processor 62 which executes computer programs represented by logic signalε stored in memory 64. The procesεor 62 is generally a commercially-available microprocessor and its supporting control circuitry and the memory 64 is typically a volatile random-access memory, which may be either static or dynamic and iε typically one or more integrated circuits. Speech signals are obtained through one of the input/output devices 66, such aε a microphone or telephone line connected to the computer through an analog-to-digital converter. A suitable sampling rate iε 12kHz, which provideε a bandwidth of 0Hz to 6kHz for analysis. A digital source of audio information may also be used. Input/output devices 66 also generally include (1) input devices such as a keyboard, mouse, trackball, touchscreen, touchpad, light pen, etc., for receiving input from a user, (2) non-volatile memory devices, including diεk drives, tape or other recordable or readable media for storing data and computer programs, (3) output devices, such as video graphicε diεplays and printers for providing information to a user, and (4) communication devices, including modems, network interfaces and networks to interconnect computer 60 to other computers. This enumeration of input/output devices iε merely exemplary and is not intended to be limiting. These elements 62, 64 and 66 are interconnected by a bus 68. The general operation and interaction of these elements in responεe to a given computer program iε well-known. Although equipment for recording, digitization and acouεtic analyεiε of εpeech iε generally available, and many commercially available εyεtemε may be used, the Loughborough Sound Images Speech Workstation 1.1 was used in one embodiment by the inventors. The inventors have also used a SPARCStation IPX workstation computer, available from SUN Microsyεtems of California, using the SunOS, version 4.1.3, operating system and the XWindows application environment. The ESPS εignal processing system (ESPS library), Version - I - 4.1, and the xwaves- computer program, verεion 2.1, both available from Entropic Research, Inc., of Washington, D.C, were used to provide signal processing capabilities, in combination with other computer programs written in ANSI-C (for phonological element, constituent and word boundary detection) and Icon (for lexical matching) . Theεe computer programs were compiled into machine-readable and executable instructions. The Appendix contains a source code listing for moεt of the portions of this system. Those which are omitted are nonetheless completely described below. An IBM-PC-compatible computer may also be used, as well as other programmable digital computers. The modules of this εyεtem shown in Fig. 2 will now be described in more detail. The word boundary detector 48 preferably detects word boundaries in a εpeech signal using a process performed by a function called "find-ep" in the ESPS library, which is a modified version of the process described in "An Algorithm for Determining the End-Points of Isolated Utterances," Bell System Technical Journal, Vol. 54, pp. 297-315 by L.R. Rabiner and :-'..R. Sambur. Thiε proceεε iε used becauεe the raw speech signal is not always reliably indicative of the presence cr absence of speech because, in εome caεeε, speech sounds are hardly distinguishable from background noise. To avoid problems with these caseε, the process of Rabiner and Sambur useε two time-domain representations of the εigna_, namely the short-time energy level and the zero-crossing rate. A plot of a typical short-time energy level is shown in Fig. 4 of the Japanese word "yozakura," which means "evening cherry". The ordinate indicateε the root-mean-square (RMS) amplitude whereas the abscissa indicates time. This plot was determined at intervals of lOmε using a rectangular window of length 15ms, into a signal sampled at 10kHz. From this plot the location of the utterance can be determined fairly preciεely by inεpection; it conεtituteε that part of the curve for which the energy is greater than zero (indicated by the vertical markers). Each peak corresponds to a nucleus. Words containing one of the known problem caεeε, however, have an energy level very close to the background silence level, so that if short-time energy alone were used as an end-point detector, errors would result in these cases. The zero-crossing rate (ZCR) is a reliable indicator of the presence of signal activity which does not rely on the energy measurement. The ZCR is defined aε the number of timeε the amplitude of the speech εignal croεεeε the zero level (or some other arbitrary level) in a given time interval, e.g., 25ms. A typical plot generated by determining the ZCR at lOmε intervalε, iε shown in Fig. 5, for the word "six". In thiε figure the ordinate indicateε the ZCR for a 25mε internal and the abscissa indicates time. Noise r rεtε and fricatives appear in the signal as high values ^f zero-crossing rate, while vowels and sonorants typica ' ' γ cause a low, steady value. The utterance in this example appears between the vertical markers 63. To identify word brundaries, upper and lower threshold values for the εhort-tir÷ energy level and the zero-croεεing rate are set. The threshold values may be determined from measurements of the bad-:rround noise or fixed experimentally. Fixed tnreεhold valueε are juεtified where working conditions are fairly stable (same microphone, same environment), but an adaptive mechanism should be used to cope with variable input, such as from a telephone line. Where the ZCR or short-time energy level is above the corresponding threshold, an utterance is aεεumed to exiεt. The output 50 of word boundary detector 48 iε a pair of markers which indicate the beginning and end of an utterance in the speech signal. The output 50 may also include the εpeech signal, but the univalent element and structure detector 52 typically receives the εpeech εignal 42 directly rather than from the word boundary detector. It should be understood that the word boundary detector 48 does not actually discriminate between speech and non-speech, but merely between the presence versuε absence of high sound intensity. It is then asεumed that the region of high sound intensity iε an utterance. Discrimination between speech and non-speech is a much more difficult problem, aε yet unεolved aε far aε we know. The detection of phonological elements aε deεcribed herein may be applicable to this problem. Word boundary detection may also be performed by calculating a spectral difference, for example, by using the spectral difference calculator 84 which is discuεεed below in connection with Fig. 5. Becauεe the human cognitive system responds to changing, rather than static, input and becauεe the frequency εpectrum of speech signals typically changes much more rapidly than that of random background noise, a measure of the spectral rate of change should be useful to detect speech. In certain cases, e.g., where an "f" is the final consonant, a spectral difference function would indicate the change rom conεonant to εilence more reliably than the ZCR function. Thuε, the spectral difference can provide a more precise location of word boundaries, eεpecially after using the Rabiner and Sambur algorithm to provide an initial rough estimate. If the spectral difference function is accurate enough to detect word boundaries, use of the Rabiner and Sambur algorithm becomes redundant. Word boundary detection is also done for purely practical reasons. In the artificial task of isolated word recognition, a speech εignal is assumed to contain an utterance somewhere. The whole εpeech signal is normally mostly silence; the word or words occurring in the signal may be only 20% or 30% of the speech signal. Because it would involve unnecessary computation to do a complete εpectral analysis of long regions of silence, the word is first isolated, using quick and computationally cheap signal procesεing. Human speech recognition probably doeε not work thiε way. This technique of word boundary detection probably does not extend to continuous εpeech where there are typically no word boundarieε in the εignal; however, it can be uεed to isolate an utterance from background silence. In a continuous speech recognition, word boundaries would have to be inferred. Detection of constituent boundaries, aε discusεed below, may be uεeful in εolving that problem aε well . The univalent element and εtructure detector 52, illustrated in more detail in Fig. 6, contains a spectral analyzer 80 which generates a spectral data file 82 from a DC-removed εpeech εignal. A spectral slice is provided at 5mε intervalε using a 20mε window, having a Hamming εhape, into the εpeech signal. A Kaiser shape window may alεo be used. Preferably, the ESPS library command "me_sgram -w20 -E0. 4-ol4 -S5 -iHAMMING" is used, aε described above, to generate pre-e tnaεized spectra using a maximum entropy method. Linear predictive coding (LPC) and faεt Fourier transforms (FFT may also be used. The 0.94 value lies on a scale in which C repreεentε no pre-emphaεiε, whereas 1.0 means a boost of 6dB per octave. A feature extractor 70 proceεεeε the speech εignal and the εpectral data file 82 between the markerε identified by the word boundary detector 48 to obtain a feature file 72. The feature file contains a description of the speech signal in terms of its signal characteristicε such as the fundamental frequency (f ) and the formant frequencies (FI, F2 and F3), the total power, mid-power, zero-croεεing rate, the probable voicing, εpectral difference computed from LPC spectra, spectral difference computed from formantε, and a ratio (kl) of the firεt two croεε-correlation coefficientε. These features are provided in the feature file 72 at 10ms intervals, i.e., at a 10ms frame rate. The spectral slices and formant values are produced every 5ms both for higher sensitivity and also because a pair of adjacent valueε iε needed to provide a εingle value for a spectral and formant difference. When formant values are stored in the feature file, every alternate value is stored, thus providing them at lOmε intervalε. Phonological element detectors, vnich are discussed below, may use these alternate values but do not appear to require any more precision than 10ms intervals. The feature file analyzer 74 analyzes the feature file 72 and the spectral data file 82 to produce the symbolic representation 54. The feature extractor 70 and feature file analyzer 72 are illustrated in more detail in Fig. 7. The speech signal and markerε 50 are fed to a formant and voicing processor 78; a DC-removec speech signal is applied to a power, mid-power and ZCR calculator 96. The removal of the DC-component of the εpeech signal 50 is obtained, for example, by a DC filter 81, which may be implemented using the ESPS command "rem_dc". A spertral difference calculator 84 receiveε the εpectral data file 82 aε an input and computes the difference between adjacent spectral εliceε. This spectral difference iε output as indicated by "εpec-diff" at 86. This spectral difference calculator 84 plots the Euclidean distance between adjacent (i.e., 5ms apart) pairs of pre-emphasized spectra from spectral data file 82. The pre-emphasiε function haε high values in transitions between nuclear and non-nuclear poεitionε, and low valueε in stable regions, i.e., nuclei. It reflects the belief that attention is drawn to regions of rapid spectral change, i.e., consonants, and that theεe regions need more precise analysis than the rest of the signal. In human εpeech processing, the ear is more sensitive to frequency change at lower frequencies, i.e., it has a greater frequency resolution at the lower end of the spectrum, decreasing roughly logarithmically aε the frequency increases. Accordingly, a log-weighted spectrum is appropriate for the εpectral difference calculation. If a log-weighted spectrum is not uεed, the εpectral difference calculator may behave erratically during fricative or silence regions, rather than providing peaks when there is a change from one type of region to another. Euclidean distance or squared Euclidean distance may be used aε the diεtance metric; however, they are prone to inεignificant phonetic detail, εuch aε sudden changes in vowel amplitude. A log-weighted diεtance metric may be implemented on a PC and may be better in thiε respect than the Sun/ΞSPS version, since in ESPS there iε no simple way to generate a log-weighted spectrum. A Mahalanobis metric may alεo be uεed. Ideally, the meaεure of εpectral difference should be inεenεitive to inεignificant phonetic detail, but senεitive to releva:. t linguiεtic changeε. In prior template-baεed automatic εpeech recognition syεtemε recognition iε baεed on the εcores obtained from matchir.r a word to its εtored spectral evolution with time. These systems have a similar problem of defining an appropriate εpectral difference function. Generally, the uεe of a proper auditory model appearε to provide the best resultε. Such models are described in "A Joint Synchrony/Mean-Rate Model of Auditory Speech Procesεing," by S. Seneff, Journal of Phonetics, Vol. 16, No. 1: pp. 55-76, 1988, and are used in the "LUTEAR Model" available as a software library from Loughborough University of Technology, in Loughborough, United Kingdom. Whole papers in the field of automatic speech recognition are devoted to discusεing thiε problem of εelecting a good meaεure of εpectral difference. See, for example, Chapter 14 of Invariance and Variability in Speech Proceεεeε, edited by Perkell and Klatt, (Lawrence Erlbaum Aεsoc: Hillsdale, N.J. 1986). This work may be useful in identifying other suitable spectral distance metrics for use with this invention. The formant and voicing processor 78 computes (1) the voicing probability ( "prob voice"), which is a value within the range of zero to one as indicated at 94; (2) the fundamental frequency ("FO"), as indicated at 88; (3) the ratio ("kl") of the first two cross-correlation coefficients of the εpeech signal as indicated at 90; and (4) the firεt, second and third formant frequencies (FI, F2 and F3) as indicated at 92. The formant and voicing processor 78 may be implemented using the ESPS function "formant". Pre-emphasis is preferably uεed by using the ESPS command "formant -p.95 -i.005 -f12000 -W.025 -Wl -ol4". The formantε should be calculated using a 25mε Hamming window at 5ms intervals. The formant frequencies 92 are fed to a formant factor calculator 100 which outputs a formant-derived εpectral difference meaεure indicated aε "factor" 102. The calculator 100 determines a spectral difference based on for ants of ad acent 5ms time periods, using the following formula: sqrt( I ((F t n - F t+1 n) / F t+1 n * 100) 2 ), n=ι where F n and F n are the formant frequencieε of succesεive frames. The power, mid-power and ZCR calculator 96 determines the power (or short-time energy level over the whole frequency range), mid-power and zero crossing rate of the DC-removed speech εignal uεing the ESPS function "acf" at lOmε intervals using a 15mε Hamming window and no pre-emphasis. The mid-power indicates the power in the frequency range of 1-2.7kHz. "Power" εignifieε the short-time energy, or total power, of the εignal. ZCR was defined above. A sample feature file for the word "eight", spoken by a female English speaker, is shown in Fig. 35. Each of the features discuεsed above is illustrated In Fig. 35 on a normalized εcale, where the abεciεεa represents time and the ordinate representε the normalized value. The features illustrated include f , the probable voicing, Kl, the mid-power, the firεt three formantε, the formant difference, the spectral difference, the zero crosεing rate, and the total power, or εhort-time energy function. The spectral data file 82 and the formantε 92 from the formant and voicing processor 78 are used by A, U, and I phonological element detectors 104. The total power, mid-power, zero crosεing rate, kl and "prob voice" featureε are used by the '?' and 'h' phonological element detector 106. The outputs of the phonological element detectorε 104 and 106 along with the "spec-diff" 86, "factor" 102 and "prob voice" 94 are input to a clasεifier and εegmenter 108 which determines and outputs the symbolic repreεentation 54 of the vord uεing a proceεε deεcribed below. The claεεifier and segmenter 108 controlε the uεe and operation of the detectorε 104 and 106 as shown by control lines Cl and C2. The clasεifier and εegmenter 108, itε interaction with detectorε 104 and 106, and the operation of theεe detectors will now be described in connection with Figε. 8-11. The proceεεing of the input εpeech εignal into voiced and voiceless regions, further subclassification and identification of a phonological elementε can be viewed aε an implementation of a machine defined by a grammar. Each word can be divided into a sequence of alternating voiced and voiceless regions. Each voiced region is a εequence of alternating regionε with either of a εonorant or a glide following or preceding a nucleus. Similarly, a voiceless region is comprised of alternating sequences of fricative and silence subregions. Prevoicing and the spectrum of a release burst following a silence region can alεo be analyzed to identify phonological elementε. By defining the speech- recognition process as a grammar, higher level linguistic knowledge can be used to constrain the recognition proceεε, thereby simplifying it and enabling the elimination of effects of speaker variation and noise. Also, it simplifieε implementation of processing of the speech signal. Thus, a number of different implementations of this recognition process are possible. The following is merely exemplary. More particularly in this embodiment of the invention, classification of the voiced and voiceless regions is based on primitive context-free "grammars" which define the well-formed sequences of sub-regions. In the case of voiced regions, the grammar is simply: Region -> N Region -> (R)NR* (* = Kleene Star) N -> Nl N -> N2 R -> R1R2 That is, a voiced region consists of either (a) a single Nucleuε (N) , or (b) a sequence beginning optionally with a sonorant followed by alternating Nucleus and sonorant (R) sub-regionε, of which the nuclei may be branching (N2) or non-branching (Nl). Thiε is an over-simplification as indicated above, omitting the category Glide. Similarly for voiceless regions the appropriate grammar is: Region -> S(B) Region -> F Region -> (F)S(B)* That is, a voiceless region consiεtε of either (a) Silence (S) followed by an optional Burεt (B), or (b) a εingle Friction εub-region (F) , or (c) a sequence optionally beginning with a Friction sub-region followed by alternating sub-regionε of Silence (plus optional Burst) and Friction. An additional constraint is that a word may not consist of a single voicelesε region, i.e., it muεt contain an obligatory voiced region. Thiε generalization applieε to all languageε with virtually no exceptionε. These simple grammars or εub-region sequences are used to implement parsing routines for the two major types of regionε. Fig. 8 iε a flowchart deεcribing the operation of the phonetic claεsifier and segmenter 108. The clasεifier and εegmenter firεt obtainε the peakε of the εpectral difference information 86 in εtep 120. The peak poεitions are output at 122. These peak positionε are uεed to εubclaεsify voiced regions in the speech signal into further subregions which are presumed to be indicative of skeletal poεitionε. Next, in εtep 123, all of the voiced and unvoiced regions are identified, using the "prob voice" value of each frame. The classification of frames int.. such regions iε performed by identifying a number of successive frames which are either voiced or voicelesε. Thiε identification may be performed by analyzing the probable voicing feature of each frame, for example, by comparing it to a threεhold. For example, if the "prob-voice" value iε leεε than 0.20, a voiceleεε sound iε presumed to exist. A sound is voiced if this value iε greater than 0.80. In εome inεtances, the "prob-voice" value of a given frame may indicate that the signal at that frame iε neither clearly voiced nor clearly voiceless. In such a case, the frame is grouped into the region having the nearest (in time) frame having a definitive value. A variable R, representing a region number being processed from left to right in the input signal, is then initialized to zero in step 124. The remaining steps performed by the clasεifier and segmenter 108, generally speaking, breakdown the εpeech εignal into a number of regionε wherein each region corresponds to a skeletal poεition and iε claεsified into a category. The categories into which the regions are clasεified are: silence (? aε head), sonorant (or resonant) (liquidε and naεalε with ? aε operator), friction (h preεent), and nucleuε (A, U and/or I preεent). Friction regions are further subcategorized as burεt or continuouε friction. One category which haε not been addreεεed iε what iε commonly called "glideε". Glides are also a form of voiced region. This kind of claεsification is only neceεεary in larger databaεeε where wordε are diεtinguished by detecting the presence of such a region. Detection of this kind of region appears not to be necessary in a limited task for example, recognizing the numbers 1 through 10 in either Japanese, Engliεh or Korean. In languageε that have no branching constituents, all non-nuclear regions are onsets. Each region is then analyzed to identify the phonological elements within it. For example, the nuclear regions are input to the A, U and I detectorε for analyεiε aε will be described below. Claεεification begins by analyzing the first region (region R=0) in the speech εignal. If a region iε voiceleεs, the region iε next broken down into εubregionε, wherein each subregion iε claεsified as being either silence or friction (step 128). The phonological elements in each of theεe subregions are also identified. This process of step 128 will be described in more detail below in connection with Figε. 18-24. For each εubregion identified, an output εtring iε updated in step 134 to indicate the kind of region and the phonological elements identified in the region. The voiceless region is continually analyzed from left to right until the entire region is claεεified into subregions. Thus, if frames remain in the region to be claεsified, as determined in step 136, stepε 128 and 134 are repeated. If a region iε voiced, aε determined in step 126, the region is broken down into subregionε wherein each εubregion iε claεsified as sonorant or a nucleus in εtep 140. The peak positions 122 assist in identifying subregionε. Phonological elements in each subregion are also identified. For example, if a subregion is a nucleuε, the U, A and I detectorε 104 are then uεed to identify phonological elements in the subregion. Whether the nucleus constituent iε branching is also considered, in the case where the language allows branching nuclei, by reference to language parameters 156 (εee tables 3 and 4 above). This procesε of εtep 140 will be described in more detail below in connection with Figε. 9-17. For each subregion identified, the output stream iε then updated in εtep 146 to indicate the claεεification of each subregion as well as the phonological elements defined therein. If more frames remain to be analyzed in this region, as determined in step 148, stepε 140 and 146 are repeated. Upon completion of analyεiε of a voiced or voiceleεε region, the variable R iε incremented in step 150. If more regionε remain to be analyzed aε determined in step 152, i.e., if more of the speech signal remains to be clasεified, processing continues by returning to step 126. Otherwiεe, the resulting output εtream, repreεentε, with a εtring of tokens, the εequence of phonological εeg entε identified in the speech εignal. This output εtring iε proceεsed by a word level constituent parser 154 which will be described in more detail below. The word level parεer 154 generally takeε the output εtream of the claεεifier 108 and convertε it into the symbolic representation 154 for use with lexical matching in accordance with language parameters 156. The subclassification of voiced regions (step 140) will now be described in more detail in connection with Figs. 9-17, Figε. 9-20 deεcribe the process performed by the computer during execution of the "parse_voiced" procedure found in the Appendix. In the caεe of inconsistencies between these and other Figures and the Appendix, the Appendix εhall be presumed to be correct. The first step of processing a voiced region involveε setup (step 300), including declaration and assignment of a number of variableε uεed in the process. These variables will be discussed in more detail below where they are used. For example, if an entire word is voiced, the local maxima of the power and mid-power are those of the entire region. Alεo, the region is presumed to include a streεsed vowel. Otherwise, the region is presumed to include a stressed vowel if the maximum power within this region is the same as the maximum power of the word uttered and whether the local maximum of the mid-power is greater than 95% of the maximum mid-power of the voiced regions. In this setup procesε, if an entire region is voiced, and if it includes more than 15 frames (has a duration of longer than 150ms), it is presumed that there are at least two nuclei within the word. Otherwise, it iε alεo preεumed that there iε only one nucleus in this region. Finally, if the mean fundamental frequency of the region is greater than 160, it is presumed that the speaker is female, otherwise, it is preεumed the εpeaker iε male. Whether this preεumption iε correct iε immaterial becauεe thiε claεsification is merely used to set other thresholdε. After setup procedureε, the computer then checks to identify any initial silence in the word (step 302). Generally, initial silence can be discarded. If the type of any previous region was unknown, or a burεt, or a low tone, and if the length of this region is greater than a minimum expected length, step 302 is performed. This procesε will be deεcribed in more detail below in connection with Fig. 10. Next, the previouε region-type iε examined in εtep 304. If the previouε type is either edge, silence, low tone, error or burst, εtep 306 of finding nuclei and sonorants iε then performed by the computer. Upon termination of thiε step 306, an output string iε constructed and processing returns to step 148 (Fig. 8). If the previouε region-type is either onset or fricative, the computer continues by performing the step 310 of finding sonorants and nuclei. The output string is then constructed in εtep 308. If the region-type iε unknown, the computer continueε by proceεsing the region as an initial unknown region before proceeding to the step of finding sonorantε and nuclei. Stepε 306, 310 and 312 will be deεcribed in more detail below in connection with Figε. 11 through 20. The proceεε of checking for an initial silence (step 302, Fig. 9) will now be described in connection with Fig. 10. The firεt εtep of thiε proceεs is finding a position of an edge in the region (εtep 320). If no edge is found, and if the previous region was a burεt as determined in εtep 322, a poεεible error iε indicated and may be correcter. in step 324. The possible error iε due to the fact that a glottal εtop εhould be found after a burεt. A glottal εtop iε typically confirmed by finding an edge. One poεεible way to correct for thiε error iε to take the indicator of the burst out of the output string (strip the 'Sh' sequence out of the output string). After correction of the error, if necesεary, step 302 is completed at step 326. The position of an edge in a region is found by locating a rising edge in the total power or mid-power curves within the region with a slope of greater than 40%. If an edge position is found and the previous region was a burst, aε determined in step 328, the existence of a glottal stop is confirmed as indicated in step 330. If the previous region was not a burst, and if the edge is close, i.e., less than four frames away, as determined in εtep 332, the type of the previouε region is then examined in εtep 334. If the type of the previous region is a low-tone, then it is presumed that the region prior to the burst should be silence and the token string is updated by the character "Z" as indicated in stepέ 336 and 338. Otherwiεe, i.e., if the previouε type is unknown, the string is updated to have the token "S" as indicated in εtep 340, and the region-type iε εet to be silence. If the position of the edge is not close to the beginning of the region, it is then assumed that there is a prevoiced region before the rising edge as checked for in step 342. Prevoicing is a short period of energy at or around only the fundamental frequency. If prevoicing iε found, the type of the region, or subregion, iε said to be a low-tone; the token string is updated to include "Z" and the εtart position from which analysis will take place within this region is set to be the edge position (step 344). Otherwise, the type of thiε subregion is set to be onset and the starting position for analysiε of the remainder of the region iε then set to the edge position (step 346). Step 306 of finding nuclei and sonorants following an edge, silence, low-tone, error or burεt εubregion will now be deεcribed. Initially, if the prior region was an edge, an "S" is inεerted to the token string. If the previous region was silence, a "." (the token delimiter character) iε inserted in the token string. It is presumed that a nucleus should follow. Accordingly, proceεεing begins by looking for a nucleuε. Thiε iε done in the following manner. In step 350, it is first determined if the reεt of thiε voiced region iε short and if the previous region was a sonorant. The rest of a region is short if its length is leεs than the minimum duration of a typical final nucleus in a region, which iε typically 60mε, or in this example, six fra eε. If the current region follows a sonorant and is short and nonfinal, a nucleus is fore*"*:! in εtep 352. A nucleuε iε forced by identifying the ■ .rεt frame εtarting from a start frame of the εubregion in which the total and mid-power is less than the total and mid-power of the start frame and in which the mid-power iε leεs than 20% of the maximum mid-power in the voiced region. If the number of frameε between the start frame and the identified frame is leεε than the number of frameε in the voiced region, a nucleus is thus forced. If the rest of the voiced region iε not short, or if the previous subregion is not a εonorant, an attempt iε then made to identify a nucleuε in the region aε indicated at εtep 354. Thiε step will be described in more detail below in connection with Figs. 12-13, and returns an indicator of the frame at which the nucleuε. if identified, ends. This procedure alεo identifieε a type of a nucleus. A region-type is then asεigned in step 356 according to thiε nucleuε-type. Thiε nucleuε-type indicates a nucleus end or and edge. If the nucleuε-type is an edge, the region-type is nucleus, otherwiεe the region-type iε εimply the value of the nucleus-type returned. After a nucleus iε identified or forced, and if the nucleuε region iε long enough aε determined at εtep 358, it iε firεt determined in εtep 360 whether any gap existed from any previous region and any gap is then adjusted. A nucleus region is long enough if it iε longer than the minimum typical nucleuε length which haε been found empirically to be about 50mε, or five frameε. A gap iε identified and adjuεted if the previous subregion was a sonorant and if the distance from the start of the nucleus region being examined and the last frame which haε been positively classified in the sonorant subregion is greater than some maximum gap size, which iε preferably about four frameε. The start of the - 6 1 nucleuε εubregion being classified is then adjusted to be the frame immediately following the last frame poεitively classified in the sonorant subregion. The branching-type of the nucleus subregion is then asεigned in step 362 in a process to be described below in connection with Fig. 15. Next, a code iε then obtained in step 364 which identifies which of the phonological elements A, U and I are present in the currently identified subregion. This step is described in more detail below in connection with Figε. 16 and 17. Based on this code, and the branching-type, the token string is updated in step 366. If the type of the nucleus is branching, the token string is updated to include "N2;", otherwise it iε updated to include "Nl;". The decimal code for the presence of the phonological elements A, U and I iε then added, followed by a ".". If the type of nucleus is a sequence, such as found in Japanese, the output string iε updated to include "O.N1;", followed by the decimal code for the preεence of phonological elements A, U and I, followed by a After the token string has been updated in step 366, it is then determined in εtep 368 whether the nucleus-type was an edge. In εuch a caεe, the previouε region-type is set to be "edge" in step 370 and thiε proceεs iε completed. The process is also completed if this voiced region haε been completely proceεsed aε determined in εtep 372. Typically, if the determined end of the nucleuε iε outεide of the voiced region or iε within two frameε of the end of the voiced region, the end of thiε region iε preεumed to have been reached. It is then determined if the end of a nucleus has certainly been reached in step 374 in order to force the detection of a sonorant in step 376. The end of a nucleuε haε certainly been reached when the nucleuε-type iε nucleuε-end, the laεt region-type iε a nucleuε, if the remainder of the voiced region is between a minimum threshold of about 40mε, or four frameε, and a maximum expected duration of a final sonorant, which is about 70ms, or 7 frames, and if the local maximum in both of the mid-power and total power curves is within the first half of the nucleuε subregion. A εoncrant is forced by returning the end of the voiced region as the end of a εonorant εubregion and by returning a εtring "εonorant." If the end of a nucleuε iε not certain, an attempt iε made to find a sonorant in εtep 378. This proceεε will be deεcribed in more detail below in connection with Figε. 12-13. This procesε returnε an indicator of the frame which iε the last frame of the sonorant region. The last region-type is then set in step 380. If the nucleus-type returned by the find sonorant proceεs iε nucleuε-end, the laεt region-type iε εet to be nucleuε, otherwiεe it iε set to be the returned nucleuε-type. If the nucleuε-type is onset, the name εtring iε εet to be "onset" rather than "sonorant." If the sonorant subregion is determined to be long enough, i.e., longer than 40mε or four frameε, aε determined in εtep 382, the presence of a gap between this sonorant region and any previous region is then detected and adjusted in step 384. After any necessary checking and adjustment for a gap, it is then determined whether the end of the voiced region haε been reached in step 386, similar to step 372. If the voiced region haε been completely proceεsed, this procedure is complete; otherwiεe the proceεε continues by checking whether the streεsed subregion has been processed in step 388, by determining (1) if the next subregion to be procesεed iε after the frame having the maximum power within the voiced region, and (2) if a nucleuε haε been assigned. If the streεsed subregion has been proceεsed, a value indicating stress has been set to false, otherwise it remains true and processing returns to εtep 350. Aε can be seen in Fig. 11. thiε process generally constrains sequences within a voiced region, generally, to be alternating between sonorant and nucleus subregions. With this procesε, it iε poεεible that a nucleuε may be empty. Thuε, at oεt two consecutive sonorantε are allowed and in thiε caεe, the preceding nucleuε must contain a stresεed vowel. If phonological segments corresponding to what are commonly called "glideε" are to be detected, one would expect an alternation between nucleuε and one of either sonorant or glide regions. The step of finding a nucleus (step 354) will now be described in more detail in connection with Figs. 12 and 13. It is first determined in step 390 whether a subregion is a εingle poεition. Generally, thiε step iε performed by examining the length of the subregion. If thiε length iε leεε than 11 frames, or 110ms, it is likely that there is only a single nuclear poεition. It is next determined whether thiε region iε εtreεsed in εtep 392. If thiε region iε not εtreεεed, the εubregion iε examined for a peak in tne total power curve in εtep 394. A peak is identified by comparing the mean power of the subregion to the expected power of a nucleus which iε greater than 500,000 in absolute terms, or by determining whether the mid-power in the subregion is high. In thiε embodiment, the mid-power is igh if at leaεt half of the values are higher than 40% of the maximum value. If no peak is found in the total power, this process returns the index of the starting frame in step 394; otherwise, the index of the end frame of the subregion iε returned and the indication of the previouε region iε set to be nucleus in step 398. If the region was determined to be εtreεεed in εtep 392, the exiεtence of a εtreεsed nucleus iε confirmed in εtep 400, for example, if the ratio of the mean total power to the mean mid-power iε greater than a given threshold and if the mean of the mid-power iε greater than a εecond threshold. Alternatively, this mean power ratio could be 50% of the first threshold and the mid-power could be generally high and also indicate a peak. By experiment, the first threshold haε been determined to be 60%. The second threshold haε been determined to be 200,000. Aε above, the mid-power iε high if at leaεt half of the valueε in thiε region are greater than 40% of the maximum of the region. If the exiεtence of a εtreεεed nucleuε iε not confirmed, the end of the nucleuε iε then identified in εtep 402. Thiε end is identified by finding the frame in which the slope of the total power and mid-power curves drop below predetermined threεholdε. For example, if the εlope of the total power curve in one frame falls below -25 and the εlope of the mid-power curve in that frame or a subεequent frame fallε below -35, the frame in which the mid-power curve fallε below -35 is selected as the end of the nucleuε. If this end frame iε εufficiently far from the end of the voiced region, e.g., four or more frames, as determined in step 404, the index of the start frame of the region is returned in step 406. Otherwise, if thiε end frame identifieε an edge (εtep 408), this edge position is then returned with the last region-type to be indicated as edge. Otherwise, the l st region-type iε indicated aε nucleuε-end and the end-print detected in εtep 402 iε then returned in εtep 412. An edge is detected if the total power of this end frame drops quickly from above the power typical for a nucleuε, e.g., 500,000. A glottal εtop may alεo be present at thiε position. If, in step 390, the subregion is likely to be more than one position, and if the region iε not streεεed aε determined in step 414, the "unstreεεed" region iε proceεsed in a manner to be deεcribed below in connection with Fig. 13, aε indicated at εtep 416. It is then determined whether the local maxima in the total power and mid-power curves are near the region's start in step 418. In particular, if the local maximum in either the mid-power curve or the total power curve iε within eight frameε of the εtart frame, or 80mε, thiε condition iε met. From the occurrence of this condition, the frame containing the last occurring power curve peak is then identified in step 420. Starting from this frame, the end of the power curve peaks iε then identified in εtep 422. This point is where the total power falls below the expected power per nucleus, or where the mid-power falls below a given factor of the expected mid-power in a voiced region. The factor is typically 20%. After the end of the power curve peak is found, the next peak in the εpectral difference curve is then found in εtep 424. If the current frame is at least beyond one-third of the subregion, a nucleuε-end iε then determined to have been found and an index to this frame is returned in step 428. Otherwise, if it is determined in step 430 that the distance from the εtart of thiε εubregion to thiε frame iε greater than the minimum length of a branching nucleuε, it iε determined that a nucleus has been found and the index to thiε frame iε returned to εtep 432. When this condition fails, the frame at the position of the lower povar ratio iε then εelected in εtep 434 and that poεition iε retimed and a nucleuε iε preεumed to exist ending at that position in step 436. When the power maxima detected in step 418 are not close to the start of the region, the power curveε are analyzed by frame, εequentially, in step 440 for a rising condition, to identify a frame in which riεing terminateε. Then, εtarting from the end of the region and working back to this identified frame, the power ratios are then examined in εtep 442 while the mean power ratio remainε above a given threεhold (half itε value at the identified frame)-. The mean power ratio iε the normalized mean from the εtart poεition to the currently εelected poεition near the end of the region. When a current frame haε been εelected through εtepε 440 and 442 it iε then determined whether the region it defineε is lcng enough in εtep 444. That iε, it εhould contain more than five frameε, or 50mε. If the region iε not long enough, the εtart poεition iε returned in εtep 446; otherwise a nucleus iε identified and the index to the current frame iε returned in εtep 448. The processing of the unstreεεed region will now be deεcribed in connection with Fig. 13. The firεt εtep in thiε proceεε is identification of the frame containing the next local maximum in the total and mid-power curve (step 450). If thiε position iε near, within 80mε of the start position, the frame after the local maximum iε εelected in εtep 452. If the length of the remainder after the εelected frame iε εhort, aε determined in εtep 454, the nucleuε iε preεumed to be found and the end of the region is returnee in step 456. Otherwise, the position after that frame in which there iε an 80% change in the power ratio iε then εelectec in step 458. If the power in this region is lesε than 20% rf the maximum power, the region iε labeled as nucleus-end in step 460, otherwise it is labeled aε a nucleuε. If the identified region is εhort, i.e., less than 60ms long, a: determined in εtep 462, then the εtart poεition iε returned in εtep 464; otherwise, the selected frame poεition is returned in step 466. The procesε of finding a εonorant will now be described in connection with Fig. 14. The first εtep of this procesε iε step 400 for determining whether the peaks of the mid-power and total power curves are far from the start of the region. This condition is true, for example, when the region is longer than lOOms and the poεition of the peakε are farther away than the typical length of a εonorant region, which is about 80ms. If the peak is far from the εtart of the region and if the mean value of the εpectral difference function in the region iε low, aε determined in εtep 502, the next peak from the spectral difference function iε then obtained in step 402, Otherwise, a peak from the formant difference is obtained in step 506. The mean value of the function is low if at least half of the values in the region are leεε than half of an upper limit, in this case 40%. After a peak is obtained in either step 504 or 506, it iε then determined whether the peak identifieε a region in εtep 508. A peak identifieε a region if the peak is between the 40ms and 140ms from the start of the region, e.g., 4 to 14 frames from the start. If the peak identifies the region, and if the ration of low power to mid-power over the region is greater than a predetermined threshold for the mean power of a sonorant region, the end of the sonorant region is identified in εtep 512 and the end of that region is returned and the type of this subregion iε εet to be εonorant in εtep 514. The predetermined threεhold for the εonorant uεed in εtep 510 iε 4,000 if the εpeaker iε preεur.ed to be female, or 300 if the εpeaker iε preεumed to be a male. Thiε distinction iε baεed on presumed gender cr.ly and iε actually based on the fundamental frequency of the speech εignal. The end of a sonorant region is identified in step 512 by identifying the frame in which the lov-power to mid-power ratio iε greater than thiε threεhold and -here the frame iε not yet the frame with the maximum low-power. In step 510, if the mean power ratio iε less than the predetermined threshold, the mean power ratio is then checked in a smaller region in step 516. If the mean power ratio of the smaller region is below the same threshold, that ε aller region iε returned as a sonorant subregion in step 518. Otherwise, it is then determined in step 520 whether the mid-power has a peak in thiε region. If the peak iε identified, the region iε returned as an onset. Otherwise, it is returned as a sonorant in step 524. If the peak in the power waε not far from the εtart of the region aε determined in εtep 500, and if the previouε type of region was a nucleus-end, the laεt frame starting with the start of the region and having a flat slope in the power curves within the length of a typical sonorant region is then identified in step 528. Thiε value may be indicated as the "end-so-far". Step 528 utilizeε 80mε aε the length of the typical sonorant region. The εlope in the power curve is flat, for these purposeε, if abεolute value of the slope is lesε than the maximum εlope expected for a εonorant region, which iε about 12%. If it iε determined that the previouε type of region iε not a nucleuε-end, the firεt frame of thiε εubregion is selected as the "end-εo-far" (εtep 530). Uεing the "end-εo-far" value, the laεt frame within a typical εonorant region having typical εonorant characteriεticε iε then identified in εtep 532. Again, the typical εonorant region iε preεumed to be, for example, 80mε. A typical εonorant characteriεtic in thiε embodiment iε that the low to mid-power ratio iε greater than a predetermined threshold (i.e., the sa-e aε diεcuεεed in εtep 510), and either that the mid-power cf the frame iε lesε than a threεhold (e.g., 3,000) or that the slope is flat, aε defined above. If thiε frame iε the laεt frame of the region, as determined in step 534, it iε returned aε the end of a εubregion of which the type is εonorant in εtep 536. If the frame identified in step 532 is not the last frame of the region, it is then determined whether the region it defines is large enough to be a true region (step 538). Thiε frame is then returned and the region iε indicated as being sonorant in step 540. Otherwise, it is then determined whether the region defined by the value "end-so-far" is large enough to be a region. If it is, that value is returned as - 1 - an indicator of the end of the region in step 544 with the region-type being sonorant. Otherwise, the "end-so-far" value is returned with the type of region unchanged. In effect, a gap is created in the classification of the voiced region, which is handled at a later time. The assignment of the branching-type of a nucleuε will now be deεcribed in connection with Fig. 15. If the parameterε given for the language to be recognized indicate that there are no branching nuclei, aε determined in εtep 550, and if a long nucleuε iε found and double nuclei sequences are possible in this language (e.g., Japanese), as determined in step 552, an output string "sequence" is generated in step 554. Otherwise, a εtring "non-branching" is output in step 556 and the proceεs is completed aε indicated at εtep 557. If the language permits branching nuclei, and if branching nuclei are headed (e.g., aε in Engliεh) aε determined in εtep 558, the length of the region is then compared to a maximum value in εtep 560, Thiε value indicateε the typical maximum length of a εingle nuclear position, and is typically 80ms. In thiε caεe, the εtring "non-branching" iε returned in εtap 562. Otherwiεe, if the nucleuε iε word-final, as determined in step 564, the εtring "branching" iε returned in εtep z -. β . If the region iε not word-final, and the length of the region iε greater than the minimum length for a branching nucleuε (typically 200mε) (step 568), the string "branching' is returned in step 570 otherwise the string "unsure" iε returned in εtep 572. If branching nuclei are not headed (εee εtep 558), the string "branching" iε returned if the length of the region iε greater than 140mε; otherwiεe the εtring "non-branching" is returned. The threshold of 140ms is determined by taking the average of the maximum single nucleuε length and the minimum branching nucleus length. Processing of a nuclear region to identify the phonological elements A, U and I, will now be deεcribed in connection with Figs. 16 and 17. In thiε process, a variable "ct" is εet to be the εtart of the region to be proceεεed in step 580. The value "ct" will now be referred to aε a current frame. The preεence of the A, U and I elementε iε then detected in the current frame in εtep 582. Detection iε performed by the A, U and I detectorε 104 (Fig. 6) wh ch will now be deεcribed in more detail. The detectorε εearch for each phonological element independently of the otherε. The search may be done either serially or in parallel. There are 3 possible outputs: the phonological element is either (i) absent, (ii) present aε the "head" (moεt εalient component) or (iii) preεent aε an "operator" (leεε εalient). Only one phonological element εhould be found aε the head in any εearch. If an element iε detected aε a head in Engliεh then a branching nucleus is preεumed to exist. The detectors return a three-bit code which repreεents the presence of U, I and A. The detectors 104 uεe tne εpectral εlice of a given frame and calculate the energy in various frequency bands, in particular, the lOOHz-lkHz, :-2kHz, and 2-4kHz bands. The position, (i.e., the frequency) in the spectrum at which the amplitude firεt falls below l.dB below the maximum value iε alεo calculated. Thiε poεition iε referred to as the "20dB" value and iε uεed as a rough guide to the property "Maεs," used in the definition of A. For A to be present aε an operator the 20dB value muεt be greater than 500Hz and FI muεt be between 450Hz and 1200Hz. For A to be preεent as a head, either (1) the 20dB value muεt be greater than lOOOHz and FI and the frequency F2 a plitudeε muεt be subεtantially equal, within lOdB, and F2 can be no more than about twice the frequency FI, or (2) with no re-triction on the relative amplitudeε of FI and F2, the 20dB value muεt be greater than 1200Hz, or the energy in the firεt quarter of the (log-warped) spectrum must be greater than a threshold value. For U to be preεent aε a head, energy is concentrated in the region below 1 Hz, and FI and F2 are below about 500Hz and 950Hz reεpectively. The amplitude of F3 iε around 35dB below that of FI . For U to be preεent aε an operator there muεt be a rapid fall-off in energy above a particular frequency which varies according to the head element. In other wordε it actε like a low-paεε filter with variable cutoff frequency. The salient property of I is a low frequency FI (less than 500Hz) and a greatly increased frequency F2 such that it approaches the frequency F3. This produces a characteristic pronounced dip in the spectrum. The detector 104 meaεureε the intervalε between FI and F2, and between F2 and F3 (which may be irrelevant), aε well aε the depth of the dip, defined aε the amplitude difference between the higheεt peak and the loweεt point of the dip. Aε an operator, I appearε with an F1-F2 (or F2-F3) interval of at leaεt 900Hz, and an F2 amplitude relative to the loweεt point greater than 0.3 timeε the depth of the dip. For I to be preεent as a head, only the F1-F2 interval is relevant: the dip depth must be greater than 20dB and the F2 factor is increased to 0.4. Further the energy in region 2-4KHz rust be greater than that in region l-2KHz. Another condition where I iε preεent aε a head, should any of the previous conditionε fail narrowly, the F2-F1 interval in the frequency domain iε greater than 68% of the frequency range. The invention iε not limited to the detailε of detection of the A, U and I detectorε aε described above. There are many ways to implement the details of detection of these elementε. For example, neural networkε could be trained to do pattern matching and diεtinguiεh among the A, U and I phonological elementε aε both head and operator elements in a phonological expresεion. The proceεε uεed to detect theεe elementε could alεo be changed due to a change in the theory as to how theεe elementε are manifeεted acoustically or are combined in phonological expreεεionε. Alεo, low level pattern recognition for the other phonological elementε can be improved. For example, a neural network or other pattern recognition device could be uεed. Next, a running count of the detection of A, U and I in thiε region iε kept and is incremented in step 584. For the current frame, an element code iε then generated in εtep 586. In thiε embodiment, the preεence of an "A" iε given a value "l", the preεence of an "I" iε given a value "2", and the preεence of an "U" iε given a value "4". The valueε of all the elementε present in a frame are then added together to obtain this element code for the current frame. The head count of each element detected aε a head element iε then incremented in εtep 538. Whenever an element iε identified or detected aε a head in a frame the total for that element iε incremented aε well aε a counter which keepε track of the number of frameε in v ich the element appearε aε a head, The current counter iε incremented in εtep 590. If the laεt frame haε been reach÷d as determined in step 592, εtep 582 is then performed, otherwise step 594 is performed to complete the generation of the element code for thiε region according to whether it iε branching or not. Thiε εtep 594 will now be deεcribed in more detail in connection with Fig. 17. The proceεs described in Fig. 16 involves handling all the caseε of the typeε of output εtring aε output in connection with the aεεign ent of the branching-type, diεcussed above, in connection with Fig. 15. If thiε output string beginε with a "N" or "B" (εtep 600), and if the number of headε appearing in the region iε greater than 50% of the length of a region, aε determined in εtep 602, the region iε indicated as headed and as being a long vowel in step 604. In either case, and if branching nuclei are headed, as determined in step 606, processing proceeds to step 608 which will be described in more detail below. If branching nuclei are headed, the element code is set in step 610 before proceeding to step 608. This step 610 will be described in more detail below. If the string indicates a sequence, (e.g., begins with "S") as determined in step 612, the region iε indicated as headed in step 614 and the element code iε εet in εtep 616 before proceeding with step 608. If the string begins with a "b" as determined in step 618, and if the number of heads is less than 3 as determined in εtep 620, the region is indicated as not headed in step 622; otherwise, it iε indicated aε headed and a long vowel in εtep 624. If the string indicating a nucleus-type indicates that it is "unsure" (e.g., it begins with "u"), aε determined in εtep 626, and if the length of the region is leεε than 120mε as indicated in εtep 628, the number of heads is examined in εtep 630. If this number of headε is lesε than 80% of the length of the region (in termε of the number of frameε), proceεεing proce÷dε with εtep 608. Otherwiεe the region is indicated aε heated and containing a long vowel in εtep 632. Proceεsing then troceedε with step 608, If the length of the unεure region iε greater than 170mε, aε determined in εtep 634, the head count of the region iε examined in εtep 636. If the number of headε iε greater than 50% of the number of frameε, the region iε headed and is a long vowel as indicated in step 638. In either of the headed or unheaded case, processing proceeds with step 608. If the length of the region is between 120ms and 170ms aε determined in εtep 628 and 634, and if the number of headε iε greater than 33% of the length, as determined in step 640, the region iε headed and containε a long vowel aε indicated in εtep 642. Proceεsing then continueε with εtep 608. If the region iε non-branching aε determined in εtep 644, the region is not headed in step 646 and processing proceeds in either case to step 608. The remainder of this procesε, starting with step 608, determines the branching-type and element code for the region. If the language under examination doeε not permit branching nuclei, the element code is returned in step 648. Otherwise, if branching nuclei in this language are not headed, as determined in step 650, the branching-type is adjuεted in εtep 652 before returning to the element code in εtep 648. If branching nuclei are headed, and if the region iε not headed aε checked in εtep 654, the element code iε εet in εtep 656 and returned in step 648. If the region is headed and if it contains a long vowel, aε checked in εtep 658, the element code iε adjuεted in εtep 660. If there iε no long vowel, the element code iε adjuεted differently in εtep 662. Steps 666, 660 and 662 will be deεcribed in more detail below. If the element code obtained after performing step 660 or 662 contains a zero, this is corrected in step 664 and the element ode iε then returned in εtep 648. The "e-rode" iε set in the steps calling for "set-e-code oy determining whether the total valueε of A, U and I are mere than 50% of the number of frames in the region. For each element satisfying this condition its appropriate code (i.e., 1, 2 or 4) is added to the element code value. If this element code is zero, after that step it can be set if the total of any of the phonological elements is 1. In εtep 662, the element code iε adjusted by multiplying the element code of the first part of the long vowel by 10 and adding the element code of the second part of the vowel. In step 660, the element code is adjuεted by εetting A, U and I to be present when their total is greater than or equal to the minimum of 20 frameε, or the length of the region. The reεultant code is then multiplied by 10. If the resulting element code is zero, the element having the higheεt head count iε then used to εet the element code. If the element code is still a zero, it is set to 4. i.e., the nucleuε iε aεsumed to contain the element U. A processing of a voiced region after an unknown, onset or fricative region, i.e., stepε 310 and 312 of Fig. 9, will now be deεcribed in more detail in connection with Fig. 18 and Fig. 19. If the previouε type of region is unknown, aε determined in εtep 1060, and the region number iε not that of the final region, as determined in step 1062, it iε likely that prevoicing is present in that region. If the length of the region is εhorter than a typical prevoicing region (εtep 1063), there iε no prevoicing. If the length of the region is εuch that prevoicing iε preεent, aε determined in εtep 1064, the previous region type is εet to low-tone. It iε possible simply to asεert that prevoicing exiεtε εo that, aε a reεult the previouε region type iε always εet to low-tone In either caεe, the laεt true value (indicating the laεt positively clasεified frame) iε εet to the end of that reg.on in εtep 1068 and proceεεing continueε with εtep 1074 to oe deεcribed below. Similarly, if the previouε type waε unknown and the region number is that of the final region, procesεing proceedε to εtep 1074. If the previous type was not unknown, and the previous type iε neither onεet nor fricative, aε determined in εtep 1070, failure iε indicated in εtep 1072. If the previouε type iε onεet or fricative, proceεεing proceedε with εtep 1074 which will now be deεcribed. The remainder of the proceεεing of thiε region iε similar to the procesεing deεcribed in Fig. 11. It iε firεt determined whether a nucleus end haε been detected with certainty in step 1074. If not, an attempt iε then made to find a sonorant in εtep 1076. The laεt region type and the current region type are then εet in εtep 1078 εimilar to εtep 380 in Fig. 11. If the nucleuε end haε been detected with certainty, and the number of peakε paεεed iε equal to the number of nuclei, aε determined in εtep 1080, a εonorant iε forced in step 1082. Otherwise, an onset iε forced in step 1084. An onset iε forced by identifying the frame at which a rise in the power curve begins. Also, the previous region type is εet to "ONSET" and a "0." iε added to the output token εtring if thiε frame iε sufficiently far (40mε) from the εtart frame. After either forcing an onεet or εonorant, or finding a sonorant, if a glottal stop has been held, as determined in step 1086 (εee Fig. 10, εtep 346), the region type iε then set in step 1088. In either case, a gap is checked for and adjusted for in step 1090. If the region has now been completed, as determined in step 1092, procesεing iε done aε indicated in εtep 1094. Otherwiεe, the remainder of the region iε proceεsed as a nucl÷uε region, εtep 1096. After proceεεing to identify a nucl÷uε subregion, if the region haε been completely proc÷sεed (εtep 1098), procesεing iε done aε indicated in εtep 1099, otherwiεe, the proceεεing returnε to εtep 1074, Proceεεing of a region aε a nucleus region will now be deεcribed in more detail in connection with Fig. 19. Firεt, the εtreεε value iε checked and updated in εtep 1100. The "εtreεsed" boolean value is set to false if the local maxima in the power curve have been paεεed and a nucleuε haε been aεsigned already within the current voiced region. If a possible εtreεsed nucleus remains, as determined in step 1102, an attempt is made to find a nucleus as indicated in εtep 1104 after which the region iε aεεigned a type with reεpect to the nucleuε type εtep 1106. If no εtreεεed nucleuε remainε, a nucleuε iε forced ir. step 1108. A nucleus iε forced if a region of εufficient length is identified by a frame in with the total power and mid power are greater than that of the first frame and where the midpower iε greater than 0.2 times the local midpower maximum. Whether εtreεεed or unεtreεεed, processing proceeds to step 1110 in which it is determined whether an identified nucleuε region iε large enough. If the region iε large enough, i.e., having a typical minimum nucleus length, it is assigned a type in step 1112, A, U and I elements are identified in step 1114, the output string is updated in step 1116. If the region is not a large enough nucleus region, or after the output string has been updated, if the nucleus type is an edge as determined in step 1116, the previous type iε set to be an edge in εtep 1120. In either caεe, proceεεing proceedε to step 1122 to determine whether the voiced region has been completely processed. If it has been completely proceεsed the previouε type iε εet in εtep 1124 to be the type of the laεt region, εuch aε may have been εet in εtep .078. In either caεe, the proceεεing is complete in step .126. After procesεing in accordance with the εtepε 306, 310 or 312, an output εtring iε conεtructed and returned from step 308, which will now be deεcribed in more detail in connection with Fig. 20. Firεt, the output εtring iε εimply returned if the last positively identified frame or clasεified frame waε not the εtart frame and if a nucleuε waε aεεigned (εtep 1130). Otherwise, if this iε the firεt region, the previouε type iε εet to low-tone and the output εtring iε returned in εtep 1136. If the region being proceεεed iε not the firεt region, and if it iε not a εtreεεed region aε determined in εtep 1138, it iε set to be either a nucleuε or a εequence of nucleus-εonorant in εtep 1140 and the output string iε updated to step 1142 and returned in step 1136. In step 1140, the region is asεumed to be an unεtreεεed rhyme if the duration iε leεε than lOOmε and the previous type is εet to be "NUCLEUS" and the output token εtring includeε "N1;0.". If the duration iε greater than lOOmε, the previouε region type is set to be "SONORANT" and the output token string is updated to include "N1;0.R.". If the region is stressed, and it is not likely to be a single nucleus, i.e., greater than 120mε, aε determined in step 1144, the region iε εet to be either nucleuε or sonorant in step 1146. In particular, the previous type is "NUCLEUS" and "N1;0." is added to the output string iε no nucleus was aεεigned and the laεt poεitively claεεified frame waε not the εtart frame. Otherwiεe the previouε type iε "SONORANT" and "N1;0.R." iε added to the output εtring. The output εtring iε updated in 1148 and returned in step 1136. If the region is leεε than 120mε, it iε aεεigned a nucleuε type in εtep 1150, the previouε type is set to be nucleus in step 1152, A, U and I elements are identified in step 1154. The output string is then updated in step 1156 and then returned in step 1136. Steps 1150-1156 are similar to steps 362-366 in Fig. 11. Having now described the proceεε of subclasεifying a voiced region, the proceεε of εubclaεsifying a voiceleεε region into εubregionε of εilence and friction (εtep 128 in Fig. 8), will now be deεcribed in connection with Figε. 21-27. Theεe figures describe the "parse_unvoiced" procedure found in the Appendix. The firεt εtep of parsing an unvoiced region is identifying whether the region iε the initial region in εtep 670. An initial region iε proceεεed differently in εtep 672, aε will be deεcribed below. After proceεεing either of an initial or non-initial region, the proceεεing of the region iε then completed in εtep 674 which iε alεo deεcribed in more detail below. In an initial region, if the mean power iε below a given threεhold, such aε 1,000, aε determined in εtep 676, an error iε returned in εtep 678. Otherwiεe, the laεt frame in the region in which the "h" phonological element iε not present, is then identified in step 680. This frame iε given the index "j" from the εtart of the region. If "j" indicateε a region which iε longer than a typical low-tone length, aε determined in step 682, a new variable "v" iε εet to the value j-1 in step 684. A typical low-tone length used in step 682 is 50mε. Firεt, the preεence of a voice bar iε detected, indicating the element L (low tone), which iε often miε-clasεified aε a voiceleεs region by the ESPS function. If the mean power between the εtart frame and the vth frame iε leεε than the minimum mean power for low-tone, e.g., 50,000 (εtep 686), an error iε returned in εtep 688. If thiε mean power iε greater than the low-tone minimum power, the first frame from the start frame with voicing probability greater than 0.2, indicating the εtart of the voice bar, iε then found in εtep 690. Thiε value iε εet to be "j". Next, if the value "v" iε near the end of the region aε determined in step 692, a previous region value is set to be "v" and the value "L" is returned. If "v" is not at the end of the region, the output string is updated to include the string "L" and a previous region value iε changed to "low-tone" (εtep 696) . If the value "j" is not longer than the typical low-tone length, or if a low-tone has been detected in step 696, the procesεing continueε with εtep 698 of finding a fricative, a proceεε to be deεcribed in more detail below in- connection with Fig. 22, which findε the end of a fricative region within a region defined by a εtart value, which iε initialized aε "j", and the end of the region being analyzed. This fricative region is then proceεsed in εtep 700, a process to also be described below, which returns a Boolean value as to whether the phonological element "h" has been found. If an "h" haε been found as determined in step 702, the output string iε updated in εtep 704. After the output εtring haε been updated or if no "h" haε been found, it iε next determined, in εtep 706, whether the end of the region haε been reached. If the end haε been reached, proceεεing of the region iε completed in εtep 674, aε will be deεcribed below. Otherwiεe, a value "s-start" is εet to the value "f-end + 1". The region from ε-εtart to the end of the region iε then proceεεed to find a εilence region in εtep 708. Thiε proceεε will be deεcribed in more detail below in connection with Fig. 25, and returnε a value "ε-end" which indicateε the end of the silence region within the region processed. If the subregion defined by ε-end iε large enough, aε indicated in εtep 710, the output εtring iε updated to include an "S" in εtep 712 and the previouε region-type iε εet to be εilence. Otherwiεe, the value s-end is set to be ε-εtart. If the end of the region haε been reached, aε determined in εtep 716, proceεεing of the region iε complete in step 674. Otherwise, the start value is then reset to be s-end + 1 in εtep 718 and proceεεing returnε to εtep 698. The proceεε of finding a fricative will now be deεcribed in more detail. The firεt εtep of finding a fricative iε to determine whether the phonological element "h" iε found aε determined in εtep 720. An "h" iε found where the zero croεεing level iε higher than a threεhold or if the value of Kl is below a given threεhold. In thiε embodiment, the zero croεεing rate threεhold iε 2,000 and the Kl threεhold iε 0.8. If an "h" iε found, for each frame in which "h" is found until the laεt frame iε identified, a head or operator iε then identified. In thiε process, an "h" iε found if it is found either as a head or an operator. It iε found aε a head if the zero croεεings are higher than the threshold of 5,000, and the value of Kl is lesε than zero. For each frame where h is identified as a head, a head count is incremented. "h" is found as an operator if the zero crosεing rate iε higher than 2,000 or if the value of Kl iε leεε than 0.5. For each frame in which "h" aε an operator iε found, an operator count iε implemented. When the laεt frame haε been identified and a value of "j" has been set as its index or if the element "h" is not found, the fricative type is then set in εtep 724. It iε identified aε a head if the head count iε greater than the operator count, otherwise it is indicated as an operator. Next, a burεt is detected in step 726. A burst is identified if the maximum zero crossing rate between the start and the end of the region being analyzed is greater than a minimum zero crosεing rate threshold for a burst, which is typically 6,000. If the index "j" is at the end of the region as determined in step 728, a burst is added to the fricative type code in step 730 and the procesε returns to the value "j". Otherwise, if the region defined by "j" is too short, i.e., leεε than 20mε, the index of a start frame, plus one, is returned in step 734. Otherwiεe, if the whole region from start to end, being analyzed, is lesε than lOOmε long, the end of that region iε returned and the fricative type iε εet to be operator if the mean ZCR in the region iε greater than 2,000 and iε set to be head if the mean ZCR iε greater than 5,000. Otherwiεe, the εtart value pluε the value "j" iε returned in εtep 740 and it iε left to the calling function to decide whether a valid friction region waε found. Proceεεing of a fricative region will now be deεcribed. Firεt, a burεt code iε detected in step 750 in the fricative type which is generated by the find fricative procedure discussed above. If the length of the region being processed is greater than or equal to the typical fricative length as determined in step 752, and if the previouε region iε silence as determined in step 756, an output εtring iε generated baεed on the fricative type. If thiε type iε "head" aε determined in εtep 758, the output εtring iε "-f" aε indicated in step 760. Otherwise, the output string iε "h" aε indicated in step 762. If the previous region iε not silence, and the previous region is also neither low-tone nor unknown, aε determined in εtep 764, the output εtring iε updated according to the fricative type which iε examined in εtep 766. The output εtring iε updated to include a "F" if the fricative type iε head, otherwise it iε updated to include "f" (εtep 770). If the previouε type of region examined in εtep 764 is either low-tone or unknown, and if either the region iε εhort or the previouε region type iε friction, aε determined in step 772, the output εtring iε updated to be "f" in εtep 774, otherwiεe it iε proceεεed by step 766-770. If the length of the region being processed is less than the typical minimum fricative length, it is proceεεed as a εhort εubregion in εtep 754, which will now be deεcribed in more detail. If the length of this region iε longer than 20mε, aε determined in εtep 780, the process returns indicating that no "h" has been found in step 782. Otherwise, if the previous region was unknown aε determined in εtep 784, and if a burεt waε detected in the region (εtep 786) the output εtring iε updated to include a "S". If the previous region, as determined in step 790, waε either εilence or low-tone, and the region includeε a burεt aε determined in εtep 792, the string is also updated to include a "h" . The previouε region value iε then εet to be "burεt" (εtep 794). Step 792 and 794 are alεo performed if the previouε region waε unknown. If no burεt waε detected or the previous region is neither unknown, nor silence, nor low-tone, the previouε region value iε set to be fricative in step 796. If the previouε region haε been εet to be fricative in εtep 796, whether the region iε headed iε examined in εtep 798. If the region iε headed, the output εtring iε updated to include a "F" in εtep 800; otherwise it is εet to be a "f" in step 802. After the output string has been updated to include a "h", "F" or "f", after εtepε 794, 800 or 802, the εtring is terminated by a " . " in step 804 and an indication that an "h" has been found is returned in step 806. The processing of a region to identify the end of a silent portion will now be described in connection with Fig. 25. If the region to be proceεεed iε εhort, aε determined in εtep 810, and if the mean mid-power and ZCR indicateε silence as determined in step 812, the end of the region is returned aε the end of the εilent portion in εtep 814. A region iε εhort if it is not greater than about 100ms. If the mean ZCR iε leεε than 1,000 and the mean mid-power iε leεε than 1%, thiε indicateε εilence. If the region iε not εhort or it doeε not indicate εilence, the laεt frame from the εtart to contain a high value of Kl iε identified in step 816. The index of this frame is then returned. A high value of Kl is identified when either the value of Kl of the given frame or its adjacent frame is greater than 0.68. Given this last frame, it is then determined whether it defines a valid silence region. A valid silence region is defined if the frame is the end of the region or if it is more than 40ms from the start of the region. If a valid silence region is identified in step 818 and if the mean mid-power indicates εilence (i.e., iε leεs than 1%), the - £ Δ - index "j" iε returned in εtep 820; otherwiεe, if the value "j" doeε not define a valid εilence region, the εtart value of the region being proceεsed is then returned in step 820. Step 672 of procesεing a non-initial region will now be described in more detail. The first step of processing a non-initial region is finding silence in step 830. The proceεs of finding silence iε diεcuεεed above. Next, in step 832, if the previous region iε either an edge or low-tone, or if the region iε long enough, it iε then determined in εtep 834 whether the region iε final. For a non-final region, if the previouε region iε low-tone, the output εtring iε updated to include a "Z", otherwise it is updated to include a "S" in εtep 840. The previouε region iε then set to be silence in εtep 842. If the region iε final, and if the region haε not yet been completely processed, aε determined in εtep 844, the previouε region-type iε εet to be εilence and the output εtring iε updated to include "S" in εtep 846. After the output εtring haε been updated or if the region haε been completely proceεεed, whether the whole region iε εilence iε then determined in εtep 848. If the previous region iε not an edge, and the whole region iε εilence, aε determined in εtep 850, an error is presumed to exist in this region aε indicated in εtep 852. Otherwiεe, the εtring iε updated in step 854 to indicate "?.". After any of stepε 848, 854, and 842, the proceεεing proceedε with εtep 858 aε will be diεcuεεed below. If the previouε region iε not an edge, iε not a low-tone, or iε not long enough, the value ε-end iε εet to be ε-εtart in εtep 856, proceεεing proceedε with εtep 858. If the region currently being examined from εtart to s-end is not big enough, aε determined in εtep 858, and if s-end is equal to zero, a value "f-start" is set to zero in step 862. Otherwise, the value of f-start is set to ε-end + 1 in step 864. Given f-start, the proceεs of finding a fricative is then performed in εtep 868. Thiε proceεs was diεcuεεed in detail above. Next, the fricative region iε proceεεed in step 868 as diεcuεsed above. If the element "h" iε found aε determined in step 870, the output string is updated in step 872. Steps 870 and 872 are similar to step 702 and 704 discussed above. If the region being examined is the final region as determined in εtep 874 and value "h" haε been found, and the value of f-end iε cloεe to the end of the region, proceεεing of the region is completed as indicated in εtep 674. If thiε iε not the final region, and the element "h" iε not found, or f-end iε not close to the end of the region, the value s-start is reεet to f-end + l in εtep 878 and proceεεing reεumeε with εtep 830. If it waε determined in εtep 858 that the εilence region identified iε large enough, and if the previouε region type waε εilence, the procesεing of thiε region is completed in step 674. If the previouε region was εilence, the previouε region-type iε then εet to be "burεt" and the output εtring iε updated to include a "h." in εtep 882. Proceεεing with this region has been completed in step 674. Step 674 will now be described in more detail. If the region being procesεed iε neither the initial or final region, as determined in step 890 and no output characters were generated by the procesεing of thiε region, and if the previouε region-type waε not unknown, a zero iε returned in step 894. If the previouε type waε unknown, an "x" value iε returned in step 896 indicating an error. If the current region is not word-internal and its length iε greater than 30mε, previous region-type is set to be onset and the output string is set to include a "O" (step 900). If the length is lesε than 30ms, an error indication is returned in step 902. After the output string is updated in step 900, proceεsing returns to ster. 892. The proceεεing of the εpeech εignal by the claεεifier 108 results in an output string which indicates the type of subregionε and phonological elementε identified in the εpeech εignal from left to right. For example, thiε output εtring may indicate "N2;30.Sh . for the Engliεh word "eight". Other typical outputε are "N2;20.S-F" or "?.N2;30.Sh" . The output εtring iε procesεed by a word parεer 154 to obtain the symbolic repreεentation 54. A εymbolic repreεentation of a word containε the structural information of a word: the constituentε, the number of poεitionε contained in each conεtituent, and the phonological expreεεionε aεεociated to theεe positions. The three constituentε recognized for all languages are those described above, namely: N(nucleuε), 0 (onset) and R (rhyme). The N constituent may contain one (Nl) or two (N2) poεitionε. The 0 conεtituent may contain zero (OO), one (01) or two (02) poεitionε. The R conεtituent alwayε cor ainε two poεitionε (R2) . Thiε latter repreεentation may εeem redundant (i.e. if "R" then 2) but iε portrayed in thiε way for clarity. Each phonological expreεεion may be repreεented by a fourteen (14)-bit word. The 10 leaεt εignificant bitε (bit numbers 0 to 9) repreεent the preεence or abεence of each of the ten phonological elementε. If a given bit is set (=1) then the phonological element iε preεent in the expreεεion; if the bit is not εet (=0) then the phonological element is absent. The moεt εignificant four bitε (bit numberε 10-13) indicate which of the elementε (if any) iε the head of the expreεεion. Thiε four bit binary number repreεentε the bit number of the head pluε one, of which bit 13 aε the moεt significant bit. For example, if A is the head of the expression, then its bit number is 0. Therefore bits 10 to 13 are: 0001. If R iε the head of the expreεεion, itε bit number iε 5. Therefore bits 10 to 13 are: 0110. A table of suitable bit assignments to phonological elements is shown in Table 6. The binary number of a phonological segment is stored in an array called "element", where the index for a phonological element in the array correspondε to the bit number of the phonological element. element(O) = "A" '1 element(1) = "I" '2 element(2) = "U" '4 element(3) = "N" '8 element(4) = "#" '16 element(5) = "R" '32 element(6) = "?" '64 element(7) = "h" '128 element(8) = "H" '256 element(9) = "L" '512 Viewed horizontally, bit: 9 8 7 6 5 4 3 2 1 0 element: L H h ? R N U I A TABLE 6 Conεider the following example repreεentationε of phonological expreεsions: Example 1: Decimal value: 3524 Binary value: 0011 0111000100 Thiε expreεεion contains the elements U, R, ? and h. The bit number of the head is 3-1=2. Therefore U is the head of this expreεεion. This expresεion correεponds to a "p" aε in the Engliεh word "put". Example 2: Decimal value: 8338 Binary value: 1000 0010010010 Thiε expreεεion containε the elementε I, # and h. The bit number of the head iε thuε 8 - 1 = 7, or h. Thiε expreεεion correεpondε to "εh" aε in the Engliεh word "shirt". This kind of representation guaranteeε that each linguistically significant phonological expresεion in any human language has a unique identifier (a number) which receiveε uniform interpretation for any human language. The number repreεenting a phonological expreεεion iε εtored in decimal form. The functions bin_to_dec and dec_to_bin (well known in the 'C programming language) handle conversion between decimal and binary forms. The operation of the word parser 154 will now be described in connection with Fig. 28. The word parser 154 receives the symbol εtring 160 which correεpondε to the output εtring updated in the procesε described in connect!:a with Fig. 8. The number of tokens in this symbol string ir then counted in step 162. If the first symbol is not an ";-", indicating a nucleus, aε determined in step 164, an onset iε built in step 166 as will be described in more detail below in connection with Figs. 29A-B. Thiε building of an onεet constructs the symbolic representation of the first onset of the sequence. The next token is obtained in step 168 and a rhyme is then built in εtep 170 aε will be deεcribed in more detail below in connection with Figε. 30A-B. The build rhyme procedure eεsentially conεtructε the symbolic representation of the εubsequent rhyme in the εpeech εignal. If the firεt εymbol is an "N", indicating a nucleus, an empty onset is inserted into a symbolic representation aε indicated in εtep 172 and the proceεsing proceeds to the "build rhyme" procedure, in step 170. If all the tokens are used, as determined in step 174, the search string, or the εymbolic repreεentation 54, iε then returned aε indicated at εtep 176. Otherwiεe, the next token iε obtained in step 178. /An onset is then built in step 180, similar to the step 166 of building an onset. If all cf the tokens are not used, i.e., tokenε remain in the εymbol string, as determined in step 182, the next token is obtained in step 184 and a rhyme is then built in step 186. Procesεing then returnε to step 174. If all the tokens are used as determined in εtep 182, it iε then determined whether domain final empty nuclei are permitted in thiε language, aε εhown in εtep 188. If domain final empty nuclei are not licenεed, an error iε returned in εtep 190. Otherwiεe, a final empty nucleus iε appended to the εymbolic repreεentation in εtep 192 and the εearch εtring iε returned in εtep 176. The proceεs of building an onεet will now be desrribed in connection with Fig. 29A-B. Thiε proceεε acceptε ar itε input a εtring correεponding to a εingle token read frr.m the token εtring output by claεsifier and εegmenter 108 ant returns a string containing one or more constituentε (depending on branching status) with the codes for their associated phonological elements aε deεcribed above. The current syllable index is held in a variable "εyll". An input token can be of any of the εet: {S, Sh, S-F, R, 0, Z, h, f, F, ?, Lf, Lh} , i.e., a token may contain 1 or 2 symbols. The O-R pair (or syllable) index is included in the output string. For example, in syllable 2, the input string* 'Sh' will be converted to the string "021; 192", because 192 = 128+64, for h and glottal εtop, reεpectively. Thiε element code is stored in the variable 'code'. This procesε uεeε the following global variables: (1) BRANCHING ONSET parameter; (2) SO: STOP parameter; (3) HV DEVOICED parameter; (4) token counter; and (5) pointer into the input symbol string. The first step of building an onεet is writing output εtring "Onl", where "n" iε the εyllable index. The next token iε then obtained in εtep 962. If thiε token iε a "L" checked in step 964, the next token is then obtained in step 966 and the value of low-tone is εet to true, If branching onεetε are uεed in thiε language, aε determined in step 968 by reference to the parameterε εet previouεly, and if low-tone is true, as checked in step 970, the output code is set to 512 in step 972 and, on the failure of any of theεe cases, the code iε εet to zero in εtep 974. The code iε then adjuεted according to the token obtained in εtep 976. Thiε adjuεtment from particular tokenε will be deεcribed in more detail in connection with Fig. 29B. After the code haε been adjuεted appropriately, whether thiε language permitε branching onεetε iε then again checked in εtep 978. If branching onεetε are permitted, and either if the firεt character iε either "S" or "f or if low-tone iε true, (theεe are the poεsible heads of branching onsets) the next token iε then obtained. If the f.rεt character of this next token iε "R" or "0" then thiε onεet iε branching and a token counter iε incremented; otherwise it is false. If this onset is branching, it is then checked in step 988, the output string is updated to include " On2;32", where "n" iε the εyllable index (εtep 990). In either caεe, the output εtring iε terminated and returned in step 992. The adjustment of the element code for the first character (step 976) will now be described in more detail in connection with Fig. 29B. If the first token is an "R" (step 1000), and if the language iε Japaneεe (εtep 1008), do the following: if the current token is the final token of the string (step 1010), a "616" (the code corresponding to a nasal sonorant) is written to the output string in step 1012, otherwise a "64" is written to the output string in step 1014. Otherwise, i.e., the language is not Japanese as determined in step 1008, if sonorants contain glottal εtopε in thiε language (step 1002), a "64" iε written to the output εtring (εtep 1004), otherwiεe a zero iε written to the output εtring in εtep 1006. This sequence of steps for processing an 'R' token is then done as indicated in step 1016. If the token is a "h" or a "f", the current code plus 128 is written to the output string. If the token is an "0" (step 1022) a "0" is written to the output string in εtep 1024. If the token iε a "F" the current code iε written to the output εtring in εtep 1028. If this language haε devoiced high vowelε, εuch aε Japanese, the next token iε then obtained in εtep 1032. If thiε next token iε a "S" aε determined in εtep 1034, the output εtring iε augmented to include "#Nnl;2" in εtep 1036. If the token iε a "Z" or a "?", "192" iε written to the output string in step 1040. Finally, if the token is a "S", aε determined in εtep 1042, and if the next character in the token εtring iε either "h" , "-" or "F", "192" lε written to the output εtring in step 1048. Otherwise the "64" is written to the output string in step 1050. Thus, with this procedure outlined in Fig. 29B, the output string is updated to include a code indicating the elements present in the onset. The process of building a rhyme will now be deεcribed in connection with Figε. 30A-B. The input and output of thiε proceεε are the εame aε for building an onset. The initial token received iε one correεponding to a nucleuε, and itε branching εtatuε iε predetermined. The only deciεion to be made iε whether we have a cloεed onset-rhyme pair, i.e., a vowel followed by a coda. So, if the language allowε branching rhymeε, the next two tokenε are examined to check whether there iε a rhymal conεonant. The second consonant must be a governor, or licensor, for the firεt, as per Coda Licensing (Kaye 1990). This is checked by a sub-function licenseε (left, right), which returnε juεt True or Falεe depending on whether the right token governε the left token. Rhyme εtringε are of the form: NO or Nl;x for non-branching nuclei; or N2;x;y for branching nuclei, where x,y are the element codeε and 0<x,y<7; y iε alwayε 0 for the moment, i.e., we ignore εequenceε like [au], [oi] and aεεume all branching nuclei are juεt long vowelε. Relevant parameterε are BR_RHYME; True for Engliεh, Falεe for Japaneεe and Korean. Firεt, the εtring "Nnl" iε written aε the output, where "n" iε the εyllable index (εtep 910). The firεt character iε then obtained from the token εtring in εtep 912. If the firεt character iε not an "n" aε determined in εtep 914, an error is returned in εtep 916. Otherwiεe, the εecond character from the token is obtained in step 918. If the εecond character iε "0", a "0" is written to the output string in step 922. If the second character is a "1", aε detected in εtep 924, it iε determined whether there iε a branching rhyme with a poεεible coda in εtepε to be deεcribed in more detail below. If the εecond chararter iε a 2 aε determined in εtep 926, the element code is then retrieved and written in step 928 and the sequence =.Nn2;0" is then written to the output string. In the default caεe, something valid is written in εtep 930 to the output εtring, εuch aε a zero. The output εtring iε then returned in εtep 932. If the second character obtained iε a 1, and branching rhymeε are permitted in thiε language, and thiε iε not the laεt token, as determined in step 934, the next token iε obtained in step 936. If this next token is a posεible coda token in the language, it iε determined in εtep 938, yet the next token iε then obtained in εtep 940. It iε then determined whether thiε moεt previouεly obtained token licenεeε the potential coda token in εtep 942. If it doeε not license the potential coda token the output string is returned in step 932, Alεo, if the next token iε not a poεsible "token" the output εtring iε then returned to εtep 932. If no branching rhymeε exist or if this token iε the iaεt token, the rhyme εtring iε terminated in εtep 944 and the output εtring is returned in step 932. If it is determined that the onset licenseε the potential coda token, the token counter iε incremented in εtep 946 and the output string is updated to include the string " Rn2;" where "n" is the syllable index (step 948) . Next, if this second token haε been determined aε a coda iε an "R" or an "S", the output εtring "64" is written and the output εtring iε returned in εtep 932. If the token iε an "f" or "F", the εtring "128" iε written to the output εtring and the output εtring iε returned in εtep 932. Otherwiεe, a "0" iε written to the output string and the output string is returned in step 932. In the current syεtem, step 942 is performed in the following manner. If the potential onset token is a "S" the condition is true. If the potential ooda token is "r" the licensing is true if the onset token s alεo either "F" or "f" or "R" . In all other caεeε, the condition failε. Theεe conditionε reεultε from the fact that a εtop in an onset will govern anything while a fricative or a sonorant can only govern εonorantε. (In fact, only a naεal εonorant can govern another sonorant.) With the proceεε of parεing completed, the εyεtem haε now provided a symbolic representation of a word or words in the speech signal. As an example, the symbolic representation of the English word "eight" could be, for the firεt example token εtring: 013;0#N11;3#N12;0#021;192#N21;0 For the second example token string for "eight," the symbolic representation would be the same, except that "013;0" would likely be replaced by "Oil;64". This should be the caεe when the word iε pronounced with an audible initial glottal εtop. In thiε εymbolic repreεentation, each field beginε with an "0" or an "N" and terminateε with a "#". The first digit following the "0" or the "R" is an index into the string, increasing by l for every O-R pair. The second digit is the number of the εkeletal position in that constituent. That is, for a branching conεtituent there would be a εecond entry, εay 012, for the contentε cf the εecond position. The second digit is followed by a ";". The decimal number following the ";" correεpondε to the phonological expreεεion aεεociated with the skeletal poεition. Thus each field (delimited by #..#) correεpondε tc a phonological εegment. To match the εymbolic representation to a word, it iε compared to a lexicon εuch aε lexicon 58 in Fig. 2. The lexicon 58 of Fig. 2 will now be teεcribed in more detail. A lexicon 58 iε preferably astabliεhed for each individual language. It may be in the form of a computer-readable file called, for example, "language.voc, " where "language" iε the name of the language in question (or the first eight characters thereof) and the extension ".voc" (vocabulary) indicates the format of the file in which lexical data iε stored. For example, a lexicon 58 for Engliεh may be labeled "engliεh.voc" . Thiε file-name format iε exemplary only, and many other formats may be suitable. Fig. 31 shows at 248 a lexicon, for the numbers 1 through 10 in Japanese, in human readable form with typical lexical entries 250 which may be used in thiε εyεtem. Each lexical entry haε two lineε: a firεt line 252 containε an indication of the εtandard orthography of the word for the language in queεtion, εuch aε an ASCII text repreεentative of the word. Thiε firεt line may εtore anything which may be used to diεplay a human-readable representation of the word. In this example, an Engliεh word and a phonetic repreεentation of the Japaneεe word iε εhown. Moεt computer εystems have the ability to display numerous Western and Eastern scriptε for thiε purpoεe. Thiε first line is for display purposes only and is not uεed in the recognition process itself. The second line 254 of each lexical entry for a word contains a lexical representation of a word. A typical lexical representation, for example the word "eight" in English would appear aε follows: 01:0, ,N1:2051.N2:2,01:6368, ,N1:0 In this lexical representation, each phonological segment is delimited by a ",". It includes an indication of the conεtituent-type (the firεt two characterε) of which the firεt character indicates either an onset ("0"), nucleuε ("N"), or rhyme ("R"). It also indicateε the number of εkeletal poεitionε in thiε constituent, which iε either 1 or 2. A rhyme conεtituent iε al" ayε marked with a "2" and only follows a non-branching nucleus ("Nl"). The phonological expreεεion aεεociated with thiε εegment follows the ";" and is generated according to the same principleε diεcusεed above uεing a 14 bit word to identify the phonological expreεεion. The lexical matching εystem 55 (of Fig. 1), which will now be described in more detail, matches a symbolic representation 54 from the parser 154 to the phonological representation of each lexical entry in the lexicon 58. A general deεcription of the concept behind lexical matching will be provided firεt. The matching to be performed can be preεumed to be a form of pattern matching uεing incomplete vectors. Thus, a variety of implementationε can be enviεaged, such as asεociative memories and content addressable memorieε. More particularly, a εyεtem can uεe a form of a weighted distance metric to identify a best match in the lexicon to the symbolic representation. More particularly, a εearch may proceed aε follows. Once all onsets have been identified, a search pattern can be produced in the form of a consonantal skeleton plus the positionε of εurrounding nuclei, which may then be input to a searching syεtem. For example, for the input word 'letter', we should have the sequence [l_t_], where the alphabetic εymbolε are only intended aε abbreviationε for the phonological expreεεionε which form each particular εegment. The "t" may well only conεiεt of 1 or 2 elementε from the εet of (R, ?, h) . Thiε is immaterial however, aε it doeε not appear as if it would produce any εeriouε miεmatcheε. For example, if only ? or h is found, the reεulting output from a lexical εearch iε only the wordε containing εtopε or fricativeε and εtopε reεpectively. In the cr-e of iεolated word recognition where no higher level conεt αentε are available, if a ? element iε found we know from the phonology of Engliεh that the εounc must be a "t" and the parser can εupply the remaining eler÷ntε to narrow down our εearch. If R and h are found, we have a poεεible miεmatch with 'lesser', but this is a common human error in any case. At this stage, the result of any preliminary search iε a short list of wordε fitting the identified conεonantal skeleton, say: 'latter', 'litter', 'letter', differing only in the content of the nuclei. In order to select the correct word, we merely need to identify the firεt nucleus, and recognition would then proceed by identifying the elements associated to that position. Another strategy would be applied in the case of the word 'potato', such as may be pronounced quickly, resulting in a dropped first 'o', resulting in the εtring [pt t_] , (where the two underscoreε indicate a branching nucleuε) . Given that the εequence 'pt' doeε not form a possible onset, the string would be modified by the parεer to [p_t t_] before εearching the lexicon, and again potential εearch errorε are avoided. In thiε caεe the skeleton will be sufficient to produce a single match in the English lexicon, and no vowel identification will then be necesεary. The process of accessing a lexicon to match a symbolic representation to a word will now be described in connection with Fig. 32. The input to the lexical matching εyεtem iε the εymbolic repreεentation 54 output by the parεer 154. This representation, as discusεed above, contains information on the structure (Onset-irucleuε pairε) identified in the εignal, pluε their aεεociated phonological elementε. Whatever εtructure has been recognized, the symbolic representation 54 will be a well-formed sequence. The first step 210 of the lexical matching proceεε is reading the contents rf the lexicon for the εpecified language, from "language.voc, " into an a two-dimenεional, n-entry array, herein called "entry", where n iε the number of entrieε in the lexicon. The firεt field, entry [n][l], stores the word's ortnography or translation, and the εecond field, entry [n][2], stores the actual phonological structure. An array, herein called "weights", is alεo created to εtore weightε or εcoreε aεεigned to each entry aε matching proceedε. All weights are initially zero. In the event of multiple matches, after all parts of the input string have been tested, the weights may be uεed to select the best match as will be described below. Another array, herein called "statuε" is alεo created to monitor which wordε are εtill candidate matches during the matching procesε. Each word in the lexicon haε an entry in the array "εtatuε" of which the value iε either "l," indicating a poεεible candidate, or "0," indicating a rejected word. The symbolic representation is then read into a string in step 212 from a file created by the parser and called "language.dat," where "language" is the name of the appropriate language: default (English), Japanese, or Korean. The firεt data field (up to the first #) is εtripped off the front of the εtring in εtep 214, and decompoεed into a ε ~t of variables corresponding to constituent type, skeletal position, index number and element data (step 214). Aε long aε there iε no input left, as determined in step 216, procesεing continueε. Properties of the source constituentε are then obtained in εtep 218. More particularly, the element data, i.e., the number between the εemi-colon and the , iε converted to binary, for example, by the C programming language command dectobin", producing a variable, herein called "bεource" . Aε diεcuεεed above, each of the more εignificant bitε m thiε number corresponds to a phonological element: a 1 indicates the phonological element is present, whereaε a 0 means the phonological element iε abεent. Only thiε first εegme t iε analyzed on the firεt run. The computer then matches this data against each lexical entry, aε will be deεcrioed below, diεcarding any entrieε which produce a εeriouε miεmatch. The computer then repeatε thiε proceεε for each field on the input εtring, one field at a time. On theεe εubsequent operations the "weights" array is carried over, and not re-initialized. Before performing the repetitions of the procesε, the computer determines the number of the nuclei in the input string. This determination is needed becauεe a word which formε a component of the target word alwayε produceε a match, e.g., in Japaneεe the input "εiti" matcheε both "εiti" and "εi". The fact that "si" only contains 1 nucleus but the target word contains 2 is then used in the matching routine to de-weight "si", but not discard it altogether. The computer matches the input string to the lexicon in the following manner. The computer loops through the lexical entries by setting a variable "word" to be equal to l in step 219, and by incrementing this variable in step 221 after matching of each word is completed, asεigning the orthography and the repreεentation to variableε "orth" and "myentry" respectively. For each word, the computer converts, in step 220, "myentry' into a set of arrays convenient for matching purposes, as follows. Arrays of variableε 0, N and R are set up, of dimension 6 x 2 (6 x 3 for Onεetε). The firεt dimension, arbitrarily set to six, refers to the O-R index, i.e., allowing for a maximum of six O-R pairε. The εecond dimenεion iε for the εkeletal pointε, of which there are either one or two. Theεe arrayε are then filled up with numberε corresponding to the element content of each poεition in the expreεsion. For example in the word 'two', there iε a glottal εtop and an h in the firεt point in the firεt onεet, εo 0[1][1] will contain the value 128 + 64 = 192. There iε no εecond point, εo 0[1][2] will remain Null, e.g., "-1", but not zero. The value of N[l][l] will be 4 (code for the U element), the value for N[l][2] will be zero (not 4 or Null, as it iε lexically empty). For longer wordε, there will be values for 0[2][1], N[2][l] etc. The 0[ ][3] position iε reserved for the empty onεet case. After the lexical entries are converted, a variable "btarget" iε created aε a binary number εpecifying the phonological element content of the εkeletal position in the target word correεponding to the one in the input field. That iε, if the input field containε N, index 2, εkeletal position 1, the εame position in the current lexical entry is examined, which will be stored in N[2][l] and called "bεource". Thiε may be either 1) null, indicating that no εuch constituent exists in that word, or 2) zero, indicating that the constituent exists but is empty, or 3) non-zero, in which caεe the number iε the phonological element content). The computer then matcheε two binary numberε "bεource" and "btarget" in step 222 in a manner described below. This matching procesε returns a weight or value corresponding to the degree of εimilarity between the two binary numberε. If the weight iε positive aε determined in step 224, the weight is added to the current weight for that word, and the word iε written cut to the file "temp.voc" 228 (both "orth" and "myentry" are written) in εtep 226. The number of matcheε, a variable named "nn," iε then incremented by 1. If the weight iε negative aε determined in εtep 224, indicating a fail, the word is eliminated in step 230 and "fail" iε written to "temp.voc," and the εtatuε value for that word iε εet to zero, εo it will not be checked on εubεequent iterationε of the matching proceεε. Whenever the number to be placed in "btarget" iε null, which meanε the lexical entry doeε not contain a conεtituent of the type given in the input εtring, the "dectobin" function -nil fail. Thiε failure doeε not reεult in an automatic matching failure, in caεe εomething waε miεsing in the input Instead the weight is reduced by two; the word fails only if the constituent is an onεet. Onsets should not be misεec, but a nucleus may be missed. In the case of a nucleus, a word fails only if the cumulative weight goes below zero. If all of the words have been analyzed, as determined in εtep 232, the new word liεt iε read in εtep 234 and processing on the next identified constituent in the symbolic representation 54 is performed by returning to step 214. Otherwise the variable "word" is incremented in step 221 and procesεing returnε to step 220. On subsequent iterations, only those wordε whoεe εtatuε iε "1" are checked againεt the new conεtituent. In one embodiment if a word failed on a previouε iteration, the weight of the word may be reduced by one, to provide a scale of failure values. That is, the weight of a failed word may be continually reduced so that the weight of the failed word should be less than the weights of words which more closely match the input. After the whole input string has been tested, as determined in step 216, or the set of candidate matches is reduced to a single word, as determined in step 236, proceεsing stops. If there is only a single word that matches, this word is output in step 238 and the computer stopε processing. If not, the result is a file "temp.voc" which contains a pair of lines for each lexical entry which iε then sorted in step 240. The first of the pair is the orthography and the weight, the second will be either the phonological structure if the word has not failed, or the word "fail". If a word is still in the running, i.e., itε εtatuε iε "1", the word and itε weight value are εtored in a table εtructure called "frame". Thiε table iε then εorted by the weight value, εo that the word with the higheεt weight becoreε the firεt entry in the table. Thiε word iε then output in εtep 238 aε the beεt match. In the event of two words having thiε εame weight, both are output. If there iε no match and there are two gueεεeε in the input file, this same procesε iε repeated with the εecond entry. The matching proceεε of step 222 will now be deεcribed in more detail. The arguments pasεed in are "bεource," "btarget" and the conεtituent type. The conεtituent type iε uεed to allow different processing for onsetε and nuclei, in that onεet matcheε/miεmatcheε are given higher/lower weight valueε than nucleuε oneε. Thiε reflectε the claim of government phonology that onεetε are more important in lexical acceεs. Zero valueε of either "bεource" or "btarget" are treated separately. If "bsource" iε zero, indicating an empty conεtituent in the εearch εtring, thiε can be matched with anything, so the weight iε εet to zero. But, if the target string is alεo zero, a higher weight is asεigned, depending on the constituent type, becauεe an exact match with an empty poεition iε just as significant aε a match between two identical non-empty positions. If "btarget" is zero, but "bsource" iε not, there iε a miεmatch (εomething that iε not there waε recognized in thiε particular word), which normally reεultε in a failure in an onset but not in a nucleuε. Aε a εpecial caεe, empty onεetε are often realized aε glottal εtopε, εo we allow for thiε poεεibility by aεεigning a zero weight. In non-zero caεeε, the "btarget" and "bεource" numberε are firεt decoded into a head value (the firεt 4 bitε of the binary number) and a phonological element value (the remaining 10 bitε). If the headε are identical, the weight is increaεed by 4. Then the phonological element value iε scanned bit by bit, and weights are asεigned according to whether both input and target are 0, 1 or different. The results vary according to constituent type as explained aoove. The number of hits and misεeε are recorded to oetermine whether an exact match waε made with the whole omary number. If εo, the weight iε increaεed by the number rf elementε matched. The reεult of thiε proceεε, for each conεtituent, iε a weight. Having now deεcribed the εpeech recognition and matching proceεε, the development of a εuitable lexicon will now be deεcribed. More particularly, the lexical entry εyεtem (56 in Fig. 2) will now be deεcribed in connection with Figε. 33 and 34. This system useε a default character εet which may be edi t ed by the user. The default character set iε εtored in a computer-readable data file, preferably called: "default.chr. " Thiε file conεiεts of a set of number pairs, such as shown below in Table 7. 0 1 1025 2066 3092 2051 3077 2067 3093 1027 The firεt number of the pair iε the decimal encoding of the phonological expreεεion. The second member of the pair represents the ASCII code of the character of the key to which the phonological expresεion is bound. The particular character εet uεed dependε on the expected phonological expreεεionε to be input, which may in turn depend on the language. The entry εyεtem iε alεo baεed on the theoretical principle that the conεtituent εtructure of any word in any language iε compriεed of a sequence of structural divisionε, each conεiεting of an onεet-rhyme pair. The only variation allowed is whether any or all of the three conεtituentε, onεet, nucleus or rhyme, may contain two poεitionε, or whether they are limited to a εingle poεition. The parameterε defining the constituents for a language stored in a file called "<language name>.par". The number "1" indicateε that the branching parameter iε εet for branching; a "0" indicates that it is not set. The "default" language is set to allow for branching in all three constituents, thus having "111" as its contents. Since Japaneεe haε no branching conεtituentε, the "japan.par" file would have aε itε contentε "000". In order to create the ".chr" and the ".par" files, a phonological analysiε of the language muεt have been performed. A lexical entry system in Fig. 33 includes a lexicon 58, and parameter and input character set files 260 and 262. These are accesεed, edited and εtored by via a parameterε module 264. The parameter module 264 receiveε an indication of a language 266 in order to determine which character and parameter fileε 260 and 262 to acceεs. The analysiε module 268 iε uεed to input new words 270 using the information obtained and maintained by the parameters module 264. Operation of the analysis and parameterε modules will now be described in connection with Fig. 34. When entries are to be added to a lexicon for a language, the user provides an indicator of the language choice (εtep 272). If this language is a new language as determined in step 274, parameters may then be entered (step 276), If the "language.chr" , "language.par" or the "language.voc" fileε do not exiεt, then thiε iε a new language. To enter the character εet, the user iε queried aε to whether the "default.chr" character εet εhould be uεed. If yes, a copy of "default.chr" iε made, uεing the name "<language name>.chr," for example. When a new language iε εtarted, the user is also prompted to supply the branching parameterε, preferably uεing a diεplay and dialog box in the form of three check boxeε, indicating whether the onεet, nucleuε or rhyme may branch. If the language exiεtε, the exiεting ".chr" and ".par" fileε are uεed. Next, new wordε are received from the uεer in εtep 280 aε a series of phonological segments, in onset-rhyme pairs, using the keys asεigned to each εegment. Wordε may be entered using standard orthography or some phonemic type spelling depending on the nature of the orthographic conventions. An input word is keyed in along with the orthography/translation. Buttonε preferably appear on the diεplay screen allowing the entry of branching constituentε if the parameter εettingε indicate that εuch conεtituentε are allowed. No buttonε appear for Japaneεe εince it haε no branching conεtituentε. If one or both memberε of a given onset-rhyme pair is branching (assuming that this is permitted by the parameter settings) then the appropriate buttons are clicked by the uεer. Thus, the computer receives and indication of a constituent, whether it is branching, and the εegment or εegments contained therein. The word is proceεεed by Onεet-Rhy e pairε from left to right in εtep 282. The Analyεiε module 252 checkε if a putative branching conεtituent is well-formed and respondε with an error message if εuch iε not the caεe. That is, the governing properties of proposed conεtituent comemberε are checked. If they have the correct governing propertieε, i.e. if the left member iε a governor and the right member, a governee, then the εtructure iε accepted; otherwiεe it iε rejected and the uεer is prompted to reenter the new entry. The input form is analyzed in a pair-wise fashion until the string is exhausted. The uεer is then prompted to provide bracketing if the form iε a compound (e.g. blackboard = [ [black] [board]]) . Finally, if the uεer iε εatiεfied and the syεtem haε accepted the form aε a legal entry, uεer may indicate thiε fact to the computer which then εtoreε the form in the language database, "<language name>.voc" (step 284). In practice, the analyεiε of a language and conεtruction of an appropriate databaεe iε performed by framed phonologiεtε. A databaεe of lexical repreεentationε of wordε of a variety of languageε maintained in computer-readable form. One may thuε acceεε thiε databaεe and εelect only thoεe wordε in which one iε conεidering to have recognized by the εpeech recognition εystem. There are numerouε advantageε of thiε approach to εpeech recognition over other typeε of εpeech recognition systemε. For example, by requiring words to have a limited εyllable structure and by restricting the signal proceεεing taεk of εpeech recognition to the detection of a small set of phonological elements the problem of phonologically-based εpeech recognition becomeε tractable. Another advantage of thiε invention iε that the εyllable εtructure of a word in a εpeech εignal need not be analyzed, determined, or identified explicitly prior to further claεεification or analyεiε. Rather, the εyllable εtructure iε inferred from other lower level ir. rmation detected in the εpeech εignal. Alεo, the determination of a εymbolic repreεentation of a word in thiε invention eliminateε any durational information about a given εound from the proceεε of matching the εymbolic repreεentation to a lexicon. Thiε iε in contrast to prior εpeech recognition εyεtemε which require εome time-dependent mapping, either with time-dependent εtatiεtical modelε or with time-warping of εpectral templateε. Another advantage of thiε automatic εpeech recognition εyεtem iε that the εame proceεεing engine can be uεed for all languageε except for the εelection of the branching parameterε, licenεing parameterε and lexicon. Given thiε deεcription of a few embodimentε of the invention, it εhould be apparent to those skilled in the art that the foregoing is merely illustrative and not limiting, having been presented by way of example only. Numerouε modifications and other embodimentε are within the εcope of one of ordinary εkill in the art. For example, non-linear models of the auditory perceptual εyεterr may alεo be used to achieve greater discrimination between apparently similar spectra. Extenεionε to tone languageε ray be made by analyzing changeε in the fundamental frequency f to identify changeε in pitch of vowelε. Alεo, there may be εome language-εpecific conεtraintε, εuch aε licenεing conεtraintε, which could be parametrized and uεed to εimplify εpeech proceεεing. Although the system described herein does not provide for detection of the phonological elementε which are related to the place of articulation in conεonantε, nor doeε it perform detection of the element N, defining naεal εoundε. Theεe phonological elementε may alεo be detected in the process of classification described above. For example, the phonological elements related to the place of articulation of a sound can be identified, for example, by analyzing the shape of the spectrum of a release burεt, which typically occurε after the ? element. Alεo, the lexicon could be arranged or εtructured in a manner εo aε to enhance the matching proceεε. For example, thoεe entrieε which are εtructurally εimilar could be made lexically proximate, for example, by uεing a tree εtructure. Alεo, the matching proceεε could be incorporated into new signal processing modules, allowing matching to be performed while the εignal iε being proceεεed. Such integrated matching and εtructured lexicon, would enable a quick elimination of non-matching wordε. Theεe and other modificationε are contemplated aε falling within the scope of the invention as defined by the appended claims and equivalentε thereto. What iε claimed iε: 87 * SPECDIF2.C * A revision of specdiff.c: spectral difference for frame n is calculated by taking the difference of two adjacent spectra as follows: 1-2, 3-4 etc #ifndef lint static char *sccs_ιd = "@(#)specdif2.c 1.0 XX/XX/XX"; #endif #define True 1 #define False 0 #defιne LIN False #defme LOG True tdefme NORMALDB 40 #defιne FORMANTS 4 char 'Version = "1.00"; ♦include <stdιo.h> #include <stdlib.h> ♦include <math.h> ♦include <esps/esps.h> ♦include <eεps/unix.h> *include <esps/lin * its.h> ♦include <esps/feasd.h> ♦include <esps/fea.h> ♦include <esps/feaspec.h> ♦include "mclude/util.h" char *ProgName = "Specdiff"; ♦include "include/spectrum.h" ♦define ax(x, y) ( (x>y) ? x : y) ♦define mιn(x, y) ( (x<y) ' x : y) ♦define FORMENTRYfx, y) (x'FORMANTS + y) ♦define SYNTAX \ USAGE("specdiff -b[start value] -e[end value] -r[range] input.feaspec input.fea output.fea specdiff.txt") /* will normally be used m a script like so: specdiff -rXX:YY lpcspec.fea input.fea output.fea specdiff.txt lpcspec.fea contains spectral records; input.fea contains power and zero-crossings; output.fea contains these plus the spectral difference data specdiff.txt contains the values of all 3 parameters 8 8 In this version, both no of formant records and no of spectral records are twice the number of feature records. This means adjacent pairs of values are used in calculating the spectral and formant difference arrays (0-1, 2-3, 4-5 etc) Formant records are usually a bit fewer than spectra because of the longer window size used in the calculations. */ ********★* Globals ****★★***** / long num_rec; /* no of data records to read */ long num_fearec; /* number of records in feature file */ long num_formrec; /* number of records in formant file */ int num_freqs; /* no of freq's in each feaspec record */ /* should be 257, but we ignore the first value, so SPECSIZE = 256 */ int debug_level = 0; int step_size = 10; /* ms */ char *debugfilename = "/home/geoff/src/DEBUG.TXT" ; ULONG data_length; /* No. of samples in file */ int f_scale_max = 4000, /* Maximum value in formant plot */ f_range; /* f_scale_max - f_scale_min */ float l_range; /* = Maxfreσuency/STARTFREQUENCY */ point curve[2] [SPECSIZE] ; /* For storing curve coordinates: */ /*******************★ external functions and prototypes *#***********★#****** void dump_curve( point curv[SPECSIZE] ); float specmaxl( float *data, long s, long *index, float limit ); int get_formants( struct header *ifh, FILE *iffile, int *peak_set ); void get_specvals( float 'source, float *dest ); char *eopen() ; void lrange_switch() ; float maxl( float data[], long s, long *index ); float axi( float data[] , int s, int *index ); float mini ( float data[] , long s, long *index ); extern unsigned char *Title; /*int e_scale_fac = 1000; FILE *fp, *fout, *flist, *fdata; int Number_output_values; int scale_max, /* Maximum value in difference plot */ scale_min, scale_fac = 10; main( int argc, char **argv ) { char *param_file = NULL; /* parameter file name */ FILE *ifile = stdin, /* input and output file streams */ "ofile = stdout, *ifea_file, *ofea_file, *iffile, /* formant file */ *fp; /* for curve dump */ struct header /* input and output file headers */ *ih, *ifh, *fh, *oh; char *iname, *oname, /* input and output file names */ *ofea_name, /* output FEA file name */ *ifea_name, ♦ ifname; /* formant file name (=formant.fb) */ struct feaspec *rec; /* record for feaspec data */ struct fea_data *p, *q; /* " fea data */ char *fields[] = { "power" , "zero_crossing" , NULL) ,- int /* split, split point used in calculating weights */ error, rval, fin; int *peaks, /* Dynamic array for storing peak values: used as peaks[Number_Of_values] [4] */ lower, upper, /* used in form_diff calculation */ factor_size; /* size of the formant factor array */ const float Min_SD_max = 1.0e4; float this_sample[SPECSIZE] , next_sample[SPECSIZE] , weight[SPECSIZE] ; /* Log weighting array */ float spec_val, tot_pwr, temp, val, xnew, yamp; /* used in calculating logf curve */ float distance, max_value, min_value, range; float *d_data, /* Dynamic array for distance data */ *d2_data, *spec_ptr, /* ptr to diff data in output FEA file */ *form_diff, /* dynamic array for formant weighted difference data */ ♦ factor, /* array of multipliers for form_diff */ 9 0 *dist_ptr; /* pointer to abbve in output file */ short pflag = 0, /* flag for range (pts.) option*/ one_val_short = 0, *Fl_ptr, *F2_ptr, *F3_ptr; char *prange = NULL; /* string for range specification (-p) */ long start, end, /* boundary points for search */ nan, index, loop, last; /* loop counter */ int startf, x_step; float sample_rate, this_pwr, next_pwr, /* tot_power of current & next spec */ this_avg, next_avg, /* averaged spectral values */ temp_zcr, /* zerocrossing value */ *src_sf, /* source sampling frequency */ *pptr_in, *zptr_in, *pptr_out, *zptr_out; double *rec_freq, /* record freq. of source */ *start_time; /* start time in see's in source file*/ char ch, *charpt; char str[255] ; long count, sample, halfnum_rec, /* no of spectral records 1 2 * 1 *start_rec; /* starting record num in source file */ register int f, ct, j, pos, xpos, ypos; /* Cursor positions */ extern int optind; extern char *optarg; short bflg=0, eflg=0, bigspec=0, /* True if num_freqs = 513 */ interval; /* =bigspec+l */ printf(" Spectral Difference ProgramXn" ) ; printf(" ==========================__\n\n") ; while ( (ch = getopt(argc, argv, "b:e:r:")) != EOF ) { switch (ch) { case 'b' : start = (long) atoi (optarg) ; Fprintf(stderr, "start value = %ld\n" , start); if (start < 0) bflg++; break; case 'e' : end = (long) atoi(optarg) ; Fprintf(stderr, "end value = %ld\n" , end); if (end < start) eflg++; 9 1 break ; case ' r ' : prange = optarg; pflag++; break; /* case ' f' : ifarg = optarg; Fprintf (stderr, "Formant file is %s\n" , ifarg) ; break; */ default: SYNTAX; break; } } if (bflg) { Fprintf (stderr, "Error: start value must be >0\n") ; exit(1) ; } if (eflg) { Fprintf (stderr, "Error: end value must be > startXn") ; exit(l) ; } if (pflag) { start = 1; end = LONG_MAX; lrange_switch(prange, &start, &end, 0) ; nan = (end == L0NG_MAX) ? 0 : end - start + 1; /* data_start_time = file_start_time + (start - 1) / sf; */ } if (optind < argc) /* input FEA_SPEC file */ iname = eopen(ProgName, argv[optind++] , "r", FT_FEA, FEA_SPEC, &ih, &ifile) ; else { Fprintf (stderr, "%s: no input FEA_SPEC file specified. \n" , ProgName) ; SYNTAX; } if (debug_level) Fprintf (stderr, "Input FEA_SPEC file is %s\n", iname); if (optind < argc) /* input FEATURE file */ ifea_name = eopen(ProgName, argv[optind++] , "r", FT_FEA, NONE, &ifh, &ifea_file) ; else { Fprintf (stderr, "%s: no input feature file specified. \n" , ProgName); SYNTAX; } if (debug_level) 92 Fprintf(stderr, "Input Feature file " is %s\n", ifea_name) ; if (optind < argc) /* output FEA file */ ofea_name = eopen(ProgName, argv[optind++] , "w" , FT_FEA, NONE, &oh, &ofea_file) ; else { Fprintf(stderr, "%s: no output FEA file specified. \n" , ProgName); SYNTA ; } if (debug_level) Fprintf(stderr, "Output Feature file is %s\n", ofea_name) ; if (optind < argc) { /* output text file */ if ((ofile = fopen(argv[optind] , "wt")) != 0) ona e = argv[optind++] ; } else { Fprintf( stderr, "%s: no output text file specified. ", ProgName); Fprintf( stderr, "Using stdout\n"); ofile = stdout; oname = "<stdout>"; } if (debug_level) Fprintf( stderr, "Outpuffile is %s\n", oname ); /****** Open formant file: NOT specified in command line ******/ ifname = eopen(ProgName, "formant.fb" , "r", FT_FEA, NONE, &fh, &iffile) ; if (strcmp(ifname, "forman .fb") != 0) { Fprintf (stderr, "%s: Error opening formant file\n"); exit(l) ; } else { Fprintf(stderr, "Input formant file is %s\n", ifname); } /* * *** * ***** Q et various data from files ** ** * ***** * *** * ** */ num_rec = ih->common.ndrec; /* get number of spectral data records */ Fprintf( stderr, "Number of spectra in input file is %ld\n", num_rec ); num_freqs = (int) *get_genhd_l ( "num_freqs", ih ) ; / ♦ Fprintf( stderr, "Number of freq's is %ld\n" , num_freqs); */ if (num_freqs < 257) { Fprintf( stderr, "%s: Exiting.\n", ProgName ); exit(l); } if (num_freqs == 513) { bigspec = True; interval = 2,- 9 3 } else interval = 1; /* get generic header items we need to copy to output file */ sample_rate = *get_genhd_f( "sf", ih ) ; /* SF of FEASPEC file */ /* Get values of header items from FEA file */ num_fearec = ifh->common.ndrec; /* no of feature file records */ Fprintf( stderr, "Number of records in feature file is %ld\n", num_fearec ); if (num_rec / num_fearec == 2) { halfnum_rec = num_fearec; } else { if (num_rec / (num_fearec-1) == 2) { halfnum_rec = num_fearec-1; one_val_short = True; } else { halfnum_rec = num_rec / 2,- one_val_short = True; } } /* do some mallocs etc */ if ( (d_data = (float *) calloc( num_rec, sizeof (float) )) == NULL ) { printf ("\n Memory allocation error; returning to workstation..." ) ; exit(l) ; } if ( (form_diff = (float *) calloc( num_fearec, sizeof (float) )) == NULL ) { printf("\n Memory allocation error; returning to workstation..." ) ; exit(l) ; } peaks = (int *) calloc( 2 * num_rec * F0RMANTS, sizeof (int) ) ; if ( !peaks) { Fprintf( stderr, "%s: Memory allocation error: returning to systemXn", ProgName) ; exit(1) ; ) /************ get formant data from formant file ************* * */ rval = get_formants ( fh, iffile, peaks ); fclose( iffile ) ; if ( rval != 0 ) { Fprintf ( stderr, "%s: Error in get_formants: exitingNn" , ProgName ) ; exit (1); } /* The above call assigns a value to global num_formrec */ /*if (num_formrec < num_rec/2) { 94 Fprintf ( stderr, "%s: Warning; More iijput spectra than formant records:", ProgName ) ; Fprintf ( stderr, "Formants=%ld; Spectra=%ld\n" , num_formrec, num_rec/2 ) ; } */ if (num_formrec % 2 == 0) factor_size = num_formrec/2; else factor_size = (num_formrec + l)/2; /* always even number */ if ( (factor = (float *) calloc( factor_size, sizeof (float) )) == NULL ) { printf("\n Memory allocation error; returning to workstation..." ) ; exit(1) ; } if (num_formrec < num_rec) { Fprintf( stderr, "%s: Warning; More spectral records than formant records:", ProgName ) ; Fprintf ( stderr, "Formants=%ld; Spectra=%ld\n" , num_formrec, num_rec ); } /* Create file for dumping curve data if ( (fp = fopen("dumpcurv.txt" , "wt")) == NULL ) { Fprintf(stderr, "Error opening dumpcurv.txt\n" ) ; exit(l) ; } */ rec_freq = get_genhd_d( "record_freq" , ifh ) ; src_sf = get_genhd_f ( "src_sf", ifh ); start_time = get_genhd_d( "start_time" , ifh ) ; start_rec = get_genhd_l ( "start", ifh ); Maxfrequency = (UINT) (sample_rate / 2); /* Freq. range = 1/2 sample rate */ Fprintf ( stderr, "Max frequency is %u\n", Maxfrequency ); fstep = Maxfrequency / (int) SPECSIZE; /* Declared in SPECTRUM.H */ temp = (float) (200.0 / fstep); startf = 5 /Mint) (ceil (temp) )*/; /* round up */ l_range = (float) loglO( (double) (Maxfrequency / 100) ); /* (usually) log (6000/100)=1.78 */ if (debug_level) Fprintf( stderr, "fstep = %d; startf = %d; l_range = %2.1f\n", fstep, startf, l_range) ; /************** calculate the formant weighted distance array ************/ Fprintf( ofile, "%s\n", get_cmd_line(argc, argv) ) ; Fprintf(ofile, "Formant weighting factors:\n") ; / ♦ for (ct=0; ct < (int) (2*num_formrec-1) ; ct++) */ /* This is for when no. of formant records = no. of spectra (ie frame length is 5ms for both) */ for (ct=0; ct < factor_size; ct++) { 9 5 temp = 0.0; pos = 2*ct; /* distance = 0; */ for(j=0; j<FORMANTS-l; j++) { /* use only firεt 3 formants */ / ♦ Fprintf(ofile, " [%d] %d\t", j, peaks [FORMENTRY(ct, j) ] ) ; ♦ / lower = peaks[FORMENTRY(pos, j) ] ; upper = peaks[FORMENTRY( (pos+1) , ) ] ; /* these brackets are CRUCIAL!!! | | */ temp += sq( (float) (upper-lower) ♦ 100 / lower); / ♦ convert to % ♦ / /* distance = distance + sq(temp) ; */ } factorfct] = (float) sqrt( (double) temp); Fprintf(ofile, "factor[%d]= %3.1f\n", ct, factor[ct] ) ; } Fprintf(ofile, "\n"); / ♦ normalise to range {0,1} ♦ / / ♦ factor[0] = 10; safe value ♦ / max_value = max( (specmaxl(factor, factor_size, &index, 120.0F)), 100.0 ) ; min_value = mini(factor, factor_size, &index) ; range = max_value - min_value; printf( "\n" ); /* in case of messages from specmaxl() */ scale_max = 100; scale_min = 0; */* */ printf ( "Formant factors: Scale Max=%4.1f; Scale min=%4.1f; Range=%4.If\n" max_value, min_value, range ) ; /* convert to (clipped) % to enhance the display */ for (j=0; j < factor_size; j++) { val = (factor[j]-min_value) / range; if (val > 1) val = 1.0; factorfj] = val; / ♦ Fprintf (ofile, " [%d] : %4.3f\t", j, factorfj]); ♦ / } factor[0] = 1.0; Fprintf (ofile, "\n"); / ♦ Allocate an feaspec record ♦ / rec = (struct feaspec *)allo_feaspec_rec( ih, FLOAT ) ; / ♦ get spec values and process them ♦ / /********.********..**,.*****..**.**.*.*****.♦,♦*,*****.****.***,****,* printf( "\nCalculating weighting factors:" ) ; NOT USED for now Weighting function gives: 0 @ loop=256, i.e. f=Maxfrequency 10 @ loop=split/fstep, i.e. f=240Hz 9 6 1 @ loop=0 to split-1, to minimise voicing contribution. split = 480/fstep; (should be 20 @ 12k, 30 @ 8k) if (debug_level) Fprintf( stderr, "split = %d\n" , split); for (loop=0; loop<split; loop++) { weight[loop] = 5.0; Fprintf( ofile, "%ld: %2.1f\t" , loop, weight[loop] ); } for (loop=0; loop<SPECS!ZE; loop++) { weight[loop] = 10 * loglO ( 10 ♦ (1 - (0.9 4 loop) /SPECSIZE) ) ; Fprintf( ofile, "%ld: %4.1f\t", loop, weight[loop] ) ; } printf ( " Done\n" ) ; / *** ** ****** . Start processing spectra * ** ** ****** * ******** * * */ /* Do a virtual log-log plot of spectrum for (f=startf; f<SPECSIZE; f++) { yamp = (float) (100.0 * (1 - (thiε_sample[f] /NO MALDB) ) ); { yamp = this_sample[f] ; ) xnew = (float) (loglO ( (double) (f * fstep)) - 2.0) * (float) (500.0 / l_range) ; ct = f-startf; curve[0] [ct] .x = curve[1] [ct] .x = (int) xnew; curve[0] [ct] .y = (int) yamp; { curve[0] [0] .y = spec[εtartf] curve[0] [1] .y = spec[startf+1] } } dump_curve( curve[0] ) ; */ **★*****************************+***************************★***★★* Main loop for calculating spectral difference printf( "XnProcessing spectra...\n" ) ; /* xpos = wherexU; ypos = whereyO; */ last = num_rec-l; for (loop=0; loop < last; loop+=2) { printf ( "%2d: " , loop+1) ; get_feaspec_rec(rec, ih, ifile) ; /* get 1st spectrum */ this_avg=0.0; for (j=startf; j<SPECSIZE; j++) { /* SPECSIZE times in all */ /* get_specvals ( rec->re_spec_val, this_sample ) ; */ this_sample[ ] = rec->re_εpec_val[j ♦ interval + 1] ; / ♦ = specarray[ (ct^SPECSIZE) + j-1] = spec_val; 9 7 Fprintf( ofile, "Spec[%d] [%d] = %f\n", ct, j, sρec_val ) ; ♦/ } this_pwr = * (rec->tot_power) ; / ♦ If next line used, don't forget to open file first: dump_spec( this_εpec ); ♦ / distance = 0.0; next_avg=0.0; / ♦ get next spectrum from input file ♦ / get_feaspec_rec(rec, ih, ifile) ; / ♦ get 2nd spectrum ♦ / / ♦ get_specvals ( rec->re_εpec_val, next_sample ); ♦ / for (j=startf; j<SPECSIZE; j++) { next_sample[j] = rec->re_spec_val [ ♦ interval + 1] ; distance+= (sq(thiε_sample[ ] - next_sample[ ] ) ) / ♦ ♦ weight[j] ♦ /; } next_pwr = ♦ (rec->tot_power) ; / ♦ dump_curve( curve[1] ) ; ♦ / / ♦ printf ( "%5.1f\t", distance ); ♦ / / ♦ now multiply by the larger of the 2 power values */ d_data[loop/2 + 1] = distance * (next_pwr > 1000); / ♦ silence threshold subject to checking ♦ / / ♦ (float) (max(this_pwr, next_pwr) ) ♦ / / ♦ thiε_avg = next_avg; ♦ / } printf( "\n" ) ; / ♦ normalise the values ♦ / d_data[0] = 1000; / ♦ safe value ♦ / /* in call to specmaxl, the limit value depends on the step size: for 5ms, use 3.0e4; for 10ms use 6.0e4 ♦ / / ♦ we don't want the max value to be lesε than 20,000, otherwise too many peakε: for step-size = 50ms use 10,000 ♦ / / ♦ Min_SD_max = 1.0e ; ♦ / max_value = max( (specmaxl(d_data, laεt, &index, 3.0E4F)), Min_SD_max) ; min_value = minl(d_data, last, &index) ; range = max_value - min_value; printf( "\n" ) ; / ♦ in case of messages from specmaxl() */ scale_max = 100; scale_min = 0; / ♦ ♦ / printf ( "Spec Diff: Scale Max=%4.1f; Scale min=%4.1f; Range=%4.If\n" , max_value, min_value, range ) ; / ♦ convert to (clipped) % to enhance the display ♦ / for (j=0; j < last; j++) { val = (d_data[ ] -min_value) ♦ 100.0 / range; if (val > 100) val = 100.0; d_data[j] = val; } 9 8 d_data[0] = scale_min; /* ♦ * ♦♦ ** ♦♦♦ write data to output text file *************/ for (loop=0; loop < (num_rec) ; loop++) { Fprintf ( ofile, "Diff[%ld] = %5.1f;\t", loop, d_data[loop] ) ; } for (j=0; j < (int) (halfnum_rec) ; j++) { form_diff[j] = d_data[j] * factor!j]; } if (one_val_short) { form_diff[num_fearec-1] = d_data[num_fearec-1] * factor[num_fearec-1] ; } /* ♦♦ *** ♦ * ♦ Create the new output Feature file **************/ oh = new_header(FT_FEA) ; oh->common.tag = YES; (void) strcpy(oh->common.prog, ProgName); (void) strcpy(oh->common.verε, Version); (void) strcpy(oh->common.progdate. Date); oh->variable.refer = ifea_name; add_source_file(oh, ifea_name, ifh); /* add input fea file as src */ add_source_file(oh, iname, ih) ; /* add input FEA_SPEC file aε source */ add_comment(oh, get_cmd_line(argc, argv) ) ; /* Add essential generic header items */ (void) add_genhd_d( "record_freq" , rec_freq, 1, oh ) ,* /* Double */ (void) add_genhd_f( "src_sf", src_sf, 1, oh ) ; /* Float */ (void) add_genhd_d( "start_time" , εtart_time, 1, oh ) ; /* Double */ (void) add_genhd_l ( "εtart", start_rec, 1, oh ); / ♦ Long */ if (copy_fea_fld(oh, ifh, "power") == -1) / ♦ trouble ♦ / exit(l) ; if (copy_fea_fld(oh, ifh, "zero_crossing") == -1) exit(l) ; / ♦ Add an extra field to output FEA file ♦ / if ( add_fea_fld("spec_diff", IL, 1, (long ♦ ) NULL, FLOAT, (char ♦♦ ) NULL, oh) == -1) { Fprintf(stderr, "Trouble adding field to output file\n"); exit(l) ; } / ♦ Add another field to output FEA file ♦ / if ( add_fea_fld("factor", IL, 1, (long *) NULL, FLOAT, (char **) NULL, oh) == -1) { Fprintf(stderr, "Trouble adding field to output file\n"); exit(l); } /* Add 3 extra fields for the formants FI, 2 &3 ♦ / if ( add_fea_fld("Fl", IL, 1, (long ♦ ) NULL, SHORT, (char ♦*) NULL, oh) == -1) { 9 9 Fprintf(stderr, "Trouble adding field' to output file\n"); exit(l) ; } if ( add_fea_fld("F2", IL, 1, (long *) NULL, SHORT, (char * ♦ ) NULL, oh) == -1) { Fprintf (stderr, "Trouble adding field to output file\n"); exit(l) ; } if ( add_fea_fld("F3", IL, 1, (long ♦ ) NULL, SHORT, (char ♦♦ ) NULL, oh) == -1) { Fprintf(stderr, "Trouble adding field to output file\n"); exit(l) ; } write_header( oh, ofea_file ) ; p = allo_fea_rec(ifh) ,- q = allo_fea_rec(oh) ; /* Allocate pointers for copying data ♦ / pptr_in = (float ♦ )get_fea_ptr( p, "power" ,ifh ) ; zptr_in = (float ♦ )get_fea_ptr( p, "zero_crossing" , ifh ) ; pptr_out = (float ♦ )get_fea_ptr( q, "power",oh ); zptr_out = (float ♦ )get_fea_ptr( q, "zero_crossing" ,oh ) ; /* now add the data for the spec_diff field */ εpec_ptr = (float *) get_fea_ptr(q, "spec_diff", oh); dist_ptr = (float *) get_fea_ptr(q, "factor", oh); Fl_ptr = (short *) get_fea_ptr(q, "FI", oh); F2_ptr = (short ♦ ) get_fea_ptr(q, "F2", oh); F3_ptr = (short ♦ ) get_fea_ptr(q, "F3", oh) ; /* = q->spec_diff ♦ / for (ct=0; ct < min(num_fearec, factor_size) ; ct++) { if ( get_fea_rec(p, ifh, ifea_file) != EOF) { ♦ pptr_out = ♦ pptr_in; ♦ zptr_out = ♦ zptr_in,- ♦ spec_ptr = d_data[ct]; ♦ dist_ptr = factor[ct]; ♦ Fl_ptr = peaks [FORMENTRY( (2 ♦ ct) , 0)] ♦ F2_ptr = peaks[FORMENTRY( (2 ♦ ct) , 1)] ♦ F3_ptr = peaks [FORMENTR ( (2 ♦ ct) , 2)] / ♦ purely formant-based difference ♦ / q->tag = p->tag; put_fea_rec(q, oh, ofea_file) ,- } } Fprintf( stderr, "OK; %ld records copied\n", ct) ; fclose(ifile) ,• 1 0 0 fclose(ofile) ; fclose(ifea_file) ; fclose(ofea_file) ; free( d_data ) ; } / ♦ End of function main ♦ / ♦define INF 1.0E30F float εpecmaxl ( float ♦ data, long ε, long ♦ index, float limit ) { float maximum = data[0]; / ♦ limit = 3.0E4 or 6.0e4; ignore values above this ♦ / long i; short bad_ct = 0; for (i = 1; i < s; i++) { if ( data[i] < limit ) { if (data[i] > maximum) { maximum = data[i]; ♦ index = i; } } else bad_ct++; } Fprintf(stderr, "Specmaxl: %hd bad values found. ", bad_ct); return maximum; ♦define avg(x, y) ( (x + y) / 2) int get_formants( struct header ♦ ifh, FILE *iffile, int ♦ pea^set ) / ♦ Reads formant values from ESPS file input.fb, created by formant program */ { char *param_file = NULL; / ♦ parameter file name */ FILE *ofile = stdout; char /* input and output file names */ ♦ oname = "formant.txt"; εtruct fea_data *formrec; / ♦ record for fea data */ double forms[FORMANTS] ; /* array for current F values ♦ / double *formptr; / ♦ pointer to formant values ♦ / register int j, ct; short row, last, next; if ((ofile = fopen(oname, "wt")) «= 0) { Fprintf( stderr, "get_formants: Error opening output file\n" ); 1 0 1 return ( -1 ) ; } num_formrec = ifh->common.ndrec; / ♦ get number of data records ♦/ Fprintf( stderr, "Number of formant records is %ld\n", num_formrec ); / ♦ Allocate an fea record ♦ / formrec = (struct fea_data *)allo_fea_rec( ifh ); formptr = (double ♦ )get_fea_ptr( formrec, "f ", ifh ); / ♦ get formant values and process them ♦ / ct=0; / ♦ Initialise loop counter*/ Fprintf( ofile, "Formant data from formant.fb\n" ) ; while ( get_fea_rec(formrec, ifh, iffile) != EOF ) { for (j=0; j<FORMANTS; j++) { forms [j] = formptrfj]; peak_set[FORMENTRY(ct, j)] = (int) (forms [j]) ; / ♦ Fprintf ( ofile, "F%d=%4.11f; ", j, forms[j] ) ; */ } Ct++; } /* fill in the gapε in the peak array by interpolation: don't need this now */ / ♦ for (ct=0; ct< num_formrec-1; ct++) { ♦ 1, 3, 5.... ♦ row = ct*2+l; last = ct*2; next = 2* (ct+1) ; for (j=0; j<FORMANTS; j++) peak_set[FORMENTRY(row, j)] = avg(peak_set[FORMENTRY(last, ) ] peak_set[FORMENTRY(next, j) ] ) ; ) */ / ♦ 2n-lth (ie laεt) value: a copy of the previous values row = 2*num_formrec-l; laεt = 2*num_formrec-2,* for (j=0; j<FORMANTS; j++) peak_εet[FORMENTRY(row, j)] = peak_εet[FORMENTRY(last, j) ] ; */ /* dump formant valueε to output file: formant.txt */ for (ct=0; ct< 2*num_formrec; ct++) { Fprintf( ofile, "Record[%d] : ", ct ) ; for (j=0; j<FORMANTS; j++) Fprintf ( ofile, "F%d=%d; ", j, peak_εet[FORMENTRY(ct, j)] ) ; Fprintf(ofile, "\n"); } fclose(ofile) ; return(0) ; } 1 0 2 void get_specvals( float ♦ source, float ♦ dest ; /* read specific frequency vals from source εpectrum, to approximate a logarithmic spectrum */ { register int j ; for (j=0; j<151; j++) /* takes uε up to 1757Hz §εf=12kHz */ dest[ ]=source[j] ; for (j=151; j<200; j++) dest[j]=source[2* (j-151) ] ; for (j=200; j<SPECSIZE; j++) dest[j]=source[5* (j-200) ] ; } void dump_curve ( point curv[SPECSIZE] ) { register int j; FILE ♦ fp; if ( (fp = fopenCdumpcurv.txt", "at")) != NULL ) { for (j=0; j<SPECSIZE; j++) fprintf(fp, "%d; %d\t", curv[j].x, curv[j].y); fprintf(fp, "\n"); fclose( fp ) ; } else { Fprintf(stderr, "Error opening dumpcurv.txt\n" ) ; exit(l) ; } } int matherr( struct exception *a ) { FILE ♦ erf; if ( a->type == DOMAIN ) { if ( strcmp( a->name, "εqrt" ) == 0 ) { a->retval = εqrt ( - ( a->argl ) ) ; return (1) ; ) else { if ( εtrcmp( a->name, "cos" ) == 0 ) { a->retval = cos(O); printf( " \tCosine domain error\n" ); return (1) ; } else { /* Must be sine error?? */ 1 0 3 a->retval = sin( PI/2 ); printf ( "\tUnknown domain error\n" ); return (-1); / ♦ This WILL halt calculation ♦ / } } } erf = fopen( debugfilename, "a+t" ); fprintf ( erf, "Maths error type was: " ) ; εwitch ( a->type ) { caεe DOMAIN: fprintf ( erf, "DOMAIN\n" ) ; break; case SING: fprintf ( erf, "SING\n" ) ; break; caεe OVERFLOW: fprintf ( erf, "OVERFLOWXn" ); break; caεe UNDERFLOW: fprintf ( erf, "UNDERFLOW\n" ); break; caεe TLOSS: fprintf( erf, "TLOSS\n" ) ; break; default: fprintf ( erf, "Unknown\n" ); } / ♦ end of switch statement ♦ / fclose( erf ) ; return (1) ,* / ♦ Some other error: thiε will not halt calculation in calculate_spec() ♦ / 1 0 4 / ♦ GET_SYL2 . C by Geoff Williams Processes an input FEA file containing power, zero-croεsing and spectral difference data, and outputs a (text) file containing boundaries of regions with their constituent type: eg: 'cat': 0-5:Onset(1) ; 6-15:Nucleus(l) ; 15-20:Onset(1) ; end:Nucleus (0) ; where (1)= non-branching; (0)=empty; (2)=branching Now combines with parse_word.c and findelem.c to produce rec_word. * / ♦ifndef lint static char ♦ sccs_id " @ (#)get_syll.c 1.0 XX/XX/XX HAL"; ♦endif char *Version = "2.00"; char *Date = "XX/XX/XX"; ♦include <stdio.h> ♦include <stdlib.h> ♦include <string.h> ♦include <ctype.h> ♦include <math.h> ♦include <esps/esps.h> ♦include <espε/unix.h> ♦include <espε/limits.h> ♦include <eεpε/feasd.h> ♦include <esps/fea.h> ♦include <esps/feaspec.h> ♦include "include/util.h" char ♦ ProgName = "rec_word"; ♦define max(x, y) ( (x>y) ? x : y) ♦define min(x, y) ( (x<y) ? x : y) ♦define between(x, y, z) ( ( (x>=y) && (x<=z) ) ? 1 : 0) ♦define SYNTAX \ USAGE( "rec_word [- (start value) -e(end value) -r(range)] [-1 language] input.fea ouput.txt" ) ♦define SPECSIZE 256 / ♦ as in SPECTRUM.H ♦ / ♦define PSIZE 500 ♦define FORMANTS 4 ♦define MAXPEAKS 20 / ♦ max number of peaks in spec_diff data ♦ / ♦define NAMESIZE 10 / ♦ length of names for regions of signal */ .1 0 5 ♦define MAXREGIONS (MAXPEAKS-1) /* max number of regions bounded by peaks ♦ / ♦define True 1 ♦define False 0 ♦define FINAL 100 / ♦ enum voice_types { UNV, TRANSITION, VCD } ; */ typedef unsigned int UINT; typedef UINT BOOLEAN; typedef struct myfeature { float zeros; float d_data; float form_diff; float power; float midpower; float kl; float voiced; / ♦ gets converted to value in the set voice_typeε ♦ / float F0; } Feature; typedef struct yspec { float amp[SPECSIZE] ; } Spectrum; Globals long num_rec, /* no of data records to read ♦ / num_frec, /* no of formant recordε */ num_srec; /* no of spectral records */ int debug_level = 1; int Spec_to_fea_ratio = 2; /* ratio of spectral records to feature records */ int Step_size = 10; /* ms */ int diff_peaks[MAXPEAKS] ,* /* holds positions of good peaks in d_data */ /* actual values at each peak held in diff_peakvals [] */ short num_bounds, /* No of peaks in specdiff data */ num_regionε; char ♦ language; /* language name, used by parεe_seg.c ♦ / int max_pwr_pos, max_mid_pos, max_vmid_pos; float maxzeros, minzeroε, maxpwr_mid, minpwr_mi , maxvoice_mid, axpower, minpower; / ♦ point curve[2] [SPECSIZE] ,- Storeε curve co-ordinates ♦ / 1 0 6 float SampRate = 12000, l_range; /* = Maxfrequency/STARTFREQUENCY ♦ / /* Parameters relating to Constituent structure */ BOOLEAN BR_ONSET, BR_NUCLEUS, BR_RHYME, DF_LEN; / ♦ Language-specific details: not parameters ♦ / BOOLEAN HV_DEVOICED = False, SON_STOP = False, BRN_IS_HEADED = False, / ♦ True if BR Nuclei must be headed as in English */ NN_SEQUENCE = False; /* True if language allows pseudo-long vowels */ Spectrum ♦ spec_array; / ♦ array for spectral data ♦ / ************ * ** ** ** * external functions ** ** ** ** **** ** * * * ** ** . char ♦ eopen() ; void lrange_switch() ; /* in parse_seg.c char ♦ get_next_token( char ♦ word, int ct ) ; char ♦ build_onset( char ♦ seg, char ♦ seg_str, int syl_ct ); char *build_rhyme( char *seg, char ♦ εeg_str, int syl_ct ) ; char ♦ parse_seg_str( char ♦ seg_str, char lang ) ; /* local functions */ int load_spectral_data( struct header ♦ ih, FILE ♦ ifile, int step, Spectrum ♦ spec_ar ay ) ; int get_for antε ( struct header *ifh, FILE ♦ iffile, int *peak_set ); float getvmax( float ♦ pwr2, int sl_array_size. Feature ♦ fea_set, int *index ); float find_riεe( float *parray, long start, int *offset ) ; void find_slopes ( long size, float ♦ parray, float ♦ sl_array ) ; short get_peaks ( float ♦ data, float *p_data, float *zeroε, int num_valε ); int get_lstmax( float *data, int start, int end ) ; int get_next_peak( float ♦ data, int start, int end, float ♦ pk.value ) ; int insert_peak( int ♦ peaks, int pos, int val ); / ♦ parse_word.c ♦ / float get_mean( float ♦ array, int start, int end ) ; short is_low( float ♦ array, int start, int end, float uplim ) ; 1 0 7 εhort is_high( float *array, int start, mt end ); int find_NUC( int start, int rest, UINT streεs, int Imaxpos, int maxmidpos, float ♦ tot_pwr, float ♦ mid_pwr, float *d_data, float *form_fac, short ♦ las^reg ) ; int find_SON( int start, int rest, int speaker, int lmax_poε, float ♦ tot_pwr, float ♦ mid_pwr, float ♦ d_data, float ♦ form_fac, short 4 nuc_end ) ; char ♦ parse_voiced( int reg_num, int start, int end, Feature *fea_set, int ♦ uai, int ♦ peaks, short 4 prev ) ; int find_FRIC( int start, int rest, float ♦ zcr, float ♦ kl, short ♦ fric_type ) ; int find_SIL( int start, int rest, float ♦ power, float ♦ zeroε, float *mid_pwr, float *kl ) ; char parse_unvoiced( int reg_num, int start, int end, Feature *fea_set, short ♦ prev, char ♦ seg_str ) ; int check_silence( int start, mt end. Feature *fea_set ); char *find_regions ( float *diffpeakε. Feature ♦ fe,a_set, char names[] [NAMESIZE] , int w uai, int ♦ peaks ) ; ********************************* M ATW *******************************/ main(int argc, char **argv) { /* char *param_file = NULL; parameter file name */ FILE *fp, *ifile = stdin, /* input and output file streams ♦ / ♦ ofile = stdout, ♦ isfile, /* spectral data file pointer ♦ / ♦ iffile; / ♦ formant file stream */ struct header /* input, output and formant file headers */ ♦ ih, ♦ ish, ♦ ifh; static char ♦ ofile_dir = "/usr/bin/HAL/ipl/progε/data/" , ♦ ofile_name /* = default,dat" ♦ /; char *iname, /* input and output file names */ *oname, ♦ ifname, ♦ spec_fname / ♦ = "rcspec.fea" ♦ /, ♦ out_εtr, ♦ temp_str, name_list[MAXREGIONS] [NAMESIZE] ; / ♦ holds names of regions of signal ♦ / struct fea_data Tec; / ♦ record for fea data */ Feature *fea_set, /* structure for my acoustic features */ ♦ mfp; / ♦ pointer used in loop ♦ / double 4 pwr2ptr, /* pointer to pwr2 field in FEA file ♦ / ♦ kl_ptr, 10P *voice_ptr, ♦ pitch_ptr; float ♦ pptr, *zptr, / ♦ pointers to power and zero__croεsing fieldε ♦ / *diff_ptr, *fact_ptr; float ♦ power, *powermid; / ♦ Arrays for power values ♦ / float *sl_array, /* slope " */ ♦ power_100, ♦ pwr2_100, ♦ zeros, *zeros_100 /* Arrays for zcr values ♦ / ♦ d_data, / Array for spec diff data ♦ / ♦ diff_peakvals; / ♦ holdε values of peaks in d_data */ float / ♦ maxpower, minpower, Now Globals maxpwr_mid, minpwr_mid, */ maxslope, minslope, risetime, tempval, voice_val, power_range, pwr_mid_range, zeros_range; char ♦ prange = NULL; / ♦ string for range specification (-p) ♦ / long start, end, /* boundary points for search */ nan, sl_array_size, /* size of sl_array ♦ / j, ct; /* loop counter ♦ / long maxdex, index, / ♦ index of max/min values in array ♦ / maxzc_dex, minzc_dex; /* " " " in zc array */ extern int optind; extern char ♦ optarg; int offεet, step = 1; register int i; int ch, pos, rval, ♦ uai, / ♦ Uεed aε uai[num_rec] ♦ / ♦ peakε; / ♦ holds formant valueε: [num_rec] [4] */ short / ♦ num_bounds, No of peaks in specdiff data: GLOBAL */ pflag = 0, / ♦ flag for range (pts.) option 4 / bflg=0, eflg=0, l_flag=0, Params[4]; / ♦ holds P-params */ /* extra vars for loading spec data float spec_val, y_scale; short interval; long num_freqs; struct feaspec *srec;*/ oname = (char *)calloc( 30, sizeof (char) ) ; iname = (char ♦ )calloc( 80, sizeof(char) ) ; 1 0 9 while ( (ch = getopt(argc, argv, "b:e:r:l:")) != EOF ) { switch (ch) { case ' ' : start = (long) atoi (optarg) ; printf ("start value = %ld\n", start); if (start < 0) bflg++; break; case 'e' : end = (long) atoi (optarg) ; printf("end value = %ld\n", end) ; if (end < start) eflg++; break; case 'r' : prange = optarg; pflag++; break; case ' 1 ' : language = optarg; l_flag++; break; default: SYNTAX; break; } ) if (bflg) { Fprintf(stderr, "Error: start value muεt be >0\n"),* exit(1) ,* } else start = 0,- if (eflg) { Fprintf(stderr, "Error: end value must be > start\n"); exit(l) ; } if (pflag) { start = 1; end = LONG_MAX; lrange_switch(prange, istart, &end, 0); nan = (end == LONG_MAX) ? 0 : end - εtart + 1; /* data_start_time = file_start_time + (εtart - 1) / sf; ♦ / } if (optind < argc) iname = eopen(ProgName, argv[optind++] , "r", FT_FEA, NONE, &ih, &ifile) ; else { 1 1 0 Fprintf (stderr, "%s: no input feature file specified. \n" , ProgName); SYNTAX; } if (debug_level) Fprintf(stderr, "Input file is %s\n", iname); if (optind < argc) { if ((ofile = fopen(argv[optind] , "wt")) != 0) oname = argv[optind++] ; ) else { Fprintf(stderr, "%ε: no output file εpecified. ", ProgName); Fprintf(stderr, "Using stdout\n"); ofile = stdout; oname = "<εtdout>"; } if (debug_level) Fprintf(stderr, "Output file is %s\n", oname) ; /* Open formant file, always formant.fb ♦ / ifname = eopen(ProgName, "formant.fb" , "r", FT_FEA, NONE, &ifh, &iffile) ; if (strcmp(ifname, "formant.fb") !=0) { Fprintf(stderr, "%s: Error opening formant file\n", ifname); exit(l); } else { Fprintf(stderr, "Input formant file is %s\n", ifname); } / ♦ get number of data records ♦ / num_rec = ih->common.ndrec; Fprintf(εtderr, "Number of feature records iε %ld\n" , num_rec) ; if (!eflg) end = num_rec-l; num_frec = ifh->common.ndrec; / ♦ get number of formant recordε */ Fprintf(stderr, "Number of formant records is %ld\n", num_frec) ,* /* Assign P-parameters and output file name based on language name */ ofile_name = (char *)calloc(48, sizeof(char) ) ; strcpy( ofile__name, ofile_dir ) ; if (l_flag) { switch (language[0] ) { case 'J' : strcatt ofile_name, "Jap.dat" ) ; for (j=0; j<4; j++) Params[ ] = 0; HV_DEVOICED = True; NN_SEQUENCE = True; 1 1 1 SON-STOP = True; /* assuming r containε ? ♦ / break; case 'j ' : strcat( ofile_name, "japanese.dat" ) ; for (j=0; j<4; j++) Para ε[j] = 0; HV_DEVOICED = True; NN_SEQUENCE = True; SON_STOP = True; /* aεsuming r contains ? ♦ / break; case 'K' : case 'k' : strcat( ofile_name, "korean.dat" ); for (j=0; j<4; j++) Params [j] = 0,* SON_STOP = True; break; strcat( ofile_name, "default.dat" ); for (j=0; j<4; j++) Params[ ] = 1; SON_STOP = True; BRN_IS_HEADED = True; break; caεe 'G' : caεe 'g' : strcat ( ofile_name, "greek.dat" ) ; for (j=0; j<4; j++) Paramε[j] = 1; SON_STOP = True; break; default: εtrcat( ofile_name, "default.dat" ); } / ♦ end of εwitch ♦ / ) else { / ♦ default case, no -1 flag used ♦ / strcat( ofile_name, "default.dat" ); language = "English"; for (j=0; j<4; j++) Params [ ] = 1; BRN_IS_HEADED = True; SON_STOP = True; } if (debug_level) 1 1 2 Fprintf (stderr, "Output language file is %ε\n", ofile_name ); BR_ONSET = Params [0] ; BR_NUCLEUS = Params[1] ; BR_RHYME = Params [2] ; DF_LEN = Params[3]; / ♦ Do some mallocs ♦ / uai = (int *) calloc( num_rec, sizeof(int) ) ; / ♦ use num_rec instead of num_frec in above as it will always be greater ♦ / /* make fea_set into an array of structures ♦ / if ( (fea_set = (Feature ♦ )calloc(num_rec, sizeof(Feature) ) ) == 0) { Fprintf (stderr, "%s: Memory allocation error; exiting.", ProgName); exit (1) ,- } if ( (power = (float ♦ )malloc(num_rec * sizeof (float)) ) == 0) { Fprintf (stderr, "%s: Memory allocation error; exiting.", ProgName); exit (1) ; } if ( (powermid = (float ♦ Jmalloc(num_rec * sizeof(float) ) ) == 0) { Fprintf(stderr, "%s: Memory allocation error; exiting.", ProgName); exit (1); } if ( (zeros = (float *Jmalloc(num_rec * sizeo (float) ) ) == 0) { Fprintf (εtderr, "%s: Memory allocation error; exiting.", ProgName) ; exit (1); } if ( (d_data = (float *(rnalloc(num_rec * sizeof (float)) ) == 0) { Fprintf (εtderr, "%s: Memory allocation error; exiting.", ProgName); exit (1); } /* Open spectral data file */ spec_fname = eopen(ProgName, "rcspec.fea" , "r", FT_FEA, FEA_SPEC, &ish, &isfile) ; if ( strcmp(spec_fname, "rcspec.fea") !=0 ) { Fprintf (stderr, "%s: Error opening FEA_SPEC file\n", spec_fname) ; retur (-1) ; } else { Fprintf(stderr, "\tlnput spectral data file is %ε\n", spec_fname) ; ) num_srec = ish->common.ndrec; /* get number of spectral records */ Fprintf(stderr, "Number of spectral records is %ld\n", num_srec) ; if ( (spec_array = (Spectrum *) calloc(num_srec, sizeof (Spectrum) ) ) == NULL) { Fprintf( stderr, "%s: Memory allocation error: returning to system\n" , 1 1 3 mfp->zeros = zeros[ct] = ♦ zptr; power[ct] = ♦ pptr; powermid[ct] = (float) 4 pwr2ptr; d_data[ct] = mfρ->d_data = *diff_ptr; mfρ->form_diff = ♦ fact_ptr; mfp->kl = (float) ♦ kl_ptr; mfp->F0 = (float) *pitch_ptr; voice_val = (float) ♦ voice_ptr; mfp->voiced = voice_val; / ♦ labels now assigned in find_regions mfp++; Ct++; } maxpower = maxl ( power, num_rec, &maxdex ) ; minpower = mini ( power, num_rec, &mindex ); maxzeroε = maxl( zeroε, num_rec, &maxzc_dex ) ; minzeros = mini ( zeroε, num_rec, &minzc_dex ) ; Fprintf( ofile, "Maximum power value = %f at index %ld\n", maxpower, maxdex ) ; Fprintf( ofile, "Minimum power value = %f at index %ld\n", minpower, index ) ; if (end > num_rec) { end = num_rec; Fprintf (εtderr, "Warning: 'end' reduced to equal no of records (%ld)\n", num_rec) ; } / ♦ Calculate slopes in selected region ♦ / if (pflag) sl_array_size=nan; elεe sl_array_size = (end-start+1) ; Fprintf(stderr, "Number of records selected is %ld\n", sl_array_size) ; sl_array = (float ♦ Jmalloc(sl_array_size * sizeof(float) ) ; power_100 = (float *)malloc(sl_array_size * sizeof(float) ) ; pwr2_100 = (float *)malloc(sl_array_size * sizeof(float) ) ; /* zeros_100 = (float *)malloc(sl_array_size * sizeof(float) ) ; */ /* Copy power and zc values for selected region into new arrays ♦ / for (j=start; j < end+1; j++) { power_100[ -start] = power[j]; pwr2_100[ -start] = powermid[ ] ; } maxpower = maxl( power_100, sl_array_size, &max_pwr_pos ) ; / ♦ local max */ minpower = mini ( power_100, sl_array_εize, &mindex ) ; / ♦ local min ♦ / power_range = maxpower-minpower; maxpwr_mid = maxl( pwr2_100, sl_array_size, imax_mid_pos ); / ♦ local max */ maxvoice_mid = getvmax( pwr2_100, sl_array_size, fea_set, &max_vmid_pos ) /* max pwr in voiced regions */ 1 t < ProgName) ; exit (1) ; ) rval = load_spectral_data( ish, isfile, εtep, εpec_array ) ; fclose( isfile ) ; if ( rval != 0 ) { Fprintf ( stderr, "%s: Error in load_εpectral_data: exiting\n", ProgName ); exit (1); } / ♦ allocate memory for peaks array: 1 entry per spectrum ♦ / if ( (peaks = (int *) calloc( (num_srec ♦ FORMANTS) , sizeof(int)) ) == NULL) { Fprintf ( stderr, "%s: Memory allocation error: returning to system\n", ProgName) ; exit(l) ; ) /* Load in formant data ♦ / rval = get_formants( ifh, iffile, peaks ); / ♦ This assigns value of global num_frec ♦ / fclose( iffile ) ,- if ( rval != 0 ) { Fprintf( stderr, "%s: Error in get_formants: exiting\n" , ProgName ); exit (1) ,- } if (num_frec < num_rec) { Fprintf( stderr, "%s: Warning; More feature records than formant records:", ProgName ) ; Fprintf ( εtderr, "Formants=%ld; Featureε=%ld\n" , num_frec, num_rec ); } / ♦ Allocate an FEA record ♦ / rec = allo_fea_rec( ih ) ; / ♦ get a pointer to the data for "power" etc ♦ / pwr2ptr = (double ♦ )get_fea_ptr( rec, "pwr2" , ih ) ; pptr = (float ♦ )get_fea_ptr( rec, "power", ih ); zptr = (float ♦ )get_fea_ptr( rec, "zero_croεsing" ,ih ) ; diff_ptr *■** (float ♦ )get_fea_ptr( rec, "εpec_diff" ,ih ),* fact_ptr = (float *)get_fea_ptr( rec, "factor" ,ih ) ; kl_ptr = (double ♦ )get_fea_ptr( rec,"kl",ih ); voice_ptr = (double ♦ )get_fea_ptr( rec, "prob_voice" , ih ); pitch_ptr = (double ♦ )get_fea_ptr( rec,"F0",ih ); / ♦ get feature values from input FEA file ♦ / Fprintf( ofile, "Command line text: %80s\n" , (char *)get_cmd_line(argc, argv) ) ; ct=0; /*Initialise loop counter*/ mfp = fea_set; while ( get_fea_rec(rec, ih, ifile) != EOF /*&& (ct < end) */) { 1 1 5 minpwr_mid = mini ( pwr2_100, sl_array_sιze, &mindex ) ; /* local min ♦ / pwr_mid_range = maxpwr_mid - minpwr_mid; if (debug_level) { Fprintf(stderr, "Local power range = %f\n", power_range) ; Fprintf (stderr, "Local mid power range = %f\n", pwr_mid_range) ; Fprintf(stderr, "Max mid-pwr (voiced) = %5.1f @ pos %d\n" , maxvoice_mid, max_vmid_poε) ; } if ( (power_range <= 0.0) | | (pwr_mid_range <=0.0)) exit(l) ; / ♦ Normalize the power and zero_croεsing valueε ♦ / mfp = fea_set; for (j=0; j < sl_array_size; j++) { mfp->power = power_100[ ] ; / ♦ store Absolute power in fea_set ♦ / mfp->midpower = pwr2_100[j] ,* / ♦ abεolute mid-range power ♦ / Fprintf ( ofile, "Power[%d]=%3.If; Mid Power[%d]=%3.If; Zeros[%d]=%3.If; D_data[%d]=%3.1f; ", j, power_100[j] , j, pwr2_l00[j] , j, zeroε[j], j, mfp->d_data) ; Fprintf( ofile, "Form_diff[%d]=%3.2f; Kl[%d]=%3.If; F0[%d]=%3.If; voiced[%d]=%2.1f\n", j, mfp->form_diff, j, mfp->kl, j, mfp->F0, j, mfp->voiced ); / ♦ ncrπ-aliεe the power arrays ♦ / power_10C I ] = (power_l00 [j]-minpower) /power_range ♦ 100; pwr2_100[j] = (pwr2_100 [j]-minpwr_mid) /pwr_mid_range ♦ 100; mfp++; ) / ♦ Get peaks in specdiff array: use total power here for checking for valid peaks */ num_bounds = get_peakε ( d_data, power_100, zeroε, (int) num_rec ) ; Fprintf (stderr, "Number of peaks found = %hd\n", num_bounds) ; /* Tidy up peak array: miεsing 1st peak: */ if (diff_peakε [0] > 10) { /* should not be more than 100ms into file */ pos = get_lεtmax( d_data, 0, diff_peaks[0] ) ; if ( insert_peak( diff_peaks, 0, pos ) == 0 ) { /* Insert value pos at peak 0 (2nd value in array */ Fprintf ( stderr, "Peak 0 added at position %d\n" , pos ) ; num_bounds++; } } /* missing final peak */ if (diff_peaks[num_bounds-l] < (num_rec-10) ) { /* not >100ms from end */ diff_peak [num_boundε] = num_rec-5; /* any old value, 50mε from end (should really find 1st peak starting from right )*/ num_bounds++; Fprintf( stderr, "Final Peak added\n" ) ; } 1 1 6 if (debug_level ) Fprintf(stderr, "Dumping peak positions to output file\n"); diff_peakvals = (float *)malloc( num_bounds ♦ sizeof (float) ) ; for (i=0; i<num_bounds; i++) { /* tempval = ; ♦ / diff_peakvals[i] = d_data[ (diff_peaks[i] ) ] ; Fprintf( ofile, "Diff_peakε[%d] at pos %d; d_data=%3.If\n" , i, diff_peaks[i] , diff_peakvals[i] ) ; ) if (debug_level) Fprintf(stderr, "Done\n"); num_regions = num_bounds - 1; / ♦ fclose( ofile ) ; ♦ / out_εtr = (char ♦ )calloc(64, sizeof(char) ) ; temp_str = (char ♦ )calloc(20, sizeof(char) ) ; temp_str = find_regions ( diff_peakvalε, fea_set, name_list, uai, peaks ); if (debug_level) Fprintf(stderr, "Back in main: segment string is %20s\n", temp_str) ; if (strstrf temp_str, "FAIL" ) != NULL) { Fprintf(stderr, "Find_regions failed: quitting!! \n") ; fcloεe(ifile) ; fcloεe(ofile) ; return 1; } out_str = parεe_seg_str( temp_εtr, language[0] ) ; printf ( "\nSearch string is %s\n" , out_str ); if ( (fp = fopen(ofile_name, "wt")) != NULL ) { fprintf( fp, "%s", out_str ); fclose( fp ) ; } / ♦ Use Absolute power, and ABSOLUTE zeros! ! ! ! ! watch the index of power if start != 0 if ((ofile = fopen(oname, "a+t")) != 0) { for (i=0; i<num_regions; i++) Fprintf( ofile, "Region %d: %s\n", i, name_list[i] ),- } */ / ♦ Get slopes in power curve : OMIT THIS FOR NOW find_slopes ( sl_array_size, power_100, εl_array ); for (ct=0; ct<εl_array_εize; ct++) Fprintf ( ofile, "Power(\%) [%d] = %4.2f; \tSlope[%d] = %4.2f\n", ct, power_100 [ct] , ct, sl_array[ct] ) ; maxslope = maxl( sl_array, sl_array_size, ϋmaxdex ); minslope = mini( sl_array, εl_array_εize, &mindex ) ; Fprintf( ofile, "Maximum slope value = %f at posn %ld\n", maxslope, maxdex ) ; Fprintf) ofile, "Minimum slope value = %f at posn %ld\n" , minslope, mindex ) ; 1 1 7 free( power ) ; free( power_100 ) free( εl_array ) ; free( zeros ) ; free( zeros_100 ) free( d_data ) ; fcloεe(ifile) ; fcloεe(ofile) ; return 0; /* **** * * * ****** * ** * ***** FUNCTIONS * ***** *** **** * * ** * * **** * * * * * * * * *** * * * / int get_formantε ( struct header *ifh, FILE *iffile, int *peak_set ) /* Readε formant values from ESPS file input.fb, created by formant program */ { char ♦ param_file = NULL; / ♦ parameter file name ♦ / FILE ♦ ofile = stdout; char / ♦ input and output file names */ ♦ oname = "formant.txt"; εtruct fea_data ♦ formrec; / ♦ record for fea data ♦ / double formε[4]; / ♦ array for current F valueε ♦ / double *formptr; /* pointer to formant valueε ♦ / / ♦ Global: long num_frec; number of records in input file */ short step_flag = 0, local_debug = 1; regiεter int j, ct; if ((ofile = fopen(oname, "wt")) == 0) { Fprintf( stderr, "get_formants: Error opening output file\n" ) ; return (-1) ; } num_frec = ifh->common.ndrec; / ♦ get number of data records ♦ / / ♦ Fprintf( stderr, "Number of formant records is %ld\n", num_frec ); ♦ / / ♦ Allocate an fea record ♦ / if (num_frec > num_srec * 1.5) { /* ie 2 formant recε to 1 spectral rec: should be false ♦ / step_flag++; if (local_debug) printf ( "Reading alternate formant entries\n"),* } formrec = (struct fea_data ♦ )allo_fea_rec( ifh ); formptr = (double ♦ )get_fea_ptr( formrec, "f ", ifh ) ; 11 8 / ♦ get formant values and procesε them ♦ / ct=0; / ♦ Initialiεe loop counterV Fprintf( ofile, "Formant data from input.fb\n" ) ; while ( get_fea_rec(formrec, ifh, iffile) != EOF ) { Fprintf( ofile, "Recorded]: ", ct ) ; for (j=0; j<4; j++) { forms[j] = formptrfj]; * (peak_set+(ct*4)+j) = (int) (forms[j]) ; Fprintf( ofile, "F%d=%4.1f; ", j, (float)forms[j] ) ; ) Fprintf(ofile, "\n"); ct++; if (step_flag) get_fea_rec(formrec, ifh, iffile) ; } fclose(ofile) ; retur (0) ; ) ♦define NORMALDB 40 / ♦ As in SPECTRUM.H ♦ / int load_spectral_data( struct header ♦ ih, FILE ♦ ifile, int step, Spectrum ♦ spec_array ) / ♦ load in εpectral data from file ./rcspec.fea into array of structureε spec_array spectrumfi,j] is spec_array[i] .amp[j] or spec_array->amp[j] . step=l or 2 (how many input spectra to each output spectrum */ { register int j, ct; FILE / ♦ ifile, 4 / ♦ fp; / ♦ struct header ♦ ih; ♦ / struct feaspec *srec; float spec_val, y_scale; short interval; long num_freqs, axdex; num_freqε = *get_genhd_l ( "num_freqs", ih ) ; Fprintf( stderr, "Number of freq's is %ld\n", num_freqs) ; if (num_freqs < 257) { Fprintf ( stderr, "%s: Exiting. \n", ProgName ) ; return (-1) ; } if (num_freqs == " 513) { / ♦ bigspec = True; 4 / interval = 2; ) else interval = 1; 1 1 9 srec = (εtruct feaspec *) llo_feaspec_rec( ih, FLOAT ); /* get εpec valueε and proceεs them */ ct = 0; /* fp = fopen( "dumpspec.txt" , "wt"); */ while (( get_feaspec_rec (srec, ih, ifile) != EOF ) /*&& (ct<10)*/ && (ct < num_rec * Spec_to_fea_ratio) ) { y_scale = maxl ( srec->re_spec_val, num_freqs, &maxdex ) ; if (maxdex == 0) /* ie maximum is at 0Hz */ y_scale = srec->re_spec_val[1] ; / ♦ take the closest value ♦ / / ♦ tot_pwr = ♦ (εrec->tot_power) ; ♦ / for (j=l; j<num_freqs; j+=interval) { / ♦ SPECSIZE times in all ♦ / spec_val = srec->re_spec_val [j ] + NORMALDB - y_scale; spec_array[ct] .amp[ (j-1) /interval] = spec_val; } / ♦ dump_sρec( sρec_array[ct] .amp ) ; */ ct++; if (step == 2) get_feaspec_rec(srec, ih, ifile); /* εkip next record */ } printf ("\t%d spectral records loaded\n", ct) ; return 0; } /***** * ***** * ****** * ****** * ****** ** * * ********************************* * , float getvma ( float *pwr2 , int εl_array_εize, Feature *fea_set, int *index ) /* Finds maximum pwr in voiced regions only. Sometimes power2 in bursts can exceed maximum in nuclear regions. We don't want this to affect normalisation of power valueε. */ { Feature *mfp; float voice [εl_array_εize] , zcr[sl_array_size] , max_val; register int j ; max_val = pwr2 [0] ; ♦ index = 0; mfp = fea_set; for (j=0; j<sl_array_size; j++) { voice[j] = mfρ->voiced; zcr[j] = mfp->zeros; mfp++; ) for (j=l; j<sl_array_size; j++) if (pwr2[j] > max_val) if (voice[j] > 0.8 && zcr[j] < 2600) { max_val = pwr2 [ ] ; ♦ index = j ; 1 2 0 } return (max_val) ; } float find_rise( float ♦ parray, long start, int ♦ offset ) / ♦ This function calculates the rate of rise in power for the first rising transition after start; the value of offset contains the difference between start and the top of the transition. ♦ / { register int ct = 0; / ♦ position relative to start ♦ / float tr_size=0; int rstart, rend; long pos = start; / ♦ absolute position in array ♦ / float val = parray[start] ; /* power value at current pos ♦ / / ♦ Look for start of rise */ while ((val > parray[pos+1] ) && (pos < (start+50))) val = parray[++pos] ; if (pos == start+50 ) { Fprintf(stderr, "%s: No rising transition found; exiting\n", ProgName); return (1) ; ) /* Now start measuring the rising transition */ rstart = pos; val = parray[rstart] ,- while ( (val < parray[pos+1] ) ._& (pos < num_rec-start) ) { val = parray[++poε] ; ) if (pos == num_rec - start) { Fprintf (stderr, "%s: No end of tranεition found; exiting\n" , ProgName); return (1) ; } if (debug_leve1) Fprintf(stderr, "Top of transition: power=%4.1f; pos=%d; num_rec=%ld\n" parray[pos] , pos, num_rec) ; rend = pos; if (debug_leve1) Fprintf(stderr, "rend = %d\n", rend); ♦ offset = pos - start; tr_size = parray[rend] - parray[rstart] ; if (debug_level) Fprintf (stderr, "Transition size = %f\n", tr_size) ; if ( (tr_size>0) && (rend > rstart) ) returnf (float) tr_size/ ( (rend-rstart) ♦ Step_size) ); else return (-1); / ♦ failure ♦ / 1 2 1 void find_slopes ( long size, float *parray, float *sl_array ) /* long size; float ♦ parray; float *sl_array; */ { long pos; pos = 0; while (pos < size-1) { sl_array[poε] = parray[pos+l] - parray[pos] ; pos++; } ) short get_peaks ( float *data, float *p_data, float *zeros, int num_vals ) { /* p_data is normalised power data: we want to ignore any spurious peaks during εilence. zeros required because we can have low power in fricatives. Useε global diff_peaks[MAXPEAKS] */ register int n, ct; short pk_count=0; int pos, temp, nmax, /* index of max value */ ctmax = 8, /* arb'y value */ last_peak; UINT valid[num_vals] ; float this, next, last, prev, maxval, /* used in calculating local max */ pit = 2.0, /* min power threshold */ zlt = 1000, / ♦ lower zeros threshold ♦ / k = 6.0, / ♦ constant determined by exp't ♦ / kl = 4.0; Fprintf(stderr, "In get_peaks:\n") ; / ♦ Clean up specdiff array first ♦ / for (n=0; n<num_vals; n++) valid[n]=True; / ♦ εet all peaks to valid */ Fprintf( stderr, "Cleaning out spurious peaks: " ) ; / ♦ we know d_data[0] is always 2.0, since set by εpec_diff */ for (n=l; n<num_vals-l; n++) { if (p_data[n] <= 100) { / ♦ Check all peaks ♦ / if ((p_data[n] < pit) && (p_data[n+l] < pit) && (zeros[n] < zlt) && (zeros[n+1] < zlt) ) { / ♦ silence level ♦ / valid[n]=False; / ♦ set this value to false */ 1 2 2 Fprintf( stderr, "%d;", n ); } } } / ♦ Check final value: n=num_vals ♦ / if (p_data[n] < 100) { / ♦ check all peaks ♦ / if ((p_data[n] < pit) &&. (zeros[n] < zlt) && (zeros[n-l] < zlt)) { /* silence level ♦ / valid[n]=False; / ♦ set thiε value to false */ Fprintf ( stderr, "%d;", n ) ; } } Fprintf( stderr, "Done\n"); pos=0; /* deal with 1st 3 values */ if <valid[0J && (data[0] > 10.0)) { if (data[0] > data[l] ) { if ( (datafl] > data[2]) || (data[0] > data[l]+40) ) { Fprintf(stderr, "Peak found at position [0]\n"); diff_peakε[0] = 0; pk_count++; } } } /* deal with next 3 values */ if ( valid[l] &.&. (pk_count == 0)) { / ♦ no peak found yet ♦ / if ((data[l] > 10.0) && (datafl] > data[0])) { if ( (datafl] > data[2]) && (data[2] > data[3])) { / ♦ Fprintf(stderr, "Peak found at position [l]\n"); */ diff_peaks[0] = 1; pk_count++; } } } n=pos=2; /* deal with most of array in loop */ while ( n<num_vals-2 ) { this = data[n] ; next = data[n+l]; last = data[n-l]; / ♦ Peak found if current value exceeds next/last by 10% and also exceeds last/next resp. ♦ / if (valid[n] ) { if (this >= 80) { if ( (this > (last+5)) && (thiε > (next+5)) ) / ♦ check for peak ♦ / diff_peaks[pk_count++] = last_peak = n; ) 1 2 3 else { if (this > 45) { if (( (this > (next+10)) && (this > (2*last) ) ) | | /* 3 was too severe */ ( (thiε > (last+10)) && (this > (2 4 next) ) )) { diff_peaks[pk_count++] = last_peak = n; } else { / ♦ l.no valid peak found: look for slow transition. look for series of values > 25, find local maximum and assign peak to that position. Value of n is NOT incremented ♦ / Fprintf(stderr, "Checking for slow transition..."); temp = n; ct = 0; nmax = n; / ♦ index of initial max value ♦ / maxval = this; while ( (this>25) && (ct<ctmax) ) { ct++; this = data[n+ct]; if ( this>maxval ) { maxval = this; nmax = n+ct; } } if (ct > 3) { Fprintf (stderr, "assigning peak.\n"); diff_ρeaks[pk_count++] = laεt_peak = nmax; ) else Fprintf (stderr, "no.\n"); ) / ♦ goes with else 1: ♦ / } else { / ♦ <= 45 */ if (this > 25) { if ( ((this > (kl ♦ laεt)) && (this > (next+10))) | | ((this > (kl ♦ next)) && (this > (last+10))) ) { diff_peaks[pk_count++] = last_peak = n; } } /* Otherwise, peak found if current value exceeds n+/-lth value and n+/-2th value */ else { /* this < 25 */ if ( this>10 ) { if ( ( (this > k 4 last) && (this > k*next) ) &&• ( ( (laεt-data[n-2] ) >1) | | ( (next-data[n+2] ) >1) ) ) { diff_peakε [pk_count++] = laεt_peak = n; ) else { / ♦ look for lone small peak: eg N to sonorant if ( (n-last_peak) > 15) ♦ / 1 2 4 } } } / ♦ end of last else clause */ } /* end of next to outer else ♦ / } /* end of outer else: if this > 90 ♦ / } / ♦ end of if(valid) */ n++; } / ♦ end of for loop: n=num_vals-2 ♦ / / ♦ deal with [num_val-2] value; remember [num_val-l] is end value */ this = data[n] ; next = data[n+l]; laεt = data[n-l] ; / ♦ Peak found if current value exceedε next/last by 10% etc as above ♦ / if ( ((this > (last + 10)) && (this > next)) | | ((this > (next + 10)) && (this > last)) ) { / ♦ Fprintf(stderr, "Peak found at position [%d]\n", n) ,- ♦ / diff_peaks[pk_count++] = n; } / ♦ Otherwise, peak found if current value > 10% and exceeds both n-lth value and n-2th value (there is no (n+2)th value) ♦ / else { if ( ((this > 10) && (this > k*next) ) && ((this > k*last) && (laεt > data[n-2])) ) { / ♦ Fprintf(stderr, "Peak found at position [%d]\n", n) ; */ diff_peakε [pk_count++] = n; } } /* deal with [num_val-l] value; ie end value ♦ / poε = ++n; / ♦ = num_val-l ♦ / if ( pos != (num_vals-l) ) { Fprintf(stderr, "Error; pos=%d; setting value to num_val-l\n" , pos); pos = num_vals-l; } prev = last; last = this; this = next; if ( (this > 10) && ((this > last) && (last > prev)) ) { / ♦ Fprintf (stderr, "Peak found at end positioned] \n" , pos); */ diff_peaks[pk_count++] = pos; } /* for (n=0; n<pk_count; n++) Fprintf(stderr, "Peak found at position [%d]\n", diff_peaks [n] ) ; */ return (pk_count) ; } /* end of get_peaks() */ 1 2 5 int get_most_peakε( float *data, float *p_data, float "zeros, int num_valε ) { int get_lstmax( float *data, int start, int end ) / ♦ finds 1st peak between start and end values of data[] ♦ / { register int n; int diff=4; float this, last, next, max=0.0; εhort found = False; /* for (n=0; n<end; n++) { thiε = data[n] ; if (thiε > max) { max = this; pos=n; ) } */ n=start; this=data[n] ; next=data[n+l] ; if ( (datafn] > 10) && (next < 3) && (next > data[n+2]) ) return n; /* stronger condition for 1st point */ elεe { n++; while ((n<end-l) __.. (! found)) { next=data[n+1] ; laεt=data[n-l] ; this=data[n] ; if (this > 5) { if ( ((this > last+1) && (this > next+diff) ) || ((this > next+1) -_& (this > last+diff) ) ) found=True; } n++; } /* end of while */ } if (found) return n; /* success */ else return 1; /* failure: set to position 1*/ } int insert_peak( int ♦ peaks, int pos, int val ) 1 2 6 { register int i; int top = MAXPEAKS-1; / ♦ check for valid value of new peak ♦ / if (val > peaks[pos]) { Fprintf( stderr, "Error in insert_peak: quitting\n" ) ; return -1; } else { for (i=top; i>pos; i--) peaks[i] = peaks[i-l]; peaks [pos] = val; return 0; ) } /* end of insert_peak ♦ / 127 /* * *************** * ***** * * * ***** * ******** * *************** * *************** ♦ FINDELEM . C ♦ by Geoff Williams * ♦ Forms part of recognition program rec_word, together with get_syl2.c, ♦ parεe_word.c and parse_seg.c. This module contains the routines for ♦ detecting elements A, U and I in Nuclei. ♦ These routines need to be modified in the light of recent revisionε ♦ to the theory. * * ** * ** * * *** * * *** * *** * ** * ****** * *** * **** * * * *** * * *** ** ** *** ** * ** * * * * * ******/ ♦include <stdlib.h> ♦include <stdio.h> ♦include <string.h> ♦include <math.h> ♦define max(x, y) ( (x>y) ? x : y) ♦if !defined) UINT) typedef unsigned int UINT; typedef int BOOLEAN; ♦endif ♦define True 1 ♦define False 0 ♦define SPECSIZE 256 / ♦ Must agree with value in SPECTRUM.H ♦ / ♦define NORMALDB 40 /* " " " */ ♦define STARTFREQUENCY 100 /* Starting frequency for log frequency plot */ typedef struct { BOOLEAN U0; BOOLEAN Al; 1 2 8 BOOLEAN 10; BOOLEAN GS; BOOLEAN hO; } E_type; ♦if !defined(_Spectrum) typedef struct yspec { float amp[SPECSIZE] ; } Spectrum; ♦endif extern point curve[2] [SPECSIZE] ; /* For storing curve coordinates */ extern int debug_level; E_type *elem_set; / ♦ used in process_region */ extern long num_rec, /* no of data records to read */ num_frec; /* no of formant records ♦ / extern float SampRate, l_range; / ♦ = Maxfrequency/STARTFREQUENCY ♦ / /* extern Spectrum *spec_array; array for spectral data */ extern UINT Maxfrequency; /* Top limit of Frequency range for spectrum ♦ / ♦if [defined (_differenced) extern BOOLEAN differenced; ♦endif /* ♦define STEPSIZE 100 Must agree with value in FINDUAI.C */ extern int fstep; /* Frequency range / no. of spectral points */ extern int Spec_to_fea_ratio; / ♦ ratio of spectral records to feature records (=2)*/ /* * . * * * ******* * ** * .** * **. * . ** * * * ** *** * * * * * *,,* * * ** * * .*, ** ** * **., * ..**** * * * FUNCTION PROTOTYPES ♦ .***.* * *** * *** * *** * ** * ** * * * .* * . ** . * . * * * * * * * .* ** * **** ** ****** , * * * ..*, * ** * / / ♦ UTIL.H */ float maxi( float data[] , int s, int ♦ index ) ; float mini( float data[] , int s, int *index ); int bandwidth( UINT peakf, /*int fεtep,*/ float εpectrum[] ) int find_20db( UINT start, float spectrum[], float halfmax ) ; float calc_spec_energy( int c_name, int lo_val, int hi_val ) ; BOOLEAN find_A( float *e_level, formant *formants, UINT db20, BOOLEAN *head ); BOOLEAN find_U( float *sρec, float *e_level, formant *formants, UINT db20, BOOLEAN *head ) ; 1 2 9 BOOLEAN find_I ( float spec [ ] , float *e_level , formant *f ormants , BOOLEAN *head ) ; void add_peak( int *peakε, int lower, int value ); int procesε_spec( int c_name, float spectrum!], int peak_set[4] , E_type *Elemε, short *head, BOOLEAN verbose ) ; int process_region( int start, int end, Spectrum *spec_array, int *peakε, int *uai, char *nuc_str int find_peaks( i r_name, float inspectrum[] , int peak_set[4], E_type *Elems , , int bandwidtht UINT peakf, /*int fstep,*/ float spectrum[] ) /* Returnε 3dB bw of peak in Hz; returns -1 or -2 if not possible. peakf is the frequency of the peak in Hz. */ { register int i; float peakamp, lastamp, amp; UINT poε, startpos; int bw; if (fstep < 15) { fprintf (stderr, "Error: fstep = %d\n", fstep); return -3 ; ) startpoε = peakf / fstep; /* Position of peak in spec units */ pos = startpos; peakamp = spectru [poε] ; lastamp = amp = peakamp; while ( (fabs (peakamp - amp) < 3.0) && ((pos > 0) && (amp <= lastamp)) ) { lastamp = amp; pos--; /* move left from peak */ amp = spectrum[poε] ; } if (fabs (peakamp - amp) >= 3.0) { bw = 2 * (startpoε - pos) * fstep; return bw; } else { if (pos == 0) /* Should only ever apply to zeroth peak */ bw = -1; else { /* This means the curve startε risin again before reaching 3dB point. Try going in the other dir'n before giving up */ pos = startpos; lastamp = amp = peakamp; while ( (fabs(peakamp - amp) < 3.0) && ((pos < SPECSIZE) && 1 3 0 '. (amp <= lastamp) ) ) { lastamp = amp; pos++; /* move right from peak ♦ / amp = spectrum[pos] ; } if (fabs(peakamp - amp) >= 3.0) { bw = 2 * (pos - startpos) * fstep; return bw; ) else { if (pos == SPECSIZE) bv- = -1; else bw = -2; ) /* End of inner else clause */ } return bw; ) / ♦ End of outer else clause ♦ / ) ♦define MAXTHRESH 96 / ♦ equiv. to 1500Hz ♦ / int find_20db( UINT start, float spectrumf], float halfmax ) { / ♦ Find 20dB point (half max. value): this will be a good measure of Mass. The value is returned in Hz. ♦ / register int n; int threshpt, / ♦ Spectral point where amp first exceeds 20dB */ droppoin ; n = start; while ( (n < SPECSIZE/2) && (spectrum[n] < halfmax) ) n++; / ♦ Find first point where curve exceeds 20dB ♦ / / ♦ This will be the start value on 2nd call ♦ / threshpt = n; if (threshpt >= MAXTHRESH) { /* ( ERROR:something wrong with this spectrum! */ return -1; ) else { / ♦ spectrum OK ♦ / while ( ( n < SPECSIZE ) && (spectru [n] > halfmax) ) n++; droppoint = n; /* Check that spectrum doesn't just dip below threshold: start again: */ if ( (droppoint - threshpt)< 3 ) { / ♦ i.e. 45Hz @ fstep=15 ♦ / 1 3 1 fprintf( stderr, "Re-calling find_20dB with n=%d\n" , n ) ; return (find_20db( droppoint, spectrum, halfmax )); ) '* while ( (n < SPECSIZE/2) && (spectrum[n] < halfmax) ) n++; ♦ 1st point where curve exceeds 20dB if ( n < 300/fstep | | (n - droppoint) < (10 * fstep)) ♦ Arb. value find_20db( n, spectrum, halfmax ) ; ■ I else return (droppoint * fstep) ; /* Value in Hz */ ) float calc_spec_energy( int c_name, int lo_val, int hi_val ) / ♦ Measures energy in specified frequency range, based on log-log spectrum. Uses the curve points stored in curve[2] [SPECSIZE] . Frequency bounds are given in freq. steps, ie 0..SPECSIZE-1. c_name is either 0 or 1, Left or Right resp. Energy in range F is defined as the area under the spectrum: F-l sum [ (x2-xl)*(yl+y2)/2 ] x=l where the y values are relative to - 0dB, and values below this count as 0. */ { register int f; int xl, x2, yl, y2; float energy = 0.0; if (lo_val<0 I I lo_val>SPECSIZE) { printf( "Range error in calc_spec_energy: lo_val = %d\n" , lo_val ) ; return 0; ) if (hi_val<lo_val | | hi_val>SPECSIZE) { printf( "Range Error in calc_spec_energy: hi_val = %d\n", hi_val ) ; return 0; } xl = curve[c_name] [lo_val] .x; 1 * * 1 x2 = curve[c_name] [lo_val + 1] .x; yl = 349 - curve[c_name] [lo_val] .y; y2 = 349 - curve[c_name] [lo_val +1] .y; for (f=lo_val; f<hi_val-l; f++) { xl = curve[c_name] [f] .x; /* */ x2 = curve[c_name] [f+1] .x; /* x2 is always > xl */ yl = 349-curve[c_name] [f] .y; /* Subtract from 349 since values inverted ♦ / y2 = 349-curve[c_name] [f+1] .y; 1 3 2 „ if ( yl<0 ) yl=0; if ( y2<0 ) y2=0; energy += ((x2-xl) ♦ (yl+y2)/2); } return (energy) ; ) void add_peak( int *peaks, int lower, int value ) { register int i; int top=5; / ♦ Must be < MAXPEAKSV for (i=top; i>lower+l; i--) { / ♦ throw away peaks[lower+top+1] */ peaks[i] = peaks[i-l], ■ } /* Now insert the new peak at position [lower+1] ♦ / peaks[lower+1] = value; } ♦define MAXPEAKS 6 int process_spec( int c_name, float spectrum[], int peak_set[4], E_type *Elems, short *head, BOOLEAN verbose ) / ♦ Does the same as find_peaks except gets formant values from ESPS file input.fb, created by formant program */ { UINT first_peak = 0; / ♦ Index of first peak that counts ♦ / BOOLEAN Ahead, Uhead, Ihead; FILE *of; formant FI, F2, F3; formant foπnant_values[3] ; register int n, i; char *clash_mesg = "head clash!!"; int db20; /* Frequency at which curve first goes below 20dB */ short head_ct, pk_count; short local_debug=0; / ♦ If true, print debugging info in this function ♦ / float start, /* temp variable for calculating start -alue ♦ / halfmax, ymax, /* Max value of curve in dB ♦ / pkO, pkl, e_level[3]; / ♦ [0] is Energy in range 0.1-lkHz ♦ / / ♦ [1] is Energy in range 1.0-2kHz ♦ / /* [2] is Energy in range 2-4kHz */ if (verbose) of = stderr; else { of = fopent "/dev/null" , "w" ) ; / ♦ fprintf( stderr, "Using /dev/null\n") ; ♦ / ) Ahead = Uhead = Ihead = False; for (n=0; n<3; n++) e_level[n] = 0.0; ymax = / ♦ maxi( spectrum, SPECSIZE )*/ 40; / ♦ spectrum already normalised ♦ / db20 = find_20db( 7, spectrum, ymax-20 ) ; /* 7=105Hz at SPECSIZE of 256, Freq. range of 4kHz */ if (db20 <0) { fprintf( of, "\nERROR: process_spec: not a vowel spectrum! \n" ) ; Elems->A1 = False; Elems->U0 = False; Elems->10 = False; Elems->hO = Elems->GS = False; return (-1) ; ) fprintf ( of, "20dB point is at %dHz;\n", db20 ); if ((c_name == 0) | | (c_name == 1)) { e_level[0] = calc_spec_energy( c_name, 6, 64 ) ; e_level[l] = calc_spec_energy( c_name, 64, 128 ) ; e_level[2] = calc_spec_energy( c_name, 128, 200 ) ; /* Frequencies for frange=4kHz, SPECΞIZE= 256: 6-64, 64-128, 128-200 Frequencies correct for frange=5kHz, SPECSIZE= 256 */ fprintf! of, "Energy < 1kHz = %5.1f; l-2kHz = %5.1f;2-4kHz = %5.1f\n", e_level[0], e_level[l], e_level[2] ); ) else { fprintf( of, "Error: c_name out of range, value %d\n" , c_name ) ; / ♦ free( fname ) ; */ return (-1) ; } .****★******************_ /* Print out peak positions and amplitudes */ if (local_debug) { fprintf( of, n \tPeak\t\tFrequency\tAmplitude\t3dB BandWidthNn" ); fprintf( of, "\t====\t\t=========\t=========\t=============\n" ) ; pk_count = 4; • for ( i=0; i < pk_count; i++ ) { fprintf( of, "\t%d\t\t%d\t\t%3.If\t\t%d\n" , i, peaklset[i] , spectrum[peak_set[i] / fstep], bandwidth(peak_set[i] , spectrum) ) ; ) 1 3 4 j n = first_peak; Fl.f = peak_set[0]; /* Frequency of FI in Hz */ Fl.peak_no = n; F2.f = peak_set[l]; / ♦ " F2 " ♦ / F2.peak_no = n+1; F3.f = peak_set[2] ; / ♦ " F3 " */ F3.peak_no = n+2; /* peak_set[3] = peaks[n+3]; */ Fl.amp = spectrumf peak_set[n] / fstep ] ; F2.amp = spectrum[ peak_set[n+1] / fstep ] ; F3.aιtιp = spectrum[ peak_set[n+2] / fstep ] ; fprintf( of, "FI " %dHz (%3.1fdB), F2 @ %dHz (%3.1fdB), F3 @ %dHz (%3.1fdB)\n ", Fl.f, Fl.amp, F2.f, F2.amp, F3.f, F3.amp ) ; formant_values[0] = FI; formant_values[1] = F2 formant_values[2] = F3 Elems->A1 = find_A( e_level, formant_values, db20, &Ahead ) ; if (Ahead) head_ct++; Elems->U0 = find_U( spectrum, e_level, formant_values, db20, &Uhead ) ; if (Uhead) head_ct++; Elems->10 = find_I ( spectrum, e_level , f ormant_values , fclhead ) ; if (local_debug) fprintf( stderr, "Back in process_spec\n") ; if (Ihead) head_ct++; if (head_ct == 0) ♦ head = 0; else { / ♦ work out code for head: A=l, 1=2, U=4 ♦ / ♦ head = (Ahead * 1) + (Ihead * 2) + (Uhead ♦ 4) ; fprintf ( of, "\tHead Code is %d; ", *head ) ; if (head_ct > 1) { printf ("\t%s\n" , clash_mesg) ; fprintf( of, "%s", clash_mesg ) ; } fprintf(of, "\n"); } Elems->h0 = Elems->GS = False; fprintf( of, "\n") ; return(0) ; } ♦define min(x, y) ( (x<y) ? x : y) / ♦ extern int step; -2 * 1 I * declared in parse_word.c: */ 5 extern int Min_N2_length; / ♦ = 20 Branching Nucleus condition: anything longer than this iε assumed to be branching */ extern int Max_Nl_length; / ♦ = 8 anything less than this probably non branching ♦ / extern BOOLEAN BR_ONSET, BR_NUCLEUS, BR_RHYME, DF_LE , BRN_IS_HEADED; int set_e_code( A_tot, I_tot, U_tot, length, headed ) { int e_code; e_code += (A_tot >= length/2 ? 1 : 0) e_code += (I_tot >= length/2 ? 2 : 0) e_code += (U_tot >= length/2 ? 4 : 0) if (e_code == 0) e_code = (A_tot > 1 ? 1 : 0) + (I_tot > 1 ? 2 0) + (U_tot > 1 ? 4 : 0) return (e_code) ; } int process_region( int start, int end, Spectrum ♦ spec_array, int ♦ peaks, int *uai, char ♦ nuc_str ) /* process a range of spectra from record [start..end] ; these will normally define a nucleus. Transform spectra to log-log and pass to procesε_spec routine (in findelem.c) nuc_str is either "branching", "SEQUENCE", "non-branching" or "unsure". If non-branching or unsure, this function sets it to one of the others. { FILE ♦ ofile = stderr; short ♦ head_array, head. Ah, Uh, Ih, A_tot, U_tot, I_tot, point_l, point_2, / ♦ 1st & 2nd points in heavy diphthong ♦ / headl = 0, head_ct = 0; int Maxplotfrequency = 5000, / ♦ this is the maximum freq in the curve plot, ie ignore 5-6k region ♦ / ct, rval, snu , / ♦ index to spectral array ♦ / index, e_code, local_debug=l, startf, x_step, 1 3 6 sttp = Spec_to_fea_ratio, la.'jt , length; BOOLEAN quiet = False, verbose = True, headed = False, long_v = False; register int f; float yamp, xnew, this_spec[SPECSIZE] , temp; E_type ♦ elem_array; / ♦ Passed to function process_spec() */ elem_array = (E_type *) calloc( num_rec, sizeof(E_type) ) ; head_array = (short *) calloc( num_rec, sizeof(short) ) ; last = min(end+l, num_frec) ; /* so we don't overrun peaks[][]; */ /* add 1 because its inclusive */ length = last - start; if (local_debug) { printf( "\tLength of region is %d:", length ) ; if (last < end+1) printf(" truncated."); printf("\n") ; } Maxfrequency = SampRate / 2; l_range = logl0( (double) (Maxplotfrequency / STARTFREQUENCY) ) ; if (local_debug) fprintf ( stderr, "l_range = %f\n", l_range) ; if (l_range < 1.5) { fprintf (stderr, "Error in value of l_range; ExitingXn"); return(-1) ; } fstep = Maxfrequency /(int) SPECSIZE; / ♦ Declared in SPECTRUM.H */ if (local_debug) fprintf( stderr, "fstep = %d\n", fstep); temp = (float) (100.0 / fstep); startf = (int) (ceil(temp) ) ; / ♦ round up ♦/ if (local_debug) fprintf( stderr, "Value of startf = %d\n" , startf ) ; if (startf < 5) { fprintf(stderr, "Error in value of startf; ExitingXn"),- return(-1) ; } x_steρ = fstep; point_l = 1; / ♦ dummy values for heavy diph: A,I ♦ / point_2 = 2; Ah β Uh = Ih = e_code = 0; A_tot = U_tot = I_tot = 0; 13 7 for (ct=start; ct<last; ct++) { snu = ct^step; fprintf ( ofile, "Spectrum %d: ", snum ); for (f=0; f<SPECSIZE; f++) { curve[0] [f] .x = 0; curve[0] [f] . = 0; } for (f=Startf; f<SPECSIZE; f++) this_spec[f] = spec_array[snum] .amp[f] ; / ♦ Do a virtual log-log plot of spectrum ♦ / for (f=startf; f<SPECSIZE; f++) { yamp = 349.0 * (float) (1.0 - (this_spec[f] / NORMALDB) ) ,- xnew = (float) ( (loglO( (double) (f * x_step) ) - 2.0) ♦ (float) (639.0 / l_range) ) ,- /* 2.0 = loglθ(lθθ) */ curve[0] [f-startf+1] .x = (int) xnew; curvefO] [f-startf+1] .y = (int) yamp; ) / ♦ If next line used, don't forget to open file first: dump_curve( curve[0] ) ; ♦ / head = 0; rval = process_spec( 0, this_spec, peaks+(snum ) , elem_array, &head, quiet ) if (rval==0) { if (elem_array->Al) { A_tot++; fprintf( ofile, "A+, " ) ,• } if (elem_array->U0) { U_tot++; fprintf( ofile, "U ," ),- } if (elem_array->10) { I_tot++; fprintf( ofile, "I " ); } uaifct] = elem_array->Al + (elem_array->10) *2 + (elem_array->U0)*4,* fprintf( stderr, " ; Element Code is %d; ", uai[ct] ) ; / ♦ e_code+=uai[ct] ; ♦ / head_array[ct-start] = head; fprintf( stderr, "Head Code is %d\n", head ) ; if (head > 0) { / ♦ heads contribute 2 to each total ♦ / head_ct++; switch (head) { case 1: Ah++; A_tOt++; break; 1 3 8 case 2: Ih++,- I_tot++; break; case 4: Uh++; U_tOt++; break; default: break; } /* end switch */ } } else { fprintf(stderr, "Trying to assign uai[%d]\n", ct) ; uai[ct] = 0; fprintf(stderr, "OK\n" ) ; ) fprintf( stderr, " \n" ); } /* end of outer for loop */ printf ( "Head count was %d\n", head_ct ) ; / ♦ Check for headedness if Branching Nuclei = TRUE ♦ / switch (nuc_str[l]) { /* first char is space */ / ♦ Upper Case means not alterable ♦ / case 'N' : / ♦ Japanese, Korean: No Branching Nuclei; Greek non-branching ones */ case 'B': /* so far only Greek, & English final stressed ♦ / if ( (head_ct > length/2) | | BRN_IS_HEADED) headed = long_v = True; if ( !BRN_IS_HEADED) e_code = set_e_code(A_tot, I_tot, U_tot, length, headed); / ♦ e_code is set later for the English case */ break; case 'S': /* Japanese allows N-N sequences separated by empty onset ♦ / headed = True; e_code = set_e_code(A_tot, I_tot, U_tot, length, headed); break; case 'b' : /* branching */ if (head_ct < 3) { headed = False; break; } /* else : check for heavy diphthong */ headed = long_v = True; brea ; case 'u * : /* 'unsure' ie in between ♦ / printf( "In the UNdecided case\n" ) ; if (length < 12) { / ♦ prob. single point ♦ / if ( head_ct > ( (4 * length) / 5) ) 1 3 9 headed = long_v = True; } else { if (length > 17) { / ♦ prob. 2 points ♦ / if (head_ct > (length / 2)) headed = long_v = True; } else { if (head_ct >= length/3) headed = long_v = True; ) } / ♦ else: check for heavy diphthong ♦ / break; case 'n': / ♦ non-branching ♦ / / ♦ if (head_ct > (4 ♦ length / 5)) headed = long_v = True; ♦ / headed = False; break; default: break; ) if (BR_NUCLEUS) { if (BRN_IS_HEADED) { printf ( "Checking for branching/non-br\n" ) ; if (headed) { strncpy( nuc_str, " branching", 12 ); if (long_v) { e_code += (A_tot >= min(20, length) ? 1 : 0) e_code += (I_tot >= min(20, length) ? 2 : 0) e_code += (U_tot >= min(20, length) ? 4 : 0) e_code *= 10; / ♦ turns n into nO for long vowel ♦ / } else { e_code += (point_l * 10); e_code+=point_2; } /* after all that: can't have a headed nucleus with e_code of 0 */ if (e_code == 0) { headl = ( (Ah > Ih ) ? 1 : 2) ; if (headl == 1) e_code = ((Ah > Uh) ? 1 : 4) ; else e_code = ( (Ih > Uh) ? 2 : 4); } /* mop up: stick an U in ♦ / if (e_code == 0) 1 4 0 e_code = 4; } else { strncpy( nuc_str, " non_branching", 15 ) ,- e_code = set_e_code( A_tot, I_tot, U_tot, length, False ); } /* end if headed ♦ / fprintf ( stderr, "Nucleus is head%s; E_code is %d\n", (headed ? "ed" : "less") , e_code ) ; } /* end if BRN_IS_HEADED */ else { fprintf( stderr, "Nucleus is %sbranching; E_code is %d\n", (nuc_str[l] == 'N' ? "non-" : "") , e_code ) ; strcpy( nuc_str, (nuc_str[1] == 'N' ? " non_branching" : " branching") ) ; } } else / ♦ Non-Branching nucleus */ fprintf ( stderr, "Non-branching Nucleus; E_code is %d\n", e_code ) ; return( e_code ) ; free( elem_array ) ; return 0; } /* * * * **************** * *************jjQ*p USED ************ * ********************** int find_peaks( int c_name, float spectrum[], int peak_set[4], E_type *Elems ) { Test routine: fp = fopen( fname, "at" ) ; fprintf ( fp, "\nEnd of find_peaks routine: c_name = %d\n" , c_name ); for ( n=0; n < SPECSIZE; n++ ) { fprintf( fp, "%d,%d\t", curve[c_name] [n] .x, curve[c_name] [n] .y ) ; } fclose( fp ) ; free( fname ) ; *******★*********************************** , BOOLEAN find_I( float spec[] , float *e_level, formant ♦ formants, BOOLEAN *Head ) { formant FI, F2, F3; BOOLEAN Ipresent, IHead, local_debug=0; UINT inter_size; UINT n; int FlF2diff, /* Difference between amplitudes of FI & F2*/ F2F3max, / ♦ Maximum of F2 and F3 ampitudes ♦ / F2F3diff, frange, index, start; 1 4 1 float dip_depth_12, / ♦ Distance in dB from largest peak ♦ / dip_depth_23; / ♦ to bottom of dip ♦ / float dip_min_12, / ♦ Lowest pt of curve between F1-F2 ♦ / dip_min_23; /' F2-F3 ♦ / float ♦ interval; FI = formants[0] F2 = formants[1] F3 = formants[2] / ♦ Adjust formant freq's to exact multiples of fstep Fl.f = fstep ♦ (int) (ceil(Fl.f /fstep)); F2.f = fstep ♦ (int) (ceil(F2.f /fstep)); F3.f = fstep ♦ (int) (ceil(F3.f /fstep)); 4 / frange = F3.f - FI.f; Ipresent = IHead = False; if (local_debug) { fprintf(stderr, "Looking for I ....") ; fprintf(stderr, "fstep = %d\n", fstep ); ) if (fstep <=0) { fprintf ( stderr, "Error: returning to process_spec\n" ) return False; ) / ♦ fprintf (stderr, "Fl.f = %d; F2.f = %d; F3.f = %d\n" , Fl.f, F2.f, F3.f); */ / ♦ Find minimum value of spec between FI & F2 ♦ / inter_size = 1 + ((F2.f - Fl.f) / fstep); / ♦ Convert to spectrum points */ interval = ( float * ) malloc( inter_size ♦ sizeof (float) ) ; start = Fl.f/fstep; for ( n=0; n < inter_size; n++ ) interval [n] = spec[n+start] ,- dip_min_12 = mini( interval, inter_size, tindex ) ; /* FlF2diff = fabs( Fl.amp - F2.amp ); */ dip_depth_12 = ((float) max(Fl.amp, F2.amp)) - dip_min_12; if (local_debug) printf( "\tFind_I: Size of Dip (FI -> F2) = %3.1f dB\n", dip_depth_12 ) ; / ♦ Find minimum value of spec between F2 & F3 ♦ / inter_size = 1 + ((F3.f - F2.f) / fstep); /^round up ♦ / free( interval ) ; interval = ( float * ) malloc( inter_size * sizeof(float) ) ; start = F2.f/fstep; for ( n=0; n < inter_εize; n++ ) interval[n] = spec[n+start] ; dip_min_23 = mini( interval, inter_size, &index ) ; F2F3max = (float) max( F2.amp, F3.amp ) ; /* F2F3diff = fabs( F2.amp - F3.amp ) ; */ 1 4 2 dip_depth_23 = F2F3max - dip_min_23; if (local_debug) printf( "\tFind_I: Size of Dip (F2 -> F3) = %3.1f dB\n", dip_depth_23 ); if ( ( (F2.f - Fl.f > 900) && ((F2.amp - dip_min_12) > 0.3 ip_depth_12 ) ) || ( (F3.f - F2.f > 900) && (dip_depth_23 > 20) && (F3.f < 3500) && /* This is to ignore formants >F3 | ♦ / (F3.amp - dip_min_23 > 0.5 4 dip_depth_23) ) ) { / ♦ The purported dip between F2 & F3 is only useful in A+I = [μ] */ if ( e_level[2] > e_level[l] ) /* ANDed with the previous condition */ Ipresent = True; /* Too simplistic ♦ / if ( (dip_depth_12 > 20) && ((F2.amp - dip_min_12) > 0.4*dip_depth_12 ) /* Λ needs normalising somehow */ && (dip_depth_12 > dip_depth_23) && (e_level[2] > e_le--3l[l]) ) Ipresent = IHead = True; } /* This condition maybe too strong */ if (ϋpresent) { /* if all else fails... */ if ( (F2.f - Fl.f) >= (0.68 ♦ frange) ) /* necessary because of y where FI can have much greater amplitude than F2, and e_level[2] = 0. Value of 0.69 determined from Wiebke's y in wb_iyu. Should do something better to fix this */ Ipresent = IHead = True; } if (Ipresent) { printf( "Element I. found!! ") ; if (IHead) printf ( "as a Head.\n" ) ; else printf( "(as an Operator.)\n" ); ♦ Head = IHead; return True; } else { printf ( "No I., found.\n" ) ; return False; } } BOOLEAN find_U( float *spec, float *e_level, formant *formants, UINT db20, BOOLEAN ♦ Head ) { /* Basically in a vowel, U is a Head when FI & F2 are < 500Hz & 1kHz, and F3 is off the bottom of the screen ( <40dB below FI ) . Often F * is not fully resolved, so I resort to energy values. Also get low FI (<350) and F2 -1800Hz but 30dB below FI; there seems to be no missing peak ♦ / 1 4 3 formant FI, F2, F3; BOOLEAN Upresent, UHead, local_debug = False; int bw2; float El_4 = e_level[l]+e_level[2] ; / ♦ Energy between 1 and 4kHz */ FI = formants[0] F2 = formantsfl] F3 = formants[2] bw2 = bandwidth( F2.f, spec ) ; / ♦ Bandwidth of F2 */ Upresent = UHead = False; if (local_debug) fprintf(stderr, "Looking for U ...\n"); / ♦ Check the bandwidth of F2: printf ( "Bandwidth of F2 is %d.\n", bw2 ); ♦ / if ( (e_level[l] < 5000) && (e_level[2] < e_level[l]) ) Upresent = True; / ♦ Doesn't work for " ♦ / if ( (Fl.f < 450) && (F2.f < 950 && bw2 > 0) ) { Upresent = True; if ( (Fl.amp - F3.amp) > 35 ) UHead = True; else { / ♦ Might fail clause 1 on bandwidth condition ♦/ if (( El_4 < 500 && e_level[0] > 30000 ) | | / ♦ As in [u] ♦ / ( e_level[0] > 60000 && El_4 < 2000 ) || /♦ As in [o] ♦/ ( e_level[0] > 10^e_level[1] )) / ♦ Exper'al, as in [o] ♦ / Upresent = UHead = True; ) ) / ♦ end of 2nd if statement */ else { if ( (Fl.f < 450) && (F2.f > 950) &•& (bw2 > 0)) if ((Fl.amp - F2.amp > 20) && (Fl.amp - F3.amp > 25)) Upresent = UHead = True; if ( (Fl.f < 450) && (e_level[0] > 10*e_level[1] ) __& (e_level[l] > e_level[2]) ) Upresent = True; /* Trial to catch U in [o] ♦ / ) / ♦ end of 2nd if..else clause ♦ / if (Upresent) { printf( "Element U found!! ' ') ; if (UHead) printf( "as a Head." ) ; else printf ( "(as an Operator.)" printf ( "\n" ) ; ♦ Head = UHead; return True; } else { - y printf( "No U. found.\n" ); return False; } ) BOOLEAN find_A( float *e_level, formant *£ormants, UINT db20, BOOLEAN *Head ) { formant FI, F2, F3; BOOLEAN Apresent, AHead, cond_l, local_debug=False; FI = formants[0] ; F2 = formants[1] ; F3 = formants[2] ; / ♦ Look for A+ element ♦ / if (local_debug) fprintf(stderr, "Looking for A+...\n") ; Apresent = AHead = cond_l = False; cond_l = (db20 > 850) && (ri.f > 450) && (Fl.f < 1200 ) ; if ( Fl.amp >= F2.amp ) { / ♦ Nearly always the case ♦ / if ( cond_l I | / ♦ I maybe K 4 F1 is better */ ( (db20 > 500) && (Fl.f > 440) ) ) { /* I (as in schwa) approx. v° FI for female speakers */ Apresent = True; /* Operator ♦ / if ( (db20 > 1000) && (( Fl.amp - F2.amp <= 10 ) && (F2.f < 2.2 * Fl.f)) ) { / ♦ Head ♦ / AHead = True; } else { if ( cond_l && ((Fl.amp - F2.amp) > 10) ) { if ( (e_level[0] > 60000) || (db20 > 1200) ) AHead = True; ) } /* end of else clause */ ) / ♦ end of if clause ♦ / } else { /* FI lower amplitude than F2 (see Turkish /a/) ♦ / if ( (F2.f > 500 && (F2.f-Fl.f) < 650) &•&■ (db20 > 850) ) { /* Turkish /a/ or English /e/ */ /* F2 is high enough and etc. ♦ / Apresent = True; if ( (db20 > 1000) || (F3.f < 1300) ) / ♦ To exclude /e/ */ AHead = True; 1 4 5 } else { /* F2 is missing and 'F3'>1200 as in phone case ♦ / if ( (Fl.f > 400) && (Fl.f < 1200) && (F2.f-Fl.f < 500) ) /* Eliminates velar burst spectra — Λ */ Apresent = True; / ♦ Need to find the missing F2 to check for Head */ } } if (Apresent) { printf( "Element A+ found!! "); if (AHead) printf ( "as a Head.Xn" ); else printf( "as an Operator. \n" ) ; ♦ Head = AHead; return True; } else { printf( "No A+ found." ); printf ( "\n" ) ; return False; } } /* End of find_A ♦ / 1 4 6 / ♦ PARSE_WORD.C by Geoff Williams This module is compiled with findelem.c, parse_seg.c and get_syl2.c to form rec_word, the main recognition routine called by the "recognize" script. This module contains the routines for parsing the feature file into. its constituents and some of their associated segmental content. Function find_regions is called by get_syl2.c, returning a character string containing the relevant information in coded form. */ ♦include <stdio.h> ♦include <stdlib.h> ♦include <string.h> ♦include <ctype.h> ♦include <math.h> extern char ♦ ProgName; ♦include "include/spectrum.h" ♦define True 1 ♦define False 0 ♦define MAXPEAKS 20 /* max number of peaks in spec_diff data ♦ / ♦define NAMESIZE 10 / ♦ length of names for regions of signal */ ♦define MAX_NUCLEI 10 / ♦ max number of Nuclei in a voiced region ♦ / ♦define FINAL 100 ♦define max(x, y) ( (x>y) ? x : y) ♦define min(x, y) ( (x<y) ? x : y) ♦define between(x, y, z) ( ( (x>=y) && (x<=z) ) ? 1 : 0) ** * ** * * * ** Globals ** ****** * ** enum voice_types { UNV, TRANSITION, VCD ) ; typedef struct myfeature { float zeros; float d_data,- float form_diff; float power; float midpower; float kl; float voiced; / ♦ now a f.p. value ♦ / float F0; } Feature; U 7 typedef struct myspec { float amp [SPECSIZE] ; } Spectrum; point curve[2] [SPECSIZE] ; / ♦ Stores curve co-ordinates ♦ / int Min_nuc_length=4; int Min_N2_length = 20; /* Branching Nucleus condition: anything longer than this is assumed to be branching */ int Max_Nl_length = 8; / ♦ anything less than this must be non_branching */ / ♦ these are declared in the main file get_syl2.c */ extern long num_rec, / ♦ no of data records to read ♦ / num_frec; / ♦ no of formant records ♦ / extern int debug_level; extern int Step_size; / ♦ ms */ extern int diff_peaks[MAXPEAKS] ; / ♦ holds positions of good peaks in d_data ♦ / / ♦ actual values at each peak held in diff_peakvals[] ♦ / extern short num_bounds, / ♦ No of peaks in specdiff data ♦ / num_regions; extern int max_pwr_pos, / ♦ location of maxpower ♦ / max_mid_pos, / ♦ location of maxpwr_mid ♦ / max_vmid_pos; / ♦ location of maxvoice_mid */ extern int Spec_to_fea_ratio; / ♦ ratio of spectral records to feature records (=2)*/ extern float maxzeros, minzeros, maxpwr_mid, minpwr_mid, maxvoice_mid, maxpower, minpower; extern float SampRate, l_range; / ♦ = Maxfrequency/STARTFREQUENCY ♦ / extern BOOLEAN BR_ONSET, BR_NUCLEUS, BR_RHYME, DF_LEN, BRN_IS_HEADED, NN_SEQUENCE; extern Spectrum ♦ spec_array; / ♦ array for spectral data ♦ / - * • ** * ** *****•*•** ** •** ••• ** ••** * / ♦ UTIL.H */ float axi( float data[] , int s, int ♦ index ) ; float mini( float data[], int s, int ♦ index ) ,- float sub_max( float ♦ data, int start, int end, int ♦ index ); 1 4 8 /* Findelem.c ♦ / int process_spec( int c_name, float ir.spectrum[] , int peaks[], E_type ♦ Elems, BOOLEAN verbose ) ; int process_region( int start, int end, Spectrum ♦ spec_array, int *peaks, int ♦ uai. char ♦ nuc_str ) ; /* parse_seg.c */ void reverse(char s[]); void itoa(int n, char *s) ; / ♦ char ♦ get_next_token( char *word, int ct ) ; char *build_onse ( char ♦ seg, char ♦ seg_str, int syl_ct ) ; char ♦ build_rhyme( char ♦ seg, char *seg_str, int syl_ct ) ; char *parse_seg_str( char *seg_str ) ; ♦ / / * * * * * * * * * * * * * * * * * * * * * * * L 0 c a ]. F U N C T I O N S * ** * ***** * ** ** **** * * ** ** * ** * * ** ** * * ** * ** * * ****/ float get_mean( float ♦ array, int start, int end ); int get_next_peak( float ♦ data, int start, int end, float *pk_val ) ; short is_low( float ♦ array, int start, int end, float uplim ) ; short is_high( float ♦ array, int start, int end ) ; void identify_regions( float ♦ peak_vals, float maxpwr, float ♦ d_data, float ♦ zeros, float ♦ tot_pwr, float ♦ mid_pwr, char names[] [NAMESIZE] ) ; short find_PV( short v_length, float ♦ voice, float ♦ tot_pwr ) ,- int get_nuclei(int n_start, int end, float ♦ tot_pwr, float ♦ mid_pwr, float tot_max, float mid_max, int *pk_array) ; void assign_ntype( short length, short tot_length, int reg_num, char ♦ type_str ) ; short find_edge_nuc(int start, int rest, UINT stress, int loc_maxpos, int mid_maxpos, float ♦ tot_pwr, float ♦ mid_pwr) ; int find_NUC( int start, int rest, UINT stress, int Imaxpos, int maxmidpos, float ♦ tot_pwr, float *mid_pwr, float ♦ d_data, float ♦ form_fac, short *last_reg ) ; int find_SON( int start, int rest, int speaker, int lmax_pos, float *tot_pwr, float *mid_pwr, float* d_data, float ♦ form_fac, short ♦ nuc_end ); char ♦ parse_voiced( int reg_num, int start, int end, Feature ♦ fea_set, int *uai, int *peaks, short *prev ) ; int find_FRIC( int start, int rest, float *zcr, float *kl, short *fric_type ); BOOLEAN process_fric( short length, short *prev, short *f_type, char *fric_string, char *seg_str ) ; int find_SIL( int start, int rest, float ♦ power, float *zeros, float ♦ mid_pwr, float 4 l ) ; 1 9 char parse_unvoiced( int reg_num, int start, int end, Feature ♦ fea_set, short ♦ prev, char *seg_str ) ; int check_εilence( int start, int end, Feature *fea_set ); char *find_regions ( float *diffpeaks, Feature *fea_set, char names[] [NAMESIZE] , int *uai, int *peaks ) ; float get_mean( float *array, int start, int end ) { register int j ; float total=0.0; for (j=start; j<end; j++) total += array[j]; return (total/ (end - start +1) ) ; ) /* end of get_mean */ short is_low( float *array, int start, int end, float uplim ) { /* are at least half the valueε in thiε region of the array < half the minimum value? returns True (1) if so, False (0) otherwise ♦/ register int ; float min=10000, enough=0.5, / ♦ constant ♦ / this; int total=0; / ♦ Find local min: is it < 50% of uplim? ♦ / fprintf (stderr, ": Checking local minimum..."); for (j=start; j<end; j++) { this = array[j] ; min = (this < min) ? this : min; } fprintf ( stderr, "Min value=%3.If;\n" , min ) ; if (min < 0.2 * uplim) return 1; / ♦ this is good enough ♦ / if (min < 0.5 ♦ uplim) { / ♦ significant max ♦ / for (j=start; j<end; j++) { if (array[j] < (2 ♦ min)) / ♦ trial value ♦ / total++; ) ) return (total >= (enough ♦ (end-start+1) ) ) ? 1 : 0; } / ♦ end of is_low ♦ / .1 5 0 short is_high( float ♦ array, int start, int end ) { /* are at least half the values in this region of the array > 40%: returns True (1) if so. False (0) otherwise. NB Works for NORMALISED arrays only ♦ / register int j; float max=0.0, enough=0.5, / ♦ constant */ this; int total=0; / ♦ Find local max: is it > 50%? ♦ / fprintf(stderr, ": Checking local max..."); for (j=start; j<end; j++) { this = array[j] ; max = (this > max) ? this : max; } fprintf( stderr, "Max value=%3.If; " , max ); if (max > 80) / ♦ ie 80% ♦ / return 1; / ♦ this is good enough */ if (max > 40) { / ♦ significant max ♦ / for (j=start; j<end; j++) { if (array[j] > (0.5 ♦ max)) / ♦ trial value ♦ / total++; ) } return (total >= (enough ♦ (end-start+1) ) ) ? 1 : 0; ) / ♦ end of is_high ♦ / int get_next_peak( float ♦ data, int start, int end, float ♦ pk.value ) / ♦ returns the position (relative to start) of the first peak in data[] , > than ♦ pk..value; or just start if no peak found. *pk_value is set to the size of the peak if peak found. ♦ / { register int n; int diff=4; float this, last, next, max=0.0; short found = False; n=start; this=data[n] ; next=data[n+l] ; if ( (datatn] > 25) && (next < 3) && (next > data[n+2]) ) { *pk_value = this; return n; / ♦ different condition for 1st point ♦ / } else { 1 5 1 while ( (n<end-l ) && ( ! found) ) { n++ ; next=data[n+l] ; this=data[n] ; if (this > 95) { if (next > 95) { n++; ♦ pk_value = next; } else *pk_value = this; found = True; break; ) last=data[n-l] ; if (this > *pk_value) { if ( ((this > last+1) && (this > next+diff) && dast>data[n-2] ) ) | ((this > next+1) && (this > last+diff) && (next>data[n+2] ) ) ) { found=True; *pk_value = this; ) ) } /* end of while */ ) if (found) return n-start; /* success */ else return 0; /* "failure": set to position 0 (relative to start) */ ) /*** ♦♦** *.****.*.**.***....****.*.**.***...*************. Now the serious stuff * *.. ***** .* * .. * ..... * . . * . ***** ** ***** . ** .. **** * *** .,. * ,/ ♦define ONSET 0 ♦define SILENCE 1 ♦define NUCLEUS 2 ♦define SONORANT 3 ♦define BURST 4 ♦define FRIC 5 ♦define LOW_TONE 6 ♦define HI_TONE 7 ♦define UNKNOWN 10 ♦define NUC_END 20 ♦define EDGE * 21 ♦define UP_EDGE 22 ♦define ERROR 30 ♦define MALE 50 ♦define FEMALE 60 1 5 2 ♦define OPERATOR 70 ♦define HEAD 80 int final_son_length = 7; / ♦ minimum duration of final SON in stressed region */ short find_edge_nuc(int start, int rest, UINT stress, int loc_maxpos, int mid_maxpos, float ♦ tot_pwr, float *mid_pwr) / ♦ Look for a rising edge with a slope greater than 40%. This will indicate glottal stop.*/ { register int i, j ; int end = start+rest-1, lastmax = max(loc_maxpoε, mid_maxpos) ; short gs_pos = -1; char *indent = "\t\t"; float *mid_slopes, ♦ slopes, *pwr_ratios, *pwr_midl00, *pwr_100, loc_max_pwr = tot_pwr[start + loc_maxpos] , thresh = 35.0, mid_thresh = 40.0; BOOLEAN gs_found = False, gs_max, gs_mid; printf ( "%sln find_edge_nuc: searching %d positions\n" , indent, lastmax ); pwr_ratios = (float ♦ )calloc(rest, sizeof(float) ) ; pwr_midl00 = (float ♦ )calloc(rest, sizeof(float) ) ; pwr_100 = (float *)calloc(rest, sizeof(float) ) ; mid_slopes = (float ♦ )calloc(rest, sizeof(float) ) ; slopes = (float *)calloc(rest, sizeof(float) ) ; for ( j=0; j< rest; j++) { /*pwr_ratios[j] = mid_pwr[ +start] / tot_pwr[j+start] ; ♦ / pwr_midl00[j] = mid_pwr[ +start] / _naxvoice_mid ♦ 100; / ♦ normalise mid_pwr ♦ / pwr_100[j] = tot_pwr[j+start] / loc_max_pwr ♦ 100; / ♦ normalise tot_pwr ♦ / } fir. _slopes( rest, pwr_100, slopes ); find_slopes( rest, pwr_midl00, mid_slopes ) ; / ♦ if not stressed nucleus, scale up the values appropriately */ 1 5 3 gs_max = gs_mid = False; for (i=0; i<lastmax; i++) { if ( (slopes[i] > thresh) && ( (pwr_100[i+1] > 75.0) | | ( (loc_maxpos-i) < 3)) ) { gs_max = True; ) else { if ( (mid_slopes[i] > mid_thresh) && ( (pwr_midl00[i+1] > 60.0) || (mid_maxpos-i < 3)) ) { gs_mid = True; ) ) if (gs_found = (gs_max | | gs_mid) ) { gs_pos = i; printf ("%sfound edge: slope=%4.1f at poεition %d\n", indent, (gs_max ? slopes[i] : mid_slopes[i] ) , i ) ; break; > ) return( gs_pos ) ; ) int find_NUC( int start, int rest, UINT stress, int Imaxpos, int maxmidpos, float *tot_pwr, float *mid_pwr, float *d_data, float *form_fac, short ♦ last_reg ) / ♦ mid_pwr is now absolute, not % Imaxpos and maxmidpos are relative to start of _voice_ region (ie can be less than start) ♦ / { register int i, j, k; / ♦ k is an end marker ♦ / int end = start+rest-1, Final_nuc_length = 6, / ♦ minimum duration of final NUC in region */ start_to_max, next_max_pos = 0, temp, leftover, peak_posl=0, peak_pos2=0; short local_debug = 1, end_pt, Near = 8; BOOLEAN k_hi, test, loc_max_near, loc_max_past, mid_max_near, max_near; float mean_pwr, mean_mid_pwr, mean_pwr_ratio, last_pwr_ratio, max_pwr, peak_mp, next_max_pwr, 1 5 4 . delta_ratio, ♦ mid_slopes, / ♦ percent slopes of pwr2 curve ♦ / ♦ pwr_midl00, *pwr_100, ♦ slopes, / ♦ percent slopes of tot_pwr curve ♦ / *pwr_ratios; / ♦ mid_power / tot_pwr */ float pk_value = 36, / ♦ min spec-diff peak value for NUC-SON transition */ pwrl=3000, / ♦ absolute */ mid_pl = 1.0, /* percent */ pwrh=60.0, /* high power threshold in % */ pwrmh=2.0e5, pwrm_son = 10000, pwr_son = 20000, /* lower power threshold for sonorants */ pwr_nuc = 5.0e5, n_fac = 0.2, /* mid-power fraction for end of nucleus */ nuc_ratio = 0.01; /* this will be sex-dependent */ printf( "\t\tln find_NUC: distance to end of region = %d\n" , rest); if (rest < 2) { printf( "\t\tEnd of region! !\n") ; return (end) ; } if ((nast_reg == NUCLEUS) || ( ♦ last_reg == NUC_END) ) { return (start) ,• } pwr_ratios = (float *)calloc(rest, sizeof (float) ) • pwr_midl00 = (float *)calloc(rest, sizeof (float) ) ; pwr_100 = (float *)calloc(rest, sizeof(float) ) ; mid_slopes = (float *)calloc(rest, sizeof(float) ) ; slopes = (float *)calloc(rest, sizeof(float) ) ; for ( j=0; j< rest; j++) { pwr_ratios[j] = mid_pwr[j+start] / tot_pwr[j+start] ; pwr_midl00[ ] = mid_pwr[j+start] / maxvoice_mid * 100; /* normalise mid_pwr ♦ / pwr_100[j] = tot_pwr[j+start] / maxpower ♦ 100; /* normalise tot_pwr */ } find_slopes( rest, pwr_100, slopes ); find_slopes ( rest, pwr_midl00, mid_slopes ); printf( "\t\tMid Power Slopes:" ); for (i=0; i<rest; i++) { if (i % 6 ss 0) printf("\n\t\t") ; printf ("%3.If\t", mid_slopes[i] ) ; ) printf("\n") ; mean_pwr_ratio = get_mean( pwr_100, 0, rest ) ; / ♦ % mean power ♦ / mean_mid_pwr = get_mean( mid_pwr, start, end ) ; 1 5 5 mean_pwr = mean_pwr_ratio / 100 * maxpower; /♦ absolute mean power */ if (local_debug) fprintf (εtderr, "\t\tMean midpower = %3.2f; mean %% power = %3.2f\n ", mean_mid_pwr, mean_pwr_ratio) ; start_to_max = Imaxpos - start; if (local_debug) fprintf(stderr, "\t\tDistance to local (tot_pwr) max=%d; ", start_to_max) ; loc_max_near = ( (start_to_max >= 0) && (start_to_max < Near)); loc_max_past = start_to_max < 0; start_to_max = maxmidpos - start; mid_max_near = ( (start_to_max >= 0) && (start_to_max < Near) ) ,- max_near = (loc_max_near | | mid_max_near) ; if ( max_near && local_debug ) fprintf( stderr, "Near local maximum!"); fprintf (stderr, "\n"); / ♦ if (*prev == SONORANT) j = start+1; else ♦ / j = start; k = min( end, 25 ) ; / ♦ don't start working backwards more than 25 positions ahead ♦ / if (rest < 11) { / ♦ prob'ly a single position ♦ / if (stress) { /* stressed nucleus condition */ if ( ( (mean_pwr_ratio > pwrh) && (mean_mid_pwr > pwrmh) ) | | ( (mean_pwr_ratio > 0.5 * pwrh) && is_high( pwr_midl00, 0, rest )) ) { *last_reg = NUCLEUS; printf("\n") ; return (end) ; } else { /* must be N+coda, or N+stop */ if (local_debug) fprintf(stderr, "\t\tDidn't confirm stressed nucleus\n") ; while ( (slopes[j-start] > -25.0) && (mid_slopes[ -start] > -35) && (j<end) ) end_pt = j+1; if ( j < end-3 ) { *last_reg = NUC_END; while (tot_pwr[j-start] > pwr_nuc) j++; if ( j < end-2 ) { printf( "\t\t? found!\n" ) ,• *last_reg = EDGE; /* abrupt end of Nucleus = ? */ return (j-1) ; } 1 5 6 else return (end_pt) ; } return (start) ; } ) else { /* no stress in rest of this region .-*********** NEEDS IMPROVING ******** /* check for peak in tot_pwr */ if ( (mean_pwr > pwr_nuc) | | is_high( pwr_midl00, 0, rest ) ) { printf ("\n") ; *last_reg = NUCLEUS; return (end) ; } else { printf ("\n") ; / ♦ in case of messages from is_high */ return (start) ; /* go looking for a Sonorant instead */ } ) ) else { /* prob'ly more than 1 position */ if ( stress ) { if ( max_near ) { /* we know this is a ÷stress N, but where does it end? */ /* move to local max position: mid_pwr usually lags tot_pwr peak */ if ( loc_max_near ) { while (j < Imaxpos) j++; ) if ( mid_max_near ) { while (j < maxmidpos) j++; } /*peak_mp = mean_mid_pwr = get_mean(mid_pwr start, j) ; ♦ / while ((tot_pwr[j] > pwr_nuc && (mid_pwr[j) > n_fac*maxvoice_mid) ) && (j<end)) { j++; /*mean_mid_pwr = get_mean(mid_pwr, start, j); */ } printf("\t\tvalue of j=%d; mid power/max mid_power = %3.2f\n", j, (mid_pwr[j] / maxvoice_mid) ); /* Check this isn't just a dip in the pwr2 curve */ temp = j ; leftover = end-temp; printf( "\t\tleftover = %d\n", leftover ) ; if ( (tot_pwr[j]>pwr_nuc) && (leftover>3) ) { for (j=temp; j<temp+3; j++) { if (mid_pwr[j] > n_fac*maxvoice_mid) break; } 1 5 7 if (j<temp+3) { /* carry on looking */ while ((tot_pwr[j] > pwr_nuc && (mid_pwr[j] > n_fac*maxvoice_mid) ) && (j<end)) } else j=temp; } if ((tot_pwr[j] > pwr_nuc) && (j<rest-final_son_length) ) { peak_posl = get_next_peak( d_data, j, end, &pk_value ) ; if (peak_posl == j) { pk_value = 36; peak_pos2 = get_next_peak( form_fac, j, end, &pk_value ) ; } / ♦ put some conditions on peak_pos ♦ / j += max(peak_posl, peak_pos2); printf ( "\t\tget_next_peak moved j to %d\n", j ) ; ) if (end-j > rest/3) { ♦ last_reg = NUC_END; / ♦ positively identified end of nucleus: ****** only if -ve slope great enough ♦/ return (j) ; } if (j-start > Min_N2_length) { ♦ last_reg = NUCLEUS; / ♦ not NUC_END, to avoid inserting a final SON */ return (j) ; ) while ( (pwr_ratios[j-start] > nuc_ratio) && (j<end)) j-*--*-; mean_pwr_ratio = (get_mean(tot_pwr, start, j)) / maxpower * 100; if (local_debug) fprintf(stderr, "\t\tvalue of j=%d, mean_pwr_ratio=%3.2f\n" , j, mean_pwr_ratio) ; if ((end-j) <= final_son_length) { if ( (end-j) <= 4 ) { *last_reg = NUCLEUS; return (end) ; } last_pwr_ratio = mean_pwr_ratio; test = False; while ( ('test) && (k > j) ) { mean_pwr_ratio = (get_mean( tot_pwr, start, k )) / maxpower * 100; teεt = (mean_pwr_ratio < 0.5*laεt_pwr_ratio) ; k—; ) *last_reg = NUCLEUS; 1 5 8 return (k) ; } if ( (j>lmaxpos) && /* well obviously it is ♦ / ( (mean_pwr_ratio > pwrh) || is_high(pwr_midlOO, start, j)) ) { / ♦ find position of next peak in spec_diff or form_factor */ printf("\n") ; *last_reg = NUCLEUS; return (j) ; ) else { / ♦ reduce j until the RHS is true */ } } else { / ♦ in stress region, more than 8 positions before peak: */ / ♦ check for rising power: */ while ((tot_pwr[j] > pwr_son) && (tot_pwr[j] < tot_pwr[ +1] ) ) / ♦ Start from end and work backwards: */ /* mean_pwr_ratio = */ while ( (mean_pwr_ratio < pwrh) && (k > j)) { k—; mean_pwr_ratio = (get_mean( tot_pwr, start, k )) / maxpower ♦ 100; } printf("\t\tFinal value of k=%d, mean power ratio=%3.2f\n" , k, mean_pwr_ratio) ; if (k - start > 5) { ♦ last_reg = NUCLEUS; return (k) ; ) else return (start) ; / ♦ go looking for a Sonorant instead ♦ / } } else { /* not primary stressed region */ next_max_pwr = maxi(pwr_100, rest, &next_max_pos) ; /* percent value */ for ( i=0; i< rest; i++) pwr_100[i] = pwr_100[i] / next_max_pwr * 100; /* re-normalise tot_pwr */ if (loc_max_past) { loc_max_near = loc_max_near | | (next_max_pos < Near) ; printf("\t\tNext max at position %d: %s\n", next_max_pos, (loc_max_near ? "near to max" : "")); } if (loc_max_near) { / ♦ unstressed N: whervs the end? ♦ / while ( (j < Imaxpos) | | (j < (start + next_max_pos) ) ) j++; /* skip past local max position ♦ / fprintf(stderr, "\t\tvalue of j=%d: ", j); 1 5 9 if (end-j < 8 ) { ♦ last_reg = NUCLEUS; printf ( "returning.\n" ); return (end) ; / ♦ make the rest of this region a N ♦ / } delta_ratio = 1 - (pwr_ratios[j-start] / pwr_ratios[j-εtart-1] ) ; / ♦ look for >80% change ♦ / while ((j < end) &.&. ( (delta_ratio < 0.8) && (pwr_100[j-start]>20) ) ) { delta_ratio = 1 - (pwr_ratios[j-start] / pwr_ratios[j-start-1] ) ; ) if ( pwr_100[j-start] < 20 ) ♦ laεt_reg = NUC_END; else ♦ last_reg = NUCLEUS; fprintf (stderr, "\t\tvalue of j=%d\n", j) ; return ((j-start > 6) ? (j-1) : (start)); / ♦ i.e only if a valid region */ ) else { printf ("\t\tTricky: giving up\n"); /* look for specdiff peak */ return( start ) ; } ) ) return (start) ; /* default case */ } / ♦ end find_nuc ♦ / / ** * ******** ***** **** .* * ..., * .*.* ** * * ,. * * *** . * .... * . * / float Max_son_pwr_slope = 12.0; /* max percent power slope allowed in Sonorant */ ♦define FLAT(x) (fabs( (double) x) < Max_son_pwr_slope) int find_SON( int start, int rest, int speaker, int lmax_pos, float *tot_pwr, float *mid_pwr, float ♦ d_data, float ♦ form_fac, short ♦ prev_type ) / ♦ In sonorants, ratio of low power to mid-range power is for Femaleε: > approx .0e3 for Males: > 1000 » / { register int i, j ; int typ_son_length = 8, end = start+rest, peak_pos, end_so_far, .1 6 0 mid_maxpos; short local_debug = 1; BOOLEAN k_hi, sonorant, far_from_peak = False; float / ♦ mean_pwr, mean_mid_pwr, ♦ / mean_pwr_ratio, loc_mid_max, son_thresh, pk_val = 20, /* height of next peak in spec_diff must be > this */ form_pk_val = 35, / ♦ height of next peak in form_fac must be > this */ *this_mid_pwr, *pwr_100, /* normalised total power between start and end */ *sl_array, /* percent slope data for total power ♦ / *low_to_mid; / ♦ ratios of low power to mid power: in voiced regions, low power = (total power - mid power) */ float pwrl=3000, / ♦ absolute ♦ / mid_pl = 1.0, / ♦ percent ♦ / kl_thresh = 0.6; printf( "\t\tln find_SON: distance to end of region = %d\n" , rest); if (speaker == FEMALE) son_thresh = 4000; else son_thresh = 300; low_to_mid = (float ♦ )calloc(res , sizeof(float) ) ; thiε_mid_pwr = (float ♦ )calloc(rest, sizeof (float) ) ; pwr_100 = (float ♦ )calloc(rest, sizeof (float) ) ; sl_array = (float ♦ )calloc(rest, sizeof(float) ) ; for ( j=0; j<rest; j++) { this_mid_pwr[j] = mid_pwr[j+start] ; low_to_mid[j] = (tot_pwr[j+start] / this_mid_pwr[j] ) - 1; pwr_100[j] = tot_pwr[j+start] / maxpower * 100; /* normalise tot_pwr */ } loc_mid_max = maxi( this_mid_pwr, rest, &mid_maxpos ) ; far_from_peak = (rest > 10) && ( min( (lmax_pos-start) , mid_maxpos) >= typ_son_length ) ; if (far_from_peak) { /* i.e. at or near start of longish voiced region */ if ( *prev_type == UNKNOWN && local_debug ) printf( "\t\tLooking for initial sonorant: ") ; if ( is_low(d_data, start, start+typ„son_length, 40) ) { /* pk_val should never be < 20; check d_data is stable */ peak_pos = get_next_pea ( d_data, start, lmax_pos, &pk_val ) ; if ( (peak_pos > 0) && (peak_pos < 3)) { /* try again ♦ / 1 6 1 pk_val = 20; / ♦ reset value ♦ / peak_pos = get_next_peak( d_data, start+peak_pos, end, 4pk_val ) + 1; } ) else { printf( "trying form_factor... " ) ; peak_pos = get_next_peak( form_fac, start, lmax_pos, &form_pk_val) ; / ♦ peak_pos should be relative to start ♦ / if ( (peak_pos > 0) && (peak_pos < 3)) { / ♦ try again ♦ / form_pk_val = 35; / ♦ reset value ♦ / peak_pos = get_next_peak( form_fac, εtart+peak_pos, end, &form_pk_val ) + 1; ) } if ( between(peak_poε, 4, 14) ) { / ♦ used to be 4, 10 ♦ / j=peak_pos; printf ( "Firεt εpec diff peak at %d\n", peak_poε ) ; if ( (mean_pwr_ratio = get_mean( low_to_mid, 0, j )) >= son_thresh) { printf ( "\t\tMean low-to-midpower ratio = %4.1f\n", mean_pwr_ratio ); while ( (low_to_mid[ ] > son_thresh) && (j < lmax_pos) ) ♦ prev_type = SONORANT; return (start+j); } / ♦ above doesn't work for l's ♦ / else { / ♦ try reducing position by 1 ♦ / if ( (mean_pwr_ratio = get_mean( low_to_mid, 0, j-1 )) >= son_thresh ) return (j+start) ; if ( (mid_maxpos - j > 3) | | (this_mid_pwr[j] < 0.2 i loc_mid_max) ) { *prev_type = SONORANT; return (j+start) ; } else { ♦ prev_type = ONSET; / ♦ label this portion as ONSET ♦ / return (j+start) ♦ 100; } } } else { printf( "get_next_peak returned value %d\n", peak_pos ) ; /* try other method ♦ / } ) / ♦ end if far_from_peak ♦ / j=0; end_so_far = 0; find_slopes( rest, pwr_100, sl_array ) ; /* printf("\t\t") ,- for (i=0; i<rest; i++) { 1 6 2 A printf("%3.If\t", sl_array[i] ) ; if (i % 4 == 0) printf("\n\t\t") ; } printf ( "\n" ) ; */ if ( (*prev_type == NUC_END) && (rest >= 4) ) { /* Must be a sonorant */ while ( FLAT(sl_array[j] ) && (j < typ_son_length) ) j++; ) end_so_f r = j ; / ♦ Not sure.... ♦ / sonorant = (this_mid_pwr[j] < pwrl) && ( (low_to_mid[j] > son_thresh) || FLAT(sl_array[j] ) ) ; while ( (sonorant) && (j<rest) &£_ (j-end_so_far < typ_son_length) ) { sonorant = (low_to_mid[j] > son_thresh) && ( (this_mid_pwr[j] < pwrl) || FLAT(sl_array[j] ) ) ; } if (local_debug) printf( "\t\tFinal value of j=%d; end so far=%d\n", j, end_so_far ) ; if (j==rest) { ♦ prev_type = SONORANT; return (end-1) ; } if (j>3) { ♦ prev_type = SONORANT; return (εtart+j-1) ; } else { if (end_so_far > 4) ♦ prev_type = SONORANT; / ♦ else: prev_type remains unchanged */ return (start+end_so_far) ; /* go looking for a Nucleus instead; ♦ / } } /* end find_SON */ ***************************** * **** *** * * **** * ************ ** * * * * *** * *****/ void a ign_ntype( short length, short tot_length, int reg_num, char *type_str ) / ♦ uses global parameters BR_NUCLEUS and BRN_IS_HEADED. General convention for code strings: changeable things are in lower case, non-changeable in UPPER case. ♦ / { short cutoff, rest = tot_length - length; BOOLEAN word_final = (rest <=2 && reg_num == FINAL) ; 1 63 if ( !BR_NUCLEUS ) { if (NN_SEQUENCE && length > Min_N2_length) strncpyf type_str, " SEQUENCE", 10 ); else strncpy( type_εtr, " NON_BR" , 8 ); return; } if ( !BRN_IS_HEADED ) { / ♦ BrNuclei are OK but do not have to be headed ♦ / cutoff = (Max_Nl_length + Min_N2_length) / 2; / ♦ = 14 ♦ / strcpy( type_str, (length > cutoff ? " BRANCHING" : " NON_BRANCHING" ) ) ; retur ; } / ♦ Now the case where Branching Nuclei must be headed: ie English ♦ / if (length < Max_Nl_length) strncpy( type_εtr, " non_branching" , 15 ) ; else { if (word_final) { strncpy( type_εtr, " BRANCHING", 12 ) ; return; ) if (length > Min_N2_length) /* assumes stressed word-final N is branching */ strncpy( type_εtr, " branching", 12 ); else strncpy( type_str, " unsure", 8 ) ; /* will be set by process_region */ } ) / ** * *** .**..... * ....... * ........*** ** . * ..... * * * . * . * . *** .. * .... * * * *..*.../ short find_PV( short v_length, float *voice, float *tot_pwr ) /* look for pre-voicing: short period of energy at or around F0 only */ { /* Not implemented yet: when it is, redo the code in parse_voiced where find_edge_nuc is called */ /* return 1; True ♦ / return 0; / ♦ False ♦ / ) int get_nuclei(int n_start, int end, float ♦ tot_pwr, float ♦ mid_pwr, float tot_max, float mid_max, int ♦ p^array) { register int j ; return 2; } 1 6 4 ******** * ******** * ********** * * * * * * ************ * ** * * ** *************** ** *********** / char ♦ parse_voiced( int reg_num, int start, int end, Feature *fea_set, int *uai, int ♦ peaks, short β prev ) 9ι I * Constraints on sequences: in general in a voiced region we must have alternating SON and NUCLEUS regions. (My voiced fricatives come out as Unvoiced). The trouble is the Nucleus can be empty, eg mem- ry, family, om+nis etc. At most 2 consecutive Sonorants are allowed, and then the preceding N must contain a stressed vowel. In Br Rhymes {lm, In) are possible but rare, and maybe non-existent in unstressed Rhymes (in English) ??? Rhymal consonants not important for now. Uses globals: max_power, maxpwr_mid, maxvoice_mid, Min_nuc_length. mid_pwr is now absolute, not %. If we want percent, work it out here. { register int j ; int n_start, r_start, n_end, r_end, reg_ct = 0, sub_num = 0, *epk_array, I * holds positions of nuclear peaks ♦ / last_used = 0, last_true = 0, / ♦ last position assigned to a region, used to check continuity of regions ♦ / e_code, / ♦ element code returned by process_region ♦ / speaker, / ♦ = MALE or FEMALE */ mid_maxpos, /* position of local max in mid_pwr */ loc_maxpos; /* position of loc_max in current region, relative to start ♦ / BOOLEAN all_done, all_voiced = False, / ♦ true if word is a single voiced region ♦ / early_max, / ■ true if the local max power value is in the first half has_stress, /* True if this region contains the overall maximum power value: doesn't get changed ♦ / stress, / ♦ true until main energy peak is passed ♦ / PV = False, / ♦ Prevoicing found ♦ / nuc_assigned = False, falling, GS_Candidate, GS_held = False; Feature *p; short length = (end - start / ♦ amount of region remaining ♦ / v_length = length+1, / ♦ size of array: 0..length ♦ / rest = length, / ♦ rest of array to be processed ♦ / last_reg = *prev, /* type of previous sub-region ♦ / 1 6 5 max_gap = 4, / ♦ size of maximum gap tolerated ♦ / Final_nuc_length = 6, / ♦ minimum duration of final NUC in region ♦ / Min_v_len = 7, / ♦ min. duration of real voiced region ♦ / nuc_type = ♦ prev, / ♦ can be any region type, plus NUC_END ♦ / seg_length, edge_pos, / ♦ result of find_edge_nuc ♦ / num_nuclei, / ♦ number of peaks in power curve(s) ♦ / peaks_past=0, / ♦ num of power peaks passed ♦ / nuc_ct, son_ct; / ♦ keeps current number of nuclei/sonorants */ float *kl, ♦ FO, ♦ voice, ♦ d_data, *form_fac, *tot_pwr, *mid_pwr, *pwr_ratios, / ♦ mid_power / tot_pwr ♦ / mean_F0, this_pwr, thiε_mid_pwr, loc_mid_max, / ♦ local max mid power ♦ / loc_max; / ♦ the local maximum total power value in this region ♦ / char ♦ type_str, ♦ name_str, ♦ nuc_str, ♦ seg_str, e_str[3] ,- printf("\tln Parse_voiced: region=%d\n" , reg_num ); type_str = (char ♦ )calloc(20, sizeof(char) ) ,- name_str = (char *)calloc(10, sizeof (char) ) ; seg_str = (char *)calloc (10, sizeof (char) ) ; kl = (float *)calloc(v_length, sizeof(float)) ; voice = (float *)calloc(v_length, εizeof(float) ) ; d_data = (float *)calloc(v_length, sizeof (float) ) ; form_fac = (float *)calloc(v_length, sizeof (float) ) ,- tot_pwr = (float *)calloc(v_length, sizeof(float) ) ; mid_pwr = (float ♦ )calloc(v_length, sizeof (float) ) ; FO = (float *)calloc(v_length, sizeof(float) ) ; p = fea_set + start; for ( j=0; j< v_length; j++) { / ♦ zcr[j] = p->zeros; ♦ / kl[j] = p->kl; voice[j] = p->voiced; / ♦ real f.p. values, not coded */ d_data[j] = p->d_data,- form_fac[j] = p->form_diff ♦ 100; tot_pwr[j] = p->power; mid_pwr[j] = p->midpower; F0[j] = p->F0; /* ♦ / 1 6 6 p++ ; } if ( ( ♦ prev==UNKNOWN) && (reg_num == FINAL)) { / ♦ whole word is 1 voiced region ♦ / all_voiced = True; sub_num = (int) ( floor( (float)v_length / 12) ); / ♦ thiε value varies A LOT ♦ / printf ( "\tAll Voiced: 1st est. no. of positions = %d\n" , sub_num) ; / ♦♦♦♦♦♦♦♦ Now look for peaks in power curve to get total number of nuclei ****/ ) / ♦ Get position and value of local power max ♦ / if (all_voiced) { loc_max = maxpower; loc_maxpoε = max_pwr_pos - start; loc_mid_max = maxvoice_mid; mid_maxpos = max_vmid_pos - start; has_stress = stress = True; ) else { loc_max = maxi ( tot_pwr, v_length, &loc_maxpos ) ; loc_mid_max = maxi( mid_pwr, v_length, &mid_maxpos ) ; fprintf( stderr, "\tLocal power max=%3.1f @ position %d\n" , loc_max, loc_maxpos ) ; if ((loc_max == maxpower) && (loc_mid_max >= 0.95 ♦ maxvoice_mid) ) { haε_stress = streεs = True; / ♦ only useful if not all_voiced) ♦ / fprintf ( stderr, "\tThis region contains stressed vowel!\n" ) ; ) else has_stress = stress = False; } if (all_voiced | | v_length > 15) { epk_array = (int ♦ )calloc(MAX_NUCLEI, sizeof (int) ) ; num_nuclei = get_nuclei(n_start, end, tot_pwr, mid_pwr, loc_max, loc_mid_max, epk_array) ; } else num_nuclei = 1; printf("\tEstimated number of Nuclei = %d\n" , num_nuclei) ; early_max = ( (loc_maxpos < length/2) && (mid_maxpos < length/2)) ? True: False; mean_F0 = get_mean( FO, 0, v_length ); fprintf( stderr, "\tLocal mean FO = %4.1f: ", mean_F0 ) ; if (mean_F0 > 160) { 1 6 7 speaker = FEMALE; printf( "Female\n" ) ; } else { speaker = MALE; printf( "Male\n" ) ; ) j=0; n_start = n_end = 0; r_start = r_end = 0; all_done = False; nuc_ct = 0; / ♦ Look for initial Glottal Stop in appropriate cases ♦ / GS_Candidate = (*prev==UNKNOWN) || (*prev==BURST) || ( (*prev==LOW_TONE) && (v_length > Min_v_len) ) ; if (GS_Candidate) { edge_pos = find_edge_nuc(n_start, rest, stress, loc_maxpos, mid_maxpos, tot_pwr, mi _pwr) ; if ( edge_pos != -1 ) { fprintf(εtderr, "\tGlottal Stop found (in Nucleus) \n") ; if (*prev == BURST) { printf( "\tGlottal Stop confirmed by find_edge\n") ; ) else { /* No previous BURST */ if (edge_poε < 4) { / ♦ assume no region before rising edge */ if (*prev == LOW_TONE) { strcat ( seg_str, "Z" ) ; *prev = SILENCE; } else { /* ie *prev == UNKNOWN ♦ / / ♦ if not word-initial, glottal stop OK, otherwise need other evidence since could be an empty onset with a phonetic glottal εtop ♦ / strcat( seg_str, "S" ) ,- ♦ prev = SILENCE; ) } else { / ♦ assume a (prob. pre-voiced) region before rising edge ♦ / if (PV = find_PV( edge_pos, voice, tot_pwr / ♦ ,spec_array ♦ / )) { *prev = LOW_TONE; strcat( seg_str, "Z." ) ; /* code for fully-voiced stop */ n_start = edge_pos; } else GS_held = True; / ♦ should find a sonorant first ♦ / / ♦ strcat(seg_str, "R."); ♦ / ♦ prev = ONSET; / ♦ ensure find_SON is called first */ 1 6 8 n_start = edge_pos ; } ) } else { / ♦ find_edge_nuc failed ♦ / if ( ♦ prev == BURST) { / ♦ We better find glottal stop... ♦ / printf ("\tUh-Oh: Glottal Stop not confirmed by find_edge\n") ; / ♦ might strip out the Sh sequence at thiε point ♦ / } } ) switch ( ♦ prev) { case EDGE: / ♦ shouldn't occur ♦ / printf( "\tWarning: previous region was EDGE n" ) ; strcat( seg_str, "S" ) ; / ♦ put in the missing S ♦ / case SILENCE: strcat( seg_str, " . " ) ; / ♦ put in the missing . ♦ / case LOW_TONE: / ♦ case SONORANT: : only occurs if glottal stop found ♦ / case ERROR: / ♦ so far this results only from pre-voicing ♦ / case BURST: / ♦ a Nucleus must follow ♦ / while ( !all_done ) { rest = length-n_start; if ((rest < Final_nuc_length) && (last_reg == SONORANT)) { if (reg_num != FINAL) { this_pwr = tot_pwr[n_start] ; this_mid_pwr = mid_pwr[n_start] ; for (j=n_start+l; j<v_length; j++) if ((tot_pwr[j] > this_pwr) && (mid_pwr[j] > thiε_mid_pwr) &._ (mid_pwr[j] > 0.2 4 maxvoice_mid) ) break; if (j < v_length) { printf( "\tNUCLEUS forced: start %d, end %d\n", start+n_start, end ); n_end = length; nuc_assigned = True; ♦ prev = NUCLEUS; ) ) ) else { n_end = find_NUC( n_start, rest, stress, loc_maxpos, mid_maxpos, tot_pwr, mid_pwr, d_data, form_fac, &nuc_type ) ; last_used = n_end; if ( (nuc_type == NUC_END) | | (nuc_type == EDGE))' / ♦ assign last_reg based on nuc_typeV last_reg = NUCLEUS; 1 6 9 else last_reg = nuc_type; ) if (n_end - n_start > Min_nuc_length) { / ♦ NUC found, min 50ms ♦ / nuc_asεigned = True; / ♦ this should be pretty rare: but anyway.... ♦ / if ( (last_reg == SONORANT) && (n_start - last_true > max_gap) ) { / ♦ Adjust start value to 1+end of SON region... ♦ / n_start = last_true+l; printf ( "\tStart of NUC adjusted to %d\n" , start+last_true+l ) ; ) printf ( "\tNUCLEUS: start %d, end %d:", start+n_start, start+n_end ) ; assign_ntype( (n_end - n_start + 1), v_length, reg_num, type_str ); printf( "%s\n", type_str) ; last_true = last_used; ♦ prev = last_reg; e_code=process_region( start+n_start, start+n_end, spec_array, peaks, uai, type_str ) ; / ♦ the above may change the br. vs non-br. decision in English ♦ / if (type_str[l] == 'b' ) strcat( seg_str, "N2;" ); else strcat( seg_str, "Nl;" ) ; itoa(e_code, e_str) ; strcat( seg_str, e_str ) ; εtrcat( seg_str, "." ) ; if (type_str[l] == 'S') { / ♦ nuclear Sequence maps on to N.O.N ♦ / strcat(seg_str, "O.N1;"); / ♦ put in empty onset ♦ / strcat(seg_εtr, e_str) ; strcat( seg_str, " . " ) ; ) / ♦ NB last_reg is assigned after call to find_NUC ♦ / ) if (nuc_type == EDGE ) { / ♦ special case: ? found ♦ / ♦ prev = EDGE; all_done = True; break; } if ( (v_length - n_end <=2) | | (n_end > v_length) ) { all_done = True; if (reg_num == FINAL) { / ♦ check for glottal stop ♦ / printf( "\tPretending to check for final ? ...\n" ); } break; } else { r_start = last_used+l; 17 0 rest = length-r_start; / ♦ Careful: only force a Sonorant if end of nucleus is certain ! ♦ / if ( early_max && (last_reg == NUCLEUS) __-_ (nuc_type == NUC_END) && between(rest, 4, final_son_length) ) { printf( "\tSONORANT forced"/ 4 , start+r_start, end */); r_end = length; name_str = "SONORANT"; } else { r_end = find_S0N( r_start, rest, speaker, loc_maxpos, tot_pwr, mid_pwr, d_data, form_fac, &nuc_type ) ; if (nuc_type == NUC_END) / ♦ should never be this ♦ / last_reg = NUCLEUS; else last_reg = nuc_type; if (r_end > 100) { name_str = "ONSET"; r_end /= 100; ) else name_str = "SONORANT"; last_used = r_end; } if (r_end - r_start >= 4) { / ♦ SON found, min 40ms ♦ / printf( "\t%s: start %d, end %d\n" , name_str, start+r_start, start+r_end ) ; if ( (r_start - last„true > max_gap) && (last_reg == NUCLEUS)) { / ♦ fill in gap ♦ / / ♦ Adjust value of end of NUC region... ♦ / printf ( "\tEnd of previous NUC adjusted to %d\n", start+r_start-l ); ) strcat( seg_str, (name_str[0] == 'S' ? "R. " : "0.")); last_true = last_used; last_reg = Λ prev = SONORANT; } if ( (v_length - r_end <=2) | | (r_end > v_length) ) { all_done = True; /* *prev = SONORANT; */ break; ) ) n_start = last_used+l; if ( (n_start > loc_maxpos) && nuc_assigned) /* we've passed the stressed sub-region and a nucleus has been assigned */ stress = False; 1 7 1 } / ♦ end of while */ if (laεt_used - last_true > max_gap) printf( "\tEnd of previous region adjusted to %d\n" , start+laεt_used ) ; break; case UNKNOWN: /* must be word-initial*/ if (reg_num != FINAL) { / ♦ whole word is 1 voiced region ♦ / printf ( "\tParsing initial voiced region... \n" ) ; if (v_length <= Min_v_len ) { / ♦ prob. pre-voicing ♦ / PV = find_PV( v_length, voice, tot_pwr ) ; if (PV) { printf( "Pre-voicing found. \n" ) ; ♦ prev = LOWJTONE; laεt_true = end; / ♦ don't write to the output εtring; taken care of in parse_unv() ♦ / } else last_true = end; / ♦ this prevents a default string being inserted later, since no nucleus is assigned ♦ / break; ) ) / ♦ don't break out, carry on ♦ / case ONSET: / ♦ a 'voiced' fricative, pre-voicing or tapped thing ♦ / case FRIC: / ♦ could be nucleus or sonorant next ♦ / while ( !all_done ) { rest = length-r_start; / ♦ Careful: only force a Sonorant if end of nucleus is certain ! ♦ / if (early_max && (last_reg == NUCLEUS) && (nuc_type == NUC_END) && between(rest, 4, final_son_length) ) { if (peaks_past == num_nuclei) { printf ( "\tSONORANT forced: "/*, start+r_start, end ♦ /),- r_end = length; name_str = "SONORANT"; } else { printf ( "\tONSET forced: " ) ; j=r_start; falling = True; while ((j < v_length) && falling) { falling = (tot_pwr[j] < tot_pwr[j-1] ) ; j++; / ♦ find start of rise in pwr curve ♦ / } printf( "j = %d: ", j ); r_end = j-1; last_true = last_used = r_end; 1 7 2 name_str = "ONSET" ; if (r_end - r_start < 4 ) { printf ( " %s : start %d, end %d\n" , name_str, start+r_start , start+r_end ) ; last_reg = # prev = ONSET; strcat (seg_str, "0. " ) ; } /* The >4 case is handled below in the normal way */ } } else { r_end = find_SON( r_start, rest, speaker, loc_maxpos, tot_pwr, mid_pwr, d_data, form_fac, &nuc_type ) ; if (nuc_type == NUC_END) / ♦ it should hardly ever be this ♦ / last_reg = NUCLEUS; else last_reg = nuc_type; if (r_end > 100) { r_end /= 100; name_str = "ONSET"; } else name_str = "SONORANT"; last_used = r_end; } if (GS_held) { last_used = r_end = edge_pos; name_str = (r_end >=4 ? "SONORANT" : "ONSET"); GS_held = False; / ♦ drop the GS once it's used ♦ / } seg_length = r_end - r_start; if ( (seg_length >= 4) | | (r_start < 2 && seg_length >= 2)) { / ♦ allow word-initial sonorants of min 20ms!! (see yy_nana) ♦ / printf ( "\t%s: start %d, end %d\n", name_str, start+r_start, start+r_end ) ; if ((r_start - last_true > max_gap) && ( 4 prev == NUCLEUS)) { /* Adjust value of end of NUC region to fill in gap... ♦ / printf( "\tEnd of previous NUC adjusted to %d\n", start+r_start-l ) ; } last_true = last_used; last_reg = 4 prev = (name_str[0] == 'O' ? ONSET : SONORANT); strcat( seg_str, (name_str[0] == '0' ? "O." : "R.")); } if ( (v_length - r_end <=2) | | (r_end > v_length) ) { all_done = True; / ♦ *prev = SONORANT; ♦ / 1 7 7> break; ) else { n_start = last_uεed+l; rest = length-n_start; if (stress && (n_start > loc_maxpoε) && nuc_assigned) / ♦ we've passed the stressed sub-region ♦ / stress = False; if ( (rest < Final_nuc_length && peaks_past < num_nuclei) && (last_reg NUCLEUS) ) { if ( !stress /*reg_num != FINAL 4 /) { this_pwr = tot_pwr[n_start] ; this_mid_pwr = mid_pwr[n_start] ; printf("\tThis_pwr = %4.1f; this_mid_pwr = %4.1f\n", this_pwr. this_mid_pwr) for (j=n_start+l; j<v_length; j++) if ((tot_pwr[j] > this_pwr) && (mid_pwr[j] > this_mid_pwr) && (mid_pwr[j] > 0.2 4 loc_mid_max) ) break; if (j v_length) { printf( "\tNUCLEUS forced''/^ start %d, end %d\n", star +n_start, endV ) ; n_end = v_length; nuc_assigned = True; ) ) else { / ♦ FINAL region: Don't get into an infinite loop ! '/ ) ) else { n_end = find_NUC( n_start, rest, stress, loc_maxpos, mid_maxpos, tot_pwr, mid_pwr, d_data, form_fac, &nuc_type ) ; last_used = n_end; if ( (nuc_type == NUC_END) | | (nuc_type == EDGE) ) / ♦ assign last_reg based on nuc_type ♦ / last_reg = NUCLEUS; else last_reg = nuc_type; ) if (n_end - n_start >= Min_nuc_length) { nuc_assigned = True; peakε_paεt++; printf( "\tNUCLEUS: εtart %d, end %d:", start+n_start, start+n_end ) ; assign_ntype( (n_end-n_start+l) , v_length, reg_num, type_str ) ; printf( "%s\n", type_str ) ; ♦ prev = NUCLEUS; . /* NB last_reg is assigned by find_NUC ♦ / last_true = last_used; e_code - process_region(start+n_start, start+n_end, spec_array, peaks, uai, type_str) ; /* the above may change the br. vs non-br. decision ♦ / strcat( seg_str, (type_str[l] == 'b' ? "N2;" : "Nl;") ); itoa( e_code, e_str ) ; strcat( seg_str, e_str ); strcat( seg_str, "." ) ; if (type_str[l] == 'S') { strcat(seg_str, "O0.N1;"); / ♦ put in empty onset ♦ / strcat(seg_str, e_str) ; strcat( seg_str, "." ); ) } if (nuc_type == EDGE) { /* special case: ? found */ *prev = EDGE; all_done = True; break; ) if ( (v_length - n_end <=2) | | (n_end > v_length) ) { ll_done = True; ♦ prev = last_reg; break; ) } r_start = last_used+l; /* safer than n_end+l, since n_end may not get assigned */ ) /* end of while */ if (last_used - last_true > max_gap) / ♦ check for only R ♦ / printf ( "\tEnd of previous region adjusted to %d\n", start+last_used ),- break; default: return "FAIL"; ) /* end switch ♦ / /♦♦♦******** Repair strategy: if no nucleus assigned: ****************/ if ( (last_true == 0) | | !nuc_assigned / ♦ && (last_true != end) ♦ / ) { printf( "No %s assigned: ", (nuc_assigned ? "regions" : "nucleus") ); if (reg_num == 0) { / ♦ probably pre-voicing ♦ / printf(" assuming Pre-voicing\n") ; ♦ prev = LOW_TONE; ) else { if (has_stress) { if (v_length <= 12) { / ♦ single nuc ♦ / 175 _ printf( "assuming single Nucleuε\n" ); assign_ntype( v_length, v_length, reg_num, type_εtr ); ♦ prev = NUCLEUS; e_code = process_region( start, end, spec_array, peaks, uai, type_str ) ; strcat( seg_str, (type_str[l] == 'b' ? "N2 ; " : "Nl;") ) ,- itoa( e_code, e_str ) ; strcat ( seg_str, e_str ); strcat( seg_str, "." ); ) else { / ♦ longer than 12 : could do better than this ♦ / printf( "too long to estimate\n" ) ; if ( !nuc_assigned && last_true > 0) { /* ie only an R ♦ / strcat( seg_str, "N1;0." ); ♦ prev = NUCLEUS; ) else { strcat( seg_str, "N1;0.R." ); ♦ prev = SONORANT; } } ) else { / ♦ unstressed nuc ♦ / printf ( "assuming unstressed rhymeXn" ); if (v_length <= 10) { strcat( εeg_str, "N1;0." ); ♦ prev = NUCLEUS; ) else { strcat ( seg_str, "N1;0.R." ) ; ♦ prev = SONORANT; ) } } } strcat( seg_str, "\0" ) ; printf ( "\tSegment string is %s\n" , seg_str ) ; / ♦ fprintf ( fd, "%s", seg_str ) ; ♦ / return (seg_str) ; } int min_sil_len = 4, / ♦ i.e. 5 x 10 = 50ms ♦ / min_fric_len = 6; / ♦ i.e. 60ms ♦ / * * ** * * * * * * * * * * * * ** * * * * * * * * * * * * * UNVOICED stuf f * * *************************** * *****/ int find_FRIC( int start, int rest, float ♦ zcr, float ♦ kl, short fric_type ) { . /♦ return the first value after start such that no h is present, start and rest are relative to starting pos of input arrays, start + rest = length of arrays ♦ / register int j ; int axdex; short local_debug = 1, head_ct = 0, / ♦ no of points s.t. h= Head ♦ / op_ct = 0, / ♦ s.t. h= Operator ♦ / end = start+rest, max_index = end-1; float Zcr, kval, max_zcr; BOOLEAN h_found, burst = False, h_head, h_operator; float burst_min = 6000, zcrh=5000, zcr_med=2000; printf ("\t\tln find_FRIC: start=%d, rest=%d\n" , start, rest ); Zcr = max(zcr[start] , zcr[start+1] ) ; / ♦ check for transition region ♦ / kval = min(kl[start] , kl[start+l] ) ; j = 1; h_found = h_head = h_operator = False; h_found = (Zcr > zcr_med) || (kval < 0.8); while ( h_found && (j < rest) ) { j++; Zcr = zcr[start+j] ; kval = kl[start+j]; h_head = (Zcr > zcrh) && (kval < 0) ; if (h_head) head_ct++; else { h_operator = (Zcr > zcr_med) || (kval < 0.5); if (h_operator) op_ct++; } h_found = h_head | | h_operator; ) ♦ fric_type = ( (head_ct > op_ct) ? HEAD : OPERATOR); max_zcr = sub_max( zcr, start, end, &maxdex ) ; if (local_debug) fprintf(stderr, "\t\tValue of j = %d; Max ZCR value = %4.0f\n", j, max_zcr ) ; burst = (max_zcr > burst_min) ; if (j == rest) { 1 7 7 *fric_type += (burst ♦ BURST); / ♦ add burst code to fric_type ♦ / return (end) ; } if (j>=2) { if ( rest < 10 ) { if ( get_mean(zcr, start, max_index) > zcr_med ) ♦ fric_type = OPERATOR; if ( get_mean(zcr, start, max_index) > zcrh ) ♦ fric_type = HEAD; ♦ fric_type += (burst ♦ BURST); / ♦ add burst code to fric_type */ return (end) ; } else return (start+j); ) else return (εtart+1) ; / ♦ to make sure we don't loop for ever ♦ / } BOOLEAN process_fric( short length, short *prev, short *fric_type, char ♦ fric_string, char ♦ seg_str ) { BOOLEAN burst = False, h_found = False; int char_ct = 0; short f_type = ♦ fric_type; if ( (f_type-BURST == HEAD) || (f_type-BURST == OPERATOR)) { burst++; f_type -= BURST; ) if (length >= min_fric_len) { strncpy(fric_string, "FRIC ", 6) ; if ( 4 prev == SILENCE) { " if (f_type == HEAD) { seg_str[char_ct++] = '-'; / ♦ result is S-F ie affricate or {k,t)**|-s */ seg_str[char_ct++] = 'F'; ) else seg_str[char_ct++] = 'h' ,- / ♦ long burst => ?+h ♦/ } else { / ♦ prev not Silence ♦ / if ( 4 prev != LOW_TONE && ^prev != UNKNOWN) seg_str[char_ct++] = (f_type == HEAD ? 'F' : ' f' ) ; else seg_str[char_ct++] = ( (length<10 || *τprev == FRIC) ? 'f : (f-type == HEAD ? 'F' : 'f' ) ) ; ) /* h can't be the head of stop, so just use f->128, in case we missed a ? ♦ / /* NB pre-voiced Fricatives give 'Lf only: ♦ LF! ! ♦ / seg_str[char_ct++] = ' . ' ; / ♦ strcat( seg_str, ♦ / h_found = True; ♦ prev = FRIC; } else { / ♦ duration of friction < min_fric_len */ if (length >= 2) { / ♦ might be a burst ♦ / /**** Make this a switch statement: UNKNOWN->Sh *********/ switch ( 4 prev) { case UNKNOWN: / ♦ if (f_type == HEAD)V if (burst) seg_str[char_ct++] = 'S'; case SILENCE: case LOW_TONE: if (burst) { strncpy(fric_string, "BURST ", 8); *prev = BURST; seg_str[char_ct++] = 'h'; } else { strncpy(fric_string, "FRIC ", 6) ; *prev = FRIC; seg_str[char_ct++] = (f_type == HEAD ? "F * : 'f • ) ; ) break; default: /* fric > 2 not following Silence or L */ strncpy(fric_string, "FRIC ", 6) ; ♦ prev = FRIC; seg_str[char_ct++] = (f_type == HEAD ? 'F' : 'f'); break; } / ♦ end switch ♦ / seg_str[char_ct++] = h_found = True; } / ♦ else no h_found ♦ / ) seg_str[char_ct++] = '\0'; ♦ fric_type = f_type; return (h_found) ; } /* end of process_fric ♦ / /************** * **** * *** * ********** * *** * * * * *** ** 1 7 9 int find_SIL( int start, int rest, float ♦ power, float ♦ zeros, float ♦ mid_pwr, float *kl ) { /* return first position s.t. not in a silence region start + rest = length of arrays tot_pwr[] is absolute, mid_pwr[] is normalised */ register int j; int end = start+rest, s_start = start; short local_debug = 1; BOOLEAN k_hi; float mean_pwr, mean_mid_pwr, mean_zcr, mean_pwr_ratio; float pwrl=3000, / ♦ absolute ♦ / mid_pl = 1.0, / ♦ percent ♦ / kl_thresh = 0.68; printf("\t\tln find_SIL: start=%d, rest=%d\n", start, rest ); / ♦ Try the whole of the rest of the region first ♦ / mean_pwr = get_mean( power, start, end ) ; /♦ ♦ / mean_mid_pwr = get_mean( mid_pwr, start, end ) ; mean_zcr = get_mean( zeros, start, end ) ; if (local_debug) fprintf (stderr, "\t\tMean midpower = %3.2f\n", mean_mid_pwr) ; if (rest < 11) { / ♦ ie duration !>100ms ♦ / if ( (mean_mid_pwr < mid_pl) && (mean_zcr < 1000)) return (end) ; } /* else ♦ / j=start; k_hi = (kl[j++] > kl_thresh) | | (kl[j++] > kl_thresh) ; if (k_hi) s_start = j-1; while ( k_hi && (j<end) ) { / ♦ j++; */ k_hi = (kl[++j] > kl_thresh) || (kl[++j] > kl_thresh) ; ) if (local_debug) fprintf (stderr, "\t\tValue of j = %d\n", j); if ((j-start >= 4) || (j==end) ) { mean_mid_pwr = get_mean( mid_pwr, s_start, j-1 ) ; if (mean_mid_pwr < mid_pl) return (/*j==end ? ♦ / j ) ; else { fprintf("\t\tFailed mid power test\n"); return (start) ; } 1 8 0 } else return (start) ; } char parse_unvoiced ( int reg_num, int start , int end, Feature ♦ fea_set , short ♦prev, char β seg_str ) { / ♦ relevant features are: kl value, zcr, specdiff ♦ / / ♦ TODO: decide what to do with regions shorter than min_si2 len. eg my [v] in seven. At the moment they get labelled as Onsets ♦ / register int j, ct; BOOLEAN all_done, h_present, h_found, burst = False; Feature ^p; short length = (end + 1 - start) , max_pos = length-1, char_ct = 0, / ♦ counter for segment string */ no_left = 3, /* min no of records left before all done */ f_type; /* type of fricative: h = HEAD or OPERATOR ♦ / int f_start, s_start, f_end, s_end, reg_ct = 0, v_end = 0, / ♦ position of end of LowTone region ♦ / L_length = 5; / ♦ min. period of Low Tone = 50ms */ float ♦ zcr, ♦ kl, ♦ fO, ♦ tot_pwr, ♦ mid_pwr; float zcrl=2000, /* lower and upper thresholds for zc and power ♦ / pwr_fric=4500, / ♦ upper power threshold for fricatives ♦ / zcrO = 600, /* upper zc threshold for silence ♦ / zcrh=4000, pwrh=80.0, / ♦ zc's are absolute, powers are % ♦ / zcr_med=2000, / ♦ interesting zc level */ zcr_top=7000, / ♦ definitely friction threshold ♦ / pwr_son = 20000, /* lower power threshold for sonorants ♦ / L_min_pwr = 5.0E4, /* min mean power for Low TOne ♦ / change, mean_zcr, mean_pwr, ♦ voice; char ♦ fric_string, / ♦ Type of fricative: FRIC or BURST */ out_str[10]-; printf("\tln Parse_unvoiced: region=%d\n" , reg_num ); fric_string = (char ♦ )calloc(10, sizeof(char) ) ; voice = (float ♦ )calloc(length, sizeof(UINT) ) ; 1 8 1 zcr = (float *)calloc(length, sizeof (float) ) ; kl = (float *)calloc (length, sizeof (float) ) ; fO = (float *)calloc (length, sizeof (float) ) ; tot_pwr = (float ♦ )calloc(length, sizeof(float) ) ; mid_pwr = (float ♦ (calloc (length, sizeof (float) ) ; p = fea_set + start; for ( j=0; j< length; j++) { / ♦ arrays are relative to start */ voice[j] = p->voiced; /* real f.p. values, not coded */ zcr[j] = p->zeros; kl[j] = p->kl; fO[j] = p->F0; /* d_data[j] = p->d_data; */ tot_pwr[j] = p->power; mid_pwr[j] = p->midpower / maxpwr_mid; /* normalise */ P++; ) j=0; f_start = f_end = 0; s_start = s_end = 0; all_done = h_found = False; f_type = UNKNOWN; char_ct = 0; h_present = (zcr[j] > zcr_med) && (kl[j] < 0.8); /* j++; */ if ( (reg_num == 0) | | ( ♦ prev == UNKNOWN)) { / ♦ we're not interested in silence firstV if ( (mean_pwr = get_mean(tot_pwr, 0, length)) < 1000 ) { printf ( "\tMean power = %3.2f: \n", mean_pwr ); printf ( "SILENCE in Region 0: ignoring\n") ; ♦ prev = UNKNOWN; seg_str[char_ct++] = 'X',- seg_str[char_ct] = '\0'; / ♦ Terminator ♦ / return ('X'); / ♦ return an exit code ♦ / } else printf ( "\tMean power = %3.2f: \n", mean_pwr ); while ( (!h_present) && ( j<max_pos ) ) { j++; h_present = (zcr[j] > zcr_med) | | (kl[j] < 0.8); ) /* Check for pre-voicing */ f_start = (zcr[j-1] > (zcr_med/2) ? —j : ) ; if (j-1 >= L_length) { v_end = j-1; mean_pwr = get_mean(tot_pwr, 0, v_end) ; if (mean_pwr > L_min_pwr) /* && (get_mean(f0) < typical_f0) ♦ / { 1 8 2 / ♦ must be Pre-voicing: join this region with next Voiced region ♦/ printf ( "\tLow Tone! " ); / ♦ now find the ε*:.art: prob_voice > 0.2 ♦ / for (j=0; j<length; j++) if (voicefj] > 0.2) break; printf( "start=%d, end=%d\n", j, v_end ) ; if ( v_end > (max_pos-2) ) { / ♦ end of region ♦ / ♦ prev = v_end; seg_str = "" ; / ♦ Put in L- later */ return( 'L' ) ; } else { / ♦ Some h left ♦ / seg_str[char_ct++] = 'L',- ♦ prev = LOW_TONE; f_start = v_end+l; } ) else { printf ( "\tSILENCE in Region 0: ignoring\n" ); ♦ prev = UNKNOWN; seg_str = "X"; / ♦ This is just silence, not necess. ? stop ♦ / return ('X'); / ♦ return an exit code ♦ / } } / ♦ f_start = j; ♦ / while ( !all_done ) { h_found = burst = False; f_end = find_FRIC( f_start, (max_pos-f_start) , zcr, kl, &f_type ) ; h_found = process_fric( f_end-f_start, prev, &f_type, fric_string, out_str ) ; if (h_found) { printf ( "\t%s start: %d, end: %d. ", fric_string, start+f_start, start+f_end ) ,- if (f_type == HEAD) printf( "h. is a Head!!\n") ; else printf( "h. is an Operator!\n" ) ; for(ct=0; ct<strlen(out_str) ; ct++) seg_str[char_ct++] = out_str[ct] ; } if ( length - f_end <=2 ) { all_done = True; break; ) else { s_start = f_end+l; s_end = find_SIL( s_start, max_pos-ε_star , tot_pwr, zcr, mid_pwr, kl ); if ( (s_end - s_start >= min_sillϊen) && { *pτev '. = UNKNOWN)) { printf ( "\tSILENCE start: %d, end: %d - ? found! \n", start+s_start, start+s_end ) ; ♦ prev = SILENCE; / ♦ strcat ( seg_str, "S" ) ; ♦ / / ♦ fprintf (stderr, "char ct = %d\n", char_ct ); ♦ / seg_str[char_ct++] = 'S'; } else s_end = s_start; / ♦ don't increment pointer in file ♦ / if ( length-s_end <=1 ) { all_done = True; ♦ prev = SILENCE; / ♦ prob'ly redundant, but just in case ♦ / break; } } f_start = s_end+l; ) / ♦ end of while loop ♦ / ) else { / ♦ not the first region in the word: includes the EDGE case ♦ / while ( !all_done ) { s_end = find_SIL( s_start, (max_pos-s_start) , tot_pwr, zcr, mid_pwr, kl ) ; if ( ( (s_end - s_start) >= min_sil_len) | | ( 4 prev == EDGE) || ( 4 prev == LOW_TONE) ) { if (reg_num != FINAL) { printf( "\tSILENCE start: %d, end: %d - ? found! !\n", start+s_start, start+s_end ) ,- seg_str[char_ct++] = ( ♦ prev==LOW_TONE ? 'Z' : 'S'); ♦ prev = SILENCE; } else { / ♦ FINAL region ♦ / if (length - s_end >= no_left) { / ♦ ie not all done yet ♦ / printf( "\tSILENCE start: %d, end: %d - ? found! !\n", start+s_start, start+s_end ) ,- / ♦ strcat( seg_str, "S" ) ; ♦ / ♦ prev = SILENCE; / ♦ fprintf(stderr, "char ct = %d\n", char_ct ); ♦ / seg_str[char_ct++] = 'S'; } / ♦ if whole region is silence, ignore it ♦ / if ( (s_end - s_start) == max_pos ) { if (*prev == EDGE) { 'seg_str[char_ct++] = ' ? ' ; seg_str[char_ct++] = ' . ' ; } else { printf( "\tIgnoring final Silence region\n"); 1 84 seg_str = "X" ; return CX'); } ) ) / ♦ end final region case ♦ / ) else / ♦ find_SIL failed ♦ / s_end = s_start; / ♦ don't increment pointer in file ♦ / if ( length - s_end <=no_left ) { ii ( (*prev == SILENCE) && (length-s_end >=2) ) { printf( "\tBURST start: %d, end: %d\n", start+s_end+l, end ) ; ♦ prev = BURST; / ♦ strcat( seg_str, "h." ) ; ♦ / / ♦ fprintf(stderr, "char ct = %d; ", char_ct ); ♦ / seg_str[char_ct++] = 'h'; seg_str[char_ct++] = ' . ' ,- } all_done = True; break; } else { f_start = (s_end > 0) ? (s_end+l) : 0; h_found = burst = False; f_end = find_FRIC( f_start, (max_pos-f_start) , zcr, kl, &f_type ); h_found = process_fric( f_end-f_start, prev, &f_type, fric_εtring, out_str ) ; if (h_found) { printf ( "\t%s start: %d, end: %d. ", fric_string, start+f_start, start+f_end ) ; if (f_type == HEAD) printf ( "h. is a Head!!\n") ; else printf( "h. is an Operator!\n") ; for(ct=0; ct<strlen(out_str) ; ct++) seg_str[char_ct++] = out_str[ct] ; } if ( (reg_num == FINAL) && (h_found) ) if ( length-f_end <=5 ) { fprintf(stderr, "\t\tIgnoring the rest\n"); all_done = True; break; } if ( length-f_end <=2 ) { all_done = True; break; } 1 8 5 } s_start = f_end+l; ) /* end of while loop */ } / ♦ strcat( seg_str, "\0" ) ; ♦ / / ♦ fprintf(stderr, "\tchar ct = %d; ", char_ct ) ; ♦ / / ♦ Check for empty strings: only do this for word-internal regions ♦ / if ( (char_ct == 0) && <reg_num != 0) && (reg_num != FINAL)) { if (length > 3) { seg_str[char_ct++] = '0'; / ♦ label as Onset ♦ / seg_str[char_ct++] = ' . ' ; / ♦ label as Onset ♦ / ♦ prev = ONSET; ) else { / ♦ don't change *prev; send a code back to find_regions to ignore this region */ printf ("\tlnvalid unvoiced region\n"); return ( 'X' ) ; } ) /* strncpy( out_str, seg_str, 10 ) ; safe copy ! printf ( "\tOutput string is %s\n", out_str ) ; fprintf( *fd, "%s", out_str ) ; */ if (*prev != UNKNOWN) { εeg_εtr[char_ct] = '\0'; /* Terminator ♦ / printf( "\tSegment εtring is %s\n", seg_str ) ; return (0) ; ) else { fprintf(stderr, "\tNon-Speech: ignoring this region! \n") ; seg_str[char_ct++] = 'X'; seg_str[char_ct] = '\0'; /* Terminator */ return ( 'X' ) ; ) } /*************** end of Parse_unvoiced * * * ** * ** * ** * ** * * * ***** ** *** / int check_εilence( int εtart, int end. Feature *fea_set ) { register int j; int length = end-start, local_debug = 1; Feature *p; BOOLEAN k_hi, unvoiced, Silence; 1 8 6 float kl_thresh = 0.6; if (local_debug) printf("\tChecking silence...start=%d, end=%d; ", start, end); if (length > 15) return False; p = fea_set + start; j=start; unvoiced = (p->voiced != VCD) || ( (p+1)->voiced != VCD); k_hi = (p->kl > kl_thresh) || ((p+l)->kl > kl_thresh) ; / ♦ never increments p • ' Silence = k. && unvoiced; p++; / ♦ NB this will fail in unlikely event that p->kl succeeds but p+l->kl fails ♦ / while ( Silence && (j<end-l) ) { k_hi = (p->kl > kl_thresh) ; unvoiced = ( (p++)->voiced != VCD); / ♦ increment p after testing value ♦ / Silence = k_hi && unvoiced; j++; ) if (j==end-l) { printf ("\n") ; return True; ) else { printf("no: position=%d\n" , j) ; return False; } } char ♦ find_regions( float ♦ diffpeaks, Feature ♦ fea_set, char names[] [NAMESIZE] , int ♦ uai, int ♦ peaks ) / ♦ outputs a list of regions, with boundaries, based on VOICED or UNVOICED ♦ / { static FILE *fp; register int j=0, n=0; char ♦ str, *v_str, Strl[NAMESIZE] , *seg_str; / ♦ stores the sequence of segment labels ♦ / / ♦ static char ♦ ofile_name = "/home/HAL/ipl/progs/data/default.fil" ; ♦ / BOOLEAN all_done; short prev_type, / ♦ coded type of previous (sub-)region: 0=FRIC, 1=SILENCE, 2=NUC, 3=SONORANT ♦ / 1 8 7 rval=0, err_ct=0, Max_final_sil = 4; / ♦ longest found so far ♦ / int ch, v_start, uv_start, v_end, uv_end, reg_ct = 0, final_pk_pos; Feature *p; float *zcr, voice_val, Zcr_hi = 2600; UINT ♦ voice; final_pk_pos = diff_peakε[num_boundε-1] ; if (final_pk_pos < (num_rec-10) ) printf ( "find_regions: Warning - final peak iε a long way from end of file!\n") ; εtr = (char *)calloc(20, sizeof(char) ) ; v_str = (char *)calloc(20, sizeof(char) ) ; seg_str = (char *)calloc(80, εizeof(char) ) ; voice = (UINT *)calloc(num_rec, sizeof (UINT) ) ; zcr = (float *)calloc(num_rec, sizeof(float) ) ; p = fea_set; /* fprintf( stderr, "Find_regions: Voice values: " ) ; 0 0.25 0.8 1.0 I UNV I TRANSITION | VCD | */ for (j=0; j< (int) num_rec; j++) { voice_val = p->voiced; if (voice_val > 0.8) voicefj] = VCD; else { if (voice_val < 0.25) voicetj] = UNV; else voicetj] = TRANSITION; } / ♦ fprintf( stderr, "%d: %d\t", j, voice[j] ); ♦ / zcr[j] = p->zeros; P++; } / ♦ first patch up any errors in the voicing data ♦ / j=l while (j < num_rec ) { if ( (voice[j-1] == UNV) && ((voicetj] == VCD) (voice[j] == TRANSITION)) ) { if ( zcr[j] > Zcr_hi ) 1 8 8 voicetj] = UNV; } } / ♦ if ((fp = fopen( ofile_name, "w+t" )) != NULL) { printf( "File %s opened OK\n", ofile_name ); } else { printf( "Failed to open output file in find_regions\n" ) ; return ("FAIL"); ) ♦ / j=0; prev_type = UNKNOWN; all_done = False; uv_start = v_start = 0; while ( !all_done ) { / ♦ assume UNVoiced first ♦ / while ( (j < num_rec) && (voice[j] != VCD) ) j++; if ( (j == num_rec) | | (j >= final_pk_pos) ) { if (reg_ct == 0) { fprintf(stderr, "find_regions: Error: no voiced region found! ! \n" ) ; all_done = True; free(voice) ; return ("FAIL"); } else { / ♦ reg_ct > 0, close to end of array ♦ / uv_end = j-1; all_done = True; if ( (uv_end-uv_start < Max_final_sil) || check_silence(uv_start, num_rec, fea_set) ) fprintf(stderr, "Ignoring invalid final 'U V" region\n"),- else { fprintf(stderr, "Region %d: UNVOICED; start=%d, end=%d\n", reg_ct, uv_start, uv_end ) ; ch = parse_unvoiced( FINAL, uv_start, uv_end, fea_set, &prev_type, str ); if ( ch == 'X' ) { fprintf(stderr, "Deleting Region %d\n" , reg_ct) ; / ♦ strcat( seg_str, "\0" ); ♦ / ) else seg_str = strca ( seg_str, str ) ; ) break; ) } v_start = j ; /* start of voiced region ♦ / 1 8 9 if ( j >0 ) { uv_end = ( j -1 ) ; fprintf(εtderr, "Region %d: UNVOICED; εtart=%d, end=%d;", reg_ct, uv_start, uv_end ) ; if ( (reg_ct == 0) && (uv_end-uv_start < 3)) fprintf (stderr, " ignoring invalid regionXn"); else { fprintf (stderr, "\n"); ch = parse_unvoiced( reg_ct, uv_start, uv_end, fea_set, &prev_type, str ) ; if ( ch != 'X' ) { / ♦ good news ♦ / if (ch == 'L' ) { v_start = prev_type; / ♦ naughty: prev_type is used to pass new starting position ♦ / prev_type = LOW_TONE; /* or ONSET? */ if (reg_ct != 0) seg_str = strcat( seg_str, "?." ) ; ) else if (ch == 0) { reg_ct++; seg_str = εtrcat( seg_str, εtr ); } ) else { /* do nothing: careful, prev_type might be invalid for voiced region*/ /*err_ct++; εtrcat( εeg_εtr, str );*/ prev_type = ERROR; ) /* all_done = True; */ } /* outer else */ ) while ( (j < num_rec) && (voicetj] != UNV) ) j++; / ♦ check for glitches in voicing curve ♦ / if ( (j < num_rec-l) && (voice[j+1] != UNV) ) { j++; while ( (j < num_rec) && (voicetj] != UNV) ) j++; ) v_end = j-1; fprintf(εtderr, "Region %d: VOICED; start=%d, end=%d\n" , reg_ct, v_start, v_end ) ; if ( (j >= num_rec-3) || ( (j>=num_rec-6) && check_silence(j, num_rec, fea_set)) ) { / ♦ look_ahead! careful with final voiced fricatives ♦ / v_str = parse_voiced( FINAL, v_start, v_end, fea_set, uai, peaks, &prev_type ); 1 9 0 all_done = True; } else { uv_start = j; if (v_end-v_start > 2) { v_str = parse_voiced( reg_ct, v_start, v_end, fea_set, uai, peaks, &prev_type ) ; reg_ct++; } else strcpy( seg_str, "" ) ; ) if ( strstr(v_str, "FAIL") == NULL) seg_str = strcat( seg_str, v_str ) ; else { printf ( "Find_regions: error in parse_voiced\n" ); err_ct++; ) } / ♦ all done ♦ / / ♦ strcat( seg_str, "\0" ) ; ♦ / printf ( "Segment string is ♦%s^\n" , seg_str ) ; / ♦ if ((fp = fopen( ofile_name, "w+t" )) != NULL) { printf( "File %s opened OK\n" , ofile_name ) ; if (fputs( seg_str, fp ) != EOF) printf ("Segment string ouput to file OK\n"); else printf ("Error writing string to ouput fileXn"); j = 0; while ( (ch = seg_str[j]) > '\0') { fprintf( fp, "%c", ch ); ) printf ( "\nSegment string copied to output fileXn" ) ; fclose( fp ) ; } ♦ / if (err. r ) { print. ( "%d error(s) occurred in find_regions\n" , err_ct ); return ("FAIL"); } fprintf (stderr, "find_regions: input exhausted. \n") ; free( voice ) ; return (seg_str) ; ) 1 9 1 / ♦ parse_seg.c Part of program rec_word. Converts a segment string (consisting of F|S(h) |h|R|N(2) ) into a search string for lexical access using Icon program 'connect'. Function parse_seg_str is called by get_syl2.c and returns the search string as a character pointer. Can be compiled and run separately by de-commenting mainO and using: gcc -o parseng eng_pars.c parεe_seg.c parsjap jap_pars.c parskor kor_pars.c " */ / ♦ Usage: parse_seg "seg_εtr" ofile language ♦ / ♦include <stdio.h> ♦include <εtdlib.h> ♦include <string.h> ♦define FINAL 100 ♦define MAXTOKLEN 6 ♦define False 0 ♦define True 1 ♦if !defined( BOOLEAN) typedef unsigned int BOOLEAN; ♦endif extern BOOLEAN BR_ONSET, BR_NUCLEUS, BR_RHYME, DF_LE ; extern BOOLEAN HV_DEVOICED, SON_STOP; void reverse(char s[]); void itoafint n, char *ε ) ; char ♦ get_next_token( char ♦ word, int ct ) ; char ♦ build_onset( char ♦ seg, char ♦ seg_str, int syl_ct ) ; char ♦ build_rhyme( char ♦ seg, char ♦ seg_str, int syl_ct ) ; char ♦ parse_seg_str( char ♦ seg_str, char lang ) ; BOOLEAN Japanese=False, Korean=False, Greek=False, Default=False; int ct, token_num=0; short last_tok = False; 1 92 int debug = 0 ; /* void main(int argc, char ♦ argvt] ) { FILE ♦ fp; char ♦ out_str, ♦ lang_name; char lang; out_εtr = (char ♦ )calloc(64, sizeof(char) ) ; if (argc == 4) { lang = argv[3] [0] ; switch (lang) { case 'j ' : case ' ' : lang_name = "Japanese"; break; case 'K' : case 'k': lang_name = "Korean"; break,- case 'G' : case 'g': lang_name = "Greek"; default: lang_name = "Default"; break; ) if (debug) printf ( "Language is %s\n", lang_name ); if ( (fp = fopen(argvt2] , "wt")) != NULL ) { printf( "Input string is %s\n", argv[l]); strcat(argv[1] , "\0"); out_str = parse_seg_str( argv[l] , lang ) ; } else { fprintf(stderr, "Error opening output fileXn"); exit (1); } } else { fprintf(stderr, "ERROR: wrong number of argumentsXn") ; fprintf(stderr, "Usage: parsexxx X"seg_str\" ofile languageXn") ,- exit (1); } printf( "Output string is %s\n", out_str ); fprintf( fp, "%s", out_str ) ; fclose( fp ) ; printf("OKXn") ; } 1 9 3 / ♦ reverse: reverse string s in place. F σ_n K&R page 62 ♦ / void reverse(char s[]) { int c, i, j ; for (i=0, j=εtrlen(ε)-1; i < j; i++, j—) { c = s [i] ; s[i] = s[j] ; s[j] = c; } } / ♦ itoa: convert n to characters in s; from K&R page 64 ♦ / void itoa(int n, char ♦ s) { int i, sign; if ((sign = n) < 0) / ♦ record sign ♦ / n = -n; / ♦ make n poεitive ♦ / i = 0; do { / ♦ generate digits in reverse order ♦ / s[i++] = n % 10 + ' 0 ' ; / ♦ get next digit ♦ / } while ( (n /= 10) > 0); /* delete it ♦ / if (εign < 0) ε[i++] = s[i] = '\0'; reverse(s) ; ) char ♦ get_next_token( char ♦ word, int ct ) { char ch, ♦ token; / ♦ this MUST BE ♦ char not (MAXTOKENLEN] ! ! ♦ / int seg_ct=0, / ♦ no of dots scanned ♦ / pos=0, / ♦ current position in string (argv[l]) ♦ / o_pos=0, / ♦ current position in output string ♦ / length = strlen(word) ; if (ct == token_num) last_tok = True; if (debug) { printf ( "Xtln get_next_token: " ) ; if (last_tok) printf ( "getting last token" ) ; printf ( "Xn* ) ; ) token = (char ♦ )calloc (MAXTOKLEN, sizeof (char) ) ; while ( (seg_ct < ct-1) && (pos < length-1) ) { 1 94 if (word[pos++] == '.') seg_ct++; } if (debug) printf( "\tseg_ct = %d\n", seg_ct ) ; / ♦ Now seg_ct = ct-1 ♦ / while ((word[pos] != '.') && (pos < length-1) ) { token[o_pos] = word[pos++]; o_pos++; ) token[o_pos] = ' \0'; if (debug) printf( "XtToken is %s\n", token ); return (token) ; ) BOOLEAN licenses( char right, char left ) { / ♦ Checks pairs of adjacent segments for governing relations. Returns True if the segment 'right' can coda-license 'left'. False otherwise. ♦ / switch (right) { case 'S': / ♦ This routine not called if left seg = "Sh 1 , so don't need to consider it ♦ / return True; case 'F' : case -f: if (left == 'R' ) return True; else return False; case 'R' : / ♦ should be a Nasal really ♦ / if (left == 'R' ) return True; else return False; default: return False; } / ♦ end switch ♦ / ) char *build_onset( char ♦ in_seg, char *seg_str, int syl_ct ) { char *new_seg, ♦ seg; char syll[3], code_str[5], first, sec- char ♦ onset; 1 9 5 int length = strlen ( in_seg) , code ; εhort branch = 0 , low_tone = 0 ; onεet = (char ♦ )calloc(12, sizeof(char) ) ; new_seg = (char ♦ )calloc(MAXTOKLEN, sizeof(char) ) ; seg = (char ♦ )calloc(length+1, sizeof(char) ) ; strncpy( seg, in_seg, length ) ; if (debug) printf ( "Xtln build_onset: seg = %s\n", seg ) ; itoa(εyl_ct, syll); if (debug) { printf ( "Xtln build_onset: syll = '%s'\n", syll ); printf ( "\t length of syll = %d\n" , strlen(εyll) ) ; ) εtrcat( onset, "0" ) ; εtrncat( onset, syll, 2 ); firεt = seg[0] ; if (debug) printf ( "Xtfirst char is %c\n", first); if (first == '0' && seg[l] == '0') strcat( onset, "3;" ),- /* pos value of 3 => empty onset */ else εtrcat( onset, "1;" ) ; if (debug) printf ( "Xtln build_onset: onset = '%s' , seg = %s\n", onset, seg ) ; if (first == 'L' ) { first = seg[l] ; low_tone++; } /* Only use presence of Low Tone in English for BrO cases; otherwise use it anyway */ if (!BR_ONSET) code = (low_tone * 512); else code = 0; switch( first ) { case 'R' : if (Japanese) { if (last_tok) strcat( onset, "616" ) ,* / ♦ R+N+L+? ♦ / else strcat( onset, "64" ) ; ) else { if (SON_STOP) 1 9 6 strcat( onset, "64" ) ; / ♦ glottal stop */ else strcat( onset, "0" ); } break; case 'h': code += 128; itoa( code, code_str ) ; / ♦ Stop Burst -> h */ strncat( onset, code_str, 5 ) ; break; case 'O' : strcat( onset, "0" ); break; case 'F': code += 8320; / ♦ h Head (s, sh) ♦ / itoa( code, code_str ) ; strncat( onset, code_str, 5 ); if (HV_DEVOICED) { /* fill in devoiced N */ new_seg = get_next_token(seg_str, ct) ; /* don't increment ct ♦ / if (new_seg[0] == 'S') { strcatf onset, "♦N" ); strncat( onset, syll, 2 ); strcat( onset, "1;2066" ); / ♦ Only works for numbers lexicon: in general could be U or I V } } break; case 'f: code += 128; / ♦ h operator ♦ / itoa( code, code_str ); strncat( onset, code_str, 5 ); break; case 'Z': / ♦ initial onsets with LowTone ♦ / case '?': strcat( onset, "192" ); / ♦ special case of final ? ♦ / break; case 'S': / ♦ printf( "XtCase SXn" ) ; ♦ / sec = seg[l] ; if (length > 1) { / ♦ ((sec == 'h') | | (sec == 'f') || (sec == -F')) */ / ♦ S-F affricate ♦ / if ( strstr("h-Ff", ++seg) !=NULL ) strcat( onset, "192" ); / ♦ Glottal stop + h ♦ / else { if (segtl] == '-') strcat( onset, "192" ) ,- else strcat( onset, "64" ) ; ) ) else / ♦ length must = 1* / if (syl_ct == 1) /* special case: could be empty onset */ strcat( onset, "64" ) ; else strcat ( onset, "64" ) ; break; case 'N': / ♦ must be the 2nd of 2 nuclei ♦ / strcat( onset, "0" ) ; ct--; / ♦ decrement token counter so thiε token iε re-used ♦ / break; default: strca ( onset, "0" ) ; ) / ♦ end switch ♦ / if (BR_ONSET) { if ((first == 'S') || (first == 'f') || (segtO] == 'L')) { /* Heads of Br. O ♦ / new_seg = get_next_token(seg_str, ct) ; if ( (new_seg[0] == 'R') || (new_seg[0] == O') ) { branch = True; / ♦ & precedes stresεed N ♦ / ct++; } } } if (branch) { strcat( onset, " *0" ); εtrncat( onεet, syll, 2 ) ,- strcat( onset, "2;32" ) ; / ♦ R element ♦ / } strcat( onset, "\0" ); return( onset ) ; ) const char ♦ Coda_set = "RFfS"; char *build_rhyme( char *in_seg, char *seg_str, int syl_ct ) { / ♦ rhyme strings are of the form: N0|l|;n where n is the element code; or N2;x;y where x, y are the element codes. 0 < n,x,y < 7 ♦ / char syll[3], first, second; char ♦ coda, ♦ nex^O, ♦ rhyme, ♦ seg; int pos=0, length = εtrlen(in_seg) ; rhyme = (char ♦ )calloc(12, sizeof(char) ) ; coda = (char ♦ )calloc(MAXTOKLEN, sizeof(char) ) ; next_0 = (char ♦ >calloc(MAXTOKLEN, sizeof (char) ) ; seg = (char ♦ )calloc(length+1, sizeof(char) ) ; strncpy( seg, in_seg, length ) ; if (debug) 1 98 printf( "Xtln build_rhyme: seg = %s\ή", seg ); itoa(syl_ct, syll); rhyme[pos++] = 'N' ; rhyme[poε++] = sylltO]; rhyme[pos++] = '1' ; rhyme[pos++] = ' ; ' ; if (debug) printf( "Xtln build_rhyme: seg = %s\n", seg ); first = segtO]; / ♦ always = N ♦ / second = segtl]; /* should be 0, 1 or 2 */ if (debug) printf ( "Xtln build_rhyme: first=%c, second=%c\n", first, second ) ; ..-witch (second) { case '0': rhyme[pos++] = '0 ' ; /* use this for unstressed N ♦ / rhyme[pos] = ' \0 ' ; break; case '1': rhyme[pos++] = seg[3]; /* Now check for closed syllable: */ if ((BR_RHYME) && ( !last_tok) ) { coda = get_next_token( seg_str, ct ); if ( strchr(Coda_set, coda[0]) != NULL && codatl] != 'h' ) { / ♦ possible Coda consonant ♦ / /* don't increment ct until we know there's a Coda ♦ / printf("XtPotential Coda consonant: checking next position: "); next_0 = get_next_token(seg_str, ct+1) ; if ( licenses(next_O[0] , codafO]) ) { ct++; printf("Yes!Xn") ; rhyme[pos++] = '♦ ' ; rhyme[pos++] = 'R' ; rhyme[poε++] = sylltO],- rhyme[pos++] = '2'; / ♦ second position */ rhyme[pos++] ~~ = ' ; ' ; rhymetpos++] = ' \0'; switch (codafO]) { case 'R' : case 'S' : strca (rhyme, "64"); / ♦ Glottal ♦ / break; case 'F' .- case 'f' : strcat(rhyme, "128"); / ♦ h ♦ / break; default: strcat(rhyme, "0"); ) /* end switch ♦ / ) else printf ("No. Xn") ,- } / ♦ else do nothing ♦ / ) elεe rhyme[poε] = '\0'; / ♦ terminate the rhyme string ♦ / break,- caεe '2': rhyme[poε++] = seg[3]; / ♦ branching N ♦ / rhyme[pos++] = '♦'; rhyme[poε++] = 'N'; rhyme[pos++] = sylltO],- rhyme[pos++] = '2 / ♦ second position ♦ / rhyme[pos++] = ';' rhyme[pos++] = '0' rhyme[poε] = ' \0 ' ,- break,- default: rhyme[poε++] = '0 / ♦ put something valid ♦ / rhyme[poε] = '\0'; break,- ) / ♦ end εwitch ♦ / return (rhyme) ,- } char ♦ parεe_seg_str( char ♦ seg_str, char lang ) { char *segment, str[3], ♦ onset, *rime, *out_str; int syl_ct=l, POS = 0; εhort all_done = 0; /* Global: int ct */ for (ct=0; ct<εtrlen(seg_str) ,- ct++) { if (seg_str[ct] == ' . ' ) token_num++,- } printf( "Number of tokens = %d\n" , token_num ) ,- ct = 1; /* No. of current token: start at 1 */ out_str = (char *)calloc( 64, sizeof(char) ) ,- segment = (char *)calloc(MAXTOKLEN, sizeof (char) ) ; onset = (char *)calloc(2 * MAXTOKLEN, sizeof(char) ) ; rime = (char ♦ )calloc(2 ♦ MAXTOKLEN, sizeof (char) ) ; switch (lang) { } if (debug) printf("Back in parse_seg_εtr: ct=%d; out_str is %s\n", ct, out_str) ; if (ct > token_num) return( out_str ) ,- syl_ct++; / ♦ = 2* 1 while ( ct <= token_num ) { ree( segment ) ; segment = (char ♦ )calloc(MAXTOKLEN, sizeof(char) ) ; segment = get_next_token(seg_str, ct) ; ct++,- onset = build_onset( segment, seg_str, syl_ct ); strcat( out_str, onset ) ,- strcatt out_str, "♦" ) ,- if (ct <= token_num) { segment = get_next_token(seg_str, ct) ,- if (segment[0] == 'N' ) { ct++; / ♦ increment token counter ♦ / rime = build_rhyme( segment, seg_str, syl_ct ) ; strcat( out_str, rime ) ,- strcat( out_str, "♦" ) ,- } / ♦ else: assume nucleus is handled by build_onset ♦ / } else { / ♦ last token used ♦ / if (DF_LEN) { / ♦ final empty nucleus ♦ / strcat( out_str, "N" ) ; itoa(syl_ct, str); strcat( out_str, str ) ; strcat( out_str, ",-0^" ) ,- } else { /* DF_LEN is False: language must be Japanese ♦ / switch (segment[0]) { case 'R' : / ♦ Japanese final eng ♦ / strcat( out_str, "N" ) ,- itoa(syl_ct, str); strcat( out_str, str ) ,- εtrcat( out_str, ";0^" ) ,- break,- case 'S' case 'F' case ■ f : / ♦ Japanese devoiced /i/ strcat( out_str, "N" ) ,- itoa(syl_ct, str) ,- strcat( out_str, str ) ,- strcat( out_εtr, "1;2066#" ) ; break; default: 2 0 0 case ' j ' : case 'J': Japanese = True; break,- case 'K' : case 'k' : Korean = True,- break,- case 'g' : case 'G' : Greek = True; break,- default: Default = True; break; ) segment = get_next_token(seg_str, ct) ; ct++; if (debug) { printf("Back in parse_seg_str: segment is %s,-", segment); printf("segment[0] is %c\n" , segment[0] ) ,- } / ♦ parse first O-R pair */ if ( segment[0] != 'N' ) { /* initial Onset ♦ / / ♦ str_cat( seg_str, "Oil,-" ); ♦ / onset = build_onset( segment, seg_str, syl_ct ) ,- strcat( out_str, onset ); strcat( out_str, "♦" ); if (debug) printf ("Back in parse_seg_str: out_str iε %s\n", out_str) ; free( segment ) ; εegment = (char ♦ )calloc(MAXTOKLEN, sizeof(char) ) ,- segment = get_next_token(seg_str, ct) ,- if (debug) printf("Back in parse_seg_str: segment is %s,-", segment); if (seg enttO] == 'N') { ct++; rime = build_rhyme( segment, seg_str, syl_ct ) ; strca ( out_str, rime ); strcat( out_str, "#" ); } else / ♦ empty nuc will be handled by build_onset ♦ / if (debug) printf( "Ignoring empty nucleus.Xn" ) ,- } else { / ♦ string begins with N: insert empty onset ♦/ strcat( out_str, "Oll;0#" ),- / ♦ remember 3 = 0 for Icon's sake ♦/ / ♦ changed to 0 : ♦ / rime = build_rhyme( segment, seg_str, syl_ct ) ,- strca ( out_str, rime ) ; , strcat( out_str, "♦" ); • 202 break ,- } /* end switch */ ) } syl_ct++; } strcat( out_str, "\0" ) ; return (out_str) ,- } 203 **** . * . **** . **** .. **** .* *** *** ** .. ** * ** . **** . **** - ******** . ************* . * SPECTRUM. H * * * * Header File containing routines for LPC spectral calculations * * and curve plotting, using data from LSI files. * * Used by LPC.C and FINDUAI.C ♦ Author: G. illiams ♦ All functionε using SPECSIZE (and fstep which is ♦ ♦ dependent on it) are now independent of the value of SPECSIZE. ' ♦ Everything is also independent of sampling rate.!! ♦ ♦ * ♦ Latest additions are curve updating routines used by FINDUAI.C * ****** ** * * ****** *** ********** * ***** * * *** **** *** ***** *** **** ****** **** **** / ♦include <stdio.h> ♦include <math.h> ♦define WINDOWSIZE 200 / ♦ was defined in LSIFILE.H ♦ / / ♦ #define STEPSIZE 50 */ ♦define SPECSIZE 256 /* Number of points in spectrum (0-6kHz in steps of xHz) */ ♦define PI 3.14159 ♦define STARTFREQUENCY 100 /* Starting frequency for log frequency plot */ ♦define NORMALDB 40 ♦define sq( x ) ( ( x ) * ( x ) ) typedef unsigned int UINT; typedef unsigned long ULONG; typedef unsigned int BOOLEAN; typedef struct { int x,- int y; } point,- typedef struct { float amp; / ♦ Amplitude in dB ♦ / UINT f; / ♦ Frequency in Hz ♦ / UINT peak_no; /* Index of peak in array peaks[] ♦ / ) formant; typedef struct { BOOLEAN U0; BOOLEAN Al; BOOLEAN 10; 204 , BOOLEAN GS; BOOLEAN hO; } E_type; #if !defined(_debugfilename) extern char *debugfilename / ♦ = "D:\\LSISWX\DEBUG.TXT" ♦ /; ♦endif ♦if !defined(_differenced) extern BOOLEAN differenced; ♦endif / ♦ extern char *ProgName = "Elemplot"; */ extern float l_range / ♦ = loglO(Maxfrequency / STARTFREQUENCY)*/; /* (usually) log (5000/100)=1.7 */ extern point curve [2 ] [SPECSIZE] ,- /* For storing curve coordinates ♦ / int fstep; /* Frequency range / no. of spectral points */ float T; /* Sampling period (1 / Sample Rate) ♦ / float r = 1.0F; / ♦ Radius of unit circle for calculating spec ♦ / UINT Maxfrequency; / ♦ Top limit of Frequency range for spectrum */ /* BOOLEAN differenced; */ char window_name[1 ] ; char window_label[25] ,- static point curve_data[SPECSIZE] ,- /* For storing curve coordinates */ / ♦ Declared as static to avoid possible unwanted modification ♦ / unsigned char *Title; /* Title of plot ♦ / /.*.*** * *.** * **.*** ** *.** * ***..*....*.*... * * * *..*. * .*....****.**.,.* * .**.* ♦ FUNCTION PROTOTYPES ♦ * * ..*** * . * . * ** * **.****** * ** * *. * **.*.*.... * ** * **.** * ** ** . * * * ** * ,. * . * . * **.*/ void dump_spec ( float spec[SPECSIZE] ),- void replot_curve( point curve[] , int colour ) ; / ♦ void get_curve_dat ( int curve_name, long time ); void update_curve( int c_name, float amplitude[2] [SPECSIZE] , float phase[2] [SPECSIZE] , float ♦ y.jnax ),- */ void dump_spec ( float spec[SPECSIZE] ) { register int j; FILE ♦ fp; if ( (fp = fopenCdumpspec.txt", "at")) != NULL ) { for (j=0; j<SPECSIZE; j++) fprintf(fp, "%f\t", spec[j]) ; fprintf(fp, "Xn"); f close ( f p ) ; ) else { fprintf (stderr, "Error opening dumpspec . tx Xn" ) ,- exit (l) ; ) } void get_curve_data( int curve_name, long time ) / ♦ Load array of spectral coordinates from global curve_data[SPECSIZE] into globals curve[0] [SPECSIZE] (left) or curve[1] [SPECSIZE] (rt) . Curves can be replotted fast using replot_curve(curve, colour) below. Values are also stored in file D:\LSISW\DATA.TXT. */ { int xpos; register int ct; FILE *flist; char *fname; if ( curve_name == 0 ) fname = "Dataleft.txt"; else fname = "Datart.txt" ; if ( (flist = fopen( fname, "wt" )) != NULL ) { fprintf( fliεt, "%s,-\n", ProgName ) ,- curvet curve_name ] [0] .x = curve_data[0] .x ,- curvet curve_name ] [0] ,y = curve_data[0] .y ; ct = 1; while ( ct < SPECSIZE ) { xpos = curve_data[ct] .x ; curvet curve_name ] [ct] .x = xpos,- curvet curve_name ] [ct] .y = curve_data[ct] .y ; fprintf( flist, "%d,%d\t", xpos, curve[ curve_name ] [ct] .y ),- ct++; } fclose( flist ) ; /* The last pair of values in the output array should be 0,0. */ ) else { printf( "XError opening DATA.TXT file.Xn" ) ; } )

Claims: 206 CLAIMS 1. A machine-implemented process for recognition of a spoken word in an utterance, comprising the steps of: receiving an speech signal containing an utterance; detecting presence of linguistic cues in the speech signal; generating, from detection of linguistic cues, a symbolic representation comprising at least one word division, each word division consisting of an onset-rhyme pair and associated phonological elements; and identifying a best match to the symbolic representation in a lexicon of symbolic representations of words, thereby recognizing the spoken word. 2. The process of claim 1, wherein the step of generating comprises the steps of: detecting presence of univalent phonological elements in the speech signal according to detected linguistic cues; and associating each detected univalent phonological element with one of an onset or a rhyme constituent. 3. The process of claim 1, wherein the step of detecting comprises the steps of: detecting voiced and voiceless regions in the speech signal; detecting sonorant and nucleus regions in the voiced regions of the speech signal; and detecting presence of phonological elements within each of any silence, fricative, sonorant and nucleus regions detected, and identifying each of the detected regions aε defining either an onset or a rhyme. 2 0 7 4. An apparatus for recognition of a spoken word in an utterance, comprising: means for receiving an speech signal containing an utterance; means for detecting presence of linguistic cues in the speech signal; means for generating, from detection of linguistic cueε, a symbolic representation comprising at least one word division, each word division consisting of an onset-rhyme pair and associated phonological elements; and means for identifying a best match to the εymbolic representation in a lexicon of symbolic representations of words, thereby recognizing the spoken word. 5. The apparatus of claim 4, wherein the means for generating a symbolic representation co priseε: means for detecting presence of univalent phonological elementε in the speech signal according to detected linguistic cues; and means for associating each detected univalent phonological element with one of an onset or a rhyme constituent. 6. The apparatus of claim 4, wherein the means for detecting comprises: means for detecting voiced and voiceless regions in the speech signal; means detecting sonorant and nucleus regions in the voiced regions of the speech signal; and means detecting presence of phonological elements within each of any silence, fricative, sonorant and nucleus regions detected, and identifying each of the detected regions as defining either an onset or a rhyme. 2 0 8 7. An apparatus for for recognition of a spoken word in an utterance, comprising: a univalent element ar.d structure detector having an input for receiving a speech signal and an output providing a symbolic representation of = word detected in the speech signal, wherein the symbolic representation comprises at least one word division, each word division consisting of an onset-rhyme pair and associated phonological elements; a lexicon of predetermined symbolic representationε of words; and a lexical matching system having a first input for receiving the symbolic representation from the output of the univalent element and εtructure detector, a second input for receiving predetermined symbolic representations from the lexiron and an output providing an indication of the predetermined symbolic representation which best matcheε the symbolic representation output by the univalent element and structure detector . 8. The apparatus of claim 7, wherein the univalent element and structure detector comprises: a phonetic clasεifier and segmenter having an input for receiving features of the speech signal and an output for providing a string of tokens indicative of presence of univalent phonological elements in the speech signal according to detected linguistic cues; and a word parser having an input for receiving the string of tokens from the phonetic classifier and segmenter and which associateε each detected univalent phonological element with one of an onset or a rhyme conεtituent so as to provide the symbolic representation. 2 0 9 9. An automatic speech recognition system comprising: means for detecting presence of at least one of a plurality of uniquely identifiable univalent phonological elements from a speech signal; means for associating each detected phonological element with either an onset or a rhyme; and means for comparing a combination of the detected phonological elements and asεociated onset and rhymes to a lexicon of poεsible combinations of such onset and rhyme divisionε and phonological elementε and for selecting from the lexicon a possible acceptable representation of a word most similar to the detected onset and rhyme divisions and phonological elements. 10. A method for automatic εpeech recognition, compriεing the εtepε of: detecting preεence of a plurality of uniquely identifiable phonological elements from a speech εignal; associating each detected phonological element with either an onset or a rhyme; and comparing a combination of the detected onset and rhyme divisions and the detected phonological elements to a lexicon of possible combinations of such onset and rhyme divisions and phonological elements and for selecting from the lexicon a poεsible acceptable representation of a word moεt εimilar to the detected onεet and rhyme divisions and phonological elements.

Inventors: KAYE JONATHAN, WILLIAMS GEOFFREY, KAYE, JONATHAN, WILLIAMS, GEOFFREY
Applicants: HAL TRUST L L C, HAL TRUST, L.L.C.
Application year: 1995
Main classification: G10L3
IPC codes: G10L11, G10L15