Method and apparatus for automatic grammar generation from data entries

A method of generating an optimized grammar, for use in speech recognition, from a data set or big list of items, is disclosed. The method includes the steps of obtaining a tree representing items in the data set, and generating the grammar using the tree. The tree or tree data structure representing items in the data set is a simulated recognition search tree, representing items in the data set, which can be automatically generated from the data set.

BACKGROUND OF THE INVENTION

The present invention generally pertains to speech recognition applications and systems. More specifically, the present invention pertains to methods and apparatus for automatically generating grammars for use by speech recognition applications.

Speech recognition applications often need to deal with big lists of proper names, symbols, numbers, ids, or other items. As an example, speech recognition is being increasingly used to recognize names spoken by a user or caller. For instance, voice dialing and other systems, a caller or user is typically asked to speak the name of the person who is to be contacted, or identified for some purpose. The system then uses a speech recognition engine to recognize the spoken name from a large list of names, often in combination with prompting for the caller or user to navigate through any name collisions or other difficulties in the identification process. Speech recognition of spoken names is also used for many purposes other than voice dialing systems.

One of the biggest challenges to using speech recognition to recognize names or other items relates to the process of building context free grammars (CFGs) to be used by the speech recognition engine. This is particularly true if the items to be recognized are from a large data list. In some speech recognition systems or applications, the number of items on the data list increases frequently, sometimes even daily, by significant numbers. In certain applications, it is possible for the number of items on the data list to increase by tens of thousands of items every day. Creating or updating CFGs to deal with these large and sometimes fast growing data lists can be very challenging, time consuming and cumbersome. In short, a challenge faced by many in speech recognition applications is to correctly and timely generate efficient grammars from those big lists.

A number of factors which affect speech recognition engine performance need to be considered when generating the CFG to be used by the speech recognition engine during the speech recognition process. To increase the ability of a speech recognition engine to accurately identify a spoken name or item, prefixing of the CFG is useful. For example, with a prefixed CFG, instead of the speech recognition engine having to process the competing common phrase “David”, the grammar recognizes “David” as a shared speech unit. The grammar then branches to possible next speech units “Ollason” and “Smith” for continued speech recognition. In other words, prefixing of a CFG allows the speech recognition engine to reduce the resource consumption, which typically improves accuracy of the recognition process. Other factors which must be considered when generating a CFG include weighting of branches of the tree structure represented in the CFG, dealing with name collisions (names sharing identical spellings or pronunciations), optimizing the size (storage and processing requirements) of the CFG, etc.

Due to the size of the task of creating or updating grammars for large lists, it is important to do so as efficiently as possible. However, accuracy is also very important. Any techniques for speeding up the grammar generation or updating process which result in a lower quality grammar will render the speech recognition system, using the CFG, less accurate. This in turn will increase the time required for users of the system to achieve a desired result, for example of being connected to a particular individual in a voice-dialing system. Many users will find the decreased accuracy and increased time required to be unacceptable.

The present invention provides solutions to one or more of the above-described problems and/or provides other advantages over the prior art.

SUMMARY OF THE INVENTION

A method of generating a grammar, for use in speech recognition, from a data set or big list of items, is disclosed. The method includes the steps of obtaining a tree representing items in the data set, and generating the grammar using the tree. The tree or tree data structure representing items in the data set is a simulated recognition search tree, representing items in the data set, which can be automatically generated from the data set.

In some specific embodiments, the present invention automatically analyzes the data set, annotates custom pronunciations, prefixes the grammar, correctly assigns language model weights (CFG phrase branching probabilities), text normalizes names (for the case of spelling grammars), performs Semantic Markup Language (SML) tag optimization, produces the grammars themselves, and raises warnings of potential inconsistencies, all using the simulated recognition search tree. In more general embodiments, the present invention performs some, but not all, of these functions.

DETAILED DESCRIPTION OF ILLUSTRATIVE EMBODIMENTS

Various aspects of the present invention pertain to methods and apparatus for automatic grammar generation from data entries. For example, a context free grammar (CFG) generated using the methods and apparatus of the present invention is used by a speech recognition engine for performing speech recognition in a desired environment. The data entries from which the CFG is automatically generated are typically in the form of a list, and the present invention is particularly useful when the list of data entries is large and/or frequently changing or quickly growing. However, the present invention is not limited to use with lists having these characteristics.

Examples of well-known computing systems, environments, and/or configurations that may be suitable for use with the invention include, but are not in any way limited to, personal computers, server computers, hand-held or laptop devices, multiprocessor systems, microprocessor-based systems, and set top boxes. Embodiments of the present invention can be implemented in a wide variety of speech recognition applications, for example including voice-dialing systems, call routing systems, voice messaging systems, order management systems, or any application where a speech recognition engine uses a grammar to recognize speech from a user. These are simply examples of systems within which embodiments of the present invention can be implemented.

Prior to discussing embodiments of the present invention in detail, exemplary computing environments within which the embodiments and their associated systems can be implemented will be discussed.

FIG. 1illustrates an example of a suitable computing environment100within which embodiments of the present invention and their associated systems may be implemented. The computing system environment100is only one example of a suitable computing environment and is not intended to suggest any limitation as to the scope of use or functionality of the invention. Neither should the computing environment100be interpreted as having any dependency or requirement relating to any one or combination of illustrated components.

The present invention is operational with numerous other general purpose or special purpose computing consumer electronics, network PCs, minicomputers, mainframe computers, telephony systems, distributed computing environments that include any of the above systems or devices, and the like.

The invention may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The invention is designed to be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules are located in both local and remote computer storage media including memory storage devices. Tasks performed by the programs and modules are described below and with the aid of figures. Those skilled in the art can implement the description and figures as processor executable instructions, which can be written on any form of a computer readable media.

It should be noted that the present invention can be carried out on a computer system such as that described with respect toFIG. 1. However, the present invention can be carried out on a server, a computer devoted to message handling, or on a distributed system in which different portions of the present invention are carried out on different parts of the distributed computing system.

II. Automatic Grammar Generation Tool

As described above, speech recognition applications often need to deal with big lists of items, including proper names, symbols, numbers, and ids. It is a significant challenge to generate efficient grammars for use in speech recognition, in a correct and timely manner, from those big lists. The present invention addresses this problem with the development of an advanced algorithm and system for automatically generating such grammars from lists. In some specific embodiments, the present invention automatically analyzes the data set, annotates custom pronunciations, prefixes the grammar, correctly assigns language model weights (CFG phrase branching probabilities), text normalizes names (for the case of spelling grammars), performs Semantic Markup Language (SML) tag optimization, produces the grammars themselves, and raises warnings of potential inconsistencies. In more general embodiments, the present invention performs some, but not all, of these functions as described below.

1. Simulated Recognition Search Tree Generation

The grammar generating tool and method of the present invention drastically reduce the development time, maintenance efforts, and human errors of grammar generation, while at the same time guaranteeing (or at least increasing the likelihood of obtaining) the optimal speech recognition accuracy and system performance. In accordance with embodiments of the present invention, a simulated recognition search tree is generated from a data set of entries (containing a list of items). An optimized grammar is then automatically generated from the simulated recognition search tree. The simulated recognition search tree data structure enables the capture of all information needed to both detect collisions in the dataset and to build the optimal grammar.

In describing the methods, apparatus or systems of the present invention, an example of a simulated recognition search tree will be described with reference toFIGS. 3-1though3-4. An example of a grammar produced from the simulated recognition search tree will be described with reference toFIG. 4. In these FIGS. and the related discussions, the dataset {‘Michael Anderson’ (Michael Anderson), ‘Michael Smith’ (Dr. Smith), ‘Michael’ (Michael), ‘David Ollason’ (David Ollason)} is used as an example. The phrases in the single quotation marks (e.g., ‘Michael Smith’) represent speech recognition matches, while the phrases in the parenthesis (e.g., (Dr. Smith)) mark the SML values to be returned in response to a match.

Shown inFIG. 2is a block diagram of an automatic grammar generation system200in accordance with some embodiments of the present invention. Grammar generation system200includes a simulated recognition search tree generating component or module210and a grammar producing component or module220. Search tree generating module210uses a data set205(or list of names or other entries) to generate a simulated search tree215. Grammar producing module220then uses simulated search tree215to generate grammar225, which can be a CFG of the type used by a speech recognition engine230during speech recognition processes.

Referring now back toFIGS. 3-1through3-4, shown are diagrammatic representations of the generation of a simulated recognition search tree300, which is shown in its final form inFIG. 3-4. Simulated search tree300is built for the example data set listed above. For discussion purposes, the search tree is generated sequentially in the order that the data set was listed: {‘Michael Anderson’ (Michael Anderson), ‘Michael Smith’ (Dr. Smith), ‘Michael’ (Michael), ‘David Ollason’ (David Ollason)}. The arcs or paths of this tree structure represents words which can be recognized. Each arc is labeled with the word to be recognized and a weight which, in most of the cases, is the count of the phrases in the data set205which share that word. A node in the tree is called a terminal node (shown with a thicker border) if the phrase reaching this node is a complete “sentence” (i.e., expression, name, etc) to be accepted. Each terminal node holds a collection of the SMLs to be returned for that terminal node.

Referring first more specifically toFIG. 3-1, the simulated search tree is initiated for the data entry or name, “Michael Anderson”. From the root node305, this results in an arc307to node310corresponding to “Michael”. Arc307is labeled with the word to be recognized (i.e., “Michael”) and a phrase count or other weight designator to be assigned to that arc in the tree. In this case, the phrase count is “1” since this is the first occurrence of the word “Michael”. From node310, the entry “Michael Anderson” also results in arc312to terminal node315. Arc312is labeled with the word to be recognized (i.e., “Anderson”) and the phrase count for that word. The SML value316to be returned, in this case ‘Michael Anderson’, is stored in association with terminal node315.

Referring next toFIG. 3-2, generation of the simulated search tree is continued for the data entry or name, “Michael Smith”. Since this is the second occurrence of the word “Michael”, the phrase count associated with arc307is updated to “2”. From node310, the entry “Michael Smith” also results in arc313to terminal node320. Arc313is labeled with the word to be recognized (i.e., “Smith”) and the phrase count for that word, which is “1” since this is the first occurrence of the word “Smith”. The SML value321to be returned, in this case ‘Dr. Smith’, is stored in association with terminal node320.

Referring next toFIG. 3-3, generation of the simulated search tree is continued for the data entry or name, “Michael”. Since this is the third occurrence of the word “Michael”, the phrase count associated with arc307is updated to “3”. Also, since “Michael” is now a complete sentence (i.e., phrase or expression) to be accepted, node310becomes a terminal node. An SML value311to be returned, in this case ‘Michael’, is therefore stored in association with terminal node310.

Referring finally toFIG. 3-4, the simulated search tree is completed after its expansion for the data entry or name, “David Ollason”. From the root node305, this results in an arc308to node325corresponding to “David”. Arc308is labeled with the word to be recognized (i.e., “David”) and a phrase count to be assigned to that arc in the tree. In this case, the phrase count is “1” since this is the first occurrence of the word “David” in tree300. From node325, the entry “David Ollason” also results in arc328to terminal node330. Arc328is labeled with the word to be recognized (i.e., “Ollason”) and the phrase count for that word. The SML value331to be returned, in this case ‘David Ollason’, is stored in association with terminal node330.

Using simulated recognition search tree300, a grammar can be automatically generated. An example of such a grammar is provided inFIG. 4. With each terminal node holding a collection of SMLs to be returned, various information can be readily ascertained from search tree300:

a) Collision Detection: Collision occurs when two exact input sentences reach the same terminal node with different SMLs to return. Using search tree300, collisions can be easily identified by checking the size of the SML collection at every terminal node.
b) CFG Weight Calculation: To properly maintain the weights for every branch of the search tree300, it is only necessary to increment the count every time an arc is re-visited (phrase is shared).
c) Grammar generation: With the weight information available, an optimal (prefixed) grammar can be readily produced directly from traversing the search tree300.
d) SML tag optimization: Statistics show that the space (memory) taken up by SML tags can reach 50% of the size of the CFG. In order to reduce the size (memory needed to load the CFG), the present invention includes a mechanism to explicitly store the SML tag in the CFG as often as possible. Since the speech application program interface (SAPI) returns the recognized phrase text (e.g., “Michael Anderson”) automatically with no extra CFG memory required, in accordance with the present invention the SML label is only stored if it can not be derived from the recognized text. A wrapper is implemented to check whether the SML was explicitly reported. If not, the empty SML label is replaced with the recognized text. An example of such a wrapper is shown in the grammar ofFIG. 4, and is repeated here for discussion purposes.

As an example, only the terminal node321inFIG. 3-4needs an SML tag portion. Here, the SML tag portion “($._attributes==undefined)” represents that the semantic tag is missing, and the SML tag portion “$._value=$recognized.text;” and “$._attributes.text=$recognized.text” dictate that the recognized text should be returned in place of the empty SML label. For a large data set, this saves a significant amount of space, resulting also in a faster speech recognition turn-around time.

2. Additional Features

Using the simulated recognition search tree methods of grammar generation described above, other features can be added to facilitate the grammar generation process. These features are illustrated in the block diagram of an automatic grammar generation system500shown inFIG. 5. System500includes the same components or modules as shown inFIG. 2, and includes components or modules505and510in accordance with some further example embodiments of the present invention. Features added with these modules include:

a) Custom pronunciation annotation: An optional Application Dictionary interface510can be added to the grammar generation to override the default pronunciations used by the speech recognition engine230. For every word the system500(via module220) writes into the CFG225, it is checked to see if it is in that dictionary510. If it is in dictionary510, then a SAPI <pron> (pronunciation) tag is added to the SML to annotate the pronunciation. To change the pronunciations, it isn't necessary to modify the CFG. Instead, the application dictionary can be updated and the CFG re-generated.
b) Text normalization for the spelling grammars: It has been discovered that many requirements/features of spelling grammars can be supported by the additional text normalization step provided by text normalization component or module505. Since all phrases that can be recognized are captured in the tree300, the text normalization step adds (additional) phrases which can also be recognized to support the flexible spelling. For example, these can include:i. Letters extraction: When a word like “Ollason” is encountered, letters “O. L. L. A. S. O. N.” can be individually added as the actual phrase to the tree. In other words, each letter forms an arc in a path toward a terminal node representing the phrase including all of the individual letters of that name.ii. Letters grouping: When a valid letter grouping is detected in the phrase, additional phrases can be added to the tree accordingly. For example, in some cases, if “letter grouping” is turned on, then both “O. L. L. A. S. O. N” and “O. double L. A. S. O. N” are added to the tree to accommodate the manner in which different speakers might sound out the spelling of “Ollason.”iii. Optional Punctuations: if punctuations are optional, additional phrases with the punctuations removed can also be added to the tree. For example, if hyphens are optional, then one can add “A. dash B. C.” and “A. B. C.” when “A-BC” is encountered.