Automated tuning of speech recognition parameters

A method for execution on a server for serving presence information, the method for providing dynamically loaded speech recognition parameters to a speech recognition engine, can be provided. The method can include storing at least one rule for selecting speech recognition parameters, wherein a rule comprises an if-portion including criteria and a then-portion specifying speech recognition parameters that must be used when the criteria is met. The method can further include receiving notice that a speech recognition session has been initiated between a user and the speech recognition engine. The method can further include selecting a first set of speech recognition parameters responsive to executing the at least one rule and providing to the speech recognition engine the first set of speech recognition parameters for performing speech recognition of the user.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to automatic speech recognition, and more particularly relates to the tuning of speech recognition parameters for automatic speech recognition engines.

2. Description of the Related Art

Speech recognition (or SR) systems translate audio information into text information. An SR system processes incoming speech and uses speech recognition parameters (i.e., grammars, weights, etc.) to determine the natural language represented by the speech. In an SR system, speech recognition occurs based on a score describing a phonetic similarity to the natural language options in a set of grammars. A grammar is an available set of natural language options in a particular context. A grammar can represent a set of words or phrases. When speech is recognized as one of the words or phrases in a grammar, the SR system returns the natural language interpretation of the speech.

The SR system computes scores for the options of the grammars for speech. The score of an option is based on two kinds of information: acoustic information and grammatical information. A probabilistic framework for the acoustic information defines the “acoustic score” as the likelihood that a particular option was spoken, given the acoustic properties of an utterance. The grammatical information biases some options in relation to others. In a probabilistic framework, the grammatical information is defined as a probability associated with each option. These probabilities are referred to herein as “grammar weights”, or simply “weights”. The score computed by the SR system for an option, given an utterance, is a combination of the acoustic score and the grammar weight. The SR system chooses the grammar option having the highest score as the natural language interpretation of the speech. Increasing the grammar weight of an option (and thus increasing the score of the option) therefore increases the chance of that option being chosen as the natural language interpretation of a given utterance.

An application author, which is a voice application programmer, defines the grammars for a speech engine. Grammar weights are defined by application authors in the course of the application programming process and are therefore alterable by the application author. The grammar weights of grammars may be determined (either assigned or tuned) according to a specific method to maximize the abilities of the SR system to correctly interpret speech. However, because acoustic scores are modeled by the manufacturer of the speech recognition software, the acoustic scores are typically fixed in a particular version of the speech recognition software. This can produce obstacles during maintenance, re-deployment, piloting and other phases of production. For example, if an SR system is originally deployed for recognizing residential addresses and then is later deployed for recognizing business addresses, the speech recognition parameters, which were originally hard-coded into the application, must then be re-worked or modified to recognize business addresses. This can be time-consuming and costly. It is therefore desirable for an SR system to have easy access to speech recognition parameters so as to allow for customization to different environments independent of applications.

Therefore, a need arises for a more efficient method for providing access to speech recognition parameters to speech recognition systems that are deployed in different environments.

BRIEF SUMMARY OF THE INVENTION

Embodiments of the present invention address deficiencies of the art in respect to automatic speech recognition and provide a novel and non-obvious method, system and computer program product for providing dynamically loaded speech recognition parameters. In one embodiment of the invention, a method for execution on a server for serving presence information, the method for providing dynamically loaded speech recognition parameters to a speech recognition engine, can be provided. The method can include storing at least one rule for selecting speech recognition parameters, wherein a rule comprises an if-portion including criteria and a then-portion specifying speech recognition parameters that must be used when the criteria is met. The method can further include receiving notice that a speech recognition session has been initiated between a user and the speech recognition engine. The method can further include selecting a first set of speech recognition parameters responsive to executing the at least one rule and providing to the speech recognition engine the first set of speech recognition parameters for performing speech recognition of the user.

In another embodiment of the invention, a method for execution on a server for serving presence information, the method for providing dynamically loaded speech recognition parameters to a speech recognition engine, is provided. The method can include storing at least one rule for selecting speech recognition parameters, wherein a rule comprises an if-portion including criteria and a then-portion specifying speech recognition parameters that must be used when the criteria is met. The method can further include storing periodically updated metadata about a plurality of speech recognition engines and selecting a first speech recognition engine based on most recently stored metadata. The method can further include receiving notice that a speech recognition session has been initiated between a user and the first speech recognition engine and executing the at least one rule. The method can further include selecting a first set of speech recognition parameters responsive to executing the at least one rule and providing to the first speech recognition engine the first set of speech recognition parameters for performing speech recognition of the user.

In yet another embodiment of the invention, a computer system comprising a server for serving presence information, the server for providing dynamically loaded speech recognition parameters to a speech recognition engine, can be provided. The system can include a repository for storing at least one rule for selecting speech recognition parameters, wherein a rule comprises an if-portion including criteria and a then-portion specifying speech recognition parameters that must be used when the criteria is met. The system further can include a processor configured for receiving notice that a speech recognition session has been initiated between a user and the speech recognition engine and executing the at least one rule. The processor may further be configured for selecting a first set of speech recognition parameters responsive to executing the at least one rule and providing to the speech recognition engine the first set of speech recognition parameters for performing speech recognition of the user.

DETAILED DESCRIPTION OF THE INVENTION

Embodiments of the present invention provide a method, system and computer program product for providing dynamically loaded speech recognition parameters. The method can include storing at least one rule for selecting speech recognition parameters, wherein a rule comprises an if-portion including criteria and a then-portion specifying speech recognition parameters that must be used when the criteria is met. The method can further include initiating a speech recognition session between with a user and the speech recognition engine and executing the at least one rule. The method can further include selecting a first set of speech recognition parameters responsive to executing the at least one rule and loading the first set of speech recognition parameters for performing speech recognition of the user.

FIG. 1is a block diagram illustrating a network architecture for a system providing for dynamically loaded speech recognition parameters, according to one embodiment of the present invention. The system ofFIG. 1includes a user102utilizing a VoIP audio device104to conduct a SIP call. The Session Initiation Protocol (SIP) is an application-layer control (signaling) protocol for creating, modifying, and terminating sessions with one or more participants. It can be used to create two-party, multiparty, or multicast sessions that include Internet telephone calls, multimedia distribution, and multimedia conferences. The device104is connected to a network106, which may be a packet switched network such as the global Internet, a private Intranet or the like.

FIG. 1also includes an interactive voice response (IVR) system106connected to the network106. An IVR is a phone technology that allows a computer to detect voice and touch tones using a normal phone call. The IVR system106can respond with pre-recorded or dynamically generated audio to further direct callers on how to proceed. The load balancer108, also connected to network106, comprises a server which services the requests of its clients, i.e., device104, by forwarding requests to other servers, such as speech recognition servers130,132and134. The device104connects to the load balancer108, requesting speech recognition services available from speech recognition servers130,132and134. The load balancer108provides the resource by connecting to the specified servers130,132and134and requesting the service on behalf of the client, device104. The load balancer108may also serve load balancing purposes by distributing the speech recognition processing burden among appropriate speech recognition servers130,132and134.

Also connected to network106are three sets of speech recognition servers130,132,134. Each type of speech recognition, such as recognizing names as opposed to numbers, necessitates a specific set of grammar and weights. Thus, each set of speech servers handle a particular type of speech recognition. The first set of speech recognition servers130comprise a set of servers that provide speech recognition for address capture, wherein an address, whether residential or business, is recognized. The second set of speech recognition servers132comprises a set of servers that provide speech recognition for cities. The third set of speech recognition servers134comprises a set of servers that provide speech recognition for a date. Note that althoughFIG. 1shows only three sets of speech recognition servers, the present invention supports any number of speech recognition servers serving various types of speech recognition.

The speech recognition servers130,132,134are configured to be used according to their abilities. Thus, a particular speech recognition server may be used for one turn. A turn is one segment of a speech recognition session. A speech recognition session may comprise various segments wherein each segment is directed towards recognizing a particular type of data. For example, a speech recognition application may be programmed to recognize an address and a city. The aforementioned speech recognition session may be divided into two segments or turns wherein the first turn is serviced by an address speech recognition server (found in group130) and the second turn is serviced by a city speech recognition server (found in group132).

Also connected to the network106is the presence server110. The presence server110serves presence information, which is a status indicator that conveys ability and willingness of an entity, such as a user or a server, to communicate or operate normally. Presence information, and related metadata, is provided by each server130,132, and134to presence server110. The presence information, and related metadata, is stored in appropriate databases116,118and can be made available for distribution to other entities. Users and servers may publish presence information and related metadata to indicate its current communication and performance status. This published information informs others that wish to contact or interact with an entity of his availability and willingness to communicate and process information.

In an embodiment of the present invention, presence server110is a modified commercially available presence server such as the IBM WebSphere Presence Server available from International Business Machines Corp. of Armonk, N.Y. Conventionally, a presence server serves presence information, which is a status indicator that conveys ability and willingness of a potential communication partner. A user's client provides presence information via a network connection to a presence server, which stores the presence information in a user's personal availability record and can be made available for distribution to other users to convey his availability for communication. The presence server110can be a commercially available presence server modified to serve additional information, besides presence information, as described below. The presence server110can further be modified to provide additional functions described below.

In an embodiment of the present invention, each speech recognition server130,132,134publishes a variety of data to the presence server110, including load data, supported grammars, availability, health, supported languages and acoustic model characteristics. Speech recognition servers may also publish performance data to the presence server110such as recognition accuracy, grammar usage and the like. The above data published by speech recognition servers may be stored in a recognition engine metadata database116. A user102, as well as other users, may publish to the presence server110such data as the current physical location of the user102, such as an address, a sphere indicator, such as “at home,” “in an office,” or “driving in a car,” and availability, which indicates whether the user102is currently available for a SIP Session. The above data published by users may be stored in a user metadata database120.

Stored in the parameters database118are speech recognition parameters such as grammars, weights, accuracy settings, threshold values and sensitivity values. Also stored in parameters database118are rules for adjusting the speech recognition parameters. A rule comprises an if-portion including criteria that must be met and a then-portion specifying speech recognition parameters that must be used when the criteria is met. Factors that may be taken into account when determining whether criteria is met include time of day, recognition accuracy of the speech recognition engine, and grammar usage of the speech recognition engine. For example, if a rules states a recognition accuracy is below 40% and the current recognition accuracy of a recognition engine server is currently 33%, then the criteria is met. Next, the then-portion of the rule dictates that a specified set of speech recognition parameters are selected.

Also connected to network106are web interface112and administrative terminal114. These interfaces are used to prompt an administrator for input in response to a situation, such as low recognition accuracy. In this process, the administrator provides commands to the system ofFIG. 1for adjusting speech recognition parameters.

FIG. 2is a flow chart depicting a general process for providing dynamically loaded speech recognition parameters, according to one embodiment of the present invention. The following flow chart depicts the process executed by the system ofFIG. 1for choosing an appropriate speech recognition engine for servicing a user at a particular turn further and dynamically providing speech recognition tuning parameters based on collected metadata. In step202, the user102uses his VoIP device104to call into the IVR106. In step204, the device104sends a SIP invite to the IVR106.

In an optional step after step204, the IVR106gathers metadata about the user102. The gathered metadata may include the current physical location of the user102, such as an address, a sphere indicator, such as “at home,” “in an office,” or “driving in a car,” and availability, which indicates whether the user102is currently available for a SIP Session. The user metadata may be gathered from a separate entity such as a location server. In a second optional step, the gathered metadata is stored by the presence server100in the user metadata database120.

In step206, the IVR106routes the original invite to the load balancer108. In step208, the load balancer108queries, via the presence server110, the recognition engine metadata database116for the most recent metadata about the recognition engine servers130,132and134. In step210, the load balancer108receives the metadata about the recognition engine servers130,132and134from the recognition engine metadata database116.

In step212, the load balancer108selects a recognition engine server within the servers130,132and134based on the received metadata. The load balancer108may take a variety of factors into account when making the selection of step212. The load balancer108takes into account the grammars and languages supported by each recognition engine server within the servers130,132and134. For example, if the IVR106is currently capturing addresses in English, only those recognition engine servers servicing address capture in English are considered. The load balancer108also takes into account load data, availability data and health data for each recognition engine server so as to determine which servers currently have enough bandwidth to service the user102at the highest capacity. The load balancer108also takes into account acoustic model characteristics so as to determine which server uses the appropriate model to service the speech recognition type of the current turn.

In step214, the load balancer108routes the original invite to the selected recognition engine server, in this case recognition engine server140. In step216, recognition engine server140receives the original invite from the device104and initiates a SIP connection with the device104. In step218, the recognition engine server140queries the presence server110for the appropriate speech recognition parameters. In step219, the presence server110executes the rules in parameter database118to determine the appropriate speech recognition parameters for loading into the recognition engine server140. The process of executing a rule is described in greater detail below.

As described earlier, a rule comprises an if-portion including criteria that must be met and a then-portion specifying speech recognition parameters that must be used when the criteria is met. Step219involves reading metadata from the parameters database118, wherein the metadata includes a least one value for at least one of time of day, recognition accuracy of the speech recognition engine, and grammar usage of the speech recognition engine. Next, it is determined whether the metadata meets criteria of the rule. For example, if the rules states a time of day between 9 am and 5 pm, then if the current time of the day is 1 pm, then the criteria is met. In another example, if the rule states that the recognition accuracy is below 40% and the current recognition accuracy of the recognition engine server140is currently 33%, then the criteria is met. Next, assuming the criteria of the if-portion of the rule is met, the then-portion of the rule dictates that a specified set of speech recognition parameters are selected.

In an optional step after step219, the presence server110takes additional user metadata, from database120, into account when selecting speech recognition parameters as in step219. For example, if the user metadata in database120indicates that the user102is driving during the SIP session, then appropriate speech recognition parameters that optimize recognition during driving are selected.

FIG. 3is a flow chart depicting a process for adjusting speech recognition parameters according to a rule, according to one embodiment of the present invention. The following flow chart depicts the process executed by the system ofFIG. 1for adjusting speech recognition parameters before a turn based on collected metadata. In step302, it is determined, by an entity such as presence server110, that a recognition engine server, such as server140, is achieving low recognition accuracy. As described earlier, each recognition engine server reports related metadata to the RE metadata database116. Thus, step302may occur in between turns wherein the presence server110queries the RE metadata database116for metadata about the speech recognition engine servers.

In step304, the presence server110executes the rules in parameter database118to determine the appropriate speech recognition parameters for loading into the recognition engine server140. The process of executing a rule is described in greater detail above. In this example, a rule is executed wherein a grammar weight is changed due to the low recognition accuracy.

In an optional step after step304, an administrator, connected via web interface112or administrative terminal114, is prompted for input in response to the low recognition accuracy. In this alternative, the administrator provides commands to the system ofFIG. 1for adjusting speech recognition parameters so as to resolve the low speech recognition accuracy.

In step306, a notification is sent via presence server110by recognition engine server140. The notification is sent to all other recognition engine servers130,132and134. The notification may be a standard text message sent via TCP/IP or SIP NOTIFY events. The notification states that new speech recognition parameters are available and shall be loaded at the next turn. In step308, the next turn is initiated.

In step310, the recognition engine server140queries the presence server110for the appropriate speech recognition parameters. In step312, the presence server110executes the rules in parameter database118to determine the appropriate speech recognition parameters for loading into the recognition engine server140. The process of executing a rule is described in greater detail above.