Patent Description:
Speech recognition refers to the transcription of spoken words into text using an automated speech recognizer (ASR). In traditional ASR systems, received audio is converted into computer-readable sounds, which are then compared to a dictionary of words that are associated with a given language. An exemplary approach for speech recognition using a geo-localized entity database is disclosed in <CIT>.

In general, one innovative aspect of the subject matter described in this specification can be embodied in methods that improve voice recognition accuracy of uncommon location-specific terms that are likely to be misrecognized. For instance, location data associated with a voice query can be used to generate additional candidate transcriptions for the voice query. The additional candidate transcriptions are generated using n-grams that are associated with multiple geographic regions relating to the location.

For example, a neighborhood model can include a collection of n-grams that are determined to be associated with a specific geographic region encompassing the location, whereas a sub-locality model can include a collection of n-grams that are determined to be associated with a larger area encompassing the location. In this regard, a hierarchy of geographic region models can be used to generate alternative candidate transcriptions that are each specific to different geographic regions relating to the location. As a result, an automated speech recognizer (ASR) can use the alternative candidate transcriptions to reduce the number of misrecognitions of location-specific terms.

Misrecognition of location-specific terms may lead to misleading or otherwise incorrect speech recognition results which, when relied upon, may lead to further inaccuracies and technical inefficiencies in a computer system. For example, in the event that the location-specific term forms part of an email address or similar, its misrecognition may lead to a computer sending an email to a non-existent email address (e.g. as part of an automated process). This may cause inefficiencies in both the computer and a communications network to which the computer is connected. For example, a bounce-back or 'undeliverable' message from a server in the network may place a load on the computer and the network. This load would not have been caused had the location-specific term been recognized correctly. Further unnecessary load may be placed on the computer by manual corrections to the misrecognized terms by a user, for example via a keyboard or other input device. The improvements to voice recognition accuracy for uncommon, location specific-terms described in this specification may reduce or altogether avoid the technical inefficiencies discussed above.

The invention is defined by a computer-implemented method according to claim <NUM>, by a system according to claim <NUM>, by a non-transitory computer-readable medium according to claim <NUM> and by a computer program according to claim <NUM>.

One or more implementations can include the following optional features. For example, in some implementations, the multiple geographic regions comprise at least a neighborhood encompassing the location, a sub-locality encompassing an entire area of the neighborhood, and a locality encompassing an entire area of the neighborhood.

In some implementations, the neighborhood is associated with a first feature score, the sub-locality is associated with a second feature score, and the locality is associated with a third feature score, the first feature score having a value greater than a value of the second feature score and the value of the second feature being greater than a value of the third feature score, and the representative transcription is selected based at least on the respective values of the first feature score, the second feature score, and the third feature score.

In some implementations, selecting the set of n-grams that are associated with each of the multiple geographic regions includes, for each of the neighborhood, sub-locality, and the locality: obtaining a collection of n-grams determined to be associated with a region, computing, for each n-gram within the collection of n-grams, (i) a first score reflecting a probability that an n-gram will be used by a particular user at the location associated with the user, and (ii) a second score reflecting a probability that the n-gram will be used by a collection of users, identifying a subset of n-grams within the collection of n-grams that each have a first score that is greater than the second score, and selecting the subset of n-grams for the region.

In some implementations, the representative transcription is selected based at least on the respective values of the first scores of n-grams included within each of the one or more additional candidate transcriptions.

In some implementations, the set of n-grams selected for each of the multiple geographic regions are entity terms representing names of points of interest within the corresponding region.

In some implementations, the set of n-grams selected for each of the multiple geographic regions are entity terms that are included in one or more queries previously submitted by the user.

In some implementations, the method further includes biasing, by the automated speech recognizer, speech recognition of the utterance using the representative transcription.

In some implementations, the one or more additional candidate descriptions generated for the one or more of the multiple geographic regions include corrected transcriptions for each of one or more candidate transcriptions that are identified as incorrect transcriptions for the utterance.

In some implementations, selecting the set of n-grams for each of the multiple geographic regions includes: comparing waveforms associated with one or more of the n-grams that occur in the initial candidate transcription and waveforms associated with one or more n-grams for each of the multiple geographic regions; and determining, based on comparing waveforms associated with one or more of the n-grams that occur in the initial candidate transcription and waveforms associated with one or more n-grams for each of the multiple geographic regions, that at least one n-gram from among the one or more n-grams for each of the multiple geographic regions is phonetically similar a term that occurs in the initial candidate transcriptions.

Other potential features and advantages will become apparent from the description, the drawings, and the claims.

Other implementations of these aspects include corresponding systems, apparatus and computer programs, configured to perform the actions of the methods, encoded on computer storage devices.

In the drawings, like reference numbers represent corresponding parts throughout.

Voice queries that include entity terms such as unique names of local restaurants are often misrecognized because they contain rare terms that are relevant only in specific locations, users, time or combinations of context data. General voice recognition techniques that do not include location data often fail to distinguish between terms that are commonly used in different user locations, and as a result, provide inaccurate transcriptions of a voice query.

Accordingly, techniques described in this specification improve voice recognition accuracy by generating multiple location-specific transcription candidates for a voice query based on data indicating a user location. For instance, prior queries submitted by users at a specified location can be extracted from various data sources such as, for example, previously submitted search queries, voice queries, or other types of user activity data. The extracted data can then be aggregated into different hierarchical region models that each specify a collection of terms for geographic regions of different sizes that are associated with the location, e.g., neighborhood, sub-locality, and a locality. As a result, terms that are only likely to be used in smaller regions such are stored within a neighborhood model, whereas terms that are likely to be used more globally are stored within a sub-locality model or a locality model. The collections of terms associated with each of the multiple region models can then be used to generate alternate transcriptions for a query subsequently transmitted from a specified location.

<FIG> illustrates an exemplary system <NUM> that may be used for location-based voice query recognition. In general, a user <NUM> provides a voice query 104a, encoded as audio data 104b, to the system <NUM>, which then generates a list of initial candidate transcriptions 104c. The system <NUM> also receives location data <NUM>, which is used to generate a list of additional candidate transcriptions 104d, select a representative candidate transcription 104e from among the list of additional candidate transcriptions 104d, and provide a recognition output 104f for output in response to the voice query 104a.

Briefly, the system <NUM> may include an automated speech recognizer (ASR) <NUM> that includes a transcription selector <NUM> and a re-scorer <NUM>, a candidate transcription expander <NUM> that includes a model selector <NUM>, regional models <NUM>, and a transcription generator <NUM>, and a language model biaser <NUM>. More particular details about the operations performed by the system <NUM> are described below.

In more detail, the user <NUM> initially provides a voice query 104a, which is then encoded as audio data 104b. In the example in <FIG>, the user <NUM> provides a voice query that includes the term "ZANH," which is the name of a fictional restaurant in Mountain View, California.

The ASR <NUM> receives the audio data 104b encoding the voice query 104a and generate a list of initial candidate transcriptions 104c for an utterance corresponding to the audio data 104b. The list 104c includes n-grams within a language model that are determined to be phonetically similar to the term "ZANH" included within the voice query 104a. In the example in <FIG>, the ASR <NUM> generates initial candidate transcriptions "ZEN," "SIGN," "SIN," and "SEEN" based on the inferred phonetics of the voice query "ZANH. " In this example, the ASR <NUM> generates such initial candidate transcriptions because the acoustic model and the language model indicate that this is the most likely recognition candidate without receiving data indicating a user location. For instance, as shown, the ASR <NUM> may use the acoustic model and the language model to generate incorrect initial candidate transcriptions because the language model may indicate that "ZEN" is the most likely transcription for the utterance "ZE-EH-EN" (shown in voice query 104a).

The candidate transcription expander <NUM> receives the list of initial candidate transcriptions 104c and additionally receive location data <NUM>. The location data <NUM> may indicate a current user location based on a global positioning system (GPS) signal from a user device <NUM>, and information related to the current user location. For example, the location data <NUM> may include terms associated with the current user location, such as names of nearby establishments, popular queries associated with the current user location, or previously submitted queries when the user <NUM> was last nearby the current location.

In some implementations, the location data <NUM> may also include context data obtained from previous user sessions such as previous search engine results from previous queries such as, for example, knowledge panels, lists, answers, or other types of information related to previous queries. In other instances, the context data may include recently clicked webpages, screen tracking data that indicates user behavior on a user interface such as, for example, click-tracking activity, amount of time spent on particular screens, or other types of user interaction. In such instances, the location data <NUM> may include n-grams that are associated with the user activity data.

In some implementations, the location data <NUM> may additionally or alternatively be extracted from a user profile that indicates user preferences that are determined based on the frequency of previously submitted voice queries, for example, commonly searched restaurants. The user profile may also indicate previous searches that include entity terms such as names of celebrities or popular locations, or different types of user activity data that represent user preferences such as, for example, "pinning" a location or setting a webpage as a "favorite.

In the example depicted in <FIG>, the candidate transcription expander <NUM> receives location data <NUM> that specifies a zip code, "<NUM>," which indicates that the current location of the user is in Mountain View, California. The model selector <NUM> of the candidate transcription expander <NUM> then uses the current user location to identify and select one or more models for geographical regions that are each associated with the user location.

Each model includes a collection of n-grams that are predetermined to be associated with a geographic region specified by the particular model. For example, the n-grams represent entity terms that correspond to, for example, names of regions of interest within the geographic region (e.g., names of businesses, neighborhood names etc.). As described more particularly with respect to <FIG>, the models may be hierarchal such that some models specify n-grams that are associated with a small geographic region such as a neighborhood (e.g., Mountain View, CA), whereas other models specify n-grams that are associated with a larger geographic region such as a sub-locality (e.g., State of California), and other models that specify n-grams that are associated with an entire locality (e.g., the United States of America).

Although region models are described throughout in reference to three levels (e.g., neighborhood, sub-locality, and locality) for simplicity, in some implementations, the system <NUM> is capable of using a greater or smaller number of hierarchical levels based on the particular voice query that is received from the user device <NUM>. For example, in one particular implementation, the system <NUM> may include models for a neighborhood, a sub-locality, a locality, and an administrative area. In addition, while the descriptions throughout are in reference to a single country (e.g., the United States), the system <NUM> may also be capable of aggregating global data for multiple counties. In this regard, the region hierarchy may include region models for different regions within a particular country, different regions within a particular continent, or different regions within the entire globe.

The different models can be used to adjust the selection of terms based on a designated specificity associated with each model. For example, a neighborhood model can include n-grams that are more likely used by users within a particular location based on the territorial boundaries for the geographic region being smaller (and thus, more specific) compared to the n-grams that are included within a sub-locality or locality models. As described in more detail below, the geographical classifications of the models can be further used to adjust the selection of n-grams to be included within additional candidate transcriptions for the voice query 104a.

In some implementations, the different hierarchies of models for a certain geographic locality can be inclusive of one another. For example, a locality model for the United States can include all n-grams that are determined to be associated with the United States, a sub-locality model for California can include a subset of the n-grams within the locality model that are determined to be associated only with California, and the neighborhood model for Mountain View can include a subset of the n-grams within the sub-locality model that are determined to be associated with Mountain View. In this example, n-grams that are included within the neighborhood model are also included within the sub-locality and locality models. In addition, there are some n-grams that may be included within the sub-locality and/or locality models but are not included within the neighborhood model (e.g., terms associated with Sacramento, CA). Thus, the different hierarchies can be used to generate alternate transcription sequences with different regional specificities with similar phonetic similarities.

In some implementations, the regional models <NUM> may periodically be updated with information transmitted from a mapping database that associates certain n-grams with hierarchies of different geographic regions. For instance, as described more particularly with respect to <FIG>, n-grams previously used by a collection of users at specified locations may be aggregated in order to identify specific n-grams that are either neighborhood-specific (e.g., more likely to be used only by users within a particular neighborhood), sub-locality-specific, or locality-specific. This is accomplished by processing and correlating location data in relation to user query data for the collection of users. In this regard, the updates to the regional models <NUM> can be used to reflect changes in usage of n-grams (e.g., local usage to global usage or vice versa).

The model selector <NUM> selects different models such that the collection of n-grams specified by each of the selected models can then be used to generate additional candidate transcriptions for the voice query 104a. In some implementations, the user location indicated by the location data <NUM> is used to identify a corresponding model within each region hierarchy (e.g., a best-matched neighborhood model, a best-matched sub-locality model, and a locality model that includes both), and then process the different collections of n-grams specified by each region model in parallel. In other implementations, the model selector <NUM> may instead select multiple models for each type of model within the region hierarchy. In such implementations, the model selector <NUM> may select a neighborhood model that includes the user location as well as other neighborhood models for neighborhoods that are nearby the neighborhood that includes the user location. Likewise, the model selector <NUM> may also select multiple sub-locality, and/or multiple locality models.

The transcription generator <NUM> uses the set of n-grams specified by each of the region models selected by the model selector <NUM> and then generates one or more additional candidate transcriptions for each of the selected region models. In the example shown in <FIG>, the transcription generator <NUM> generates a single additional candidate transcription for each of the neighborhood model, the sub-locality model, and the locality model selected based on the user location specified by the location data <NUM>. In other examples, the transcription generator <NUM> may select multiple additional candidate transcriptions for each model.

The transcription generator <NUM> generates the additional candidate transcriptions included within the table 104d based on comparing the collection of n-grams specified by each of the region models to the initial candidate transcriptions included within the table 104c. For each region model, the transcription generator <NUM> then generates one or more additional candidates that include n-grams from the collection of n-grams for the region model that (i) do not occur within the table 104c, and (ii) are indicated as being phonetically similar to the terms that do occur within the table 104c. As an example, for the neighborhood model, the transcription generator <NUM> identifies "XAHN" as an additional candidate transcription because this n-gram was not identified within the table 104c as an initial candidate transcription but is phonetically similar to the initial candidate transcription "ZEN.

The transcription generator <NUM> may additionally compute a recognition score for each of the additional candidate transcriptions generated for the selected region models. The recognition score can be used to reflect a likelihood that the additional candidate transcription reflects an accurate transcription of the voice query 104a. In some implementations, the recognition score is computed based on comparing the probability that a particular n-gram specified within a regional model is likely to be used by a user within the geographical region specified by the model, and the global probability that the particular n-gram is likely to be used by any user within any location (e.g., a probability indicated within a baseline language model used by the ASR <NUM>). In this regard, the recognition score enables the system <NUM> to compare the usage of n-grams within different geographic regions relative to global usage to determine whether a likelihood of an additional candidate recognition is highly likely to be an accurate transcription for the voice query 104b given the user's present location.

The ASR <NUM> receives the list of additional candidate transcriptions 104d and selects a representative candidate transcription for output as the recognition output 104e. For instance, in some implementations, the ASR <NUM> may initially generate respective confidence scores for each of the initial transcriptions within the list 104c and the additional candidate transcriptions within the list 104d. The confidence score may represent the likelihood that each particular candidate transcription represents an accurate transcription of the voice query 104a. As shown in the example in <FIG>, the ASR <NUM> determines, based on both the audio data 104b and the location data <NUM>, a low confidence score for the initial candidate transcription "ZEN" because of the current user location indicated by zip code in the location data <NUM>. Alternatively, the ASR <NUM> determines a high confidence score for the additional candidate transcription "ZANH" because it is phonetically similar to the audio data 104b and because the term "ZANH" is associated with the zip code in the location data <NUM>.

In more detail, transcription selector <NUM> initially selects a particular candidate transcription for output from among the initial candidate transcriptions and the additional candidate transcriptions that represents the candidate transcription that is most likely to represent an accurate transcription. For instance, the transcription selector <NUM> may compare the respective confidence scores assigned to each of the initial candidate transcription and the additional candidate transcription, and select the particular candidate transcription with the highest confidence score. As shown in the example in <FIG>, the transcription selector <NUM> chooses the additional candidate transcription "ZANH" for this transcription having a higher confidence score given the location data <NUM> indicating the current location of the user <NUM>. The ASR <NUM> then provides the recognition output 104e as "ZANH" based on selecting the additional candidate transcription.

In some implementations, the ASR <NUM> may additionally include the re-scorer <NUM>, which rescores the confidence scores calculated by ASR <NUM> for each particular candidate transcription. For instance, the re-scorer <NUM> may additionally compare the phonetic similarity between each of candidate transcription and the audio data 104b to determine which individual candidate transcription represents the transcription that is most likely to be the correct transcription. For example, if the audio data 104b includes significant amounts of noise, the re-scorer <NUM> may adjust the confidence scores assigned to each of the initial candidate transcription 104c and the additional candidate transcription 104d such that the ASR <NUM> appropriately select the candidate transcription that is most likely to be an accurate transcription.

Although <FIG> illustrates that the representative candidate transcription as being selected by the ASR <NUM>, in other implementations, the representative candidate transcription may alternatively be selected by the candidate transcription expander <NUM>. For example, in such implementations, the transcription generator <NUM> may also perform the operations of the transcription selector <NUM> to in order to provide the ASR <NUM> with a representative transcription that is determined to be the most accurate transcription for the voice query 104a. In this example, processing of additional candidate transcriptions 104d is externalized from the ASR <NUM> such that the ASR <NUM> only transmits the representative transcription selected by the candidate transcription expander <NUM> for output to the user device <NUM>.

In some implementations, the representative transcription may additionally or alternatively be selected based on biasing a baseline language model used by the ASR <NUM> to generate recognition outputs for the voice query 104a. For example, the n-grams included within the table 104d can be sent to the language model biaser <NUM> to positively bias the baseline language model such that there is an increased probability that such n-grams are selected as a candidate transcription sequence by the ASR <NUM>. In such examples, the language model biaser <NUM> may increase the baseline probability of a particular n-gram using Equation (<NUM>), below: <MAT>.

In Equation (<NUM>), LMBoost represents positive bias to the baseline language model probability for a particular n-gram, P(n-gram | location) represents the probability of the n-gram being used at the user location indicated by the location data <NUM>, P(n-gram) represents the probability of the n-gram within the baseline language model, and FS represents a feature score associated with a region model that includes the n-gram.

The value of the feature scores can be used to adjust the biasing of individual n-grams included within the table 104d based on the associated region models. For example, a neighborhood model may be associated with a larger feature score compared to a local model such that an n-gram identified within the neighborhood model is more heavily biased compared to a related n-gram identified within the locality module. In this example, n-grams from neighborhood models are more heavily biased because the neighborhood model is associated with a smaller geographic region, increasing the probability that a user within the user location indicated by the location data <NUM> is likely to use an n-gram from the neighborhood model compared to a related n-gram from the locality model.

<FIG> illustrates an exemplary process <NUM> for generating hierarchical region models used to improve voice query recognition. Briefly, the process <NUM> can be executed by a server <NUM> that obtains query log data 204a, 204b, and 204c for users 202a, 202b, and 202c, respectively. The server <NUM> processes and aggregates the n-grams identified within the query log data 204a, 204b, and 204c into storage <NUM>. The server <NUM> then categorizes the identified n-grams in order to generate a set of region models (e.g., the region models <NUM>) that includes a neighborhood model 214a, a sub-locality model 214b, and a locality model 214c.

In more detail, the region models can correspond to geographic regions of different sizes such that each model specifies a collection of n-grams associated with a different region size. In the example depicted, a neighborhood model 214a specifies a collection of entity n-grams that are all associated with "Mountain View, CA," a sub-locality model 214b specifies a collection of entity n-grams that are all associated with California, USA," and a locality model 214c specifies a collection of entity n-grams that are all associated with the "West Coast" region of the United States that includes multiple states.

The server <NUM> initially obtains query log data 204a, 204b, and 204c associated with the users 202a, 202b, and 202c, respectively. As described above, the query log data can include historical data such as voice or text queries submitted by the users 202a-c, among other types of data such as search query data, or application input data. The query log data include a collection of n-grams that were included within the historical data associated with each user.

The server <NUM> performs various processing operations in order to select n-grams that are extracted from the query log data and stored in the storage <NUM>. For example, the server <NUM> may annotate text segments to extract entity n-grams and remove non-entity n-grams. For instance, in the example, the text segment "IN CANON CITY" is annotated to extract "CANON CITY" and remove "IN.

In addition, the server <NUM> uses various heuristics to filter out n-grams that are not to be included within the region models. For example, the server <NUM> may filter out misspellings, n-grams that only few users use, and/or n-grams that are only popular within the location due to misrecognitions. For instance, in the example, the n-gram "OBAMA" is removed from the query log data since its frequency of usage by all users is higher than a specified threshold for including within a region model. In another example, server <NUM> may filter out n-grams where the ratio of correctly spelled queries to incorrectly spelled queries within a location is below a threshold, indicating that the n-gram is due to a common misrecognition error across all users. In another example, the server <NUM> may filter not n-grams where the different between the subset spell-correction-ratio and global spell-correction-ratio exceed a threshold, indicating that the n-gram is a local spelling error. In yet another example, the server <NUM> may also filter out n-grams that were included in a lot of voice queries but not in text queries, indicating that the n-gram is only mistakenly recognized. In yet another example, the server <NUM> may filter out n-grams where the difference between the subset voice-ratio and the global voice-ratio is above a threshold, indicating that the n-gram is a local misrecognition, e.g., "canon city" being misrecognized as "canyon city.

In the example depicted, each of the users 202a, 202b, and 202c each submit queries in different locations, e.g., "Mountain View, CA," "Sacramento, CA," and "Phoenix. " Based on these locations, the server <NUM> generates the neighborhood model 214a such that it only includes n-grams obtained from the query log 204a. In addition, because the user 202a and the user 202b, but not the user 202c, are located in California, the sub-locality model 214b includes terms from the query logs 204a and 204b but not from the query log 204c. Finally, since all of the users 202a, 202b, and 202c are located on the west coast of the United States, e.g., California and Arizona, the locality model 214c includes n-grams obtain from each of the query logs 204a, 204b, and 204c.

<FIG> illustrates an exemplary process <NUM> for improving voice query recognition using hierarchical region models. Briefly, the process <NUM> may include receiving audio data and data indicating a location associated with a user (<NUM>), obtaining an initial candidate transcription (<NUM>), selecting a set of n-grams (<NUM>), generating one or more additional candidate transcriptions (<NUM>), selecting a representative transcription (<NUM>), and providing the representative transcription to an automated speech recognizer (<NUM>).

In more detail, the process <NUM> may include receiving audio data and data indicating a location associated with a user (<NUM>). For instance, the ASR <NUM> may obtain the audio data 104b encoding an utterance of the voice query 104a submitted by the user <NUM> on the user device <NUM>. The ASR <NUM> may also obtain location data <NUM> based on a GPS signal of the user device <NUM>.

In some implementations, the ASR <NUM> may also obtain context data extracted from previous user sessions such as, for example, previous search queries, search engine results, or answers to voice queries. In other instances, the context data may be extracted from user activity data such as screen tracking data, recently clicked hyperlinks, or user profiles indicating preferences. The context may additionally or alternatively include a current user location and terms associated with the user location such as names of nearby locations, or common queries submitted by the user <NUM> or other users within the specified location.

The process <NUM> may include obtaining an initial candidate transcription (<NUM>). For example, the candidate transcription expander <NUM> may obtain an initial candidate transcription included within the list of initial candidate transcriptions 104c generated by the ASR <NUM>. As described above, the initial candidate transcriptions can be identified by the ASR <NUM> based on an acoustic model and a language model used to recognize the voice query 104a.

The process <NUM> selecting a set of n-grams (<NUM>). For instance, the candidate transcription expander <NUM> may select a set of n-grams from among a collection of n-grams of each of the region models <NUM>. In some implementations, the model selector <NUM> may initially select a neighborhood model, a sub-locality model, and a locality model based on the user location indicated by the location data <NUM>. In other implementations, the model selector <NUM> may select multiple region models within the model hierarchy, e.g., multiple neighborhood models, multiple sub-locality models, and multiple locality models.

The process <NUM> may include generating one or more additional candidate transcriptions (<NUM>). For example, the candidate transcription expander <NUM> may generate additional candidate transcriptions for one or more of the region models selected by the model selector <NUM>. The additional candidate transcriptions that are generated by the transcription generator <NUM> can then be included within the table 104d. As described above, each additional candidate includes n-grams that were not included within the list of initial candidate transcriptions 104c, and that are indicated as being phonetically similar to one of more terms that do exist within the list of initial candidate transcriptions 104c.

The process <NUM> may include selecting a representative transcription (<NUM>). For example, either the ASR <NUM> and/or the candidate transcription expander <NUM> may select a representative transcription from among the initial candidate transcriptions in the table 104c and the additional candidate transcriptions in the table 104d. As described above, the selection may be based on the respective confidence scores for each of the individual candidate transcriptions. In some implementations, prior to the selection of the representative transcription, the computed confidence scores for the initial candidate transcriptions within the table 104c may be adjusted based on the generation of the additional candidate transcription scores. In such implementations, the candidate transcription with the greatest confidence score is then selected as the representative transcription.

In some implementations, the representative transcription is based on biasing a baseline language model used by the ASR <NUM> to generate a recognition output for the voice query 104a. For instance, as described above, n-grams within the additional candidate transcriptions may be sent to the language model biaser <NUM> to positively bias the baseline language model to increase the respective probabilities associated with the n-grams included within the additional candidate transcriptions. The language model biaser <NUM> may positively bias the baseline language model based on computing a language probability boost according to equation (<NUM>) described above. In such implementations, a respective probability within the baseline language model may be increased if the probability of an n-gram being used within a geographic region specified by a region model is greater than the probability of the n-gram within the language model.

The process <NUM> may include providing the representative transcription to an automated speech recognizer (<NUM>). For example, the candidate transcription expander <NUM> may provide the representative transcription to the ASR <NUM>. The representative transcription can then be used by the ASR <NUM> to generate the recognition output 104e.

<FIG> is a schematic diagram of an example of a generic computer system <NUM>. The system <NUM> can be used for the operations described in association with <FIG> according to some implementations. The system <NUM> may be included in the systems <NUM> and <NUM>.

The system <NUM> includes a processor <NUM>, a memory <NUM>, a storage device <NUM>, and an input/output device <NUM>. Each of the components <NUM>, <NUM>, <NUM>, and <NUM> are interconnected using a system bus <NUM>. The processor <NUM> is capable of processing instructions for execution within the system <NUM>. In one implementation, the processor <NUM> is a single-threaded processor. In another implementation, the processor <NUM> is a multi-threaded processor. The processor <NUM> is capable of processing instructions stored in the memory <NUM> or on the storage device <NUM> to display graphical information for a user interface on the input/output device <NUM>.

The features described can be implemented in digital electronic circuitry, or in computer hardware, firmware, software, or in combinations of them. The apparatus can be implemented in a computer program product tangibly embodied in an information carrier, e.g., in a machine-readable storage device, for execution by a programmable processor; and method steps can be performed by a programmable processor executing a program of instructions to perform functions of the described implementations by operating on input data and generating output. The described features can be implemented advantageously in one or more computer programs that are executable on a programmable system including at least one programmable processor coupled to receive data and instructions from, and to transmit data and instructions to, a data storage system, at least one input device, and at least one output device. A computer program is a set of instructions that can be used, directly or indirectly, in a computer to perform a certain activity or bring about a certain result.

The elements of a computer are a processor for executing instructions and one or more memories for storing instructions and data.

Claim 1:
A computer-implemented method comprising:
receiving, by an automated speech recognizer, audio data (104b) corresponding to an utterance of a user (<NUM>, <NUM>),
receiving, by a candidate transcription expander, data indicating a location associated with the user (<NUM>, <NUM>);
obtaining, by the automated speech recognizer, an initial candidate transcription (104c, <NUM>) of the utterance using an acoustic model and a baseline language model, the baseline language model including n-grams;
selecting, by a model selector of the candidate transcription expander, for each of multiple geographic regions that are associated with the location, a set of n-grams (<NUM>) that are associated with a region;
for one or more of the multiple geographic regions, generating, by a transcription generator of the candidate transcription expander, one or more additional candidate transcriptions (<NUM>) based on (i) the initial candidate (104c, <NUM>) transcription, and (ii) one or more terms of the set of n-grams (<NUM>) that are associated with the region, that do not occur in the initial candidate transcription (104c, <NUM>), and that are indicated as phonetically similar to one or more terms that do occur in the initial candidate transcription (104c, <NUM>);
selecting, by the automated speech recognizer, from among the initial candidate transcription (104c, <NUM>) and the one or more additional candidate transcriptions (<NUM>), a representative transcription (<NUM>) of the utterance, the representative transcription being based on a biasing of the baseline language model by a language model biaser, the baseline language model being positively biased by n-grams included within the additional candidate transcriptions (<NUM>) that increase respective probabilities associated with the n-grams included within the additional candidate transcriptions (<NUM>); and
outputting the representative transcription (<NUM>) by the automated speech recognizer (<NUM>, <NUM>).