Patent Description:
Recent technological advances have allowed the somewhat widespread use of automated speech recognition (ASR) tools, by which computing devices convert speech to text without human intervention. ASR tools have proven useful in numerous applications, including voice user interfaces that rely on speech to text tools to convert spoken commands to text that can be interpreted, and speech to text processing that allows people to perform word processing tasks without typing.

ASR currently suffers from significant limitations, however. Unassisted ASR tools often suffer from limited accuracy. In particular, ASR systems often have difficulty when dealing with words that sound identical or similar yet have different meanings such as "meet" and "meat", difficult to pronounce words that are often spoken incorrectly to the ASR tool, mispronounced words, and noise in the speech input signal. These and other factors result in decreased accuracy of the ASR system and user frustration.

<CIT> describes performing a search for content via the Internet, wherein user input specifying a search query is supplied to a mobile communications device, such as, for example, a smartphone. The mobile communications device separately issues the search query to a plurality of search engines and can receive the results from each search engine and display the results to the user. Thus, the user does not have to separately issue the query to each of the plurality of search engines.

<CIT> discloses an offline semantic processor of a resource-constrained voice-enabled device such as a mobile device utilizes an offline grammar model with reduced resource requirements to parse voice-based queries received by the device. The offline grammar model may be generated from a larger and more comprehensive grammar model used by an online voice-based query processor, and the generation of the offline grammar model may be based upon query usage data collected from one or more users to enable a subset of more popular voice-based queries from the online grammar model to be incorporated into the offline grammar model. In addition, such a device may collect query usage data and upload such data to an online service to enable an updated offline grammar model to be generated and downloaded back to the device and thereby enable a dynamic update of the offline grammar model to be performed.

Accordingly, to overcome the lack of accuracy in ASR systems, systems and methods are described herein for a computer-based process that generates hint words to assist in the automated speech recognition process. The use of hint words, or additional words input to the ASR system along with speech input, helps provide context for the spoken words and thus increases accuracy. Embodiments of the disclosure describe improvements in the determination of hint words, which thus results in increased accuracy of ASR systems.

In one embodiment, it is noted that speech to text operations are at times carried out in relation to the use of a voice interface system. More specifically, users of voice interface systems may issue voice queries or commands to initiate various operations such as conduct an electronic search for information, purchase one or more products, direct the operation of various electronic devices, and the like. ASR systems interpreting these voice commands are aided by contextual information, which can be determined from prior operation of the voice interface system. Accordingly, hint words are taken from terms that have arisen during operation of the voice interface system. This provides valuable context information for voice commands issued in connection with use of the voice interface system, thus improving the accuracy of any speech to text operations. These terms can be any terms of search queries spoken to the voice interface system, or any terms of commands uttered to the system. Terms can also be any other word or phrase, including terms such as names of consumer goods, tasks, reminders, calendar items, dates, or items of a list of items, as many of these may be uttered to voice interface systems.

As one example, hint words for voice queries issued in connection with a voice interface system can be determined according to the most frequently occurring terms arising in operation of this voice interface system. These terms can be determined from any such operation. For example, terms can be selected from electronic search queries issued through the voice interface system. Terms can also be taken from commands issued through or to the voice interface system. These terms, or some subset thereof, can be selected as hint words for transmission to, and use by, an ASR application.

Hint words can be taken from this set of terms in any manner. As one example, a predetermined number of the most frequently occurring terms may be selected as the hint words. For instance, the top <NUM>, <NUM>, or <NUM> most frequently occurring terms may be used as hint words. As another example, hint words can be taken from a predetermined number of the terms that occur most frequently during some predetermined time period. Alternatively, a combination of the above two examples may be used to determine hint words. That is, a predetermined number of the most frequently occurring terms, and a predetermined number of the most frequently occurring terms over some particular time period, can be used as the hint words. In another embodiment, common terms between the most frequently occurring terms and the most frequently occurring terms over a particular time period can be used as hint words.

Selecting hint words from those terms common to the set of most frequently occurring terms, and the set of most frequently occurring terms over a particular time period, may result in an insufficient number of terms, as these two sets may sometimes have few terms in common. Thus, additional terms may be further selected as hint words. In particular, if less than some predetermined number of terms are common to both sets, additional terms can be selected from the set of most frequently occurring terms, to take the total number of hint words up to the predetermined number. That is, additional terms can be picked from those terms of the first set that are not common to the second set, until the total number of terms reaches the predetermined number. These additional terms can be selected from the first set in any manner, such as in order of most frequent occurrence.

In the alternative, additional terms can be picked from the second set rather than the first set. That is, additional terms can be picked from the set of most frequently occurring terms over a particular time period, until the total number of terms reaches the predetermined number. In this case, additional terms can be taken from those members of the second set that are not common to the first set, in order of most frequent occurrence or in any other order.

As described above, some embodiments may select terms from one or more of terms of electronic search queries, or commands spoken. It is noted, though, that this is not an exhaustive list, and terms may be selected from other sources. For example, terms may be selected from some predetermined number of the most recent electronic search queries, or terms of some predetermined number of commands most recently issued. As another example, phonemes or phonetic neighbors of any terms may also be used as hint words.

Hint words are determined using a graph data structure. More specifically, terms can be taken from a predetermined number of recent electronic searches. A graph of terms is consulted. This graph is a graph data structure which may be constructed from any group of terms. Each term taken from the electronic searches is compared to the graph. For each such term that appears in the graph, proximate (e.g., neighboring) terms can be selected. The sets of such proximate terms are then be compared to each other. Terms common to each of these sets form a set of hint words. These hint words are then transmitted to an ASR application to assist in converting voice commands or other speech to text.

The graph can be any graph of terms. As one example, the graph can be constructed from a set of entertainment terms, or any other group of terms that are likely to appear in the context of, e.g., a voice interface system.

Any proximate terms can be selected. For example, terms can be selected in order of proximity. In one such embodiment, nearest connected terms can be selected first and, if an insufficient number of nearest terms exists, next-nearest connected terms can also be selected, with this process repeating until a sufficient number of terms have been selected for use as hint words.

Common terms may also be selected in any manner. For instance, terms common to every group of proximate terms may be selected as hint words. If this results in an insufficient number of hint words, one group of proximate terms may be removed, and terms common to every remaining group of proximate terms may be selected as additional hint words. If the total number of hint words remains insufficient, this process may be repeated with successive groups of proximate terms removed, with terms common to the remaining groups of proximate terms also selected as hint words, until a sufficient number of hint words has been selected.

Each term may be a single word, i.e., a natural language word, or may be a collection of words such as a natural language phrase.

Hint words may be taken from proximate terms on a graph. These proximate terms may be any other words associated with the terms in some manner. Thus, for example, hint words may include phonemes of the terms selected, as well as phonetic neighbors of those terms.

It is also noted that the number of search queries considered, and the number of proximate terms, may be the same number or different numbers.

It is noted that embodiments of the disclosure are not limited to voice interface systems, and encompass any system employing hint words. For example, embodiments of the disclosure may be employed in connection with content display systems such as media guides. In particular, media guides that employ voice interfaces allow users to search for and select content via voice commands. When such content display systems are used, terms can be taken from search terms, the titles of content played by the display system, from titles of content the user has liked or otherwise indicated as his or her favored content, or from terms of content that the user has disliked in some manner.

The disclosure relates to systems and methods for determining hint words that improve the accuracy of ASR systems. Hint words are determined in the context of a user issuing voice commands in connection with a voice interface system. Terms are initially taken from most frequently occurring terms arising in operation of a voice interface system. For example, most frequently occurring terms that arise in electronic search queries or user-issued commands are selected. Certain of these terms are selected as hint words, and the selected hint words are then transmitted to an ASR system to assist in translation of speech to text. Selection of terms as hint words may be accomplished according to any criteria and in any manner. For example, a certain number of the most frequently occurring terms may be selected as hint words. As another example, a specified number of the terms that occur most frequently in some predetermined time period may be selected as hint words. Hint words may be selected in various other ways as well.

<FIG> illustrates operation of an ASR system without hint words, and operation of an ASR system with hint words determined in accordance with embodiments of the disclosure. Here, an ASR system <NUM>, which can be any ASR system implementing any speech to text methods and processes, receives an audio signal as input and outputs a corresponding text translation of the audio signal. In this case, the input audio signal is an audio recording of the spoken statement "Movies with no censor. " The output may be either the text "Movies with no sensor" or "Movies with no censor. " As the ASR module <NUM> does not receive any context information, it cannot distinguish between two different words that sound identical. That is, the ASR module <NUM> correctly notes that the sequence of sounds it receives for the spoken term "censor" can be the text word "censor" or the text word "sensor," and without any knowledge of the context of the input sentence, it cannot reliably determine which of the two is correct.

In contrast, ASR module <NUM> receives both an input audio signal and a set of hint words. These hint words provide context information to the ASR module <NUM> that was missing from the input to previous ASR module <NUM>. In particular, the hint words "movie," "tv show," "censor," and "rating" indicate an entertainment-related context for the input audio signal. That is, the terms or words of the input audio signal may be entertainment-related. Thus, the ASR module <NUM> selects the term "censor" as being within the context of the input audio signal, rather than the term "sensor" which is unrelated to entertainment. Accordingly, ASR module <NUM> utilizes input hint words to select the correct translation "Movies with no censor" rather than the incorrect "Movies with no sensor.

The hint words input to ASR module <NUM> can be any terms, i.e., any one or more words, that provide an accurate context for an input audio or speech signal. As an example, hint words can be taken from frequent words used by a user or reflecting his or her behavior, e.g., words from electronic searches the user initiates, or words the user frequently utters when issuing voice commands to his or her voice interface system.

<FIG> is a block diagram illustration of a system for implementing processes of hint word determination in accordance with embodiments of the disclosure. A computing device <NUM> may be in communication with an ASR server <NUM> through, for example, a communications network <NUM>. ASR server <NUM> is also in electronic communication with conversation processing server <NUM> also through, for example, the communications network <NUM>. Computing device <NUM> may be any computing device running a user interface, such as a voice assistant, voice interface allowing for voice-based communication with a user, or an electronic content display system for a user. Examples of such computing devices are a smart home assistant similar to a Google Home® device or an Amazon® Alexa® or Echo® device, a smartphone or laptop computer with a voice interface application for receiving and broadcasting information in voice format, a set-top box or television running a media guide program or other content display program for a user, or a server executing a content display application for generating content for display to a user. ASR server <NUM> may be any server running an ASR application. Conversation processing server <NUM> may be any server programmed to determine hint words in accordance with embodiments of the disclosure, and to transmit the hint words to the ASR server <NUM>. For example, conversation processing server <NUM> may be a server programmed to determine hint words by retrieving terms entered into computing device <NUM> when the user is operating device <NUM> to view content.

The computing device <NUM>, e.g., device <NUM>, may be any device capable of acting as a voice interface system such as by running one or more application programs implementing voice-based communication with a user, and engaging in electronic communication with server <NUM>. For example, computing device <NUM> may be a voice assistant, smart home assistant, digital TV, laptop computer, smartphone, tablet computer, or the like. <FIG> shows a generalized embodiment of an illustrative user equipment device <NUM> that may serve as a computing device <NUM>. User equipment device <NUM> may receive content and data via input/output (hereinafter "I/O") path <NUM>. I/O path <NUM> may provide content (e.g., broadcast programming, on-demand programming, Internet content, content available over a local area network (LAN) or wide area network (WAN), and/or other content) and data to control circuitry <NUM>, which includes processing circuitry <NUM> and storage <NUM>. Control circuitry <NUM> may be used to send and receive commands, requests, and other suitable data using I/O path <NUM>. I/O path <NUM> may connect control circuitry <NUM> (and specifically processing circuitry <NUM>) to one or more communications paths (described below). I/O functions may be provided by one or more of these communications paths but are shown as a single path in <FIG> to avoid overcomplicating the drawing.

Control circuitry <NUM> may be based on any suitable processing circuitry such as processing circuitry <NUM>. As referred to herein, processing circuitry should be understood to mean circuitry based on one or more microprocessors, microcontrollers, digital signal processors, programmable logic devices, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), etc., and may include a multi-core processor (e.g., dual-core, quad-core, hexa-core, or any suitable number of cores). In some embodiments, processing circuitry may be distributed across multiple separate processors or processing units, for example, multiple of the same type of processing units (e.g., two Intel Core i7 processors) or multiple different processors (e.g., an Intel Core i5 processor and an Intel Core i7 processor). In some embodiments, control circuitry <NUM> executes instructions for receiving streamed content and executing its display, such as executing application programs that provide interfaces for content providers to stream and display content on display <NUM>.

Control circuitry <NUM> may thus include communications circuitry suitable for communicating with a content provider <NUM> server or other networks or servers. Communications circuitry may include a cable modem, an integrated services digital network (ISDN) modem, a digital subscriber line (DSL) modem, a telephone modem, Ethernet card, or a wireless modem for communications with other equipment, or any other suitable communications circuitry. Such communications may involve the Internet or any other suitable communications networks or paths. In addition, communications circuitry may include circuitry that enables peer-to-peer communication of user equipment devices, or communication of user equipment devices in locations remote from each other.

Memory may be an electronic storage device provided as storage <NUM> that is part of control circuitry <NUM>. As referred to herein, the phrase "electronic storage device" or "storage device" should be understood to mean any device for storing electronic data, computer software, or firmware, such as random-access memory, read-only memory, hard drives, optical drives, digital video disc (DVD) recorders, compact disc (CD) recorders, BLU-RAY disc (BD) recorders, BLU-RAY 3D disc recorders, digital video recorders (DVR, sometimes called a personal video recorder, or PVR), solid state devices, quantum storage devices, gaming consoles, gaming media, or any other suitable fixed or removable storage devices, and/or any combination of the same. Storage <NUM> may be used to store various types of content described herein as well as media guidance data described above. Nonvolatile memory may also be used (e.g., to launch a boot-up routine and other instructions). Cloud-based storage may be used to supplement storage <NUM> or instead of storage <NUM>.

Storage <NUM> may also store instructions or code for an operating system and any number of application programs to be executed by the operating system. In operation, processing circuitry <NUM> retrieves and executes the instructions stored in storage <NUM>, to run both the operating system and any application programs started by the user. The application programs can include one or more voice interface applications for implementing voice communication with a user, and/or content display applications which implement an interface allowing users to select and display content on display <NUM> or another display.

Control circuitry <NUM> may include video generating circuitry and tuning circuitry, such as one or more analog tuners, one or more MPEG-<NUM> decoders or other digital decoding circuitry, high-definition tuners, or any other suitable tuning or video circuits or combinations of such circuits. Encoding circuitry (e.g., for converting over-the-air, analog, or digital signals to MPEG signals for storage) may also be included. Control circuitry <NUM> may also include scaler circuitry for upconverting and downconverting content into the preferred output format of the user equipment <NUM>. Circuitry <NUM> may also include digital-to-analog converter circuitry and analog-to-digital converter circuitry for converting between digital and analog signals. The tuning and encoding circuitry may be used by the user equipment device to receive and to display, to play, or to record content. The tuning and encoding circuitry may also be used to receive guidance data. The circuitry described herein, including for example, the tuning, video generating, encoding, decoding, encrypting, decrypting, scaler, and analog/digital circuitry, may be implemented using software running on one or more general purpose or specialized processors. Multiple tuners may be provided to handle simultaneous tuning functions (e.g., watch and record functions, picture-in-picture (PIP) functions, multiple-tuner recording, etc.). If storage <NUM> is provided as a separate device from user equipment <NUM>, the tuning and encoding circuitry (including multiple tuners) may be associated with storage <NUM>.

A user may send instructions to control circuitry <NUM> using user input interface <NUM>. User input interface <NUM> may be any suitable user interface, such as a remote control, mouse, trackball, keypad, keyboard, touch screen, touchpad, stylus input, joystick, voice recognition interface, or other user input interfaces. Display <NUM> may be provided as a stand-alone device or integrated with other elements of user equipment device <NUM>. For example, display <NUM> may be a touchscreen or touch-sensitive display. In such circumstances, user input interface <NUM> may be integrated with or combined with display <NUM>. Display <NUM> may be one or more of a monitor, a television, a liquid crystal display (LCD) for a mobile device, amorphous silicon display, low temperature poly silicon display, electronic ink display, electrophoretic display, active matrix display, electro-wetting display, electrofluidic display, cathode ray tube display, light-emitting diode display, electroluminescent display, plasma display panel, high-performance addressing display, thin-film transistor display, organic light-emitting diode display, surface-conduction electron-emitter display (SED), laser television, carbon nanotubes, quantum dot display, interferometric modulator display, or any other suitable equipment for displaying visual images. In some embodiments, display <NUM> may be HDTV-capable. In some embodiments, display <NUM> may be a 3D display, and the interactive media guidance application and any suitable content may be displayed in 3D. A video card or graphics card may generate the output to the display <NUM>. The video card may offer various functions such as accelerated rendering of 3D scenes and 2D graphics, MPEG-<NUM>/MPEG-<NUM> decoding, TV output, or the ability to connect multiple monitors. The video card may be any processing circuitry described above in relation to control circuitry <NUM>. The video card may be integrated with the control circuitry <NUM>. Speakers <NUM> may be provided as integrated with other elements of user equipment device <NUM> or may be stand-alone units. The audio component of videos and other content displayed on display <NUM> may be played through speakers <NUM>. In some embodiments, the audio may be distributed to a receiver (not shown), which processes and outputs the audio via speakers <NUM>.

<FIG> is a generalized embodiment of an illustrative conversation processing server <NUM> constructed for use according to embodiments of the disclosure. Here, device <NUM> may serve as a conversation processing server. Device <NUM> may receive content and data via I/O paths <NUM> and <NUM>. I/O path <NUM> may provide content and data to the various content consumption devices <NUM> and <NUM>, while I/O path <NUM> may provide data to, and receive content from, one or more content providers <NUM>. Like the user equipment device <NUM>, the device <NUM> has control circuitry <NUM> which includes processing circuitry <NUM> and storage <NUM>. The control circuitry <NUM>, processing circuitry <NUM>, and storage <NUM> may be constructed, and may operate, in similar manner to the respective components of user equipment device <NUM>.

Storage <NUM> is a memory that stores a number of programs for execution by processing circuitry <NUM>. In particular, storage <NUM> may store a number of device interfaces <NUM>, an ASR interface <NUM>, hint words module <NUM> for retrieving terms from device <NUM> and selecting hint words therefrom, and storage <NUM>. The device interfaces <NUM> are interface programs for handling the exchange of commands and data with the various devices <NUM>. ASR interface <NUM> is an interface program for handling the exchange of commands with and transmission of hint words to various ASR servers <NUM>. A separate interface <NUM> may exist for each different ASR server <NUM> that has its own format for commands or content. Hint words module <NUM> includes code for executing all of the above described functions for selecting hint words, including retrieving terms from devices <NUM>, selecting hint words therefrom, and sending the selected hint words to ASR interface <NUM> for transmission to ASR server <NUM>. Storage <NUM> is memory available for any application, and is available for storage of terms or other data retrieved from device <NUM>, selected hint words, or the like.

The device <NUM> may be any electronic device capable of electronic communication with other devices and selection of hint words. For example, the device <NUM> may be a server, or a networked in-home smart device connected to a home modem and thereby to various devices <NUM>. The device <NUM> may alternatively be a laptop computer or desktop computer configured as above.

ASR server <NUM> may be any server configured to run an ASR application program, and may be configured similar to server <NUM> of <FIG> with the exception of storing one or more ASR modules in memory <NUM> rather than device interfaces <NUM>, ASR interface <NUM>, and hint words module <NUM>.

<FIG> is a flowchart illustrating process steps for determining hint words for an ASR system, in accordance with embodiments of the disclosure. Initially, hint words module <NUM> determines the most frequently occurring terms arising during operation of a voice interface system on device <NUM> (Step <NUM>). The terms may arise during any operation of the voice interface system, and can include any terms spoken by a user to the device <NUM>, such as search query terms or command words. Terms can also be any other word or phrase spoken to the voice interface system, including terms such as names of consumer goods, tasks, reminders, calendar items, dates, or items of a list of items. When device <NUM> implements another interface such as a content display system, terms may for example be those terms entered in search queries executed by the electronic content display system, terms in any titles of content played by the content display system, and any terms of content liked, disliked, or otherwise positively/negatively described through the content display system, as well as any phonetic neighbors, phonemes, synonyms, and the like.

Terms may be sent to the hint words module <NUM> in any manner. For example, terms may be compiled and transmitted by the device <NUM>, either by its voice interface application or by another application program. Transmission can be performed automatically, or in response to a request from conversation processing server <NUM>. Alternatively, interactions between the user and device <NUM> may be sent to server <NUM>. For example, device <NUM> may transmit to server <NUM> any or all user interactions with its voice interface application, including search queries received, commands issued, or the like. Hint words module <NUM> may then parse each of these and log each individual term they contain, along with a count of their number of times each term appears. Module <NUM> may also add any phonetic neighbors, phonemes, synonyms, or the like, which may be assigned the same frequency as their base terms or may be given any other frequency. This log of terms may then be used to determine hint words.

Once the hint words module <NUM> retrieves and/or determines each term, the most frequently occurring of these terms are selected as hint words (Step <NUM>). That is, the terms most frequently arising in the user's interactions with their voice interface application are deemed likely to provide meaningful context for any voice queries or commands the user may issue for that same voice interface application. Accordingly, such frequently arising terms are deemed good candidates for effective hint words. Selection of frequently arising terms may be performed in any manner, and exemplary selection processes are described further in connection with <FIG> below.

The hint words determined in Step <NUM> are then transmitted to ASR server <NUM> (Step <NUM>), where they are used to improve the accuracy of speech to text operation. Steps <NUM>-<NUM> may be performed at any one or more times, automatically or in response to any signal. For example, Steps <NUM>-<NUM> may be performed in response to a voice query sent to ASR server <NUM> by device <NUM>, such as by a signal from device <NUM> transmitted to server <NUM> when device <NUM> sends its voice query to ASR server <NUM>. Alternatively, Steps <NUM>-<NUM> may be performed automatically upon receipt of new terms or user interaction information from device <NUM>. As a further alternative, device <NUM> may transmit voice queries directly to conversation processing server <NUM> instead of ASR server <NUM>, and server <NUM> may forward the query to ASR server <NUM> along with accompanying hint words after executing Steps <NUM>-<NUM>.

<FIG> is a flowchart illustrating further details of hint word selection, in accordance with embodiments of the disclosure. As described above, the hint words may simply be a predetermined number of the most frequently appearing terms logged by server <NUM> at Step <NUM>. That is, hint words may be taken from the terms most commonly arising in the user's interactions with the voice interface application of his or her device <NUM>. Alternatively, hint words may be taken from the terms most commonly arising in the user's interactions with the voice interface application during a predetermined time period. For example, as above, the hint words module <NUM> may determine a predetermined number of the most frequently appearing terms. This may be referred to as a first set of terms. Additionally, hint words module <NUM> may determine a second set of terms which are the most frequently occurring terms arising during a predetermined time period of operation of the display system (Step <NUM>). Here, device <NUM> transmits not only user interactions with its voice interface application, but also dates and times at which these interactions occur. Hint words module <NUM> may thus log terms and their frequencies as a function of time, for example by logging only those terms that arise within a predetermined time period such as the past week. That is, hint words module <NUM> may keep a running log of only those terms whose interaction dates and times fall within, e.g., the past week. Alternatively, hint words module <NUM> may keep a running log of terms and their interaction dates and times, and filter the log by time to determine only those terms that have arisen within the predetermined time period. The predetermined time period of Step <NUM> may be any time period, e.g., the past week, the past two weeks, last week, last month, any time period specified by the user whether ongoing or already ended, or the like.

Once this second set of terms is determined, hint words module <NUM> can select hint words from the first set of terms and the second set of terms as desired (Step <NUM>). Selection can occur from between the first and second sets of terms in any manner. As one example, module <NUM> may select a predetermined number terms from the first set of terms, and a predetermined number of terms from the second set of terms, and select common terms as hint words. That is, for instance, module <NUM> may select the <NUM> most frequently occurring terms of the first set, and the <NUM> most frequently occurring terms of the second set, and only those terms common to each selected group of <NUM> terms may be picked as hint words. The hint words may also be selected from the most frequently occurring terms in any other manner, such as by simply using every selected term (excluding duplicates) as hint words. The numbers of terms selected from the first and second sets may be any numbers, and the number of terms selected from each set may vary by set.

If common terms are selected, a situation may arise in which there are insufficient common terms. That is, the set of common terms may be too small a set to act as effective hint words. Thus, if less than some predetermined number of common terms is selected, hint words module <NUM> may also select terms from among the first and second sets that are not common to both sets, to bring the total number of selected terms up to some predetermined desired number of terms (Step <NUM>). This predetermined desired number may be any number deemed sufficient to provide a corpus of hint words that will be effective in assisting the ASR server <NUM>, e.g., <NUM> or any other number. These additional terms may be selected from among the first and second sets in any manner. For example, a predetermined number of terms may be selected from each set to bring the sum total of terms to some desired number, e.g., an equal number of terms may be selected from each set, in order of frequency of occurrence, until the desired total number of terms is reached. Alternatively, term selection may be weighted toward the second set if the second set is a set of terms arising within an ongoing or current time period, as such terms may provide more recent contextual information. The desired number of total terms may be any number.

<FIG> is a flowchart illustrating process steps for determining hint words for an ASR system, in accordance with further embodiments of the disclosure. Here, hint words are determined with reference to a graph data structure, i.e., a representation of a set of terms, in which terms are interconnected by links. Graphs are known data structures used to represent terms that have some relation to each other. The process of <FIG> uses graphs of terms taken from the voice interface application of device <NUM>, to determine a set of hint words. Initially, hint words module <NUM> determines terms from a first predetermined number of recent electronic search queries entered in connection with the voice interface application of device <NUM> (Step <NUM>). As above, this Step may be performed in any manner. For example, terms may be compiled and transmitted by the device <NUM>, either by its voice interface application or by another application program. Transmission can be performed automatically, or in response to a request from conversation processing server <NUM>. Alternatively, interactions between the user and device <NUM> may be sent to server <NUM>. For example, device <NUM> may transmit to server <NUM> any or all search queries entered to the voice interface application. Hint words module <NUM> may then parse each of these and log each individual term they contain, along with a count of their number of times each term appears. Module <NUM> may also add any phonetic neighbors, phonemes, synonyms, or the like, which may be assigned the same frequency as their base terms or may be given any other frequency. This log of terms may then be used to determine hint words.

The logged terms are compared to a graph of terms, i.e., a graph data structure populated with terms such as entertainment-related terms. Such graphs are known. As such, existing graphs may be employed, or graphs may be constructed using known methods from terms such as general entertainment-related terms, known entertainment domain entities such as personalities, programs, channels, genres, etc., terms arising in content genres of interest to the user of device <NUM>, and the like. In such graphs, nodes are entities while edges represent relationships between nodes, e.g., "director of," "episode of," "co-actor," and the like. Graphs may be unweighted, or weighted accordingly to how closely connected nodes are. Thus, such graphs can identify words, phrases or entities in previous queries, determine their closely connected nodes, and include those terms in the list of hint words.

Accordingly, for each logged term that is present in the graph, hint words module <NUM> selects a predetermined number of proximate terms from the graph of terms (Step <NUM>). That is, hint words module <NUM> determines a set of proximate terms for each logged term that is also present in the graph. Proximate terms are any nearby terms in the graph, as described further below.

The hint words module <NUM> then selects terms common to each set of proximate terms, with these common terms then forming the set of hint words (Step <NUM>). The set of hint words is then transmitted to the ASR server <NUM> (Step <NUM>) for use in converting speech to text.

Nearby graph terms are selected for use as hint words in any manner. <FIG> are flowcharts illustrating further details of exemplary process steps for selecting nearby terms for use as hint words, in accordance with embodiments of the disclosure. More specifically, <FIG> illustrates further details of Step <NUM> above, as carried out in one embodiment. Here, once terms are determined and logged from device <NUM>, the terms are compared to the graph of terms. For each logged term which also appears in the graph, hint words module <NUM> selects the nearest connected terms, i.e., terms connected by one edge. The logged term and its nearest connected terms together form a set of terms (Step <NUM>). The number of terms in this set is then compared to a predetermined value and, if the value is greater than the number of terms in the set, the set is too small. Next-nearest connected terms are thus determined, i.e., terms two edges distant, and these terms are added to the set of terms. This process repeats with successively more distantly-connected terms, until the number of terms in the set meets or exceeds the predetermined value (Step <NUM>). The predetermined value may be any value.

The selection of terms common to each set may be accomplished in any manner. <FIG> illustrates further details of Step <NUM> above, as carried out in one embodiment. The embodiment of <FIG> contemplates the possibility that there may be too few common terms to serve effectively as hint words. In particular, the number of terms common to every set of proximate terms may be too few to provide context to an ASR program. The process of <FIG> thus reduces the number of sets required by one at each step, until sufficient common terms exist. For example, if N sets of proximate terms exist (N logged terms that appear in the graph, and their proximate graph terms), and there are an insufficient number of terms common to all N sets, the hint words module <NUM> then selects all terms common to N-<NUM> sets and adds those terms to the list of common terms (Step <NUM>), minus duplicates. If insufficient common terms still exist, the hint words module <NUM> then selects all terms common to N-<NUM> sets and adds those terms to the list of common terms, minus duplicates. This process continues (Step <NUM>) until a sufficient number of common terms exists. This number may be any value. Additionally, sets may be removed in any manner. That is, when the process transitions from examining N sets to examining N-<NUM> sets, the set removed is determined in any manner. For instance, smallest sets may be removed first, largest sets may be removed first, sets with the most/least frequently occurring terms may be removed first, or the like.

<FIG> are tables illustrating word frequencies, for selection of terms in accordance with embodiments of the disclosure. As above, once hint words module <NUM> determines terms from recent electronic search queries as in Step <NUM>, module <NUM> generates a log of terms. This log allows module <NUM> to easily determine and retrieve the most frequently occurring terms. <FIG> illustrates one such log, showing terms received from device <NUM> and the number of times each term has arisen in connection with operation of its voice interface application, in order of frequency. This log may be used, for example, to determine hint words in embodiments that consider the most frequently occurring terms as hint word candidates. In contrast, <FIG> illustrates another such log, showing terms received and the number of times each term has arisen during a specified time frame. This log may be used, for example, to determine hint words in embodiments that consider the most frequently occurring terms over a specified time frame.

Claim 1:
A method of determining hint words for automated speech recognition, the method comprising:
determining (<NUM>), using processing circuitry (<NUM>, <NUM>), a set of logged terms from a first predetermined number of recent electronic search queries;
accessing a graph of terms, wherein the graph of terms represents terms and their relations;
comparing each logged term of the set of logged terms to the graph of terms;
for each of the set of logged terms that is present in the graph of terms, selecting (<NUM>) a second predetermined number of proximate terms from the graph of terms, so as to form sets of proximate terms respectively corresponding to each logged term of the set of logged terms present in the graph of terms;
selecting (<NUM>) terms common to each one of said sets of proximate terms, so as to form a set of hint words; and
transmitting (<NUM>), using the processing circuitry (<NUM>), the hint words to the automated speech recognition application (<NUM>).