Patent ID: 12204588

DETAILED DESCRIPTION

FIG.1shows an illustrative example of natural-language user interface100(henceforth NLUI100) that provides a personalized answer to a user's query using a learned query term, in accordance with some embodiments of the disclosure. NLUI100may be an interface for an AI virtual assistant (e.g., Google Assistant™), a search-based application, a database search application (e.g., Amazon Prime™ UI), or any application that provides targeted answers to an input query. Input query102represents a user command/question that can be provided verbally, through typing, through clickable selections, etc. InFIG.1, input query102is a command stating, “Show me flicks with Tom Cruise.” In response to receiving this query, NLUI100outputs response106“We found several flicks with Tom Cruise for you” and results110featuring posters of movies featuring Tom Cruise. In order to fully appreciate how response106is generated by NLUI100,FIG.2andFIG.3should be considered.FIG.2shows illustrative example200depicting query parsing and response templates, in accordance with some embodiments of the disclosure. Input query102is labeled input query202inFIG.2. Using natural language processing, NLUI100parses input query202to determine what the user is asking and how to respond. Display action204is a command “Show me,” that indicates that NLUI100should generate, for display, requested data (e.g., images). Requested information206represents the output that the user is requesting. In this example, the user is requesting flicks. Information modifier208represents clarifications or filters that the user adds in relation to requested information206. In this example, the user specifically wants see flicks including Tom Cruise. NLUI100may refer to a response templates database to retrieve a template that best answers the user query. Because of the potentially large size of the database, NLUI100may use indexing for efficient data lookups. For example, in response to parsing display action204, NLUI100may search only portions of the database associated with displaying images, videos, and text (e.g., skip looking up response templates in a “Play a” or “Tell me” action category). NLUI100may identify multiple response templates that can be used to answer the user query. Response templates210represent potential candidates for generating response106. The template responses may be ranked by NLUI100in order of relevance, as determined by natural language processing techniques. The top template in response templates210, for example, is “We found several movies with <information modifier> for you.”

It should be noted that even though the term “flicks” is used in response template106, the templates in response templates210use the term “movies” instead of “flicks.” Input term104may not be a word in the response templates database. Accordingly, NLUI100determines whether input term104has a definition similar to the definition of a template term in the templates of the response templates database. Identifying a similar definition between two terms enables NLUI100to select the relevant response template to use when generating a response to the query (e.g., select a response template associated with movies), and allows NLUI100to replace the template term with the input term.FIG.3shows illustrative example300for replacing an input term with a template term, in accordance with some embodiments of the disclosure. Input term104is labelled input term302inFIG.3. Template term304“movies” is derived from response templates210inFIG.2. In the example, NLUI100determines that the respective singular versions of the terms “flicks” and “movies” are synonyms and share the definition “a motion picture,” depicted as definition306. It should be noted that the phrase “shares a definition” refers to having matching or similar definitions. In this example, the definition of “flick” is “a motion picture” and the definition of “movie” is “a motion picture.” The respective individual definitions match and this match is phrased as “sharing a definition.” Definitions do not necessarily need to match precisely in order to be considered as “shared.” This is further described in the description ofFIG.16.

In response to determining that input query102includes input term104, which shares definition306with template term304(i.e., the respective terms are synonyms), NLUI100generates a modified template. This modified template may have the same terms and sentence structure as the stored response template, but the template term in the modified template is specifically replaced with input term104. Depicted in modified response templates308, the modified template reads, “We found several flicks with <Information Modifier> for you.” NLUI100then generates and outputs response106“We found several flicks with Tom Cruise for you” based on this modified template. As a result, response106contains input term104“flicks” in place of the template term “movies” found in response templates210. NLUI100may proceed to replace all occurrences of the template term “movies” with input term104(e.g., the other templates of response templates210have been modified to recite “flicks” instead of “movies” in modified response template308).

In some embodiments, rather than generating a response based on modified response templates308, NLUI100generates a response based on response templates210. For example, NLUI100generates a response “We found several movies with Tom Cruise for you.” In response to determining that input term104shares definition306with template term304, NLUI100identifies input term104as a replacement term of template term304. NLUI100then modifies the generated response by replacing the template term “movies” in the generated response with replacement term108“flicks.” The generated modified response is outputted by NLUI100as response106. In some embodiments, NLUI100may also replace the term “movies” by replacement term108“flicks” in results110. Doing so makes results110and response106more accessible and personalized, ensuring that the user can comprehend the response.

It should be noted that input term104appearing in response106is not due to a simple regurgitation of input query102. In other words, some approaches may simply reorganize an input query when creating a response. Such approaches are deficient because they do not adjust how the NLUI application responds to queries in which the input term is not used. In contrast, NLUI100replaces template terms from response templates programmed by a developer with input terms appearing in user queries. NLUI100adapts its own response templates and/or vocabulary database, rather than providing isolated paraphrased input queries as responses. As an example of this adapting process,FIG.4shows illustrative example400in which NLUI100provides an answer comprising a learned input term from previous user queries, in accordance with some embodiments of the disclosure. Input query402may be received by NLUI100after receiving input query102. Input query402states, “What are the showtimes at my local theater?” NLUI100may parse input query402to determine that the user is asking about movie screening times at the user's nearby movie theater. Although input query402does not include input term104“flicks,” because NLUI100has generated modified response templates308featuring “flicks” as a replacement term for “movies,” NLUI100may answer input query402with “Here are some flicks playing near you.” Here, input term102appears as input term406in response404. Had input term104“flicks” not existed in input query102, NLUI100may have replied with “Here are some movies playing near you,” as listed in the original response templates210. NLUI100then outputs results408and response404.

In some embodiments, NLUI100adds “flicks” to a vocabulary database as a replacement term of the template term “movies.” The vocabulary database may be a data structure such as an array or linked list.FIG.5shows diagram500of template terms that are connected with various terms of a vocabulary database, in accordance with some embodiments of the disclosure. Template term502“movie” is depicted in an oval, alongside other template terms: “song,” “lead actor,” “tv show,” and “air.” These template terms may be stored in a vocabulary database, which includes the terms that NLUI100uses to generate responses to queries. Input term504“flick” is depicted in a rectangle, alongside other input terms (e.g., “picture,” “film,” “hero,” “lead star,” etc.) that the user may have used in previous queries. InFIG.5, the input terms that share a definition with a particular template term are visually connected. For example, “picture,” “flick,” and “film” are connected to the template term “movie” because the three terms are determined to be synonyms of “movie.” NLUI100may generate new entries in the vocabulary database with these three words and may add pointers or index values, which point to the new generated entries, to the entry of the template term “movie.” When performing a lookup of replacement terms for template term502, NLUI100retrieves the index values and pointers in the entry of template term502. The retrieved index values and/or pointers may direct NLUI100to input term504.

In some embodiments, NLUI100monitors the number of times an input term is used and stores the counter values in the vocabulary database. For example, counter506represents the number of times input term504was presented in all queries received by NLUI100. Counter506may also be the number of queries, of all queries received by NLUI100, that included input term504. Revisiting example400ofFIG.4, in response to receiving input query402, NLUI100may generate a response such as, “Here are some movies playing near you,” based on the original response templates210. NLUI100may then retrieve a replacement term for the template term “movies” from the vocabulary database. In particular, the NLUI may identify the input term linked with the template term “movie” that is used the most by the user. Based onFIG.5, the input term linked with “movie” that is most used is “film.” Thus, NLUI100may generate response404such that it states, “Here are some films playing near you.”

FIG.5also includes user identifier508. This indicates that NLUI100may create personalized vocabulary databases with terms that are used specifically by a particular user. User identifier508may be a text string, a fingerprint, a login credential to a user profile, or a vocal pattern. For example, if the user is performing a query while logged in to his/her user profile, NLUI100may retrieve a vocabulary database associated with that particular user profile.

FIG.6Ashows illustrative dialogue600between a user and media device602(e.g., a smart speaker) implementing the nature-language user interface application, in accordance with some embodiments of the disclosure. InFIG.1, NLUI100is only presented as a visual interface. However, the NLUI application may also be implemented as an audio-based user interface. In illustrative dialogue600, media device602receives input query604from the user, stating, “Does Tom Cruise have any flicks coming out soon?” The input term is “flicks.” The NLUI application determines that the input query is answered by a stored response template stating, “An upcoming movie with <information modifier> is <result>.” The template term “movie” is further determined by the NLUI application to share a definition with the input term “flick.” In response to determining that the respective terms have similar definitions, the NLUI application may generate a modified template reciting, “An upcoming flick with <information modifier> is <result>” and/or generate a response based on the stored response template while ultimately replacing the template term with the input term in the generated response. In the former, the NLUI may generate a modified template of all stored response templates that include the template term. The audio output from the NLUI application in media device602is response606“An upcoming flick with Tom Cruise is ‘Top Gun Maverick.’”

FIG.6Bshows illustrative dialogue608between the user and media device602where media device602responds to a user query with an input term used in a previous user query, in accordance with some embodiments of the disclosure. InFIG.6A, the NLUI application may have created modified templates for all stored response templates including the template term “movie.” The modified templates may have the template term “movie” replaced with “flick.” Accordingly, in illustrative dialogue608, when receiving input query608“I'm bored, what should I do?” the NLUI application may generate a response based on a modified template. Response610by media device602is “Why not watch a flick such as ‘Halloween’?” The term “flick” represents a replacement term of a template term such as “movie” (i.e., the original response template may have been “Why not watch a movie such as ‘Halloween’?”). If a modified template does not exist to respond to input query608, the NLUI application generates a response based on an appropriate stored response template (e.g., “Why not watch a movie such as <recommended movie>?”) and then replaces the template term “movie” with an input term from the vocabulary database discussed inFIG.5. For example, the NLUI application may generate the response “Why not watch a movie such as ‘Halloween’?” and modify the response to read, “Why not watch a flick such as ‘Halloween’?” The NLUI application outputs the modified response on media device602.

FIG.7shows a generalized embodiment of illustrative media device700. As referred to herein, the phrase “media device” should be understood to mean any device that can output results to user queries. As depicted inFIG.7, media device700is a smartphone. However, media device700is not limited to smartphones and may be any computing device. For example, media device700ofFIG.7can be implemented in system800ofFIG.8as media device802(e.g., a smartphone, a video game console, a smart television, a smart speaker, a computer, or any combination thereof).

Media device700may receive data via input/output (hereinafter “I/O”) path702. I/O path702may provide received data to control circuitry704, which includes processing circuitry706and storage708. Control circuitry704may be used to send and receive commands, requests, and other suitable data using I/O path702. I/O path702may connect control circuitry704(and specifically processing circuitry706) to one or more communications paths (described below). I/O functions may be provided by one or more of these communications paths but are shown as a single path inFIG.7to avoid overcomplicating the drawing.

Control circuitry704may be based on any suitable processing circuitry such as processing circuitry706. As referred to herein, processing circuitry should be understood to mean circuitry based on one or more microprocessors, microcontrollers, digital signal processors, programmable logic devices, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), etc., and may include a multi-core processor (e.g., dual-core, quad-core, hexa-core, or any suitable number of cores). In some embodiments, processing circuitry may be distributed across multiple separate processors or processing units, for example, multiple of the same type of processing units (e.g., two Intel Core i7 processors) or multiple different processors (e.g., an Intel Core i5 processor and an Intel Core i7 processor). In some embodiments, control circuitry704executes instructions for a NLUI application stored in memory (i.e., storage708).

A NLUI application may be a stand-alone application implemented on a media device or a server. The NLUI application may be implemented as software or a set of executable instructions. The instructions for performing any of the embodiments discussed herein of the NLUI application may be encoded on non-transitory computer readable media (e.g., a hard drive, random-access memory on a DRAM integrated circuit, read-only memory on a BLU-RAY disk, etc.) or transitory computer readable media (e.g., propagating signals carrying data and/or instructions). For example, inFIG.7the instructions may be stored in storage708and executed by control circuitry704of a media device700.

In some embodiments, a NLUI application may be a client-server application where only the client application resides on media device700(e.g., media device802), and a server application resides on an external server (e.g., server806). For example, a NLUI application may be implemented partially as a client application on control circuitry704of media device700and partially on server806as a server application running on control circuitry of server806. Server806may be a part of a local area network with media device802or may be part of a cloud computing environment accessed via the Internet. In a cloud computing environment, various types of computing services for performing searches on the Internet or informational databases, providing storage (e.g., for the vocabulary database) or parsing data are provided by a collection of network-accessible computing and storage resources (e.g., server806), referred to as “the cloud.” Media device700may be a cloud client that relies on the cloud computing capabilities from server806to generate the personalized answers of the NLUI application. When executed by control circuitry of server806, the NLUI application may instruct the control circuitry to generate the NLUI application output (e.g., the personalized answer) and transmit the generated output to media device802. The client application may instruct control circuitry of the receiving media device802to generate the NLUI application output. Alternatively, media device802may perform all computations locally via control circuitry704without relying on server806.

Control circuitry704may include communications circuitry suitable for communicating with a NLUI application server or other networks or servers. The instructions for carrying out the above-mentioned functionality may be stored and executed on server806. Communications circuitry may include a cable modem, an integrated services digital network (ISDN) modem, a digital subscriber line (DSL) modem, a telephone modem, an Ethernet card, or a wireless modem for communications with other equipment, or any other suitable communications circuitry. Such communications may involve the Internet or any other suitable communications networks or paths. In addition, communications circuitry may include circuitry that enables peer-to-peer communication of media devices, or communication of media devices in locations remote from each other.

Memory may be an electronic storage device provided as storage708that is part of control circuitry704. As referred to herein, the phrase “electronic storage device” or “storage device” should be understood to mean any device for storing electronic data, computer software, or firmware, such as random-access memory, read-only memory, hard drives, optical drives, solid state devices, quantum storage devices, gaming consoles, or any other suitable fixed or removable storage devices, and/or any combination of the same. Nonvolatile memory may also be used (e.g., to launch a boot-up routine and other instructions). Cloud-based storage (e.g., on server806) may be used to supplement storage708or instead of storage708.

A user may send instructions (e.g., search a query) to control circuitry704using user input interface710of media device700. User input interface710may be any suitable user interface touchscreen, touchpad, stylus and may be responsive to external device add-ons such as a remote control, mouse, trackball, keypad, keyboard, joystick, voice recognition interface, or other user input interfaces. Display710may be a touchscreen or touch-sensitive display. In such circumstances, user input interface710may be integrated with or combined with display712. Display712may be one or more of a monitor, a television, a liquid crystal display (LCD) for a mobile device, amorphous silicon display, low temperature poly silicon display, electronic ink display, electrophoretic display, active matrix display, electro-wetting display, electro-fluidic display, cathode ray tube display, light-emitting diode display, electroluminescent display, plasma display panel, high-performance addressing display, thin-film transistor display, organic light-emitting diode display, surface-conduction electron-emitter display (SED), laser television, carbon nanotubes, quantum dot display, interferometric modulator display, or any other suitable equipment for displaying visual images. A video card or graphics card may generate the output to the display712. Speakers714may be provided as integrated with other elements of user equipment device700or may be stand-alone units. An audio component of the personalized answer and other content displayed on display712may be played through speakers714. In some embodiments, the audio may be distributed to a receiver (not shown), which processes and outputs the audio via speakers714.

Control circuitry704may allow a user to provide user profile information or may automatically compile user profile information. For example, control circuitry704may monitor the words the user inputs in his/her queries. In some embodiments, control circuitry704monitors user inputs that are not queries, such as texts, calls, conversation audio, social media posts, etc., to detect input terms that share definitions with template terms in stored response templates. Control circuitry704may store the detected input terms in a vocabulary database linked to the user profile. Additionally, control circuitry704may obtain all or part of other user profiles that are related to a particular user (e.g., via social media networks), and/or obtain information about the user from other sources that control circuitry704may access. As a result, a user can be provided with a unified experience across the user's different media devices.

As depicted inFIG.8, media device802may be coupled to communication network804. Communication network804may be one or more networks including the Internet, a mobile phone network, mobile voice or data network (e.g., a 4G or LTE network), cable network, public switched telephone network, Bluetooth, or other types of communications network or combinations of communications networks. Thus, media device802may communicate with server806over communication network804via communications circuitry described above. In should be noted that there may be more than one server806, but only one is shown inFIG.8to avoid overcomplicating the drawing. The arrows connecting the respective device(s) and server(s) represent communication paths, which may include a satellite path, a fiber-optic path, a cable path, a path that supports Internet communications (e.g., IPTV), free-space connections (e.g., for broadcast or other wireless signals), or any other suitable wired or wireless communications path or combination of such paths.

FIG.9is a flowchart of detailed illustrative process900for providing a personalized answer to a user's query by generating a modified response template including a learned input term, in accordance with some embodiments of the disclosure. At902, control circuitry704receives a first query via I/O Path702on input interface710of a media device. For example, the first query may be input query102if the media device has a display, or input query604if the output of the media device is audio-based. If the first query is a verbal query (e.g., an audio clip received via a microphone), control circuitry704converts, using voice recognition, the audio clip of the first query into text. At902, control circuitry704additionally parses the first query to determine how to answer the first query. This process is described inFIG.2.

At904, control circuitry704accesses a response template database from memory (e.g., storage708) in order to identify an appropriate response template for answering the first query. For example, input query102states, “Show me flicks with Tom Cruise” which is interpreted by control circuitry704as a command to display movies starring Tom Cruise. Control circuitry704executes a search through the response template database to identify relevant templates that can be used to respond to the user. At906, control circuitry704selects a response template for responding to the first query. For example, control circuitry704may select the response template “We found several movies with <information modifier> for you.”

At908, control circuitry704extracts an input term. The first query includes six terms: “show,” “me,” “flicks,” “with,” “Tom,” and “Cruise.” When parsing the first query, control circuitry704may consider each term as an input term. At910, control circuitry704determines whether an input term shares a definition with a template term in the selected template for responding to the first query. For example, control circuitry704compares the definition of the first term “show” with the definitions of all terms in the selected template (e.g., “we,” “found,” “several,” etc.). Control circuitry704may retrieve the definitions from the Internet or a digital dictionary stored in storage708(e.g., on media device802and/or server806). Control circuitry704may also determine whether the input term appears in an online and/or local digital dictionary as a synonym of any terms in the selected template. The processes for determining whether an input term shares a definition with a template term are further described inFIG.12andFIG.16. In response to determining that the input term “show” does not share a definition with any template term in the selected response template, process900may advance to920, where control circuitry704determines whether there are more terms in the first query to analyze. In this example, four input terms remain for analysis. Thus, in response to determining that there are additional input terms to analyze, process900returns to908where control circuitry704extracts the term “me.” This loop between908,910, and920continues until all input terms are analyzed.

At910, control circuitry704may also determine that the input term does share a definition with a template term. For example, after extracting the term “flicks” from the first query, at910, control circuitry704may compare the definitions of “flicks” and the template term “movies.” In response to determining that the respective terms have similar definitions “a motion picture,” at912, control circuitry704generates a modified template in which the template term is replaced by the input term. For example, control circuitry704generates a new template “We found several movies with <information modifier> for you” and replaces the term “movies” in the new template with “flicks.” The generated modified template thus reads, “We found several flicks with <information modifier> for you.”

At914, control circuitry704generates a response to the first query based on the modified template. As discussed inFIG.2, when parsing the first query in902, control circuitry704may categorize each term of the input query (e.g., as display action204, requested information206, information modifier208, etc.). The modified template features a field in which the “information modifier” string from the input query should be inserted. The input query specifically requests movies with Tom Cruise. Accordingly, control circuitry704inserts the text string “Tom Cruise” into the information modifier field of the modified template. Once all fields of a response template are filled by the required information, the control circuitry704will have generated a response. In this example, control circuitry704generates a response stating, “We found several flicks with Tom Cruise for you.”

From912, process900may proceed to916or918or both916and918. For example, at916, control circuitry704creates modified templates of other stored response templates to replace the template term (e.g., “movies”) in the other response templates with the input term (“flicks”). At918, control circuitry704stores in memory (e.g., storage708) a copy of each modified template generated at912and916.

In the event that there are no input terms that share a definition with a template term of the selected template, control circuitry704may determine whether there is a template term from any of the stored response templates that share a definition with the input term. For example, there may be a response template that includes the term “movie” and is not the selected template. Control circuitry704generates modified templates for those specific response templates with the template term and stores them in memory (e.g., storage708).

At920, if control circuitry704determines that there are no more terms to analyze and no input terms match a template term, process900advances to922where control circuitry704generates a response to the first query based on the selected template. In this case, a modified template is not generated for the selected template. Referring to the overarching example, the response may be “We found several movies with Tom Cruise for you.”

At924, control circuitry704outputs (e.g., via speakers714, display712, or a combination thereof), the generated response. If process900advanced to914, where control circuitry704generated the response based on the modified template, the outputted response may be “We found several flicks with Tom Cruise for you,” as depicted inFIG.1. If process900advanced to922instead, control circuitry704outputs “We found several movies with Tom Cruise for you.”

It should be noted that control circuitry704may detect multiple input terms that share definitions with respective template terms in the selected response template. In response, control circuitry704generates a modified response template based on the selected response template and makes multiple replacements (i.e., replace each input term with the corresponding template term that shares a definition with the respective input term).

FIG.10is a flowchart of illustrative process1000for providing a personalized answer to a user's query by modifying the answer to include a learned input term, in accordance with some embodiments of the disclosure. Process1000begins at1002, where control circuitry704receives a first query. Suppose that the first query is input query604“Does Tom Cruise have any flicks coming out soon?”. At1004, control circuitry704accesses the response template database from memory (e.g., storage708of media device802or server806). At1006, control circuitry704selects a template for responding to the first query. The selected template is retrieved by control circuitry704from the response template database. By employing natural language processing techniques, control circuitry704parses the first query (e.g., identifies keywords and grammatical attributes) and determines the response template in the response template database that shares the keywords and grammatical attributes, thus answering the question/command in the first query. In this example, the selected response template may be “an upcoming movie with <information modifier> is <result>.” At1008, control circuitry704generates a response to the first query based on the selected template. For example, control circuitry704fills in the fields of “information modifier” and “result” (e.g., inserting text strings in the text string of the response template), creating a response that reads, “An upcoming movie with Tom Cruise is ‘Top Gun Maverick.’”

At1010, control circuitry704extracts an input term from the first query. As mentioned previously, control circuitry704identifies all terms in the first query as input terms. The first input term extracted by control circuitry704may be “does.” At1012, control circuitry704determines whether the input term shares a definition with a template term in the generated response of1008. In response to determining that the first input term “does” is not a synonym of any template term in the generated response, at1018, control circuitry704determines whether there are additional terms to analyze. In response to determining that there are additional terms to analyze (e.g., “Tom,” “Cruise,” “flicks,” etc.), process1000returns to1010, where control circuitry704extracts the next input term. In this example, the loop between1010,1012and1018may continue until control circuitry704extracts the term “flicks.” At1012, control circuitry704thus determines that the input term “flicks” shares a definition with the template term “movie.”

In response to determining that an input term shares a definition with a template term, at1014control circuitry704modifies the response to the first query (i.e., the generated response of1008) by replacing the template term with the input term. For example, control circuitry704replaces the term “movie” in the generated response with “flick.” Thus, the modified response reads, “An upcoming flick with Tom Cruise is ‘Top Gun Maverick.’” At1016, control circuitry704stores the input term in a vocabulary database (e.g., in storage708of media device802or server806) as a replacement term of the template term. The vocabulary data is discussed in further detail in the description ofFIG.5. If control circuitry704determines that the term already exists the in the vocabulary database, control circuitry704increments, in the entry of the input term in the vocabulary database, the number of times the input term was used. It should be noted that control circuitry704may identify multiple input terms that share definitions with template terms in the generated response. Accordingly, control circuitry704replaces the respective template terms that share a definition with the corresponding the input term.

At1020, control circuitry704outputs (e.g., via speakers714, display712, or a combination thereof) the generated response. If the generated response of1008was modified by control circuitry704at1014, control circuitry704outputs the modified response.FIG.6Adepicts an example of control circuitry704outputting a modified response. If control circuitry704determines, based on1012and1018, that there are no input terms that share a definition with any template term in the response, at1020, control circuitry704outputs the generated response of1008.

FIG.11is a flowchart of illustrative process1100for providing a personalized answer to a second query that does not comprise the input term by generating a modified template with the input term, in accordance with some embodiments of the disclosure. Control circuitry704may execute process1100subsequent to receiving the first query. Specifically, at1102, control circuitry704receives (e.g., via I/O Path702of input interface710) a second query not comprising the input term. Examples of the second query include input query402and input query608. The input term “flicks” is not included in either example. At1104, control circuitry704accesses in memory (e.g., storage708of media device802or server806) the response template database. At1106, control circuitry704selects a response template for responding to the second query. Although the second query does not include the input term, control circuitry704employs natural language processing techniques to identify a response template that appropriately answers the query. In the case of input query608, the selected template is “Why not watch a movie such as <movie name>?”

At1108, control circuitry704determines whether the selected template includes a template term that can be replaced with an input term. For example, control circuitry704accesses the vocabulary database in memory (e.g., in storage708of media device802or server806) and determines whether the template terms in the selected template have linked replacement terms. For example, as depicted inFIG.5, the template term “movie” can be replaced with the input terms “picture,” “flick,” or “film.” In response to determining that the selected template includes a template term that can be replaced with an input term, at1112control circuitry704generates a modified template of the selected template in which the template term is replaced by the input term (e.g., “Why not watch a flick such as <movie name>?”). At1114, control circuitry704generates a response to the second query based on the modified template by inserting the movie title into the “movie name” field of the template. In response to determining that the selected template does not include a template term that can be replaced with an input term, at1110, control circuitry704generates a response to the second query based on the selected template (without modifications). At1116, control circuitry704outputs (e.g., via speakers714, display712, or a combination thereof) the generated response to the second query. For example, as depicted inFIG.6, control circuitry704outputs, “Why not watch a flick such as ‘Halloween?’”

FIG.12is a flowchart of illustrative process1200for determining whether an input term and a template term have similar definitions based on a clarifying user query, in accordance with some embodiments of the disclosure. When parsing the input query at902or1002, control circuitry704may be unable to determine a template that should be used to respond to the input query. This is further elaborated in process1200where at1202, control circuitry704determines whether the response to the first query cannot be generated. For example, the input query may include an input term that control circuitry704does not understand (e.g., is not present in the vocabulary database and cannot be defined). In response to determining that the response cannot be generated, control circuitry704may generate an error message. For example, in response to receiving the query “Show me flicks with Tom Cruise,” control circuitry704may output, “Sorry, I do not understand.” At1204, control circuitry704monitors for new queries. For example, control circuitry704may wait for a query received subsequent to outputting the error message.

At1206, control circuitry704determines whether a modified version of the first query has been received. The modified version of the first query may follow the same sentence structure as the first query. For example, control circuitry704may receive a query such as “Show me movies with Tom Cruise.” This query may be the user's attempt to clarify his/her original query. Control circuitry704determines that five out of six words match with the original query and are in the same position in the respective sentences. Control circuitry704may compare the number of matching terms between the first query and the modified version and determine that the number (i.e.,5) exceeds a threshold number of matching terms (e.g.,3). In response to determining that the number exceeds the threshold number of matching terms, control circuitry704determines that a modified version of the first query has been received and advances to1208where control circuitry704determines whether the modified version of the first query comprises the template term.

In response to determining that the modified version of the first query has not been received, process1200returns to1204. The loop between1204and1206may continue until a modified version of the first query is received. Control circuitry704may additionally measure a period of time since the error message was outputted. In response to determining that the period of time exceeds a threshold period of time, control circuitry704may end the loop and advance process1200to1212.

At1208, control circuitry704identifies the term in the first query that cannot not be parsed, preventing a response from being generated at1202. In the overarching example, this term is “flicks.” In response to identifying the term, control circuitry704determines whether the term has been replaced in the modified version of the first query. As discussed previously, the modified version of the first query includes the word “movies” instead of “flicks.” Control circuitry704determines that the term “movies” is a template term found in the stored response templates and the vocabulary database. In response to determining that the modified version of the first query comprises the template term (e.g., “movies”), at1210, control circuitry704determines that the input term in the first query (e.g., “flicks”) shares a definition with the template term (e.g., “movies”). If control circuitry704still does not identify a template term in the modified version of the first query at1208, process1200ends at1212.

FIG.13is a flowchart of illustrative process1300for providing personalized answers based on the number of times an input term is used in a plurality of queries, in accordance with some embodiments of the disclosure. In some cases, control circuitry704may determine that there are multiple candidates that can be replacement terms for a template term. For example, at times a user may use the term “flicks” when describing “movies” and at times the user may use the term “films.” In order to determine which input term to use when replacing a template term in a generated response or modified template, control circuitry704executes process1300.

At1302, control circuitry704receives a third query comprising a different input term (e.g., “films”) that shares a definition with the template term (e.g., “movies”). At1304, control circuitry704accesses a query history from memory (e.g., storage708of media device802or server806). The query history is a list of queries received by control circuitry704and may be linked to a user profile. At1306, control circuitry704retrieves a previously received query from the history (e.g., “play a flick on my television”). At1308and1312, control circuitry704determines whether the retrieved query includes the input term or the different input term, respectively. In response to determining that the retrieve query includes the input term (e.g., “flicks”), at1310, control circuitry704increments a first count (C1) of previously received queries that comprise the input term. At1316, control circuitry704determines whether there are additional previously received queries. If control circuitry704determines that there are additional previously received queries, process1300returns to1306where control circuitry704retrieves another previously receive query (e.g., “what film starred Tom Cruise as a character named Jack?”). In this iteration, control circuitry704determines, at1312, that the retrieved query includes the different input term (e.g., “film”) and increments a second count (C2) of previously received queries that comprise the different input term. The loop between1306and1316continues until all previously received queries have been analyzed by control circuitry704.

At1318, control circuitry704compares C1and C2in order to determine which input term to use. Suppose that after several loops between1306and1316, C1equals 15 and C2equals 10. In some embodiments, control circuitry704stores these values in the vocabulary database (specifically in the entries of the input term and the different input term, respectively). In response to determining that C1is greater than or equal to C2, at1320, control circuitry704creates a version of the template in which the template term (e.g., “movie”) is replaced by the input term (“flick”). In another example, suppose that C1equals 10 and C2equals 15. In response to determining that that C2is greater than C1, at1322, control circuitry704creates a version of the template in which the template term (e.g., “movie” is replaced by the different input term (e.g., “film”).

FIG.14is a flowchart of illustrative process1400for providing answers to a different user, in accordance with some embodiments of the disclosure. In some cases, different users may provide queries to the NLUI application. For example, in a household that has a smart speaker, multiple individuals may share the smart speaker. Process1400may thus may executed to distinguish between users. At1402, control circuitry704receives a third query. Suppose that the third query is “I'm bored, what should I do?”. As depicted inFIG.6B, when a first user provided input query608“I'm bored, what should I do?” control circuitry704outputted “Why not watch a flick such as ‘Halloween?”’ via media device602. At1404, control circuitry704determines whether the third query was received from a different user than the first user. For example, if the third query is an audio clip, control circuitry704may use voice recognition to determine whether the voice model associated with the first query is different from the voice model associated with the third query. Control circuitry704may alternatively determine whether the third query is linked with a particular user profile that is different from the user profile linked with the first query (e.g., received from a different login account, different device, etc.).

In response to determining that the third query was received from a different user, control circuitry704generates a response to the third query based on a response template. For example, control circuitry704may identify a response template in the response template database that answers the third query. Control circuitry704may retrieve template terms and replacement terms from the vocabulary database, based on user identifier508. The user identifier for the different user may be, for example, a particular name or an indicator of his/her voice pattern (e.g., a sine wave). Using the replacement terms, at1410control circuitry704may generate a response that uses the replacement terms. For example, the different user may use the input term “film” instead of “flicks.” Accordingly, control circuitry704generates the response “Why not watch a film such as ‘Halloween?’”. In some embodiments, control circuitry704may simply generate a response based on the original response template (e.g., “Why not watch a movie such as ‘Halloween?’”).

In response to determining that the third query was not received from the different user, at1406, control circuitry704generates a modified template in which the template term is replaced by the input term (e.g., “Why not watch a flick such as <movie name>?”). At1408, control circuitry704generates a response to the third query based on the modified template. At1412, control circuitry704outputs the response generated at either1410or1408depending on the determination made by control circuitry704at1404.

FIG.15is a flowchart of illustrative process1500for substituting template terms with input terms, in accordance with some embodiments of the disclosure. When generating a modified template at912(e.g. labelled as1502inFIG.15), control circuitry704may execute process1500. At1504, control circuitry704generates a copy of the selected response template. At1506, control circuitry704extracts a term from the copy of the response template. For example, the copy of selected response template may be read, “We found several movies with <information modifier> for you.” The extracted term may be “movies.” At1508, control circuitry704determines whether the extracted term (e.g., “movies”) matches the template term (e.g., “movies”). In response to determining that the respective terms match, at1512, control circuitry704substitutes the extracted term in the copy of the selected response template with the input term (e.g., “flicks”). In response to determining that the respective terms do not match, at1510, control circuitry704determines whether there are additional terms in the copy to analyze (e.g., “we,” “found,” etc.). If control circuitry704determines that there are additional terms to analyze, process1500returns to1506. At1514, control circuitry704stores in memory (e.g., storage708of media device802or server806) the copy of the selected response template as the modified template.

FIG.16is a flowchart of illustrative process1600for determining whether the definitions of the input term and template term match, in accordance with some embodiments of the disclosure. When determining whether two terms share a definition at910or1012, control circuitry704may execute process1600. At1602, control circuitry704retrieves (e.g., from the Internet via communication network804or storage708) a first definition of the input term (e.g., definition306of input term302). At1604, control circuitry704retrieves a second definition of the template term (e.g., definition306of template term304). At1606, control circuitry704compares the first definition with the second definition. At1608, control circuitry704determines whether the definitions match. For example, control circuitry704may determine the number of words in each definition that are shared and whether they are in the same positions in the respective definitions. This number may be compared with a threshold value indicating the minimum number of words that need to be the same and in the same position in the respective definitions to be considered the matching definitions. In response to determining that the definitions match, at1610, control circuitry704determines that the input term shares the definition with the template term. In response to determining that the definitions do not match, process1600ends at1612.

It should be noted that processes900-1600or any step thereof could be performed on, or provided by, any of the devices shown inFIGS.7-8. For example, the processes may be executed by control circuitry704(FIG.7) as instructed by a NLUI application implemented on media device802and/or server808. In addition, one or more steps of a process may be incorporated into or combined with one or more steps of any other process or embodiment (e.g., steps from process900may be combined with steps from process1100). In addition, the steps and descriptions described in relation toFIGS.9-16may be done in alternative orders or in parallel to further the purposes of this disclosure. For example, each of these steps may be performed in any order or in parallel or substantially simultaneously to reduce lag or increase the speed of the system or method.

The processes discussed above are intended to be illustrative and not limiting. One skilled in the art would appreciate that the steps of the processes discussed herein may be omitted, modified, combined, and/or rearranged, and any additional steps may be performed without departing from the scope of the invention. More generally, the above disclosure is meant to be exemplary and not limiting. Only the claims that follow are meant to set bounds as to what the present invention includes. In addition, the systems and methods described herein may be performed in real time. It should also be noted, the systems and/or methods described above may be applied to, or used in accordance with, other systems and/or methods.