Patent ID: 12217008

DETAILED DESCRIPTION

In the following description, for the purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the embodiments of this disclosure. It will be appreciated, however, by those having skill in the art, that the embodiments of the invention may be practiced without these specific details or with an equivalent arrangement. In other cases, well-known structures and devices are shown in block diagram form in order to avoid unnecessarily obscuring the embodiments of the invention.

FIG.1shows an illustrative user interface for presenting dynamic conversational responses using machine learning models, in accordance with one or more embodiments. For example,FIG.1shows user interfaces100,120, and130. The conversational system (e.g., via a mobile application) may generate and respond to user interactions in the user interfaces (e.g., user interfaces100,120, and130) in order to engage in a conversational interaction with the user. The conversational interaction may include a back-and-forth exchange of ideas and information between the system and the user. The conversational interaction may proceed through one or more mediums (e.g., text, video, audio, etc.).

The conversational system may generate prompt102upon activation inviting the user to type in a question or a request. Although prompt102is shown as a visual output (e.g., on a display screen), prompt102may be an audio output (e.g., via one or more speakers). In some embodiments, prompt102may be an audio-visual prompt. The conversational system may receive a user input (e.g., user input104) from a user, during a conversational interaction with a user interface. Although user input104is shown as a text input, user input104may be audio, visual, or audio-visual. For example, the user may speak his/her input into a microphone associated with a user device and that input may be communicated to the conversational system.

In some embodiments, the conversational system may process user input104to generate dynamic conversational responses sensitive to different emotional contexts using machine learning models. An exemplary conversational system is illustrated within ecosystem200ofFIG.2. Conversational system202may include communication subsystem212, processing subsystem214, machine learning subsystem216, and output subsystem218. Ecosystem200may also include data node204that may store training datasets and/or other data (e.g., in databases). Ecosystem200may also include user device206that enables user input as described below. Conversational system202, data node204, and user device206may be connected by network250(e.g., the Internet, intranet, or a combination of both).

The conversational system may receive a user input, from a user, during a conversational interaction with a user interface. The user input may be received via communication subsystem212from a device associated with the user. Communication subsystem212may include software components, hardware components, or a combination of both. For example, communication subsystem212may include a network card (e.g., a wireless network card and/or a wired network card) that is coupled with software to drive the card. When the user input is received, communication subsystem212may pass the received user input to processing subsystem214.

Processing subsystem214may include hardware components, software components, or a combination of both. For example, processing subsystem214may use processor(s), memory, and software to use those hardware components. Processing subsystem214may identify, in the user input, a term that matches a stored term with an associated social closeness value. The associated social closeness value may indicate a potential social distance between the user and an entity represented by the stored term. For example, a term such as “grandmother” or “mother” may have a higher social closeness value than the term “neighbor”.FIG.3illustrates an exemplary table300that may store terms and social closeness values. Column302stores the terms, while column304stores the corresponding social closeness values. Although the social closeness values are displayed as integer values, other data types to store social closeness values may be used (e.g., percentages, ratios, or other types of values).

Thus, in some embodiments, processing subsystem214may compare a plurality of terms within the user input with a plurality of stored terms. Each term of the plurality of stored terms may be associated with a corresponding social closeness value, and a particular social closeness value may indicate a social distance between the user and an entity represented by a corresponding stored term (as illustrated inFIG.3). For example, processing subsystem214may select each term within the input and compare those terms with stored terms (e.g., sequentially or in parallel) to determine whether any of the terms within the user input match a stored term. If processing subsystem214finds a match, processing subsystem214may retrieve the social closeness value associated with the term. As referred to herein, “term” refers to a word or phrase. Thus, in some embodiments, processing subsystem214may execute a function to split the user input into terms.

Data processing subsystem214may determine, using the user input, an emotional context associated with the user. Data processing subsystem214may make the determination using various operations. For example, data processing subsystem214may compare the terms within the user input with stored terms that have a stored emotional context. For example, the term “funeral” may be associated with “sadness” as an emotional context. The term “graduation” may be associated with “happiness” as an emotional context. Those terms may be pre-stored and data processing subsystem214may compare those pre-stored terms with terms within the user input.

In some embodiments, data processing subsystem214may use an additional machine learning model to determine an emotional context associated with the user input. For example, data processing subsystem214may input the user input into a second machine learning model that is trained to identify context from the user input. The additional machine learning model may be trained to identify the context using a training dataset. For example, a training dataset may be used that includes statements (e.g., various user inputs) and labels (e.g., emotional contexts) associated with those statements to train the model. In some embodiments, the additional (second) machine learning model may be trained using a training dataset that includes “terms” and associated labels for those terms. For example, the term “wedding” may be labeled “happy” as a corresponding context, while the term “funeral” may have an associated label of “sad.” Other terms may be included in the training dataset with the associated labels (e.g., “wedding reception”, “graduation”, “graduation ceremony”, etc.).

In some embodiments, the additional (second) machine learning model may be trained to identify an event associated with the user input. Thus, data processing subsystem214may input the user input into a second machine learning model, where the second machine learning model is trained to identify events from user input. Prior to input, data processing subsystem214may split the user input into terms and generate a vector for input into the machine learning model using those terms. The machine learning model may be trained using a training dataset to identify the events. For example, the training dataset may include a plurality of terms (e.g., “wedding”, “wedding reception”, “graduation”, “graduation ceremony”, “funeral”, “wake”, etc.) and corresponding labels (e.g., “graduation”, “funeral”, “wedding”, etc.). The additional (second) machine learning model may be trained using that training dataset.

Data processing subsystem214may receive, from the second machine learning model, an identifier of an event associated with the user input. For example, if the user input states, “I need to buy my sister a graduation present,” the machine learning model trained to identify events may identify a graduation as the event associated with the user input. Data processing subsystem214may determine the emotional context associated with the event.

For example, data processing subsystem214may compare an identifier associated with the event (e.g., event name) with a plurality of pre-stored events. Each pre-stored event may have an associated emotional context. Thus, when data processing subsystem214identifies a matching event, data processing subsystem214may retrieve an emotional context value (e.g., happiness, sadness, etc.) associated with the event.

In some instances, identifying an event using a machine learning model may be more advantageous than performing term matching on the user input. For example, the user may state “I need to buy a gift for a diploma ceremony.” In this instance, it may be possible that the event “diploma ceremony” may not be part of the listed events. However, a machine learning model may still recognize that a “diploma ceremony” may be equivalent to “graduation.”

Processing subsystem214may generate a feature input based on the term, the emotional context, and the associated social closeness value. For example, processing subsystem214may generate a vectorized representation of the user input, the emotional context and the associated closeness value. The vectorization operation may be performed by another machine learning model. Processing subsystem214may pass the feature input to machine learning subsystem216.

Machine learning model subsystem216may input the feature input into a first machine learning model to obtain a dynamic conversational response from a plurality of dynamic conversational responses based on the feature input. For example, the machine learning model may output a plurality of possible interpretations of the user's query and a corresponding probability for each interpretation. In addition, the machine learning model may output an indication of additional response language to be used in responding to the user. For example, the machine learning model may output an indication that there is a 90% probability that the user is asking for an account balance and may also output an indication that a particular language template (e.g., including language that is sensitive to the particular determined emotional context) should be used to deliver the results to the user. For example, the system may include a plurality of language templates for a plurality of emotional contexts and a plurality of socially close entities. The machine learning model may output a template identifier suitable for the combination of the socially close entity and the emotional context provided. The machine learning model may be trained using the template identifiers for a socially close entity and the emotional contexts.

In some embodiments, machine learning subsystem216may receive, from the first machine learning model, a plurality of probable contexts and a plurality of corresponding probabilities. Machine learning subsystem216may receive a context of “sadness” with a probability of 75% and a context of “anger” with a probability of 50%. Machine learning subsystem216may select, from the plurality of probable contexts, one or more contexts with highest probabilities. In some embodiments, machine learning subsystem216may select one context with the highest probability (e.g., sadness with a 75% probability), while in other embodiments, machine learning subsystem216may select multiple contexts with the highest probabilities (e.g., when probabilities are within a specific threshold value of each other). For example, if the “sadness” context and the “anger” context are within a particular percentage (e.g., within 5%), machine learning subsystem216may select both of those contexts.

Machine learning subsystem216may then select conversational data based on the one or more contexts. For example, machine learning subsystem216may select a template for the one or more contexts selected. Machine learning subsystem216may then update the dynamic conversational response with the conversational data. For example, if machine learning subsystem216determines that the context is “sadness” and it is determined that the user is asking about his/her account balance, machine learning subsystem216may select a template associated with the “sadness” context. However, if machine learning subsystem216determines that the context is “sadness” and “anger” and it is determined that the user is asking about his/her account balance, machine learning subsystem216may select a template associated with the “sadness” and “anger” contexts.

Machine learning subsystem216may then update the dynamic conversational response with the conversational data. Machine learning subsystem216may make the update using the following operations. Machine learning subsystem216may compare each response term with a plurality of contextual terms. In response to determining that a response term should be replaced with a contextual term of the plurality of contextual terms, machine learning subsystem216may replace the response term with the contextual term. For example, if the conversation response includes a phrase “Have a nice day,” machine learning subsystem216may replace that phrase with “My condolences” in the case where a user's socially close person has died.

Machine learning subsystem216may pass the conversational response to output subsystem218. Output subsystem218may receive the conversational response and generate and/or display, at the user interface, the dynamic conversational response. For example, the output subsystem may determine a type of user interface that is being used by the user (e.g., graphical, audio, or a combination of the two) and adopt the conversational response based on the user interface. Thus, in response to determining that the user interface is graphical, output subsystem218may generate for display the conversational response as text. In response to determining that the user interface is voice based, output subsystem218may adapt the conversational response into a voice output and output the conversational response using a speaker. For example, output subsystem218may generate for display output106(FIG.1).

FIG.4is an illustrative system for generating dynamic conversational responses using a machine learning model, in accordance with one or more embodiments. For example, system400may represent the components used for generating dynamic conversational responses as shown inFIG.1. As shown inFIG.4, system400may include mobile device422and user terminal424. While shown as a smartphone and personal computer, respectively, inFIG.4, it should be noted that mobile device422and user terminal424may be any computing device, including, but not limited to, a laptop computer, a tablet computer, a hand-held computer, other computer equipment (e.g., a server), including “smart,” wireless, wearable, and/or mobile devices.FIG.4also includes cloud components410. Cloud components410may alternatively be any computing device as described above and may include any type of mobile terminal, fixed terminal, or other device. For example, cloud components410may be implemented as a cloud computing system and may feature one or more component devices. It should also be noted that system400is not limited to three devices. Users may, for instance, utilize one or more devices to interact with one another, one or more servers, or other components of system400. It should be noted, that, while one or more operations are described herein as being performed by particular components of system400, those operations may, in some embodiments, be performed by other components of system400. As an example, while one or more operations are described herein as being performed by components of mobile device422, those operations, may, in some embodiments, be performed by components of cloud components410. In some embodiments, the various computers and systems described herein may include one or more computing devices that are programmed to perform the described functions. Additionally, or alternatively, multiple users may interact with system400and/or one or more components of system400. For example, in one embodiment, a first user and a second user may interact with system400using two different components.

With respect to the components of mobile device422, user terminal424, and cloud components410, each of these devices may receive content and data via input/output (hereinafter “I/O”) paths. Each of these devices may also include processors and/or control circuitry to send and receive commands, requests, and other suitable data using the I/O paths. The control circuitry may comprise any suitable processing, storage, and/or input/output circuitry. Each of these devices may also include a user input interface and/or user output interface (e.g., a display) for use in receiving and displaying data. For example, as shown inFIG.4, both mobile device422and user terminal424include a display upon which to display data (e.g., based on recommended contact strategies).

Additionally, as mobile device422and user terminal424are shown as touchscreen smartphones, these displays also act as user input interfaces. It should be noted that in some embodiments, the devices may have neither user input interface nor displays and may instead receive and display content using another device (e.g., a dedicated display device such as a computer screen and/or a dedicated input device such as a remote control, mouse, voice input, etc.). Additionally, the devices in system400may run an application (or another suitable program). The application may cause the processors and/or control circuitry to perform operations related to generating dynamic conversational responses using machine learning models.

Each of these devices may also include electronic storages. The electronic storages may include non-transitory storage media that electronically stores information. The electronic storage media of the electronic storages may include one or both of (i) system storage that is provided integrally (e.g., substantially non-removable) with servers or client devices, or (ii) removable storage that is removably connectable to the servers or client devices via, for example, a port (e.g., a USB port, a firewire port, etc.) or a drive (e.g., a disk drive, etc.). The electronic storages may include one or more of optically readable storage media (e.g., optical disks, etc.), magnetically readable storage media (e.g., magnetic tape, magnetic hard drive, floppy drive, etc.), electrical charge-based storage media (e.g., EEPROM, RAM, etc.), solid-state storage media (e.g., flash drive, etc.), and/or other electronically readable storage media. The electronic storages may include one or more virtual storage resources (e.g., cloud storage, a virtual private network, and/or other virtual storage resources). The electronic storages may store software algorithms, information determined by the processors, information obtained from servers, information obtained from client devices, or other information that enables the functionality as described herein.

FIG.4also includes communication paths428,430, and432. Communication paths428,430, and432may include the Internet, a mobile phone network, a mobile voice or data network (e.g., a 4G or LTE network), a cable network, a public switched telephone network, or other types of communications networks or combinations of communications networks. Communication paths428,430, and432may separately or together include one or more communications paths, such as a satellite path, a fiber-optic path, a cable path, a path that supports Internet communications (e.g., IPTV), free-space connections (e.g., for broadcast or other wireless signals), or any other suitable wired or wireless communications path or combination of such paths. The computing devices may include additional communication paths linking a plurality of hardware, software, and/or firmware components operating together. For example, the computing devices may be implemented by a cloud of computing platforms operating together as the computing devices.

Cloud components410may be a database configured to store user data for a user. For example, the database may include user data that the system has collected about the user through prior transactions. Alternatively, or additionally, the system may act as a clearinghouse for multiple sources of information about the user. Cloud components410may also include control circuitry configured to perform the various operations needed to generate responses. For example, the cloud components410may include cloud-based storage circuitry configured to store a first machine learning model that is trained to select a dynamic conversational response from a plurality of dynamic conversational responses based on first feature input. Cloud components410may also include cloud-based control circuitry configured to determine an intent of the user based on a machine learning model. Cloud components410may also include cloud-based input/output circuitry configured to generate the dynamic conversational response during a conversational interaction.

Cloud components410include machine learning model402. Machine learning model402may take inputs404and provide outputs406. The inputs may include multiple datasets, such as a training dataset and a test dataset. Each of the plurality of datasets (e.g., inputs404) may include data subsets related to user data, contact strategies, and results. In some embodiments, outputs406may be fed back to machine learning model402as input to train machine learning model402(e.g., alone or in conjunction with user indications of the accuracy of outputs406, labels associated with the inputs, or with other reference feedback information). For example, the system may receive a first labeled feature input, wherein the first labeled feature input is labeled with a known dynamic conversational response for the first labeled feature input. The system may then train the first machine learning model to classify the first labeled feature input with the known dynamic conversational response.

In another embodiment, machine learning model402may update its configurations (e.g., weights, biases, or other parameters) based on the assessment of its prediction (e.g., outputs406) and reference feedback information (e.g., user indication of accuracy, reference labels, or other information). In another embodiment, where machine learning model402is a neural network, connection weights may be adjusted to reconcile differences between the neural network's prediction and the reference feedback. In a further use case, one or more neurons (or nodes) of the neural network may require that their respective errors are sent backward through the neural network to facilitate the update process (e.g., backpropagation of error). Updates to the connection weights may, for example, be reflective of the magnitude of error propagated backward after a forward pass has been completed. In this way, for example, the machine learning model402may be trained to generate better predictions.

In some embodiments, machine learning model402may include an artificial neural network (e.g., as described inFIG.4below). In such embodiments, machine learning model402may include an input layer and one or more hidden layers. Each neural unit of machine learning model402may be connected with many other neural units of machine learning model402. Such connections can be enforcing or inhibitory in their effect on the activation state of connected neural units. In some embodiments, each individual neural unit may have a summation function that combines the values of all of its inputs together. In some embodiments, each connection (or the neural unit itself) may have a threshold function such that the signal must surpass before it propagates to other neural units. Machine learning model402may be self-learning and trained, rather than explicitly programmed, and can perform significantly better in certain areas of problem solving, as compared to traditional computer programs. During training, an output layer of machine learning model402may correspond to a classification of machine learning model402and an input known to correspond to that classification may be input into an input layer of machine learning model402during training. During testing, an input without a known classification may be input into the input layer, and a determined classification may be output.

In some embodiments, machine learning model402may include multiple layers (e.g., where a signal path traverses from front layers to back layers). In some embodiments, back propagation techniques may be utilized by machine learning model402where forward stimulation is used to reset weights on the “front” neural units. In some embodiments, stimulation and inhibition for machine learning model402may be more free-flowing, with connections interacting in a more chaotic and complex fashion. During testing, an output layer of machine learning model402may indicate whether or not a given input corresponds to a classification of machine learning model402.

In some embodiments, model402may predict a goal or intent of a user. This goal or intent may be selected from a plurality of goals and/or intents stored by the system. For example, the system may determine that users who ask different questions about payment have similar account information and digital activities. In some embodiments, the model (e.g., model402) may automatically perform actions based on output406.

In some embodiments, machine learning model402may be a machine learning model that determines a socially close entity. In some embodiments, a second machine learning model402may be used to determine a context associated with the user input. Input404may be a vector generated based on the user input (e.g., including a social closeness value, one or more terms, and/or the event), and output406may be an output that indicates the context of the user input. In some embodiments, input404may be input into the additional (second) machine learning model that may determine an event and/or context.

FIG.5shows a flowchart of the steps involved in generating dynamic conversational responses using a machine learning model, in accordance with one or more embodiments. For example, process500may represent operations by one or more components as shown inFIG.2when generating dynamic conversational responses using one or more machine learning models (e.g., as shown inFIG.4). At502, process500(e.g., using one or more components in system400(FIG.4)) receives a first user input during a conversational interaction. For example, the system may receive, using control circuitry, a user action, from a user, during a conversational interaction with a user interface. For example, the system may receive a user action as described inFIGS.1-2.

At504, process500(e.g., using one or more components in system400(FIG.4)) identifies a term that matches a stored term with an associated social closeness value. For example, the system may perform the identification operation, using the control circuitry. At step506, process500(e.g., using one or more components in system400(FIG.4)) determines an emotional context associated with the user. For example, process500may use a machine learning model ofFIG.4or another suitable component. At508, process500(e.g., using one or more components in system400(FIG.4)) generates a feature input based on the term, the emotional context, and the associated social closeness value. For example, the system may generate, using the control circuitry, the feature input.

At510, process500(e.g., using one or more components in system400(FIG.4)) inputs the feature input into a machine learning model. For example, the system may input, using the control circuitry, the feature input into a machine learning model (e.g., machine learning model ofFIG.4). The machine learning model may be trained to select a dynamic conversational response from a plurality of dynamic conversational responses based on the feature input. At512, process500(e.g., using one or more components in system400(FIG.4)) generates a dynamic conversational response based on a response selected by the machine learning model. For example, the system may generate and/or display, at the user interface, the dynamic conversational response based on a response selected by a machine learning model during the conversational interaction.

It is contemplated that the steps or descriptions ofFIG.5may be used with any other embodiment of this disclosure. In addition, the operations and descriptions described in relation toFIG.5may be done in alternative orders or in parallel to further the purposes of this disclosure. For example, each of these steps may be performed in any order, in parallel, or simultaneously to reduce lag or increase the speed of the system or method. Furthermore, it should be noted that any of the devices or equipment discussed in relation toFIGS.1-4could be used to perform one or more of the steps inFIG.5.

The above-described embodiments of the present disclosure are presented for purposes of illustration and not of limitation, and the present disclosure is limited only by the claims which follow. Furthermore, it should be noted that the features and limitations described in any one embodiment may be applied to any other embodiment herein, and flowcharts or examples relating to one embodiment may be combined with any other embodiment in a suitable manner, done in different orders, or done in parallel. In addition, the systems and methods described herein may be performed in real time. It should also be noted that the systems and/or methods described above may be applied to, or used in accordance with, other systems and/or methods.

The present techniques will be better understood with reference to the following enumerated embodiments:

1. A method for generating dynamic conversational responses sensitive to different emotional contexts using machine learning models, the method comprising: receiving a first user input from a user during a conversational interaction with a user interface; identifying, in the first user input, a term that matches a stored term with an associated social closeness value, wherein the associated social closeness value indicates a social distance between the user and an entity represented by the stored term; determining, using the first user input, an emotional context associated with the user; generating a feature input based on the first user input, the term, the emotional context, and the associated social closeness value; inputting the feature input into a first machine learning model to obtain a dynamic conversational response from a plurality of dynamic conversational responses based on the feature input; and generating, at the user interface, the dynamic conversational response based on a response selected by the first machine learning model during the conversational interaction.

2. Any of the preceding embodiments, wherein determining the emotional context associated with the user comprises: inputting the first user input into a second machine learning model, wherein the second machine learning model is trained to identify events from user input; receiving, from the second machine learning model, an identifier of an event associated with the first user input; and determining the emotional context associated with the event.

3. Any of the preceding embodiments, wherein generating the feature input based on the first user input, the term, the emotional context, and the associated social closeness value further comprises generating the feature input based on the event.

4. Any of the preceding embodiments, wherein determining the emotional context associated with the event comprises: comparing the event with a list of stored events; identifying, based on comparing the event with the list of stored events, a matching event; and retrieving the emotional context associated with the matching event.

5. Any of the preceding embodiments, further comprising: inputting the first user input into a second machine learning model, wherein the second machine learning model is trained to identify a context from user input; and receiving, from the second machine learning model, an identifier of the context associated with the first user input.

6. Any of the preceding embodiments, wherein inputting the feature input into the first machine learning model to obtain the dynamic conversational response from the plurality of dynamic conversational responses comprises: receiving, from the first machine learning model, a plurality of probable contexts and a plurality of corresponding probabilities; selecting, from the plurality of probable contexts, one or more contexts with highest probabilities; selecting conversational data based on the one or more contexts; and updating the dynamic conversational response with the conversational data.

7. Any of the preceding embodiments, wherein updating the dynamic conversational response with the conversational data comprises: comparing each response term with a plurality of contextual terms; and in response to determining that a response term matches a contextual term of the plurality of contextual terms, replacing the response term with the contextual term.

8. Any of the preceding embodiments, wherein identifying, in the first user input, the term that matches the stored term with the associated social closeness value comprises comparing a plurality of terms within the first user input with a plurality of stored terms, wherein each of the plurality of stored terms is associated with a corresponding social closeness value.

9. A tangible, non-transitory, machine-readable medium storing instructions that, when executed by a data processing apparatus, cause the data processing apparatus to perform operations comprising those of any of embodiments 1-8.

10. A system comprising: one or more processors; and memory storing instructions that, when executed by the processors, cause the processors to effectuate operations comprising those of any of embodiments 1-8.

11. A system comprising means for performing any of embodiments 1-8.