Dialogue management system with hierarchical classification and progression

A dialogue management system applies hierarchical classifiers and other natural language processing to dialogue input, and determines whether performance of an action is likely to occur. The dialogue management system may process dialogue input to assess a dialogue participant's current position in various hierarchies or other classification schemes associated with performance of a desired action. The system may then present results of the assessment to another dialogue participant or provide the results to another system. In some embodiments, the dialogue management system may automatically generate responses or questions designed to engage a dialogue participant and cause the participant to progress through the levels of a hierarchy toward performance of a desired action.

BACKGROUND

Natural language processing systems include various components for receiving input from a user and processing the input to determine what the user means. In some implementations, a natural language processing system receives textual input, such as text entered by a user or a transcription of a user's utterance. The natural language processing system can determine the meaning of the textual input in a way that can be acted upon by a computer application. For example, a user of a mobile phone may speak a command to initiate a phone call. Audio of the spoken command can be transcribed by an automatic speech recognition module. The natural language processing system can then determine the user's intent from the transcription (e.g., that the user wants to initiate the phone call feature) and generate a command to initiate the phone call.

DETAILED DESCRIPTION

Computing systems may use natural language processing (“NLP”) subsystems to process natural language input (e.g., text-based input typed by a user, speech input that has been processed into a textual transcription, etc.) and extract information from the input. In some implementations, NLP subsystems may be used to determine information about a source of the input and/or to determine information that the system is to output. From the perspective of a user interacting with an NLP-based system, the system's performance may be defined in terms of the degree to which output from the system and actions taken by the system comport with the input provided to the system. From the perspective of a developer or operator of a natural language processing system, the system's performance may be defined in terms of the number or proportion of transactions performed automatically and successfully.

Some conventional dialogue management systems use NLP to determine which command a dialogue participant is requesting the system to execute. Other conventional dialogue management systems attempt to generate a profile of a dialogue participant. In some cases, a dialogue management system attempts to obtain as much information as possible during a multi-turn dialogue by making the dialogue last as long as possible. However, these and other such systems fail to assess the current state of the dialogue or dialogue participant, and then generate output specifically tailored to change the state of the dialogue or dialogue participant towards a desired action. Instead, such conventional systems merely aim to understand the action that a user wishes to initiate, or they aim to collect information about a user.

Aspects of the present disclosure are directed to a dialogue management system that applies hierarchical classifiers and other natural language processing methods to dialogue input, and determines whether performance of an action is likely to occur. The dialogue management system may process dialogue input to assess a current position in various hierarchies or other classification schemes associated with performance of a desired action. The system may then automatically generate output (e.g., responses or questions) based on the assessment. A system that implements some or all of these features can intelligently engage a dialogue participant such that over the course of a multi-turn dialogue the participant progresses through the levels of a hierarchy toward performance of a desired action. Moreover, a system that implements these features can perform functions not previously performable by automated systems, including generating output specifically formulated to provoke input that progresses the dialogue through the levels of a hierarchy and toward performance of a desired action. By implementing these features, a computing system can go beyond mere dialogue automation, and may provide intelligent steering of a dialogue and analysis thereof to provide insights and provoke responses not previously possible in a single, real-time dialogue of any kind.

Some aspects of the present disclosure relate to using NLP and classifiers to assess a likelihood associated with a dialogue (e.g., a likelihood that a dialogue participant will perform a desired action). A dialogue participant—also referred to as a participant, user, client, or customer—may access a dialogue-based interface to obtain information about a possible action, such as scheduling an appointment or making a purchase. The participant may engage in a multi-turn dialogue in which the participant asks questions, answers questions, makes selections, and/or provides other input. The input can be processed using one or more classifiers to determine a current hierarchal level for the participant. For example, a particular hierarchy may consist of levels through which a participant is expected to progress before performing an action. The levels may not be “checkpoints” through which a user progresses in a strictly linear fashion. Rather, a participant may start off at a lowest level (farthest away from performing an action) and progress to a higher level over the course of time (closer to performing an action). The participant may then continue progressing to higher levels, remain at a particular level, or regress to a lower level through which the participant had previously progressed. The progression/regression may occur indefinitely, and there may be no requirement that the participant reaches or remains at any particular level.

Additional aspects of the present disclosure relate to intelligently generating responses or other dialogue turns such that subsequent input from the dialogue participant will result in a change to the hierarchy level classification described above. The dialogue management system can process input from the dialogue participant over the course of multiple dialogue turns and determine various classifications from the processed input. In some embodiments, the dialogue management system can select or generate a dialogue response based on some combination of classification results. For example, the dialogue management system may have a collection of dialogue responses or response templates. Individual responses or response templates may be associated with a set of classifications, and may be designed to elicit input from the dialogue participant that causes a progression in the hierarchy level classification that the system determines for the participant. The elicited input may directly affect the level classification (e.g., the level classification may progress to the next higher level making it more likely that the participant will perform a desired action). In some cases the elicited input may be used to select or generate a subsequent response or prompt that causes a change to the level classification. In addition, the real-time or substantially real-time manner in which the system intelligently generates responses ensures that the current state of the dialogue and/or dialogue participant, as assessed using the classifiers described herein, has not changed due to the mere passage of time between input and subsequent output. By ensuring that the dialogue remains active in substantially real-time (e.g., a response is generated by the system within about 30 seconds, or within about 15 seconds, or within about 5 seconds, or within about 1 second of receiving an input), the assessment remains valid through the generation of output and receipt of subsequent input, if any.

Although aspects of some embodiments described in the disclosure will focus, for the purpose of illustration, on particular examples of classifiers, dialogues, and desired actions, the examples are illustrative only and are not intended to be limiting. In some embodiments, the techniques described herein may be applied to additional or alternative classifiers, dialogue, or actions. Various aspects of the disclosure will now be described with regard to certain examples and embodiments, which are intended to illustrate but not limit the disclosure.

Network-Based Dialogue Management Environment

With reference to an illustrative embodiment,FIG. 1shows a network environment in which aspects of the present disclosure may be implemented. As shown, the network environment may include a dialogue management system100, various user devices102a,102b, and102c(collectively “user devices102”), and various data consumers104aand104b(collectively “data consumers104”). The user devices102and dialogue management system100may communicate with each other via one or more communication networks150. The data consumers104and dialogue management system100may also communicate with each other via one or more communication networks160. A communication network150or160may be a publicly accessible network of linked networks, possibly operated by various distinct parties, such as the Internet. In other embodiments, a network150or160may include a private network, personal area network, local area network, wide area network, cable network, satellite network, cellular telephone network, etc. or a combination thereof, some or all of which may or may not have access to and/or from the Internet. Although two separate communication networks150and160are shown, in some embodiments the user devices102and data consumers104(or some subset thereof) may each communicate with the dialogue management system100via the same communication network.

The dialogue management system100may include various components for providing the features described herein. Illustratively, the dialogue management system100may include a chat server110to obtain input from user devices102, provide such input (or data derived therefrom) to other components of the dialogue management system100, and provide output to user devices102. The dialogue management system100may also include a classification-based processor112that receives dialogue input via the chat server110, performs classification and other natural language processing, and generates responses for presentation via the chat server110. The dialogue management system100may also include a data manager114that maintains data regarding dialogues and users, and manages the provision of such data to data consumers104. The example components and data stores of the dialogue management system100shown inFIG. 1are illustrative only, and are not intended to be limiting. In some embodiments, a dialogue management system100may have fewer, additional, and/or alternative components and data stores.

The dialogue management system100may be implemented on one or more physical server computing devices that provide computing services and resources to user devices102and data consumers104. In some embodiments, the dialogue management system100(or individual components thereof, such as the chat server110, classification-based processor112, etc.) may be implemented on one or more host devices, such as blade servers, midrange computing devices, mainframe computers, desktop computers, or any other computing device configured to provide computing services and resources. For example, a single host device may execute one or more chat servers110, one or more classification-based processor112, one or more data managers114, some combination thereof, etc. The dialogue management system100may include any number of such hosts.

In some embodiments, the features and services provided by the dialogue management system100may be implemented as web services consumable via communication networks150and160. In further embodiments, the dialogue management system100(or individual components thereof) is provided by one more virtual machines implemented in a hosted computing environment. The hosted computing environment may include one or more rapidly provisioned and released computing resources, such as computing devices, networking devices, and/or storage devices. A hosted computing environment may also be referred to as a “cloud” computing environment.

The individual user devices102may be any of a wide variety of computing devices, including personal computing devices, terminal computing devices, laptop computing devices, tablet computing devices, electronic reader devices, wearable computing devices, mobile devices (e.g., smart phones, media players, handheld gaming devices, etc.), and various other electronic devices and appliances. A dialogue participant may operate a user device102to access and interact with the dialogue management system100. In some embodiments, a dialogue participant may launch specialized application software, such as a mobile application106executing on a particular user device, such as a smart phone102aor tablet computer102b. The application106may be specifically designed to interface with the dialogue management system100(or chat servers in general) for conducting multi-turn dialogs or “chats.” In some embodiments, a user may use other application software, such as a browser application108, to interact with the dialogue management system100.

The individual data consumers104may be any of a wide variety of computing systems, including the computing systems described above with respect to the dialogue management system and/or the user devices102. The data consumers104may be operated by or otherwise associated with entities that use data obtained or generated by the dialogue management system100. For example, a data consumer104amay be operated by an automobile dealership to obtain data from the dialogue management system100regarding sales leads, committed test drive appointments, likely sales, and the like. As another example, a data consumer104bmay be operated by a therapist to obtain data regarding potential clients, committed appointments, and the like. In some embodiments there may be no separation or physical distinction between a dialogue management system100and data consumer104. Instead, an entity (e.g., car dealership, therapist, etc.) may operate its own dialogue management system100and consume data generated or obtained by the dialogue management system100. In some embodiments, a data consumer may provide information or functionality to the dialogue management system100for use in managing dialogues. For example, a car dealership can provide information regarding the cars currently available for sale.

When a user accesses an application106on a user device102ato retrieve content, the user device102amay establish a connection with the dialogue management system100via the network150. The user device102amay receive an initial prompt from a chat server110of the dialogue management system100via the connection. The prompt may be a general greeting, a specific question, a request for login credentials, etc. Illustratively, the prompt may be displayed in text form or presented audibly using text-to-speech (“TTS”) processing. A user of the user device102acan respond to the prompt (or otherwise interact with the dialogue management system100) by entering text or uttering a response that is converted to text using automatic speech recognition (“ASR”) processing. The user's response can be transmitted to the dialogue management system100for processing by the chat server110.

The chat server110can provide the user's input (or data derived therefrom, such as an ASR transcript of a spoken utterance) to the classification-based processor112. The classification-based processor112may use one or more classifiers to determine one or more classifications of the current state of the dialogue with the user. A response to the user's input may then be selected to further the dialogue (e.g., to respond to a specific question of the user, to obtain additional required or desired information from the user, to elicit a user response that will alter the classification or other aspects of the dialogue state, etc.). The response may then be provided to the chat server110, which can handle transmission of the response to the appropriate user device102ain the appropriate manner. The interactions may continue each time an input is received from the same user device102aand/or from other user devices102bor102c.

In some embodiments, responses to user input may be provided by another person rather than generated via an automatic process. For example, a system administrator or representative associated with a data consumer104may engage in a real-time (or substantially real-time) conversation with a user. In such cases, the dialogue management system100may serve as an intermediary. A user (e.g., a customer, sales lead, patient, potential patient, etc.) may provide input as discussed above and in greater detail below. The input can then be processed by the classification-based processor112to determine classifications as discussed above and in greater detail below. However, rather than automatically generating a response, the dialogue management system100can provide the user's input and the determined classifications to another person. A display interface, such as the graphical user interface (“GUI”) shown inFIG. 12, may be presented to the other person. The other person can then read the conversation thus far, see the classifications determined by the classification-based processor112, and provide a response to the latest user input (or provide ad-hoc output to the other user). The chat server110can receive the response and transmit the response to the appropriate user device102.

Example Classification-Based Processor

FIG. 2shows an example of a classification-based processor112that may be implemented in a dialogue management system100. As shown, the classification-based processor112may include any number of classifiers202for processing input text200(and other data, such as contextual data210) to determine various classifications for a particular user or dialogue. The classification-based processor112may also include a context engine204for maintaining the context of a dialogue over multiple turns, such as prior classification results, changes to classification results over the course of the dialogue, a transcript or other representation of the dialogue communications, etc. The classification-based processor112may also include a response engine that generates a response208(or some other output, such as a prompt). The classification-based processor112may also include various data stores, such as a context data store210to store contextual data used by other components of the classification-based processor112, and a templates data store212to store templates for responses to be provided to user devices102. The example components and data stores shown inFIG. 2are illustrative only, and are not intended to be limiting. In some embodiments, a classification-based processor112may have fewer, additional, and/or alternative components and data stores.

Generally described, classifiers202assign inputs to one or more predetermined classes based on various models, such as statistical models, neural-network-based models, criteria-based and/or rule-based models, etc. The classifiers analyze the input for signals of certain characteristics or states, and produce output representative of the identified characteristics or states. In the classification-based processor112shown inFIG. 2, multiple classifiers202assign textual input received from a user (or derived from user input) to various classes. Some classifiers may also use other input, such as contextual data regarding the user and/or the current multi-turn dialogue, in performing classification.

FIG. 3shows example classifiers that may be implemented in a classification-based processor112. The example classifiers202shown inFIG. 3are illustrative only, and are not intended to be limiting. In some embodiments, a classification-based processor112may implement additional, fewer, and/or alternative classifiers. Individual classifiers may use statistical models, neural-network-based models, criteria-based and/or rule-based models, other types of models, or any combination thereof to facilitate classification of input into one or more classes. In addition, individual classifiers202may be implemented in hardware, or in a combination of software and hardware. Each of the example classifiers shown inFIG. 3is discussed below.

An intent classifier310can classify input as one of a set of possible “intents” that the classification-based processor112is configured to recognize. An “intent” corresponds to the meaning of an input or the motivation behind an input, such as an action that the provider of the input wishes to perform or have performed. For example, a user of a user device102may access a dialogue management system100and provide textual input200. The textual input200may in some way reflect the user's intent to set up an appointment, obtain information about a product, initiate a transaction, or the like. The intent classifier310can use a natural language understanding (“NLU”) model to classify dialogue text200as representative of a “schedule appointment” intent, a “browse products” intent, a “purchase product” intent, or some other intent. Such a classification can be used to control subsequent operations of the dialogue management system100. For example, the output provided to the user device102in response to the input200can vary depending upon the intent, other classifiers that are used to process the text200can vary depending upon the intent, etc. In some embodiments, the intent classifier310(or some other classifier or component of the classification-based processor112) may classify users into one of a set of classes based on a current phase or stage associated with the determined intent of the user. For example, in the context of selling products or services, the intent classifier310may classify a user as currently being in the “lead,” “appointment,” “contract,” or “closing” phase.

An action hierarchy classifier312can classify input on a hierarchy of levels through which a user is expected to progress before performing a desired action. The levels of the hierarchy may include a first or lowest level that corresponds to an initial state of “desire” by the user, such as the desire to perform some action. The levels of the hierarchy may include a last or highest level that corresponds to a final state of “commitment” by the user, such as a commitment to perform the desired action. In some embodiments, the hierarchy may also include any number of intermediate levels between initial desire and final commitment, such as: a second level (e.g., above the initial “desire” level) that corresponds a recognition by the user of an “ability” to perform the desired action; a third level (e.g., above the second “ability” level) that corresponds to a state of “reason” or logical motivation going beyond mere desire; and a fourth level (e.g., above the third “reason” level and before the final “commitment” level) that corresponds to a state of “need” to perform the action, beyond merely having a logical reason to perform the desired action. In some embodiments, the levels of the action hierarchy are not checkpoints to pass in a strictly linear manner. Rather, a user's input may result in classification in a lower level after previously being classified in a higher level. In addition, a user's input may result in an initial classification at any level, without necessarily beginning at the lowest “desire” level. For example, a user's input may be indicative of proper classification in the “reason” level the first time the action hierarchy classifier312is run on input from the user. Subsequent input from the user may be indicative of a “need” level classification, showing that the user has regressed backwards and away from performing the desired action, or that the user had not actually reached the state of reason yet.

The action hierarchy classifier312can use a textual analysis of user input to determine a class according to the action hierarchy. For example, the action hierarchy classifier312may analyze the frequency with which individual words or word groups appear in input text200. Words or word groups may be associated with individual levels of the action hierarchy. When a user's input includes words associated with a particular hierarchy level at a higher frequency than words or word groups associated with other levels, then the user (or the user's input) may be determined to be in the particular hierarchy level. An example of an action hierarchy classifier312implemented using a word frequency model is described in greater detail below with respect toFIGS. 4 and 5.

A dialogue vector classifier314can classify input as one of a set of possible “vectors” that the dialogue vector classifier314is configured to recognize. A “vector” corresponds to a direction in which a dialogue is moving, or in which an input is moving the dialogue. For example, a particular multi-turn dialogue may relate to scheduling an appointment, such as a test drive of a car. A user may use words such as “but,” “yet,” or “instead,” which tend to redirect the dialogue (e.g., if the dialogue is progressing towards finalizing a test drive, the use of “but” by the user making the appointment may indicate that the user is now withdrawing from finalizing the test drive). The dialogue vector classifier314can classify such input as a “redirect” dialogue vector. This classification can be useful in generating a response. For example, the dialogue management system100may respond with vector words to redirect the dialogue back towards a desired outcome (e.g., scheduling the test drive). In some embodiments, the dialogue vector classifier314can use a textual analysis of user input to determine a dialogue vector classification, such as the frequency-based classifier described below with respect to the action hierarchy classifier312.

A perception classifier316can classify input as indicative of mode of perception preferred by a user. A particular user may be more visually oriented not only in the way they most effectively perceive external events, but also in the way that they process events internally (e.g., the user may react most strongly to visual stimuli, and may also think in terms of visual concepts such as “seeing things from a particular point of view”). Another user may be more aurally oriented in both perception of external events and processing of internal events (e.g., the user may react most strongly to the way things sound, and may also think in terms of aural concepts such as “being in harmony”). Yet another user may be more kinesthetically oriented in both perception of external events and processing of internal events (e.g., the user may react most strongly or favorably to being active, and may also think in terms of kinesthetic concepts). Determination of a user's preferred mode of perception can be useful in generating a response to the user's input. For example, if a user is classified as “visual,” the dialogue management system100may focus on the way a car looks and may use phrasing such as “it looks like you're interested scheduling a test drive.” Such responses may be more effective in getting the user to actually take the test drive than focusing on the way the car's stereo sounds or using phrasing such as “it sounds like you're interested in scheduling a test drive.” In some embodiments, the perception classifier316can use a textual analysis of user input to determine a perception mode classification, such as the frequency-based classifier described below with respect to the action hierarchy classifier312.

A preferences classifier318can classify input as indicative of one of a set of possible preferences or “values” held by the user providing the input. A preference corresponds to a feature or concept that a user values over other features or concepts. For example, a particular multi-turn dialogue may relate to purchase of a product, such as a car. A user may provide input regarding possibly driving the user's young children in the car, or input regarding discussing the car with the user's father or financial advisor. The preferences classifier318can classify such input as indicative of a preference for “safety” (for the user who will be driving with young children”) or for “resale value” (for the user who will be discussing the car with the user's father). This classification can be useful in generating a response. For example, the dialogue management system100may highlight the advanced safety features of the car when generating responses to users classified as having a “safety” preference or value. In some embodiments, the preferences classifier318can perform NLU on textual input using a neural network that is trained to generate classification scores for each of a predetermined set of preferences or values. An example of a preferences classifier318implemented using a neural network model is described in greater detail below with respect toFIGS. 6 and 7.

An expectation classifier320can classify input as indicative of a user's expectation for a current multi-turn dialogue. A user's expectation may correspond to whether the user expects to be able to perform a desired action and/or how much dialogue or process the user expects to engage in before performance of the desired action seems realistic. For example, a particular multi-turn dialogue may relate to purchase of a product, such as a car. A user may provide input indicating that the user expects to purchase a car, with the primary question being which car to purchase or where to make the purchase. Another user may provide input indicating that the user expects to obtain a significant amount of information and take a significant amount of time to carefully consider the decision. Assignment of these users to different classes can be useful in generating a response. For example, the dialogue management system100may streamline the prompts and accelerate the process towards scheduling a test drive or completing a sale for users who have been assigned to a class indicating their expectation is to purchase a car soon. The dialogue management system100may break the process down into smaller steps and generate more prompts or provide additional information for users who have been assigned to a class indicating their expectation is to cautiously obtain information and consider options before committing to making any purchase. In some embodiments, the expectation classifier320is implemented using a neural network model trained to generate scores for individual expectation classes based on textual input, such as the neural-network-based classifier described below with respect to the preferences classifier318.

An exchange classifier322can classify input with respect to the exchange of resources between dialogue participants. The resources measured by the exchange classifier322may be the resources expected to be exchanged (e.g., provided by a user to the dialogue management system100or some other user) while the user is providing input to the dialogue management system100on the way to performance of a desired action. In some embodiments, the desired action may be purchase of a product, and the resources measured by the exchange classifier322may be “attention” (the user is giving attention to the dialogue management system100), “time” (the user is spending time conversing with the dialogue management system100), “trust” (the user is developing trust in the dialogue management system's responses and/or what is being provided or promised to the user), and “action” (the user is committing to performance of an action—in the case of a product purchase, the user is exchanging money for the product). The resources may be assumed to be exchanged in a linear manner; that is, if the exchange classifier322determines that user input is in the “trust” class, then the user may be assumed to have exchanged attention and time in addition to trust. Assignment of a user to a class representative of a particular exchanged resource can be useful in generating a response. For example, the dialogue management system100may move on from attempting to build credibility and trust if the exchange classifier322determines that the user input is in the “trust” class. In some embodiments, the exchange classifier322is implemented using a criteria-based model and/or a rule-based model in which aspects of the user's textual input and, optionally, contextual information regarding the dialogue are analyzed with respect to criteria and/or rules that may be different for each class.

In some embodiments, an additional classifier may be used to further analyze and classify input with respect to one of the resources of the exchange classifier322, or the exchange classifier322may include one or more sub-classifiers to implement these features. For example, a separate action set classifier may classify input, context, other information, some combination thereof, etc. on a hierarchy of commitment with respect to the “action” resource described above. In one specific, non-limiting embodiment, the commitment hierarchy may include a “possibility” level, an “eventuality” level, a “reality” level, and a “responsibility” level. In this example, the “possibility” level is associated with the lowest likelihood of being ready to perform an action (e.g., exchange the “action” resource), whereas the “responsibility” level may be associated with the highest likelihood of being ready to perform the action. In some embodiments, an action set classifier can use a textual analysis of user input to determine a dialogue vector classification, such as the frequency-based classifier described below with respect to the action hierarchy classifier312. The levels of the hierarchy may be inversely correlated with ease with which they are implemented. For example, use of words like “if” may be easy to introduce into a dialogue, may be indicative of mere “possibility,” and may not represent any commitment to perform an action. Use of words like “when” may be indicative of “eventuality,” and use of words like “as” may be indicative of “reality.” Use of words like “am” may be require greater context to introduce in a dialogue and be more challenging to introduce, and may therefore represent the highest measured level of commitment to perform an action: “responsibility.”

A personality classifier324can classify input as representative of one of a set of possible personality types, such as one of the Myers-Briggs personality types. This classification can be useful in generating a response. For example, the dialogue management system100may respond with different words, phrases, or entirely different lines of questions or prompts based on the differences between the personality types (e.g., some prompts may be more effective with users of particular personality types than with users of other personality types). In some embodiments, the personality classifier324can use a neural network or a frequency-based textual analysis of user input to determine a personality type.

A sentiment classifier326can classify input as representative of one of a set of possible sentiments. The possible sentiments may be determined with respect to particular topics, such as being “positive,” “negative,” or “neutral” with respect to the last dialogue prompt. The possible sentiments may also or alternatively be determined with respect to the user's general outlook or state, such as “angry,” “sad,” and “happy.” Sentiment classification can be useful in generating a response. For example, the dialogue management system100may respond with different words, phrases, or entirely different lines of questions or prompts based on the differences between the sentiments. In some embodiments, the sentiment classifier326can use a neural network or a frequency-based textual analysis of user input to determine a sentiment class.

Although the classifiers shown inFIG. 3have been discussed above—and will be further discussed below—in the context of specific implementations, the examples are illustrative only and are not intended to be limiting. In some embodiments, any of the classifiers discussed herein may be implemented using a frequency-based textual model, a neural-network based processing model, a criteria/rule-based model, some other type of model, or any combination thereof. In addition, althoughFIG. 3shows the classifiers in a parallel configuration, in some embodiments the classifiers (or some subset thereof) may execute sequentially or asynchronously. In some embodiments, the output (or a portion thereof) of some classifiers may serve as the input (or a portion thereof) to other classifiers.

Example Classification Processes

FIG. 4is a flow diagram of an illustrative process400that may be executed by a classifier to determine classes for input based on the frequency with which tokens appear in the textual input. The process400will be described with respect to the illustrative action hierarchy classifier shown inFIG. 5.

The process400shown inFIG. 4begins at block402. The process400may begin in response to an event, such as when input is received by a dialogue management system100, when a classification-based processor112is instantiated or invoked, etc. When the process400is initiated, a set of executable program instructions stored on one or more non-transitory computer-readable media (e.g., hard drive, flash memory, removable media, etc.) may be loaded into memory (e.g., random access memory or “RAM”) of a computing device of the dialogue management system100. The executable instructions may then be executed by a hardware-based computer processor (e.g., a central processing unit or “CPU”) of the computing device. In some embodiments, the process400or portions thereof may be implemented on multiple processors, serially or in parallel.

At block404, the classifier executing the process400(in this example, an action hierarchy classifier) can obtain an input. The input may be text provided by a user of a user device102(or generated by an ASR module) during a multi-turn dialogue with the dialogue management system100. For example, as shown inFIG. 5, the text may be a transcript500of an entire multi-turn dialogue, or of the i-most-recent entries by the user during a multi-turn dialogue (where i is a positive integer).

At block406, the classifier executing the process400can obtain contextual information associated with the current input, the current multi-turn dialogue, or the user in general. For example, previous multi-turn dialogues in which the user has engaged may be accessed and used during the action hierarchy classification process.

At block408, the classifier executing the process400can extract tokens associated with the various classes to which the classifier is configured to assign input. For example, the action hierarchy classifier shown inFIG. 5may be configured to assign input to one of five classes: desire, ability, reason, need, and commitment. There may be a predetermined set of tokens (words or short phrases) associated with each of the classes. Table 1, below, shows an illustrative list of such tokens for each of the classes.

At block410, the classifier executing the process400can determine the frequency with which each of the tokens appears in the input. For example, as shown inFIG. 5, each occurrence of each token may be counted, and a counter502may be maintained to track the number of occurrences of each token. In some embodiments, each occurrence of a token may be counted separately, even if multiple occurrences of a single token appear in a single dialogue turn. In some embodiments, multiple occurrences of a token in a given dialogue turn may be discounted or weighted. For example, multiple occurrences in a single turn may only be counted as one occurrence regardless of how many occurrences are in the dialogue turn. In some embodiments, occurrences of tokens in previous or “older” dialogue turns are discounted or weighted such that they contribute less to the overall frequency counter for the token.

At block412, the classifier executing the process400can determine classification scores for each class. The classification scores may be determined based on the frequency counter generated above in block410.FIG. 5shows an example chart of classification scores504. In some embodiments, the frequency counts for each token of a particular class are summed, and the total of all frequency counts is then summed. The frequency count for each class may then be divided by the sum of all frequency counts to generate a score for each class. In these cases, the classes with higher frequency counts, and therefore higher proportions of the overall frequency count, will be assigned higher scores than classes with lower frequency counts. In some embodiments, the frequency count ratios may serve as the basis for the overall class score, while other information may also be factored into the final classification score or used to adjust the classification score. For example, context information associated with the dialogue or the user may be used to adjust the classification scores. In some embodiments, individual tokens or groups of tokens may be weighted more heavily than other tokens. For example, a particular token may be assigned a weighting factor that is applied (e.g., multiplied to produce a product) either to the frequency count for the token, or used in determining class scores. Illustratively, a frequency of a particular token may be weighted more heavily than other tokens for a particular class, or more heavily for one class and less heavily for a different class. In some embodiments, other information (e.g., output from other classifiers, or other information altogether) may be factored into the final classification score and used to signal and adjust the classification score. The other information may include whether a communication is inbound or outbound, output of an exchange classifier, etc.

FIG. 6is a flow diagram of an illustrative process600that may be executed by a classifier to determine classes for input based on a neural network specifically trained for the task. The process600will be described with respect to the illustrative preferences classifier shown inFIG. 7.

The process600shown inFIG. 6begins at block602. The process600may begin in response to an event, such as when input is received by a dialogue management system100, when a classification-based processor112is instantiated or invoked, etc. When the process600is initiated, a set of executable program instructions stored on one or more non-transitory computer-readable media (e.g., hard drive, flash memory, removable media, etc.) may be loaded into memory (e.g., random access memory or “RAM”) of a computing device of the dialogue management system100. The executable instructions may then be executed by a hardware-based computer processor (e.g., a central processing unit or “CPU”) of the computing device. In some embodiments, the process600or portions thereof may be implemented on multiple processors, serially or in parallel.

At block604, the classifier executing the process600(in this example, a preferences classifier) can obtain an input. As shown inFIG. 7, the input may be text200provided by a user of a user device102(or generated by an ASR module) during a multi-turn dialogue with the dialogue management system100. In some embodiments, the text may be a transcript of an entire multi-turn dialogue, or of the i-most-recent entries by the user during a multi-turn dialogue (where i is a positive integer).

At block606, the classifier executing the process600can obtain contextual information associated with the current input, the current multi-turn dialogue, or the user in general. For example, previous multi-turn dialogues in which the user engaged may be accessed and used during the preferences classification process.

At block608, the classifier executing the process600can generate one or more input vectors or “feature” vectors for processing by a neural network. For example, the preferences classifier shown inFIG. 7may use a neural network702to generate classification scores704for multiple preference classes. The neural network702may take, as input, a vector700of data to be processed by the neural network702. The vector700may include data regarding the words in the input text200, various items of context data300, other information, some combination thereof, etc. Illustratively, the n items in the vector700may be referred to as dimensions, and thus the vector700may be an n-dimensional vector. In some embodiments, output of other classifiers may be used to signal information to the neural network702as information to be incorporated into the vector700. For example, the output of the action hierarchy classifier312and/or various contextual information may be incorporated into the vector700.

The value for an individual dimension may be a number that is processed by the neural network702as described below. In some embodiments, text data (e.g., dialogue input and/or dialogue output) may be incorporated into the vector700directly (e.g., as text data) or after processing into numeric form. For example, if prior dialogue output asked the user a specific question, then the next dialogue input may be processed into numeric form. Illustratively, if a user is asked a question with a limited number of possible responses, then the particular response may be mapped to a value that is included in the vector100. Each value may correspond to a particular category or classification, and therefore multiple inputs responsive to such dialogue prompts can be aggregated, averaged, or otherwise analyzed and used to determine the strength of the input signal with respect to the particular category or classification.

At block610, the classifier executing the process600can perform a neural network forward pass on the vector700. The forward pass involves multiplying a weight matrix, representing a subset of the parameters of the neural network702, by the input vector and then applying an activation function to generate an initial result. This initial result may be referred to as an “internal” or “hidden” layer because most neural networks multiply the initial result by at least one other matrix and apply at least one other activation function to generate at least one other result. The process of matrix multiplication and activation function application may be repeated for each internal layer of the neural network702.

At block612, the classifier executing the process600can determine classification scores for each class. In classification systems, neural-network-based models may generate scores via the forward pass. For example, the last set of results generated by the forward pass (e.g., the “output layer”) may include the scores for each class. Individual scores may indicate the probability that the input corresponds to a particular class. In the example shown inFIG. 7, the “reliability” preference may be the class that is most likely correct for the given input text200and context300.

FIG. 8is a flow diagram of an illustrative process800that may be executed by a classifier to determine classes for input based on various criteria and/or rules. The process800will be described with respect to the illustrative exchange classifier shown inFIG. 9.

The process800shown inFIG. 8begins at block802. The process800may begin in response to an event, such as when input is received by a dialogue management system100, when a classification-based processor112is instantiated or invoked, etc. When the process800is initiated, a set of executable program instructions stored on one or more non-transitory computer-readable media (e.g., hard drive, flash memory, removable media, etc.) may be loaded into memory (e.g., random access memory or “RAM”) of a computing device of the dialogue management system100. The executable instructions may then be executed by a hardware-based computer processor (e.g., a central processing unit or “CPU”) of the computing device. In some embodiments, the process800or portions thereof may be implemented on multiple processors, serially or in parallel.

At block804, the classifier executing the process800(in this example, an exchange classifier) can obtain an input. As shown inFIG. 9, the input may be text200provided by a user of a user device102(or generated by an ASR module) during a multi-turn dialogue with the dialogue management system100. In some embodiments, the text may be a transcript of an entire multi-turn dialogue, or of the i-most-recent entries by the user during a multi-turn dialogue (where i is a positive integer).

At block806, the classifier executing the process800can obtain contextual information associated with the current input, the current multi-turn dialogue, or the user in general. For example, previous multi-turn dialogues in which the user engaged may be accessed and used during the preferences classification process.

At block808, the classifier executing the process800can apply classification criteria and/or rules associated with a class. For example, the exchange classifier shown inFIG. 9can assess criteria or apply rules900that may be specific for each class. The criteria may be rule-based (e.g., if a criterion is met then a value is assigned) or computed (e.g., input analyzed to count or aggregate instances of events). For the “attention” class, the exchange classifier may consider whether the user is responding to the prompts of the dialogue management system100, has initiated contact with the system in an unscheduled manner, etc. If the user is being responsive and/or has initiated contact with the system in an unprompted or unscheduled manner, the exchange of attention may be noted. For the “time” class, the exchange classifier may consider certain actions the user has performed or commit to perform, such as scheduling a test drive. This consideration may be based on the action hierarchy discussed above (e.g., if the user has reached the “commitment” level, then the exchange of time may be noted). For the “trust” class, the exchange classifier may consider a frequency-based analysis of text. If the user has used a threshold number or percentage of words indicative of trust, then the exchange of trust may be noted. For the “action” class, the exchange classifier may consider certain actions that the user has performed or committed to perform, such as signing a contract or transferring funds.

At decision block810, the classifier executing the process800can determine whether there are additional classes to assess. If so, the process800can return to block808. Otherwise, the process800can terminate at block812. For example, as shown inFIG. 9, the exchange classifier can assess classification criteria and/or rules for each of four different classes. The assessments may occur on a class-by-class basis, and therefore portions of the process800may be repeated for each class, or performed in parallel for each class.

Example Response Generation Process

FIG. 10is a flow diagram of an illustrative process1000that may be executed by a dialogue management system100to generate dialogue responses or other prompts based on user input and classifications determined by one or more classifiers. The process1000will be described with reference to the illustrative response engine shown inFIG. 11.

The process1000shown inFIG. 10begins at block1002. The process1000may begin in response to an event, such as when input is received by a dialogue management system100. When the process1000is initiated, a set of executable program instructions stored on one or more non-transitory computer-readable media (e.g., hard drive, flash memory, removable media, etc.) may be loaded into memory (e.g., random access memory or “RAM”) of a computing device of the dialogue management system100. The executable instructions may then be executed by a hardware-based computer processor (e.g., a central processing unit or “CPU”) of the computing device. In some embodiments, the process1000or portions thereof may be implemented on multiple processors, serially or in parallel.

At block1004, the dialogue management system100can obtain an input. The input may be text provided by a user of a user device102(or generated by an ASR module) during a multi-turn dialogue with the dialogue management system1000.

At block1006, the dialogue management system100or some component thereof, such as a classification-based processor112, can generate classification results using the input. For example, the various classifiers shown inFIG. 3may be used to assign the input to various classes as described in greater detail above.

At block1008, the dialogue management system100or some component thereof, such as a response engine206, can identify a response template that is likely to advance progress of the dialogue toward a desired result, such as a level of the action hierarchy. As shown inFIG. 11, a template selector1100can obtain the classification results302generated above, and determine which template (or templates) in the templates data store212corresponds to the classification results. For example, the templates data store212may include a different response template for each possible combination of classification results (e.g., each possible combination of intents, action hierarchy levels, perception modes, preferences, expectations, exchange levels, personality types, and sentiments). In some embodiments, a single template may be associated with multiple different combinations of possible classifications. In some embodiments, a single combination of classifications may be associated with multiple templates, such as a variety of different templates for use depending upon the current context of the dialogue (e.g., what information is required to progress to the next level in the action hierarchy). In some embodiments, templates may not be statically assigned to individual sets of classification results. Instead, templates may be associated with quantifiable characteristics or properties, such as scores representing the relative relevance of the templates for various classifications or combinations of classification results. Then, a set of current classification results and, optionally, additional context is analyzed and the various templates may be scored with respect to the current set of classification results (and context). The highest-scoring template may then be chosen for further processing as described below.

At block1010, the dialogue management system100or some component thereof, such as a response engine206, can determine one or more customizations to the response template identified above. As shown inFIG. 11, a response generator1102can obtain a template selected by the template selector1100, classification results302, and/or various other data (e.g., contextual data). The response generator1102can then customize the template to generate a processed response208. For example, a particular response template may be “You <perception class> a <car in context>.” The contextual information may show that the user is considering scheduling test drive of a Chevy Camaro. If the perception classifier has determined that the user is in the visually-oriented class, the template may be customized to read “You would look good in a Chevy Camaro.” If the perception classifier has determined that the user is in the auditory-oriented class, the template may be customized to read “You'll love the engine sound of a Chevy Camaro.” If the perception classifier has determined that the user is in the kinesthetically-oriented class, the template may be customized to read “You'll love the feel of the open road in a Chevy Camaro.”

In some embodiments, the response engine206can determine which customizations to make to the template using a model that accepts, as input, the identified template, the current classification results, additional context of the dialogue, or some combination thereof. The model (e.g., a neural network, conditional random field, etc.) may generate output that is indicative of which words to place in which portions of the template. For example, if the template has two portions to be customized, as in the “You <perception class> a <car in context>” template described above, the model may generate an output vector comprises two elements: one for each of the portions to be customized. The elements may have numeric values that can be translated, cross-referenced, or otherwise transformed into a word or phrase to be inserted into the appropriate location of the template.

In some embodiments, additional or alternative factors may be used to customize a template at block1010, or select a template at block1008. For example, different verb tenses may be used move the user through a process, such as using the future tense in cases where it is desired to have the user plan an action, using the past tense in cases where it is desired to have the user visualize looking back on a desired action, using the present tense to motivate the user to act now, etc.

In some embodiments, templates may be chosen and/or customized such that responses include dialogue vector words to move the user through a process. Dialogue vector words are words (or phrases) that steer and shape the productivity of a conversation. Redirection words such as “but,” “yet,” “instead,” and “so” can be used to redirect a dialogue toward a desired outcome (e.g., if a user backs away from a desired action by using the word “but,” the same word can be used to redirect the user back in the direction of the desired action). Linking words such as “and” can be used to link concepts, such as linking a desired action to an action that the user has indicated a willingness to perform. Directional words such as “to” or “because” can continue to move the dialogue in a direction and amplify certain points. Defensive words such as “why” provoke a user to defend a decision or assertion. Invitation words, such as “if” invite a user to move past resistance and take an action without the need for logical leaps. Words such as “or” indicate different options from which a user may choose. A closed “or” can limit the options, while an open-ended “or” can focus the user on certain options while allowing for the possibility of using other options. Equivalence words such as “means” or “is” can be used to indicate the equivalence of two concepts (such as a concept about which the user has shown a positive sentiment, and a concept about which it is desired that the user show a positive sentiment).

In some embodiments, a response template may occasionally be chosen and/or customized in a semi-random manner, or in a manner which does not otherwise follow the predetermined assignments and determinations of the system. For example, a particular dialogue context and set of classification results may lead to selection of a particular response template that has been previously assigned to the set of classification results and that corresponds to the dialogue context. However, rather than choosing that particular template every time the assigned set of classification results and dialogue context correspondence is observed, a different template (or customization to a template) can be chosen in a small, pseudo-random subset of such cases. For example, a template that is next-most-closely associated with the set of classification results and dialogue context may be chosen. As another example, a template that has not been used recently but which still makes sense for the current dialogue context may be chosen (e.g., a template that is still responsive to a question from the dialogue participant, or a template for which all required customization information is available and which will is still likely to move the dialogue forward toward performance of a desired action or another desired goal). By using such alternative responses, the system can obtain real-world feedback regarding the effect of different templates. Such data can be used to automatically or manually adjust the manner in which templates are chosen and/or customized in the future to achieve more desirable results.

In some embodiments, response templates and/or customizations may provide information in an unprompted manner, or request information that can be used to provide such unprompted information. The system can then use participant responses to continue to move the dialogue forward toward performance of a desired action or another desired goal. For example, in a dialogue regarding a possible purchase of a car, a response may be generated that prompts the dialogue participant to provide an estimated daily commute time (e.g., “How long is your daily commute?”). Based on the participant's response, the system can select a follow-up response that provides interesting-but-unprompted information designed to move the user's subsequent responses toward a different classification result (e.g., “Wow, that's a long commute. The average is x minutes. You should consider comfort and interior amenities when choosing a car.”).

At block1012, the dialogue management system100or some component thereof, such as the chat server110, can provide the processed response to the user device102.

At decision block1014, the dialogue management system100or some component thereof, such as the chat server110, can determine whether additional input has been received. If so, the process1000can return to block1006. Otherwise, the process1000may terminate at block1016.

In some embodiments, the dialogue management system100does not identify and customize a response template. Rather, the dialogue management system100generates a response word-by-word or phrase-by-phrase. For example, the dialogue management system100may use a generative model that takes input, such as the classification results302, context300, and/or other information. The generative model may then “translate” the input into a response (e.g., a question, comment, or answer) without accessing a template and customizing the template. Illustratively, the generative model may be a recurrent neural network (“RNN”) that identifies features and dependencies across sequences of events. Because multi-turn dialogues include input and responses occurring in a sequence, a recurrent neural network may provide desired performance.

The generative model may be trained using a corpus of training data that includes sets of training input and desired output. The sets of training input may be examples of multi-turn dialogue input and, in some cases, corresponding system output (e.g., input to the system, output from the system, etc.) over the course of a dialogue leading up to the latest dialogue input. Each set of training data input may be assigned to or otherwise associated with a corresponding desired output, such as the output the is desired from the system when presented with a particular dialogue input or multi-turn dialogue state. The training data inputs may then be processed by the model, and the results compared to the results that correspond to the desired output. For example, classifier data from any or all of the classifiers described above, context data, text data representative of dialogue turns (input and/or output), other information, some combination thereof, etc. may be provided to the model. Input may be processed into vectors, which may be multi-dimensional data structures that include values of classifier output, tokens extracted from dialogue turns, etc. The vectors may then be multiplied by one or more matrices of model parameters to generate results. In some embodiments, regularization functions (e.g., sigmoid) may be applied to transform the results into probabilities or scores.

In some embodiments, output of the model may be a number or collection of numbers that can be translated or otherwise transformed into text data, such as a sentence or phrase to be used to respond to a dialogue input. The desired output may be the number or collection of numbers that, when transformed into text data, represent the “gold standard” or otherwise desired response of the system when presented with such dialogue input or such a dialogue state. Based on the comparison of output to desired output, some or all parameters of the model may be adjusted so that in subsequent executions the results come closer to the desired output. This process may be repeated a predetermined or dynamically determined number of times or until some convergence criteria is met (e.g., the output is within a threshold distance of the desired output).

In use, the generative model may receive input as described above, and generate output. However, as the desired output may not be known ahead of time for this exact set of real-world input, there may not be a step of comparing the output to desired output. Rather, the response generator may proceed with generation of the response208.

In some embodiments, the dialogue management system100may use a hybrid retrieval/generative approach to generating responses. For example, the response engine206may first determine a template or set of templates that appear to be most relevant (e.g., highest-scoring) for the current input (e.g., classification results302, context300, etc.). If none of the templates appears to be particularly relevant (e.g., if no template is associated with a score that satisfies a threshold), then the response generator1102may proceed to generate the response208using the generative model, rather than by customizing a template. In some embodiments, the response engine206may generate responses using both a retrieval approach and a generative approach, and then select a most-relevant response (e.g., highest-scoring) to use as the response208.

Example User Interface

FIG. 12shows an example user interface1200that the dialogue management system100can use to present the results of an ongoing or previous dialogue. For example, the user interface1200can be presented to a person (e.g., customer service representative, therapist, etc.) currently engaging in an interactive dialogue with a user of a user device102. The person can see the transcript1202of the dialogue, profile information1204about the user with whom they are communicating, classification results1206generated from the transcript1202, and summary results1208that may correspond to the current “bottom line” estimates for this dialogue. In some embodiments, the user interface1200can be presented to a person after an automated dialogue between a user of a user device102and the dialogue management system100. For example, if a user accesses the dialogue management system100to obtain information about a car and schedule a test drive, the results of that dialogue can be transmitted to a data consumer104, such as the car dealership at which the user is to take the test drive. A customer service representative at the car dealership may access the user interface1200before meeting the user for the test drive. In this way, the customer service representative can obtain information regarding what the user communicated to the dialogue management system100during the dialogue, and what classes the dialogue management system100has determined for the user.

In some embodiments, the user interface1200can present accumulated information regarding multiple dialogues, possibly with multiple parties. For example, dialogue transcripts and/or classification results may be accumulated over the course of interactions with an automated chat server for (1) setting up a test drive appointment and (2) conversing with a dealer representative regarding the test drive or follow-up negotiations. Dealer representatives may supplement the information with their own observations, copies of emails exchanged with the dialogue participant, etc. In this way, the classifications can continue to be updated and leveraged by other staff at the dealership even after the initial computer-based dialogue, such as by financing specialists who typically interact with a person only after the purchase decision has been finalized.

Some inventive aspects of the disclosure are set forth in the following clauses:

under control of a computing system comprising one or more computing devices configured to execute specific instructions,receiving text corresponding to one or more inputs of a user; andin response to an input of the one or more inputs:processing at least a portion of the text in a first classifier, wherein the first classifier generates a first plurality of scores, wherein individual scores of the first plurality of scores correspond to individual levels in a hierarchy, wherein individual levels in the hierarchy correspond to individual phases preceding performance of an action, and wherein the first plurality of scores are generated using a model that maps individual words of the text to individual levels in the hierarchy;processing at least a portion of the text in a second classifier, wherein the second classifier generates a second plurality of scores, wherein individual scores of the second plurality of scores correspond to individual user preferences associated with the action, and wherein the second plurality of scores are generated using a neural network trained to generate scores corresponding to a plurality of user preferences associated with the action;identifying a response template based at least partly on a level of the hierarchy associated with a highest score of the first plurality of scores;generating a response using the response template and a customization to the response template, wherein the customization is based at least partly on a user preference associated with a highest score of the second plurality of scores; andpresenting the response.

2. The computer-implemented method of clause 1, further comprising identifying the template based at least partly on a likelihood that processing text, corresponding to a subsequent input of the user, in the first classifier will produce a result corresponding to a progression of the user to a higher level of the hierarchy.

3. The computer-implemented method of clause 1, further comprising processing at least a portion of the text in a third classifier, wherein the third classifier generates a third plurality of scores, wherein generating the response is further based at least partly on a highest score of the third plurality of scores.

4. The computer-implemented method of clause 3, wherein individual scores of the third plurality of scores correspond to individual resources of a plurality of resources expected to be exchanged by the user preceding performance of the action, and wherein the highest score of the third plurality of scores corresponds to a resource, of the plurality of resources, likely to be exchanged by the user.

5. The computer-implemented method of clause 3, wherein individual scores of the third plurality of scores correspond to individual modes of perception of a plurality of modes of perception, and wherein the highest score of the third plurality of scores corresponds to a mode of perception, of the plurality of modes of perception, likely to be preferred by the user.

6. The computer-implemented method of clause 3, wherein individual scores of the third plurality of scores correspond to individual expectations of a plurality of expectations regarding successfully performing action, and wherein the highest score of the third plurality of scores corresponds to an expectation, of the plurality of expectations, likely to be associated with the user.

7. The computer-implemented method of clause 1, further comprising identifying a vector word to be included in the response, wherein the vector word is identified based at least partly on a desired direction of a dialogue, and wherein the response comprises the vector word.

8. The computer-implemented method of clause 1, wherein the action comprises one from a group consisting of: making a purchase, and changing a behavior.

9. The computer-implemented method of clause 1, further comprising providing to a second computing system:at least a portion of the text;data indicative of the level corresponding to the highest score of the first plurality of scores; anddata indicative of the preference corresponding to the highest score of the second plurality of scores.

under control of a computing system comprising one or more computing devices configured to execute specific instructions,obtaining text corresponding to one or more inputs of a user, wherein the one or more inputs are provided by the user to the computing system during a multi-turn dialogue associated with an action;obtaining a textual model comprising a plurality of words, a plurality of classes, and a plurality of mappings,wherein individual mappings of the plurality of mappings map a word of the plurality of words to a class of the plurality of classes, andwherein individual classes of the plurality of classes correspond to individual levels of a hierarchy comprising a plurality of levels through which classification of input is expected to progress before performance of the action;determining frequencies with which at least a subset of plurality of words occurs in the text;identifying a level of the plurality of levels based at least partly on the level being associated with higher frequencies than other levels of the plurality of levels; andclassifying the user as likely being associated with the level.

11. The computer-implemented method of clause 10, further comprising:processing at least a portion of the text in a classifier, wherein the classifier generates a plurality of scores; andclassifying the user as also being associated with a class corresponding to a highest score of the plurality of scores.

12. The computer-implemented method of clause 11, wherein individual scores of the plurality of scores correspond to individual resources of a plurality of resources expected to be exchanged by the user preceding performance of the action, and wherein the highest score of the plurality of scores corresponds to a resource, of the plurality of resources, likely to be exchanged by the user.

13. The computer-implemented method of clause 11, wherein individual scores of the plurality of scores correspond to individual modes of perception of a plurality of modes of perception, and wherein the highest score of the plurality of scores corresponds to a mode of perception, of the plurality of modes of perception, likely to be preferred by the user.

14. The computer-implemented method of clause 11, wherein individual scores of the plurality of scores correspond to individual expectations of a plurality of expectations associated with action, and wherein the highest score of the plurality of scores corresponds to an expectation, of the plurality of expectations, likely to be associated with the user.

15. The computer-implemented method of clause 10, wherein the action comprises one from a group consisting of: making a purchase, and changing a behavior.

16. The computer-implemented method of clause 10, further comprising providing to a second computing system:at least a portion of the text; anddata indicative of the level with which the user is associated.
Terminology