Patent ID: 12223511

DETAILED DESCRIPTION

FIG.1is a block diagram illustrating an example operation of an emotion classification engine in accordance with techniques of this disclosure.

The emotion classification engine illustrated inFIG.1is DIVA engine4, which refers to the four emotion factor values of Determination, Inquisitiveness, Valence, and Aggression that the engine is configured to determine from input communication data and use to classify the communication data as being associated with a given emotion state. In the illustrated example ofFIG.1, DIVA engine4includes a DIVA indexer10and an emotion classifier20. The DIVA indexer10may comprise four machine learning models trained to output the four different emotion factor values for communication data. The emotion factor values may be represented as numerical numbers (e.g., between 0 and 1, between −2 and 2, or the like) reflecting the intensity of a specified emotion present in the communication data. For communication data representing a customer communication, for example, a determination model12may be trained to output a determination value, an inquisitiveness model14may be trained to output an inquisitiveness value, a valence model16may be trained to output a valence value, and an aggression model18may be trained to output an aggression value. DIVA indexer10may output the four emotion factor values to an emotion classifier20, which may be a machine learning model or a rule-based model, configured to classify the communication data into an associated emotion state (e.g., angry, curious, happy, etc.) based on the four emotion factor values.

DIVA engine4may be supported on one or more servers or other computing systems or devices within an organization network. For example, DIVA engine4may comprise software code executing on processors or processing circuitry of one or more computing systems that may be included in a centralized or distributed network of disparate computing devices. In some examples, one or more of the emotion factor models12,14,16, and18and emotion classifier20may each be supported by different computing systems or devices within the network. In other examples, DIVA indexer10may be supported on the same computing system, and emotion classifier20may be supported on the same computing system or a different computing system within the network.

Upon receipt of communication data representing a customer communication for processing, data pre-processor2may perform preprocessing to prepare the communication data for application to the DIVA engine4machine learning models. The communication data representing a customer communication may also be saved to a database in memory. The four machine learning models12,14,16, and18of DIVA indexer10are trained to recognize certain emotion factors within communication data and output emotion factor values reflecting the presence and intensity of those emotions. The emotion factor values output from DIVA indexer10may be saved to an emotion factor index database22and an identification number may be assigned to the communication data in memory to associate the communication data with the emotion factor values. The saved emotion factor values may also be associated with a customer who is the source of the communication data. In some examples, the emotion classifier20may also retrieve historic, saved emotion factor values for previous communication data associated with the customer from the emotion factor index database22as additional input to determine an emotion state of current communication data of the customer. The use of historic emotion factor values of the customer may enable emotion classifier20to more accurately classify an emotion state of a customer associated with the current communication data by identifying trends or sudden changes in the emotion factor values of the customer over time. Emotion states may be saved to an emotion state database (not shown inFIG.1) and associated with the respective communication data, originating customer, and/or emotion factor values.

In some examples, instead of only relying on the specific customer's own historic emotion factor values, emotion classifier20may use historic emotion factor values associated with a grouping or profile of customers that includes the specific customer. For example, customer profiles may be identified for groups of customers based on geographical location, education level, age, profession, socioeconomic status, or other categorization. The use of customer profiles may provide a larger historic data set from which emotion classifier20may learn to identify emotional trends over time.

Conventional sentiment analysis systems generally classify communication data as being positive, neutral, or negative. Unlike these conventional systems, the emotion classification engine described herein, e.g., DIVA engine4ofFIG.1, includes an indexer, e.g., DIVA indexer10ofFIG.1, configured to identify the existence and intensity of four emotions specifically useful for financial institutions when handling customer communications. In addition, the machine learning models included within the indexer, e.g., models12,14,16, and18ofFIG.1, may be trained using communication data received by financial institutions, rather than general communication data from different environments. In this way, the training of the machine learning models within DIVA indexer10may be more specific to financial institutions and more accurate in identifying emotive content in communications with a financial institution.

The four emotion factor values output by the machine learning models within DIVA indexer10may correspond to a customer's perceived determination, inquisitiveness, valence, and aggression within an inquiry, complaint, or other customer communication. The determination factor value may correspond to a level of purposefulness of the speech of the customer. In some examples, a customer communication that is highly focused on a specific topic may have a high determination factor value. In some examples, a customer communication that repeats itself may have a high determination factor value. In some examples, a customer communication that makes only a short, single statement or a broad, indefinite statement may have a low determination factor value.

The inquisitiveness value may be a measure of the level of curiosity of the speech of the customer communication. In some examples, a customer service inquiry that is probing for information about various aspects of the customer's account or the organization may have a high inquisitiveness value. In some examples, the customer communication that does not indicate an interest in learning anything may have a low inquisitiveness value. The customer who submits a communication with a low inquisitiveness value may wish to resolve any issues without receiving further information.

The valence value may be a measure of the attitude conveyed by the speech of the customer. In some examples, a customer communication that is very negative may have a low valence value. In some examples, a customer communication that is cheerful may have a high valence value. The aggression value may be a measure of the aggressiveness of the speech of the customer communication. In some examples, a customer communication that is brusque may have a high aggression value. In some examples, a customer communication that uses an authoritative tone of voice may have a high aggression value. In some examples, a customer communication that sounds meek or pathetic may have a low aggression value.

An emotion classifier, e.g., emotion classifier20ofFIG.1, within the emotion classification engine described herein is configured to classify an emotional state of the customer communication based at least in part on the four emotion factor values output from the machine learning models within DIVA indexer10for the customer communication. In some examples, emotion classifier20may comprise a machine learning-based or rule-based algorithmic model configured to map the emotion factor values for the customer communication to an emotional state, emotion score, or other emotional indicator. In one example, emotion classifier20may have access to a library of emotional states algorithmically tied to different combinations of emotion factor value inputs. The emotional state library may contain emotion states such as “curious,” “trusting,” “disgruntled,” “interesting” etc. As described in this disclosure, the emotion classifier included within the emotion classification engine described herein may be use case specific. In accordance with the techniques described in this disclosure, the emotion classifier may comprise an emotion classification model configured to classify the current communication into an emotional state based on the emotion factor values.

DIVA indexer10and emotion classifier20may include functions (e.g., machine learning algorithms and/or rule-based algorithms) configured to be executed by processors. In some examples, the machine learning models within DIVA indexer10implement supervised learning, e.g., classify sets of data into groups. For example, a set of data, such as a sequence of code pairs representing customer communication data, may be classified into four values (determination, inquisitiveness, valence, and aggression). The function may include nodes, layers, and connections, and the function may be represented by equations having a plurality of variables and a plurality of known coefficients.

Machine learning algorithms, such as those of DIVA engine4, may be trained using a training process to create data-specific models. After the training process, the created models may be capable of determining an output data set based on an input data set (e.g., match a sequence of text data strings representing a customer service inquiry to one or more known emotion factor values or emotion states). The training process may implement one or more sets of training data to create the models.

A computing system may be configured to train the deep learning models of the DIVA indexer (determination model12, inquisitiveness model14, valence model16, and aggression model18) based on a set of training data that includes a plurality of customer communications in one or more memories or storage systems within the organization network, in which each customer communication of the plurality of customer communications is pre-categorized as associated with at least one emotion factor value. The deep learning models may include an artificial neural network, such as an RNN. During training of the model, an RNN may identify a plurality of patterns in a plurality of sequences of events. For example, the RNN may observe word phrases of customer communications known to be indicative of an aggressive emotion. After the model is trained, the model may accept a customer communication as an input and output an emotion factor value (e.g., an integer between −2 and 2, inclusive) as an output, a process known as sequence classification.

A computing system may be configured to train a deep learning model like emotion classifier20based on a set of training data that includes a plurality of emotion factor value sets in one or more memories or storage systems within the organization network, in which each set of emotion factor values the plurality of sets of emotion factor values is pre-categorized as associated with at least a certain emotion state. The deep learning model may include an artificial neural network, such as an RNN. During training of the model, an RNN may identify a plurality of patterns in a plurality of sequences of events. For example, the RNN may observe emotion factor value combinations known to be indicative of a depressed emotion. After the model is trained, the model may accept a set of emotion factor values as an input and output an emotion classification (e.g., angry, interested, joyful, trusting, depressed, etc.) as an output, a process known as sequence classification.

The DIVA engine4may be implemented on any suitable computing system, such as one or more server computers, workstations, mainframes, appliances, cloud computing systems, and/or other computing systems that may be capable of performing operations and/or functions described in accordance with one or more aspects of the present disclosure. In some examples, the DIVA engine4may be implemented on a computing system that represents a cloud computing system, server farm, and/or server cluster (or portion thereof) that provides services to customer devices and other devices or systems, e.g., agent workstations within a financial institution. In other examples, the DIVA engine4may represent or be implemented through one or more virtualized compute instances (e.g., virtual machines, containers) of a data center, cloud computing system, server farm, and/or server cluster.

FIG.2is a block diagram illustrating an example emotion classification system214, in accordance with the techniques of this disclosure. The emotion classification system includes DIVA engine4as described inFIG.1in which the emotion classifier is a use case specific emotion classification model220configured to classify a customer communication into an emotion state based on emotion factor values output by the machine learning models of DIVA indexer10.

As illustrated inFIG.2, one or more user devices206A-206N (collectively “user devices206”) are in communication with emotion classification system214via a network204. Emotion classification system214may comprise a network run partially on devices in a facility configured to handle incoming messages from user devices206operated by users that may be customers or potential customers of a business or organization. Emotion classification system214may include several disparate computing systems configured to handle customer communications focused on customer accounts with the business or other services provided by the business, e.g., servicing existing accounts, opening new accounts, servicing existing loans, and opening new loans. In some examples described in this disclosure, emotion classification system214may comprise a customer service center of a bank or other financial institution. A contact center of the emotion classification system214may allow customers to speak to a live person when resolving service issues and/or leave a voice message detailing one or more service issues. Additionally, or alternatively, customers may submit messages (e.g., communications or complaints) via text channels such as email, text messaging, and social media messaging.

User devices206may be any suitable communication or computing device, such as a conventional or landline phone, or a mobile, non-mobile, wearable, and/or non-wearable computing device capable of communicating over network204. For example, each user device of user devices206may include any one or combination of a landline phone, a conventional mobile phone, a smart phone, a tablet computer, a computerized watch, a computerized glove or gloves, a personal digital assistant, a virtual assistant, a gaming system, a media player, an e-book reader, a television or television platform, a bicycle, automobile, or navigation, information and/or entertainment system for a bicycle, automobile or other vehicle, a laptop or notebook computer, a desktop computer, or any other type of wearable, non-wearable, mobile, and non-mobile computing device that may perform operations in accordance with one or more aspects of the present disclosure. One or more of user devices206may support communication services over packet-switched networks, e.g., the public Internet, including Voice over Internet Protocol (VOIP). One or more of user devices206may also support communication services over circuit-switched networks, e.g., the public switched telephone network (PSTN).

Each of user devices206is operated by a user that may be a customer or a potential customer of the business or organization that provides emotion classification system214. In the case of a business or corporate customer, the user may be a representative of the business or a corporate customer. In some examples, the user may be a customer who interacts with the business or organization through account inquiries, service inquiries etc. In one or more cases where the user has previously interacted with the business or organization by communications sent to the business or organization, the emotion factor values and emotion states for those communications may be stored in databases in memory of emotion classification system214.

The emotion classification system214may comprise one or more physical entities (e.g., computing devices, computer servers, quantum computers, desktop computers, tablet computers, laptop computers, smartphones, etc.) and/or virtual entities (e.g., virtual machines, application software in computing machines, a cloud computing system, etc.). In certain examples, the emotion classification system214may include one or more computers that process information and/or devices with embedded computers.

Network204and emotion classification system214may comprise computer networks (e.g., a wide area network (WAN), such as the Internet, a local area network (LAN), or a virtual private network (VPN)), a telephone network (e.g., the PSTN or a wireless network), or another wired or wireless communication network. Although illustrated as single entities, each of network204and emotion classification system214may include a combination of multiple networks. In some examples, network204may comprise a public network or a private access network through which user devices206may access emotion classification system214. In some examples, emotion classification system214may comprise a private network of a business or organization, e.g., a bank or other financial institutions.

Emotion classification system214may be implemented as any suitable computing system, such as one or more server computers, workstations, mainframes, appliances, cloud computing systems, and/or other computing systems that may be capable of performing operations and/or functions described in accordance with one or more aspects of the present disclosure. In some examples, emotion classification system214represents cloud computing systems, server farms, and/or server clusters (or portions thereof) that provide services to customer devices and other devices or systems. In other examples, emotion classification system214may represent or be implemented through one or more virtualized compute instances (e.g., virtual machines, containers) of a data center, cloud computing system, server farm, and/or server cluster. Emotion classification system214may communicate with external systems via one or more networks (e.g., network204). In some examples, emotion classification system214may use network interfaces (such as Ethernet interfaces, optical transceivers, radio frequency (RF) transceivers, Wi-Fi or Bluetooth radios, or the like), telephony interfaces, or any other type of device that can send and receive information to wirelessly communicate with external systems, e.g., network204, user device206, agent devices224, etc.

Emotion classification system214may include a centralized or distributed network of disparate computing systems made up of interconnected desktop computers, laptops, workstations, wireless devices, network-ready appliances, file servers, print servers, or other computing devices. For example, emotion classification system214may include one or more data centers including a plurality of servers configured to provide account services interconnected with a plurality of databases and other storage facilities in which customer credentials, customer profiles, and customer accounts are stored.

Emotion classification system214may include systems with which a user may interact, including one or more agent devices224used by a number of human agents that are representatives of the business or organization. The one or more agent devices224may be configured to display or otherwise provide data about the customer or the customer's communications with the business or organization to the human agents, including current and/or historic emotion states for communications from the customer.

In the illustrated example ofFIG.2, emotion classification system214also includes data pre-preprocessor2, DIVA engine4that includes one or more machine learning models within DIVA indexer10, as described with respect toFIG.1, and emotion classification model220. Data pre-processor2may prepare communications data received from user device206for application to the one or more machine learning models within DIVA engine4. Emotion classification system214may also include components necessary for collecting, storing, and maintaining data used by emotion classification system214. The architecture of emotion classification system214illustrated inFIG.2is shown for exemplary purposes only and should not be limited to this architecture. In other examples, emotion classification system214may include more, fewer, or different computing systems configured to handle customer messages.

In some examples, emotion classification system214receives an inbound message from a user device, e.g., user device206A, via network204and determines whether to route the inbound message to data pre-processor2. In accordance with one or more techniques of this disclosure, the message may comprise communication data in the form of text or audio, such as emails, scanned letters, online chat, telephone calls, etc. A speech recognition model may be used to convert audio customer service inquiries to plain text data via natural language processing. A text image recognition model may be used to convert hand- or typewritten customer service inquiries to plain text data or text-based annotation data.

Text-based annotation data may be a combination of two sets of plain text data. The first set of plain text data may comprise the words and/or text of a customer's message to an organization, while the second set of plain text data may comprise annotations by an agent of the organization. In some examples, the customer may send communication data to an organization in the form of a visual data (e.g., letter, fax, video call, etc.), or audio data (e.g., phone call, web call, video call, etc.). The visual or audio data may have indications of emotive content not captured by the words of the message alone. In that case, an agent of the organization may add annotations, in the form of plain-text data, to the communication data for the customer's message. For example, annotations may describe the customer's behavior during a phone call, including shouting, pleading, sincere gratitude, and other descriptions of displayed emotion not conveyed through the words of the conversation alone. In some examples, annotations may describe a letter as smudged with tears, or showing red circle marks around certain words, and other descriptions of displayed emotion not conveyed through the words in the letter alone.

Data pre-processor2may prepare communication data indicative of a customer interaction/message for submission to the machine learning models of DIVA engine4. The machine learning models of DIVA indexer10may receive pre-processed communication data as input, and output a set of four emotion factor values indicative of the emotive content of the communication data. The four emotion factor values may include a determination value, an inquisitiveness value, a valence value, and an aggression value. The set of emotion factor values may be stored in an emotion factor index database22along with a time stamp and associated with the communication data as well as the customer associated with the communication data. An emotion classification model220may use the current emotion factor value set as input, as well as one or more historic emotion factor value sets from the emotion factor index database22corresponding to historic communications associated with the customer over time, where the historic communications occurred prior to the current communication. Emotion classification model220may classify the current communication into an emotion state (e.g., “angry,” “scared,” “nervous,” “excited”) according to the emotive content of the current communication based on the current emotion factor value set and/or historic emotion factor value sets. Emotion classification system214may store the emotion state in emotion state database414and assign an ID to the emotion factor value sets in emotion factor index database22to associate the set or sets with the communication data and the emotion state.

Emotion state database414may contain a plurality of emotional states, where each of the plurality of emotion states is associated with one or more messages from one or more customers with similar emotive contents. The similar emotive contents may be represented by similar emotion factor values determined for the messages received from the customers. In some examples, an emotion state may be a string category. For example, the different states could be titled “frustrated,” “angry,” “curious,” or “cheerful.” In another example, an emotion state may be a numeric category (e.g., one through ten) representing the negativity or positivity of the message, where messages classified in category one have the most positive content and messages in category ten have the most negative content.

In some examples, emotion classification model220may transmit the emotion state to one or more agent devices224for use in determining how to handle the current communication. In some examples, emotion classification system214may transmit the emotion state to one or more agent devices from emotion state database414. AlthoughFIG.2depicts a flow of information to agent devices224only through emotion state database414, in some examples agent devices224may receive information directly from emotion classification model220, and in some examples, agent devices224may receive information directly from emotion state database414. In some examples, emotion classification model220or emotion classification system214may transmit the emotion state automatically from emotion classification model220or emotion state database414to agent devices224after a message is received and classified with the emotion state. In some examples, an agent operating one or more agent devices224may request the emotion state from emotion state database414when the agent is ready to begin work associated with the message. In some examples, agent devices224and/or emotion classification system may retrieve the emotion state for a customer message from emotion state database414when considering whether to solicit to the customer associated with the message, when determining what order in which to respond to customer messages, when determining whether a customer message may be fraudulent, when routing a customer message to a particular agent or agents capable of most effectively responding to the customer, when considering loan risks for the customer, or in any other scenario where the emotion state may assist in the decisions and processes of the business or organization.

By taking into account the emotive content of customer communications, emotion classification system214may provide more effective and informed services to customers of the business or organization.

FIG.3is a block diagram illustrating an example computing system300for running a DIVA indexer, in accordance with the techniques of this disclosure. The architecture of computing system300illustrated inFIG.3is shown for exemplary purposes only. Computing system300should not be limited to the illustrated example architecture. In other examples, computing system300may be configured in a variety of ways.

As shown in the example ofFIG.3, a computing system300includes one or more processors302, one or more interfaces304, and one or more storage units310. The one or more storage units310may house training data312, and an emotion factor index database22. The computing system300also includes the DIVA indexer10, and a training unit320, which may be implemented as program instructions and/or data stored in the storage units310and executable by the processors302. The DIVA indexer10may comprise a determination model12, an inquisitiveness model14, a valence model16, and an aggression model18.

Computing system300may be implemented as any suitable computing system, such as one or more server computers, workstations, mainframes, appliances, cloud computing systems, and/or other computing systems that may be capable of performing operations and/or functions described in accordance with one or more aspects of the present disclosure. In some examples, computing system300represents a cloud computing system, server farm, and/or server cluster (or portion thereof) that provides services to customer devices and other devices or systems. In other examples, computing system300may represent or be implemented through one or more virtualized compute instances (e.g., virtual machines, containers) of a data center, cloud computing system, server farm, and/or server cluster.

The storage units310of computing system300may also store an operating system (not shown) executable by the processors302to control the operation of components of the computing system300. The components, units, or modules of the computing system300are coupled (physically, communicatively, and/or operatively) using communication channels for inter-component communications. In some examples, the communication channels may include a system bus, a network connection, an inter-process communication data structure, or any other method for communicating data.

The processors302, in one example, may comprise one or more processors that are configured to implement functionality and/or process instructions for execution within the computing system300. For example, processors302may be capable of processing instructions stored by storage units310. Processors302may include, for example, microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate array (FPGAs), or equivalent discrete or integrated logic circuitry, or a combination of any of the foregoing devices or circuitry.

The computing system300may utilize interfaces304to communicate with external systems via one or more networks, e.g., a customer service center. Interfaces304may be network interfaces (such as Ethernet interfaces, optical transceivers, radio frequency (RF) transceivers, Wi-Fi or Bluetooth radios, or the like), telephony interfaces, or any other type of devices that can send and receive information. In some examples, the computing system300utilizes interfaces304to wirelessly communicate with external systems, e.g., other computing devices or systems within emotion classification system214ofFIG.2.

Storage units310may be configured to store information within the computing system300during operation. Storage units310may include a computer-readable storage medium or computer-readable storage device. In some examples, storage units310include one or more of a short-term memory or a long-term memory. Storage units310may include, for example, random access memories (RAM), dynamic random-access memories (DRAM), static random-access memories (SRAM), magnetic discs, optical discs, flash memories, or forms of electrically programmable memories (EPROM) or electrically erasable and programmable memories (EEPROM). In some examples, storage units310are used to store program instructions for execution by processors302. Storage units310may be used by software or applications running on the computing system300to temporarily store information during program execution.

Computing system300includes one or more machine learning models of DIVA indexer10and a training unit320used to train each of the machine learning models of DIVA indexer10using training data312. As seen inFIG.3, DIVA indexer10includes determination model12, inquisitiveness model14, valence model16, and aggression model18. The training unit320includes validation unit322and performance monitoring unit324.

Machine learning algorithms or functions (e.g., a word embedding algorithm) are trained to create the machine learning models within DIVA indexer10, configured to accept an input sequence of plain text data or text-based annotation data associated with a message and output, using determination model12, inquisitiveness model14, valence model16, and aggression model18, four emotion factor values including a determination value, an inquisitiveness value, a valence value, and an aggression value, where each value is an integer between −2 and 2 inclusive (although the integer range could consist of any range useful for the application) representing the intensity of the respective emotion contained within the message. For example, for each emotion factor value, −2 and −1 may be considered low values, while 1 and 2 may be considered high values. For example, a message could be scored with a determination value of negative one, a valence value of zero, an inquisitiveness value of two, and aggression value of two. The machine learning models within DIVA indexer10may generate emotion factor values based on text characteristics. For example, aggression model18may generate an aggression value of two for an incoming message if a set of text data associated with the incoming message has greater than a threshold level of similarity to known characteristics of messages with aggression values of two, as identified by aggression model18.

Processors302may be configured to train each train each machine learning model of the set of machine learning models to determine the measure of the particular emotion factor of the set of emotion factors based on the set of training data.

For example, determination model12may be trained to determine a determination value based on input communication data representing a message. A machine learning model may be trained using a training process to create a data-specific model, such as determination model12, based on training data312. After the training process, the created model may be capable of determining an output data set based on an input data set (e.g., generate a determination value representing a level of determination emotion in a message based on communication data). The training process may implement a set of training data (e.g., training data312) to create the model.

Determination model12may include functions configured to be executed by processors302. In some examples, determination model12implements supervised learning, e.g., classifies sets of data into groups. For example, a set of data, such as communication data indicative of a message to a financial institution, may be classified with a determination value of negative two, negative one, zero, one, or two. In some examples, the function may include nodes, layers, and connections, and the function may be represented by equations having a plurality of variables and a plurality of known coefficients.

For example, determination model12may receive communication data in the form of plain-text or text-based annotation data and may parse the communication data to identify a sequence of items including any one or combination of words, phrases, characters (e.g., punctuation), and numerical values corresponding to a determination emotion and an intensity of the determination emotion. Determination model12may output a determination value comprised of an integer between −2 and 2 inclusive, where a higher number represents a higher determination emotion in the message. For example, a determination value of negative two may represent a lowest determination emotion within the message, a determination value of two may represent a highest determination emotion in the message, and a determination value of zero may represent a neutral determination emotion in the message. DIVA indexer10may store the determination value in an emotion factor index database22and assign an ID to the determination value to associate it with the message and the other emotion factor values generated for the message.

Processors302may be configured to create a set of training data312that includes a plurality of customer communications (i.e., messages), wherein each customer communication comprises communication data and a set of labels indicating the set of emotion factor values for the customer communication.

For example, training data312may include data indicative of a plurality of messages. At least some of the plurality of messages may represent customer complaints, responses, transcripts of calls or letters submitted to emotion classification system214. The plurality of messages may include a group of messages with a determination value of negative 2, a group of messages with a determination value of negative one, a group of messages with a determination value of zero, a group of messages with a determination value of one, and a group of messages with a determination value of two, where each message of the plurality of messages is known to have a determination value of negative two, negative one, zero, one, or two. In one example, training data312contains data representing about equal numbers of messages with determination values of each number negative two through two. In another example, training data312contains data including a greater number of messages with a determination value of two than messages with a determination value of zero. In another example, training data312contains data including a greater number of messages with a determination value of zero than messages with a determination value of two. Other examples are contemplated wherein training data312contains data including a greater number of messages with any particular determination value than a number of messages with any other particular determination value. Training unit320may access training data312stored in storage units310, and training unit320may train determination model12using training data312.

Validation unit322may be configured to determine an accuracy of determination model12. For example, validation unit322may use determination model12to determine if an example message corresponding to a known determination value has a determination value of negative two, negative one, zero, one, or two. Validation unit322may determine if determination model12was able to correctly score the incoming message. Additionally, validation unit322may be configured to determine the accuracy of determination model12for a plurality of messages example each corresponding to a determination value, and validation unit322may be configured to identify an accuracy (e.g., a success rate) in which determination model12correctly scores the messages for each determination value. If the accuracy is above a threshold accuracy value, determination model12may be used to classify incoming messages to emotion classification system214. If the accuracy is below the threshold accuracy value, training unit320may re-train determination model12based on an updated set of training data. In some examples, the threshold accuracy value in which determination model12may be used may be greater than or equal to 90%. In some examples, validation unit322may be configured to identify an accuracy in which determination model12correctly scores the determination values of a plurality of messages.

Training unit320may include performance monitoring unit324. Performance monitoring unit324may monitor a performance of determination model12after it is applied to score incoming messages to emotion classification system214(e.g., score the four emotion factor values as integers between −2 and 2 inclusive). In some examples, performance monitoring unit324may determine an accuracy of determination model12by comparing determination values scored by determination model12with known determination values of a plurality of messages. For example, if determination model12determines that an incoming message has a determination value of negative two, and the incoming message is discovered to have a determination value of one, performance monitoring unit324may record that an incorrect determination value was generated. Performance monitoring unit324may continuously monitor an accuracy of determination model12. Performance monitoring unit324may determine a fraction of messages in which determination model12correctly scores an incoming message. The fraction may represent a measured accuracy of determination model12. New messages may be analyzed by performance monitoring unit324, the new messages representing data that was not used by training unit320to create determination model12. In other words, performance monitoring unit324may test the accuracy of determination model12continuously using new data. In some examples, if performance monitoring unit324determines that the accuracy of determination model12is below a threshold accuracy value (e.g., 90%), performance monitoring unit324may output an instruction to re-train determination model12.

Training unit320may periodically (e.g., monthly, bi-monthly, yearly, or the like) re-train determination model12based on an updated set of training data. The updated set of training data may include part or all of the plurality of messages of training data312. Additionally, the updated set of training data may include a set of messages that are received by emotion classification system214during a time since determination model12was last trained by training unit320.

Processors302may be configured to train each train each machine learning model of the set of machine learning models to determine the measure of the particular emotion factor of the set of emotion factors based on the set of training data.

For example, inquisitiveness model14may be trained to determine an inquisitiveness value based on input communication data representing a message. A machine learning model may be trained using a training process to create a data-specific model, such as inquisitiveness model14, based on training data312. After the training process, the created model may be capable of determining an output data set based on an input data set (e.g., generate an inquisitiveness value representing a level of inquisitive emotion in a message based on communication data). The training process may implement a set of training data (e.g., training data312) to create the model.

Inquisitiveness model14may include functions configured to be executed by processors302. In some examples, inquisitiveness model14implements supervised learning, e.g., classifies sets of data into groups. For example, a set of data, such as communication data indicative of a message to a financial institution, may be classified with an inquisitiveness value of negative two, negative one, zero, one, or two. In some examples, the function may include nodes, layers, and connections, and the function may be represented by equations having a plurality of variables and a plurality of known coefficients.

For example, inquisitiveness model14may receive communication data in the form of plain-text or text-based annotation data and may parse the communication data to identify a sequence of items including any one or combination of words, phrases, characters (e.g., punctuation), and numerical values corresponding to an inquisitive emotion and an intensity of the inquisitive emotion. Inquisitiveness model14may output an inquisitiveness value comprised of an integer between −2 and 2 inclusive, where a higher number represents a higher inquisitive emotion in the message. For example, an inquisitiveness value of negative two may represent a lowest inquisitive emotion within the message, an inquisitiveness value of two may represent a highest inquisitive emotion in the message, and an inquisitiveness value of zero may represent a neutral inquisitive emotion in the message. DIVA indexer10may store the inquisitiveness value in an emotion factor index database22and assign an ID to the inquisitiveness value to associate it with the message and the other emotion factor values generated for the message.

Processors302may be configured to create a set of training data312that includes a plurality of customer communications (i.e., messages), wherein each customer communication comprises communication data and a set of labels indicating the set of emotion factor values for the customer communication.

For example, training data312may include data indicative of a plurality of messages. At least some of the plurality of messages may represent customer complaints, responses, transcripts of calls or letters submitted to emotion classification system214. The plurality of messages may include a group of messages with an inquisitiveness value of negative 2, a group of messages with an inquisitiveness value of negative one, a group of messages with an inquisitiveness value of zero, a group of messages with an inquisitiveness value of one, and a group of messages with an inquisitiveness value of two, where each message of the plurality of messages is known to have an inquisitiveness value of negative two, negative one, zero, one, or two. In one example, training data312contains data representing about equal numbers of messages with inquisitiveness values of each number negative two through two. In another example, training data312contains data including a greater number of messages with an inquisitiveness value of two than messages with an inquisitiveness value of zero. In another example, training data312contains data including a greater number of messages with an inquisitiveness value of zero than messages with an inquisitiveness value of two. Other examples are contemplated wherein training data312contains data including a greater number of messages with any particular inquisitiveness value than a number of messages with any other particular inquisitiveness value. Training unit320may access training data312stored in storage units310, and training unit320may train inquisitiveness model14using training data312.

Validation unit322may be configured to determine an accuracy of inquisitiveness model14. For example, validation unit322may use inquisitiveness model14to determine if an example message corresponding to a known inquisitiveness value has an inquisitiveness value of negative two, negative one, zero, one, or two. Validation unit322may determine if inquisitiveness model14was able to correctly score the incoming message. Additionally, validation unit322may be configured to determine the accuracy of inquisitiveness model14for a plurality of messages example each corresponding to an inquisitiveness value, and validation unit322may be configured to identify an accuracy (e.g., a success rate) in which inquisitiveness model14correctly scores the messages for each inquisitiveness value. If the accuracy is above a threshold accuracy value, inquisitiveness model14may be used to classify incoming messages to emotion classification system214. If the accuracy is below the threshold accuracy value, training unit320may re-train inquisitiveness model14based on an updated set of training data. In some examples, the threshold accuracy value in which inquisitiveness model14may be used may be greater than or equal to 90%. In some examples, validation unit322may be configured to identify an accuracy in which inquisitiveness model14correctly scores the inquisitiveness values of a plurality of messages.

Training unit320may include performance monitoring unit324. Performance monitoring unit324may monitor a performance of inquisitiveness model14after it is applied to score incoming messages to emotion classification system214(e.g., score the four emotion factor values as integers between −2 and 2 inclusive). In some examples, performance monitoring unit324may determine an accuracy of inquisitiveness model14by comparing inquisitiveness values scored by inquisitiveness model14with known inquisitiveness values of a plurality of messages. For example, if inquisitiveness model14determines that an incoming message has an inquisitiveness value of negative two, and the incoming message is discovered to have an inquisitiveness value of one, performance monitoring unit324may record that an incorrect inquisitiveness value was generated. Performance monitoring unit324may continuously monitor an accuracy of inquisitiveness model14. Performance monitoring unit324may determine a fraction of messages in which inquisitiveness model14correctly scores an incoming message. The fraction may represent a measured accuracy of inquisitiveness model14. New messages may be analyzed by performance monitoring unit324, the new messages representing data that was not used by training unit320to create inquisitiveness model14. In other words, performance monitoring unit324may test the accuracy of inquisitiveness model14continuously using new data. In some examples, if performance monitoring unit324determines that the accuracy of inquisitiveness model14is below a threshold accuracy value (e.g., 90%), performance monitoring unit324may output an instruction to re-train inquisitiveness model14.

Training unit320may periodically (e.g., monthly, bi-monthly, yearly, or the like) re-train inquisitiveness model14based on an updated set of training data. The updated set of training data may include part or all of the plurality of messages of training data312. Additionally, the updated set of training data may include a set of messages that are received by emotion classification system214during a time since inquisitiveness model14was last trained by training unit320.

Processors302may be configured to train each train each machine learning model of the set of machine learning models to determine the measure of the particular emotion factor of the set of emotion factors based on the set of training data.

For example, valence model16may be trained to determine a valence value based on input communication data representing a message. A machine learning model may be trained using a training process to create a data-specific model, such as valence model16, based on training data312. After the training process, the created model may be capable of determining an output data set based on an input data set (e.g., generate a valence value representing a level of negative or positive emotion in a message based on communication data). The training process may implement a set of training data (e.g., training data312) to create the model.

Valence model16may include functions configured to be executed by processors302. In some examples, valence model16implements supervised learning, e.g., classifies sets of data into groups. For example, a set of data, such as communication data indicative of a message to a financial institution, may be classified with a valence value of negative two, negative one, zero, one, or two. In some examples, the function may include nodes, layers, and connections, and the function may be represented by equations having a plurality of variables and a plurality of known coefficients.

For example, valence model16may receive communication data in the form of plain-text or text-based annotation data and may parse the communication data to identify a sequence of items including any one or combination of words, phrases, characters (e.g., punctuation), and numerical values corresponding to a positive or negative emotion and an intensity of the emotion. Valence model16may output a valence value comprised of an integer between −2 and 2 inclusive, where a higher number represents a more positive emotion in the message. For example, a valence value of negative two may represent a very negative emotion within the message, a valence value of two may represent a very positive emotion in the message, and a valence value of zero may represent a neutral emotion in the message. DIVA indexer10may store the valence value in an emotion factor index database22and assign an ID to the valence value to associate it with the message and the other emotion factor values generated for the message.

Processors302may be configured to create a set of training data312that includes a plurality of customer communications (i.e., messages), wherein each customer communication comprises communication data and a set of labels indicating the set of emotion factor values for the customer communication.

For example, training data312may include data indicative of a plurality of messages. At least some of the plurality of messages may represent customer complaints, responses, transcripts of calls or letters submitted to emotion classification system214. The plurality of messages may include a group of messages with a valence value of negative 2, a group of messages with a valence value of negative one, a group of messages with a valence value of zero, a group of messages with a valence value of one, and a group of messages with a valence value of two, where each message of the plurality of messages is known to have a valence value of negative two, negative one, zero, one, or two. In one example, training data312contains data representing about equal numbers of messages with valence values of each number negative two through two. In another example, training data312contains data including a greater number of messages with a valence value of two than messages with a valence value of zero. In another example, training data312contains data including a greater number of messages with a valence value of zero than messages with a valence value of two. Other examples are contemplated wherein training data312contains data including a greater number of messages with any particular valence value than a number of messages with any other particular valence value. Training unit320may access training data312stored in storage units310, and training unit320may train valence model16using training data312.

Validation unit322may be configured to determine an accuracy of valence model16. For example, validation unit322may use valence model16to determine if an example message corresponding to a known valence value has a valence value of negative two, negative one, zero, one, or two. Validation unit322may determine if valence model16was able to correctly score the incoming message. Additionally, validation unit322may be configured to determine the accuracy of valence model16for a plurality of messages example each corresponding to a valence value, and validation unit322may be configured to identify an accuracy (e.g., a success rate) in which valence model16correctly scores the messages for each valence value. If the accuracy is above a threshold accuracy value, valence model16may be used to classify incoming messages to emotion classification system214. If the accuracy is below the threshold accuracy value, training unit320may re-train valence model16based on an updated set of training data. In some examples, the threshold accuracy value in which valence model16may be used may be greater than or equal to 90%. In some examples, validation unit322may be configured to identify an accuracy in which valence model16correctly scores the valence values of a plurality of messages.

Training unit320may include performance monitoring unit324. Performance monitoring unit324may monitor a performance of valence model16after it is applied to score incoming messages to emotion classification system214(e.g., score the four emotion factor values as integers between −2 and 2 inclusive). In some examples, performance monitoring unit324may determine an accuracy of valence model16by comparing valence values scored by valence model16with known valence values of a plurality of messages. For example, if valence model16determines that an incoming message has a valence value of negative two, and the incoming message is discovered to have a valence value of one, performance monitoring unit324may record that an incorrect valence value was generated. Performance monitoring unit324may continuously monitor an accuracy of valence model16. Performance monitoring unit324may determine a fraction of messages in which valence model16correctly scores an incoming message. The fraction may represent a measured accuracy of valence model16. New messages may be analyzed by performance monitoring unit324, the new messages representing data that was not used by training unit320to create valence model16. In other words, performance monitoring unit324may test the accuracy of valence model16continuously using new data. In some examples, if performance monitoring unit324determines that the accuracy of valence model16is below a threshold accuracy value (e.g., 90%), performance monitoring unit324may output an instruction to re-train valence model16.

Training unit320may periodically (e.g., monthly, bi-monthly, yearly, or the like) re-train valence model16based on an updated set of training data. The updated set of training data may include part or all of the plurality of messages of training data312. Additionally, the updated set of training data may include a set of messages that are received by emotion classification system214during a time since valence model16was last trained by training unit320.

Processors302may be configured to train each train each machine learning model of the set of machine learning models to determine the measure of the particular emotion factor of the set of emotion factors based on the set of training data.

For example, aggression model18may be trained to determine an aggression value based on input communication data representing a message. A machine learning model may be trained using a training process to create a data-specific model, such as aggression model18, based on training data312. After the training process, the created model may be capable of determining an output data set based on an input data set (e.g., generate an aggression value representing a level of aggressive emotion in a message based on communication data). The training process may implement a set of training data (e.g., training data312) to create the model.

Aggression model18may include functions configured to be executed by processors302. In some examples, aggression model18implements supervised learning, e.g., classifies sets of data into groups. For example, a set of data, such as communication data indicative of a message to a financial institution, may be classified with an aggression value of negative two, negative one, zero, one, or two. In some examples, the function may include nodes, layers, and connections, and the function may be represented by equations having a plurality of variables and a plurality of known coefficients.

For example, aggression model18may receive communication data in the form of plain-text or text-based annotation data and may parse the communication data to identify a sequence of items including any one or combination of words, phrases, characters (e.g., punctuation), and numerical values corresponding to an aggressive emotion and an intensity of the aggressive emotion. Aggression model18may output an aggression value comprised of an integer between −2 and 2 inclusive, where a higher number represents a higher aggressive emotion in the message. For example, an aggression value of negative two may represent a lowest aggressive emotion within the message, an aggression value of two may represent a highest aggressive emotion in the message, and an aggression value of zero may represent a neutral aggressive emotion in the message. DIVA indexer10may store the aggression value in an emotion factor index database22and assign an ID to the aggression value to associate it with the message and the other emotion factor values generated for the message.

Processors302may be configured to create a set of training data312that includes a plurality of customer communications (i.e., messages), wherein each customer communication comprises communication data and a set of labels indicating the set of emotion factor values for the customer communication.

For example, training data312may include data indicative of a plurality of messages. At least some of the plurality of messages may represent customer complaints, responses, transcripts of calls or letters submitted to emotion classification system214. The plurality of messages may include a group of messages with an aggression value of negative 2, a group of messages with an aggression value of negative one, a group of messages with an aggression value of zero, a group of messages with an aggression value of one, and a group of messages with an aggression value of two, where each message of the plurality of messages is known to have an aggression value of negative two, negative one, zero, one, or two. In one example, training data312contains data representing about equal numbers of messages with aggression values of each number negative two through two. In another example, training data312contains data including a greater number of messages with an aggression value of two than messages with an aggression value of zero. In another example, training data312contains data including a greater number of messages with an aggression value of zero than messages with an aggression value of two. Other examples are contemplated wherein training data312contains data including a greater number of messages with any particular aggression value than a number of messages with any other particular aggression value. Training unit320may access training data312stored in storage units310, and training unit320may train aggression model18using training data312.

Validation unit322may be configured to determine an accuracy of aggression model18. For example, validation unit322may use aggression model18to determine if an example message corresponding to a known aggression value has an aggression value of negative two, negative one, zero, one, or two. Validation unit322may determine if aggression model18was able to correctly score the incoming message. Additionally, validation unit322may be configured to determine the accuracy of aggression model18for a plurality of messages example each corresponding to an aggression value, and validation unit322may be configured to identify an accuracy (e.g., a success rate) in which aggression model18correctly scores the messages for each aggression value. If the accuracy is above a threshold accuracy value, aggression model18may be used to classify incoming messages to emotion classification system214. If the accuracy is below the threshold accuracy value, training unit320may re-train aggression model18based on an updated set of training data. In some examples, the threshold accuracy value in which aggression model18may be used may be greater than or equal to 90%. In some examples, validation unit322may be configured to identify an accuracy in which aggression model18correctly scores the aggression values of a plurality of messages.

Training unit320may include performance monitoring unit324. Performance monitoring unit324may monitor a performance of aggression model18after it is applied to score incoming messages to emotion classification system214(e.g., score the four emotion factor values as integers between −2 and 2 inclusive). In some examples, performance monitoring unit324may determine an accuracy of aggression model18by comparing aggression values scored by aggression model18with known aggression values of a plurality of messages. For example, if aggression model18determines that an incoming message has an aggression value of negative two, and the incoming message is discovered to have an aggression value of one, performance monitoring unit324may record that an incorrect aggression value was generated. Performance monitoring unit324may continuously monitor an accuracy of aggression model18. Performance monitoring unit324may determine a fraction of messages in which aggression model18correctly scores an incoming message. The fraction may represent a measured accuracy of aggression model18. New messages may be analyzed by performance monitoring unit324, the new messages representing data that was not used by training unit320to create aggression model18. In other words, performance monitoring unit324may test the accuracy of aggression model18continuously using new data. In some examples, if performance monitoring unit324determines that the accuracy of aggression model18is below a threshold accuracy value (e.g., 90%), performance monitoring unit324may output an instruction to re-train aggression model18.

Training unit320may periodically (e.g., monthly, bi-monthly, yearly, or the like) re-train aggression model18based on an updated set of training data. The updated set of training data may include part or all of the plurality of messages of training data312. Additionally, the updated set of training data may include a set of messages that are received by emotion classification system214during a time since aggression model18was last trained by training unit320.

In some examples, the business or organization may be a financial institution, and training data312may include data exclusively from a financial institution context. For example, the plurality of messages contained within training data312may all be messages to a financial institution. In this manner, training data312may be used to train the machine learning models of DIVA indexer10to be most effective in the financial institution context.

Computing system300may receive plain text data of communication data indicative of a current communication associated with a customer. Computing system300may then apply the communication data to an emotion-based indexer as input. For example, processors of computing system300may be configured to input the plain text data of the communication data to DIVA indexer10. The emotion-based indexer may include a set of machine learning models for a set of emotion factor values, wherein each machine learning model is trained to determine a measure of a particular emotion factor of the set of emotion factors. For example, DIVA indexer10may include determination model12trained to determine a determination value, inquisitiveness model14trained to determine an inquisitiveness value, valence model16trained to determine a valence value, and aggression model18trained to determine an aggression value. Computing system300may apply the communication data for the current communication as input to each of the four machine learning models of DIVA indexer10(i.e., determination model12, inquisitiveness model14, valence model16, and aggression model18). Computing system300may generate, as output from the emotion-based indexer, a set of emotion factor values for the current communication, wherein each emotion factor value of the set of emotion factor values indicates the measure of the particular emotion factor in the current communication. For example, DIVA indexer10may output the four emotion factor values, where each value indicates the measure of its respective emotion factor (i.e., an amount of determination, inquisitiveness, valence, and aggression). Processors302may be configured to indicate, as output from each machine learning model of DIVA indexer10, the corresponding emotion factor value for the current communication (i.e., the determination value, inquisitiveness value, valence value, and aggression value).

In addition, computing system300may store the emotion factor values in emotion factor index database22. Computing system300may then send the emotion factor values, e.g., using interfaces304, to another computing system executing an emotion classification model220configured to classify the customer communication into an emotion state based on the set of emotion factor values for the current communication associated with the customer.

FIG.4is a block diagram illustrating an example computing system400for running an emotion classification model220, in accordance with the techniques of this disclosure. The architecture of computing system400illustrated inFIG.4is shown for exemplary purposes only. Computing system400should not be limited to the illustrated example architecture. In other examples, computing system400may be configured in a variety of ways. Although computing system300and computing system400are illustrated herein as separate systems, in other examples DIVA indexer10and emotion classification model220may be run on a single, shared computing system.

Computing system400may be implemented as any suitable computing system, such as one or more server computers, workstations, mainframes, appliances, cloud computing systems, and/or other computing systems that may be capable of performing operations and/or functions described in accordance with one or more aspects of the present disclosure. In some examples, computing system400represents a cloud computing system, server farm, and/or server cluster (or portion thereof) that provides services to customer devices and other devices or systems. In other examples, computing system400may represent or be implemented through one or more virtualized compute instances (e.g., virtual machines, containers) of a data center, cloud computing system, server farm, and/or server cluster.

As shown in the example ofFIG.4, a computing system400includes one or more processors402, one or more interfaces404, and one or more storage units410. The one or more storage units410may store training data412, and/or emotion state database414. The computing system400also includes the emotion classification model220, and a training unit420, which may be implemented as program instructions and/or data stored in the storage units410and executable by the processors402.

The storage units410of the computing system400may also store an operating system (not shown) executable by the processors402to control the operation of components of the computing system400. The components, units, or modules of the computing system400are coupled (physically, communicatively, and/or operatively) using communication channels for inter-component communications. In some examples, the communication channels may include a system bus, a network connection, an inter-process communication data structure, or any other method for communicating data.

The processors402, in one example, may comprise one or more processors that are configured to implement functionality and/or process instructions for execution within the computing system400. For example, processors402may be capable of processing instructions stored by storage units410. Processors402may include, for example, microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate array (FPGAs), or equivalent discrete or integrated logic circuitry, or a combination of any of the foregoing devices or circuitry.

Computing system400may utilize interfaces404to communicate with external systems via one or more networks, e.g., a customer service center. Interfaces404may be network interfaces (such as Ethernet interfaces, optical transceivers, radio frequency (RF) transceivers, Wi-Fi or Bluetooth radios, or the like), telephony interfaces, or any other type of devices that can send and receive information. In some examples, the computing system400utilizes interfaces404to wirelessly communicate with external systems, e.g., other computing devices or systems within emotion classification system214ofFIG.2.

Storage units410may be configured to store information within the computing system400during operation. Storage units410may include a computer-readable storage medium or computer-readable storage device. In some examples, storage units410include one or more of a short-term memory or a long-term memory. Storage units410may include, for example, random access memories (RAM), dynamic random-access memories (DRAM), static random access memories (SRAM), magnetic discs, optical discs, flash memories, or forms of electrically programmable memories (EPROM) or electrically erasable and programmable memories (EEPROM). In some examples, storage units410are used to store program instructions for execution by processors402. Storage units410may be used by software or applications running on the computing system400to temporarily store information during program execution.

Computing system400includes a machine learning emotion classification model220with a classification unit406. Computing system400also includes a training unit420. As seen inFIG.4, training unit420includes validation unit422and performance monitoring unit424.

In some examples, computing system400may receive a set of emotion factor values for a current communication associated with a customer and apply the set of emotion factor values for the current communication to emotion classification model220as input. The set of emotion factor values may include a determination value, an inquisitiveness value, a valence value, and an aggression value. Computing system400may retrieve, e.g., using interfaces404, the emotion factor values associated with the current communication from emotion factor index database22. In some examples, emotion classification model220may be configured to classify the current communication into an emotion state based on the set of emotion factor values for the current communication associated with the customer. Emotion classification model220may receive one or more of the emotion factor values as input and output an emotion state. In some examples, the emotion state may be a text string (e.g., “frustrated,” “angry,” “curious,” “cheerful”) indicative of the emotive content of the communication. In some examples, the emotion state may be a number (e.g., between one and ten) representing the negativity or positivity of the message, where messages classified in category one have the most positive content and messages in category ten have the most negative content. The emotion state may be stored in an emotion state database414and associated with the current communication. Computing system400may send the emotion state as associated with the customer communication to one or more agent devices for use in determining how to handle the current communication. In this way, emotion classification model220may allow agents of the business or organization to handle customer communications more efficiently.

In some examples, emotion classification model220may be configured to classify a current communication into an emotion state based on a set of emotion factor values for the current communication associated with a customer, as well as one or more historic emotion factor value sets from the emotion factor index database22corresponding to historic communications associated with the customer over time, where the historic communications occurred prior to the current communication. Computing system400may retrieve, e.g., using interfaces404, a current set of emotion factor values associated with a current communication from emotion factor index database22. In some examples, computing system400may receive the current set of emotion factor values from DIVA indexer10. Computing system400may retrieve historic emotion factor values for historic communications associated with a customer over time from emotion factor index database22. Emotion classification model220may receive the one or more sets of emotion factor values as input and output an emotion state. The emotion state may be stored in an emotion state database414and associated with the current communication. In this way, emotion classification model220may take into account the emotive content of past customer communications for a given customer when classifying a current communication for the given customer into an emotion state.

Emotion classification model220may include functions configured to be executed by processors402. In some examples, emotion classification model220implements supervised learning, e.g., classifies sets of data into groups. For example, a set of data, such as a set of one or more emotion factor values indicative of the emotive content in a communication sent to a business or organization (e.g., a financial institution), may be classified with an emotion state. In some examples, the function may include nodes, layers, and connections, and the function may be represented by equations having a plurality of variables and a plurality of known coefficients.

Machine learning algorithms, such as some examples of emotion classification model220, may be trained using a training process to create data-specific models, such as emotion classification model220based on training data412. After the training process, the created model may be capable of determining an output data set based on an input data set (e.g., classify a communication into an emotion state based on a set of one or more emotion factor values). The training process may implement a set of training data (e.g., training data412) to create the model.

Training data412may include data indicative of a plurality of sets of emotion factor values. At least some of the plurality of sets of emotion factor values may represent the emotive content of customer communications submitted to computing system400.

In some examples training data312may include data exclusively from a financial institution context. For example, the plurality of communication data contained within training data312may all be communication data for messages or communications to a financial institution. In this way, the training of the machine learning models within DIVA indexer10may be more specific to financial institutions and more accurate in identifying emotive content in communications with a financial institution.

Processors402may be configured to create a set of training data that includes a plurality of communications, wherein each communication of the plurality of communications comprises a corresponding set of emotion factor values and a label identifying an associated emotion state.

In one example, the plurality of sets of emotion factor values may include a group of sets of emotion factor values labeled with a first emotion state, a group of sets of emotion factor values labeled with a second emotion state, and so on for each emotion state of a plurality of emotion states, where each group of sets of emotion factor values of the plurality of sets of emotion factor values is known to have a particular emotion state. In some examples, training data412contains data representing about equal numbers of sets of emotion factor values labeled with each emotion state of the plurality of emotion states. In another example, training data412contains data including a greater number of sets of emotion factor values labeled with a first emotion state than a number of sets of emotion factor values labeled with a second emotion state. Other examples are contemplated wherein training data412contains data including a greater number of sets of emotion factor values labeled with any particular emotion state than a number of sets of emotion factor values labeled with any other particular emotion state.

In some examples, a machine learning algorithm or function (e.g., a word embedding algorithm) is trained to create the emotion classification model220configured to accept an input set of emotion factor values associated with a current customer communication for a particular customer and output, using classification unit406, an emotion state classification for the customer communication indicative of the emotive content of the communication. For example, emotion classification model220may output classifications based on mapped patterns of emotion factor values. For example, emotion classification model220may classify a current communication as “cheerful” if the input set of emotion factor values associated with the current communication has a greater than threshold level of similarity to known characteristics of sets of emotion factor values classified as “cheerful”, as identified by classification unit406with reference to emotion state database414. Training unit420may output emotion states to storage units410.

Validation unit422may be configured to determine an accuracy of emotion classification model220. In some examples, validation unit422may use emotion classification model220to determine if example sets of emotion factor values for customer communications correspond to a known emotion state classification. Validation unit422may determine if emotion classification model220is able to correctly classify the set of emotion factor values. Additionally, validation unit422may be configured to determine the accuracy of emotion classification model220for a plurality of example emotion state classifications each corresponding to one or more sets of emotion factor values associated with customer communications, and validation unit422may be configured to identify an accuracy (e.g., a success rate) in which emotion classification model220correctly classifies the one or more sets of emotion factor values for each emotion state. If the accuracy is above a threshold accuracy value, emotion classification model220may be used to classify sets of emotion factor values output by DIVA indexer10. If the accuracy is below the threshold accuracy value, training unit420may re-train emotion classification model220based on an updated set of training data. In some examples, the threshold accuracy value in which emotion classification model220may be used may be greater than or equal to 90%. In some examples, validation unit422may be configured to identify an accuracy in which emotion classification model220correctly classifies a plurality of sets of emotion factor values into emotion states.

Training unit420may include performance monitoring unit424. Performance monitoring unit424may monitor a performance of emotion classification model220after it is applied to classify communications based on sets of emotion factor values.

In some examples, performance monitoring unit424may determine an accuracy of emotion classification model220by comparing emotion state classifications generated by emotion classification model220with known emotion state classifications of a plurality of sets of emotion factor values. For example, if emotion classification model220classifies an incoming set of emotion factor values into an emotion state of “cheerful,” and the set of emotion factor values is discovered to have a classification of “angry,” performance monitoring unit424may record that a communication was classified into an incorrect emotion state. Performance monitoring unit424may continuously monitor an accuracy of emotion classification model220. Performance monitoring unit424may determine a fraction of sets of emotion factor values in which emotion classification model220correctly classifies a communication into an emotion state. The fraction may represent a measured accuracy of the model. New sets of emotion factor values may be analyzed by performance monitoring unit424, the new sets of emotion factor values representing data that was not used by training unit420to create the model. In other words, performance monitoring unit424may test the accuracy of the model continuously using new data. In some examples, if performance monitoring unit424determines that the accuracy of emotion classification model220is below a threshold accuracy value (e.g., 90%), performance monitoring unit424may output an instruction to re-train emotion classification model220.

Training unit420may periodically (e.g., monthly, bi-monthly, yearly, or the like) re-train emotion classification model220based on an updated set of training data. The updated set of training data may include part or all of the plurality of sets of emotion factor values of training data412. Additionally, the updated set of training data may include a plurality of sets of emotion factor values for communications that are received by computing system400during a time since emotion classification model220was last trained by training unit420.

Emotion classification model220or computing system400may transmit the emotion state to one or more agent devices for use in determining how to handle the current communication. In some examples, emotion classification model220or computing system400may transmit the emotion state automatically from emotion classification model220or emotion state database414to one or more agent devices after receiving a customer communication and classifying the communication with the emotion state. In some examples, an agent operating the one or more agent devices may request the emotion state from emotion state database414when the agent is ready to begin work associated with the communication. In some examples, the one or more agent devices and/or computing system400may retrieve the emotion state for a customer communication from emotion state database414when considering whether to solicit to the customer associated with the message, when determining what order in which to respond to customer messages, when determining whether a customer message may be fraudulent, when routing a customer message to a particular agent or agents capable of most effectively responding to the customer, when considering loan risks for the customer, or in any other scenario where the emotion state may assist in the decisions and processes of the business or organization.

FIG.5is a flow diagram illustrating an example process for classifying a current communication into an emotion state, in accordance with the techniques of this disclosure. The example process ofFIG.5may be performed by emotion classification system214ofFIG.2, including DIVA indexer10, e.g., running on computing system300ofFIG.3, emotion classification model220, e.g., running on computing system400ofFIG.4, and/or processing circuitry of one or more agent device.

Emotion classification system214associated with an organization or business receives communication data representing a current customer communication from a customer that can be used to determine the emotive content of the communication (502). The customer may send the current communication from user device206to one or more servers or other computing devices of emotion classification system214. The current communication may be in the form of a text, call, letter, email, or other form of communication.

Once emotion classification system214receives the current communication, data pre-processor2may pre-process the current communication into communication data for further processing (504). The communication data may be in plain text format, where data pre-processor2digitally transcribes audio messages into plain text format, or an employee of the organization manually transcribes the audio message into plain text format. In some examples, data pre-processor2transcribes visual data (from scanned documents, pdf files, image files etc.) into plain text format, or an employee of the organization manually transcribes visual data into plain text format. Data pre-processor2may include a speech recognition model, e.g., a natural language processing (NLP) engine, configured to convert audio customer service inquiries to plain text data via natural language processing. In other examples, data pre-processor2may include a text image recognition model configured to convert hand- or typewritten customer service inquiries to plain text data or text-based annotation data.

Processors of computing system300may apply the processed communication data as input to DIVA indexer10(506). DIVA indexer10includes multiple machine learning models, including a determination model12, inquisitiveness model14, valence model16, and aggression model18. Applying the communication data as input to DIVA indexer10may include applying the communication data as input to each of the machine learning models of DIVA indexer10. The machine learning models within DIVA indexer10may accept plain text or text-based annotation data as input, where the communication data received is associated with a single communication submitted by the customer. Each machine learning model of DIVA indexer10may be trained to determine a measure of a particular emotion factor of the set of emotion factors.

DIVA indexer10generates, as output, a current set of emotion factor values, where each of the machine learning models within DIVA indexer10is configured to generate a single emotion factor value as output (508). Each emotion factor value of the set of emotion factor values may indicate the measure of the particular emotion factor in the current communication. For example, each of the emotion factor values may be indicative of an emotive intensity present in the communication data. For example, determination model12may generate a determination value comprising an integer between −2 and 2 (inclusive). A determination value of −2 may indicate that the customer communication conveys a low determination, where the customer may feel undecided on an issue. A determination value of 2 may indicate that the customer communication conveys a high determination value, where the customer may feel fixated on an issue. Similarly, inquisitiveness model14may generate an inquisitiveness value, valence model16may generate a valence value, and aggression model18may generate an aggression value, where each emotion factor value may be an integer between −2 and 2 (inclusive), representing the intensity of the respective emotion as conveyed in the customer communication. The four emotion factor values may be saved in an emotion factor index database22as associated with the communication data and the customer who sent the current communication.

Processors of computing system400may apply the set or sets of emotion factor values for the current communication to emotion classification model220as input (510). The processors may retrieve the one or more sets of emotion factor values associated with the customer from the emotion factor index database22, or may receive the one or more sets of emotion factor values from DIVA indexer10. Emotion classification model220may be a machine learning model or a business rule-based model. In some examples, emotion classification model retrieves the current set of emotion factor values, as well as historic sets of emotion factor values from emotion factor index database22.

Emotion classification model220then classifies the current communication into an emotion state based on the one or more sets of emotion factor values associated with the customer (512). In some examples emotion classification model220is a machine learning model, trained as described below with respect toFIG.7to classify a communication into an emotion state indicative of the emotive content of the communication. In some examples, the emotion state may be a text string (e.g., “frustrated,” “angry,” “curious,” “cheerful”) indicative of the emotive content of the communication. In some examples, the emotion state may be a number (e.g., between one and ten) representing the negativity or positivity of the message, where messages classified in category one have the most positive content and messages in category ten have the most negative content.

Emotion classification model220or emotion classification system214may store the emotion state in an emotion state database414as associated with the communication (514). Emotion classification system214retrieve the emotion state as associated with the customer communication and send the emotion state to one or more agent devices for use in determining how to handle the communication. In this way, emotion classification model220may allow agents of the business or organization to handle customer communications more efficiently. In some examples, an agent operating one or more agent devices224may request the emotion state from emotion state database414when the agent is ready to begin work associated with the message. In some examples, agent devices224and/or emotion classification system may retrieve the emotion state for a customer message from emotion state database414when considering whether to solicit to the customer associated with the message, when determining what order in which to respond to customer messages, when determining whether a customer message may be fraudulent, when routing a customer message to a particular agent or agents capable of most effectively responding to the customer, when considering loan risks for the customer, or in any other scenario where the emotion state may assist in the decisions and processes of the business or organization.

FIG.6is a flow diagram illustrating an example process for training an emotion-based indexer machine learning model, in accordance with the techniques of this disclosure. The example operation ofFIG.6is described with respect to computing system300ofFIG.3including DIVA indexer10and training unit320. The emotion-based indexer may comprise determination model12, inquisitiveness model14, valence model16, and aggression model18within DIVA indexer10. Each model within DIVA indexer10may need to be trained individually with communication data labeled with their respective emotion factor values.

Training unit320may receive a set of training data312including data indicative of a first set of communication data associated with a first set of emotion factor values and at least a second set of communication data associated with a second set of emotion factor values (602). Computing system300may create the set of training data including a plurality of customer communications in storage units310, and a plurality of sets of emotion factor values wherein each customer communication includes communication data and a set of labels indicating the values for a corresponding set of emotion factor values for the customer communication. The sets of emotion factor values may each be comprised of a determination value, an inquisitiveness value, a valence value, and an aggression value, where each emotion factor value is comprised of an integer between negative two and two inclusive. In some examples, the first set of emotion factor values has different integer values than the second set of emotion factor values. In some examples, the set of communication data associated with the first set of emotion factor values may be approximately equal in size to the set of communication data associated with the second set of emotion factor values.

Training unit320may train the machine learning models within DIVA indexer10using training data312(604). Training unit320may train each machine learning model of the set of machine learning models to determine the measure of the particular emotion factor of the set of emotion factors based on the set of training data. Determination model12may be trained to determine a determination value based on input communication data representing a message. A machine learning model may be trained using a training process to create a data-specific model, such as determination model12, based on training data312. After the training process, determination model12may be capable of outputting a determination value representing a level of determination emotion in a message based on an input of communication data. The training process may implement a set of training data (e.g., training data312) to create determination model12. Computing system300may create the set of training data from a plurality of customer communications in storage units310, and a plurality of determination values in emotion factor index database22wherein each customer communication includes communication data and a label indicating the corresponding determination value.

Training data312may include a plurality of sets of communication data and emotion factor values as described above, wherein the determination values from the sets of emotion factor values are used to train determination model12. The plurality of sets of communication data may include a group of communication data labeled with a determination value of negative 2, a group of communication data labeled with a determination value of negative one, a group of communication data labeled with a determination value of zero, a group of communication data labeled with a determination value of one, and a group of communication data labeled with a determination value of two, where each group of communication data of the plurality of sets of communication data is known to be labeled with a determination value of negative two, negative one, zero, one, or two. In one example, training data312contains data representing about equal numbers of sets of communication data labeled with determination values of each number negative two through two. In another example, training data312contains data including a greater number of sets of communication data labeled with a determination value of two than sets of communication data labeled with a determination value of zero. Other examples are contemplated wherein training data312contains data including an equal to or greater number of sets of communication data labeled with any particular determination value than a number of sets of communication data labeled with any other particular determination value. Training unit320may access training data312stored in storage units310, and training unit320may train determination model12using training data312.

Inquisitiveness model14may be trained to determine an inquisitiveness value based on input communication data representing a message. A machine learning model may be trained using a training process to create a data-specific model, such as inquisitiveness model14, based on training data312. After the training process, inquisitiveness model14may be capable of outputting an inquisitiveness value representing a level of inquisitiveness emotion in a message based on an input of communication data. The training process may implement a set of training data (e.g., training data312) to create inquisitiveness model14. Computing system300may create the set of training data from a plurality of customer communications in storage units310, and a plurality of inquisitiveness values in emotion factor index database22wherein each customer communication includes communication data and a label indicating the corresponding inquisitiveness value.

Training data312may include a plurality of sets of communication data and emotion factor values as described above, wherein the inquisitiveness values from the sets of emotion factor values are used to train inquisitiveness model14. The plurality of sets of communication data may include a group of communication data labeled with an inquisitiveness value of negative 2, a group of communication data labeled with an inquisitiveness value of negative one, a group of communication data labeled with an inquisitiveness value of zero, a group of communication data labeled with an inquisitiveness value of one, and a group of communication data labeled with an inquisitiveness value of two, where each group of communication data of the plurality of sets of communication data is known to be labeled with an inquisitiveness value of negative two, negative one, zero, one, or two. In one example, training data312contains data representing about equal numbers of sets of communication data labeled with inquisitiveness values of each number negative two through two. In another example, training data312contains data including a greater number of sets of communication data labeled with an inquisitiveness value of two than sets of communication data labeled with an inquisitiveness value of zero. Other examples are contemplated wherein training data312contains data including an equal to or greater number of sets of communication data labeled with any particular inquisitiveness value than a number of sets of communication data labeled with any other particular inquisitiveness value. Training unit320may access training data312stored in storage units310, and training unit320may train inquisitiveness model14using training data312.

Valence model16may be trained to determine a valence value based on input communication data representing a message. A machine learning model may be trained using a training process to create a data-specific model, such as valence model16, based on training data312. After the training process, valence model16may be capable of outputting a valence value representing a level of valence emotion in a message based on an input of communication data. The training process may implement a set of training data (e.g., training data312) to create valence model16. Computing system300may create the set of training data from a plurality of customer communications in storage units310, and a plurality of valence values in emotion factor index database22wherein each customer communication includes communication data and a label indicating the corresponding valence value.

Training data312may include a plurality of sets of communication data and emotion factor values as described above, wherein the valence values from the sets of emotion factor values are used to train valence model16. The plurality of sets of communication data may include a group of communication data labeled with a valence value of negative 2, a group of communication data labeled with a valence value of negative one, a group of communication data labeled with a valence value of zero, a group of communication data labeled with a valence value of one, and a group of communication data labeled with a valence value of two, where each group of communication data of the plurality of sets of communication data is known to be labeled with a valence value of negative two, negative one, zero, one, or two. In one example, training data312contains data representing about equal numbers of sets of communication data labeled with valence values of each number negative two through two. In another example, training data312contains data including a greater number of sets of communication data labeled with a valence value of two than sets of communication data labeled with a valence value of zero. Other examples are contemplated wherein training data312contains data including an equal to or greater number of sets of communication data labeled with any particular valence value than a number of sets of communication data labeled with any other particular valence value. Training unit320may access training data312stored in storage units310, and training unit320may train valence model16using training data312.

Aggression model18may be trained to determine an aggression value based on input communication data representing a message. A machine learning model may be trained using a training process to create a data-specific model, such as aggression model18, based on training data312. After the training process, aggression model18may be capable of outputting an aggression value representing a level of aggression emotion in a message based on an input of communication data. The training process may implement a set of training data (e.g., training data312) to create aggression model18. Computing system300may create the set of training data from a plurality of customer communications in storage units310, and a plurality of aggression values in emotion factor index database22wherein each customer communication includes communication data and a label indicating the corresponding aggression value.

Training data312may include a plurality of sets of communication data and emotion factor values as described above, wherein the aggression values from the sets of emotion factor values are used to train aggression model18. The plurality of sets of communication data may include a group of communication data labeled with an aggression value of negative 2, a group of communication data labeled with an aggression value of negative one, a group of communication data labeled with an aggression value of zero, a group of communication data labeled with an aggression value of one, and a group of communication data labeled with an aggression value of two, where each group of communication data of the plurality of sets of communication data is known to be labeled with an aggression value of negative two, negative one, zero, one, or two. In one example, training data312contains data representing about equal numbers of sets of communication data labeled with aggression values of each number negative two through two. In another example, training data312contains data including a greater number of sets of communication data labeled with an aggression value of two than sets of communication data labeled with an aggression value of zero. Other examples are contemplated wherein training data312contains data including an equal to or greater number of sets of communication data labeled with any particular aggression value than a number of sets of communication data labeled with any other particular aggression value. Training unit320may access training data312stored in storage units310, and training unit320may train aggression model18using training data312.

By training the machine learning models within DIVA indexer10, training unit320may generate an emotion factor index database22(606). The emotion factor index database22may include a plurality of emotion factor value sets, where each emotion factor value set of the plurality of emotion factor value sets corresponds to a respective message or communication data. Communication data may include words of the English language or other languages, single numerals, groups of single numerals, numerical strings, groups of numerical strings, single characters, groups of single characters, character strings, or groups of character strings in plain text format. As such, using emotion factor index database22, the machine learning models within DIVA indexer10may determine a set of emotion factor values for a message or communication data. Training unit320may store emotion factor index database22in storage units310(608).

FIG.7is a flow diagram illustrating an example process for training an emotion classification model220, in accordance with the techniques of this disclosure. The example operation ofFIG.7is described with respect to computing system400ofFIG.4including the machine learning model, emotion classification model220, and training unit420.

The emotion classification model220machine learning model may be trained to classify a communication into an emotion state indicative of the emotive content of the communication based on a set of emotion factor values for the communication data associated with the communication.

Training unit420may receive a set of training data412including a group of sets of emotion factor values associated with customer communications and labeled with a first emotion state, and a group of sets of emotion factor values associated with customer communications and labeled with a second emotion state (702). In some examples, the group of sets of emotion factor values labeled with the first emotion state may be approximately equal in size to the group of sets of emotion factor values labeled with the second emotion state. Computing system400may create the set of training data412from a plurality of sets of emotion factor values in emotion factor index database22and a plurality of emotion states in emotion state database414, wherein each set of emotion factor values of the plurality of sets of emotion factor values includes a label indicating the corresponding emotion state.

Training unit420may train emotion classification model220using training data412(704). In a first example, emotion classification model220may be trained to classify a communication into an emotion state based on an input set of emotion factor values associated with the communication. In a second example, emotion classification model220may be trained to classify a communication into an emotion state based on an input set of emotion factor values associated with the communication and one or more input sets of historical emotion factor values associated with historical communications of the customer associated with the communication. A machine learning algorithm may be trained using a training process to create a data-specific model, such as emotion classification model220based on training data412. After the training process, emotion classification model220may be capable of classifying a communication into an emotion state based on a set of emotion factor values in a first example, or emotion classification model220may be capable of classifying a communication into an emotion state based on a set of emotion factor values and one or more sets of historic emotion factor values in a second example. The training process may implement a set of training data (e.g., training data412) to create the emotion classification model220.

Training data412may include data indicative of a plurality of sets of emotion factor values labeled with a plurality of emotion states, wherein the plurality of sets of emotion factor values labeled with a plurality of emotion states comprises a first set of emotion factor values labeled with a first emotion state and at least a second set of emotion factor values labeled with a second emotion state. The plurality of sets of emotion factor values may include a particular number of groups (e.g., ten groups) of sets of emotion factor values where each of the groups includes data that is labeled with a particular emotion state. In one example, training data412contains data representing about equal numbers of sets of emotion factor values labeled with each emotion state. In another example, training data412contains data including a greater number of sets of emotion factor values labeled with a first emotion state than a number of sets of emotion factor values labeled with a second emotion state. Other examples are contemplated wherein training data412contains data including a greater number of sets of emotion factor values labeled with any particular emotion state than a number of sets of emotion factor values labeled with any other particular emotion state. Training unit420may access training data412stored in storage units410, and training unit420may train the emotion classification model220using training data412.

By training emotion classification model220, training unit420may generate an emotion state database414(706). The emotion state database414may include a plurality of emotion states, where each emotion state of the plurality of emotion states corresponds to one or more sets of emotion factor values associated with customer communications. Each emotion state of the plurality of emotion states is associated with an emotive content of a communication. Using emotion state database414, emotion classification model220may classify a communication into an emotion state indicative of the emotive content of the communication.

Training unit420may store emotion state database414in storage units410(708). Computing system400may retrieve an emotion state associated with a customer communication and send the emotion state to one or more agent devices for use in determining how to handle the communication. In this way, emotion classification model220may allow agents of the business or organization to handle customer communications more efficiently. In some examples, an agent operating one or more agent devices may request the emotion state for a customer communication from emotion state database414when the agent is ready to begin work associated with the communication. In some examples, agent devices and/or computing system400may retrieve the emotion state for a customer message from emotion state database414when considering whether to solicit to the customer associated with the message, when determining what order in which to respond to customer messages, when determining whether a customer message may be fraudulent, when routing a customer message to a particular agent or agents capable of most effectively responding to the customer, when considering loan risks for the customer, or in any other scenario where the emotion state may assist in the decisions and processes of the business or organization.

In one or more examples, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored on or transmitted over a computer-readable medium as one or more instructions or code and executed by a hardware-based processing unit. Computer-readable media may include computer-readable storage media, which corresponds to a tangible medium such as data storage media, or communication media including any medium that facilitates transfer of a computer program from one place to another, e.g., according to a communication protocol. In this manner, computer-readable media generally may correspond to (1) tangible computer-readable storage media which is non-transitory or (2) a communication medium such as a signal or carrier wave. Data storage media may be any available media that can be accessed by one or more computers or one or more processors to retrieve instructions, code and/or data structures for implementation of the techniques described in this disclosure. A computer program product may include a computer-readable medium.

By way of example, and not limitation, such computer-readable storage media can include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage, or other magnetic storage devices, flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer. Also, any connection is properly termed a computer-readable medium. For example, if instructions are transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of medium. It should be understood, however, that computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other transitory media, but are instead directed to non-transitory, tangible storage media. Disk and disc, as used herein, includes compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk and Blu-ray disc, where disks usually reproduce data magnetically, while discs reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media.

Instructions may be executed by one or more processors, such as one or more DSPs, general purpose microprocessors, ASICs, FPGAs, or other equivalent integrated or discrete logic circuitry, as well as any combination of such components. Accordingly, the term “processor,” as used herein may refer to any of the foregoing structures or any other structure suitable for implementation of the techniques described herein. In addition, in some aspects, the functionality described herein may be provided within dedicated hardware and/or software modules. Also, the techniques could be fully implemented in one or more circuits or logic elements.

The techniques of this disclosure may be implemented in a wide variety of devices or apparatuses, including a wireless communication device or wireless handset, a microprocessor, an integrated circuit (IC) or a set of ICs (e.g., a chip set). Various components, modules, or units are described in this disclosure to emphasize functional aspects of devices configured to perform the disclosed techniques, but do not necessarily require realization by different hardware units. Rather, as described above, various units may be combined in a hardware unit or provided by a collection of interoperative hardware units, including one or more processors as described above, in conjunction with suitable software and/or firmware.

Various examples have been described. These and other examples are within the scope of the following claims.