Multi-channel customer sentiment determination system and graphical user interface

Techniques described herein are directed to a sentiment analysis engine for classifying and quantifying customer sentiments between a customer and an agent. The sentiment analysis engine includes a data acquisition engine configured to receive data from a plurality of different communication channels. A preprocessing engine of the sentiment analysis engine is configured to apply rule(s) that are configured to perform formatting and/or filtering operations on the received data and/or isolate customer-provided messages from agent-provided messages. A customization engine of the sentiment analysis engine is configured to perform channel-specific customizations on the received data, such as, but not limited to interpreting the data in a channel-specific manner A batch scoring engine of the sentiment analysis engine is configured to quantify customer sentiments in real-time (e.g., during a communication session between the customer and agent), and such sentiments may be displayed via a GUI utilized by an authorized individual.

BACKGROUND

Support centers exist to resolve issues that prevent customers from using their products, and to educate customers on how to better use their products. In other words, support centers exist to safeguard and increase customer loyalty. Conventional sentiment analysis solutions are trained from whole documents, e.g. news articles, product/movie/restaurant reviews or forum posts, or short text fragments such as tweets. However, such techniques cannot gauge customer sentiment in a communication sessional platform (such as phone calls, agent chats, or artificial intelligence-based bot chats, etc.) used to communicate with support centers.

SUMMARY

Techniques described herein are directed to a sentiment analysis engine for classifying and quantifying customer sentiments between a customer and one or more agents (e.g., customer service representatives, technical support specialists, etc.). The sentiment analysis engine may comprise a data acquisition engine that is configured to receive data from a plurality of different communication channels (e.g., a phone call, an Internet chat session, an email, a social media posting etc.). A preprocessing engine of the sentiment analysis engine may be configured to apply one or more rules that are configured to perform formatting and/or filtering operations on the received data and/or isolate customer-provided messages from agent-provided messages. A customization engine of the sentiment analysis engine may be configured to perform channel-specific customizations on the received data, such as, but not limited to interpreting the data in a channel-specific manner A batch scoring engine of the sentiment analysis engine may be configured or modified to quantify customer sentiments in real-time (e.g., during a communication session between the customer and agent), and such sentiments may be displayed via a graphical user interface (GUI) utilized by the agent or an authorized individual (e.g., a manager or supervisor of the agent).

Further features and advantages of the disclosed embodiments, as well as the structure and operation of various embodiments disclosed herein, are described in detail below with reference to the accompanying drawings. It is noted that the disclosed embodiments are not limited to the specific embodiments described herein. Such embodiments are presented herein for illustrative purposes only. Additional embodiments will be apparent to persons skilled in the relevant art(s) based on the teachings contained herein.

DETAILED DESCRIPTION

The present specification and accompanying drawings disclose one or more embodiments that incorporate the features disclosed herein. The scope of the present embodiments is not limited to the description provided herein. The features disclosed herein merely exemplify the disclosed embodiments, and modified versions of the features disclosed herein are also encompassed by the present embodiments. The embodiments described herein are defined by the claims appended hereto.

II. Example Embodiments for Sentiment Analysis and Determination

A. Support Sentiment Analysis Engine

Industry research shows that the best way to predict customer loyalty, and thus measure the effectiveness of the offerings of a support center, is to measure the way that customers feel, i.e., through sentiment and emotion detection. Sentiment refers to a view of or attitude toward a support-related situation (e.g., a customer problem/product satisfaction, etc.) or event (e.g., support experience), an opinion that can be used as a filter to identify customer pain (e.g., “I'm disappointed in Company A support”, “You are giving me stupid answers”, etc.).

Techniques described herein are directed to a sentiment analysis engine for classifying and quantifying customer sentiments between a customer and an agent (e.g., a customer service representative, technical support, etc.). The sentiment analysis engine may comprise a data acquisition engine that is configured to receive data from a plurality of different communication channels (e.g., a phone call, an Internet chat session, emails, etc.). A preprocessing engine of the sentiment analysis engine may be configured to apply one or more rules that are configured to perform formatting and/or filtering operations on the received data and/or isolate customer-provided messages from agent-provided messages. A customization engine of the sentiment analysis engine may be configured to perform channel-specific customizations on the received data, such as, but not limited to interpreting the data in a channel-specific manner A batch scoring engine of the sentiment analysis engine may be configured to quantify customer sentiments in real-time (e.g., during a communication session between the customer and agent), and such sentiments may be displayed via a GUI utilized by an authorized individual (e.g., a manager or supervisor of the agent) or the agents themselves.

Certain components of the sentiment analysis engine (e.g., the data acquisition engine) may be utilized across the plurality of different communication channels, thereby advantageously reducing the processing overhead and/or storage required to perform the techniques described herein. Such a reduction in computing resources also results in a smaller consumption of power on which such techniques are executed.

The intensity of the customer's sentiment and the direction in which the sentiment is trending may be also determined and displayed in real-time via the GUI. The GUI may also display quantified customer sentiments for a plurality of different communication sessions between different agents and customers. In particular, the GUI simultaneously displays a plurality of different display boxes, where each box displays information pertaining to communications session(s) between an agent and customer(s) and customer sentiment and/or sentiment trend for each of the customer(s). This improved GUI structure advantageously enhances the ability for a manager/supervisor to quickly assess the performance of a plurality of agents, thereby enabling targeted real-time corrective supervisor intervention.

In addition, it has been observed that how a communication session ends is very predictive of overall customer satisfaction. Accordingly, embodiments described herein analyze the last N statements (e.g., the last 6 statements) of a customer to determine the customer's sentiment of the issue being discussed with the agent. By only analyzing a limited subset of messages of a communication session (as opposed to analyzing all the messages of a communication session), the amount of computing resources (e.g., processing, memory, etc.) required to determine the customer's sentiment is significantly reduced.

The techniques described herein also advantageously allow a company (e.g., that maintains the support center) to tap into how customers feel at scale and highlight the strengths and opportunities related to how the business supports its products.

For example,FIG. 1shows a block diagram of an example network-based computing system100configured for determine sentiment, according to an example embodiment. As shown inFIG. 1, system100includes a plurality of clusters102A,102B and102N. A computing device104is communicatively coupled with system100via a network116. Furthermore, each of clusters102A,102B and102N are communicatively coupled to each other via network116, as well as being communicatively coupled with computing device104through network116. Network116may comprise one or more networks such as local area networks (LANs), wide area networks (WANs), enterprise networks, the Internet, etc., and may include one or more of wired and/or wireless portions.

Clusters102A,102B and102N may form a network-accessible server set. Each of clusters102A,102B and102N may comprise a group of one or more nodes and/or a group of one or more storage nodes. For example, as shown inFIG. 1, cluster102A includes nodes108A-108N and one or more storage nodes110, cluster102B includes nodes112A-112N, and cluster102N includes nodes114A-114N. Each of nodes108A-108N,112A-112N and/or114A-114N are accessible via network116(e.g., in a “cloud-based” embodiment) to build, deploy, and manage applications and services. Each of storage node(s)110comprises a plurality of physical storage disks122that is accessible via network116and is configured to store data associated with the applications and services managed by nodes108A-108N,112A-112N, and/or114A-114N.

In an embodiment, one or more of clusters102A,102B and102N may be co-located (e.g., housed in one or more nearby buildings with associated components such as backup power supplies, redundant data communications, environmental controls, etc.) to form a datacenter, or may be arranged in other manners. Accordingly, in an embodiment, one or more of clusters102A,102B and102N may be a datacenter in a distributed collection of datacenters.

Each of node(s)108A-108N,112A-112N and114A-114N may be configured to execute one or more software applications (or “applications”) and/or manage hardware resources (e.g., processors, memory, etc.), which may be utilized by users (e.g., customers) of the network-accessible server set. Node(s)108A-108N,112A-112N and114A-114N may also be configured for specific uses. For example, as shown inFIG. 1, node108A is configured to execute a sentiment analysis engine118. It is noted that cluster102B and/or cluster102N may also include storage node(s)110.

Sentiment analysis engine118may be configured to determine a customer's sentiment towards a particular issue (e.g., a customer problem/product satisfaction, etc.) or event (e.g., support experience), or an opinion that can be used as a filter to identify customer pain (e.g., “I'm disappointed in Company A support”, “You are giving me stupid answers”, etc.). The customer's sentiment may be determined by analyzing customer messages. The messages may comprise customer content received via one or more data sources (also referred to as communication channels). Examples of data sources include, but are not limited to, a communication session between a customer and an agent (e.g., a telephone (i.e., a voice) call between the customer and the agent, an Internet chat session between the customer and the agent, an email thread between the customer and the agent, etc.). Sentiment analysis engine118may be further configured to determine the customer sentiment with respect to a particular issue based on other data sources, such as, but not limited to, the messages provided by the customer on the customer's social media feed, verbatim responses provided by the customer (e.g., rankings, ratings, reviews, comments, etc. for example, provided via a survey filled out by the customer, etc.).

Based on the content of the messages, sentiment analysis engine118may generate a sentiment classification and/or score for each message and/or for the communication session as a whole. Sentiment analysis engine118may be also configured to determine a trend in the user's sentiment (e.g., whether the user's sentiment is trending positively or negatively). Sentiment analysis engine118may be further configured to determine when the customer's sentiment shifts (e.g., from positive to negative, or negative to positive), and the driver of that shift in sentiment (e.g., a particular statement made by the agent, a particular issue experienced by the user, etc.). The sentiment classification, score, sentiment trend, sentiment shift and/or drivers thereof may be determined in real-time (e.g., during the communication session between the customer and the agent) and/or presented via a GUI provided by sentiment analysis engine118. For example, the GUI may display one or more symbols to represent the sentiment classification and/or score (e.g., a numerical value, an up arrow for a relatively high score, a down arrow for a relatively low score, etc.) and/or an intensity of the score. For example, the symbol may be color-coded based on how negative or positive the sentiment is (e.g., red for negative, and green for positive). The GUI may also display a symbol that represents the trend in the customer's sentiment (e.g., an arrow pointing in a direction in which the sentiment is trending). The GUI may be utilized by a manager of the agent to assess how well the call is going. The GUI may further simultaneously display the sentiment classification, score (and intensity thereof), and/or sentiment trend for a plurality of different communication sessions between different agents and customers.

A user (e.g., a manager) may be enabled to view and interact with the GUI via computing device104. As shown inFIG. 1, computing device104includes a display screen124and an application126. A user may access the GUI provided by sentiment analysis engine118by interacting with an application capable of accessing sentiment analysis engine118. For example, in an embodiment in which application126is a browser application, the user may use application126to traverse a network address (e.g., a uniform resource locator) to a web page associated with sentiment analysis engine118, which invokes a user interface128(e.g., a web page) in a browser window rendered on computing device104. Computing device104may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a Microsoft® Surface® device, a laptop computer, a notebook computer, a tablet computer such as an Apple iPad™, a netbook, a mobile phone device, such as an Apple iPhone™ or Google Pixel™, etc.), a wearable computing device (e.g., a head-mounted device including smart glasses such as Google® Glass™, etc.), or a stationary computing device such as a desktop computer or PC (personal computer).

FIG. 2shows a block diagram of an example sentiment analysis engine200, according to an example embodiment. Sentiment analysis engine200is an example of sentiment analysis engine118, as described above with reference toFIG. 1. As shown inFIG. 2, sentiment analysis engine200comprises a data acquisition engine216, a preprocessing engine222, a customization engine224, a sentiment driver engine238, application programming interface240, a batch soring engine242, and a governance and continuous improvement engine244. Data acquisition engine216may be configured to capture support-specific data from a plurality of different data sources202(or communication channels).

Data sources202may comprise agent chat sessions204, virtual agent chat sessions206, verbatim responses208, social media feeds210, emails212, and/or voice data214. Agent chat sessions204may comprise messages provided by the customer via a chat session with support staff, customer service representatives, etc. The messages may be stored in a log file and provided to data acquisition engine216after the chat session has ended. Alternatively, the messages may be provided to data acquisition engine216in real-time (e.g., during the chat session). Virtual agent chat sessions206may comprise messages provided by the customer via a chat session with virtual support staff or virtual customer service representatives, such as a chatbot and/or the like. The messages may be stored in a log file and provided to data acquisition engine216after the chat session has ended. Alternatively, the messages may be provided to data acquisition engine216in real-time (e.g., during the chat session). Verbatim responses208may comprise messages (or responses) provided via surveys (which may include customer rankings, ratings, reviews, comments, etc.). Social media content210may be content provided by the customer pertaining to a particular issue via one or more social media services, such as, Facebook®, Twitter®, MSDN, Stack Overflow, etc. The content may comprise one or more postings made by the customer and/or direct messages to an agent provided via the social media service(s) (e.g., which provides a social media forum, such as a website for posting messages).

Emails212may comprise one or more emails between the customer and the agent pertaining to a particular issue. Voice data214may comprise voice calls with support staff, customer service representatives, etc. Voice data214may be received by data acquisition engine216as one or more audio signals. The audio signals may be received after the voice call has ended. Alternatively, the audio signals may be provided to data acquisition engine216in real-time (e.g., during the call).

Data acquisition engine216may comprise a speech-to-text converter218and an audio analyzer220. Speech-to-text converter218may be configured to convert the voice data to text in accordance to a particular language (e.g., English, French, Spanish, etc.). Audio analyzer220may be configured to detect characteristics (e.g., the volume, tone, inflection, etc.) of voice content (e.g., words, phrases, sentences, etc.) uttered by the customer during the voice call, including if there was a pause between utterances without an agent response. Data acquisition engine216may associate each word, phrase, and or sentence uttered by a customer with an indicator that specifies the characteristics thereof. Data acquisition engine216provides the data collected from data sources202to preprocessing engine222.

Preprocessing engine222may be configured to perform spell checking operations on the messages received via agent chat sessions204, virtual agent chat sessions206, verbatim responses208, social media content210, emails, and/or the voice data converted into text. Preprocessing engine222may be further configured to apply support-specific rules, which may be configured to filter the received data in accordance with the rules. For example, preprocessing engine222may remove messages that may be interpreted as having neutral sentiments (e.g., by removing/ignoring certain messages from the collected data, such as, but not limited to, problem statements (“My Surface Pro died last night”), formality niceness (e.g. “Hello,” “How are you?” “Doing great.”), etc. Preprocessing engine222may further be configured to isolate messages provided by the customer from messages provided by the agent. For example, messages provided by the customer may be stored in one log, and messages provided by the agent may be stored in another log. In another example, messages provided by the customer may be marked in a first manner, and the messages provided by the agent may be marked in a different, second manner. In yet another example, the messages provided by the agent may be removed, and the messages provided by the customer may be maintained.

Customization engine224may be configured to perform data source (or channel)-specific customizations for data received from each of agent chat sessions204, virtual agent chat sessions206, verbatim responses208, social media content210, emails212and/or voice data214. The data received via each of these data sources may be in a format specific to that data source. For example, chat messages, emails, social media posts may include symbols, such as emotion icons (also known as “emoticons”). Verbatim responses may include data representative of rankings or ratings, etc., such as stars, numerical values, etc. Voice data may comprise characteristics, such as volume, tone, inflection, etc., as determined by audio analyzer220. The customizations may be performed in accordance with a customization model for each of data sources202, where each customization model may specify one or more rules for interpreting data in a data source-specific manner.

For example, as shown inFIG. 2, customization engine224includes an agent chat model226, a virtual agent model228, a verbatim model230, a social media model232, an email model234, and a voice data model236. Each of these models is configured to process the data received from a respective data source in a certain manner. For example, agent chat model226, virtual agent model228, email model234, and/or social media model232may be configured to analyze and/or interpret emotion icons (also known as “emoticons”) provided by the customers via messages provide during a chat session (e.g., with an agent or virtual agent), in an email or a social media post or message. In another example, social media model232may be configured to analyze and/or interpret “likes,” “hearts,”, “dislikes,” etc. selected by the customer with respect to a particular post or message. In yet another example, voice data model236may be configured to analyze and/or interpret the manner in which the customer uttered a word, phrase, sentence, etc. (e.g., by analyzing the volume, tone, inflection, etc. as determined by audio analyzer220). In a further example, verbatim model230may be configured to analyze and/or interpret, ratings, rankings, etc. (e.g., associate a particular number of stars as having a particular sentiment) provided by the user.

Batch scoring engine242may be configured to generate a sentiment classification and/or score for the type of data provided thereto. Batch scoring engine242may utilize natural language processing and/or machine-learning techniques to generate the score. For example, such techniques may classify certain words, phrases, sentences, etc., as having a negative sentiment, positive sentiment or a neutral sentiment, and the score may be determined based on such classification. In accordance with an embodiment, batch scoring engine242may utilize a supervised machine learning algorithm to classify words, phrases, sentences, etc. The machine learning algorithm learns what types of words, phrases, sentences, etc. are indicative of having a positive sentiment, what types of words, phrases, sentences, etc. are indicative of having a negative sentiment, and/or what types of words, phrases, sentences, etc. are indicative of having a neutral sentiment. The machine learning algorithm may be provided with training data that comprises positively-labeled words, phrases, sentences, etc., (i.e., labeled has having a positive sentiment) and negatively-labeled words, phrases, sentences, etc., (i.e., labeled has having a negative sentiment). Using these inputs, the machine learning algorithm learns what constitutes a word, phrase, sentence, etc. as having a positive sentiment or a negative sentiment and generates a classification model that is utilized to classify new words, phrases, sentences, etc. provided thereto as having a positive sentiment or a negative sentiment. The classification model may output an indication (e.g., prediction) as to whether the inputted word, phrase, sentence, etc., has a positive sentiment, neutral, or a negative sentiment. The indication may be a numerical value. For example, a word, phrase, sentence, etc. that is assigned a value between 0-40 may be labelled as having a negative sentiment, and a word, phrase, sentence, etc., that is assigned a value between 60-100 may be labelled as having a positive sentiment. A word, phrase, sentence, etc., assigned a value between 40-60 may be labelled as having a neutral sentiment. Sentiment related KPIs (Key Performance Indexes) may be defined in accordance with Table 1, which is provided below. It is noted that the KPIs provided in Table 1 are exemplary and that other KPIs may be utilized:

TABLE 1KPIAbbreviationDescriptionFormatCustomerCSPThe predicted customer sentiment in aNumber rangingSentimentgiven timeframe for a setfrom 0 to 100,Pulsequeue/site/scope which indicates thewhere 0 is worstdegree of positive sentiment expressed byand 100 bestcustomers at the end of their interactions.sentimentPositiveCSP % posCount of positive sentiment zone% of totalSentimentconversations divided by total chatPulsesessionsNeutralCSP % neuCount of neutral sentiment zone% of totalSentimentconversations divided by total chatPulsesessionsNegativeCSP % negCount of negative sentiment zone% of totalSentimentconversations divided by total chatPulsesessionsCustomerCSTAverage of sentiment transition for allNumber rangingSentimentchat sessionsfrom −100 to 100TransitionPositiveCST % posCount of sessions where the sentiment% of totalSentimenttransitioned upwards to the positive zoneTransitiondivided by total chat sessionsNegativeCST % negCount of sessions where the sentiment% of totalSentimenttransitioned down to the negative zoneTransitiondivided by total chat sessions

Batch scoring engine242may be further configured to output (e.g., via a GUI) the sentiment classification and/or score for the statements uttered by a user after the communication session (e.g., a chat or phone call) has ended. Additional details regarding batch scoring engine242are provided below with reference to Subsection B. Application programming interface240may be configured to output (e.g., via a GUI) a sentiment classification and/or score for a message provided by a user in real-time (i.e., during a live communication session (e.g., via chat or a phone call). Additional details regarding application programming interface240are provided below with reference to Subsection C. Sentiment driver engine238may be configured to determine a shift in customer sentiment (e.g., from negative to positive, to positive to negative, etc.) and the driver of that sentiment shift (i.e., the reason why the customer's sentiment shifted) based on the sentiment classification and/or score for customer messages provided by batch scoring engine242. Additional details regarding sentiment driver engine238are provided below with reference to Subsection D. Governance and continuous improvement engine244may be configured to obtain feedback (e.g., via an administrator, an agent manager, a supervisor, etc.) and/or refine each of agent chat model226, virtual agent model228, verbatim model230, social model232, email model234, and/or voice data model236based on the feedback. For example, the feedback may be utilized to update and/or create rules in which data from a particular data source is to be interpreted. For instance, the feedback may specify additional emoticons that are to be supported by agent chat model226, virtual agent model228, social media model232, and/or email model234.

B. Customer Sentiment Classification and Quantitation

It has been observed that how a communication session ends is very predictive of overall customer satisfaction. Accordingly, embodiments described herein analyze the last N statements of a customer to determine the customer's sentiment of the issue being discussed with the agent. In accordance with an embodiment, the last six statements of a communication session are analyzed.

For example,FIG. 3shows a table300of sentiment classifications for a plurality of customer messages for different communication sessions in accordance with an embodiment. In particular, table300depicts the classification of the last 6 (although it can be any positive integer number that renders the best predictive performance of the model) statements (shown as End to E-6, where End represents the last message in a communication session, E-1 represents the second to last message in the communication session, and so on and so forth) of 10 different communication sessions302,304,306,308,310,312,314,316318, and320. Messages that are classified as having a neutral sentiment are labeled with a “0”, messages that are classified as having a negative sentiment are labeled with a “−”, and messages that are classified as having a positive sentiment are labeled with a “+”. As further shown inFIG. 3, each of communication sessions302,304,306,308,310,312,314,316,318, and320may be classified based on the classification of the messages contained therein in accordance with four different classifications, although the embodiments described herein are not so limited. The four classifications shown inFIG. 3are All “Neutral”, “Negative”, “Positive”, and “Positive” or “Negative”. In particular, each of the 6 messages of communication session302has been classified as having a neutral sentiment, and therefore has been classified as being All “Neutral”. None of the last 6 messages of communication sessions304,306, and308were classified as having a positive sentiment, but some were classified as having a negative sentiment. Thus, these communication sessions have been classified as being “Negative”. In particular, the fifth to the last message (E-4) in communication session304, the fourth to last (E-3) message and second to last message (E-1) in communication session306, and all the messages in communication session308have been labeled as having a negative sentiment. None of the last 6 statements of communication sessions310,312, and314were classified as having a negative sentiment, but some were classified as having a positive sentiment. Thus, these communication sessions have been classified as being “Positive”. In particular, the second to last message (E-1) in communication session310, the sixth to last message (E-5) and third to last message (E-2) in communication session312, and the fifth to last message (E-4) through the last message (End) in communication session314were classified as having a positive sentiment. Communication sessions316,318, and320include messages labeled as having a positive sentiment and other statements labeled as having a negative sentiment. Thus, these communication sessions have been labeled as “Positive” or “Negative” In particular, the sixth to last message (E-5) and fifth to last message (E-4) in communication session316are labeled as having a negative sentiment, and second to last message (E-1) in communication session316is labeled as having a positive sentiment. The fifth to last message (E-4) in communication session318is labeled as having a positive sentiment, and the third to last message (E-2) and the last message (End) in communication session318are labeled as having a negative sentiment. The sixth to last message (E-6) and third to last message (E-2) in communication session320are labeled as having a negative sentiment, and the fourth to last message (E-3) and the second to last message (E-1) in communication session320are labeled as having a positive sentiment. All other messages shown inFIG. 3are labeled as having a neutral sentiment.

The whole communication session may be classified based on the classification of the last N messages (e.g., the last 6 messages as described above). For example, as described above, communication sessions that only include messages that are classified as having a neutral sentiment (e.g., communication session302) are classified as having a neutral sentiment (i.e., All “Neutral”), communication sessions that include messages being classified as having negative sentiments, but no positive sentiments (e.g., communication sessions304,306, and308), are classified as having a negative sentiment (i.e., “Negative”), and communication sessions that include messages being classified as having positive sentiments, but no negative sentiments (communication sessions310,312, and314), are classified as having a positive sentiment (i.e. “Positive”). To determine the sentiment of a communication session that includes messages having both negative and positive sentiments (e.g., communication sessions316,318, and320), the last message that is classified as being positive or negative is used to classify the communication session. For example, referring again toFIG. 3, communication sessions316and318are classified as having a positive sentiment, because the last non-neutral message (message E-1) in both communication sessions316and320are classified as being positive. Communication session318is classified as having a negative sentiment, because the last non-neutral message (message End) is classified as having a negative sentiment.

It is noted that whileFIG. 3shows classifications being represented by a “+” (for positive sentiments), “−” (for negative sentiments), and “0” (for neutral sentiments), classifications may be represented as a score (e.g., from 0-100, where 0 represents the most negatively-ranked sentiment, and 100 represents the most positively-ranked sentiment). In accordance with such an embodiment, the following techniques are utilized to generate a score for the whole communication session.

For example, with respect to communication session302, the average of the scores for all the messages (which are all classified as having a neutral sentiment) may be used as the score for the communication session. For communication sessions having both messages that are classified as having a positive sentiment and other messages that are classified as having a negative sentiment (e.g., communication sessions316,318, and320), the score of the last non-neutral message may be used as the score for that communication session. For communication sessions where only one message has been classified as having a negative sentiment (e.g., communication session304), the score of that message is used as the score for the communication session. Similarly, for communication sessions where only one message has been classified as having a positive sentiment (e.g., communication session310), the score of that message is used as the score for the communication session. For communication sessions having a plurality of messages that are classified as being negative and no messages that are classified as being positive (e.g., communication sessions306and308), and for communication sessions having a plurality of messages that are classified as being positive and no messages that are classified as being negative (e.g., communication sessions312and314), a “half the distance to the goal” rule is used to determine the score for such communication sessions.

When applying the rule to a communication session that only include messages that are classified as having negative and neutral sentiments, the overall score for the communication session decreases for each additional message that is classified as having a negative sentiment. For example,FIG. 4Ashows a table400A of a plurality of customer messages of different communication sessions that are classified as having either a negative sentiment or a neutral sentiment in accordance with an embodiment. As shown inFIG. 4A, the last six messages (E-5 to End) of six different communication sessions402,404,406,408,410, and412have been assigned a sentiment score. In the example shown inFIG. 4A, a message assigned a sentiment score of less than 50 (e.g.,30) is considered to have a negative sentiment, and a message assigned a sentiment score of 50 is considered to have a neutral sentiment. Accordingly, as shown inFIG. 4A, communication session402includes one message that has been classified as having a negative sentiment (i.e., the third to last message (E-2)), communication session404includes two messages that have been classified as having a negative sentiment (i.e., the fifth to last message (E-4) and the second to last message (E-1)), communication session406includes three messages that have been classified as having a negative sentiment (i.e., the fifth to last message (E-4), the fourth to last message (E-3), and the third to last message (E-2)), communication session408includes four messages that have been classified as having a negative sentiment (i.e., the sixth to last message (E-5), the fifth to last message (E-4), the third to last message (E-2), and the second to last message (E-1)), communication session410includes five messages that have been classified as having a negative sentiment (i.e., the sixth to last message (E-5), the fifth to last message (E-4), the fourth to last message (E-3), the third to last message (E-2), and the second to last message (E-1)), and communication session412includes six messages that have been classified has having a negative sentiment (i.e., messages E-5 through End).

FIG. 4Bshows a graph400B illustrating overall sentiment scores for each of communication sessions402,404,406,408,410, and412. As described above, batch scoring engine242may utilize a “half the distance to the goal” rule to determine the score for each of communication sessions402,404,406,408,410, and412. When applying the rule to communication sessions in which messages are assigned a negative sentiment score, but no messages are assigned a positive sentiment score, the lowest sentiment score that a communication session can receive is the difference between the lowest score determined for a particular message and the lowest possible score (e.g., 0) divided by 2, which equals “half of the distance to 0”. In particular, as shown inFIG. 4B, since the lowest score for a message is 30, the lowest possible score (represented by line414) that can be assigned to a communication session is 15 (i.e., (30−0)/2). As the number of messages assigned to a negative sentiment score increase, the overall sentiment score for the communication session decreases. The overall sentiment score may be decreased in accordance with a step function. For example, as shown inFIG. 4B, the overall sentiment score decreases for a communication session for each additional message that is assigned a negative sentiment score. In particular, communication session402is assigned an overall sentiment score of 30, communication session404is assigned an overall sentiment score of 27, communication session406is assigned an overall sentiment score of 24, communication session408is assigned an overall sentiment score of 21, communication session410is assigned an overall sentiment score of 18, and communication session412is assigned an overall sentiment score of 15.

FIG. 5Ashows a table500A of a plurality of customer messages of different communication sessions that are classified as having either a negative sentiment or a neutral sentiment in accordance with another embodiment. As shown table500A, each of communication sessions502,504,506,508,510, and512have at least one message that was assigned a negative score. However, such messages are assigned a lower sentiment score than the messages described above with reference toFIG. 4A. In particular, such messages are assigned a score of 20, meaning that such messages have an even lower sentiment than the messages described above with reference toFIG. 4A. Accordingly, the overall sentiment score for communication sessions502,504,506,508,510, and512differs than communication sessions402,404,406,408,410, and412when applying the “half the distance to the goal” rule.

For example,FIG. 5Bshows a graph500B illustrating the overall sentiment score for each of communication sessions502,504,506,508,510, and512. As shown inFIG. 5B, since the lowest score for a message is 20, the lowest possible score (represented by line514) that can be assigned to a communication session is 10 (i.e., (20−0)/2). As the number of messages assigned to a negative sentiment score increases, the overall sentiment score for the communication session decreases (e.g., in accordance with a step function). In particular, communication session502is assigned an overall sentiment score of 20, communication session504is assigned an overall sentiment score of 18, communication session506is assigned an overall sentiment score of 16, communication session508is assigned an overall sentiment score of 14, communication session510is assigned an overall sentiment score of 12, and communication session512is assigned an overall sentiment score of 10.

When applying the rule to a communication session that only includes messages that are classified as having positive and neutral sentiments, the overall score for the communication session increases for each additional message that is classified as having a positive sentiment. For example,FIG. 6Ashows a table600A of a plurality of customer messages of different communication sessions that are classified as having either a positive sentiment or a neutral sentiment in accordance with an embodiment. As shown inFIG. 6A, the last six messages (E-5 to End) of six different communication sessions602,604,606,608,610, and612have been assigned a sentiment score. In the example shown inFIG. 6A, a message assigned a sentiment score greater than 50 (e.g.,70) is considered to have a positive sentiment, and a message assigned a sentiment score of 50 is considered to have a neutral sentiment. Accordingly, as shown inFIG. 6A, communication session602includes one message that has been classified as having a positive sentiment (i.e., the second to last message (E-1)), communication session604includes two messages that have been classified as having a positive sentiment (i.e., the sixth to last message (E-5) and the last message (End)), communication session606includes three messages that have been classified as having a positive sentiment (i.e., the fifth to last message (E-4), the third to last message (E-2), and the second to last message (E-1)), communication session608includes four messages that have been classified as having a positive sentiment (i.e., the sixth to last message (E-5), the fourth to last message (E-3), the third to last message (E-2), and the last message (End)), communication session610includes five messages that have been classified as having a positive sentiment (i.e., the sixth to last message (E-5), the fifth to last message (E-4), the fourth to last message (E-3), the second to last message (E-1), and the last message (End)), and communication session612includes six messages that have been classified has having a positive sentiment (i.e., messages E-5 through End).

FIG. 6Bshows a graph600B illustrating the overall sentiment score for each of communication sessions602,604,606,608,610, and612. As described above, batch scoring engine242may utilize a “half the distance to the goal” rule to determine the score for each of communication sessions602,604,606,608,610, and612. When applying the rule to a communication session in which messages are assigned a positive sentiment score, but no messages are assigned a negative sentiment score, the highest sentiment score that a communication session can receive is the difference between the maximum possible score (e.g., 100) and half the delta between the highest possible score and the highest score determined for a particular message, which equals “half the distance to 100”. In particular, as shown inFIG. 6B, since the highest score for a message is 70, the highest possible score (represented by line614) that can be assigned to a communication session is 85 (i.e., 100−((100−70)/2)). As the number of messages assigned to a positive sentiment score increases, the overall sentiment score for the communication session increases. The overall sentiment score may be increased in accordance with a step function. For example, as shown inFIG. 6B, the overall sentiment score for a communication session increases for each additional message that is assigned a positive sentiment score. In particular, communication session602is assigned an overall sentiment score of 70, communication session604is assigned an overall sentiment score of 73, communication session606is assigned an overall sentiment score of 76, communication session608is assigned an overall sentiment score of 79, communication session610is assigned an overall sentiment score of 82, and communication session612is assigned an overall sentiment score of 85.

FIG. 7Ashows a table700A of a plurality of customer messages of different communication sessions that are classified as having either a positive sentiment or a neutral sentiment in accordance with another embodiment. As shown table700A, each of communication sessions702,704,706,708,710, and712have at least one message that was assigned a positive score. However, such messages are assigned a higher sentiment score than the messages described above with reference toFIG. 6A. In particular, such messages are assigned a score of 80, meaning that such messages have an even higher sentiment than the messages described above with reference toFIG. 6A. Accordingly, the overall sentiment score for communication sessions702,704,706,708,710, and712differs than communication sessions602,604,606,608,610, and612when applying the “half the distance to the goal” rule.

For example,FIG. 7Bshows a graph700B illustrating the overall sentiment score for each of communication sessions702,704,706,708,710, and712. As shown inFIG. 7B, since the highest score for a message is 80, the highest possible score (represented by line714) that can be assigned to a communication session is 90 (i.e., 100−(100−20)/2). As the number of messages assigned to a positive sentiment score increase, the overall sentiment score for the communication session increases (e.g., in accordance with a step function). In particular, communication session702is assigned an overall sentiment score of 80, communication session704is assigned an overall sentiment score of 82, communication session706is assigned an overall sentiment score of 84, communication session708is assigned an overall sentiment score of 86, communication session710is assigned an overall sentiment score of 88, and communication session712is assigned an overall sentiment score of 90.

C. Dynamic Sentiment Determination in a Real-Time Customer Interaction Environment

Today, support agent managers have no way to identify chat or phone sessions in need of corrective intervention. They are left to poke around based on chat or call durations, which is ineffective, usually too late, and limits the number of agents a supervisor can effectively coach. Real-time customer sentiment monitoring is critical to ensuring support quality as well as providing timely recommendations for intervention and improvement during the communication sessions. Leveraging the above-described techniques for classifying and quantifying customer sentiments, a real-time sentiment intensity, e.g., ranging from −3 to 3, and a sentiment trend, may be determined within a configurable rolling message window (a.k.a. “micro session”) for a given ongoing communication (e.g., chat/phone) session. The real-time sentiment intensity and/or sentiment trend may be displayed to a call center manager via a GUI. The GUI may display the real-time sentiment intensity and/or sentiment trend for a plurality of different agents. This may advantageously enable call center managers to easily identify customer support interactions' sentiment and sentiment trend and quickly identify which agents are struggling and which ones are doing well, thereby enabling targeted real-time corrective supervisor intervention.

Real-time sentiment may be reported upon a customer entering or uttering an N number (e.g., six) of messages. The sentiment determination techniques described above in Subsection B may be utilized to classify the N number of messages into different classes (e.g., All “Neutral”, “Negative”, “Positive”, and “‘Positive’ or ‘Negative’”, as described above). The sentiment determination may be performed for a rolling window of an N number of messages (i.e., the last N number of messages of any given communication session). Messages that are classified as having a positive or negative sentiment are considered to be strong signals and always turn the trend, so any positive intensity will become slightly negative if the last message is negative. The same is true for positive inflections from negative. That is, any negative intensity will become slightly positive if the last message is positive. Every positive intensity is increased to the next level (or value) if the last message is classified as having a positive sentiment, and every negative intensity is increased to the next level if the last message is classified as having a negative sentiment. Positive intensity may be maxed out at a maximum positive intensity value (e.g., 3), and negative intensity may be maxed out at a maximum negative intensity value (e.g., −3). Messages classified as having a neutral sentiment may soften the intensity (intensity entropy) depending on the amount of remaining positive (or negative) messages in the N number of messages since a positive-to-slightly negative (or a negative to slightly-positive) sentiment shift. Neutral intensity requires all N messages to be classified as having a neutral. A slightly positive or slightly negative intensity stays at that intensity when the last message is classified as having a neutral intensity, except in the case where the previous N−1 messages are classified as having a neutral sentiment. In this case, the sentiment, intensity becomes neutral, as described above.

Table 2, below, illustrates how batch scoring engine242may determine one of seven different sentiment intensity levels based on an analysis of the last N messages of an ongoing communication session. Table 2 is described in accordance with an embodiment in which N is equal to six, although the embodiments described herein are not so limited:

TABLE 2Real Time SentimentIntensity (7 level)DeterminationFully Positivea) when the current sentiment intensity is a medium(e.g., a sentimentpositive intensity, the last message is classified as havingintensity value of 3)a positive sentiment, and the number of messages in therolling window that is classified as having a positivesentiment since a negative-to-slightly positive shift is atleast three;b) when the current sentiment intensity is a fully positiveintensity and the last message is classified as having apositive sentiment; orc) when the current sentiment intensity is a fully positiveintensity, the last message is classified as having a neutralsentiment, and the number of messages in the rollingwindow that is classified as having a positive sentimentsince a negative-to-slightly positive shift is at least three;Medium Positivea) when the current sentiment intensity is a fully positive(e.g., a sentimentintensity, the last message is classified as having a neutralintensity value of 2)sentiment, and the number of messages in the rollingwindow that is classified as having a positive sentimentsince a negative-to-slightly positive shift is two;b) when the current sentiment intensity is a slightlypositive intensity, the last message is classified as havinga positive sentiment, and the number of messages in therolling window that is classified as having a positivesentiment since a negative-to-slightly positive shift istwo; orc) when the current sentiment intensity is a mediumpositive intensity, the last message is classified as havinga neutral sentiment, and the number of messages in therolling window that is classified as having a positivesentiment since a negative-to-slightly positive shift istwo;Slightly Positivea) When the current sentiment intensity is a medium(e.g., a sentimentpositive intensity, the last message is classified as havingintensity value of 1)a neutral sentiment, and the number of messages in therolling window that is classified as having a positivesentiment since a negative-to-slightly positive shift isone;b) when the current sentiment intensity is a neutralintensity, the last message is classified as having apositive sentiment, and the number of messages in therolling window that is classified as having a positivesentiment since a negative-to-slightly positive shift isone;c) when the current sentiment is a negative intensity andthe last message is classified as having a positivesentiment; ord) when the current sentiment is a slightly positiveintensity, the last message is classified as having a neutralsentiment, and the number of messages in the rollingwindow that is classified as having a positive sentimentsince a negative-to-slightly positive shift is oneNeutralWhen all N messages are classified as having a neutral(e.g., a sentimentsentimentintensity value of 0)Slightly Negativea) When the current sentiment intensity is a medium(e.g., a sentimentnegative intensity, the last message is classified as havingintensity value of −1)a neutral sentiment, and the number of messages in therolling window that is classified as having a negativesentiment since a positive-to-slightly negative shift isone;b) when the current sentiment intensity is a neutralintensity, the last message is classified as having anegative sentiment, and the number of messages in therolling window that is classified as having a negativesentiment since a positive-to-slightly negative shift isone;c) when the current sentiment is a positive intensity andthe last message is classified as having a negativesentiment; ord) when the current sentiment is a slightly negativeintensity and the last message is classified as having aneutral sentiment, and the number of messages in therolling window that is classified as having a negativesentiment since a positive-to-slightly negative shift is oneMedium Negativea) when the current sentiment intensity is a fully negative(e.g., a sentimentintensity, the last message is classified as having a neutralintensity value of −2)sentiment, and the number of messages in the rollingwindow that is classified as having a negative sentimentsince a positive-to-slightly negative shift is two;b) when the current sentiment intensity is a slightlynegative intensity, the last message is classified as havinga negative sentiment, and the number of messages in therolling window that is classified as having a negativesentiment since a positive-to-slightly negative shift istwo; orc) when the current sentiment intensity is a mediumnegative intensity, the last message is classified as havinga neutral sentiment, and the number of messages in therolling window that is classified as having a negativesentiment since a positive-to-slightly negative shift is twoFully Negativea) when the current sentiment intensity is a medium(e.g., a sentimentnegative intensity, the last message is classified as havingintensity value of −3)a negative sentiment, and the number of messages in therolling window that is classified as having a negativesentiment since a positive-to-slightly negative shift is atleast three;b) when the current sentiment intensity is a fully negativeintensity and the last message is classified as having anegative sentiment; orc) when the current sentiment is a fully negative intensity,the last message is classified as having a neutralsentiment, and the number of messages in the rollingwindow that is classified as having a negative sentimentsince a positive-to-slightly-negative shift is at least three

Table 3, below, illustrates five trend indicators and how each may be determined by batch scoring engine242:

TABLE 3Trend StateComputationStrong UpTwo or more last massages areclassified as having a positivesentimentUpLast (one) message is classified ashaving a positive sentimentNeutralLast message is classified as having aneutral sentimentDownLast (one) message is classified ashaving a negative sentimentStrong DownTwo or more last messages areclassified as having a negativesentiment

Batch scoring engine242may provide the determined sentiment intensity level and/or sentiment trend to application programming interface240in real-time (e.g., during a communication session), and application programming interface240may cause one or more symbols representative of the sentiment intensity and/or sentiment trend to be rendered via a GUI in real-time. For example,FIG. 8depicts a table800of symbols that may be used to represent real-time sentiment intensity via a GUI in accordance with an example embodiment.FIG. 9depicts a table900of symbols that may be used to represent real-time sentiment trend via a GUI in accordance with an example embodiment.

As shown inFIG. 8, a fully positive sentiment intensity may be represented by a first symbol802. First symbol802is depicted as a filled, upward-facing arrow head. A medium positive sentiment intensity may be represented by a second symbol804. Second symbol802is depicted as a partially-filled, upward-facing arrow head. A slightly positive sentiment intensity may be represented by a third symbol806. Third symbol806is depicted as an unfilled (or empty) arrow head. A neutral sentiment intensity may be represented by a fourth symbol808. Fourth symbol808is depicted as an unfilled (or empty). rectangular shape. A slightly negative sentiment intensity may be represented by a fifth symbol810. Fifth symbol810is depicted as an unfilled (or empty), downward-facing arrow head. A medium negative sentiment intensity may be represented by a sixth symbol812. Sixth symbol812is depicted as a partially-filled, downward-facing arrow head. A fully negative sentiment intensity may be represented by a seventh symbol814. Seventh symbol814is depicted as a filled, downward-facing arrow head. It is noted that the depictions of first symbol802, second symbol804, third symbol806, fourth symbol808, fifth symbol810, sixth symbol812, and seventh symbol814are purely exemplary, and that other symbols may be utilized and/or other attributes may be associated with the symbols described above. For example, each of first symbol802, second symbol804, third symbol806, fourth symbol808, fifth symbol810, sixth symbol812, and seventh symbol814may be color coded, where positive intensity symbols (i.e., symbols802,804, and806) are green and negative intensity arrows (e.g., symbols810,812, and814) are red. Symbol808may be visually distinct from the other symbols (e.g., it may have a different shape and/or have a different color, such as gray), which advantageously avoids visual overload for this non-actionable intensity.

As shown inFIG. 9, a strongly positive sentiment trend (“Strong Up”) may be represented by a first symbol902. First symbol902is depicted as an up arrow. A slightly positive sentiment trend (“Up”) may be represented by a second symbol904. Second symbol904is depicted as an upwardly-angled arrow. A neutral sentiment trend (“Neutral”) may be represented by a third symbol906. Third symbol906is depicted as an arrow pointing to the right. A slightly negative sentiment trend (“Down”) may be represented by a fourth symbol908. Fourth symbol908is depicted as a downwardly-angled arrow. A strongly negative sentiment trend (“Strong Down”) may be represented by a fifth symbol910. Fifth symbol910is depicted as a down arrow. It is noted that the depictions of first symbol902, second symbol904, third symbol906, fourth symbol908, and fifth symbol910, are purely exemplary, and that other symbols may be utilized and/or other attributes may be associated with the symbols described above. For example, each of first symbol902, second symbol904, third symbol906, fourth symbol908, and fifth symbol910may be color coded, where positive sentiment trends (i.e., symbols902and904) are green and negative sentiment trends (e.g., symbols908and910) are red. Symbol906may be visually distinct from the other symbols (e.g., it may have a different shape and/or a different color, such as gray), which advantageously avoids visual overload for this non-actionable intensity.

FIG. 10shows a graph1000depicting sentiment scores for a plurality of different messages in a communication session in accordance with an embodiment. The x-axis (“Sequence Number”) represents the number of the message in the communication session, and the y-axis represents the sentiment score, where scores over upper bound1001(e.g., sentiment scores over 60) are representative of a positive sentiment, scores less than lower bound1003(e.g., sentiment scores less than 40) are representative of a negative sentiment, and scores between upper bound1001and lower bound (e.g., sentiment scores ranging from 40 to 60) are representative of a neutral sentiment.FIGS. 11A-11Cshow tables1100A-1100C specifying the determined real time sentiment and sentiment trend for the messages represented in graph1000ofFIG. 10. The real-time sentiment starts with the 6thcustomer message (i.e. N=6). Every row represents the message-to-message change in real time sentiment and sentiment trend.

As shown inFIG. 10, the communication session begins with considerable sentiment fluctuations (see e.g., sentiment scores for the first six messages (represented by points1002,1004,1006,1008,1010, and1012), then stabilizes and trends mostly positive. During the first 6 messages, the real time sentiment may default to neutral, although the embodiments described herein are not so limited. The sentiment classifications for these 6 messages (i.e., the current rolling window) is shown in the first row of table1100A, column1102. In particular, the sentiment classification for point1002is neutral (“0”), the sentiment classification for point1004is positive (“+”), the sentiment classification for point1006is negative (“−”), the sentiment classification for point1008is positive (“+”), the sentiment classification for point1010is neutral (“0”), and the sentiment classification for point1012is negative (“−”). In accordance with Table 2 described above, because the sentiment intensity was neutral and the last message (point1012) is classified as having a negative sentiment, the current real time sentiment is “slightly negative,” and symbol810(as described above with reference toFIG. 8) would be rendered via a GUI. In accordance with Table 3, because the last message is classified as having a negative sentiment, symbol908(as described above with reference toFIG. 9) would be rendered via a GUI.

The seventh message (shown as point1014inFIG. 10) is classified as having a neutral sentiment. The current rolling window of sentiment classifications no longer includes the sentiment classification of the first message (i.e., point1002), but now includes the sentiment classification for the seventh message. The current rolling window of sentiment classifications is shown in the second row of table1100A (i.e., “+−+0−0”). In accordance with Table 2 described above, because the sentiment intensity was “slightly negative” and the last message (point1014) is classified as having a neutral sentiment, the current real time sentiment is “slightly negative,” and symbol810(as described above with reference toFIG. 8) would be rendered via a GUI. In accordance with Table 3, because the last message is classified as having a neutral sentiment, symbol906(as described above with reference toFIG. 9) would be rendered via a GUI.

The eighth message (shown as point1016inFIG. 10) is classified as having a positive sentiment. The current rolling window of sentiment classifications no longer includes the sentiment classification of the second message (i.e., point1004), but now includes the sentiment classification for the eighth message. The current rolling window of sentiment classifications is shown in the third row of table1100A (i.e., “−+0−0+”). In accordance with Table 2 described above, because the sentiment intensity was “slightly negative” and the last message (point1016) is classified as having a positive sentiment, the current real time sentiment is “slightly positive,” and symbol806(as described above with reference toFIG. 8) would be rendered via a GUI. In accordance with Table 3, because the last message is classified as having a positive sentiment, symbol904(as described above with reference toFIG. 9) would be rendered via a GUI.

The ninth message (shown as point1018inFIG. 10) is classified as having a neutral sentiment. The current rolling window of sentiment classifications no longer includes the sentiment classification of the third message (i.e., point1006), but now includes the sentiment classification for the ninth message. The current rolling window of sentiment classifications is shown in the fourth row of table1100A (i.e., “+0−0+0”). In accordance with Table 2 described above, because the sentiment intensity was “slightly positive” and the last message (point1018) is classified as having a neutral sentiment, the current real time sentiment is “slightly positive,” and symbol806(as described above with reference toFIG. 8) would be rendered via a GUI. In accordance with Table 3, because the last message is classified as having a neutral sentiment, symbol906(as described above with reference toFIG. 9) would be rendered via a GUI.

The tenth message (shown as point1020inFIG. 10) is classified as having a negative sentiment. The current rolling window of sentiment classifications no longer includes the sentiment classification of the fourth message (i.e., point1008), but now includes the sentiment classification for the tenth message. The current rolling window of sentiment classifications is shown in the fifth row of table1100A (i.e., “0−0+0−”). In accordance with Table 2 described above, because the sentiment intensity was “slightly positive” and the last message (point1020) is classified as having a negative sentiment, the current real time sentiment is “slightly negative,” and symbol810(as described above with reference toFIG. 8) would be rendered via a GUI. In accordance with Table 3, because the last message is classified as having a negative sentiment, symbol908(as described above with reference toFIG. 9) would be rendered via a GUI.

The eleventh message (shown as point1022inFIG. 10) is classified as having a neutral sentiment. The current rolling window of sentiment classifications no longer includes the sentiment classification of the fifth message (i.e., point1010), but now includes the sentiment classification for the eleventh message. The current rolling window of sentiment classifications is shown in the sixth row of table1100A (i.e., “−0+0−0”). In accordance with Table 2 described above, because the sentiment intensity was “slightly negative” and the last message (point1022) is classified as having a neutral sentiment, the current real time sentiment is “slightly negative,” and symbol810(as described above with reference toFIG. 8) would be rendered via a GUI. In accordance with Table 3, because the last message is classified as having a neutral sentiment, symbol906(as described above with reference toFIG. 9) would be rendered via a GUI.

The twelfth message (shown as point1024inFIG. 10) is classified as having a neutral sentiment. The current rolling window of sentiment classifications no longer includes the sentiment classification of the sixth message (i.e., point1012), but now includes the sentiment classification for the twelfth message. The current rolling window of sentiment classifications is shown in the seventh row of table1100A (i.e., “0+0−00”). In accordance with Table 2 described above, because the sentiment intensity was “slightly negative” and the last message (point1024) is classified as having a neutral sentiment, the current real time sentiment is “slightly negative,” and symbol810(as described above with reference toFIG. 8) would be rendered via a GUI. In accordance with Table 3, because the last message is classified as having a neutral sentiment, symbol906(as described above with reference toFIG. 9) would be rendered via a GUI.

The thirteenth message (shown as point1026inFIG. 10) is classified as having a neutral sentiment. The current rolling window of sentiment classifications no longer includes the sentiment classification of the seventh message (i.e., point1014), but now includes the sentiment classification for the thirteenth message. The current rolling window of sentiment classifications is shown in the eighth row of table1100A (i.e., “+0−000”). In accordance with Table 2 described above, because the sentiment intensity was “slightly negative” and the last message (point1026) is classified as having a neutral sentiment, the current real time sentiment is “slightly negative,” and symbol810(as described above with reference toFIG. 8) would be rendered via a GUI. In accordance with Table 3, because the last message is classified as having a neutral sentiment, symbol906(as described above with reference toFIG. 9) would be rendered via a GUI.

The fourteenth message (shown as point1028inFIG. 10) is classified as having a negative sentiment. The current rolling window of sentiment classifications no longer includes the sentiment classification of the eighth message (i.e., point1016), but now includes the sentiment classification for the fourteenth message. The current rolling window of sentiment classifications is shown in the ninth row of table1100A (i.e., “0−000−”). In accordance with Table 2 described above, because the sentiment intensity was “slightly negative,” the last message (point1028) is classified as having a negative sentiment, and the number of messages in the rolling window that are classified as having a negative sentiment is two, the current real time sentiment is “medium negative,” and symbol812(as described above with reference toFIG. 8) would be rendered via a GUI. In accordance with Table 3, because the last message is classified as having a negative sentiment, symbol908(as described above with reference toFIG. 9) would be rendered via a GUI.

The fifteenth message (shown as point1030inFIG. 10) is classified as having a positive sentiment. The current rolling window of sentiment classifications no longer includes the sentiment classification of the ninth message (i.e., point1018), but now includes the sentiment classification for the fifteenth message. The current rolling window of sentiment classifications is shown in the tenth row of table1100A (i.e., “−000−+”). In accordance with Table 2 described above, because the sentiment intensity was “medium negative” and the last message (point1030) is classified as having a positive sentiment, the current real time sentiment is “slightly positive,” and symbol806(as described above with reference toFIG. 8) would be rendered via a GUI. In accordance with Table 3, because the last message is classified as having a positive sentiment, symbol904(as described above with reference toFIG. 9) would be rendered via a GUI.

The sixteenth message (shown as point1032inFIG. 10) is classified as having a neutral sentiment. The current rolling window of sentiment classifications no longer includes the sentiment classification of the tenth message (i.e., point1020), but now includes the sentiment classification for the sixteenth message. The current rolling window of sentiment classifications is shown in the eleventh row of table1100A (i.e., “000−+0”). In accordance with Table 2 described above, because the sentiment intensity was “slightly positive” and the last message (point1032) is classified as having a neutral sentiment, the current real time sentiment is “slightly positive,” and symbol806(as described above with reference toFIG. 8) would be rendered via a GUI. In accordance with Table 3, because the last message is classified as having a neutral sentiment, symbol906(as described above with reference toFIG. 9) would be rendered via a GUI.

The seventeenth message (shown as point1034inFIG. 10) is classified as having a positive sentiment. The current rolling window of sentiment classifications no longer includes the sentiment classification of the eleventh message (i.e., point1022), but now includes the sentiment classification for the seventeenth message. The current rolling window of sentiment classifications is shown in the twelfth row of table1100A (i.e., “00−+0+”). In accordance with Table 2 described above, because the sentiment intensity was “slightly positive”, the last message (point1034) is classified as having a positive sentiment, and the number of messages in the rolling window that are classified as having a positive sentiment since the negative-to-positive negative shift (as occurred via the fifteenth message) is two, the current real time sentiment is “medium positive,” and symbol804(as described above with reference toFIG. 8) would be rendered via a GUI. In accordance with Table 3, because the last message is classified as having a positive sentiment, symbol904(as described above with reference toFIG. 9) would be rendered via a GUI.

The eighteenth message (shown as point1036inFIG. 10) is classified as having a neutral sentiment. The current rolling window of sentiment classifications no longer includes the sentiment classification of the twelfth message (i.e., point1024), but now includes the sentiment classification for the eighteenth message. The current rolling window of sentiment classifications is shown in the thirteenth row of table1100A (i.e., “0−+0+0”). In accordance with Table 2 described above, because the sentiment intensity was “medium positive”, the last message (point1036) is classified as having a neutral sentiment, and the number of messages in the rolling window that are classified as having a positive sentiment since the negative-to-slightly positive shift (as occurred via the fifteenth message) is two, the current real time sentiment is “medium positive,” and symbol804(as described above with reference toFIG. 8) would be rendered via a GUI. In accordance with Table 3, because the last message is classified as having a neutral sentiment, symbol906(as described above with reference toFIG. 9) would be rendered via a GUI.

The nineteenth message (shown as point1038inFIG. 10) is classified as having a neutral sentiment. The current rolling window of sentiment classifications no longer includes the sentiment classification of the thirteenth message (i.e., point1026), but now includes the sentiment classification for the nineteenth message. The current rolling window of sentiment classifications is shown in the first row of table1100B (i.e., “−+0+00”). In accordance with Table 2 described above, because the sentiment intensity was “medium positive”, the last message (point1038) is classified as having a neutral sentiment, and the number of messages in the rolling window that are classified as having a positive sentiment since the negative-to-slightly positive shift (as occurred via the fifteenth message) is two, the current real time sentiment is “medium positive,” and symbol804(as described above with reference toFIG. 8) would be rendered via a GUI. In accordance with Table 3, because the last message is classified as having a neutral sentiment, symbol906(as described above with reference toFIG. 9) would be rendered via a GUI.

The twentieth message (shown as point1040inFIG. 10) is classified as having a positive sentiment. The current rolling window of sentiment classifications no longer includes the sentiment classification of the fourteenth message (i.e., point1028), but now includes the sentiment classification for the twentieth message. The current rolling window of sentiment classifications is shown in the second row of table1100B (i.e., “+0+00+”). In accordance with Table 2 described above, because the sentiment intensity was “medium positive” the last message (point1040) is classified as having a positive sentiment, and the number of messages in the rolling window that are classified as having a positive sentiment is three, the current real time sentiment is “fully positive,” and symbol802(as described above with reference toFIG. 8) would be rendered via a GUI. In accordance with Table 3, because the last message is classified as having a positive sentiment, symbol904(as described above with reference toFIG. 9) would be rendered via a GUI.

The twenty first message (shown as point1042inFIG. 10) is classified as having a positive sentiment. The current rolling window of sentiment classifications no longer includes the sentiment classification of the fifteenth message (i.e., point1030), but now includes the sentiment classification for the twenty first message. The current rolling window of sentiment classifications is shown in the third row of table1100B (i.e., “0+0 0++”). In accordance with Table 2 described above, because the sentiment intensity was “fully positive” and the last message (point1042) is classified as having a positive sentiment, the current real time sentiment is “fully positive,” and symbol802(as described above with reference toFIG. 8) would be rendered via a GUI. In accordance with Table 3, because the last two messages are classified as having a positive sentiment, symbol902(as described above with reference toFIG. 9) would be rendered via a GUI.

The twenty second message (shown as point1044inFIG. 10) is classified as having a negative sentiment. The current rolling window of sentiment classifications no longer includes the sentiment classification of the sixteenth message (i.e., point1032), but now includes the sentiment classification for the twenty second message. The current rolling window of sentiment classifications is shown in the fourth row of table1100B (i.e., “+0 0++−”). In accordance with Table 2 described above, because the sentiment intensity was “fully positive” and the last message (point1044) is classified as having a negative sentiment, the current real time sentiment is “slightly negative,” and symbol810(as described above with reference toFIG. 8) would be rendered via a GUI. In accordance with Table 3, because the last message is classified as having a negative sentiment, symbol908(as described above with reference toFIG. 9) would be rendered via a GUI.

The twenty third message (shown as point1046inFIG. 10) is classified as having a positive sentiment. The current rolling window of sentiment classifications no longer includes the sentiment classification of the seventeenth message (i.e., point1034), but now includes the sentiment classification for the twenty third message. The current rolling window of sentiment classifications is shown in the fifth row of table1100B (i.e., “00++−+”). In accordance with Table 2 described above, because the sentiment intensity was “slightly negative” and the last message (point1046) is classified as having a positive sentiment, the current real time sentiment is “slightly positive,” and symbol806(as described above with reference toFIG. 8) would be rendered via a GUI. In accordance with Table 3, because the last message is classified as having a positive sentiment, symbol904(as described above with reference toFIG. 9) would be rendered via a GUI.

The twenty fourth message (shown as point1048inFIG. 10) is classified as having a neutral sentiment. The current rolling window of sentiment classifications no longer includes the sentiment classification of the eighteenth message (i.e., point1036), but now includes the sentiment classification for the twenty fourth message. The current rolling window of sentiment classifications is shown in the sixth row of table1100B (i.e., “0++−+0”). In accordance with Table 2 described above, because the sentiment intensity was “slightly positive” and the last message (point1048) is classified as having a neutral sentiment, the current real time sentiment is “slightly positive,” and symbol806(as described above with reference toFIG. 8) would be rendered via a GUI. In accordance with Table 3, because the last message is classified as having a neutral sentiment, symbol906(as described above with reference toFIG. 9) would be rendered via a GUI.

The twenty fifth message (shown as point1050inFIG. 10) is classified as having a positive sentiment. The current rolling window of sentiment classifications no longer includes the sentiment classification of the nineteenth message (i.e., point1038), but now includes the sentiment classification for the twenty fifth message. The current rolling window of sentiment classifications is shown in the seventh row of table1100B (i.e., “++−+0+”). In accordance with Table 2 described above, because the sentiment intensity was “slightly positive”, the last message (point1050) is classified as having a positive sentiment, and the number of messages in the rolling window that is classified as having a positive sentiment is two since the negative-to-slightly positive sentiment shift (as occurred via the twenty third message) is two, the current real time sentiment is “medium positive,” and symbol804(as described above with reference toFIG. 8) would be rendered via a GUI. In accordance with Table 3, because the last message is classified as having a positive sentiment, symbol904(as described above with reference toFIG. 9) would be rendered via a GUI.

The twenty sixth message (shown as point1052inFIG. 10) is classified as having a positive sentiment. The current rolling window of sentiment classifications no longer includes the sentiment classification of the twentieth message (i.e., point1040), but now includes the sentiment classification for the twenty sixth message. The current rolling window of sentiment classifications is shown in the eighth row of table1100B (i.e., “+−+0++”). In accordance with Table 2 described above, because the sentiment intensity was “medium positive”, the last message (point1052) is classified as having a positive sentiment, and the number of messages in the rolling window that is classified as having a positive sentiment is two since the negative-to-slightly positive sentiment shift (as occurred via the twenty third message) is at least three, the current real time sentiment is “fully positive,” and symbol802(as described above with reference toFIG. 8) would be rendered via a GUI. In accordance with Table 3, because the last two messages are classified as having a positive sentiment, symbol902(as described above with reference toFIG. 9) would be rendered via a GUI.

The twenty seventh message (shown as point1054inFIG. 10) is classified as having a positive sentiment. The current rolling window of sentiment classifications no longer includes the sentiment classification of the twenty first message (i.e., point1042), but now includes the sentiment classification for the twenty seventh message. The current rolling window of sentiment classifications is shown in the ninth row of table1100B (i.e., “−+0+++”). In accordance with Table 2 described above, because the sentiment intensity was “fully positive” and the last message (point1054) is classified as having a positive sentiment, the current real time sentiment is “fully positive,” and symbol802(as described above with reference toFIG. 8) would be rendered via a GUI. In accordance with Table 3, because the last two (or more) messages are classified as having a positive sentiment, symbol902(as described above with reference toFIG. 9) would be rendered via a GUI.

The twenty eighth message (shown as point1056inFIG. 10) is classified as having a neutral sentiment. The current rolling window of sentiment classifications no longer includes the sentiment classification of the twenty second message (i.e., point1044), but now includes the sentiment classification for the twenty eighth message. The current rolling window of sentiment classifications is shown in the tenth row of table1100B (i.e., “+0+++0”). In accordance with Table 2 described above, because the sentiment intensity was “fully positive”, the last message (point1056) is classified as having a neutral sentiment, and the number of messages in the rolling window that is classified as having a positive sentiment is at least three, the current real time sentiment is “fully positive,” and symbol802(as described above with reference toFIG. 8) would be rendered via a GUI. In accordance with Table 3, because the last message is classified as having a neutral sentiment, symbol906(as described above with reference toFIG. 9) would be rendered via a GUI.

The twenty ninth message (shown as point1058inFIG. 10) is classified as having a positive sentiment. The current rolling window of sentiment classifications no longer includes the sentiment classification of the twenty third message (i.e., point1046), but now includes the sentiment classification for the twenty ninth message. The current rolling window of sentiment classifications is shown in the eleventh row of table1100B (i.e., “0+++0+”). In accordance with Table 2 described above, because the sentiment intensity was “fully positive” and the last message (point1058) is classified as having a positive sentiment, the current real time sentiment is “fully positive,” and symbol802(as described above with reference toFIG. 8) would be rendered via a GUI. In accordance with Table 3, because the last message is classified as having a positive sentiment, symbol904(as described above with reference toFIG. 9) would be rendered via a GUI.

The thirtieth message (shown as point1060inFIG. 10) is classified as having a positive sentiment. The current rolling window of sentiment classifications no longer includes the sentiment classification of the twenty fourth message (i.e., point1048), but now includes the sentiment classification for the thirtieth message. The current rolling window of sentiment classifications is shown in the twelfth row of table1100B (i.e., “+++0++”). In accordance with Table 2 described above, because the sentiment intensity was “fully positive” and the last message (point1060) is classified as having a positive sentiment, the current real time sentiment is “fully positive,” and symbol802(as described above with reference toFIG. 8) would be rendered via a GUI. In accordance with Table 3, because the last two messages are classified as having a positive sentiment, symbol902(as described above with reference toFIG. 9) would be rendered via a GUI.

The thirty first message (shown as point1062inFIG. 10) is classified as having a positive sentiment. The current rolling window of sentiment classifications no longer includes the sentiment classification of the twenty fifth message (i.e., point1050), but now includes the sentiment classification for the thirty first message. The current rolling window of sentiment classifications is shown in the thirteenth row of table1100B (i.e., “++0+++”). In accordance with Table 2 described above, because the sentiment intensity was “fully positive” and the last message (point1062) is classified as having a positive sentiment, the current real time sentiment is “fully positive,” and symbol802(as described above with reference toFIG. 8) would be rendered via a GUI. In accordance with Table 3, because the last two (or more) messages are classified as having a positive sentiment, symbol902(as described above with reference toFIG. 9) would be rendered via a GUI.

The thirty second message (shown as point1064inFIG. 10) is classified as having a negative sentiment. The current rolling window of sentiment classifications no longer includes the sentiment classification of the twenty sixth message (i.e., point1052), but now includes the sentiment classification for the thirty second message. The current rolling window of sentiment classifications is shown in the fourteenth row of table1100B (i.e., “+0+++−”). In accordance with Table 2 described above, because the sentiment intensity was “fully positive” and the last message (point1064) is classified as having a negative sentiment, the current real time sentiment is “slightly negative,” and symbol810(as described above with reference toFIG. 8) would be rendered via a GUI. In accordance with Table 3, because the last message is classified as having a negative sentiment, symbol908(as described above with reference toFIG. 9) would be rendered via a GUI.

The thirty third message (shown as point1066inFIG. 10) is classified as having a positive sentiment. The current rolling window of sentiment classifications no longer includes the sentiment classification of the twenty seventh message (i.e., point1054), but now includes the sentiment classification for the thirty third message. The current rolling window of sentiment classifications is shown in the first row of table1100C (i.e., “0+++−+”). In accordance with Table 2 described above, because the sentiment intensity was “slightly negative” and the last message (point1066) is classified as having a positive sentiment, the current real time sentiment is “slightly positive,” and symbol806(as described above with reference toFIG. 8) would be rendered via a GUI. In accordance with Table 3, because the last message is classified as having a positive sentiment, symbol904(as described above with reference toFIG. 9) would be rendered via a GUI.

The thirty fourth message (shown as point1068inFIG. 10) is classified as having a positive sentiment. The current rolling window of sentiment classifications no longer includes the sentiment classification of the twenty eighth message (i.e., point1056), but now includes the sentiment classification for the thirty fourth message. The current rolling window of sentiment classifications is shown in the second row of table1100C (i.e., “+++−++”). In accordance with Table 2 described above, because the sentiment intensity was “slightly positive”, the last message (point1068) is classified as having a positive sentiment, and the number of messages in the rolling window that is classified as having a positive sentiment since the negative-to-slightly positive shift (as occurred via the thirty third message) is two, the current real time sentiment is “medium positive,” and symbol804(as described above with reference toFIG. 8) would be rendered via a GUI. In accordance with Table 3, because the last two messages are classified as having a positive sentiment, symbol902(as described above with reference toFIG. 9) would be rendered via a GUI.

The thirty fifth message (shown as point1070inFIG. 10) is classified as having a positive sentiment. The current rolling window of sentiment classifications no longer includes the sentiment classification of the twenty ninth message (i.e., point1060), but now includes the sentiment classification for the thirty fifth message. The current rolling window of sentiment classifications is shown in the third row of table1100C (i.e., “++−+++”). In accordance with Table 2 described above, because the sentiment intensity was “medium positive”, the last message (point1070) is classified as having a positive sentiment, and the number of messages in the rolling window that is classified as having a positive sentiment since the negative-to-slightly positive shift (as occurred via the thirty third message) is at least three, the current real time sentiment is “fully positive,” and symbol802(as described above with reference toFIG. 8) would be rendered via a GUI. In accordance with Table 3, because the last two (or more) messages are classified as having a positive sentiment, symbol902(as described above with reference toFIG. 9) would be rendered via a GUI.

The thirty sixth message (shown as point1072inFIG. 10) is classified as having a positive sentiment. The current rolling window of sentiment classifications no longer includes the sentiment classification of the thirtieth message (i.e., point1062), but now includes the sentiment classification for the thirty sixth message. The current rolling window of sentiment classifications is shown in the fourth row of table1100C (i.e., “+−++++”). In accordance with Table 2 described above, because the sentiment intensity was “fully positive” and the last message (point1072) is classified as having a positive sentiment, the current real time sentiment is “fully positive,” and symbol802(as described above with reference toFIG. 8) would be rendered via a GUI. In accordance with Table 3, because the last two (or more) messages are classified as having a positive sentiment, symbol902(as described above with reference toFIG. 9) would be rendered via a GUI.

The thirty seventh message (shown as point1074inFIG. 10) is classified as having a positive sentiment. The current rolling window of sentiment classifications no longer includes the sentiment classification of the thirty first message (i.e., point1064), but now includes the sentiment classification for the thirty seventh message. The current rolling window of sentiment classifications is shown in the fifth row of table1100C (i.e., “−+++++”). In accordance with Table 2 described above, because the sentiment intensity was “fully positive” and the last message (point1074) is classified as having a positive sentiment, the current real time sentiment is “fully positive,” and symbol802(as described above with reference toFIG. 8) would be rendered via a GUI. In accordance with Table 3, because the last two (or more) messages are classified as having a positive sentiment, symbol902(as described above with reference toFIG. 9) would be rendered via a GUI.

The final and thirty eighth message (shown as point1076inFIG. 10) is classified as having a positive sentiment. The current rolling window of sentiment classifications no longer includes the sentiment classification of the thirty second message (i.e., point1066), but now includes the sentiment classification for the thirty eighth message. The current rolling window of sentiment classifications is shown in the sixth row of table1100C (i.e., “++++++”). In accordance with Table 2 described above, because the sentiment intensity was “fully positive” and the last message (point1076) is classified as having a positive sentiment, the current real time sentiment is “fully positive,” and symbol802(as described above with reference toFIG. 8) would be rendered via a GUI. In accordance with Table 3, because the last two (or more) messages are classified as having a positive sentiment, symbol902(as described above with reference toFIG. 9) would be rendered via a GUI.

As shown inFIG. 10, there are no strong negative sections in this communication session and, as also shown inFIGS. 11A-11C, the sentiment score never goes below medium negative. Most likely supervisor intervention would occur at message count 14. Yet other candidates are message counts 6, 10, 22, 32 where sentiment is negative, and the sentiment is trending down.

As explained above, the GUI may simultaneously display the sentiment classification, score (and intensity thereof), and/or sentiment trend for a plurality of different communication sessions between different agents and customers. The GUI may be utilized by a manager of the agent to assess how well a communication session is going and to determine whether an intervention is required. For example,FIG. 12depicts an example GUI screen1200depicting real-time sentiment and sentiment trend for a plurality of different communication sessions in accordance with an example embodiment. As shown inFIG. 12, GUI screen1200includes a first display box1202, a second display box1204, and a third display box1206.

First display box1202displays information pertaining to an ongoing communication session between a first agent and a customer via a phone call. For example, first display box1202displays a username1208of the first agent (“cgriffith”), an indicator1210that displays the current length of the phone call (i.e., 7 minutes), a first symbol1212that displays the intensity of the customer's current sentiment during the ongoing phone call, and a second symbol1214that displays the sentiment trend during the ongoing phone call. First symbol1212is an example of symbol810, as described above with reference toFIG. 8, and second symbol1214is an example of symbol908, as described above with reference toFIG. 9. As shown inFIG. 12, the current sentiment intensity is “slightly negative,” and the sentiment trend is “down.”

Second display box1204displays information pertaining to ongoing communication sessions between a second agent and two different customers via different chat sessions. For example, second display box1204displays a username1216of the second agent (“sburke”), an indicator1218that displays the current length of the first chat session (i.e., 8 minutes), a first symbol1220that displays the intensity of the first customer's current sentiment during the first chat session, a second symbol1222that displays the current sentiment trend during the first chat session, an indicator1224that displays the current length of the second chat session (i.e., 27 minutes), a second symbol1226that displays the intensity of the second customer's current sentiment during the second chat session, and a second symbol1228that displays the current sentiment trend during the second chat session. First symbol1220is an example of symbol806, as described above with reference toFIG. 8, second symbol1222is an example of symbol904, as described above with reference toFIG. 9, first symbol1226is an example of symbol804, as described above with reference toFIG. 8, and second symbol1228is an example of symbol902, as described above with reference toFIG. 9. As shown inFIG. 12, the current sentiment intensity for the first chat session is “slightly positive,” and the sentiment trend is “Up”, and the current sentiment intensity for the second chat session is “medium positive,” and the sentiment trend is “Up”.

Third display box1206displays information pertaining to ongoing communication sessions between a third agent and two different customers via different chat sessions. For example, third display box1206displays a username1230of the third agent (“jlawson”), an indicator1232that displays the current length of the first chat session (i.e., 28 minutes), a first symbol1234that displays the intensity of the first customer's current sentiment during the first chat session, a second symbol1236that displays the current sentiment trend during the first chat session, an indicator1238that displays the current length of the second chat session (i.e., 17 minutes), a second symbol1240that displays the intensity of the second customer's current sentiment during the second chat session, and a second symbol1242that displays the current sentiment trend during the second chat session. First symbol1234is an example of symbol810, as described above with reference toFIG. 8, second symbol1236is an example of symbol908, as described above with reference toFIG. 9, first symbol1240is an example of symbol814, as described above with reference toFIG. 8, and second symbol1242is an example of symbol910, as described above with reference toFIG. 9. As shown inFIG. 12, the current sentiment intensity for the first chat session is “slightly negative,” and the sentiment trend is “Down”, and the current sentiment intensity for the second chat session is “fully negative,” and the sentiment trend is “Strong Down”.

D. Sentiment Shift and Driver(s) Thereof Determination Techniques

In the past, a lot of manual investigations were necessary to identify actionable insights into support sessions (what works well and where do the agents fail). While issue classification can identify the volume of certain types of issues, issue classification does not provide insight into support channel issues and does not quantify customer pain points.

The ability to identify negative sentiment drivers (also referred to as “breakpoints”) and positive sentiment drivers (also referred to as “connect points”) can be used to garner actionable insights into sentiment drivers, which may enable support agent supervisors to address support channel failures and build on positive sentiment drivers that can be applied more broadly.

Techniques described herein may enable support agent supervisors to identify active support sessions, breakpoints that negatively impact the session, and intervene in real time, equipped with the right insights to take effective corrective actions.

For example, referring again toFIG. 2, sentiment driver engine238may be configured to determine the driver in a customer's shift in sentiment. Sentiment driver engine238may be configured to determine the driver in response to a determination that the sentiment has shifted (e.g., from neutral to slightly positive, slightly positive to medium positive, medium positive to fully positive, neutral to slightly negative, slightly negative to medium negative, or medium negative to fully negative). For instance, batch scoring engine242may provide the sentiment score for each message in a communication session to sentiment driver engine238. In response, sentiment driver engine238may determine that a shift in sentiment occurred based on an increase (or decrease) in the sentiment score. For example, if the last message has a score of 61 (“slightly positive”), and the previous message has a score of 50 (“neutral”), then sentiment driver engine238may determine that a shift in sentiment has occurred (i.e., from a neutral sentiment to a slightly positive sentiment). When a shift in sentiment has been determined, sentiment driver engine238may be configured to analyze the last message and/or an N number of messages preceding the last message to determine the driver of that sentiment shift. The shift in sentiment may sometimes be driven based on what the customer has said and/or what the agent has said. Accordingly, sentiment driver engine238may analyze both the customer and agent messages to determine the driver of the sentiment shift.

The drivers in sentiment shift may be classified into a plurality of different categories. Such categories include, but are not limited to, a process issue, technology issues, agent issues, product issues, etc. Examples of process issues include, but are not limited to, subscription/service cancellation-related issues, subscription/service renewal-related issues, refund/return timeframe issues, wait time-related issues (e.g., the wait time to talk with an agent). Examples of technology issues include, but are not limited to, phone quality issues, phone/chat disconnections, audio issues, background noise issues, etc. Examples of agent issues include, but are not limited to, communication-related issues (e.g., is the agent hard to understand?, is the agent able to explain the issue coherently?, etc.), attentiveness of the agent, agent behavior (friendliness, rudeness, empathy (or lack thereof) towards the customer), etc. Examples of product issues include, but are not limited to, product (or product feature) satisfaction/dissatisfaction, integration with other devices, etc. Examples of policy issues include, but are not limited to, payment offerings, refund/exchange policy, etc.

Sentiment driver engine238may utilize natural language processing and/or machine-learning techniques to determine the driver of the sentiment shift during an ongoing communication session. For example, such techniques may classify certain words, phrases, sentences, etc., as being drivers of sentiment shift. In accordance with an embodiment, sentiment driver engine238may utilize a supervised machine learning algorithm to classify words, phrases, sentences, etc. The machine learning algorithm learns what types of words, phrases, sentences, etc. are representative of drivers of drivers of sentiment shift. The machine learning algorithm may be provided with training data that comprises positively-labeled words, phrases, sentences, etc., (i.e., labeled as being indicative of a driver of sentiment shift) and negatively-labeled words, phrases, sentences, etc., (i.e., labeled as not being indicative of a driver of sentiment shift). Using these inputs, the machine learning algorithm learns what constitutes a word, phrase, sentence, etc. as being indicative of a driver of sentiment shift and generates a classification model that is utilized to classify new words, phrases, sentences, etc. provided thereto as being indicative of a driver of sentiment shift. The classification model may output an indication (e.g., prediction or likelihood) as to whether the inputted word, phrase, sentence, etc., is indicative of being a driver of sentiment shift.

Sentiment driver engine238may be further configured to output (e.g., via a graphical user interface (GUI) (e.g., to be utilized by a call center manager) the message that caused the sentiment shift and/or the message that drove the sentiment shift during a communication session. For example,FIG. 13depicts an example GUI screen1300that provides an identification of a breakpoint message and an identification of the driver of that breakpoint message in accordance with an example embodiment. As shown inFIG. 13, GUI screen1300depicts the last 13 messages of an ongoing communication session. The customer sentiment shifts from neutral to slightly negative at message 13, as indicated by symbol1302. Accordingly, sentiment driver engine238identifies message 13 (“That is the conflict”) as the breakpoint message. As shown inFIG. 13, this identification is made via a check mark symbol1304in the “shift” column of table1300. As further shown inFIG. 13, sentiment driver engine238identifies message 12 (“Your system is not allowing me to merge with another existing business account”) as the driver of the sentiment shift. This identification is made via a check mark symbol1306in the “driver” column of table1300. It is noted that the other identification schemes may be used in addition to or in lieu of check mark symbols1304and1306. For example, the box containing message 12 may be highlighted a first color, and the box containing message 13 may be highlighted a second color that is different than the first color.

FIG. 14depicts an example GUI screen1400that provides an identification of a connect point message and an identification of the driver of that connect point message in accordance with an example embodiment. As shown inFIG. 14, GUI screen1400depicts the last 11 messages of an ongoing communication session. The customer sentiment shifts from neutral to slightly positive at message 11, as indicated by symbol1402. Accordingly, sentiment driver engine238identifies message 11 (“Wow! That's great news!”) as the connect point message. As shown inFIG. 14, this identification is made via a check mark symbol1404in the “shift” column of table1400. As further shown inFIG. 14, sentiment driver engine238identifies message 10 (“Ok, great! That is within our 30 day return policy window. I can handle the return for you.”) as the driver of the sentiment shift. This identification is made via a check mark symbol1406in the “driver” column of table1400. It is noted that the other identification schemes may be used in addition to or in lieu of check mark symbols1404and1406. For example, the box containing message 10 may be highlighted a first color, and the box containing message 11 may be highlighted a second color that is different than the first color.

E. Method for Sentiment Determination

FIG. 15depicts a flowchart1500for determining a customer's sentiment in accordance with an embodiment. The method of flowchart1500is described with continued reference toFIG. 2. Other structural and operational embodiments will be apparent to persons skilled in the relevant art(s) based on the following discussion regarding flowchart1500and system200.

Flowchart1500ofFIG. 15begins with step1502. At step1502, first messages from a first data source are received by a data acquisition engine. The first messages are pertaining to a first communication session between a first customer and a first agent. The data acquisition engine is further operable to receive second messages from a second data source, the second messages pertaining to a second communication session between a second customer and a second agent. For example, with reference toFIG. 2, data acquisition engine216receives first messages from a first data source (e.g., agent chat sessions204) and is operable to receive second messages from a second data source (e.g., voice data214).

In accordance with one or more embodiments, the first data source and the second data source comprise different ones of a first log comprising an Internet chat session between at least one of the first customer and the first agent or the second customer and the second agent (e.g., agent chat sessions204), one or more audio signals comprising a voice call between at least one of the first customer and the first agent or the second customer and the second agent (e.g., voice data214), a second log comprising one or more emails between at least one of the first customer and the first agent or the second customer and the second agent (e.g., emails212), and a third log comprising at least one of one or more postings from the first customer in a first social media forum or one or more postings from the second customer in a second social media forum (e.g., social media content210).

At step1504, first customer-provided messages of the first messages are isolated from first agent-provided messages of the first messages by a preprocessing engine. The preprocessing engine is further operable to isolate second customer-provided messages of the second messages from second agent-provided messages of the second messages. For example, with reference toFIG. 2, preprocessing engine222is configured to isolate first customer-provided messages of the first messages from first agent-provided messages of the first messages and is further operable to isolate second customer-provided messages of the second messages from second agent-provided messages of the second messages.

In accordance with an embodiment, step1504comprises isolating the first customer-provided messages from the first agent-provided messages during the first communication session.

At step1506, the first customer-provided messages of the first messages are interpreted by a customization engine in a manner specific to the first data source. The customization engine is further operable to interpret the second customer-provided messages of the second messages in a manner specific to the second data source. For example, with reference toFIG. 2, customization engine224interprets the first customer-provided messages of the first messages in a manner specific to the first data source and is further operable to interpret the second customer-provided messages of the second messages in a manner specific to the second data source.

In accordance with one or more embodiments, step1506comprises determining the first sentiment score during the first communication session.

At step1508, a first sentiment score for a first communication session is determined by a batch scoring engine based on the interpreted first customer-provided messages of the first messages. The batch scoring engine is further operable to determine a second sentiment score for the second communication session based on the interpreted second customer-provided messages of the second messages. For example, with reference toFIG. 2, batch scoring engine242is configured to determine a first sentiment score for the first communication session based on the interpreted first customer-provided messages of the first messages and is further operable to determine a second sentiment score for the second communication session based on the interpreted second customer-provided messages of the second messages.

In accordance with one or more embodiments, a corresponding first sentiment score is continuously determined for a rolling window of interpreted first customer-provided messages during the first communication session, the rolling window comprising a last N number of interpreted first customer-provided messages, where N is a positive integer. For example, with reference toFIG. 2, batch scoring engine242is configured to continuously determine a corresponding first sentiment score for a rolling window of interpreted first customer-provided messages during the first communication session.

In accordance with one or more embodiments, a first sentiment trend is continuously determined based on an analysis of interpreted first customer-provided messages of a plurality of rolling windows of the first communication session. For example, with reference toFIG. 2, batch scoring engine242is configured to continuously determine a first sentiment trend based on an analysis of interpreted first customer-provided messages of a plurality of rolling windows of the first communication session.

In accordance with one or more embodiments, a shift in sentiment of the first customer during the first communication is determined based on the analysis of interpreted first customer-provided messages of the plurality of rolling windows, and a driver of the determined shift is determined based on an analysis of at least one of the interpreted first customer-provided messages of the first communication session and the first agent-provided messages of the first communication session. For example, with reference toFIG. 2, sentiment driver engine238is configured to determine a shift in sentiment of the first customer during the first communication session based on the analysis of interpreted first customer-provided messages of the plurality of rolling windows and determine a driver of the determined shift based on an analysis of at least one of the interpreted first customer-provided messages of the first communication session and the first agent-provided messages of the first communication session.

In accordance with one or more embodiments, an application programming interface causes at least one of a first symbol representative of the first sentiment score and a second symbol representative of the first sentiment trend to be displayed via a graphical user interface during the first communication session, the first symbol being configured to indicate an intensity of the first sentiment score. For example, with reference toFIG. 2, application programming interface240is configured to cause at least one of a first symbol representative of the first sentiment score and a second symbol representative of the first sentiment trend to be displayed via a graphical user interface during the first communication session, the first symbol being configured to indicate an intensity of the first sentiment score.

In accordance with one or more embodiments, the graphical user interface is configured to display the first sentiment score and one or more second sentiment scores, each of the one or more second sentiment scores being associated with a communication session that is different than the first communication session and display the first sentiment trend and one or more second sentiment trends, each of the one or more second sentiment trends being associated with a communication session that is different than the first communication session.

III. Example Computer System Implementation

The systems and methods described above in reference toFIGS. 1-15, may be implemented in hardware, or hardware combined with one or both of software and/or firmware. For example, sentiment analysis engine118sentiment analysis engine200, data acquisition engine216, speech-to-text converter218, audio analyzer220, preprocessing engine222, customization engine224, agent chat model226, virtual agent model228, verbatim model230, social media model232, email model234, voice data model236, sentiment driver engine238, application programming interface240, batch scoring engine242, and governance and continuous improvement engine244, and flowchart1500may be each implemented as computer program code/instructions configured to be executed in one or more processors and stored in a computer readable storage medium. Alternatively, sentiment analysis engine118sentiment analysis engine200, data acquisition engine216, speech-to-text converter218, audio analyzer220, preprocessing engine222, customization engine224, agent chat model226, virtual agent model228, verbatim model230, social media model232, email model234, voice data model236, sentiment driver engine238, application programming interface240, batch scoring engine242, and governance and continuous improvement engine244, and flowchart1500may be implemented in one or more SoCs (system on chip). An SoC may include an integrated circuit chip that includes one or more of a processor (e.g., a central processing unit (CPU), microcontroller, microprocessor, digital signal processor (DSP), etc.), memory, one or more communication interfaces, and/or further circuits, and may optionally execute received program code and/or include embedded firmware to perform functions.

Furthermore,FIG. 16depicts an exemplary implementation of a computing device1600in which embodiments may be implemented, including nodes108A-108N,112A-112N,114A-114N, storage node(s)110, computing device104, sentiment analysis engine118sentiment analysis engine200, data acquisition engine216, speech-to-text converter218, audio analyzer220, preprocessing engine222, customization engine224, agent chat model226, virtual agent model228, verbatim model230, social media model232, email model234, voice data model236, sentiment driver engine238, application programming interface240, batch scoring engine242, and governance and continuous improvement engine244, and/or each of the components described therein, and flowchart1500. The description of computing device1500provided herein is provided for purposes of illustration, and is not intended to be limiting. Embodiments may be implemented in further types of computer systems, as would be known to persons skilled in the relevant art(s).

As shown inFIG. 16, computing device1600includes one or more processors, referred to as processor circuit1602, a system memory1604, and a bus1606that couples various system components including system memory1604to processor circuit1602. Processor circuit1602is an electrical and/or optical circuit implemented in one or more physical hardware electrical circuit device elements and/or integrated circuit devices (semiconductor material chips or dies) as a central processing unit (CPU), a microcontroller, a microprocessor, and/or other physical hardware processor circuit. Processor circuit1602may execute program code stored in a computer readable medium, such as program code of operating system1630, application programs1632, other programs1634, etc. Bus1606represents one or more of any of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, and a processor or local bus using any of a variety of bus architectures. System memory1604includes read only memory (ROM)1608and random access memory (RAM)1610. A basic input/output system1612(BIOS) is stored in ROM1608.

Computing device1600also has one or more of the following drives: a disk drive1614for reading from and writing to a hard disk or a solid state drive, a magnetic disk drive1616for reading from or writing to a removable magnetic disk1618, and an optical disk drive1620for reading from or writing to a removable optical disk1622such as a CD ROM, DVD ROM, or other optical media. Hard disk drive1614, magnetic disk drive1616, and optical disk drive1620are connected to bus1606by a hard disk drive interface1624, a magnetic disk drive interface1626, and an optical drive interface1628, respectively. The drives and their associated computer-readable media provide nonvolatile storage of computer-readable instructions, data structures, program modules and other data for the computer. Although a hard disk, a removable magnetic disk and a removable optical disk are described, other types of hardware-based computer-readable storage media can be used to store data, such as flash memory cards, digital video disks, RAMs, ROMs, and other hardware storage media.

A number of program modules may be stored on the hard disk, magnetic disk, optical disk, ROM, or RAM. These programs include operating system1630, one or more application programs1632, other programs1634, and program data1636. Application programs1632or other programs1634may include, for example, computer program logic (e.g., computer program code or instructions) for implementing the systems described above, including the sentiment determination techniques described in reference toFIGS. 1-15.

A user may enter commands and information into the computing device1600through input devices such as keyboard1638and pointing device1640. Other input devices (not shown) may include a microphone, joystick, game pad, satellite dish, scanner, a touch screen and/or touch pad, a voice recognition system to receive voice input, a gesture recognition system to receive gesture input, or the like. These and other input devices are often connected to processor circuit1602through a serial port interface1642that is coupled to bus1606, but may be connected by other interfaces, such as a parallel port, game port, or a universal serial bus (USB).

A display screen1644is also connected to bus1606via an interface, such as a video adapter1646. Display screen1644may be external to, or incorporated in computing device1600. Display screen1644may display information, as well as being a user interface for receiving user commands and/or other information (e.g., by touch, finger gestures, virtual keyboard, etc.). In addition to display screen1644, computing device1600may include other peripheral output devices (not shown) such as speakers and printers.

Computing device1600is connected to a network1648(e.g., the Internet) through an adaptor or network interface1650, a modem1652, or other means for establishing communications over the network. Modem1652, which may be internal or external, may be connected to bus1606via serial port interface1642, as shown inFIG. 16, or may be connected to bus1606using another interface type, including a parallel interface.

As noted above, computer programs and modules (including application programs1632and other programs1634) may be stored on the hard disk, magnetic disk, optical disk, ROM, RAM, or other hardware storage medium. Such computer programs may also be received via network interface1650, serial port interface1642, or any other interface type. Such computer programs, when executed or loaded by an application, enable computing device1600to implement features of embodiments discussed herein. Accordingly, such computer programs represent controllers of the computing device1600.

Embodiments described herein may also be implemented and/or executed via virtual or emulated hardware (e.g., virtual machines) and/or other environments, including, but not limited to, a cloud-computing based environment (e.g., Microsoft Azure by Microsoft Corporation®), such as the cloud-computing based environment described above with reference toFIG. 1.

A system is described herein. The system includes: at least one processor circuit; and at least one memory that stores program code configured to be executed by the at least one processor circuit, the program code comprising: a sentiment analysis engine, comprising: a data acquisition engine configured to receive first messages from a first data source and second messages from a second data source, the first messages pertaining to a first communication session between a first customer and a first agent and the second messages pertaining to a second communication session between a second customer and a second agent; a preprocessing engine configured to isolate first customer-provided messages of the first messages from first agent-provided messages of the first messages and isolate second customer-provided messages of the second messages from second agent-provided messages of the second messages; a customization engine configured to interpret the first customer-provided messages of the first messages in a manner specific to the first data source and interpret the second customer-provided messages of the second messages in a manner specific to the second data source; and a batch scoring engine configured to: determine a first sentiment score for the first communication session based on the interpreted first customer-provided messages of the first messages; and determine a second sentiment score for the second communication session based on the interpreted second customer-provided messages of the second messages.

In one embodiment of the foregoing system, the preprocessing engine is configured to isolate the first customer-provided messages from the first agent-provided messages during the first communication session, the customization engine is configured to interpret the first customer-provided messages during the first communication session, and the batch scoring engine is configured to determine the first sentiment score during the first communication session.

In another embodiment of the foregoing system, the batch scoring engine is further configured to continuously determine a corresponding first sentiment score for a rolling window of interpreted first customer-provided messages during the first communication session, the rolling window comprising a last N number of interpreted first customer-provided messages.

In a further embodiment of the foregoing system, the batch scoring engine is further configured to continuously determine a first sentiment trend based on an analysis of interpreted first customer-provided messages of a plurality of rolling windows of the first communication session.

In yet another embodiment of the foregoing system, the batch scoring engine is further configured to: determine a shift in sentiment of the first customer during the first communication session based on the analysis of interpreted first customer-provided messages of the plurality of rolling windows; and determine a driver of the determined shift based on an analysis of at least one of the interpreted first customer-provided messages of the first communication session and the first agent-provided messages of the first communication session.

In still another embodiment of the foregoing system, the system further comprises: an application programming interface configured to: cause at least one of a first symbol representative of the first sentiment score and a second symbol representative of the first sentiment trend to be displayed via a graphical user interface during the first communication session, the first symbol being configured to indicate an intensity of the first sentiment score.

In another embodiment of the foregoing system, the graphical user interface is configured to: display the first sentiment score and one or more second sentiment scores, each of the one or more second sentiment scores being associated with a communication session that is different than the first communication session; and display the first sentiment trend one or more second sentiment trends, each of the one or more second sentiment trends being associated with a communication session that is different than the first communication session.

In still another embodiment of the foregoing system, the first data source and the second data source comprise different ones of: a first log comprising an Internet chat session between at least one of the first customer and the first agent or the second customer and the second agent; one or more audio signals comprising a voice call between at least one of the first customer and the first agent or the second customer and the second agent; a second log comprising one or more emails between at least one of the first customer and the first agent or the second customer and the second agent; and a third log comprising at least one of one or more postings from the first customer in a first social media forum or one or more postings from the second customer in a second social media forum.

A method is also described herein. The method comprises: receiving, by a data acquisition engine, first messages from a first data source, the first messages pertaining to a first communication session between a first customer and a first agent via a first data source, the data acquisition engine being further operable to receive second messages from a second data source, the second messages pertaining to a second communication session between a second customer and a second agent via a second data source; isolating, by a preprocessing engine, first customer-provided messages of the first messages from first agent-provided messages of the first messages, the preprocessing engine being further operable to isolate second customer-provided messages of the second messages from second agent-provided messages of the second messages; interpreting, by a customization engine, the first customer-provided messages of the first messages in a manner specific to the first data source, the customization engine being further operable to interpret the second customer-provided messages of the second messages in a manner specific to the second data source; and determining, by a batch scoring engine, a first sentiment score for the first communication session based on the interpreted first customer-provided messages of the first messages, the batch scoring engine being further operable to determine a second sentiment score for the second communication session based on the interpreted second customer-provided messages of the second messages.

In one embodiment of the method, said isolating comprises: isolating the first customer-provided messages from the first agent-provided messages during the first communication session, wherein said determining comprises: determining the first sentiment score during the first communication session.

In another embodiment of the method, the method further comprises: continuously determining a corresponding first sentiment score for a rolling window of interpreted first customer-provided messages during the first communication session, the rolling window comprising a last N number of interpreted first customer-provided messages.

In yet another embodiment of the method, the method further comprises: continuously determining a first sentiment trend based on an analysis of interpreted first customer-provided messages of a plurality of rolling windows of the first communication session.

In still another embodiment of the method, the method further comprises: determining a shift in sentiment of the first customer during the first communication session based on the analysis of interpreted first customer-provided messages of the plurality of rolling windows; and determining a driver of the determined shift based on an analysis of at least one of the interpreted first customer-provided messages of the first communication session and the first agent-provided messages of the first communication session.

In another embodiment of the method, the method further comprises: causing, by an application programming interface, at least one of a first symbol representative of the first sentiment score and a second symbol representative of the first sentiment trend to be displayed via a graphical user interface during the first communication session, the first symbol being configured to indicate an intensity of the first sentiment score.

In still another embodiment of the method, the graphical user interface is configured to: display the first sentiment score and one or more second sentiment scores, each of the one or more second sentiment scores being associated with a communication session that is different than the first communication session; and display the first sentiment trend and one or more second sentiment trends, each of the one or more second sentiment trends being associated with a communication session that is different than the first communication session.

In yet another embodiment of the method, the first data source and the second data source comprise different ones of: a first log comprising an Internet chat session between at least one of the first customer and the first agent or the second customer and the second agent; one or more audio signals comprising a voice call between at least one of the first customer and the first agent or the second customer and the second agent; a second log comprising one or more emails between at least one of the first customer and the first agent or the second customer and the second agent; and a third log comprising at least one of one or more postings from the first customer in a first social media forum or one or more postings from the second customer in a second social media forum.

A computer-readable storage medium having program instructions recorded thereon that, when executed by at least one processor, perform a method is further described herein. The method comprises: receiving, by a data acquisition engine, first messages from a first data source, the first messages pertaining to a first communication session between a first customer and a first agent via a first data source, the data acquisition engine being further operable to receive second messages from a second data source, the second messages pertaining to a second communication session between a second customer and a second agent via a second data source; isolating, by a preprocessing engine, first customer-provided messages of the first messages from first agent-provided messages of the first messages, the preprocessing engine being further operable to isolate second customer-provided messages of the second messages from second agent-provided messages of the second messages; interpreting, by a customization engine, the first customer-provided messages of the first messages in a manner specific to the first data source, the customization engine being further operable to interpret the second customer-provided messages of the second messages in a manner specific to the second data source; and determining, by a batch scoring engine, a first sentiment score for the first communication session based on the interpreted first customer-provided messages of the first messages, the batch scoring engine being further operable to determine a second sentiment score for the second communication session based on the interpreted second customer-provided messages of the second messages.

In an embodiment of the computer-readable storage medium, said isolating comprises: isolating the first customer-provided messages from the first agent-provided messages during the first communication session, and said determining comprises: determining the first sentiment score during the first communication session.

In another embodiment of the computer-readable storage medium, the method further comprises: continuously determining a corresponding first sentiment score for a rolling window of interpreted first customer-provided messages during the first communication session, the rolling window comprising a last N number of interpreted first customer-provided messages.

In yet another embodiment of the computer-readable storage medium, the method further comprises: continuously determining a first sentiment trend based on an analysis of interpreted first customer-provided messages of a plurality of rolling windows of the first communication session.