LANGUAGE AND SENTIMENT ANALYSIS FOR GENERATING SUPPORT SUMMARIES

A method of generating a support summary includes extracting chat information including natural language data from a support chat with a user. Tokenized language is generated by performing feature extraction on this natural language data. This tokenized language is subjected to sentiment analysis to produce sentiment data reflecting sentiment of the support user during the support chat, and to semantic analysis to extract support-relevant features. A support summary made up of natural language text identifying a support issue and information germane to the support issue is then generated from the extracted chat information using a language model, at least in part from the extracted support-relevant features and the sentiment data.

FIELD OF THE INVENTION

The present disclosure relates generally to language processing, and more particularly to the generation and distillation of natural language information for technical support into a support summary using language model.

BACKGROUND

Users in need of technical support often have multiple interactions with support providers, both in the course of resolving or attempting to resolve a single issue, and more broadly over the course of a history of support interactions. Users can experience the same or related issues on multiple occasions, or issues for which a common set of facts are relevant. In many instances, at least some interactions between support providers and users take the form of automated troubleshooting or information collection wherein a user is prompted to answer questions (or provide information, more broadly) relevant to their support issue. In instances where automated support tools are unavailable or insufficient to address user needs or questions, a support query issue can be escalated to a human technician. In many cases, support technicians record notes on issues, tests, dispositions, and other information relevant to current and/or future support issues while or after attempting to address support issues. These notes can be useful for future tech support, especially when a different support technician may be involved.

When a human support technician is first introduced to a support issue, it is common for that technician to review at least a portion of prior user interactions, technician notes, and other user- or project-specific information to best assess and address the new support issue. This review process is time consuming and vulnerable to human error, especially when the support technician must attempt to extract relevant information quickly from a large volume of historical data. Support technicians may repeat questions or request data that users have already provided, but doing so risks increasing user impatience and frustration.

SUMMARY

This disclosure presents a method of generating a support summary from extracting chat information including natural language data from a support chat with a user. According to this method, tokenized language is generated by tokenizing the natural language data. This tokenized language is subjected to sentiment analysis to produce sentiment data reflecting sentiment of the support user during the support chat, and to semantic analysis to extract support-relevant features. A support summary made up of natural language text identifying a support issue and information germane to the support issue is then generated from the extracted chat information using a language model, at least in part from the extracted support-relevant features and the sentiment data.

This disclosure also presents a system for generating a support summary from a natural language chat record of a support interaction with a support user. This support summary includes natural language text identifying a support problem and information germane to the support issue. The system includes a tokenization module, a sentiment extraction module, a semantic analysis module, and a language model. The tokenization module is configured to tokenize the natural language chat record, and the sentiment identification module is configured to produce sentiment data reflecting sentiment of the support user during the support interaction from the tokenized natural language chat record. The semantic analysis module is configured to extract support-relevant features from the tokenized natural language chat record, the support-relevant features including at least one support topic. The language model is configured to generate the support summary from the support-relevant features and the sentiment data.

DETAILED DESCRIPTION

The present disclosure provides multiple embodiments of systems and an accompanying generalizable method for improving technical support by producing a support summary for review by support technicians before and while assisting users with support issues. Specifically, this support summary is a natural language report of support-relevant information generated using a language model from semantic and sentiment analyses of historical text data (including user chat data). This historical text data can include chat transcripts from human and/or automated support chats with the user requesting technical assistance. By collecting and distilling data from prior user interactions into a summary of support-relevant data, the systems and method described herein facilitate faster and more accurate catchup by the support technician with respect to information already provided by the user, improving support efficiency and reducing likelihood of user frustration or impatience.

FIGS.1and2are schematic module diagrams of technical support systems100and200, respectively. Technical support system200generally parallels technical support system100except where otherwise specified. More specifically, although the following description is framed largely in terms of technical support system100, elements2xxof technical support system200generally operate similarly to corresponding elements1xxof technical support system100, with exceptions as noted below. Where elements ofFIG.2are not separately described, description of corresponding elements ofFIG.1may be presumed to carry over.

Technical support system100includes support summary system102accessible to human operators104and including processor106, user interface108, and memory110. Human operators104can includes support technicians, support-seeking users, and others (e.g. system administrators) who provide information used by, or receive information generated by, support system102via user interface108. Support system102is a system capable of generating a summary of support-relevant information from historical data, as described above. Although only a single support summary system102is illustrated inFIG.1, some embodiments of support system102can include multiple parallel modules instantiated or distributed across separate hardware.

User interface108is an input and/or output device and/or software interface, and enables human operator104to input and/or output data and/or otherwise interact with support summary system102. For example, user interface108can be configured to receive text inputs from a support user including support chat, and to provide a support technician with a support summary generated by support summary system102. In some embodiments user interface108can include one or more direct input/output devices facilitating input and/or output of information in a form understandable to users and/or machines, such as a sound card, a video graphics card, a speaker, a display device (such as a liquid crystal display (LCD), a light emitting diode (LED) display, an organic light emitting diode (OLED) display, etc.), a touchscreen, a keyboard, a mouse, or a joystick. In further embodiments, user interface108can additionally or alternatively include one or more wired or wireless transceivers capable of receiving digital communications generated via such input/output devices. More specifically, in the illustrated embodiment user interface108can optionally include audiovisual capture128such as a camera and/or microphone. For the purposes of this description, audiovisual capture128enables user interface108to record voice and/or facial expression information of a support user, as noted below.

Processor106is a logic-capable device or group of devices capable of executing software, applications, and/or programs stored on memory110. Examples of processor106can include one or more of a processor, a microprocessor, a controller, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other equivalent discrete or integrated logic circuitry. Processor106can be entirely or partially mounted on one or more circuit boards.

Memory110is a machine-readable storage medium or media configured to store information and accessible to processor106. In some examples, a machine-readable storage medium can include a non-transitory medium. The term “non-transitory” can indicate that the storage medium is not embodied in a carrier wave or a propagated signal. In certain examples, a non-transitory storage medium can store data that can, over time, change (e.g., in RAM or cache). In some examples, memory110is or includes a temporary memory. As used herein, a temporary memory refers to a memory having a primary purpose that is not long-term storage. In an exemplary embodiment, at least a portion of memory110is described as volatile memory. As used herein, a volatile memory refers to a memory that that the memory does not maintain stored contents when power to the Memory110is turned off. Examples of volatile memories can include random access memories (RAM), dynamic random access memories (DRAM), static random access memories (SRAM), and other forms of volatile memories. In the illustrated embodiment, memory110is used to store program instructions for execution by processor106, and is used by software and/or applications running on support summary system102to temporarily store information during program execution.

In some embodiments memory110can also include non-volatile machine-readable memory for long-term storage of information. Examples of such non-volatile storage elements can include, for example, magnetic hard discs, optical discs, floppy discs, flash memories, or forms of electrically programmable memories (EPROM) or electrically erasable and programmable (EEPROM) memories. Although memory110is illustrated inFIG.1as a single module situated within support summary system102, this module can be distributed across multiple physical locations, and long-term storage elements of memory110can include cloud-accessible data storage. In at least some examples, memory110can house support-relevant information stored in databases. For example, data stored in memory110can be structured in relational databases and can organize data using a relational database management system (RDBMS), object-relational database management system (ORDBMS), columnar database management systems (CDBMS), document-oriented database management systems (DoDBMS) and/or a multi-model database management system (MMDBMS), among other options.

FIG.1illustrates several modules of support summary system110implemented in memory110by processor106, including text record112, tokenization module118, sentiment analysis module120, semantic analysis module122, language model124, and summary output126. These modules constitute specific data sets (text record112, summary output126) and/or software program components (tokenization module118, sentiment analysis module120, semantic analysis module122, language model124) instantiated in memory110and defined functionally for the sake of this description.

Text record112is an archive of text processed by support summary system102to generate a support summary for consumption by a support technician. Text record112can, for example, include chat information114, technical notes116, and other text-based information. Chat information114can, for example, constitute one or more transcripts of natural language chat between a support user and an automated system, a support technician, or both. Chat information114can include text strings provided as text by a support user and support technician or automated tool. In further embodiments, some or all of chat information114can consist of a text transcript of audio or video chat with the support user. In some embodiments chat information114can include only recent chat, e.g. a natural language transcript of chat directed to a current support issue. In other embodiments chat information114can also include older historical chat records potentially reflecting separate (and potentially already resolved) support issues. Chat information114can include user identifications of current support issues requiring resolution, i.e. questions or problems with which the support user seeks assistance from technical support system100. Such identifications need not be explicitly separated from other chat information114. Text record112can also include technical notes116recorded by one or more support technicians while or after providing support assistance to the support user, e.g. to summarize open questions, actions taken or pending, possible and excluded support diagnoses, and other commentary. In some embodiments technical notes116, like chat information114, can take the form of natural language text related to a current technical support issue, or related to older historical technical support issues which may have already been resolved. In additional to records of natural language, text record112can include timestamps (i.e. dates and times) associated with each record (i.e. each chat message, support note, or other text input received via input device108). In some embodiments, some or all records can also be labeled with related information, such as names and/or roles of participants (e.g. specific support technicians or users) or the status of a related support ticket (i.e.

open or closed). Aside from this labeling, however, the contents of text record112can be principally strings of natural language.

Tokenization module118is a pre-processing tool configured to extract semantically significant text from the contents of text record112. Specifically, each chat message, note, or other text element is tokenized, i.e. broken down into parts of speech (noun, verb, adverb, etc.) based on words and structure. Conjunctions and articles can then, in some embodiments, be discarded. Each remaining word is then stemmed, i.e. truncated to its root word—“running” to “run,” for example. These tokenized, stemmed words are then filtered to remove stop words or phrases, i.e. phrases that are merely responsive rather than additive to the semantic content of the selected portion of conversation. Stemming, tokenizing, and stop word removal can be performed using known approaches, e.g. with publicly available libraries such as the StanfordNLP Python library made available by the Stanford Natural Language Processing group. In some exemplary embodiments, the extraction of relevant text can include simplification and potentially tokenization of idiomatic language or non-textual semantic content such as emojis and/or stamps, e.g. by reference to an incorporated library or libraries (not separately shown) also present in memory110. In some embodiments, preprocessing performed by tokenization module118can also include noise removal, lowercasing, normalization, and/or lemmatization. Regardless of the particular model(s) used, tokenization module118transforms natural language from text record112into tokenized language.

Tokenized language generated by tokenization module118is evaluated by sentiment analysis module120. Sentiment analysis module120assigns a user sentiment to elements of chat information114, i.e. to some or each chat information record. User sentiment can, for example, be binary (i.e. positive/negative) or trinary (positive/negative/neutral), and/or can include additional qualitative characterizations of apparent user sentiment (e.g. angry, frustrated, relieved). In a non-limited example, sentiment identification can be performed using VADER NLTK or textblob semantic labeling. In some embodiments video and/or audio of support user interactions can also be captured via audiovisual capture128and used to supplement identification of user sentiment, e.g. by facial expression recognition, voice waveform and/or spectrogram analysis, and other means known in the art. Where chat information114and/or audiovisual information is timestamped, sentiment analysis120can also identify sentiment transitions representing changes in user sentiment (e.g. positive-to-negative, frustrated-to-relieved) associated with particular text chat information records. In an illustrative example, sentiment analysis module120may label initial user chat records as displaying negative sentiment, reflecting user frustration or anxiety related to the support issue prompting the request for technical support. After resolution of this issue, subsequent user sentiment may shift to positive, reflecting user satisfaction or relief. In this example, sentiment analysis module120is capable of flagging the negative-to-positive sentiment transition associated with the message and timestamp corresponding to the resolution of the support issue.

In some embodiments support summary system102can also include semantic analysis module122. Semantic analysis module122is an NLP-based filter capable of evaluating tokenized language provided by tokenization module118to identify support-relevant features and/or exclude support-irrelevant features, i.e. to distinguish information germane to potential support issues based on semantic content. Semantic analysis module122can, for example, perform topic modelling or topic classification based on supervised or unsupervised machine learning. In an illustrative embodiment, semantic analysis module122only evaluates relevance to technical support topics generally, and does not filter text based on identification of any specific support issue. In this embodiments, semantic analysis module122might, for example, identify a side conversation regarding sports or a pet as not germane to support topics. In another embodiment, semantic analysis module122can be configured to flag specific tokenized language provided by tokenization module118as either (1) identifying a possible support issue; or (2) related to an identified support issue. Semantic analysis module122can, for example, be a tokenization module configured to identify semantic features within tokenized language by means of feature extraction techniques such as bag-of-words (BOW) analysis, bag-of-n-grams analysis, term frequency-inverse document frequency (TF-IDF) vectorization analysis, One Hot encoding, or other known techniques. These features can, for example by highly predictive or support relevant features, where semantic analysis module122is training via supervised or unsupervised machine learning to identify “key” support features within text.

Support summary system102also includes language model124. While support summary system202has not been separately described, elements of technical support system200generally mirror corresponding elements of technical support system100described up to this point. As noted below, however, language model224functions somewhat differently in support summary system202from language model124in support summary system102. Both language model124and language model224generate a summary of a support issue and information germane to the support issue in the form of a natural language text provided as summary output126/226.

Language models124/224include one or more computer-implemented machine learning models configured to generate natural language text data in response to language inputs. The computer-implemented machine learning model(s) of language models124/224can be referred to as a generative AI. In some examples, language models124/224are large language models trained at least in part through semi-supervised or self-supervised learning on large quantities of human-generated text. Where language models124/224are large language models, language model160can be, for example, a deep learning model, such as a deep neural network. It at least some examples, language models124/224are trained transformer models or transformer networks.

As illustrated inFIG.1, language model124receives tokenized language extracted from text record112as processed through sentiment analysis120(to assign sentiment values to tokenized text) and semantic analysis122(to identify support-relevant features). Thus, language model124generates summary output126on the basis of tokenized text with labeling provided by these analysis modules. In some embodiments of support summary system102, language model124need not be capable of ingesting natural language, and serves only to assemble a natural language summary of tokenized text information tagged or labeled based on the outputs of sentiment and semantic modules120,122. Labeling by sentiment analysis module120can, for example, flag text associated with positive or negative emotional states of the user, or with sentiment transitions potentially indicative of resolutions or problems during communication with the user. User sentiment can also be summarized directly in summary output126(e.g. “user has grown increasingly frustrated”) for provision to the support technician. Labeling by semantic analysis module122can, for example, identify text data for inclusion in or exclusion from summary output126based on general support relevance or relevance to specific support issues.

As illustrated inFIG.2, language model224receives natural language from text record212(with or without text tokenization by tokenization module218), and assembles summary output226directly therefrom. To assist language model224in identifying support issues and information germane to those issues, sentiment outputs from sentiment analysis module220and support-relevance outputs from semantic analysis module222can be supplied to language model224for retrieval-augmented generation (RAG) or other forms of context injection, to define boundary conditions or otherwise constrain outputs of language model224. In particular, sentiment analyses from sentiment analysis module220can be used to flag transformative interactions from within chat information214on the basis of detected sentiment or transitions in sentiment, and semantic analyses from semantic analysis module222can be used to flag specific support issues and information germane thereto. Although tokenization modules118/218, sentiment analysis modules120/220, semantic analysis modules122/222, and language models124/224are presented herein as discrete components of support summary system102/204, the functions represented by these components can in some embodiments be performed together, or in different orders. Functions of sentiment analysis modules120/220and semantic analysis modules122/222, for example, can be performed in parallel (as shown) or in series, in any order. Similarly, though sentiment and semantic analysis modules120/220and122/222are illustrated as separate from language models124/224, some embodiments of support summary systems102/202may integrate semantic and/or sentiment analysis into a single generative AI trained to incorporate semantic and/or sentiment analyses generally as described above.

Summary output126or226, produced by language model124or224, respectively, provides a summary in natural language of support-relevant portions of text record112. Support output126/226is tailored to distil information for a support technician so as to allow the technician to quickly become familiar with any technical issues and relevant information already captured in text record112/212before and while engaging a user to provide technical support. By so doing, technical support systems100/200shorten times and avoid repetitions in technical support resolutions for both users and technicians, improving technician efficiency and improving user satisfaction with the technical support process.

FIG.3is a method flowchart and serves mainly as a recap of tech support methods already discussed above. Specifically,FIG.3illustrates support method300for generating and using support summaries using technical support systems100and/or200.

As described above, support activity reflecting user interactions and other support-relevant information is received via user interface108/208and recorded in memory110/210(Step302), and used to update a support record (Step304). More specifically, text record112/212is updated to include natural language chat text between the user and an automated support system and/or human technician is recorded in memory110/210. (Step304a) In some embodiments, audio and/or video data of support interactions with the user can also be recoded, e.g. for transcription into text for text record112/212and/or for analysis of user sentiment. (Step304b). Natural language in text record112/212is then tokenized by tokenization module118/218. (Step306). For each support chat record, this tokenized language is used by semantic analysis module122/222to generate content-based relevance assessments (Step308), and by sentiment analysis module120/220to generate identifications of associated sentiment (Step310). Language model124/224produces a support summary as described above using the sentiment identifications provided by sentiment analysis module120/220and the relevance analysis generated by semantic analysis module122/222, the tokenized text produced by tokenization module118/218, and in some embodiments (seeFIG.2) the natural language support chats recorded in chat information214of text record212. A support technician reviews this support summary (Step314) and provides technical support to the user, accordingly (Step316). During and after providing technical support, the support record may be updated again to reflect the additional text (and audiovisual information, if available) from this support interaction, and to reflect technical notes and other records added manually by the support technician. This updating ensures that text record112remains up-to-date for future technical support relating to the same or different support issues.

As noted above with respect toFIGS.1and2, the operation of technical support system100/200in steps302-312enables more efficient ingestion of historical information by a support technician, such that step314is simultaneously faster and more comprehensive than manual review, allowing technical support (Step316) to proceed quickly and efficiently, without undue delay or cause for user frustration.

Discussion of Possible Embodiments

A method of generating a support summary, the method comprising: extracting chat information from a support chat with a support user via a processor, the chat information including natural language data; tokenizing the natural language data to generate tokenized language; performing sentiment analysis on the tokenized language to produce sentiment data reflecting sentiment of the support user during the support chat; performing semantic analysis on the tokenized language to extract support-relevant features; producing the support summary using a language model, wherein: the support summary comprises a natural language text identifying a support issue and information germane to the support issue, from the extracted chat information; and the language model produces the support summary at least in part from the extracted support-relevant features and the sentiment data.

A further embodiment of the foregoing method, further comprising: recording audiovisual (AV) data including audio and/or video corresponding from the support chat with the support user; and generating the sentiment identification based in part on the AV data.

A further embodiment of the foregoing method, wherein performing sentiment analysis on the AV data comprises: extracting audio waveform data from the AV data; and generating the sentiment identification based in part on the extracted audio waveform data.

A further embodiment of the foregoing method, wherein performing sentiment analysis on the AV data comprises: extracting facial expression information from the AV data; and generating the sentiment identification based in part on the facial expression information.

A further embodiment of the foregoing method, wherein the sentiment data comprises: identification of a plurality of user sentiments expressed by the support user during the support chat; and identification of a sentiment transition between the plurality of user sentiments. A further embodiment of the foregoing method, wherein the support summary includes an identification of user sentiment.

A further embodiment of the foregoing method, wherein producing the support summary using the language model comprises correlating the user sentiment and a change in the user sentiment with at least one chat string included among the chat information.

A further embodiment of the foregoing method, further comprising performing feature extraction on natural language technical notes recorded by a support technician, and the language model produces the support summary at least in part based on features extracted from the natural language technical notes.

A further embodiment of the foregoing method, wherein the language model produces the support summary at least in part from the extracted support-relevant features, the sentiment data, and the natural language data.

A further embodiment of the foregoing method, wherein the language model receives the extracted support-relevant features and the sentiment data in the form of context injection for the generative production of the support summary.

A further embodiment of the foregoing method, wherein performing semantic analysis on the tokenized language comprises classifying the tokenized language according to semantic features.

A further embodiment of the foregoing method, further comprising performing feature extraction on the chat information to flag the semantic features, wherein the feature extraction includes at least one of bag-of-words (BOW) analysis, bag-of-n-grams analysis, term frequency-inverse document frequency (TF-IDF) vectorization analysis, and One Hot encoding.

A further embodiment of the foregoing method, wherein the language model is a large language model, and wherein producing the support summary using the language model comprises generating the support summary at least in part from the extracted chat information.

A further embodiment of the foregoing method, wherein the generation of the support summary is constrained by retrieval augmented generation (RAG) using at least one of the sentiment data and the support-relevant features.

A method of providing technical support to a user, the method comprising: generating a support summary according to the foregoing method; reviewing the support summary; providing technical support related to the support issue identified in the support summary, to the user.

A further embodiment of the foregoing method, wherein the language model produces the support summary at least in part base on features extracted from natural language technical notes, the method further comprising updating the technical notes while or after providing the technical support to the user, the technical notes including at least one of open questions, actions taken or pending, and possible or excluded diagnoses.

A system for generating a support summary from a natural language chat record of a support interaction with a support user, the support summary comprising natural language text identifying a support issue and information germane to the support issue, the system comprising: a tokenization module configured to tokenize the natural language chat record; a sentiment identification module configured to produce sentiment data from the tokenized natural language chat record, the sentiment data reflecting sentiment of the support user during the support interaction; a semantic analysis module configured to extract support-relevant features from the tokenized natural language chat record; and a language model configured to generate the support summary from the support-relevant features and the sentiment data.

A further embodiment of the foregoing system, wherein the support-relevant features include at least one support issue, and the support summary identifies the support issue.

A further embodiment of the foregoing system, wherein the language model is a large language model configured to generate the support summary from the natural language chat record using the sentiment data and the support-relevant features for context injection.

A further embodiment of the foregoing system, further comprising an audiovisual capture device configured to capture audio or video data corresponding to the support interaction with the support user, wherein the sentiment identification is configured to produce the sentiment data at least in part from the captured audio or video data.

Summation

Any relative terms or terms of degree used herein, such as “substantially”, “essentially”, “generally”, “approximately” and the like, should be interpreted in accordance with and subject to any applicable definitions or limits expressly stated herein. In all instances, any relative terms or terms of degree used herein should be interpreted to broadly encompass any relevant disclosed embodiments as well as such ranges or variations as would be understood by a person of ordinary skill in the art in view of the entirety of the present disclosure, such as to encompass ordinary manufacturing tolerance variations, incidental alignment variations, alignment or shape variations induced by thermal, rotational or vibrational operational conditions, and the like.