Discovering the Undercurrent of a Conversation using AI: An NLP Model using Corpus of Interviews
By: Humans For Ai
Tags: Machine Learning, NLP, Sentiment Analysis, Conversations, Artificial Intelligence

By: Contributors

Anshuman Neog, Consultant|ML & DL, Affine Analytics,

Rayudu Pujitha, Data Scientist, Affine Analytics

Ever stepped out of a meeting and wondered how it went off? What if there was some intelligent software or gadget that synthesized your conversation and gave you a report on how it went — good, bad, ugly?

With Artificial Intelligence you can do just that — well almost!

The Power of NLP

Natural Language Processing (NLP) is a stream of artificial intelligence which deals with the ability of a computer to understand human natural language and its ability to process and analyse a large amount of natural language data.

( Read our earlier blog “NLP for All” for more details on the basics)

One of the basic foundation blocks of any AI is the process of making the software intelligent using data sets to train NLP algorithms and build an NLP data model which can then be shared with the community. Newbies to NLP, can use existing data-sets if one exists or create a new one if there doesn’t exist one — like in our case.

Training the NLP Algorithm — Building the NLP Model

Our objective is to study undercurrent of a long conversation and provide meaningful interpretation. For our particular scenario, we will be using a set of Interviews which are part of Earning Call Transcripts as our data-set to train the NLP algorithm and build the NLP Model.

The Challenge — Using Long Interviews as NLP Training Data

Double problem: your corpus is huge, and each document is an interview

The Approach we use here is to look at this data set as a classification problem, which means that using techniques of machine learning, the software will group together data by a particular criterion. Our aim will be to find this grouping or the classification.

Problems while solving a Classification Problem where your Corpus is a set of Interviews

(1) Large Text: You are dealing with a document with at least 5000 words. This is quite different from say when you are dealing with Twitter (average of 50 words). Imagine the chances of polarities being lost amidst the thousand words for a simple Sentiment Analysis

(2) Interviews do not contain discernible topics. There is a reason behind this. In any interview there will be information hiding. The interview will be bordering along Positive things. For example, if a CXO is being interviewed, rest assured the CXO will never say about negative topics. And even if a difficult question is put across to the CXO, the answer will be mostly placed in a positive manner

(3) Interviews are mostly Neutral

Most NLP problems can be viewed as having two aspects: The HOW aspect and the WHAT aspect

(1) The “How aspect”, how is the person replying to a question. Consider the case of a question being asked and the person is not confident about the answer versus the person is confident about the answer.

(2) The “What aspect”, what is the person talking about. The Topics which the person is talking about. Say for a CXO, is he talking about new Markets, Revenue, Loss etc.

The Interviews marvel in the “How” aspect.

We implemented this on Earning Call Transcripts, created features which captures the “How” aspect and then created a ML model which accurately predicts risk of Class Action Lawsuits given how an Interview went in the immediate Call Transcripts before the Class Action.

Feature Creation pitched to capture the how aspect:

We capture the “how” by creating Features which falls into the following Categories:

(1) Tonality

(2) Modality

(3) Complexity

(4) Sentiment Extraction on small snippets/ Chunks of the entire Interviews. These locations are determined using NLP processes as well as Domain Knowledge

Tonality:

Tone, in written composition, is an attitude of a writer toward a subject or an audience. Tone is generally conveyed through the choice of words, or the viewpoint of a writer on a specific subject. The tone can be formal, informal, serious, sarcastic, sad or cheerful, or it may be any other existing attitude. The formality of tone is the use of contractions, words like we’ve, I’d, they’re, and it’s. Using ‘it’s’ instead of ‘it is’ kind of informal tone.

Consider the following examples of tone:

“I want to ask the authorities what is the big deal? Why do they not control the epidemic? It is eating up lives like a monster.”

“I want to draw the attention of the appropriate authorities toward damage caused by the epidemic. If steps are not taken to curb it, it will further injure our community.”

The theme of both tone examples is the same. The only way we can differentiate between them is their separate tone. The tone in the first example is casual or informal while, it is more formal in the second

“Can someone tell me what the hell is going on here?” which has an aggressive tone.

“That’s great!!!” which has a cheerful tone and sometimes it can be sarcastic as well. So, it depends on the complete context of the paragraph.

Modality:

Modality is the semantic information associated with the speaker’s opinion on what was said. It captures the certainty, possibility, willingness, obligation and necessity from the text. It is the non — propositional part of the sentence which concerns to the factual status.

Low modality shows less certainty and high modality shows more certainty. Flags of high, medium, low modal words are created and based on these flags it calculates the number of words which have high modality, medium and low modality words in the transcript.

“I am almost certain; every investor has to look at their own situation and conclude the things that they think are best. I think the active devices are definitely” . In the above text words like ‘almost’ , ‘definitely’ are strong modal words

It is connected to act of ‘Assertives’, ‘Directives’, ‘Commissives’, ‘Declaratives’, ‘Expressives’

Assertive — ‘Socrates is bald’

Directives — ‘Pass the Mike’

Commissives — ‘I solemnly swear to truth’

Declaratives — I promise to complete it by today

Expressives — Thank you for giving me this opportunity

Let’s say, some of the tokens are given as ‘might happen’, ‘could possibly happen’, ‘will definitely happen’. Here the modality of sentence is increasing from low modality to high modality.

Complexity:

Complexity measures the standard of the text or how much text can be easily understandable. It is important to analyse the complexity of statements since it is difficult to capture the actual sense of the text when speaker is trying to hide information. For example, if analyst asks some difficult question, then the executive if wants to evade the question, the sentence structure is less likely to be straight forward and more likely to be complex. The complexity comes out with various different motivations at both micro and macro level of meaning. One motivation can be to divert the topic. But the most common will be as a human response to evade or feeling of uneasiness. They tend to use complicated, ambiguous words in their answers or make insignificant and sophisticated statements instead of giving more simple answers. For example, consider the following 2 sentences whose meaning is same but the difficulty of words is different. If the manager asks the employee about the progress of the task. It can be as follows

‘I completed the task and will forward it to you immediately’ — which is less complex and more clear

‘The task is quite strenuous, but I am almost done with that and going through it to so there won’t be any white spots in it’ — using unnecessary words and hiding information

Here, we are using readability index to measure complexity of the text. It can also be captured using quantitative features like

· Word length (e.g., simple versus multisyllabic words)

· Frequency of unfamiliar or new vocabulary terms

· Sentence length and syllables used

· Text cohesion

Qualitative evaluation of the text can be possible by understanding the Levels of meaning, structure, language clarity.

Sentiment Analysis of specific sections of the text

Sentiment is the opinion of the speaker on certain topic. Many techniques are being evolved on how to extract the sentiment from text but as discussed it is difficult to extract sentiment for long discussions/ Interviews. By normal sentiment analysis it is not possible to get the exact views but a very Strategic Sentiment Analysis must be performed.

Ex: The company performed very well in last 2 quarters and the revenue has been increased significantly. — tells positively about the company

One of the approaches is splitting the whole text into small paragraphs / smaller chunks based on some predefined ruleset and extract SA on them individually which makes more robust. Defining ruleset is a very important and proper, and it will be different for different kinds of problem/text.

For example, let’s take extracting the sentiment from interview between the Analyst and CXO’s of the company. Our hypothesis is that CXO’s will answer positively about his company even if the analyst asked the complex question using some negative terms about the company. Similarly, if we compare answers given by CEO and Vice President, we assume that the answers given by CEO will be stronger and more positive without hesitation and less use of unnecessary words when compared with the answer given by Vice President. These can be the set of rules while extracting the information from interviews and this rule set will vary according the problem statement

Conclusion:

As discussed above, when it comes to extracting information from interviews, “how” aspect matters a lot. In fact, interviews especially an interview related to financial health of the company, there will be a lot of disclosure on the bad aspects. This indirectly means that extracting the “What” aspect which comes via Topic Modelling is not so reasonable. However, if we address “how the replies are being given?” and extract information along the same, we can engineer a ton load of Features which we can use in creation of ML/ DL model. This is captured by Tonality, Modality and Complexity and Sentiment on important parts of document.

Note: For each of Modality, Tone and Complexity there will be multiple features.

References:

Modality

(1) https://www.det.nsw.edu.au/eppcontent/glossary/app/resource/factsheet/4091.pdf

(2) https://www.slideshare.net/merykul95/modality-61463910

Tone of text

(3) https://literarydevices.net/tone/

Complexity of text

(4) https://iris.peabody.vanderbilt.edu/module/sec-rdng2/cr_assess/#content

(5) http://missionliteracy.com/complex-text.html

(6) https://www.sc3ta.org/downloads/CommonCore4ELL/Mini_Lesson_7-Text_Complexity.pdf

About Affine

Affine has been a provider of analytical solutions, working with global organizations solving their strategic and day to day business problems.

We combine data-driven statistical and AI analysis with heuristic domain expertise to provide actionable insights empowering organizations to make informed decisions and take proactive action. Our value lies in enabling our clientele to implement and realize the ROI of the recommendations.

About Humans For AI

Humans For AI is a non-profit organization focused on building a more diverse workforce for the future leveraging AI technologies. Learn more about us and join us as we embark on this journey to make a difference!