“Am I Answering My Job Interview Questions Right?”: A NLP Approach to Predict Degree of Explanation in Job Interview Responses (2022)

Abstract: Providing the right amount of explanation in an employment interview can help the interviewee effectively communicate their skills and experience to the interviewer and convince the she/he is the right candidate for the job. This paper examines natural language processing (NLP) approaches, including word-based tokenization, lexicon-based representations, and pre-trained embeddings with deep learning models, for detecting the degree of explanation in a job interview response. These are exemplified in a study of 24 military veterans who are the focal group of this study, since they can experience unique challenges in job interviews due to the unique verbal communication style that is prevalent in the military. Military veterans conducted mock interviews with industry recruiters and data from these interviews were transcribed and analyzed. Results indicate that the feasibility of automated NLP methods for detecting the degree of explanation in an interview response. Features based on tokenizer analysis are the most effective in detecting under-explained responses (i.e., 0.29 F1-score), while lexicon-based methods depict the higher performance in detecting over-explanation (i.e., 0.51 F1-score). Findings from this work lay the foundation for the design of intelligent assistive technologies that can provide personalized learning pathways to job candidates, especially those belonging to sensitive or under-represented populations, and helping them succeed in employment job interviews, ultimately contributing to an inclusive workforce.

## introduction artificial intelligence (ai) can empower a plethora of assistive tools for enhancing one's visual, hearing, communication, cognitive, and motor skills @xcite . by automating natural language processing (nlp) and understanding, ai technologies can enable individuals who belong to sensitive populations, to better express themselves or better understand the world around them. intelligent interview training is one such technology that can facilitate training in a safe environment on specific verbal and nonverbal behaviors and can help individuals effectively adapt to cognitively demanding and socially challenging interview situations @xcite . this technology can further contribute to an inclusive workforce. since the employment interview comprises the first step of the job hiring process, intelligent interview training augmented with nlp can detect linguistic and semantic communicative behaviors that might jeopardize candidates' performance in the interview, suggest the exact modifications needed to effectively communicate their skills, and facilitate access to training material and information in a personalized manner @xcite . military veterans is a group that can particularly benefit from assistive interview training technologies. in many countries around the world, military veterans face major barriers to participating in the civilian workforce after separation from active duty @xcite . the military background and training of most veterans is significantly different compared to the general job candidate population, who usually comprise of relatively younger fresh college graduates. military veterans often find it challenging to clearly articulate their strengths and "brag" about their achievements in the civilian employment interview setting. particularly, they can experience unique verbal communication gaps, such as ineffective translation of relevant military experience and technical skills, over-explaining their responses, and excessive use of military jargon, that hamper them from successfully obtaining a job in the civilian workforce @xcite . intelligent job interview training systems can potentially track these linguistic behaviors of interest and provide military veterans the right feedback at the right time. we conduct a linguistic analysis of veterans' responses in civilian interview settings. we focus on the degree of explanation in the response, since this construct is particularly relevant to the interview success and unexplored by previous work, and particularly we investigate a range of nlp systems to detect over/under-explained, succinct, and comprehensive responses @xcite . to accomplish this task, we examine nlp systems that rely on text tokenization, lexicon-based analysis, and deep learning methods. these are evaluated on transcripts from mock interviews between 24 military veterans and 5 industry recruiters. a total of 163 responses provided during the interviews were coded by third-party annotators with respect to the degree of explanation. results indicate the feasibility of automated nlp analysis for detecting the outcome of interest. particularly, features based on tokenizer analysis are the most effective in detecting under-explained responses (i.e., 0.29 f1-score), while lexicon-based methods depict the higher performance in detecting over-explanation (i.e., 0.51 f1-score). challenges that were met during data analysis, namely, the small data sample, subjectivity in coding, and uneven class distributions, are described. discussion of these results further provides ways in which the proposed nlp analysis can contribute to the design of assistive technologies for interview training. ## related work prior work in assistive technologies for interview training has focused on helping users demonstrate effective social skills and positive personality cues. the tardis project, for example, designed a game simulation platform through which interviewees interacted with a virtual agent in an effort to improve social cues and affective expressions during the interview @xcite . the system automatically detected and analyzed smiles, head nods, and body movements, which were used by a machine learning algorithm to classify the mental state (e.g., stressed, bored, hesitant) and affective state (e.g., positive/negative mood) of the user. during the virtual interview, the user received credits in the game when depicting behaviors that were deemed as effective for the interview. at the end, users received a series of statistics for each of the focal behaviors, which were also visualized over time. mach-my automated conversation coach is another automated interview training system that provided feedback to the user regarding their performance based on the analysis of facial expressions, speech, and prosody @xcite . similarly, hartholt et al. designed a virtual reality system that simulated various interview settings, including the interviewer's propensity toward the interviewee (i.e., friendly, neutral, unfriendly) and the physical space of the interview (e.g., break room, office) @xcite . a user would interact with the training system by starting from easy to more challenging scenarios. no additional feedback was provided to the user. another line of work has evaluated interviewees based on multimodal data that were mostly collected in an asynchronous manner. chen et al. estimated applicants' personality traits based on the audiovisual analysis of monologue job interviews @xcite . linguistic analysis was conducted with a bag-of-words text representation. hemamou et al. designed a hierarchical attention model, called "hirenet" that predicted the hirability of an interviewee based on asynchronous video interviewing. hirenet relied on multimodal information from text, audio, and video @xcite . similarly, ngugen & gatricia-perez and muralidhar et al. analyzed acoustic and visual cues of video resumes and examined their effectiveness in estimating the candidate's hireability and social and communication skills @xcite . finally, naim et al. analyzed interviewees' performance in mock job interviews using their facial expressions (e.g., smiles, head gestures, facial tracking points), language (e.g., word counts, topic modeling), and prosodic information (e.g., pitch, intonation, and pauses). results presented in the mit interview dataset suggest that the use of unique words and personal pronouns, and the degree of speech fluency significantly affect one's interview performance @xcite . the contributions of this paper in comparison to prior work are: (1) while previous work focuses on global characteristics of the interviewee (e.g., personality, social/communication skills) and overall descriptors of the interview outcome (e.g., hireability, performance), this paper provides a closer study to turn-level behaviors that can affect the job interview outcome, thus laying out the foundation toward intelligent assistive technologies that can analyze micro-level data and provide users with detailed feedback at the turn-level; (2) in contrast to the majority of prior work, this paper analyzes data from synchronous interactions between an interviewer and an interviewee, which are more dynamic and diverse; and (3) prior work has mostly focused on college students or fresh college graduates, while this research investigates a unique population that comprises of military veterans facing unique challenges when preparing for a job interview, thus outlining unique design characteristics when it comes to creating assistive technologies for this population. ## discussion the increasingly complex and demanding employment market and future workforce requires mature handling of content and emotions by the job candidates, therefore failing to explain one's skills or over-sharing information can be detrimental to succeeding in the employment interview @xcite . results from this study indicate that various types of nlp techniques can be effective in automatically identifying the degree of explanation in job interview responses, which can be particularly valuable when designing training technologies to prepare candidates for future employment. while previous work has focused on behavioral impressions that can affect the overall outcome of the interview @xcite @xcite , this paper focuses on linguistic behaviors at the turnlevel, which can serve as the foundation for providing tangible low-level feedback to the interviewee. training technologies that rely on automated nlp systems, such as the ones examined in this paper, can help pinpoint exact turns in the dialog that effectively serve the job interview outcome (i.e., succinct, comprehensive responses), as well as turns that might hurt the interview outcome (i.e., underexplaining, over-explaining). intelligent cognitive enhancement technologies can potentially assist job candidates in helping them effectively communicate their skills to the interviewers. such technologies need to rely on robust nlp approaches, that are adequately generalizable to unseen users and new contexts and depict reliable performance, especially for the detection of classes of interest, such as the under-explaining and over-explaining classes in our case. in addition, nlp technologies need to be effectively meshed with human-computer interaction (hci) interfaces, in order to provide feedback in the right form (e.g., visual, tactile) and the right time (e.g., during practice, post-practice). in addition to detecting points of improvement, explaining their role in interview performance and suggesting appropriate changes to those responses would pave the way for personalized learning pathways. it is also essential to consider the degree of explanation in the context of other linguistic behaviors (e.g., excessive use of military jargon, ineffective translation of military experience to the civilian job context), gestures (e.g., rigidity in posture), and vocal expressions (e.g., voice loudness), which will allow us to design technologies that can assist veteran interviewees in a holistic manner. user studies are needed to be conducted so that we can better understand the effectiveness of these technologies in the overarching goal of assisting military veterans to succeed in civilian job interviews. ## conclusion we examined linguistic behaviors of military veterans that are indicative of the degree of explanation in job interview responses. we investigated different types of linguistic descriptors, ranging from word-based tokenization and lexicon-based representations, to pre-trained embeddings with deep learning models. our results indicate that pretrained embeddings are effective in detecting succinct and comprehensive responses, which contain the majority of samples. lexicon-based features can reliably detect over-explained responses, potentially because of their unique psycholinguistic characteristics related to affect, work experience, and complex expression. finally, under-explained answers are best recognized via the token-based approach, which might be due to the fact that these are characterized by significantly different frequency of tokens compared to the succinct responses. results from this study lay the foundation toward intelligent interview training technologies that provide personalized learning by detecting verbal behaviors important for the job interview, explaining their role to the user, and suggesting appropriate changes that can effectively help users secure their desired job.