Automatic Social Role Recognition In Professional Meetings

ABSTRACTThis paper investigates the inuence of social roles on the conversa-tion style and linguistic usage of participants in professional meet-ing recordings. At rst, we implement a generative model to capturethe sequential nature of conversations in terms of participants, turn-taking behavior. In parallel, the system also employs a probabilisticdiscriminativeclassieronasetofhighlevelfeatures. Thenalstepinvolves combining evidence from both generative and discriminatemodels. Experimentssuggestthatbothgenerativeanddiscriminativemodels can reach a recognition accuracy of65%in classifying foursocial roles. Moreover, the recognition accuracy increases to69%when information from both models is taken into consideration.Index Terms Social Role Labeling, Turn Taking, Linguistic,Lexical and Prosody.1. INTRODUCTIONAnalyzing spoken documents in terms of speaker role informationis useful for enriching the content description of multimedia data.It can be used in applications like information retrieval, enhancingmultimedia content browsing and allowing summarization of multi-media documents [1]. Speaker roles are stable behavioral patternsin an audio recording and the problem of role recognition consistsin assigning a label, i.e., a role to each of the speakers. Automaticlabeling of speaker roles has been widely studied in case of Broad-cast News (BN) recordings. These roles are imposed from the newsformat and relate to the task each participant performs in the conver-sation like anchorman, journalists, interviewees, etc. In the last fewyearsautomaticrolerecognitionhasalsobeeninvestigatedformeet-ing recordings and broadcast conversations. Typical roles in thesestudies can vary with environment and applications such as ProjectManager in AMI corpus [2], student, faculty member in ICSI cor-pus[3]. Mostoftheresearchhasfocusedonusesupervisedmethods(bothgenerativeordiscriminativeclassiers)thoughsomeworks[4]havealsoexploredunsupervisedmethods. Commonfeaturesusedinthesestudiesextractrelevantinformationfromconversationfeatures,lexical features, prosody and Dialog act tags [5, 6, 4]. For the studies mentioned above participants role is formal andconsidered to remain constant over the duration of entire audiorecording. Other role coding schemes have also been proposed inliterature which put roles in a more dynamic setting, such as socio-emotional roles (here after referred to as social roles) [7, 8, 9, 10]. Social roles describe relation between conversation participants andtheir roles oriented towards functioning of group as a group. So-cialrolesareusefultocharacterizethedynamicsoftheconversation,i.e., the interaction between the participants and can be generalizedacross any type of conversation. They are also related to phenomenastudiedinmeetingslikesocialdominance,engagementandalsohot-spots [11]. Besides these social roles can also provide cues aboutstate of meeting. Meeting segments where participants take moreactive roles are likely to have richer information o w compared tosegments where participants only take passive roles.Previous approaches to automatic social role recognition canbe broadly classied into discriminative and generative approaches.Common format in many of these studies is to predict social rolewithin a segment of recording, where the role of each participantis assumed to stay constant. Among the discriminative approaches,automatic social role recognition was rst investigated in [7]. Theyused a support vector machine classier to discriminate between so-cial roles in meetings recorded for problem solving sessions. Thefeature set used in this work was extracted from audio and video ac-tivity states within a segment of meeting recording. Other studieshave also investigated social role recognition in professional meet-ings (AMI corpus). The research in [12] revealed that automaticallyextracted subjectivity features from lexical and prosodic cues arecorrelated with social roles. They also report that combining speechactivity features with subjectivity features can improve recognitionover using activity features alone. More recently, in [10] a multi-class boosting classier was used to integrate evidence from severalinformation streams i.e. speech activity, dialog act tags, lexical andprosody for social role recognition. Investigations in this work alsohighlighted that some social roles are more correlated with lexicalcontent and dialog act tags. The generative models for social rolerecognition were considered in [9, 8]. While the work in [8] usedspeech activity and video features, in [9] the generative frameworkwas used to combine prosody and turn duration. These works alsoinvestigated the inuence of other participants on the distribution ofthese features.In this paper, we propose a novel method, which combines thestrengthofbothdiscriminativeandgenerativemodelswhichhavesofar been used separately. The generative framework is used to de-velop a model for analyzing conversations in terms of turn takingsequences while the discriminative model is trained on a novel setof high level features. The generative approach models conversationsequencesasoutputsofaMarkovrandomprocess. Itinvestigatesef-fect of social roles on long pauses and overlaps in turn taking style.In comparison the study in [9] captures speaker change informationin turn taking. The conversational model framework in this paper issimilar to research in [13]. However, while they investigate relationof correlates such as education, gender etc. on turn taking styles intwopersondialogs, theproposedworkconsiderstheinuenceofso-cial roles on turn taking patterns in multiparty conversations. In ad-dition to modeling conversation styles, the present work also inves-tigates correlation between social roles and a novel set of high levelfeatures which includes prosodic, structural and linguistic informa-tion extracted over a segment in meeting. While structural features