Dialogue act classification is a central challenge for dialogue systems . Although the importance of emotion in human dialogue is widely recognized , most dialogue act classification models make limited or no use of affective channels in dialogue act classification . This paper presents a novel affect-enriched dialogue act classifier for task-oriented dialogue that models facial expressions of users , in particular , facial expressions related to confusion . The findings indicate that the affectenriched classifiers perform significantly better for distinguishing user requests for feedback and grounding dialogue acts within textual dialogue . The results point to ways in which dialogue systems can effectively leverage affective channels to improve dialogue act classification . Dialogue systems aim to engage users in rich , adaptive natural language conversation . For these systems , understanding the role of a user 's utterance in the broader context of the dialogue is a key challenge ( Sridhar , Bangalore , & Narayanan , 2009 ) . Central to this endeavor is dialogue act classification , which categorizes the intention behind the user 's move ( e.g. , asking a question , providing declarative information ) . Automatic dialogue act classification has been the focus of a large body of research , and a variety of approaches , including sequential models ( Stolcke et al . , 2000 ) , vector-based models ( Sridhar , Bangalore , & Narayanan , 2009 ) , and most recently , featureenhanced latent semantic analysis ( Di Eugenio , Xie , & Serafin , 2010 ) , have shown promise . These models may be further improved by leveraging regularities of the dialogue from both linguistic and extra-linguistic sources . Users ' expressions of emotion are one such source . Human interaction has long been understood to include rich phenomena consisting of verbal and nonverbal cues , with facial expressions playing a vital role ( Knapp & Hall , 2006 ; McNeill , 1992 ; Mehrabian , 2007 ; Russell , Bachorowski , & Fernandez-Dols , 2003 ; Schmidt & Cohn , 2001 ) . While the importance of emotional expressions in dialogue is widely recognized , the majority of dialogue act classification projects have focused either peripherally ( or not at all ) on emotion , such as by leveraging acoustic and prosodic features of spoken utterances to aid in online dialogue act classification ( Sridhar , Bangalore , & Narayanan , 2009 ) . Other research on emotion in dialogue has involved detecting affect and adapting to it within a dialogue system ( Forbes-Riley , Rotaru , Litman , & Tetreault , 2009 ; López-Cózar , Silovsky , & Griol , 2010 ) , but this work has not explored leveraging affect information for automatic user dialogue act classification . Outside of dialogue , sentiment analysis within discourse is an active area of research ( López-Cózar et al . , 2010 ) , but it is generally lim-ited to modeling textual features and not multimodal expressions of emotion such as facial actions . Such multimodal expressions have only just begun to be explored within corpus-based dialogue research ( Calvo & D'Mello , 2010 ; Cavicchio , 2009 ) . This paper presents a novel affect-enriched dialogue act classification approach that leverages knowledge of users ' facial expressions during computer-mediated textual human-human dialogue . Intuitively , the user 's affective state is a promising source of information that may help to distinguish between particular dialogue acts ( e.g. , a confused user may be more likely to ask a question ) . We focus specifically on occurrences of students ' confusion-related facial actions during taskoriented tutorial dialogue . Confusion was selected as the focus of this work for several reasons . First , confusion is known to be prevalent within tutoring , and its implications for student learning are thought to run deep ( Graesser , Lu , Olde , Cooper-Pye , & Whitten , 2005 ) . Second , while identifying the " ground truth " of emotion based on any external display by a user presents challenges , prior research has demonstrated a correlation between particular facial action units and confusion during learning ( Craig , D'Mello , Witherspoon , Sullins , & Graesser , 2004 ; D'Mello , Craig , Sullins , & Graesser , 2006 ; McDaniel et al . , 2007 ) . Finally , automatic facial action recognition technologies are developing rapidly , and confusion-related facial action events are among those that can be reliably recognized automatically ( Bartlett et al . , 2006 ; Cohn , Reed , Ambadar , Xiao , & Moriyama , 2004 ; Pantic & Bartlett , 2007 ; Zeng , Pantic , Roisman , & Huang , 2009 ) . This promising development bodes well for the feasibility of automatic real-time confusion detection within dialogue systems . Emotion plays a vital role in human interactions . In particular , the role of facial expressions in humanhuman dialogue is widely recognized . Facial expressions offer a promising channel for understanding the emotions experienced by users of dialogue systems , particularly given the ubiquity of webcam technologies and the increasing number of dialogue systems that are deployed on webcamenabled devices . This paper has reported on a first step toward using knowledge of user facial expressions to improve a dialogue act classification model for tutorial dialogue , and the results demonstrate that facial expressions hold great promise for distinguishing the pedagogically relevant dialogue act REQUEST FOR FEEDBACK , and the conversational moves of GROUNDING . These early findings highlight the importance of future work in this area . Dialogue act classification models have not fully leveraged some of the techniques emerging from work on sentiment analysis . These approaches may prove particularly useful for identifying emotions in dialogue utterances . Another important direction for future work involves more fully exploring the ways in which affect expression differs between textual and spoken dialogue . Finally , as automatic facial tagging technologies mature , they may prove powerful enough to enable broadly deployed dialogue systems to feasibly leverage facial expression data in the near future .

Challenge: Dialogue act classification which is a central challenge for dialogue systems currently makes limited or no use of emotional expressions while they are widely recognized.
Approach: They propose an affect-enriched classifier for task-oriented and computer-mediated textual human-human tutorial dialogues which uses confusion-related facial expressions by students.
Outcome: The proposed classifier can leverage affective channels and improve on distinguishing user requests for feedback and grounding dialogue acts within a textual dialogue.