“ Y ou might think about slightly revising the title”: Identifying Hedges in Peer-tutoring Interactions (2022)

Abstract: Hedges have an important role in the management of rapport. In peer-tutoring, they are notably used by tutors in dyads experiencing low rapport to tone down the impact of instructions and negative feedback. Pursuing the objective of building a tutoring agent that manages rapport with teenagers in order to improve learning, we used a multimodal peer-tutoring dataset to construct a computational framework for identifying hedges. We compared approaches relying on pre-trained resources with others that integrate insights from the social science literature. Our best performance involved a hybrid approach that outperforms the existing baseline while being easier to interpret. We employ a model explainability tool to explore the features that characterize hedges in peer-tutoring conversations, and we identify some novel features, and the benefits of a such a hybrid model approach.

## introduction rapport, most simply defined as the ". . . relative harmony and smoothness of relations between people . . . " @xcite , has been shown to play a role in the success of activities as varied as psychotherapy @xcite and survey interviewing @xcite . in peer-tutoring, rapport, as measured by the annotation of thin slices of video, has been shown to be beneficial for learning outcomes @xcite . the level of rapport rises and falls with conversational strategies deployed by tutors and tutees at appropriate times, and as a function of the content of prior turns. these strategies include selfdisclosure, referring to shared experience, and, on the part of tutors, giving instructions in an indirect manner. some work has attempted to automatically detect these strategies in the service of intelligent tutors @xcite , but only a few strategies have been attempted. other work has con-centrated on a "social reasoning module" @xcite to decide which strategies should be generated in a given context, but indirectness was not among the strategies targeted. in this paper, we focus on the automatic classification of one specific strategy that is particularly important for the tutoring domain, and therefore important for intelligent tutors: hedging, a sub-part of indirectness that "softens" what we say. this work is part of a larger research program with the long-term goal of automatically generating indirectness behaviors for a tutoring agent. according to @xcite , hedges are part of the linguistic tools that interlocutors use to produce politeness, by limiting the face threat to the interlocutor (basically by limiting the extent to which the interlocutor might experience embarrassment because of some kind of poor performance). an example is "that's kind of a wrong answer". hedges are also found when speakers wish to avoid losing face themselves, for example when saying ("i think i might have to add 6."). @xcite found that in a peer-tutoring task, when rapport between interlocutors is low, tutees attempted more problems and correctly solved more problems when their tutors hedged instruc-tions, which likewise points towards a "mitigation of face threat" function. hedges can also be associated with a nonverbal component, for example averted eye gaze during criticism @xcite . hedges are not, however, always appropriate, as in "i kind of think it's raining today." when the interlocutors can both see rain (although it might be taken as humorous). these facts about hedges motivate a way to automatically detect them and, ultimately (although not in the current work) also generate them. in both cases we first have to be able to characterize them using interpretable linguistic features, which is what we address in the current paper. thus, in the work described here, based on linguistic descriptions of hedges @xcite , we built a rule-based classifier. we show that this classifier in combination with additional multimodal interpretable context-dependent features significantly improves the performance of a machine learning model for hedges, compared to a less interpretable deep learning baseline from @xcite using word embeddings. we also relied on a machine learning model explanation tool @xcite to investigate the linguistic features related to hedges in the context of peer-tutoring, primarily to see if we could discover surprising features that the classification model would associate to hedges in this context, and we describe those below. the code of the models described in the paper is also provided. foot_0 2 related work hedges: according to @xcite , hedging is a rhetorical strategy that attenuates the strength of a statement. one way to produce a hedge is by altering the full semantic value of a particular expression through propositional hedges (also called approximators in @xcite ), as in "you are kind of wrong," that reduce prototypicality (i.e accuracy of the correspondence between the proposition and the reality that the speaker seeks to describe). propositional hedges are related to fuzzy language @xcite , and therefore to the production of vagueness @xcite and uncertainty @xcite . a second kind are relational hedges (also called shields in @xcite ), such as "i think that you are wrong." or "the doctor wants you to stop smoking.", conveying that the proposition is considered by the speaker as subjective. in a further sub-division, attribution shields, as in "the doctor wants you ...", the involvement of the speaker in the truth value of the proposition is not made explicit, which allows speakers not to take a stance. as described above, @xcite found that tutors who showed lower rapport with their tutees used more hedged instructions (they also employed more positive feedback), however this was only the case for tutors with a greater belief in their ability to tutor. tutees in this context solved more problems correctly when their tutors hedged instructions. no effect of hedging was found for dyads (pairs of interlocutors) with greater social closeness. however, the authors did not look at the specific linguistic forms these teenagers used. @xcite also describes the role that hedging plays in this age group, showing that students use both relational ("i think that john is smart.") and propositional ("john is kind of smart.") hedges for much the same shielding function of demonstrating uncertainty, to save them from the risk of embarrassment if they are wrong. the author observed that teens used few adaptors (kind of, somewhat) and preferred to use rounders (around, close to). however, this study was performed with an adult and two children, possibly biasing the results due to the participation of the adult investigator. hedges have been included in virtual tutoring agents before now. @xcite integrated hedges in a tutor agent for undergraduates in cs, as a way to encourage the student to take the initiative. hedges have also been used as a way of integrating brown and levinson's politeness framework @xcite in virtual tutoring agents. results were not broken out by strategy, but politeness in general was shown to positively influence motivation and learning, in certain conditions. ## problem statement we consider a set d of conversations @xmath0, c 2 , ..., c |d| ), where each conversation is composed of a sequence of independent syntactic clauses c @xmath1, u 2 , ..., u m ), where m is the number of clauses in the conversation. note that two consecutive clauses can be produced by the same speaker. each clause is associated with a unique label corresponding to the differ-ent hedge classes described in