Embodied Interaction in Mental Health Consultations: Some Observations on Grounding and Repair (2022)

Abstract: Shared physical space is an important resource for face-to-face interaction. People use the position and orientation of their bodies—relative to each other and relative to the physical environment—to determine who is part of a conversation, to manage conversational roles (e.g. speaker, addressee, side-participant) and to help co-ordinate turn-taking. These embodied uses of shared space also extend to more fine-grained aspects of interaction, such as gestures and body movements, to support topic management, orchestration of turns and grounding. This paper explores the role of embodied resources in (mis)communication in a corpus of mental health consultations. We illustrate some of the specific ways in which clinicians and patients can exploit embodiment and the position of objects in shared space to diagnose and manage moments of misunderstanding.

## background non-verbal signals are integral to natural human interaction. the best known are the facial expressions of emotion (e.g. anger, fear, sadness, surprise, happiness) @xcite . however, there are also a range of non-verbal signals that are specific to conversation @xcite @xcite . these include large-scale configurations of body position and orientation that can tell us e.g., who is participating in a conversation, what their role is (e.g. speaker, addressee, listener or bystander) and their relative levels of interest and engagement @xcite @xcite . there are also a range of small-scale conversational gestures. for example, the use of hand gestures to hold or yield a turn, to enlist help with finding a word-or expression and the use of facial gestures such as raised eyebrows to emphasise particular words or to display "thinking" @xcite @xcite . embodied signals can be produced in parallel with verbal contributions by both the speaker and by other participants @xcite @xcite . this facilitates real-time, incremental checking and feedback. speakers can produce gestures that complement or augment their speech and listeners can simultaneously display their reactions through concurrent backchannel signals @xcite . these concurrent signals shape a speaker's turn in realtime and-if problems are apparent-can cause a speaker to rephrase, change direction or cut-off their turn @xcite . some non-verbal signals are associated with potential problems with shared understanding. a frown and briefly averted gaze before a turn can suggest that a speaker is about to say something potentially problematic @xcite and gaze aversion by an addressee following a turn can prompt the speaker to rephrase what they said @xcite . some facial gestures, such as raised eyebrows and widening of the eyes, can act as stand-alone clarification requests @xcite . similarly, temporary suspension of hand movements described as non-verbal 'holds' or 'freezes' can provide signals of ongoing repairs @xcite @xcite . quantitative data from motion captured conversations shows that the velocity and height of head and hand movements changes during both selfrepairs/disfluencies and other-repairs-and that these changes are different for speakers and listeners @xcite @xcite . ## communication in healthcare communication in healthcare settings is critical to the quality of patient-clinician relationships, affecting outcomes such as patient satisfaction and treat-ment adherence. it is arguably even more important in mental healthcare settings, since talk is the primary means of diagnosis and treatment @xcite @xcite @xcite . however, difficulties in balancing good communication practices with work pressures have often been reported in the nhs @xcite . this has prompted an interest in developing protocols and tools to help healthcare professionals organise their thoughts and structure consultations. however, overly rigid protocols can have a problematic impact on the naturalness of healthcare professional's interaction, limiting their ability to adapt to individual patient's needs, concerns or understanding @xcite . at its worst, the artificial structure imposed by the protocol disrupts the flow of conversation and can become counterproductive for clinical interactions @xcite . here we examine these issues in the context of a tablet application (dialog+) designed to promote communication in face-to-face mental health consultations. we focus on a detailed qualitative analysis of the moments where misunderstandings ariseand the combination of verbal and non-verbal resources that are used to address such problems. our analysis shows how the position of the physical device and people's orientation toward it plays a role in both causing and mitigating misunderstandings. ## the current study the data for this study comes from video recordings of mental health consultations. extracts from these interactions are described below using conversation analysis (ca) techniques. ca involves detailed qualitative analysis of conversations in naturally occurring circumstances-and the material for such analyses is recorded and/or transcribed talk @xcite @xcite . the focus of the analysis is on the organisation of fine-grained features of interaction such as timing, pauses, repetitions, restarts and the details of concurrent non-verbal signals such as gaze and gesture. the examples considered below are selected to illustrate the types and trajectories of (mis)communication observed in this corpus, how they are affected by the presence of dialog+ application, and how this affects the management of shared understanding. the two overarching research questions in this study are: ## methods design. the dialog+ trials are described elsewhere @xcite . our dataset consists of 40 video recordings of 32 clinical consultations; 16 are from a control group receiving treatmentas-usual and 17 from an intervention group using dialog+. the average length of the recordings is 30.12 minutes in the control group and 39.68 minutes in the intervention group. participants and ethics. participants who consented to join the original dialog+ trials provided informed consent to be video-taped or audiotaped on at least one session for use in future studies on potential improvements to the dialog+ procedure and technology. the study protocol, including data collection and storage procedures, was audited by the national research ethics service (nres) london, stanmore (12/lo/1145). all personal data has been removed from the extracts and faces have been blurred. procedure and data analysis. the authors familiarised themselves with the data by watching and listening to the recordings repeatedly, making brief notes about the general form of the conversations and any features that appeared striking. a second pass then focused on selecting particular episodes with overt evidence of misunderstanding. the clearest of these were transcribed in detail using jefferson's orthography which includes features like pauses, overlap and intonation @xcite . embodied conduct that is relevant to the interaction-such as nods, gestures and postures shifts-were are also included in the transcriptions. ## discussion the examples presented above raise a number of basic points about the organisation of face-to-face interaction and the use of embodiment in shared space as a resource for communication. the data presented highlight the tension between the use of the protocol as a standardised assessment instrument and its function as a tool to promote effective conversation. ideally, quantitative assessments of quality of life should be consistent across different participants and different contexts. in practice, the meaning of the different assessment items and even the meaning of the numbers on the assessment scale varies across consultations. patients and clinicians routinely engage in active, collaborative re-interpretation of the protocol in order to complete the assessment. standard phrases such as best case scenario and practical help take on specific meanings depending on individual circumstances. what is ostensibly the same question means different things to different people and can also mean different things to the same people in different sequential contexts. the observations show the process of detecting and dealing with differences in interpretation is fundamental to effective communication @xcite . although this recurrent interpretive work means the application of the dialog+ protocol is not strictly standardised, it is part of normal interaction and arguably central to the therapeutic effectiveness of the intervention. the assessment items provide prompts that encourage a wider ranging conversation and greater continuity across sessions than observed in the control groups. the more strictly the protocol is applied, the more friction it would cause to the conversation @xcite . the work people do to bridge the gap between the protocol and the details of individual's lives can play an important role in uncovering the different combinations of practical circumstances and constraints that influence long term outcomes. moving to remote delivery directly alters the configuration of resources available to participants in the interaction. one effect will be to change the visibility and control of actions in the application. as noted, input is currently led by clinicians and often not directly visible to the patient. if the ap-plication is running in a shared window all updates will be immediately visible to both participantsand control of input by one person will not automatically restrict input by the other. this should mitigate, for example, problems with coordinating when a concurrent action (typing notes) starts and finishes-and also what item is currently under discussion. however, the work of (re)interpreting items and actions would still be required. other possible advantages of remote interaction are the savings in cost and time, as well as the potential for improved access for some patients. nonetheless, some important features will be removed by remote interaction. one is the use of the embodiment of the protocol itself as a shared screen positioned in space between the participants. there are two ways in which this matters. first, changes in people's overall physical orientation, through head movements or posture shifts signal important changes in their focus of attention and engagement. this is especially true of the clinicians, who often use posture changes to and from the device and to and from the patient to mark changes in engagement e.g. to introduce a direct question to the patient (see e.g. the first line of excerpt 1). secondly, clinicians (and occasionally patients) have the ability to point at a question and, for example, gesture from the question to a person. these movements, in effect, use the shared space to provide a useful spatial map for the embodied coordination of topics @xcite @xcite . it is also noticeable from the preceding examples that where a gesture is specifically placed in space is significant. the pointing gestures in figures 2 and 3 have the same form, but their different speed of execution, orientation and placement give them a different interpretation. in an example from the same session not included in the extracts, there is a rapid shift by the clinician from a short two figure point at the screen to a two finger point at the patient. the form and speed of the gesture is very similar but its significance is different for the participants because of where it is placed, a phenomenon also noted in other face-to-face contexts @xcite . it also illustrates the ways in which people can use contrasting head and hand orientation as a means of concurrent triangulation of different reference points (e.g. people and objects) to help coordinate understanding @xcite . these examples also illustrate how these additional, embodied resources seem to become especially significant at the points where shared understanding is threatened @xcite @xcite . the space between participants becomes an important extra resource for detecting and dealing with these misunderstandings. remote video-mediated interactions flatten the three-dimensional world of face-to-face interaction in shared space into a two-dimensional window @xcite . a shared application and a video window impede the forms of interaction highlighted above. working out what someone is orienting to requires more effort-and although posture shifts and gestures may be visible on camera, they are attenuated and cannot take advantage of relative position in a shared space. people are able to compensate in these situations and it is an open question what the cost of adapting could be. shared applications that enable people to see each other's cursors can partially replicate a sense of the current focus of attention. this may help with the redirection of attention, but provides a significantly reduced set of cues. one way in which remote interaction can give greater access is to allow asynchronous updates. for example, allowing patients to add notes or modify ratings in response to events outside the clinical context. this could give patients the opportunity to understand the protocol and application better and also feel that they are on a more equal footing with clinicians. it could also provide clinicians with potentially useful additional context updates for use in the face-to-face sessions. this shift in the distribution of control over the tool might impact on the dynamics of clinical interaction and careful design would be needed to avoid it becoming overused as a communication channel. there are some qualifications to the findings. the observations are selective and are based on a specific population of patients with a diagnosis of psychosis. although we think the general principles should apply in other contexts, different issues will be encountered with other communication tools and/or other client groups e.g. telephone delivery @xcite . nevertheless, this study has shed light on some of the concerns and challenges related to designing health communication protocols and some specific issues for remote settings. in the wake of the pandemic, while remote consultations are unlikely to fully replace face-to-face consultations, it will no doubt become a feature increasingly integrated into current health systems and used alongside conventional practice @xcite . care needs to be taken to ensure that developments surrounding technologies like dialog+ are balanced with appropriate flexibility, as every nuance in communication between patients and clinicians can have a role to play in influencing the quality of therapeutic relationships and the effectiveness of clinical encounters. ## conclusion structured protocols are increasingly used in community mental healthcare consultations. detailed analysis of the interactions using one of these protocols (dialog+) and its associated tablet application shows some of the advantages and pitfalls of the approach. the application provides a useful tool to support engagement in the consultations, but in practice, deviations from the protocol play an important role in the success of the consultations. the interactions are characterised by collaborative work done to (re)interpret the assessment items in the context of each client's and each clinician's practical circumstances. participants use embodiment in shared space as an important, flexible interactional resource in doing this. with remote consultations increasingly integrated into healthcare settings, our findings provide a starting point for thinking about how software like the dialog+ application can be redesigned for these environments.