# Creation, Analysis and Evaluation of AnnoMI, a Dataset of Expert-Annotated Counselling Dialogues +
## 4 Annotation Scheme
### Therapist Utterance Attributes  
#### 4.1.1 (Main) Behaviour  
In MI, three fundamental yet crucial skills to achieve effective counselling are asking, informing, and listening [1]. In view of this principle and the related components of mainstream coding schemes for MI, we consider Question, Input, and Reflection as major therapist behaviours that correspond to asking, informing, and listening, respectively. In cases where more than one behaviour is present in an utterance, e.g., a question after an input, the expert is asked to further select the Main Behaviour. Other is listed as the fourth and default option, where no Question, Input, or Reflection appears in the utterance.  
We also list Question, Input, and Reflection as separate attributes of therapist utterances in order to investigate their sub-types, as laid out in the sections below.  
Figure 2: Results of survey of annotators regarding whether AnnoMI reflects real-world high- and low-quality MI.  
We note that this work is more focused on the use of asking, informing, and listening in the AnnoMI dialogues. For this purpose, the original annotation scheme was more ambitious and had several non-MITI/MISC annotation fields, but they are not included in this paper due to their very low IAAs (Fleiss' kappa), and thus the annotation scheme presented in this section may resemble a subset/regrouping of MISC to some readers. This work does not seek to compare directly with previous studies that use the complete MISC/MITI for annotation.  
\begin{table}
\begin{tabular}{c c} \hline \hline
**Therapist Utterance Attributes** & **Label** \\ \hline \hline \multirow{4}{*}{(Main) Behaviour} & Question \\  & Input \\  & Reflection \\  & Other \\ \hline \multirow{2}{*}{Question} & Open question (+) \\  & Closed question (+) \\  & No question (-) \\ \hline \multirow{4}{*}{Input} & Information (+) \\  & Advice (+) \\  & Options (+) \\  & Negotiation/goal setting (+) \\  & No input (-) \\ \hline \multirow{2}{*}{Reflection} & Simple reflection (+) \\  & Complex reflection (+) \\  & No reflection (-) \\ \hline \multirow{2}{*}{**Client Utterance Attribute**} & **Label** \\ \hline \multirow{2}{*}{Talk Type} & Change \\  & Neutral \\ \multicolumn{2}{c}{Sustain} \\ \end{tabular}
\end{table}
Table 4: Utterance-level multi-choice annotation scheme. (+) implies presence of utterance attribute (e.g., “Simple reflection” entails that Reflection exists in utterance), while (-) indicates absence thereof (e.g., “No reflection” label implies that Reflection is not present in utterance).  
\begin{table}
\begin{tabular}{c c} \hline \hline
**Topic** & **Dialogues** \\ \hline Reducing alcohol consumption & 28 (21.1\%) \\ Smoking cessation & 21 (15.8\%) \\ Weight loss & 9 (6.8\%) \\ Taking medicine/following medical procedure & 9 (6.8\%) \\ More exercise/increasing activity & 9 (6.8\%) \\ Reducing drug use & 8 (6.0\%) \\ Reducing recidivism & 7 (5.3\%) \\ Compliance with rules & 5 (3.8\%) \\ Asthma management & 5 (3.8\%) \\ Diabetes management & 5 (3.8\%) \\ Other & 33 (24.8\%) \\ \hline \multirow{2}{*}{**Topic**} & **Uterances** \\ \hline Reducing alcohol consumption & 1914 (19.7\%) \\ Reducing recidivism & 1303 (13.4\%) \\ Smoking cessation & 1106 (11.4\%) \\ Diabetes management & 709 (7.3\%) \\ Reducing drug use & 578 (6.0\%) \\ Taking medicine/following medical procedure & 574 (5.9\%) \\ More exercise/increasing activity & 525 (5.4\%) \\ Weight loss & 396 (4.1\%) \\ Avoiding DUI & 394 (4.1\%) \\ Changing approach to disease & 315 (3.2\%) \\ Other & 2107 (21.7\%) \\ \hline \hline \end{tabular}
\end{table}
Table 3: Top 10 topics in AnnoMI in terms of (a) number and percentage of dialogues that have those topics, and (b) total number and percentage of utterances in those dialogues.