VERVE: Template-based ReflectiVE Rewriting for MotiVational IntErviewing
Authors: Do June Min
Veronica Perez-Rosas
Ken Resnicow
Rada Mihalcea
Venue: EMNLP 2023 Findings (2023)
Keywords: motivational interviewing
rewriting
counseling
TL;DR: We introduce VERVE, a template-based rewriting system with paraphrase-augmented training and adaptive template updating.

Abstract: Reflective listening is a fundamental skill that counselors must acquire to achieve proficiency in motivational interviewing (MI). It involves responding in a manner that acknowledges and explores the meaning of what the client has expressed in the conversation. In this work, we introduce the task of counseling response rewriting, which transforms non-reflective statements into reflective responses. We introduce VERVE, a template-based rewriting system with paraphrase-augmented training and adaptive template updating. VERVE first creates a template by identifying and filtering out tokens that are not relevant to reflections and constructs a reflective response using the template. Paraphrase-augmented training allows the model to learn less-strict fillings of masked spans, and adaptive template updating helps discover effective templates for rewriting without significantly removing the original content. Using both automatic and human evaluations, we compare our method against text rewriting baselines and show that our framework is effective in turning non-reflective statements into more reflective responses while achieving a good content preservation-reflection style trade-off.

VERVE: Template-based ReflectiVE Rewriting for MotiVational IntErviewing
Do June Min', Verónica Pérez-Rosas', Kenneth Resnicow?, and Rada Mihalcea'
'Department of Electrical Engineering and Computer Science, "School of Public Health University of Michigan, Ann Arbor, MI, USA {dojmin, vrncapr, kresnic, mihalcea @ @umich.edu
Abstract
Diets don't work for me. Initially feel great, up feeling and in the back of my mind.
Reflective listening a fundamental skill that counselors must acquire achieve proficiency in motivational interviewing (MI). It involves responding a manner that acknowledges and explores the meaning of what the client has pressed in the conversation. In this work, we in- troduce the task counseling response rewrit- ing. which transforms non-reflective state- ments into reflective responses. We introduce VERVE. template-birs based rewriting system with paraphrase-augmennte training and adap- tive template updating. VERVE first creates a template by identifying and filtering out to- kens that are not relevant to reflections and constructs a reflective response using the tem- plate. Paraphrase-augmmetted training allows the model to learn less-strict fillings of masked spans, and adaptive template updating helps dis- cover effective templates for rewriting without significantly removing the original content. Us- ing both automatic and human evaluations, compare our method against text rewriting base- lines and show that our framework is effective in turning non-reflective statements into more reflective responses while achieving good con- tent preservation- reflectioon style trade-off.
Client Prompt
find healthy food that find enjoyable to est. I'm sure they exist.
Rewrite Behavior: Advice
Original Response
finding healthy food that you find enjoyable to eat.
You're having
Agent
Behavior: Reflection
Rewrite
Figure 1: In this example of counselor response rewrit- ing, a counseling trainee is asked to provide reflective response given the client prompt and produces poor re- sponse by giving a piece of advice rather than reflecting the client's concerns. Our system generates an improved response that preserves content and increases the of reflective language.
have been several efforts to use NLP to assist coun- selor training, including automatic coding of coun- selor behavior (Flemotomos et al., 2021b), provid- ing timing and language suggestions during client interactions (Miner et al., 2022; Creed et al., 2022), and evaluating the quality of specific counseling skills (Shen et al., 2020; Min et al., 2022).
1 Introduction
During the Covid-19 pandemic, the number of peo- ple living with anxicty and depression more than four times, thus aggravating the ongoing dis- parity between unmet needs for mental health treat- ment and increased mental health disorders (Coley and Baum, 2021).
However, the progress in developing tools that can fulfill a "mentoring role" and offer alternative language suggestions for counselors training has been limited. To fill this gap. we introduce the task of counselor response rewriting, which involves rephrasing trainees' responses with basic counsel- ing skills into alternative responses that reflect a more advanced level of counseling proficiency. We focus on reflective listening skill, and on Motivational Interviewing (Miller and Rollnick, 2013) the counseling strategy.
One driving cause behind this discrepancy is the shortage of mental health professionals, which is exacerbated by the fact that becoming a counselor requires extensive training (Lyon et al., 2010). In particular, counselor training is difficult to speed up due to several factors, such as the need for expert su- pervision, and the laborious and enme-extensive pro- cess needed to provide evaluative feedback. There
main counseling
We show an example of our system output Figure 1. In this case, providing a numerical score

---

reference reflection (i.e., a high-quality reflec- tion) does not help the counselor understand what parts of their answer could be improved. Our sys- tem addresses this shortcoming by separating the behavior-relevant (e.g., reflection-like language) and the behavior-non-relevan parts, and using the latter as template for creating an improved rewrite of the original.
parency is valued. Moreover, content preservation is another advantage of prototype-baset editing, since the template generation process can be con- trolled to vary the amount of original content pre- served in the rewrite. An important difference from previous studies is that we address text rewriting in dialog context, whereas previous studies are mostly concerned with transforming isolated text, such product reviews (Mir et al., 2019).
We introduce VERVE (ReflectiVE Rewriting for MotiVational IntErviewing), a framework based on template editing methods from text style transfer that do not require parallel data, since ex- pert annotation of rewritten responses is expensive and time-consuming. We propose two simple tech- niques to adapt template- semplate-based text rewriting the train- ing, and adaptively template updating. The first helps the text generator to learn a more flexible mapping between a masked template and a full re- sponse so that the structure of the final rewrite is not constrained by the template. The second han- dles the content-edit trade-off (e.g.. preserving part of the user response rather than completely rewrit- ing) by iteratively updating the masked template based on the effect of the rewrite. We evaluate our framework against several baselines from previous text style transfer works using automatic evalua- tion and demonstrate that our system outperforms baselines in achieved reflection scores while still preserving content from the original response.
Since counseling reflections often include em- pathy (Lord et al., 2014), empathetic text genera- tion and rewriting are also relevant. While most of the empathetic generation literature focuses on modeling emotion for generating responses from scratch, Sharma et al. (2021) directly models mul- tiple aspects of empathy and applics reinforcement learning (RL)-based training for rewriting online mental health comments. Similarly, we leverage a classifier model for discriminating attribute labels for text but use simple supervised learning instead of policy gradient RL training.
counseling domain:
Our work is also related to recent work on NLP for the counseling domain aiming to assist coun- selors during their practice and ongoing training. Reflection is an important construct in counseling strategies such as MI, and previous works have studied how the frequency or quality of reflections can be used to evaluate counseling (Pérez-Rosal et al., 2017; Flemotomos et al., 2021a; Ardulov et al., 2022). There also have been studies on gen- erating reflections (Shen et al., 2020, 2022). How- ever, the best of our knowledge, our work is the first to consider rewriting non-reflections into reflections.
2
Related Work
Our work builds upon previous work in text style transfer, text rewriting, and NLP for counseling. Broadly, counselor response rewriting is related to text rewriting in NLP, which includes, text style transfer, content debiasing, and controlled gener- ation (Li et al., 2018; Madaan et al., 2020). In this work, we focus on rewriting through template- based editing (or prototype- based in other text style transfer literature (Jin et al.. 2022). These systems offer several advantages over alternative frame- works such as latent style transfer or LLM-based methods (Dai et al., 2019; Sharma et al., 2023). First, template- editing systems offer high in- terpretability they rely on predefined templates patterns. Users can have precise control over the editing process by selecting specific templates or designing new ones. This allows for easier under- standing and manipulation of the output, which is particularly important in applications where trans-
3 Counselor Response Rewriting
3.1 Task and Application
Reflection is key skill for empathetic listening in motivational interviewing (Miller and Rollnick, 2013; McMaster and Resnicow, 2015; Moyers et al., 2016a). Recently, there has been increas- ing interest in how language models can be used to understand and generate reflections to assist coun- selor practice and ongoing training (Flemotomos et al., 2021 a; Shen et al., 2020, 2022). Our work follows the same research direction, however, we focus on the new task of reflection rewriting rather than reflection writing from scratch. We argue that response rewriting can provide more detailed feed- back while coaching and training counselors, since users' responses are considered by the model, al-

---

Statisties
PAIR
AnnoMI
tion of the datasets and the filtering procedure we used in Appendix A. The final datasct statistics are shown in Table 1.
# of Exchange Pairs
2544
450
Avg of Words
32.39
39.50
# of Complex Reflection
636
of Simple Reflection
318
Methodology
of Non-Reflection
1590
450
Our VERVE framework, shown in Figure 2, is based on template editing- based approach that does not require parallel data. Below, we describe system details.
Table Annotation statics for PAIR and AnnoMI datasets.
lowing the user to compare the original and rewrit- ten responses.
5.1 Template-based Response Rewriting
For example, given client prompt describing their struggles while losing weight, a poorly made counselor response such as "Are you sure you'l 've given up all unhealthy food?" contains unsolicited advice rather than listening and acknowledging the client's experience. Given this response, our system can suggest the following rewrite "You't 've given up all unhealthy foods and you' re sure that di- eting doesn't 't work for you." alternative higher quality reflection that preserves content from the original response.
VERVE follows two-step process in which attribute-relevant tokens in the counselor response are first identified and masked. The resulting tem- plate, along with the original prompt, are then pro- vided as input to the generator to obtain a rewritten response filled with relevant spans.
Template Extraction (Masking). The goal of this step is to create masked version of the origi- nal response to be used by the generator plate for the rewritten version. We start by training transformer model discriminate between the three levels of reflections in the PAIR dataset i.e., non-reflection, simple reflection, and complex re- flection.' 1 Next, we use the attention scores of the discriminator to identify tokens that contribute to the low reflection level in the original response. Our intuition is that the reflection scoring model has learned to attend to key tokens that are relevant to reflection qualityso. their attention scores can be used signal token importance. We use the model's penultimate self-attention layer to identify tokens to be masked and then normalize the atten- tion scores across tokens, per attention head. We then apply max-pooling over the heads, obtaining single attention map A over tokens. Using token type ids, we then the client prompt tokens. This final map is then compared to the average attention score Á across response tokens. For each response token A,, we mask if
tem-
4
Datasets
We use two publicly available MI datasets from PAIR (Min et al., 2022) and AnnoMI (Wu et al., 2022). While PAIR is a collection of single turn exchanges, AnnoMI is set of counseling con- versations consisting of multiple conversational turns. PAIR contains client prompts along with counselor responses with varying reflection quality levels, including simple and complex reflections or non-reflection. Complex reflections go further than simple reflections (i..., simple repetition or slight rephrasing of client's 's statement) by inferring unstated feelings and concerns of the client and are often preferred over simple reflections in MI counseling (Miller and Rollnick, 2013). Note that although PAIR contains multiple responses for given prompt they were designed as rewrites, and thus they cannot be used directly as parallel data to train supervised end-to-end rewriter.
out the attention scores of
(1) Rewriting from Template (Filling). The next step is to input the resulting template into the gen- erator model. This is a transformer- encoder- decoder model that receives the concatenation of prompt and template as input, separated by spe- cial token. We train the generator the original
A -= À
Preprocessing. We preprocess AnnoMI to fo- cus on single exchanges between counselors and clients. Also, since AnnoMI does not include anno- tations for reflection type we the subset of ut- terances labeled no-reflections only. We extract pairs consisting of a single client turn followed by a counselor non-reflection, with constraints on the length of the utterances to filter out short utterances or disfluencies. We include a more detailed descrip-
'The reflection discriminator achieves ~883 label accu- racy.

---

Paraphrase-augmentee Training
Inference w/ Adaptive Template Update
Prompt: start diet every day once get hungry, lose
Non-Refiection: Why do you you can't slick to your goals?
Reflection: Why you you can't stick to your goals?
discipline.
Mask
Mask
Reflection: You really struggle to stick with goels when you're temptation.
Template: <mask> you <mask> you can <mask> stick to your goals <mask>
Template: you
stick to your goals <mask>
Paraphrase
Fill
Fill
When you are faced with temptation, struggle to your goals.
Reflection:
Generated: you considered why you can't stick to your goals?
You're
you
can't stick your goals.
Mask
<mask> are faced with temptation, <mask> struggle to stick your goals.
Template:
Score
Score
Result: Non-Reflection
Result: Reflection
MLE Training:
Increase attribute removal rate
Prompt; Template Reflection
Iteration
Iteration
Figure 2: Overview the VERVE framework. During training, we use attribute-masked versions of paraphrases of reflections as templates for the MLE training for generator training. In the inference time, we adjust the content weight iteratively to achieve the desired edit effect.
response tokens using maximum likelihood estima- tion (MLE) loss. Importantly, only reflec- tions for training the generator (fill-in model).
5.3 Inference with Adaptive Template Updating
One key challenge in text rewriting is the trade-off between content preservation and edit effect since heavy editing of the input (to increase reflection quality/score) leads to less content preservation from the original text. To address this issue, we add a thresholding step during the masking process. Our strategy similar to Li et al. (2018)'s, who use tunable thresholding value at test time. We con- trol content masking by weighting the thresholding term À with weight C. using A, > C Á. Intu- itively, higher C' values make the content masking more conservative, only tokens that highly attended by the predictor would be masked. In contrast, lower C' values lead to higher content masking, allowing more room for the generator to fill in.
5.2 Paraphrase-augmmente Training
One shortcoming we observed while using the template-based editing approach is that con- strains the generator to output responses that are too dependent on the template, thus biasing the gen- eration towards the same type of utterances. For example, "What do you know about yourself?", results in the template: "<mask> do you <mask> about yourself <mask>"" with the bigram "do you" biasing the generator towards generating a question rather than a reflection.
To mitigate this problem, we experiment with auggented training, which helps the generator to learn more flexible mapping be- tween the template and the output by paraphrasing the input template. We use publicly available transformer-based model (details in A.3) gener- ate multiple paraphrases for a given response, then we select the version with the highest Levenshtein edit distance from the original response.
During inference time, use C' to adaptively adjust the degree of content preservation in the rewrite. We begin with basc C value (c.g., 1.0). and incrementally decrease (e.g., by 0.1) if the resulting rewrite is not a reflection (or obtains a very low reflection score).
"To evaluate the quality of the reflection, we use the PAIR scorer (Min et al., 2022).

---

6 Experiments
define it as the fraction of keyphrases from the orig- inal response found in the rewritten response. We extract keyphrases using the TopicRank algorithm (Bougouin et al., 2013).
During our experiments, we use a 75%/5% split of the PAIR data for train, development, and test sets, and use AnnoMI for evaluation only. We report the average scores for five runs based different random seeds.
Perplexity, Coherence, Specificity. Following PARTNER (Sharma et al., 2021), we also measure the perplexity, coherence, and specificity of the rewrite using pre-trained language models.
6.1 Baselines
We compare VERVE against two template-based text style transfer baselines: Delete, Retrieve, and Generate (DRG) (Li et al., 2018) and Tag and Gen- erate (TG) (Madaan et al., 2020). For fair compar- ison with our models, we reimplement these base- lines using the same base architecture (transformer- based LM) and pretrained weights. We adjust the format of each generator input they work with a target text and a context prompt in the same way as VERVE. Also, we only implement the template generation methods, to separate implementation details 3 from our comparison and focus on the template generation and filling strategies.
6.2.1 Results
Our evaluation results are shown in Tables 2 and 3. First, we note that VERVE achieves the largest edit effect gain among baselines. We find that while TG performs poorly, DRG shows higher edit effect, although trailing significantly behind VERVE. VERVE and DRG have similar perfor- mances across different metrics, except for changes in edit effect and perplexity. Notably, the two mod- els preserve similar amounts of content, both terms of keyphrase preservation and edit rate. One interpretation is that VERVE benefits from the edit- ing "room" space for increasing the reflection score, while DRG uses it to fill in the most likely tokens.
6.2 Automatic Evaluations
During automatic evaluations, focus rewriting effectiveness (i.e., whether leads change in reflection score), content preservation, fluency, and relevance.
Moreover, we find that although the perfor- mances across datasets are slightly different, the general trends are similar, thus indicating that our framework performs well even when applied to unseen data. Finally, we attribute the poor per- formance of the TG model to the limited size of our corpora. In TG, style markers are selected via salience among n o-grams found in both corpora. We observe that this choice limits the candidates' size, leading fewer tokens being masked in the resulting template.
Edit Effect (Reflection Score). We implement the reflection scorer introduced by Min et al. (2022) measure the rellection quality of the rewrite. The reflection scorer uses client prompt and coun- selor response as input and outputs scalar value in the range [0, measuring the reflection quality in the response. We use the same training and testing split as in our rewriting model so that the test set is unobserved for the scorer. We use the scorer to compute the amount of change in reflection.
We also perform ablations for the paraphrase augmented training (paraphrase), and adaptive template updating (adaptive) meth- ods. Across datasets, these methods lead to higher change in reflection, compared the base model. Similarly, coherence and specificity are also in- creased when combining these two strategics. As expected, performance gains for both methods re- sult in lower content preservation.
A blation Results.
Content Preservation. We are also interested in measuring how much content preserved in the rewrite, as lower content preservation would reduce the "rewrite"nes ness of the generation, thus limiting its utility to the user. We use two automatic metrics measure content preservation: translation edit rate and keyphrase coverage. Translation edit rate measures the number of changes between the origi- nal and rewritten responses. Keyphrase coverage measures how much key information or concepts are included in the rewrite (Snover et al., 2006). We
Interestingly, both methods seem to increase coherence, while paraphrase training is associ- ated with higher specificity. One explanation is that paraphrase training preserves the original text keywords and key phrases. This leads to higher keyphrase coverage (paraphrase only) but a lower edit rate result than adaptive only.
Forr instance, retrieval from corpus (Li et al., 2018), or
training a separate tagger.

---

Model Metries
Change Reflection (%)
Keyphrase Coverage (%) f
Edit Rate (%)
Perplexity
Coherence (%)
Specificity (%)
VERVE adaptive only paraphrase only base VERVE
79.86
44.30
101.33
36,97
93.35
79.21
44.58
51.84
50.23
36.66
82.18
73.15
49.63
63.69
76.47
39.43
81.89
79.04
17.02
73.68
29.25
37.33
75.69
73.82
DRG (Li et al., 2018) TG (Madaan et al., 2020)
44.56
43.37
114.56
20.82
91.06
79.81
14.66
16.80
86.11
72.09
85.62
74.43
Table 2: Evaluation results for the PAIR dataset. indicates higher score better, & otherwise.
/ Metrics
Change Reflection (%) f
Keyphrase Coverage (%)
Edit Rate (%)
Perplexity
Coherence (%) f
Specificity (%)
VERVE adaptive only paraphrase only base VERVE
74.71
34.44
87.69
34.66
91.46
74.85
43.03
37.58
55.63
35.40
83.04
70.71
40.87
48.03
69.14
33,90
81.28
72.14
12.81
58.31
43.90
32.39
73.26
70.93
DRG (Li et al., 2018) TR (Madaan al., 2020)
48.60
26.58
96.49
18.43
88.24
75.15
6.93
17.14
90.84
60.60
76.44
71.58
Table Evaluation results the AnnoMI dataset. indicates higher score better, 4 otherwise.
coverage results by the reflection level of the orig- inal counselor response, using PAIR's annotation of reflection levels (Complex, Simple, and Non- reflection). Table
10
Avg Change Reflection
0.8
that VERVE im- proves reflection scores for no-reflection (NR) and simple reflections (SR). Although edit effect gains decrease for SR (since has less room for im- provement due to already being a reflection), the absolute reflection level is similar for both levels (0.88, 0.87), indicating that VERVE can handle user inputs of varying qualities. In addition, we observe that poorly performing models (TG) actually reduce the reflection quality of responses, Moreover, we observe that keyphrase coverage is greater for simple reflections. Intuitively, this is likely due to the fact that simple reflections already contain words and spans that also appear in the original response.
0.4
0.2
PAIR question
PAIR advice Original Counselor Behavior
AnnoMi question
AnnoM input
Figure 3: Analysis of edit effect by original counselor behavior. The error bars are 95% confidence intervals.
We also observe interesting differences when comparing paraphrase only and DRG, and adaptive only and DRG. In these comparisons the edit effect results similar but the con- tent preservation scores are far apart. Overall, adaptive only and paraphrase only are bet- ter at preserving content from the original response while achieving similar edit effect as DRG. This suggests that our framework provides effective way to explore the trade-off between content and edit effect.
Analysis by Original Counselor Behavior. We also analyze the changes reflection quality of the rewrite given the counselor behavior in the original response. We use PAIR's 's counselor annotations for no-reflections, including "advice" and "question" and AnnoMI annotations for "question", "therapist input". and "other's". From Figure 3, we see that VERVE performs better for "therapist input" than "question" across both datasets. This suggests that in response rewriting, beneficial to con- sider the dialog act of the original utterance.
Analysis by Original Reflection Level. Addi- tionally, we analyze the edit effect and keyphrase

---

Change in Reflection (%)
Keyphrase Coverage (%)
NR
SR
NR
SR
VERVE
78.69
32.08
VERVE
35.95
60.48
DRG
48.13
23.56
DRG
30.91
62.85
TG
10.28
-15.52
TG
15.48
21.51
Table Analysis by Original Response Level, PAIR + AnnoMI dataset. NR and SR refer to non-reflection and simple reflection.
6.3
Human Evaluation
full test split of the PAIR dataset. Since the flections used in this evaluation are not previously seen by our models, the degree of similarity is an indication of how closely model rewrites resemble MI expert reflections.
For our human evaluation, consider both experts (counseling coaches, counseling trainees) and non- experts (clients) users. Although our framework is intended for training counselors, we also evaluate with non-experts to ensure that our system can cre- ate reflections that sound fluent and empathetic to clients, who are non-MI experts.
6.3.1 Results
Non-expert Evaluation Results for the A/B test- ing comparison of the models are shown in Table 5, We measure the fraction of times VERVE is pre- ferred against each baseline. Similar to automatic evaluation results, VERVE significantly outper- forms TG while having smaller gap over DRG. It is notable that VERVE surpass DRG on fluency and specificity, while in automatic evaluation DRG outperforms in perplexity and specificity. Our evaluation focuses on comparing the qual- ity of the rewritten samples, rather than evaluating whether the generations are indeed rewrites for the original response. Annotation of the usefulness or faithfulness of rewrites is difficult and subject to in- dividual preferences variations. Overall, ob- serve that VERVE maintains competitive higher rates of content preservation while outperforming the baselines edit effect and conversational qual- ity on both automatic and non-expert evaluation thus showing its potential for response rewriting.
Non-expert Evaluation. To conduct a human evaluation of the models by non-expert users, we recruit four graduate students without expertise in mental health counseling motivational interview- ing. This setting is intended evaluate our system from the perspective of counseling patients, who are not experts in MI. To this end, we sample a col- lection of 50 rewrites, each generated by VERVE and the two baselines, and ask the participants to compare the generations of VERVE against base- lines, across four dimensions: fluency, coherence, specificity, and empathy level. We allow no ties during the annotation process.
Expert Evaluation. We also evaluate our system from the perspective of MI experts. To this end, we recruit two MI experts (professional MI coaches) to evaluate model rewrites against MI-expert reflec- tions. We use a set of 46 parallel samples from PAIR test split set. Annotators were asked to in- dicate whether they prefer "A, B, or Tie" when randomly shown reflections either written by our models (rewrites) or by MI experts in response to given prompt. For a fair comparison of the model and human experts, original responses were hid- den. During our evaluations, we opt for comparing VERVE and DRG only against MI experts to miti- gate annotation expenses.
Expert Evaluation Results for our A/B testing of the rewriting models against MI experts shown in Table 6. Unsurprisingly, reflections rewritten by MI experts are generally preferred over model generations. Nonetheless, we find that VERVE is more frequently preferred over experts when compared to DRG by a large margin (36.96% vs 18. 83.48)). Additionally, in one-to-one comparison our model outperforms DRG with a win rate of 68.48% and tic rate of 8. 3.70%. Morcover we found that VERVE most similar to MI expert reflections by large margin in all metrics, as shown Table 7. These results indicate that our system capable of producing more expert-like reflections than the baseline models.
We also conducted comparisons between rewrites and expert-written complex reflections us- ing text similarity measures, such BLEURT, Me-
teor, BLEU,, and BERTScore (Sellam et al., 2020; Banerjee and Lavie, 2005; Papineni et al., 2002; Zhang et al., 2020). For this analysis, we use the

---

Comparison
Fluency
Coherence
Specificity
Empathy
Against DRG (%)
61.5
56.5
58.5
62.0
Against TG (%)
87.5
84.0
87.0
90.5
Table 5: Human comparisons of VERVE vs Baselines using fluency, coherence, specificity, and empathy. The percentages indicate the ratio of VERVE win against respective baselines.
Model
Win (56)
Lose (9)
Tie (%)
pared to advice or input. Thus, these cases repre- sent situations with greater opportunities for useful feedback. We conjecture that this is due to the overall differences in style and semantics of utter- ances with different conversational functions and behaviors. For instance, we expect that the direc- tive language in "advice" "input" responses is more difficult to turn into reflective language.
VERVE
36.96
54.35
8.70
DRG
18.48
73.91
7.61
Table 6: Model vs MI expert reflections. The percent- ages indicate the ratio of model wins against MI experts.
Metric
||
Meteor
BLEU
BERTScore
BLEURT
VERVE
67.30
35.47
58.25
-29.04
DRG
50.23
19.84
44.58
-54.04
Conclusion
TG
27.29
3.61
37.00
-76.73
In recent years, the disparity between accessible and timely mental health care and the increasing demand for psychotherapy has significantly dete- riorated, highlighting the need for more scalable and efficient ways to train new counselors. NLP can assist this counselor training process through automated feedback, which previously was only available through expert supervision.
Table 7: Text similarity scores of the different model rewrites against MI-experts.
7 Discussion
Does template editing work for counselor re- sponse rewriting? We argue that template edit- ing is useful strategy for rewriting counselor re- sponses. However, several adaptations are needed apply the counseling domain. For instance, the prompt should be considered as an additional input and the mask template should be modified accordingly. We found that attention-ban based token masking is a better fit for response rewriting than n-gram n-gram-based masking in Madaan et al. (2020); Li et al. (2018) since the relationship between prompts and responses be naturally modeled by the for- mer. Moreover, it is helpful model flexible mapping between templates and reconstructions, because response rewriting may require greater amount of text editing than style transfer.
In this paper, we introduced the task of counselor response rewriting generate automatic counsel- ing feedback. We introduced VERVE, template- based approach with paraphrase-augmenttd train- ing and adaptive template updating, which can transform non-reflective counselor responses into reflective responses. Without access to parallel data, VERVE achieves higher editing effect than other baseline systems by using flexible template reconstruction approaches. also has the ability to adjust the attribute masking step without unnec- essarily sacrificing content preservation. future work, we plan to pilot our system in educational settings and explore how VERVE can provide sup- port for student training or coaching.
When should rewrites be suggested? Measur- ing content preservation and usefulness of text rewriting are still open problems. However, in the context of counselor response rewriting in MI, we can provide few guidelines based on our empiri- cal findings. First, we should consider the quality of the original counselor response. Table shows that rewrites of simple reflections have higher con- tent preservation. Second, when rewriting non- reflections, the original intent of the response (coun- selor verbal behavior) likely matters. In Figure 3, we see that questions have larger edit effects com-
The VERVE system is publicly available from https://gi thu::://ggttb..../eee com/ wtt.c/om//mmmii wwt.///iiooonn//eeereeoeoeemeeee
Limitations
The central intuition behind rewriting as a training or coaching feedback tool is that rewriting can pre- serve core ideas already present in responses and repurpose them to increase response quality. How- ever, some responses, especially responses con- taining prescriptive language, may not have sal- vageable phrases. Although we analyze the impact

---

of original response behavior on rewriting results, future work on how to identify ideal rewrite oppor- tunities is needed.
tion and/or Summarization, pages 65-72, Ann Arbor, Michigan. Association for Computational Linguis- tics.
Also, measuring content preservation is still an open problem. In this work, we use the fraction of keyphrases in the rewrite as a proxy for measuring content preservation. However, this measure does not fully capture situations where ideas or concepts are expressed in different ways.
Adrien Bougouin, Florian Boudin, and Béatrice Daille. 2013. TopicRank: Graph-based topic ranking for keyphrase extraction. In Proceedings of the Sixth International Joint Conference on Natural Language Processing, pages 543-551, Nagoya, Japan. Asian Federation of Natural Language Processing.
Moreover, we note the reflection scorer (PAIR) used in this project is not flawless. We found that the scorer better at identifying "reflection- sounding" language and often gave high scorers to incoherent or factually wrong responses. Therefore, we used PAIR in conjunction with an evaluation conducted by non-experts and MI experts.
Rebekah Coley and Christopher Baum. 2021. Trends in mental health symptoms, service use, and unmet need for services among adults through the first nine months of the covid-19 pandemic.
Torrey A. Creed, Leah Salama, Roisín A. Slevin, Michael Tanana, 7ac E. Imel, Shrikanth S. Narayanan, and David C. Atkins. 2022. Enhancing the quality of cognitive behavioral therapy in com- munity mental health through artificial intelligence generated fidelity feedback (project affect): a study protocol. BMC Health Services Research, 22.
Finally, in this project, do not consider how large language models (LLMs) can be incorporated as a component in our rewriting framework. In- stead, we focus on using smaller, finetunable mod- els that are relatively easier to train, while also being transparent in terms of having components that can be directly observed and examined, such as the attention weights used for template extraction. For future work, we plan to explore how LLMs can augment or complement systems like ours.
Ning Dai, Jianze Liang, Xipeng Qiu, and Xuanjing Huang. 2019. Style transformer: Unpaired text style transfer without disentangled latent representation. In Proceedings of the 57th Annual Meeting of the As- sociation for Computational Linguistics, pages 5997- 6007, Florence, Italy. Association for Computational Linguistics.
Jacob Devlin, Ming-Wei Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language under- standing. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Tech- nologies, Volume (Long and Short Papers), pages 4171-4186, Minneapolis, Minnesota. Association for Computational Linguistics.
Ethics Statement
We emphasize that our framework is not a tool to process and improve counselor utterances in real counseling practice, nor is it meant replace or execute the work of human counselors. Rather, is a tool that focuses on training and coaching learn- ers, and since we find that the generator can make incorrect edits, the rewrites are to be considered only as suggestions. We recommend that in educa- tional deployments safeguards are placed to filter out harmful or toxic edits.
Nikolaos Flemotomos. Victor R. Martinez, Zhuohao Chen, Torrey A. Creed, David C. Atkins, and Shrikanth Narayanan. 2021a. Automated quality sessment of cognitive behavioral therapy sessions through highly contextualized language representa- tions. CoRR, abs/2102 11573.
References
Nikolaos Flemotomos, Victor R. Martinez, Zhuohao Chen, Karan Singla, Victor Ardulov, Raghuveer Peri, Derek D. Caperton, James Gibson, Michael J Tanana, Panayiotis G. Georgiou, Jake Van Epps, S. P. Lord, Tad Hirsch, Zac Imel, David C. Atkins, and Shrikanth S. Narayanan. 202 "am a good thera- pist?" automated evaluation of psychotherapy skills using speech and language technologies. Behavior research methods.
Victor Ardulov, Torrey A. Creed, David C. Atkins, and Shrikanth Narayanan. 2022. Local dynamic mode of cognitive behavioral therapy.
John Langshaw Austin. 1962. How to do things with words. William James Lectures. Oxford University Press.
Satanjeev Banerjee and Alon Lavie. 2005. METEOR:
An automatic metric for MT evaluation with im- proved correlation with human judgments. In Pro- ceedings of the ACL Workshop on Intrinsic and Ex- Irinsic Evaluation Measures for Machine Transla-
Di Jin, Zhijing Jin, Zhiting Hu, Olga Vechtomova, and Rada Mihalcea. 2022. Deep learning for text style transfer: survey. Computational Linguisties. 48(1): 1055-200.

---

Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2019. BART: denoising sequence-to-sseqenee pre-training for natural language generation, translation, and com- prehension. CoRR, abs/1910. 134411.
Theresa Moyers, Lauren Rowell, Jennifer Manuel, Denise Ernst, and Jon Houck. 2016a. The motiva- tional interviewing treatment inlegrity code (miti 4): Rationale, preliminary reliability and validity. Jour- nal of Substance Abuse Trealment, 65.
Theresa Moyers, Lauren Rowell, Jennifer Manuel, Denise Ernst, and Jon Houck. 2016b. The motiva- tional interviewing treatment integrity code (miti 4): Rationale, preliminary reliability and validity. Jour- nal of Substance Abuse Treetment, 65.
Juncen Li, Robin Jia, He He, and Percy Liang. 2018. Delete, retrieve, generate: simple approach senti- ment and style transfer. In North American Chapter of the Association for Computational Linguistics.
Sarah Lord, Flliaa Sheng, 7ac Tmel, John Baer, and David Atkins. 2014, More than reflections: Empathy motivational interviewing includes language style synchrony between therapist and client. Behavior Therapy, 46.
Kishore Papineni. Salim Roukos, Todd Ward, and Wei- Jing Zhu. 2002. Bleu: a method for automatic evalu- ation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Compu- tational Linguistics, pages 31 1-318, Philadelphia, Pennsylvania, USA. Association for Computational Linguistics.
Aaron Lyon, Shannon Stirman, Suzanne Kerns, and Eric Bruns. 2010. Developing the mental health work- force: Review and upplication of training approaches from multiple disciplines. Administration and policy in mental health, 38:238-53.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury. Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang. Zachary Vito,, Martin Raison, Alykhan Te- jani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang. Junjie Bai, and Soumith Chintala. 2019. Pytorch: An imperative style, high-performance deep learning library. In Advances in Neural Information Process- ing Systems 32, pages 8024-8035. Curran Associates, Inc.
Aman Madaan, Amrith Sethur, Tanmay Parekh, Barn- abas Poczos, Graham Neubig, Yiming Yang. Ruslan Salakhutdinov, Alan W Black, and Shrimai Prabhu- moye. 2020. Politeness transfer: A tag and generate approach. In Proceedings of the 58th Annual Meet- ing of the Association for Computational Linguistics, pages 1 1869-1881, Online. Association for Computa- tional Linguistics.
Fiona McMaster and Kenneth Resnicow, 2015. Valida- tion of the one pass measure for motivational inter- viewing competence. Patient education and counsel- ing, 98 4:499-505.
Verónica Pérez-Rosas, Rada Mihalcea, Kenneth Resni- cow, Satinder Singh, Lawrence An, Kathy J. Goggin, and Delwyn Catley. 2017. Predicting counselor be- haviors in motivational interviewing encounters. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Lin- guistics: Volume 1, Long Papers, pages 122-1137. Valencia, Spain. Association for Computational Lin- guistics.
William R Miller and Stephen Rollnick. 2013. Motivu- tional interviewing: Helping people change, Third edition. The Guilford Press.
Do June Min, Verónica Pésez-Rosas, Kenneth Resni- cow, and Rada Mihalcca. 2022. PAIR: Prompt-aware margin ranking for counselor reflection scoring in motivational interviewing. Proceedings of the 2022 Conference on Empirical Methods in Natural Lan- guage Processing.
Gerard Schippers and Cas Schaap 2005. The moti- vational interviewing skill code: Reliability and a critical appraisal. Behavioural and Cognitive Psy- chotherapy, 33:285 - 298.
Adam S. Miner, S. Fleming, Albert Haque, Jason Alan Fries, Tim Althoff, Denise E. Wilfley, William Stew- art Agras, Amold Milstein, Jeff Hancock, Steven M. Asch, Shannon Wiltsey Stirman, Bruce A. Arnow, and Nigam Haresh Shah. 2022. A computational approach to measure the linguistic characteristics of psychotherapy timing, responsiveness, and consis- tency. npj Mental Health Research, I.
Thibault Sellam, Dipanjan Das, and Ankur Parikh. 2020. BLEURT: Learning robust metrics for text genera- tion. In Proceedings of the 58th Annual Meeting of the A Association for Computational Linguistics, pages 7881-7892. Online. Association for Computational Linguistics.
Ashish Sharma, Inna Wanyin Lin, Adan S. Miner, David C. Atkins, and Tim Althoff. 2021. Towards facilitating empathic conversations in online mental health support: A reinforcement learning approach. Proceedings of the Web Conference 2021.
Remi Mir, Bjarke Felbo, Nick Obradovich, and Iyad Rahwan. 2019. Evaluating style transfer for text. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies, Volume I (Long and Short Papers), pages 495-504, Minneapolis, Minnesota. Association for Computa- tional Linguistics.
Ashish Sharma, Adam S Miner, David C Atkins, and Tim Althoff. 2020. A computational approach un- derstanding empathy expressed text-based mental health support. In EMNLP.

---

Ashish Sharma, Kevin Rushton, Inna Wanyin Lin, David Wadden, Khendra G. Lucas, Adam S. Miner, Theresa Nguyen, and Tim Althoff. 2023. Cogni- tive reframing of negative thoughts through human- language model interaction.
Text Style Transfer. Text style transfer tasks on various styles or attributes have been well-studied in NLP, including sentiment, formality, or toxicity transfer to name a few (Jin et al., 2022). One no- table difference between style transfer and response rewriting is that reflection is a verbal strategy that is closer to a dialog act, than a style or sentiment of an utterance. A dialogue act (DA) is defined as an utterance that serves function in the context of a conversation, such as questioning, making a statement, or requesting an action (Austin, 1962). Commonly used MI coding schemes such as MITI or MISC use DA-like codes such as questions, giv- ing information, etc (Schippers and Schaap 2005; Moyers et al., 201 On the other hand, style transfer is not expected to alter the dialog act of an utterance.
Siqi Shen, Veronica Perez-Rosas, Charles Welch, Sou- janya Poria, and Rada Mihalcea. 2022. Knowledge enhanced reflection generation for counseling dia- logues. In Proceedings of the 60th Annual Meet- ing of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3096-3100 Dublin, Ireland. Association for Computational Linguisties.
Siqi Shen. Charles Welch, Rada Mihalcea, and Verónica Pérez-Rosas. 2020. Counseling-styy reflection gen- cration using generative pretrained transformers with augmented context. In Proceedings of the 21th An- nual Meeting of the Special Interest Group on Dis- course and Dialogue, pages 10-20, 1st virtual meet- ing. Association for Computational Linguisties.
Matthew Snover, Bonnie Dorr, Rich Schwartz, Linnea Micciulla, and John Makhoul. 2006. A study of trans- lation edit rate with targeted human annotation. In Proceedings of the 7th Conference of the Association for Machine Translation in the Americas: Technical Papers. pages 223 231, Cambridge, Massachusetts, USA. Association for Machine Translation in the Americas.
Empathetic Rewriting. Another highly related task is empathetic rewriting, first proposed by Sharma et al. (2021) as the PARTNER system. We first note that as with text style transfer, empathetic rewriting also should not alter the function or dialog act of an utterance. Moreover, PARTNER targets online text-based comments that usually longer than counseling utterances. Also, although PART- NER can theoretically make fine tineggrinedd token- level edits, its edit scope is at a sentence level and largely operates by inserting sentences. Finally, PARTNER focuses on sentence-level edits (remov- ing and adding sentences), and uses a warm-start strategy where pseudo-paralle corpus is created by identifying high-empatti sentences from the text to create low-high empathy pairs. This is dif- ferent from our response rewriting since we focus on transforming a relatively short utterance consist- ing of a few sentences.
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier- ric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush. 2020. Transform- ers: State-of-the-art natural language processing. In Proceedings of the 2020 Conference Empirical Methods in Natural Language Processing: System Demonstrations, pages 38-45, Online. Association for Computational Linguistics.
Zixiu Wu, Simone Balloceu, Vivek Kumar, Rim Helaoui, Ehud Reiter, Diego Reforgiato Recupero, and Daniele Riboni. 2022. Anno-mi: A dataset of expert-annotated counselling dialogues. ICASSP 2022 2022 IEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 6177-
A.1.1 PAIR Dataset
PAIR collection of single-turn client-counselor exchanges, collected by (Min et al., 2022). The au- thors use both expert and crowdsource annotation, using the former for reflection annotation which requires MI expertise, and the latter for collecting non-reflections containing prescriptive language. Following MI literature (Moyers et al., 201 each counselor response is coded to one of Com- plex Reflection (CR), Simple Reflection (SR), Non-Reflection (NR). Examples are shown in Ta- ble CRs are considered higher-quality responses compared to SRs, which are ranked above NRs. In this project, we consider CRs as the gold standard.
Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. 2020. Bertscore: Eval- uating text generation with bert. In International Conference on Learning Representations.
A Appendix
A.1 Comparison with Similar Tasks
Here, we briefly discuss how counselor response rewriting is distinguished from related tasks such as text style transfer and empathetic rewriting.

---

Dataset
Prompt
Response
Label
Your mother's death was devastating You re worried you may die the same way she did. You believe you will die from breast cancer, just like your mom. Are you giving up?
CR
PAIR
My mother died breast cancer, know I'm going die of too.
SR
NR
Well, I'd my children settled and my grandchildren growing up and should be exumple ihem.
for your -your fumily's important you?
So
AnnoMI
NR
Table 8: Sampled client-counselor exchanges from PAIR and AnnoMI datasets
That is, we aim rewrite SRs and NRs into CRs.
A.3
VERVE Implementation Details
We list the transformer architectures and pretrained weights used for the models used in the project.
A.1.2 AnnoMI Dataset
AnnoMI is a conversation dataset comprising 133 carefully transcribed xxpert-annotated demonstra- tions of MI counseling, collected from educational video sources, such as AlexanderStreent" (Wu et al.. 2022). Although AnnoMI datasets annotated with session-level counseling quality labels (high low), we only use utterance-leve behavioral codes.
Template Extraction: bert-base-uncanl (Devlin et al., 2019). We use BERT to lever- age its well-trained pretrained weights, but any transformer model that can be trained a classifier and whose attention weights can be extracted can be used.
Template Filling: facebook/bart-large (Lewis et al., 2019). Our choice of BART a template filling model is motivated by the fact that BART is trained with sequence denoising objective, which involves filling in corrupted (masked) tokens.
AnnoMI consists of full session-length conver- sations and is different from PAIR exchanges in several ways. Since the dataset is transcribed from audiovisual sources, it includes many speech disflu- encies, repetitions, interruptions ("Um", "Uh", "I mean-" etc). Thus, we process them to extract single-turn exchanges (client prompt and counselor response).
Paraphrase Model: tuner007/pegasioe paraphrase https://hugginggace co/ tuner tunerr@77/pegasu__pprre that any reasonably well-performing paraphrase model can be used for the paraphrase-augmente training step.
We note
A.2 AnnoMI Processing Step
To extract prompt & non-reflection pairs from the AnnoMI dataset, we take the following steps:
1. We flatten the transcripts into consecutive client utterance and counselor utterance pairs.
For the sampling algorithm used in the gener- ation of responses, we used beam sampling with num beams=5.
2. We filter out pairs that meet any of the follow- ing criteria:
Adaptive Template Updating. At maximum of 5 iterations, we decrease the content weight C by 0.1 if the difference is <= 0.2.
The counselor behavior not annotated as a reflection.
A.4 Baseline Hyperparemeters
The client utterance string starts or ends with "-" This is to filter out interruptions or continued utterances.
The baselines tested in this project require setting hyperparameter values for the template creation process. Since the domain of the text is different, manually tune the threshold parameters by mon- itoring the reflection score.
remove common speech disfluencies that we manually identified.
The client utterance is shorter than 16 words.
n grams considered: 1,2.3-grams.
The counselor utterance shorter than words.
DRG: Threshold: 0.3.
"nttps://aleeanderstreettt
TG: y 0.75, threshold: 0.5.

---

A.5 Computational Resources
For training, we use NVIDIA GeForce RTX 2080 Ti for 5 epochs, resulting in a training time of 0.5 hours. We use Pytorch and Huggingface libraries to implement and run our models (Paszke et al., 2019; Wolf et al., 2020).
A.6 Automatic Metrics Implementation
For the implementation of automatic metrics (perplexity, coherence, and specificity), we fol- low the implementation in https://g/t tttbb/...tuu.ccom. behavioral-datt//PARTNEE
A.7 Human Evaluation Details
A.7.1 Non-expert Evaluation
Instead of asking participants to evaluate the reflec- tion level of responses, we choose empathy evaluation criterion, because the participants non-experts. MI literature emphasizes that at lower empathy levels, reflective listening is absent, while high empathy is related to skillful use of reflective listening (Miller and Rollnick, 2013). Moreover, the empathetic communication mechanism of "ex- ploration" analyzed in Sharma et al. (2020) is simi- lar to reflection in that both strategies acknowledge the concerns of the client while also actively infer- ring the client's unstated feelings and expectations.
Recruitment and Informed Consent. For our human evaluation, we recruit four PhD students from the department who are in the third or later stages of the degree program. The students are proficient in English and do not have expertise in mental health or MI. They are volunteers and gave informed consent to the usage of their annotation in our project.
Annotator Instructions.
Fluency: How fluent does the response sound?
Coherence: Is the response coherent and relevant reply to the prompt?
Specificity: Is the response specific to the given prompt? (A response is not specific if it can serve as a sensible response to many other prompts.)
Empathy: How empathetic the response. given the client prompt?
A.8 Sample Generations
We show sample generations from VERVE and the baselines in Table 9.

---

Prompt
Response (Non-Refiection)
Model
Rewritten Response
My mom said that I can't It cat thything after dinner (which is around 5pm) because all that food goes right to your thighs. (PAIR)
Has that been your expe- rience when you'se ve gained weight in the past?
VERVE
You' ve gained weight in the past and you're concerned that eating after dinner could be one of the reasons why.
DRG
Since you't ve lost weight in the past, has that been your experience when you' 've been trying to lose weight?
TG
What impact eating has on your thighs has?
Expert-written
frustrating when you get hungry after Spm but feel like you shouldn't 't eat any- thing becuase of your mom's rule.
Well, I'd like to see my chil- dren settled and my grand- children growing up and should, suppose should be an example to them. (An- noMI)
So can be in there for your- your family's 's important to you?
VERVE
Your family is important to you so you's 'd like to see be in there for your family.
DRG
It's important to you to he role model for your chil- dren and grandchildren, so can be in there for your- your family's 's importance to you.
TG
So much so that your-your family feels important to you?
Table 9; Sample Rewritings from the models, For the AnnoMi dataset, counselor non reflections are not paired with parallel reflections.


Reviews:
[{"review_id": "qs9UXwvSOS", "reviewer": "Reviewer_LWEi", "summary": "", "questions": "The paper raises an important problem to solve. However, the authors have made some assumptions on which I would like to gain clarity. How did the authors segregate between behavior-relevant and non-relevant parts of the dataset? I would like to see some explanation here.", "limitations": "", "rating": 2, "confidence": 5, "soundness": 2, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the reviewer's careful consideration of our paper and their thoughtful feedback. We are glad they found our work to be addressing an important problem. We would like to address the concerns raised in the review and provide further insight into our work.\n\n[Regarding the concern about the novelty and contribution of our work]. We acknowledge that there is prior research in the area of response rewriting. Our focus, however, is on the specific context of counseling response rewriting, which has unique nuances due to the requirement for reflective listening in MI. While the concept of paraphrase-augmented training is not entirely novel, we emphasize its application to the counseling domain, which has distinct linguistic and psychological aspects. We agree that domain shifting is a concern, and we have taken steps to address it by utilizing an adaptive template updating mechanism, which we believe mitigates the problem to a degree by incrementally updating the original response.\n\nWe agree knowing \"when not to take action\" is a critical problem. In our previous studies and deployments, our approach to determining the appropriate instances for suggesting rewrites or providing exemplary counselor responses for MI counselor trainers was to employ a reflection detector/scoring model. This model would identify cases where the output fell below a certain threshold, akin to the process that governs the termination of adaptive template updating. However, this approach is a very simple method that may fail in the case of domain shifts, as pointed out by the reviewer. To address this limitation, we have curated a dataset in which each conversation has been annotated with a specific counseling topic, such as eating disorders or smoking cessation. In an upcoming follow-up study, we plan to leverage this dataset to investigate the impact of domain shifts and to devise more effective countermeasures.\n\n[Regarding domain-centric metrics] Our evaluation encompassed a mix of expert and non-expert validation, with our expert assessment aligning with how MI experts gauge and identify reflective listening in practice. However, we agree that further studying the effect of rewritten reflections through psychiatric validation as suggested by the reviewer would further ground our results. We are consulting with our domain expert cooperators about this step, and plan to include additional analyses in the camera-ready.\n\n[Regarding the choice of baselines] We agree that a broader selection of baselines could provide a more comprehensive performance comparison. Our main focus in our work was to compare against template-based editing methods. \n\nIn response to this point, we implemented and tested an LLM-prompting-based rewriting approach (in-context learning with instructions and examples of reflections) on the automatic evaluation metrics and also did a qualitative review of the generations. We used OpenAI’s closed-sourced gpt-3.5-turbo-16k-0613 model. The following are the results:\nPAIR: Change in Reflection: 85.00, Keyphrase Coverage: 13.26, Edit Rate: 246.12, Perplexity: 17.2271, Coherence: 97.58, Specificity: 83.02\nAnnoMI: Change in Reflection: 60.86, Keyphrase Coverage: 18.41, Edit Rate: 131.73, Perplexity: 18.1430, Coherence: 93.64, Specificity: 77.14\nOverall, we observe that while LLM method is capable of producing highly reflective and fluent rewrites, at the same time it is also the furthest from the original text thus having the least contextual coherence.\n\nWe posit that LLM rewriting functions within a distinct paradigm, neither superior nor inferior, but different from ours. Our strategy revolves around templates, enabling the model to operate within predefined structures and facilitate incremental adjustments. In contrast, LLM-driven prompts appear to enjoy a higher degree of content addition and removal flexibility. Managing this variance can be challenging, although refining prompts presents a potential solution. We think that a hybrid approach, where templates are provided to the LLM for completion, presents an opportunity we hope to explore subsequently.\n\nAdditionally, we think that a locally hosted model for counselor response rewriting can be preferable over LLMs (especially closed-source ones only accessible via API) for privacy concerns and resource efficiency. It ensures that sensitive information is kept in a controlled environment and provides more direct control over computational resources and time constraints.\n\nWe will include details about our LLM methodology (prompt, model, example selection) in the revision, and also discuss recent methodologies rooted in LLMs within analogous tasks (for instance, cognitive reframing, by Sharma et al: https://aclanthology.org/2023.acl-long.555/). We intend to elaborate on both the connections and distinctions between our approach and these LLM-based strategies. \n\nConcerning the query about segregating behavior-relevant and non-relevant portions of the dataset, we primarily relied on the annotations available within the datasets. In the PAIR dataset, each data sample constitutes a single exchange addressing a counseling topic/issue between a client and a counselor. The AnnoMI dataset comprises counseling dialogues spanning entire sessions. To process this, we identified client-counselor utterance turns concluding with a counselor reflection (identified by the dataset labels). We also filtered out irrelevant behaviors such as disfluencies or interruptions (uh, umm, etc.).\n\nWe thank the reviewer for the careful and valuable feedback as it helps us improve the clarity and rigor of our work. We are committed to addressing the concerns raised and enhancing the quality of our paper.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The authors targeted an essential problem of counselor training in a very sensitive research space of mental health.\n\n2. The proposed system is a better fit in the current research scope, which targets adapting and rewriting responses rather than generating new ones. This approach is more realistic than a complete replacement of a human therapist.   \n\n3. Additional expert validation and ethical considerations are ensured to justify the research standard in this space.", "reasons_to_reject": "While the paper addresses an important problem related to rewriting responses in the counseling education domain, there are some concerns regarding its novelty and contribution. \n\n1. The literature review shows that ample research already exists in this area. Although the authors present a paraphrase-augmented training basis as their USP, the problem of topic/domain shifting in paraphrase-augmented training is a common concern. The paper fails to discuss the intricacies involved with the approach and \"when not to take action\" especially in the case of uncontrolled domain shift.\n\n2. Although a mix of expert and non-expert validation has been included, no domain-centric or expert-guided metric is discussed. Being from a very similar field, I believe it is important to involve metrics or methods in psychiatric validation that lead to computational and psychological performance comparison. The question is why authors missed presenting such domain-centric metrics even with the presence of expert evaluators. \n\n3. Despite the existence of ample work, the selection of baselines is very limited to naive/old models. I encourage authors to add more baselines to perform robust performance comparisons.\n\n4. The proposed work lacks novelty. Several methods in the past have tried to apply very similar incorporation to state-of-the-art paradigms and have reported gains in response rewriting space and, in some cases, counseling rewriting too.", "questions_to_authors": "The paper raises an important problem to solve. However, the authors have made some assumptions on which I would like to gain clarity. How did the authors segregate between behavior-relevant and non-relevant parts of the dataset? I would like to see some explanation here.", "paper_topic_and_main_contributions": "The paper tackles the problem of response rewriting in the mental health space. The authors exploited two specific techniques involving paraphrase-augmented training and template updating. Further, the results inherit both automatic and human evaluation to show the effectiveness of the proposed system.", "excitement": 2, "reproducibility": null, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "-", "Typos_Grammar_Style_and_Presentation_Improvements": "-"}, "extra_text": {}}, {"review_id": "CY9i1qXXSK", "reviewer": "Reviewer_DEMb", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": 2, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the reviewer's thoughtful evaluation of our paper and their valuable feedback regarding the alignment of our work with the current state of technology and the absence of a comparison with large language models (LLMs).\n\n[Regarding contextualization within the latest technological developments] We acknowledge the importance of considering the impact of LLMs on the field of NLP. While our paper was conceived before the widespread adoption of these models in the field of counseling response generation, we understand that LLMs have indeed made significant strides in generating empathy-aware responses. We will revise our introduction and motivation sections to better situate our work within the context of LLMs and their capabilities. \n\nWe also recognize the omission of a comparison with LLMs in our analysis, which could provide a more comprehensive evaluation of our proposed techniques. It is true that LLMs have become benchmarks in various NLP tasks, including response rewriting. \n\nIn response to this point, we implemented and tested an LLM-prompting-based rewriting approach (in-context learning with instructions and examples of reflections) on the automatic evaluation metrics and also did a qualitative review of the generations. The following are the results:\nPAIR: Change in Reflection: 85.00, Keyphrase Coverage: 13.26, Edit Rate: 246.12, Perplexity: 17.2271, Coherence: 97.58, Specificity: 83.02\nAnnoMI: Change in Reflection: 60.86, Keyphrase Coverage: 18.41, Edit Rate: 131.73, Perplexity: 18.1430, Coherence: 93.64, Specificity: 77.14\nOverall, we observe that the LLM method is capable of producing highly reflective and fluent rewrites, surpassing even the performance of VERVE in the PAIR dataset. However, we also note that the LLM approach results in the most significant departure from the original text.\n\nWe posit that LLM rewriting functions within a distinct paradigm, neither superior nor inferior, but different from ours. Our strategy revolves around templates, enabling the model to operate within predefined structures and facilitate incremental adjustments. In contrast, LLM-driven prompts appear to enjoy a higher degree of content addition and removal flexibility. Managing this variance can be challenging, although refining prompts presents a potential solution. We think that a hybrid approach, where templates are provided to the LLM for completion, presents an opportunity we hope to explore subsequently.\n\nAdditionally, we think that a locally hosted model for counselor response rewriting can be preferable over LLMs (especially closed-source ones only accessible via API) for privacy concerns and resource efficiency. It ensures that sensitive information is kept in a controlled environment and provides more direct control over computational resources and time constraints.\n\nWe also will include a discussion of recent LLM-based approaches in related tasks (such as cognitive reframing, by Sharma et al https://aclanthology.org/2023.acl-long.555/, and how our approach is related to and different from them.\n\nThank you for your insightful review. We are committed to addressing these concerns to enhance the relevance, rigor, and impact of our work.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "* The study on reflective listening is important for advanced conversational agents. \n* This paper is well-written. \n* The proposed approach is interesting and show its effectiveness in experiments. \n", "reasons_to_reject": "* Given that the study is being conducted in 2023, the motivation and the research background of this work should be more closely aligned to the current state of technology, specifically the advent and impact of large language models. It's essential for research to be contextualized within the most recent and significant developments in the field to ensure its relevance and applicability. As LLMs have shown their ability in generating empathy-aware responses, it is important to check if the kind of reflective listening addressed in this work is still a challenge.  \n* The paper exhibits a significant oversight by not comparing its techniques with the capabilities of large language models. LLMs have become benchmarks in numerous NLP tasks, including conditional paraphrasing. In this work, LLMs' absence in the comparative analysis leaves a substantial gap in the research. Without this comparison, it's challenging to ascertain the true efficacy and novelty of the proposed methods in light of existing cutting-edge technologies.", "questions_to_authors": "", "paper_topic_and_main_contributions": "Reflective listening is essential for counselors practicing motivational interviewing, as it focuses on understanding and delving deeper into the client's expressions. This study introduces a task named \"counseling response rewriting,\" aiming to transform non-reflective statements into reflective ones. For this, the VERVE framework was created, utilizing template editing techniques from text style transfer. Unique about VERVE is that it doesn't demand parallel data, acknowledging that obtaining expertly annotated rewritten responses is both costly and time-intensive. Two key methods were introduced: paraphrase-augmented training and adaptively template updating. The former encourages the text generator to create a varied connection between templates and full responses, ensuring the end rewrite isn't strictly bound by the template. The latter manages the balance between content editing, focusing on maintaining aspects of the original user response. \n\nThe motivation of applying reflective listening is a very important topic in natural language generation. This work proposes an approach that diversity the outcome from a template-based rewrite approach. When compared with previous benchmarks, the proposed framework VERVE showed superior performance in reflection scores while still retaining the original content. However, a notable shortcoming is the absence of comparisons with Large Language Models (LLMs). Given that LLMs have demonstrated proficiency in empathy-aware conversations and paraphrasing, they could serve as potential benchmarks or even offer supplementary techniques in both data generation and the main task of reflective listening. The exclusion of LLMs as a comparative baseline may limit the comprehensiveness of the evaluation, potentially overlooking the advancements and capabilities that these models bring to the domain. Incorporating LLMs in iterations would provide a more holistic view of the landscape and might lead to further optimizations or integrations that could enhance the effectiveness of the proposed framework. In other words, with comparison with LLMs, the contribution of this work can be clearer. ", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "gdqvzIpV0V", "reviewer": "Reviewer_AcJk", "summary": "", "questions": "- Why is the reflection behavior more preferable than the one that gives advice in Figure 1?\n- Why is the penultimate self-attention layer used rather than the final layer? (lines 231-233)\n- Is the research on reflective rewriting relatively new? The suggested baselines seem to be not specifically trained for this task and wonder whether there are stronger baselines that VERVE should be compared against.", "limitations": "", "rating": 4, "confidence": 2, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the detailed review and valuable feedback provided by the reviewer. We are glad they found our paper well-written and easy to understand, and our analyses comprehensive. Their insights are very helpful in refining and enhancing our paper.\n\n[Regarding the significance of reflective listening and motivational interviewing] We recognize the importance of providing a clear context for readers unfamiliar with these concepts. We will incorporate a more comprehensive explanation of these concepts in the introduction, detailing why reflective listening is valuable in counseling and elaborating on the principles of motivational interviewing. This will ensure that readers can better appreciate the foundation on which our work is built.\n\n[Regarding Figure 1] We will further elaborate on why reflection behavior is preferable over giving advice by emphasizing the importance of reflective listening in MI. We will also discuss how reflection promotes client autonomy, encourages self-exploration, and aligns with the core principles of MI.\n\n[Regarding the choice of the penultimate self-attention layer] The decision was made by (1) practical advice/wisdom from colleagues and the ML engineering community and (2) our findings that qualitatively the attention scores did seem to correlate with our intuition. We will state this explicitly in the paper.\n\nWe acknowledge the point raised about the relatively new nature of research on reflective rewriting, and how different methods and baselines will make our case stronger. In this study, we primarily focused on previous template-based editing methods and aimed to provide improvements over them, but we also recognize the fact that there have been strides in the field of response rewriting, including latent space-based methods and more recently, LLM-based rewriting. Although we have made sure to train the previous template-based baselines on our task using the same model size/training parameters for a fair comparison, we agree that a wider comparison of our methodology will better contextualize the performance of our VERVE model.\n\nIn response to this point, we implemented and tested an LLM-prompting-based rewriting approach (in-context learning with instructions and examples of reflections) on the automatic evaluation metrics and also did a qualitative review of the generations. The following are the results:\nPAIR: Change in Reflection: 85.00, Keyphrase Coverage: 13.26, Edit Rate: 246.12, Perplexity: 17.2271, Coherence: 97.58, Specificity: 83.02\nAnnoMI: Change in Reflection: 60.86, Keyphrase Coverage: 18.41, Edit Rate: 131.73, Perplexity: 18.1430, Coherence: 93.64, Specificity: 77.14\nOverall, we observe that the LLM method is capable of producing highly reflective and fluent rewrites, surpassing even the performance of VERVE in the PAIR dataset. However, we also note that the LLM approach results in the most significant departure from the original text.\n\nWe posit that LLM rewriting functions within a distinct paradigm, neither superior nor inferior, but different from ours. Our strategy revolves around templates, enabling the model to operate within predefined structures and facilitate incremental adjustments. In contrast, LLM-driven prompts appear to enjoy a higher degree of content addition and removal flexibility. Managing this variance can be challenging, although refining prompts presents a potential solution. We think that a hybrid approach, where templates are provided to the LLM for completion, presents an opportunity we hope to explore subsequently.\n\nAdditionally, we think that a locally hosted model for counselor response rewriting can be preferable over LLMs (especially closed-source ones only accessible via API) for privacy concerns and resource efficiency. It ensures that sensitive information is kept in a controlled environment and provides more direct control over computational resources and time constraints.\n\nWe appreciate the suggestion to include more references in the related work section related to counseling skills and strategies. We will incorporate additional references that help elaborate on the importance of reflective listening and motivational interviewing, providing a more comprehensive context for our work.\n\n[Regarding the acronym \"VERVE,\"] we appreciate the reviewer's proposal of \"MIRROR.\" We find it to be a compelling name that better captures the nature of the reflection rewriting task. We are considering this suggestion seriously and will either adopt a name that more accurately resonates with our approach or potentially opt for the reviewer's suggested term.\n\nWe will make the necessary adjustments to Table 4 and Table 5 to enhance readability and provide more background on counseling skills and strategies to demonstrate their relevance for improved counseling outcomes.\n\nFinally, we'll address the grammar and style improvements pointed out by the reviewer to ensure the clarity and cohesiveness of the paper.\n\nWe thank the reviewer once again for their thorough review and constructive feedback, which will undoubtedly contribute to the overall quality and impact of our paper.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The paper is well-written and easy to follow except for the gap on why reflective listening is useful for counseling and what motivational interviewing is.\n- Each step of the proposed framework is well-motivated and easy to understand.\n- The authors perform a comprehensive analysis for comparing against baselines and expert references using both human evaluation and automatic evaluation.\n- The paper has a solid discussions section and a limitations section.", "reasons_to_reject": "No major reason for rejection other than that the gap on why reflective listening is useful for counseling and what motivational interviewing is. Without this information, readers who are not familiar with the two concepts and their importance to counseling will not fully appreciate the significance of this work.", "questions_to_authors": "- Why is the reflection behavior more preferable than the one that gives advice in Figure 1?\n- Why is the penultimate self-attention layer used rather than the final layer? (lines 231-233)\n- Is the research on reflective rewriting relatively new? The suggested baselines seem to be not specifically trained for this task and wonder whether there are stronger baselines that VERVE should be compared against.", "paper_topic_and_main_contributions": "The paper describes a template-driven reflective rewriting technique to perform motivational interviewing, which is a counseling strategy that aims to improve counseling outcomes. This is important for scaling efforts in teaching new counselors for mental health support. The proposed technique yields significant improvements to the compared baselines in both human evaluation and automatic evaluation. ", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Missing_References": "I wished there were more references in the related work section on counseling skills and strategies other than the ones employed in this work and elaborate why reflective listening and motivational interviewing were chosen.", "Typos_Grammar_Style_and_Presentation_Improvements": "- The introduction should introduce, at least briefly, what motivational interviewing is. It should be explained in more detail in 3.1 when it is heavily referenced by a lot of work (lines 149-151).\n- There is a gap in how reflective listening improves counseling outcomes. This should be filled in the introduction.\n- (very minor and personal) I'm not a fan of the acronym as it is really forced and also the definition of verve doesn't really resonate with what the model is trying to do. What about Motivational Interviewing with Reflecting Rewriting for COunseling ImpRovement (MIRROR)?\n- Table 4 and Table 5 can be column-width tables to reduce space and add more background on counseling skills and strategies and why they are useful for better counseling outcomes.\n- line 408: need space after . in \".In Table 4\""}}]

Author rebuttal:

Meta-review: This paper presents a method for rewriting junior counselor's responses using a system of templates to offer better responses in terms of reflective listening, a key skill for mental health counselors.  The templates themselves are too restrictive, and so paraphrasing is used to broaden their matching power.

One of the reviewers cited several concerns with the lack of comparison with a broader set of relevant methods, and multiple reviewers pointed out the lack of adequate comparison to LLM's.  The paper now argues for why template-based approaches are superior to any LLM based approach on the grounds that they provide more controllability and preservation of the original content.  Controlled generation is an active area of research for LLM's, but an LLM, particularly one that has been fine-tuned, offers quite a high degree of controllability for a given task, and so this claim should be properly evaluated.  Moreover, saying that templates provide a better ability to preserve original content is also a claim that needs evaluation.  The authors note in one of their rebuttals that they have, in fact, tried to use LLM's and that they were superior to their own method, but less good at preserve original text, but this was without sufficient attempts to improve this capability.

This paper explores an important area, is (generally) clearly written and offers good results in its proposed direction.  However, the reviewers here have pointed out legitimate concerns with this work and its place alongside similar work in the area of response rewriting and, indeed, in the broader area of NLP.
Decision: Accept-Findings