SlowGuess/ACL_2020_OCR huggingface dataset:
Bo Sun, Yinghui Zhang, Jun He, Lejun Yu, Qihua Xu, Dongliang Li, and Zhaoying Wang. 2017. A Random Forest Regression Method With Selected-Text Feature For Depression Assessment. In Proceedings of the 7th Annual Workshop on Audio/Visual Emotion Challenge - AVEC '17, pages 61-68, Mountain View, California, USA. ACM Press.
Jackson Tolins and Jean E. Fox Tree. 2014. Addressee backchannels steer narrative development. Journal of Pragmatics, 70:152-164.
Chrystal Vergara and John E. Roberts. 2011. Motivation and goal orientation in vulnerability to depression. Cognition and Emotion, 25(7):1281-1290.
A. H. Weinberger, M. Gbedemah, A. M. Martinez, D. Nash, S. Galea, and R. D. Goodwin. 2018. Trends in depression prevalence in the USA from 2005 to 2015: widening disparities in vulnerable groups. Psychological Medicine, 48(8):1308-1315.
Le Yang, Dongmei Jiang, Lang He, Ercheng Pei, Meshia Cédric Oveneke, and Hichem Sahli. 2016. Decision Tree Based Depression Classification from Audio Video and Language Information. In Proceedings of the 6th International Workshop on Audio/Visual Emotion Challenge - AVEC '16, pages 89-96, Amsterdam, The Netherlands. ACM Press.
Le Yang, Dongmei Jiang, Xiaohan Xia, Ercheng Pei, Meshia Cedric Oveneke, and Hichem Sahli. 2017. Multimodal Measurement of Depression Using Deep Learning Models. In Proceedings of the 7th Annual Workshop on Audio/Visual Emotion Challenge - AVEC '17, pages 53-59, Mountain View, California, USA. ACM Press.
V. H. Yngve. 1970. On getting a word in edgewise. In Chicago Linguistics Society, 6th Meeting, pages 567-578.
Justine Zhang, Robert Filbin, Christine Morrison, Jaclyn Weiser, and Cristian Danescu-Niculescu-Mizil. 2019. Finding Your Voice: The Linguistic Development of Mental Health Counselors. In Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Papers, pages 946-947.
A.1 Continuous representation of utterances
For continuous representation using the GloVe model, we use the pretrained 100-dimensional embeddings (Pennington et al., 2014). The representation of an utterance is computed as the average of embeddings for words in the utterance, with  $\mathbf{0}^{100}$  used to represent words not in the pretrained vocabulary. Based on the pretrained vocabulary, contractions (e.g. "can't") are decomposed. For continuous representation with the
BERT model, utterances are split into sequences of sub-word tokens following the authors' specifications (Devlin et al., 2019), and the pretrained BERT (Base, Uncased) model computes a 768-dimensional position-dependent representation.
A.2 Training the BERT Model
For the BERT model, all interviews were truncated to fit the maximum sequence length of the pretrained BERT model (Base, Uncased): 512 subword tokens. Truncation occurs by alternating between removing prompt and response tokens until the interview length in tokens is adequate.
Devlin et al. (2019) suggest trying a limited number of combinations of learning rate and training epochs to optimize the BERT classification model. Specifically, the paper recommends combinations of 2, 3, or 4 epochs and learning rates of 2E-5, 3E-5, and 5E-5. We noted that validation and test scores were surprisingly low (significantly below random) using these combinations, and posited that the small number of suggested epochs could have resulted from the authors only evaluating BERT on certain types of datasets. Accordingly, we evaluated up to 50 epochs with the suggested learning rates and selected a learning rate of 2E-5 with 15 epochs based on validation results.
A.3 Exclusion of prompts
The goal of removing prompts is to prevent a classifier from identifying participants as depressed based on certain prompts simply being present in the interview, such as "How long ago were you diagnosed [with depression]?" While some prompts are clear indicators, early tests showed that even with these prompts removed, other prompts were predictors for the participant being depressed for no obvious reason, indicating a bias in the design in the interview. Rather than using arbitrary means to determine whether prompts could be predictive, we used a machine-learning based algorithm to identify and remove predictive prompts from interviews.
After the division of interviews into turns as described in Section 3.1, we extracted the set of distinct prompts  $P_{distinct}$  from all interviews (with no additional preprocessing). We then iteratively performed 10 logistic regression experiments using the same set of splits described in Section 4.2.
In a given experiment, each interview was represented as an indicator vector with  $|P_{distinct}|$  di
mensions, such that position  $p$  is set to 1 if prompt  $p \in \{1, \dots, |P_{\text{distinct}}|\}$  is present in the interview, and 0 otherwise. Logistic Regression was optimized on the vector representations for the training interviews. The predicted F1 score for the depressed class on the validation set was recorded for each experiment.
The average weight vector for the 10 Logistic regression models was computed. The prompt corresponding to the highest weight was removed from  $P_{distinct}$  and added to a separate set  $D$  of predictive prompts. The process was repeated until the mean validation F1 score was less than the random baseline for the dataset (see Section 4.3).
The final set of 31 prompts  $D$  had to be removed from the dataset before the baselines and proposed approaches could be evaluated. The design of the DAIC interview posed a challenge, however: the same prompt can appear in many interviews, but preceded by unique interjections by the interviewer, such as "mhm," "nice," and "I see". We refer to this interjections as "prefixes." We manually compiled a list of 37 prefixes that commonly reoccur in interviews. For all interviews, if a prompt from  $P_{distinct}$  occurred in the interview after prefixes were ignored, then both the prompt and its corresponding response were removed from the interview before training. This resulted in an removing an average of 13.64 turns from each interview in the dataset.