SlowGuess/ACL_2020_OCR huggingface dataset:
In our experimentation, we use the University of Maryland Reddit Suicidality Dataset, v.2 (Shing et al., 2018; Zirikly et al., 2019). This English-language dataset, derived from the 2015 Full Reddit Submission Corpus (2006-2015), includes 11,129 potentially at-risk individuals who posted on r/SuicideWatch (a subreddit dense in self-reports about suicidality, henceforth SW), as well as 11,129 control individuals who never posted on any mental-health related subreddit. Entire posting histories (not just from SW, but all Reddit forums) were collected. An individual's number of posts can range from 10 to 1,326. See Table 2 for a detailed breakdown of number of posts per individual across datasets and risk categories.
The full dataset has three subsets with disjoint individuals. The first, which we term the WEAK SUPERVISION dataset, includes 10,263 individuals who posted in SW and 10,263 control individuals who did not; they are respectively considered to be indirectly positively and negatively labeled, very noisily since posting on SW does not necessary imply suicidal ideation.[8] The second set is the CROWDSOURCE dataset, including 621 individuals annotated by crowdsourcers with four risk levels: No Risk, Low Risk, Moderate Risk, and Severe Risk.
See Appendix A for IRB and ethical considerations.
See Gaffney and Matias (2018) for caveats.
E.g. seeking help for a friend, or offering support.
8129
(a) 3HAN
(b) Seq2Vec with Attention
Figure 4: An illustration of the three-level Hierarchical Attention Network (3HAN) model
# Posts	10-20	20-40	40-60	60-100	100-200	200-500	500-1,000	1,000-1,500
CrowdSource	No Risk	31	42	25	27	18	12	4	0
Low Risk	19	22	5	11	2	4	0	0
Moderate Risk	46	45	19	14	9	7	1	0
Severe Risk	80	79	37	19	28	12	3	0
Expert	No Risk	3	7	2	5	7	8	3	0
Low Risk	6	11	5	11	8	7	1	1
Moderate Risk	23	19	12	26	13	14	5	3
Severe Risk	7	2	5	9	10	4	4	1
Table 2: Number of individuals with the number (range) of posts, by dataset and risk category.
The last is the EXPERT dataset, including 242 individuals with the same four-level annotation, by four suicide risk assessment experts. Along with the level of risk for each individual, the expert annotators also designated the single post that most strongly supported each of their low, moderate, or severe risk labels.
5.2 Evaluating with hTBG
As TBG and hTBG are measures designed for binary relevance judgements, we map the Severe Risk category to at-risk, and everything else to not-at-risk. For word counts, we directly use the token counts in documents. We use the parameters that Smucker and Clarke (2012) estimated for TBG in user studies (Table 1). As discussed in Section 3.2, we assume there exists a maximum number of documents the clinician can read for each individual.
We set that number to 50 for the calculation of hTBG; if no relevant document exists in the top 50 documents, we consider that individual a miss and set the gain to zero. $^{11}$
To rank individuals using our classification models, we use a standard conversion method to convert four-class probability to a single score:
$$
\sum_ {\operatorname {r e l} _ {i}} ^ {R} P \left(\hat {y} _ {i} = \operatorname {r e l} _ {i}\right) \operatorname {s c o r e} _ {\operatorname {r e l} _ {i}} \tag {14}
$$
where  $R$  is  $\{\mathrm{No, Low, Moderate, Severe}\}$ , and  $\mathrm{score}_{\mathrm{rel}_i}$  is the real number that maps to the risk-level of the individual  $i$ . We use  $\{\mathrm{No} = 0, \mathrm{Low} = 1, \mathrm{Moderate} = 2, \mathrm{Severe} = 4\}$  as our mapping — No Risk can plausibly be treated the same as a post with no annotation (e.g. a control individual), and exponential scaling also seems plausible although just one of many possibilities, which we leave for future work.
The hTBG metric also requires a stopping probability for each document,  $R_{i,l}$ . Assuming that the more severe the risk associated with a document is, the more likely the assessor is to stop and flag the
11 All parameters were frozen prior to testing. We plan to estimate hyperparameters in our own user studies in the future.
9Shing et al. (2018) report reliable expert annotation, Krippendorff's  $\alpha = .81$ . The original EXPERT dataset had 245 individuals; we exclude three owing to errors in processing.
<sup>10</sup>Since the label definitions distinguish severe from moderate by focusing on the risk of an attempt in the near future, this binary distinction is aligned with recent work in suicidology that focuses specifically on characterizing "the acute mental state that is associated with near-term suicidal behavior" (Schuck et al., 2019).
individual, on the EXPERT dataset where we have document-level annotations, we can estimate the expected stopping probability as:
$$
R _ {i, l} = 1 - \prod_ {c = 1} ^ {C} \left(1 - \frac {\operatorname {s c o r e} _ {\text {r e l} _ {i , l , c}}}{\operatorname {s c o r e} _ {\max }}\right) \tag {15}
$$
where  $C$  annotators annotated the post as most strongly supporting their judgment.  $\mathrm{Score}_{\mathrm{rel}_{i,l,c}}$  is a mapping from the document-level risk by annotator  $c$  to a real number, with the same mapping used in Equation 14.  $\mathrm{Score}_{\max} = 4$  is the maximum in that mapping.
To reflect different time budgets, we report results with the half-life parameter ranging from 1 to 6 hours, which corresponds to expected reading time budgets from 1.4 to 8.7 hours.
5.3 Models for Ranking Individuals
3HAN. 3HAN is first pretrained on the binary WEAK SUPERVISION dataset. The model is then further tuned on the four-class CROWDSOURCE dataset by transferring the weights (except the last fully-connected prediction layer) over. We initialized and fixed the word embedding using the 200-dimensional Glove embedding trained on Twitter (Pennington et al., 2014).
3HAN_Av. 3HAN Average is trained the same way as 3HAN, except that the last Seq2Vec layer (the layer that aggregates a sequence of document vectors to an individual vector) is averaged instead of using attention, which can be achieved by fixing  $a_{i,j} = \frac{1}{m}$  in Equation 10. This is similar to the HN-AVE baseline in Yang et al. (2016). Note that 3HAN Av cannot rank documents, as it lacks document attention.
LR. A logistic regression model is trained on the CROWDSOURCE dataset. The feature vector for an individual is computed by converting documents into document-level feature vectors, and then averaging them to obtain an individual-level feature vector. For each document, we concatenate four feature sets: (1) bag-of-words for vocabulary count larger than three, (2) Glove embedding summing over words, (3) 194 features representing emotional topics from Empath (Fast et al., 2016),
and (4) seven scores measuring document readability. $^{13}$  This model is included as a conventional baseline in suicide risk assessment, similar to the baseline found in Shing et al. (2018).
5.4 Models for Ranking Documents
3HAN_At. Document attention learned jointly with 3HAN. As a side effect to training our 3HAN model, we learn document attention scores, see Equation 10. This score can then be used to rank documents in terms of their relevance to the judgement. This availability of document ranking, despite a lack of document annotations, is a significant advantage of hierarchical attention networks, since fine-grained document annotations are difficult to obtain on a large scale. Sentence- and word-level attention are a further advantage, in terms of potentially facilitating user review (see Figure 1), although exploring that awaits future work.
Forward and Backward. Ranking an individual's documents in either chronological order or reverse chronological order is an obvious default in the absence of a trained model for document ranking, important baselines for testing whether a document ranking model actually adds value.
Our model, 3HAN+3HAN ATT, the only joint model, achieves the best performance on hTBG compared to all other combinations of individual rankers and document rankers across three different time budgets (Table 3). The result is significant except when compared to 3HAN_AV+3HAN ATT. However, using 3HAN ATT to rank documents implies that you have already trained 3HAN. Therefore, a more reasonable combination to compare with is 3HAN_AV+BACKWARD, which we outperform by a significant margin.
Overall, the effect of document ranking is larger than the effect of individual ranking. Notably, the FORWARD document ranker always yields the worst performance. BACKWARD, on the other hand, is surprisingly competitive. We hypothesize that this may be an indication that suicidal ideation worsens over time, or perhaps of the unfortunate
$^{13}$ Flesch-Kincaid Grade Level, Flesch Reading Ease, Dale Chall Readability, Automated Readability Index (ARI), Coleman Liau Index, Gunning Fog Index, and Linsear Write.
$^{14}$ Paired bootstrap resampling test, repeated 1000 times,  $p < 0.05$ .
<sup>12</sup>We experimented with trainable Glove embedding as well as BERT, but saw little to no improvement in performance using cross-validation. We plan to explore fine-tuning BERT on Reddit in future work.
8131
Individual Ranker	Document Ranker	Half-life h
1 hr	3 hrs	6 hrs
LR	FORWARD	7.51	10.05	10.89
3HAN_AV	FORWARD	7.76	10.15	10.94
3HAN	FORWARD	7.40	9.98	10.84
LR	BACKWARD	8.75	11.70	12.68
3HAN_AV	BACKWARD	9.65	12.09	12.89
3HAN	BACKWARD	9.73	12.17	12.95
LR	3HAN_ATT	9.44	12.05	12.88
3HAN_AV	3HAN_ATT	10.16	12.35	13.04
3HAN	3HAN_ATT	10.39	12.49	13.12
Optimal hTBG	19.78	20.39	20.54
event of suicide attempts following posting a Severe Risk document. This motivates the importance of prioritizing the reading order of documents: being able to find evidence early in suicide assessment leaves more time for other individuals, and will reduce probability of misses.
Document ranking alone does not decide everything, as 3HAN+BACKWARD outperforms LR+3HAN_ATT. It is the combination of 3HAN and its document attentions that produce our best model. This makes sense, as 3HAN, while learning to predict the level of risk, also learns which documents are important to make the prediction.
Figure 1 shows the top 3 documents in a summary-style view for each of the highest ranked 3 individuals, with word-level attention shown using shading. Words without attention are obfuscated; others are altered to preserve privacy.
Previously Existing Measures. For previously existing measures, e.g. TBG and NDCG@20, document ranking has no effect, and thus these are not suitable measures in our scenario. However, we include results here for reference (Table 4). Since 3HAN Av. and LR cannot rank documents, it is impossible to calculate hTBG, so we report results on the chronologically backward ranking strategy. NDCG@20 is NDCG score cut off at 20, chosen based on the optimal hTBG value.
We introduced hTBG, a new evaluation measure, as a step toward moving beyond risk classification to a paradigm in which prioritization is the focus, and where time matters. Like TBG, the hTBG score is interpretable as a lower bound on the expected
Table 3: hTBG scores with three different time budgets, all combinations of individual and document rankers.
Ranker	hTBG	TBG	NDCG@20
3HAN+3HAN ATT.	12.49	11.46	70.90
3HAN AV.+BACKWARD	12.09	11.40	68.28
LR+BACKWARD	11.70	10.98	69.44
Optimal	20.39	19.75	100.00
Table 4: TBG and NDCG@20 listed to compare with hTBG. Both hTBG's and TBG's half lives are set at 3 hrs, and maximum document cutoff is set at 50.
number of relevant items found in a ranking, given a time budget. In our experiment, a "relevant item" is a person classified by experts as being at risk of attempting suicide in the near future.
Measured at an expected reading time budget of about half a day (4hr20min, half-life 3hrs), our joint ranking approach achieved hTBG of 12.49 compared with 11.70 for a plausible baseline from prior art: using logistic regression to rank individuals, and then looking at a individual's posts in backward chronological order. That increase is just a bit short of identifying one more person in need of immediate help in the experiment's population of 242 individuals. There are certainly limitations in our study and miles to go before validating our approach in the real world, but our framework should make it easy to integrate and explore other individual rankers, document rankers and explanation mechanisms, and to actually build user interfaces like the schematic in Figure 1.
This work has been supported in part by a University of Maryland Strategic Partnership (MPower) seed grant, an AWS Machine Learning Research Award, and an AI + Medicine for High Impact (AIM-HI) Challenge Award. We are immensely grateful to Glen Coppersmith, Michelle Colder Carras, April Foreman, Michelle Kuchuk, Beau Pinkham, Rebecca Resnik, Katherine Musacchio Schafer, Jonathan Singer, Raymond Tucker, Tony Wood, Ayah Zirikly, members of the UMIACS CLIP lab, and participants at the Workshops on Computational Linguistics and Clinical Psychology for valuable discussions related to this work.
Rakesh Agrawal, Sreenivas Gollapudi, Alan Halverson, and Samuel Leong. 2009. Diversifying search results. In Proceedings of the Second ACM International Conference on Web Search and Data Mining,
8132
WSDM '09, page 5-14, New York, NY, USA. Association for Computing Machinery.
Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate. In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings.
Krisztian Balog. 2018. Entity-Oriented Search, volume 39 of The Information Retrieval Series. Springer.
Krisztian Balog, Yi Fang, Maarten de Rijke, Pavel Serdyukov, and Luo Si. 2012. Expertise retrieval. Foundations and Trends in Information Retrieval, 6(2-3):127-256.
Tal Baumel, Jumana Nassour-Kassis, Raphael Cohen, Michael Elhadad, and Noémie Elhadad. 2018. Multi-label classification of patient notes: Case study on ICD code assignment. In *The Workshops of the Thirty-Second AAAI Conference on Artificial Intelligence*, pages 409–416. AAAI Press.
Adrian Benton, Glen Coppersmith, and Mark Dredze. 2017. Ethical research protocols for social media health research. In Proceedings of the First ACL Workshop on Ethics in Natural Language Processing, EthNLP@EACL, pages 94-102. Association for Computational Linguistics.
David E. Bloom, Elizabeth Cafiero, Eva Jané-Llopis, Shafika Abrahams-Gessel, Lakshmi Reddy Bloom, Sana Fathima, Andrea B. Feigl, Tom Gaziano, Ali Hamandi, Mona Mowafi, Danny O'Farrell, and Emre. 2012. The Global Economic Burden of Noncommunicable Diseases. PGDA Working Papers 8712, Program on the Global Demography of Aging.
Bureau of Health Workforce. 2020. Designated health professional shortage areas: Statistics, second quarter of fiscal year 2020, designated HPSA quarterly summary.
Rafael A. Calvo, David N. Milne, M. Sazzad Hussain, and Helen Christensen. 2017. Natural language processing in mental health applications using non-clinical texts. Nat. Lang. Eng., 23(5):649-685.
Stevie Chancellor, Michael L. Birnbaum, Eric D. Caine, Vincent M. B. Silenzio, and Munmun De Choudhury. 2019. A Taxonomy of Ethical Tensions in Inferring Mental Health States from Social Media. In Proceedings of the Conference on Fairness, Accountability, and Transparency.
Olivier Chapelle, Donald Metlzer, Ya Zhang, and Pierre Grinspan. 2009. Expected reciprocal rank for graded relevance. In Proceedings of the 18th ACM Conference on Information and Knowledge Management, CIKM 2009, pages 621-630. ACM.
Munmun De Choudhury. 2013. Role of social media in tackling challenges in mental health. In Proceedings of the 2nd international workshop on Socially-aware multimedia, SAM@ACM Multimedia 2013, pages 49-52. ACM.
Cindy Chung and James W Pennebaker. 2007. The psychological functions of function words. Social communication, 1:343-359.
Glen Coppersmith, Ryan Leary, Patrick Crutchley, and Alex Fine. 2018. Natural Language Processing of Social Media as Screening for Suicide Risk. Biomedical Informatics Insights, 10:117822261879286.
Ethan Fast, Binbin Chen, and Michael S. Bernstein. 2016. Empath: Understanding topic signals in largescale text. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, pages 4647-4657. ACM.
Joseph C. Franklin, Jessica D. Ribeiro, Kathryn R. Fox, Kate H. Bentley, Evan M. Kleiman, Xieyining Huang, Katherine M. Musacchio, Adam C. Jaroszewski, Bernard P. Chang, and Matthew K. Nock. 2017. Risk factors for suicidal thoughts and behaviors: A meta-analysis of 50 years of research. Psychological Bulletin, 143(2):187-232.
Devin Gaffney and J. Nathan Matias. 2018. Caveat emptor, computational social science: Large-scale missing data in a widely-published reddit corpus. PLOS ONE, 13(7):1-13.
Matt Gardner, Joel Grus, Mark Neumann, Oyvind Tafjord, Pradeep Dasigi, Nelson F. Liu, Matthew Peters, Michael Schmitz, and Luke Zettlemoyer. 2018. AllenNLP: A deep semantic natural language processing platform. In Proceedings of Workshop for NLP Open Source Software (NLP-OSS), pages 1-6. Association for Computational Linguistics.
Holly Hedegaard, Sally C Curtin, and Margaret Warner. 2018. Suicide rates in the United States continue to increase. National Center for Health Statistics.
Matthew Honnibal and Mark Johnson. 2015. An improved non-monotonic transition system for dependency parsing. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, EMNLP 2015, pages 1373-1378. The Association for Computational Linguistics.
Sarthak Jain and Byron C. Wallace. 2019. Attention is not explanation. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, pages 3543-3556. Association for Computational Linguistics.
Kalervo Jarvelin and Jaana Kekäläinen. 2002. Cumulated gain-based evaluation of IR techniques. ACM Transactions on Information Systems (TOIS), 20(4):422-446.
8133
Kathryn P. Linthicum, Katherine Musacchio Schafer, and Jessica D. Ribeiro. 2019. Machine learning in suicide science: Applications and ethics. Behavioral Sciences & the Law, 37(3):214-222.
David E. Losada, Fabio Crestani, and Javier Parapar. 2020. eRisk 2020: Self-harm and depression challenges. In Advances in Information Retrieval - 42nd European Conference on IR Research, ECIR 2020, Lisbon, Portugal, April 14-17, 2020, Proceedings, Part II, volume 12036 of Lecture Notes in Computer Science, pages 557-563. Springer.
David N. Milne, Glen Pink, Ben Hachey, and Rafael A. Calvo. 2016. CLPsych 2016 shared task: Triaging content in online peer-support forums. In Proceedings of the 3rd Workshop on Computational Linguistics and Clinical Psychology: From Linguistic Signal to Clinical Reality, CLPsych@NAACL-HLT 2016, pages 118-127. The Association for Computational Linguistics.
James Mullenbach, Sarah Wiegreffe, Jon Duke, Jimeng Sun, and Jacob Eisenstein. 2018. Explanable prediction of medical codes from clinical text. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2018, pages 1101-1111. Association for Computational Linguistics.
Jeffrey Pennington, Richard Socher, and Christopher D. Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014, pages 1532-1543. ACL.
Tetsuya Sakai. 2019. Graded relevance assessments and graded relevance measures of NTCIR: A survey of the first twenty years. CoRR, abs/1903.11272.
SAMHSA. 2019. National Survey on Drug Use and Health, 2017 and 2018. Center for Behavioral Health Statistics and Quality. Table 8.58B.
Allison Schuck, Raffaella Calati, Shira Barzilay, Sarah Bloch-Elkouby, and Igor Galynker. 2019. Suicide Crisis Syndrome: A review of supporting evidence for a new suicide-specific diagnosis. Behavioral sciences & the law, 37(3):223-239.
Han-Chin Shing, Suraj Nair, Ayah Zirikly, Meir Friedenberg, Hal Daumé III, and Philip Resnik. 2018. Expert, crowdsourced, and machine assessment of suicide risk via online postings. In Proceedings of the Fifth Workshop on Computational Linguistics and Clinical Psychology: From Keyboard to Clinic, CLPsych@NAACL-HTL, pages 25-36. Association for Computational Linguistics.
Han-Chin Shing, Guoli Wang, and Philip Resnik. 2019. Assigning medical codes at the encounter level by paying attention to documents. In ML4H, Machine Learning for Health Workshop at NeurIPS.
Mark D. Smucker and Charles L. A. Clarke. 2012. Time-based calibration of effectiveness measures. In The 35th International ACM SIGIR conference on research and development in Information Retrieval, SIGIR '12, pages 95-104. ACM.
Ellen M. Voorhees. 2001. The philosophy of information retrieval evaluation. In Evaluation of Cross-Language Information Retrieval Systems, Second Workshop of the Cross-Language Evaluation Forum, CLEF 2001, volume 2406 of Lecture Notes in Computer Science, pages 355-370. Springer.
Byron C. Wallace. 2019. Thoughts on "attention is not not explanation". Medium, Accessed: December, 2019.
Sarah Wiegrefe and Yuval Pinter. 2019. Attention is not not explanation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019, pages 11-20. Association for Computational Linguistics.
Zichao Yang, Diyi Yang, Chris Dyer, Xiaodong He, Alexander J. Smola, and Eduard H. Hovy. 2016. Hierarchical attention networks for document classification. In *NAACL HLT* 2016, The 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies*, pages 1480-1489. The Association for Computational Linguistics.
Andrew Yates, Arman Cohan, and Nazli Goharian. 2017. Depression and Self-Harm Risk Assessment in Online Forums. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2968-2978.
Michael Zimmer. 2010. "But the data is already public": on the ethics of research in Facebook. Ethics and Information Technology, 12(4):313-325.
Ayah Zirikly, Philip Resnik, Ozlem Uzuner, and Kristy Hollingshead. 2019. CLPsych 2019 shared task: Predicting the degree of suicide risk in Reddit posts. In Proceedings of the Sixth Workshop on Computational Linguistics and Clinical Psychology, pages 24-33. Association for Computational Linguistics.
8134
A Appendix: Ethical Considerations
Our research involving the University of Maryland Reddit Suicide Dataset has undergone review by the University of Maryland Institutional Review Board with a determination of Category 4 Exempt status under U.S. federal regulations. For this dataset, (a) the original data are publicly available, and (b) the originating site (Reddit) is intended for anonymous posting. In addition, since Reddit is officially anonymous, but that is not enforced on the site, the dataset has undergone automatic de-identification using named entity recognition aggressively to identify and mask out potential personally identifiable information such as personal names and organizations, in order to create an additional layer of protection (Zirikly et al., 2019). In an assessment of de-identification quality, we manually reviewed a sample of 200 randomly selected posts (100 from the SuicideWatch subreddit and 100 from other subreddits), revealing zero instances of personally identifiable information.
Following Benton et al. (2017), we treat the data (even though de-identified) as sensitive and restrict access to it, we use obfuscated and minimal examples in papers and presentations, and we do not engage in linkage with other datasets.
The dataset is available to other researchers via an application process put in place with the American Association of Suicidology that requires IRB or equivalent ethical review, a commitment to appropriate data management, and, since ethical research practice is not just a matter of publicly available data or even IRB approval (Zimmer, 2010; Benton et al., 2017; Chancellor et al., 2019), a commitment to following additional ethical guidelines. Interested researchers can find information at http://umiacs.umd.edu/\~resnik/umd_redit_suicidality_dataset.html.
B Appendix: Proofs
B.1 Time-Biased Gain