SlowGuess/NAACL_2022_OCR huggingface dataset:
David Hecht. 2013. The neural basis of optimism and pessimism. Experimental neurobiology, 22(3):173.
Karl Moritz Hermann, Tomas Kocisky, Edward Grefenstette, Lasse Espeholt, Will Kay, Mustafa Suleyman, and Phil Blunsom. 2015. Teaching machines to read and comprehend. In Advances in neural information processing systems, pages 1693-1701.
Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. *Neural computation*, 9(8):1735-1780.
Dirk Hovy and Shannon L Spruit. 2016. The social impact of natural language processing. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 591-598.
Pei Huo, Yan Yang, Jie Zhou, Chengcai Chen, and Liang He. 2020. Terg: Topic-aware emotional response generation for chatbot. In 2020 International Joint Conference on Neural Networks (IJCNN), pages 1-8. IEEE.
Tatsuya Ide and Daisuke Kawahara. 2021. Multi-task learning of generation and classification for emotion-aware dialogue response generation. arXiv preprint arXiv:2105.11696.
Mustafa Jahanara. 2017. Optimism, hope and mental health: Optimism, hope, psychological well-being and psychological distress among students, university of Pune, india. International Journal of Psychological and Behavioral Sciences, 11(8):452-455.
Shaoxiong Ji, Xue Li, Zi Huang, and Erik Cambria. 2020. Suicidal ideation and mental disorder detection with attentive relation networks. arXiv preprint arXiv:2004.07601.
Shaoxiong Ji, Tianlin Zhang, Luna Ansari, Jie Fu, Prayag Tiwari, and Erik Cambria. 2021. Mentalbert: Publicly available pretrained language models for mental healthcare. arXiv preprint arXiv:2110.15621.
Allison Lahnala, Yuntian Zhao, Charles Welch, Jonathan K. Kummerfeld, Lawrence C. An, Kenneth Resnicow, Rada Mihalcea, and Verónica Pérez-Rosas. 2021. Exploring self-identified counseling expertise in online support forums. In Findings of the Association for Computational Linguistics: ACL/IJCNLP 2021, Online Event, August 1-6, 2021, volume ACL/IJCNLP 2021 of Findings of ACL, pages 4467-4480. Association for Computational Linguistics.
Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Ves Stoyanov, and Luke Zettlemoyer. 2019. Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. arXiv preprint arXiv:1910.13461.
Jiwei Li, Will Monroe, Alan Ritter, Dan Jurafsky, Michel Galley, and Jianfeng Gao. 2016. Deep reinforcement learning for dialogue generation. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, EMNLP 2016.
Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pages 74-81.
Rodrigo Martínez-Castano, Amal Htait, Leif Azzopardi, and Yashar Moshfeghi. 2021. Bert-based transformers for early detection of mental health illnesses. In International Conference of the Cross-Language Evaluation Forum for European Languages, pages 189-200. Springer.
Robert R Morris, Kareem Kouddous, Rohan Kshirsagar, and Stephen M Schueller. 2018. Towards an artificially empathic conversational agent for mental health applications: system design and user perceptions. Journal of medical Internet research, 20(6):e10148.
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, 2002.
Braja Gopal Patra, Reshma Kar, Kirk Roberts, and Hulin Wu. 2020. Mental health severity detection from psychological forum data using domain-specific unlabelled data. AMIA Summits on Translational Science Proceedings, 2020:487.
Verónica Pérez-Rosas, Xinyi Wu, Kenneth Resnicow, and Rada Mihalcea. 2019. What makes a good counselor? learning to distinguish between high-quality and low-quality counseling conversations. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 926-935.
Syed Arbaaz Qureshi, Sriparna Saha, Mohammed Hasanuzzaman, and Gael Dias. 2019. Multitask representation learning for multimodal estimation of depression level. IEEE Intell. Syst., 34(5):45-52.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9.
Guozheng Rao, Chengxia Peng, Li Zhang, Xin Wang, and Zhiyong Feng. 2020. A knowledge enhanced ensemble learning model for mental disorder detection on social media. In International Conference on Knowledge Science, Engineering and Management, pages 181-192. Springer.
Tulika Saha, Saraansh Chopra, Sriparna Saha, and Pushpak Bhattacharyya. 2020a. Reinforcement learning based personalized neural dialogue generation. In International Conference on Neural Information Processing, pages 709-716. Springer.
Tulika Saha, Dhawal Gupta, Sriparna Saha, and Pushpak Bhattacharyya. 2018. Reinforcement learning based dialogue management strategy. In International Conference on Neural Information Processing, pages 359-372. Springer.
Tulika Saha, Dhawal Gupta, Sriparna Saha, and Pushpak Bhattacharyya. 2021a. Emotion aided dialogue act classification for task-independent conversations in a multi-modal framework. Cognitive Computation, 13(2):277-289.
Tulika Saha, Aditya Patra, Sriparna Saha, and Pushpak Bhattacharyya. 2020b. Towards emotion-aided multimodal dialogue act classification. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4361-4372.
Tulika Saha, Saichethan Miriyala Reddy, Sriparna Saha, and Pushpak Bhattacharyya. 2022. Mental health disorder identification from motivational conversations. IEEE Transactions on Computational Social Systems.
Tulika Saha, Sriparna Saha, and Pushpak Bhattacharyya. 2020c. Towards sentiment aided dialogue policy learning for multi-intent conversations using hierarchical reinforcement learning. *PloS one*, 15(7):e0235367.
Tulika Saha, Sriparna Saha, and Pushpak Bhattacharyya. 2020d. Towards sentiment-aware multi-modal dialogue policy learning. Cognitive Computation, pages 1-15.
Tulika Saha, Apoorva Upadhyaya, Sriparna Saha, and Pushpak Bhattacharyya. 2021b. A multitask multimodal ensemble model for sentiment-and emotion-aided tweet act classification. IEEE Transactions on Computational Social Systems.
Tulika Saha, Apoorva Upadhyaya, Sriparna Saha, and Pushpak Bhattacharyya. 2021c. Towards sentiment and emotion aided multi-modal speech act classification in twitter. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 5727-5737.
Victor Sanh, Lysandre Debut, Julien Chaumont, and Thomas Wolf. 2019. Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108.
Iulian Vlad Serban, Alessandro Sordoni, Yoshua Bengio, Aaron C. Courville, and Joelle Pineau. 2016. Building end-to-end dialogue systems using generative hierarchical neural network models. In Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence.
Iulian Vlad Serban, Alessandro Sordoni, Ryan Lowe, Laurent Charlin, Joelle Pineau, Aaron C. Courville, and Yoshua Bengio. 2017. A hierarchical latent variable encoder-decoder model for generating dialogues. In Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, USA.
Ashish Sharma, Inna W. Lin, Adam S. Miner, David C. Atkins, and Tim Althoff. 2021. Towards facilitating empathic conversations in online mental health support: A reinforcement learning approach. In WWW '21: The Web Conference 2021, Virtual Event / Ljubljana, Slovenia, April 19-23, 2021, pages 194-205. ACM / IW3C2.
Ashish Sharma, Adam S. Miner, David C. Atkins, and Tim Althoff. 2020. A computational approach to understanding empathy expressed in text-based mental health support. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, November 16-20, 2020, pages 5263-5276. Association for Computational Linguistics.
Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. 2014. Sequence to sequence learning with neural networks. In Advances in Neural Information Processing Systems 27: Annual Conference on Neural Information Processing Systems 2014.
Iwan Syarif, Nadia Ningtias, and Tessy Badriyah. 2019. Study on mental disorder detection via social media mining. In 2019 4th International Conference on Computing, Communications and Security (ICCCS), pages 1-6. IEEE.
Rupa Valdez and Jessica Keim-Malpass. 2019. Ethics in health research using social media. In Social Web and Health Research, pages 259-269. Springer.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, 4-9 December 2017, Long Beach, CA, USA, pages 5998-6008.
Wei Wei, Jiayi Liu, Xianling Mao, Guibing Guo, Feida Zhu, Pan Zhou, and Yuchong Hu. 2019. Emotion-aware chat machine: Automatic emotional response generation for human-like emotional interaction. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management, pages 1401-1410.
Bernard L Welch. 1947. The generalization of student's problem when several different population variances are involved. Biometrika.
Amir Hossein Yazdavar, Hussein S. Al-Olimat, Monireh Ebrahimi, Goonmeet Bajaj, Tanvi Banerjee, Krishnaprasad Thirunarayan, Jyotishman Pathak, and Amit P. Sheth. 2017. Semi-supervised approach to monitoring clinical depressive symptoms in social media. In Proceedings of the 2017 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining 2017, Sydney, Australia, July 31 - August 03, 2017, pages 1191-1198. ACM.
Amir Hossein Yazdavar, Mohammad Saeid Mahdavinejad, Goonmeet Bajaj, William Romine, Amit
Sheth, Amir Hassan Monadjemi, Krishnaprasad Thirunarayan, John M Meddar, Annie Myers, Jyotishman Pathak, et al. 2020. Multimodal mental health analysis in social media. Plos one, 15(4):e0226248.
Amir Hossein Yazdavar, Mohammad Saeid Mahdavinejad, Goonmeet Bajaj, Krishnaprasad Thirunarayan, Jyotishman Pathak, and Amit P. Sheth. 2018. Mental health analysis via social media data. In IEEE International Conference on Healthcare Informatics, ICHI 2018, New York City, NY, USA, June 4-7, 2018, pages 459-460. IEEE Computer Society.
Wojciech Zaremba and Ilya Sutskever. 2015. Reinforcement learning neural tuning machines-revised. arXiv preprint arXiv:1505.00521.
Yizhe Zhang, Siqi Sun, Michel Galley, Yen-Chun Chen, Chris Brockett, Xiang Gao, Jianfeng Gao, Jingjing Liu, and Bill Dolan. 2020. DIALOGPT: Large-scale generative pre-training for conversational response generation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, ACL 2020, Online, July 5-10, 2020, pages 270-278. Association for Computational Linguistics.
Motivational VA : MotiVAte Dataset The description of the mental disorders (considered in this paper) as mentioned in ICD-10 is listed in Table 5.
Interactive Training of Crowd-workers. The crowd-workers were initially provided with the entire guidelines for modifying the conversations along with ten such examples of raw and modified conversation pair. After this initial training, we scheduled an hour long phone call with them to discuss our instruction guidelines. Crowd-workers also raised questions about the guidelines during the phone conference, which substantially aided in resolving any potential issues. We gave them each 20 instances to modify after the phone call (randomly chosen; different for each crowd-worker). We manually evaluated the modified dyadic conversation on those 20 raw multi-party conversation. We either decided to discontinue with the crowd-worker (there was one) or gave them further manual input based on the results of the evaluation. Crowd-workers actively asked questions depending on their reservations during the process. We also ran spot checks on quality beyond the initial training phase (at least two times for each crowdworker; on more than 10 conversation each) to offer them with additional feedback. This check on quality was also conducted by the psychiatrist
with whom we collaborated for preparing the guidelines for modifying the source conversation. His feedback was also conveyed to the crowd-workers.
Guidelines Prepared. Some of the other important guidelines for modifying the conversations were : (i) The poster's messages/responses were changed to remove any references to a group of people as a whole. For example, phrase such as "does anyone here go through" was converted to "do you go through", similarly "thank you friends for helping me out" to "thank you for helping me" and so on. Similarly, a VA cannot respond by sharing its experiences because it is a machine robot with no life experience to draw on for example, "I have also faced a similar thing" etc. Also, a VA cannot refer to the poster about an anonymous seeker and share the seeker's experience (seen in the source conversation), as the communication between VA and the seeker is meant to be purely confidential and anonymized. As a result, the commenter's comments or utterances of these patterns were removed from the conversation in the context of VA; (ii) In the changed version, source conversations relating to the original topic of the subforum, such as MDD, were marked as MDD. We made no attempt to further categorise the chats by the contemplated category because we assumed that the poster would have picked the right category based on their needs. This is due to the fact that we have no other evidence to base our analysis on than what the poster chose for themselves. We recognise that this is a potential drawback because posters may not always have the mental health status that they perceive, and its impact should be examined further in the future.
Inter-annotator Agreement. The modified conversation from each of the crowd-workers were inter-changed and presented to the remaining crowd-workers to approve the quality of the modified conversations (in the sense that the modified conversations should be aligned with the guidelines provided). When any of the quality check crowd-workers disapproved of a chat that did not match the standards, it was removed from the MotiVAte dataset's final set. Only those dialogues were included in the final set that received unanimous approval from all crowd-workers. Following this criteria, we rejected 3k conversations from the 10k modified conversation and only 7067 conversation were included in the MotiVAte dataset. As a re
Category	Description listed in ICD-10
MDD	mood disorder, hopelessness, worthlessness, lack of energy, reduced activity
OCD	repeated unwanted thoughts (obsession), urge to continuously repeat something (compulsion)
Anxiety	nervous disorder, worry, uncontrollable racing thoughts, difficulty in concentrating, sleeping
PTSD	recurrent distressing memories of the traumatic event, negative thoughts about oneself, difficulty maintaining close relationships
sult, we observed a  $71\%$  inter-annotator agreement, which is regarded credible. To extend the scale of the MotiVAte dataset, we expect to clarify ambiguities in the rejected modified chats in the future.
Ethical Concerns. The acquisition of raw data and, as a result, the development of the dataset were done in accordance with all ethical principles or codes of conduct. Initially, an opinion on data usage and privacy was requested from an IPR lawyer during the data creation stage, and the response stated that "Section 107 of the U.S. Copyright Law which provides that "the fair use of a copyrighted work ... for purposes such as ... research, is not an infringement of copyright." Similarly, Section 52 of the Copyright Act, 1957 provides that "fair dealing with any work, for the purposes of — (i) private or personal use, including research" does not constitute an infringement of copyright in the said work. This statutory exception of fair use/ fair dealing in the website's content is also reflected in the Terms of Use of PsychCentral.org: "provided however, that users may download one copy of any Content on any single computer and print a copy of that Content solely for their personal, private, non-commercial use." The use of the content for research may be deemed to fall within this exception provided it was "personal, private, non-commercial use". Following that, the current study is being conducted in collaboration with a psychiatrist from a nationally recognised institution. The psychiatrist has assisted us with every element of this work, including developing data annotation criteria and carefully reviewing the quality of the data.
Table 5: Description of different mental disorders in ICD-10
Model	k=1	k=2	k=t
Prec.	Rec.	Prec.	Rec.	Prec.	Rec.
BERT+CNN (NA+NSenti)	0.4630	0.4725	0.4616	0.4690	0.4720	0.5625
BERT+CNN+Senti (NA)	0.4668	0.4771	0.4650	0.4722	0.5562	0.5688
BERT+Bi-GRU (NA+NSenti)	0.3858	0.4231	0.4537	0.4629	0.4785	0.5680
BERT+Bi-GRU+Senti (NA)	0.4538	0.4615	0.4726	0.4718	0.5270	0.5753
BERT+Bi-LSTM (NA+NSenti)	0.4735	0.4813	0.4663	0.4772	0.4872	0.5745
BERT+Bi-LSTM+Senti (NA)	0.4752	0.4938	0.4771	0.4879	0.5601	0.5703
BERT+Bi-LSTM+Senti (only SA)	0.5035	0.5039	0.4935	0.4912	0.5617	0.5725
BERT+Bi-LSTM+Senti (only CA)	-	-	0.4960	0.5049	0.4800	0.5888
MIC Model	0.5035	0.5039	0.5162	0.5083	0.5730	0.6016
Table 6: Results of the MIC framework and its varying combinations in terms of precision and recall metrics