pwc-archive/papers-with-abstracts huggingface dataset:
TMU System for SLAM-2018
Authors: Masahiro Kaneko, Tomoyuki Kajiwara, Mamoru Komachi
Date: 2018-06-01 00:00:00
Proceeding: WS 2018 6
Tasks: Language Acquisition

We introduce the TMU systems for the second language acquisition modeling shared task 2018 (Settles et al., 2018). To model learner error patterns, it is necessary to maintain a considerable amount of information regarding the type of exercises learners have been learning in the past and the manner in which they answered them. Tracking an enormous learner{'}s learning history and their correct and mistaken answers is essential to predict the learner{'}s future mistakes. Therefore, we propose a model which tracks the learner{'}s learning history efficiently. Our systems ranked fourth in the English and Spanish subtasks, and fifth in the French subtask.
Deep Factorization Machines for Knowledge Tracing
Authors: Jill-J{\^e}nn Vie
Date: 2018-06-01 00:00:00
Proceeding: WS 2018 6
Tasks: Knowledge Tracing, Language Acquisition, regression

This paper introduces our solution to the 2018 Duolingo Shared Task on Second Language Acquisition Modeling (SLAM). We used deep factorization machines, a wide and deep learning model of pairwise relationships between users, items, skills, and other entities considered. Our solution (AUC 0.815) hopefully managed to beat the logistic regression baseline (AUC 0.774) but not the top performing model (AUC 0.861) and reveals interesting strategies to build upon item response theory models.
CLUF: a Neural Model for Second Language Acquisition Modeling
Authors: Shuyao Xu, Jin Chen, Long Qin
Date: 2018-06-01 00:00:00
Proceeding: WS 2018 6
Tasks: Decoder, Knowledge Tracing, Language Acquisition, Language Modeling, Language Modelling, Machine Reading Comprehension, Machine Translation, Reading Comprehension, Sentiment Analysis, Text Classification

Second Language Acquisition Modeling is the task to predict whether a second language learner would respond correctly in future exercises based on their learning history. In this paper, we propose a neural network based system to utilize rich contextual, linguistic and user information. Our neural model consists of a Context encoder, a Linguistic feature encoder, a User information encoder and a Format information encoder (CLUF). Furthermore, a decoder is introduced to combine such encoded features and make final predictions. Our system ranked in first place in the English track and second place in the Spanish and French track with an AUROC score of 0.861, 0.835 and 0.854 respectively.
Neural sequence modelling for learner error prediction
Authors: Zheng Yuan
Date: 2018-06-01 00:00:00
Proceeding: WS 2018 6
Tasks: Grammatical Error Detection, Language Acquisition, Prediction

This paper describes our use of two recurrent neural network sequence models: sequence labelling and sequence-to-sequence models, for the prediction of future learner errors in our submission to the 2018 Duolingo Shared Task on Second Language Acquisition Modeling (SLAM). We show that these two models capture complementary information as combining them improves performance. Furthermore, the same network architecture and group of features can be used directly to build competitive prediction models in all three language tracks, demonstrating that our approach generalises well across languages.
Automatic Distractor Suggestion for Multiple-Choice Tests Using Concept Embeddings and Information Retrieval
Authors: Le An Ha, Victoria Yaneva
Date: 2018-06-01 00:00:00
Proceeding: WS 2018 6
Tasks: Information Retrieval, Multiple-choice, Retrieval

Developing plausible distractors (wrong answer options) when writing multiple-choice questions has been described as one of the most challenging and time-consuming parts of the item-writing process. In this paper we propose a fully automatic method for generating distractor suggestions for multiple-choice questions used in high-stakes medical exams. The system uses a question stem and the correct answer as an input and produces a list of suggested distractors ranked based on their similarity to the stem and the correct answer. To do this we use a novel approach of combining concept embeddings with information retrieval methods. We frame the evaluation as a prediction task where we aim to {``}predict{''} the human-produced distractors used in large sets of medical questions, i.e. if a distractor generated by our system is good enough it is likely to feature among the list of distractors produced by the human item-writers. The results reveal that combining concept embeddings with information retrieval approaches significantly improves the generation of plausible distractors and enables us to match around 1 in 5 of the human-produced distractors. The approach proposed in this paper is generalisable to all scenarios where the distractors refer to concepts.
Cross-Lingual Content Scoring
Authors: Andrea Horbach, Sebastian Stennmanns, Torsten Zesch
Date: 2018-06-01 00:00:00
Proceeding: WS 2018 6
Tasks: Machine Translation

We investigate the feasibility of cross-lingual content scoring, a scenario where training and test data in an automatic scoring task are from two different languages. Cross-lingual scoring can contribute to educational equality by allowing answers in multiple languages. Training a model in one language and applying it to another language might also help to overcome data sparsity issues by re-using trained models from other languages. As there is no suitable dataset available for this new task, we create a comparable bi-lingual corpus by extending the English ASAP dataset with German answers. Our experiments with cross-lingual scoring based on machine-translating either training or test data show a considerable drop in scoring quality.
What type of happiness are you looking for? - A closer look at detecting mental health from language
Authors: Alina Arseniev-Koehler, Sharon Mozgai, Stefan Scherer
Date: 2018-06-01 00:00:00
Proceeding: WS 2018 6
Tasks: Diagnostic

Computational models to detect mental illnesses from text and speech could enhance our understanding of mental health while offering opportunities for early detection and intervention. However, these models are often disconnected from the lived experience of depression and the larger diagnostic debates in mental health. This article investigates these disconnects, primarily focusing on the labels used to diagnose depression, how these labels are computationally represented, and the performance metrics used to evaluate computational models. We also consider how medical instruments used to measure depression, such as the Patient Health Questionnaire (PHQ), contribute to these disconnects. To illustrate our points, we incorporate mixed-methods analyses of 698 interviews on emotional health, which are coupled with self-report PHQ screens for depression. We propose possible strategies to bridge these gaps between modern psychiatric understandings of depression, lay experience of depression, and computational representation.
A Linguistically-Informed Fusion Approach for Multimodal Depression Detection
Authors: Michelle Morales, Stefan Scherer, Rivka Levitan
Date: 2018-06-01 00:00:00
Proceeding: WS 2018 6
Tasks: Depression Detection

Automated depression detection is inherently a multimodal problem. Therefore, it is critical that researchers investigate fusion techniques for multimodal design. This paper presents the first-ever comprehensive study of fusion techniques for depression detection. In addition, we present novel linguistically-motivated fusion techniques, which we find outperform existing approaches.
Expert, Crowdsourced, and Machine Assessment of Suicide Risk via Online Postings
Authors: Han-Chin Shing, Suraj Nair, Ayah Zirikly, Meir Friedenberg, Hal Daum{\'e} III, Philip Resnik
Date: 2018-06-01 00:00:00
Proceeding: WS 2018 6

We report on the creation of a dataset for studying assessment of suicide risk via online postings in Reddit. Evaluation of risk-level annotations by experts yields what is, to our knowledge, the first demonstration of reliability in risk assessment by clinicians based on social media postings. We also introduce and demonstrate the value of a new, detailed rubric for assessing suicide risk, compare crowdsourced with expert performance, and present baseline predictive modeling experiments using the new dataset, which will be made available to researchers through the American Association of Suicidology.
CLPsych 2018 Shared Task: Predicting Current and Future Psychological Health from Childhood Essays
Authors: Veronica Lynn, Alissa Goodman, Kate Niederhoffer, Kate Loveys, Philip Resnik, H. Andrew Schwartz
Date: 2018-06-01 00:00:00
Proceeding: WS 2018 6

We describe the shared task for the CLPsych 2018 workshop, which focused on predicting current and future psychological health from an essay authored in childhood. Language-based predictions of a person{'}s current health have the potential to supplement traditional psychological assessment such as questionnaires, improving intake risk measurement and monitoring. Predictions of future psychological health can aid with both early detection and the development of preventative care. Research into the mental health trajectory of people, beginning from their childhood, has thus far been an area of little work within the NLP community. This shared task represents one of the first attempts to evaluate the use of early language to predict future health; this has the potential to support a wide variety of clinical health care tasks, from early assessment of lifetime risk for mental health problems, to optimal timing for targeted interventions aimed at both prevention and treatment.
Using contextual information for automatic triage of posts in a peer-support forum
Authors: Edgar Altszyler, Ariel J. Berenstein, David Milne, Rafael A. Calvo, Fern, Diego ez Slezak
Date: 2018-06-01 00:00:00
Proceeding: WS 2018 6

Mental health forums are online spaces where people can share their experiences anonymously and get peer support. These forums, require the supervision of moderators to provide support in delicate cases, such as posts expressing suicide ideation. The large increase in the number of forum users makes the task of the moderators unmanageable without the help of automatic triage systems. In the present paper, we present a Machine Learning approach for the triage of posts. Most approaches in the literature focus on the content of the posts, but only a few authors take advantage of features extracted from the context in which they appear. Our approach consists of the development and implementation of a large variety of new features from both, the content and the context of posts, such as previous messages, interaction with other users and author{'}s history. Our method has competed in the CLPsych 2017 Shared Task, obtaining the first place for several of the subtasks. Moreover, we also found that models that take advantage of post context improve significantly its performance in the detection of flagged posts (posts that require moderators attention), as well as those that focus on post content outperforms in the detection of most urgent events.
Hierarchical neural model with attention mechanisms for the classification of social media text related to mental health
Authors: Julia Ive, George Gkotsis, Rina Dutta, Robert Stewart, Sumithra Velupillai
Date: 2018-06-01 00:00:00
Proceeding: WS 2018 6
Tasks: Classification, Decision Making, Feature Engineering, General Classification, text-classification, Text Classification

Mental health problems represent a major public health challenge. Automated analysis of text related to mental health is aimed to help medical decision-making, public health policies and to improve health care. Such analysis may involve text classification. Traditionally, automated classification has been performed mainly using machine learning methods involving costly feature engineering. Recently, the performance of those methods has been dramatically improved by neural methods. However, mainly Convolutional neural networks (CNNs) have been explored. In this paper, we apply a hierarchical Recurrent neural network (RNN) architecture with an attention mechanism on social media data related to mental health. We show that this architecture improves overall classification results as compared to previously reported results on the same data. Benefitting from the attention mechanism, it can also efficiently select text elements crucial for classification decisions, which can also be used for in-depth analysis.
Cross-cultural differences in language markers of depression online
Authors: Kate Loveys, Jonathan Torrez, Alex Fine, Glen Moriarty, Glen Coppersmith
Date: 2018-06-01 00:00:00
Proceeding: WS 2018 6
Tasks: Cultural Vocal Bursts Intensity Prediction, Data Visualization

Depression is a global mental health condition that affects all cultures. Despite this, the way depression is expressed varies by culture. Uptake of machine learning technology for diagnosing mental health conditions means that increasingly more depression classifiers are created from online language data. Yet, culture is rarely considered as a factor affecting online language in this literature. This study explores cultural differences in online language data of users with depression. Written language data from 1,593 users with self-reported depression from the online peer support community 7 Cups of Tea was analyzed using the Linguistic Inquiry and Word Count (LIWC), topic modeling, data visualization, and other techniques. We compared the language of users identifying as White, Black or African American, Hispanic or Latino, and Asian or Pacific Islander. Exploratory analyses revealed cross-cultural differences in depression expression in online language data, particularly in relation to emotion expression, cognition, and functioning. The results have important implications for avoiding depression misclassification from machine-driven assessments when used in a clinical setting, and for avoiding inadvertent cultural biases in this line of research more broadly.
Deep Learning for Depression Detection of Twitter Users
Authors: Ahmed Husseini Orabi, Prasadith Buddhitha, Mahmoud Husseini Orabi, Diana Inkpen
Date: 2018-06-01 00:00:00
Proceeding: WS 2018 6
Tasks: Deep Learning, Depression Detection

Mental illness detection in social media can be considered a complex task, mainly due to the complicated nature of mental disorders. In recent years, this research area has started to evolve with the continuous increase in popularity of social media platforms that became an integral part of people{'}s life. This close relationship between social media platforms and their users has made these platforms to reflect the users{'} personal life with different limitations. In such an environment, researchers are presented with a wealth of information regarding one{'}s life. In addition to the level of complexity in identifying mental illnesses through social media platforms, adopting supervised machine learning approaches such as deep neural networks have not been widely accepted due to the difficulties in obtaining sufficient amounts of annotated training data. Due to these reasons, we try to identify the most effective deep neural network architecture among a few of selected architectures that were successfully used in natural language processing tasks. The chosen architectures are used to detect users with signs of mental illnesses (depression in our case) given limited unstructured text data extracted from the Twitter social media platform.
Current and Future Psychological Health Prediction using Language and Socio-Demographics of Children for the CLPysch 2018 Shared Task
Authors: Sharath Ch Guntuku, ra, Salvatore Giorgi, Lyle Ungar
Date: 2018-06-01 00:00:00
Proceeding: WS 2018 6
Tasks: Clustering

This article is a system description and report on the submission of a team from the University of Pennsylvania in the {'}CLPsych 2018{'} shared task. The goal of the shared task was to use childhood language as a marker for both current and future psychological health over individual lifetimes. Our system employs multiple textual features derived from the essays written and individuals{'} socio-demographic variables at the age of 11. We considered several word clustering approaches, and explore the use of linear regression based on different feature sets. Our approach showed best results for predicting distress at the age of 42 and for predicting current anxiety on Disattenuated Pearson Correlation, and ranked fourth in the future health prediction task. In addition to the subtasks presented, we attempted to provide insight into mental health aspects at different ages. Our findings indicate that misspellings, words with illegible letters and increased use of personal pronouns are correlated with poor mental health at age 11, while descriptions about future physical activity, family and friends are correlated with good mental health.
Predicting Psychological Health from Childhood Essays with Convolutional Neural Networks for the CLPsych 2018 Shared Task (Team UKNLP)
Authors: Anthony Rios, Tung Tran, Ramakanth Kavuluru
Date: 2018-06-01 00:00:00
Proceeding: WS 2018 6
Tasks: regression

This paper describes the systems we developed for tasks A and B of the 2018 CLPsych shared task. The first task (task A) focuses on predicting behavioral health scores at age 11 using childhood essays. The second task (task B) asks participants to predict future psychological distress at ages 23, 33, 42, and 50 using the age 11 essays. We propose two convolutional neural network based methods that map each task to a regression problem. Among seven teams we ranked third on task A with disattenuated Pearson correlation (DPC) score of 0.5587. Likewise, we ranked third on task B with an average DPC score of 0.3062.
A Psychologically Informed Approach to CLPsych Shared Task 2018
Authors: Almog Simchon, Michael Gilead
Date: 2018-06-01 00:00:00
Proceeding: WS 2018 6

This paper describes our approach to the CLPsych 2018 Shared Task, in which we attempted to predict cross-sectional psychological health at age 11 and future psychological distress based on childhood essays. We attempted several modeling approaches and observed best cross-validated prediction accuracy with relatively simple models based on psychological theory. The models provided reasonable predictions in most outcomes. Notably, our model was especially successful in predicting out-of-sample psychological distress (across people and across time) at age 50.
Predicting Psychological Health from Childhood Essays. The UGent-IDLab CLPsych 2018 Shared Task System.
Authors: Klim Zaporojets, Lucas Sterckx, Johannes Deleu, Thomas Demeester, Chris Develder
Date: 2018-06-01 00:00:00
Proceeding: WS 2018 6

This paper describes the IDLab system submitted to Task A of the CLPsych 2018 shared task. The goal of this task is predicting psychological health of children based on language used in hand-written essays and socio-demographic control variables. Our entry uses word- and character-based features as well as lexicon-based features and features derived from the essays such as the quality of the language. We apply linear models, gradient boosting as well as neural-network based regressors (feed-forward, CNNs and RNNs) to predict scores. We then make ensembles of our best performing models using a weighted average.
Can adult mental health be predicted by childhood future-self narratives? Insights from the CLPsych 2018 Shared Task
Authors: Kylie Radford, Louise Lavrencic, Ruth Peters, Kim Kiely, Ben Hachey, Scott Nowson, Will Radford
Date: 2018-06-01 00:00:00
Proceeding: WS 2018 6
Tasks: Epidemiology, regression

The CLPsych 2018 Shared Task B explores how childhood essays can predict psychological distress throughout the author{'}s life. Our main aim was to build tools to help our psychologists understand the data, propose features and interpret predictions. We submitted two linear regression models: ModelA uses simple demographic and word-count features, while ModelB uses linguistic, entity, typographic, expert-gazetteer, and readability features. Our models perform best at younger prediction ages, with our best unofficial score at 23 of 0.426 disattenuated Pearson correlation. This task is challenging and although predictive performance is limited, we propose that tight integration of expertise across computational linguistics and clinical psychology is a productive direction.
Automatic Detection of Incoherent Speech for Diagnosing Schizophrenia
Authors: Dan Iter, Jong Yoon, Dan Jurafsky
Date: 2018-06-01 00:00:00
Proceeding: WS 2018 6
Tasks: Sentence, Sentence Embedding, Sentence-Embedding, Word Embeddings

Schizophrenia is a mental disorder which afflicts an estimated 0.7{\%} of adults world wide. It affects many areas of mental function, often evident from incoherent speech. Diagnosing schizophrenia relies on subjective judgments resulting in disagreements even among trained clinicians. Recent studies have proposed the use of natural language processing for diagnosis by drawing on automatically-extracted linguistic features like discourse coherence and lexicon. Here, we present the first benchmark comparison of previously proposed coherence models for detecting symptoms of schizophrenia and evaluate their performance on a new dataset of recorded interviews between subjects and clinicians. We also present two alternative coherence metrics based on modern sentence embedding techniques that outperform the previous methods on our dataset. Lastly, we propose a novel computational model for reference incoherence based on ambiguous pronoun usage and show that it is a highly predictive feature on our data. While the number of subjects is limited in this pilot study, our results suggest new directions for diagnosing common symptoms of schizophrenia.
Oral-Motor and Lexical Diversity During Naturalistic Conversations in Adults with Autism Spectrum Disorder
Authors: Julia Parish-Morris, Evangelos Sariyanidi, Casey Zampella, G. Keith Bartley, Emily Ferguson, Ashley A. Pallathra, Leila Bateman, Samantha Plate, Meredith Cola, P, Juhi ey, Edward S. Brodkin, Robert T. Schultz, Birkan Tun{\c{c}}
Date: 2018-06-01 00:00:00
Proceeding: WS 2018 6
Tasks: Decision Making, Diversity

Autism spectrum disorder (ASD) is a neurodevelopmental condition characterized by impaired social communication and the presence of restricted, repetitive patterns of behaviors and interests. Prior research suggests that restricted patterns of behavior in ASD may be cross-domain phenomena that are evident in a variety of modalities. Computational studies of language in ASD provide support for the existence of an underlying dimension of restriction that emerges during a conversation. Similar evidence exists for restricted patterns of facial movement. Using tools from computational linguistics, computer vision, and information theory, this study tests whether cognitive-motor restriction can be detected across multiple behavioral domains in adults with ASD during a naturalistic conversation. Our methods identify restricted behavioral patterns, as measured by entropy in word use and mouth movement. Results suggest that adults with ASD produce significantly less diverse mouth movements and words than neurotypical adults, with an increased reliance on repeated patterns in both domains. The diversity values of the two domains are not significantly correlated, suggesting that they provide complementary information.