Investigating non lexical markers of the language of schizophrenia in spontaneous conversations (2021)

Abstract: We investigate linguistic markers associated with schizophrenia in clinical conversations by detecting predictive features among French-speaking patients. Dealing with human-human dialogues makes for a realistic situation, but it calls for strategies to represent the context and face data sparsity. We compare different approaches for data representation – from individual speech turns to entire conversations –, and data modeling, using lexical, morphological, syntactic, and discourse features, dimensions presumed to be tightly connected to the language of schizophrenia. Previous English models were mostly lexical and reached high performance, here replicated (93.7% acc.). However, our analysis reveals that these models are heavily biased, which probably concerns most datasets on this task. Our new delexicalized models are more general and robust, with the best accuracy score at 77.9%.

## introduction schizophrenia is defined as a severe mental illness (apa, 2015) that comes with varied symptoms, ranging from delirium to hallucinations. among these symptoms, there are language disorders, especially the so-called positive thought disorder (i.e., disorganized language output such as derailment and tangentiality) foot_0 and negative thought disorder foot_1 @xcite . schizophrenia affects about 1% of the world's adult population, with cognitive troubles for 70-80% of the patients @xcite . since the symptoms often affect language skills, several studies proposed using nlp techniques on patients' productions (section 2) to identify what is affected in language, thus understand better the disease and its symptoms and how language works in general. in this paper, we explore linguistic markers of schizophrenia through feature exploration within a classification system. we do so on spontaneous dialogues in french where all the previous work was in english and most used social media data or monologues. replicating state-of-the-art results allows us to confirm some previous findings of specific features of the language of schizophrenia. our study focuses on two aspects: carefully exploring data representations and investigating preliminary modeling of dialogues, both with scarce data. using spontaneous conversations makes for a realistic scenario -the patient is merely talking with her clinician. however, representing dialogues is not easy: we restrict ourselves to patients' speech turns, and test varied context windows to tackle data sparsity. additionally, we compare several representations and confirm that lexicon is a good indicator, making for high-performing models with at best 93.7% (acc.). nevertheless, our analysis demonstrates that it probably corresponds to a bias in our data caused by the constraints imposed during the collection process. most of the datasets are likely biased the same way. this analysis led us to delexicalized models while focusing on dimensions presumed to be affected in schizophrenia: morphosyntactic, syntactic, dialogue, and discourse information are therefore considered. our best delexicalized model gets 77.9% (acc.) and shows the importance of morpho-syntactic information and high-level features in dialogue. when dealing with medical data, ethical questions arise. the diagnosis of schizophrenia is complex and relies on many indices. automatic systems could provide psychiatrists with further clues, possibly alleviating the need for the patients to go through several cognitive tests, but this is a longreach goal. it is clear that the systems developed can not substitute for a human expert, as a diagnosis is a medical act. moreover, linguistic clues, while crucial, have to be interpreted within the patient's social environment. contributions this study: (i) is the first in french, replicating english studies with comparable results with less data and resources; (ii) continues seminal work on schizophrenia detection in dialogues but with a focus on modeling and bias -two crucial issues for a task inherently data-scarce; (iii) reveals language features of schizophrenia, confirming psychologists' descriptions on the use of complex structures or the capacity to maintain conversation. 3 2 related work psychiatrists rely on language and speech behavior as one of the main clues in psychiatric diagnosis @xcite . they found that these patients' speech tends to be less predictable @xcite @xcite , with a poorer vocabulary @xcite . it has also been found that their productions tend to be more grammatically deviant @xcite and less syntactically complex than that of controls @xcite . at the discourse level, they associate words within a larger context than controls @xcite with often more diffuse associations @xcite . they also present referential impairments -categorized as vagueness, missing information, or confusing reference @xcite -, and specific discontinuities at the discourse level @xcite . on the other hand, many researchers have used nlp methods to help to identify mental disorders, such as depression @xcite @xcite , posttraumatic stress disorder @xcite , suicide risk @xcite , alzheimer's disease @xcite , and autism @xcite . for schizophrenia, previous work has mainly focused on lexical information @xcite @xcite . unlike ours, these studies rely on linguistic inquiry word count (liwc) cate-gories @xcite ) -psycho-metrically validated lexicon mapping words to psychological concepts), latent dirichlet allocation (lda) @xcite -inferring topics in each document, and brown clustering @xcite ) -grouping contextually similar words into the same cluster. however, most of these resources are only available in english. more recent approaches considered syntactic, semantic, and sentiment information @xcite . both studies show good performance with morpho-syntactic features, especially with part-of-speech (pos) tags. 4 they were based on narrative texts (essays and tweets). we here demonstrate that some findings can generalize to spontaneous conversations. @xcite proposed the first study on detecting schizophrenia patients from conversations, mostly limited to lexical features. also, close to our work, @xcite investigated linguistic features in transcripts of conversations between patients and clinicians. the authors tried to predict patient satisfaction and adherence to treatment on the concatenation of speech turns of the patient. inspired by the work of @xcite , we also use higher-level features (see section 3) on real conversations but directly investigating a model of detecting patients with schizophrenia symptoms. furthermore, we extend previous work by varying the length of dialogues and testing more complex features, including sequences of pos tags, finer tree representations, and dialogical information. ## approach varying dialogue size: our data are composed of 41 dialogues with 2, 811 words, and 268 speech turns on average (when limited to patients/controls). the clinician's speech turns are ignored in all dialogues to reduce their impact on classification, but further studies should also include the interaction. first, we concatenate all the speech turns of a patient/control (full setting), thus making for a large document that contains the whole context. since the documents are long, it could be hard for the system to find regularities, especially with only a few classification instances (i.e., 41). the opposite 4 pos tagging is a process of marking up a word in a text to a particular part of speech. @xcite tested two types of pos tags: a general one called meta-pos (12 labels) and a precise one pos (160 labels). both allow performance higher than chance. option is to classify each speech turn individually (indiv.): this leads to more instances (10, 319), but we lose the context of the neighboring speech turns. moreover, the speech turns are of varied length with an average of 11 words; some of them contain too few words to be informative. the last option is in between: we use a window of at least n words (w-n), always going until the end of the current speech turn, to assess the possibility of identifying distinctive features already in smaller blocks of conversation. we test @xmath0, 256, 512}, providing with middle representations (see comparing representations: existing work on schizophrenia language demonstrated the importance of lexical features. for french, as for many languages, we do not have access to a resource such as liwc. we thus propose to simply include bag-of-words (bow) and n-grams (@xmath1, 3}) to our models as a proxy for topic identification. @xcite showed the importance of features specific to spontaneous dialogues that do involve lexicon but in a more generic way: ocr corresponds to open class repair initiators (pardon?, huh?); backchannel (bc) responses are phatic expressions (yeah, hum mm). to reflect text organization, we also include discourse features by extracting the forms (without disambiguation) corresponding to connectives (but, because, since) as identified in lexconn @xcite . finally, we test the two following non-lexical features: part-of-speech n-grams and treelets. @xcite demonstrated that pos tags are effective features. we also test for larger patterns with sequences, pos n-gram with @xmath2, 2, 3}. @xcite only used the dependencies as syntactic features. we extend to treelet features @xcite based on the dependency parse trees: 2-treelet corresponds to 2 tokens with a syntactic relation between a head and a dependent, e.g., '@xmath3, and 3-treelet corresponds to 3 tokens with 2 syntactic relations: could be 1 head dominates 2 dependents or a chain of dependencies, e.g., 'pron←poss←noun←nsubj←verb'. see figure 1 for an illustration. ## experimental setting data: forty-one conversations between patients (18) or controls (23) and a psychologist come from slam project @xcite . the transcripts are standardized and follow a transcription guide. the groups are balanced with gender, age, intelligence quotient (iq) score, years of studies, and three cognitive tests' results (wais-iii, tmt, cvlt) foot_3 . they are free exchanges carried out in a medical setting where the psychologist is not personally involved -her main action is to maintain the exchange. preliminary experiments showed that we could distinguish the two groups with relatively high accuracy with the clinician's data. we thus removed clinician's speech turns to reduce this impact and only focused on patients' factors. further studies are needed to decide how to take into account the entire interaction. classification: we compare several classification algorithms: support vector machines (svm) @xcite , logistic regression (lr), random forest (rf), perceptron (perc), and naive bayes (nb), without and with feature selection based on importance weight, all implemented in scikit-learn @xcite . hyperparameters are: thresholds for feature selection are the range of 10 values equally distributed from 1e -5 to the weight of the 50 th most important feature (thus allowing to keep at least 50 features), plus the mean and median of the weights. since our dataset is minimal, we use nested cross-validation to assess the performance of our system: tune hyper-parameters on k -1 folds and then evaluate on the left-out fold, repeating the whole process m times (@xmath4. we report average accuracy over the m out folds. best hyper-parameters values and algorithms are given in appendix a.2. ## conclusion we used conversations involving patients with schizophrenia in order to learn about language features associated with the disease. we compared various settings to represent dialogues and several representations to deal with data scarcity and lexical bias. our experiments replicate performances as high as previous studies in english. further experiments will be designed to take into account the entire interaction, probably with neural networks. we would also like to investigate the effect of adversarial loss in mitigating the bias within a neural model. we hope that this paper will remind us of the importance of looking for bias in data and exploring higher-level, less language-dependent information to produce robust systems and draw more general conclusions on conversational data. masahito sakishita, chihiro ogawa, kenji j tsuchiya, toshiki iwabuchi, taishiro kishimoto, and yoshinobu kano. 2019. autism spectrum disorder's severity prediction model using utterance features for automatic diagnosis support. in international workshop on health intelligence, pages 83-95. springer. k salzinger. 1979. ecolinguistics: a radical behavior theory approach to language behavior. kurt salzinger and muriel hammer. 1963. some formal characteristics of schizophrenic speech as a measure of social deviance. annals of the new york academy of sciences. kurt salzinger, stephanie portnoy, and richard s feldman. 1964. verbal behavior of schizophrenic and normal subjects. annals of the new york academy of sciences, 105(15):845-860. kurt salzinger, stephanie portnoy, david b pisoni, and richard s feldman. 1970. the immediacy hypothesis and response-produced stimuli in schizophrenic speech. journal of abnormal psychology, 76(2):258. ivan sekulić and michael strube. 2019. adapting deep learning methods for mental health prediction on social media. in proceedings of the 5th workshop on noisy user-generated text (w-nut 2019), pages 322-327. shihao xu, zixu yang, debsubhra chakraborty, yasir tahir, tomasz maszczyk, yi han victoria chua, justin dauwels, daniel thalmann, nadia magnenat thalmann, bhing-leet tan, et al. 2019. automated lexical analysis of interviews with individuals with schizophrenia. in proceedings of the 9th international workshop on spoken dialogue system technology, pages 185-197. springer.