Utterance Classification with Logical Neural Network: Explainable AI for Mental Disorder Diagnosis (2023)

Abstract: In response to the global challenge of mental health problems, we proposes a Logical Neural Network (LNN) based Neuro-Symbolic AI method for the diagnosis of mental disorders. Due to the lack of effective therapy coverage for mental disorders, there is a need for an AI solution that can assist therapists with the diagnosis. However, current Neural Network models lack explainability and may not be trusted by therapists. The LNN is a Recurrent Neural Network architecture that combines the learning capabilities of neural networks with the reasoning capabilities of classical logic-based AI. The proposed system uses input predicates from clinical interviews to output a mental disorder class, and different predicate pruning techniques are used to achieve scalability and higher scores. In addition, we provide an insight extraction method to aid therapists with their diagnosis. The proposed system addresses the lack of explainability of current Neural Network models and provides a more trustworthy solution for mental disorder diagnosis.

## introduction a mental disorder is a significant deterioration of human thinking, emotional control, or behavior, which is diagnosed clinically and can affect key areas of life. due to the covid-19 pandemic, the number of people who suffer from anxiety and depressive illnesses greatly increased in 2020. initial projections indicate a 26% and 28% increase in anxiety and major depressive disorders respectively during the first year of the pandemic (who). moreover, every year, 703 000 people commit suicide, with many more attempting to do so. although people of all ages commit suicides, it is alarming that in 2019 suicide was one of the leading causes of death among young people worldwide (sui). furthermore, around 24 million people, or 1 in 300 persons (0.32%), globally suffer from schizophrenia. although it is not common as other mental disorders, schizophrenia produces psychosis, is associated with significant disability, and may have an impact on all aspects of life, including personal, family, social, educational, and occupational functioning (sch). diagnosis of mental disorders is accomplished through a clinical interview, where a therapist evaluates the mental health of the patient and identifies possible disorders based on symptoms. however, although many mental health issues may be properly treated at low cost, there is still a wide gap between those who need care and those who have access to it. despite the progress in some countries, there is still a severe lack of effective therapy coverage. therefore, there is a need for an ai solution that can assist therapists with a diagnosis of mental disorders. although current neural network (nn) models are powerful and can operate in a wide range of tasks, their effectiveness in mental disorder classification is questionable due to their black-box nature. in this regard, the model explainability is a vital property, which is required to make a diagnosis of mental disorders. while neural network models can achieve high scores, therapists may be hesitant to trust such tools and accept classification results if proper explanations are not provided. because of nn nature, it is impossible to tell whether their predictions are the result of robust features or some spurious clues @xcite . there are attempts to provide interpretable insights in mental disorder diagnosis, such as using topic modeling to extract concepts @xcite or inferring psychological properties such as working alliance @xcite . although such approaches can enable explainable ai systems for passive assistance @xcite or interventional recommendations @xcite to the therapists, applying these insights directly to the classification problem yields suboptimal performance @xcite . furthermore, despite being able to provide global explanations for the prediction @xcite , traditional ml models lack scalability and they are not generalizable for broader tasks. in this regard, logical neural network @xcite ) might be a good solution to the problem. it is a neuro-symbolic ai method (nsai), that combines the learning capabilities of neural networks with the reasoning capabilities of classical logic-based ai. the lnn is a recurrent neural network architecture in which neurons represent a precisely defined notion of weighted real-valued logic. it has a 1-to-1 relationship to a system of logical formulae. the main problem related to this approach is that it has not been implemented for the supervised learning utterance classification task. therefore, this work proposes an lnn-based explainable nsai utterance classification method for mental disorder diagnosis. the model was trained with different predicate pruning techniques to achieve scalability and higher scores. the advantages of the proposed system can be summarized via the following points: ## supervised learning with lnn although nsai supports data driven training of the network, it encodes knowledge into logic rules with predicates as inputs, where predicates represents a property or a relation. therefore, nsai method requires special preprocessing of the dataset to generate predicates and data samples for training and testing purposes. the proposed system consists of two parts abstract meaning representation (amr) @xcite semantic parser and lnn. fig. 1 shows overall pipeline of the system, first part containing amr parser is used to convert raw text into classifier input data, and second part is an lnn model which performs rule-based classification. ## experiment results in this section experiment results for predicate pruning and lnn model evaluation will be provided. the lnn models have been trained using different pruning methods and have been compared with deep learning (dl) and lnn baselines. the number of predicates and training samples are shown in the according to the ## discussions and future work scaling of the lnn is a significant issue which requires selection of the right predicates. pruning of the predicates essentially limits the knowledge base of the lnn, thus it is important to understand the effect of the predicates on model performance. frequency predicate models have not shown great results, the possible explanation for that behavior can be found in predicate analysis. the analysis shows that predicates with higher frequencies also tend to be inclusive for several classes. such predicates might be extracted from common dialogue phrases, that are common to regular conversations. thus, it is more difficult to learn for lnn in such circumstances and it might lead to a behavior similar to the random classifiers'. moreover, variation in the frequency thresholds did not affect the overall performance of the lnn model. thus, it can be concluded that frequency predicates cannot provide a quality selection of the predicates when they are applied alone. furthermore, in the case of exclusive predicates, the model has learned depression class better than others. it can be explained by the depression class possessing more exclusive predicates compared to other classes. interestingly, the model has learned to identify non-schizophrenia samples better than schizophrenia samples. possible reasoning for that is fewer predicates for schizophrenia compared to other classes. furthermore, some mental disorders have the same symptoms, and exclusive pruning eliminate such predicates from the training, which might lead to limited diagnostic abilities. thus, exclusive predicates should be combined with other methods to provide trade-off between generalization and exclusivity of predicates. another challenge of this line of work is the usage of ai for mental disorder diagnosis. as pointed out in @xcite , one significant challenge is related to the privacy and security of patient data. to train the model, the system requires access to sensitive patient data, which must be protected from unauthorized access or misuse. there is also a concern that the use of ai in mental health diagnosis may lead to the stigmatization of individuals with mental disorders. in this work, we have deidentified all the sessions and all the transcripts are obtained under proper license and consent. we would also like to point out that the system may not work for all individuals, which could lead to misdiagnosis or lack of diagnosis, leading to harm to the patient. therefore, the ethical challenge lies in ensuring the system's reliability, fairness, and transparency and balancing the use of ai with the need for human involvement in mental health diagnosis and treatment, as part of the future work. the main advantage of the lnn over dl is in its explainability. it is possible to extract predicates with high weights for the each class and to examine which predicates contribute to the result significantly. ## conclusion mental disorders are a significant issue that is affecting more people every year. therefore, explainable ai mental disorder diagnosis through utterance classification can aid the therapist in their practice. in this work, a supervised learning setting for the lnn has been proposed to address this issue. moreover, predicate pruning methods based on the similarity, frequency, and exclusivity of the predicates are analyzed in terms of training performance. overall, the model trained with exclusive predicates shows the best results among the pruning methods, and acheived auc roc of 0.79 for the depression disorder. finally, explainability of the lnn diagnosis has been shown by analyzing significant predicates for each class and extracting the predicates with high weights.