Large language models as windows on the mental structure of psychopathology
Authors: José Guillen, Eugenio Herrera-Berg, Apoorva Bhandari, Pablo León-Villagrá, Marc-Lluís Vives, Cristian Buc Calderon


Reviewer paper summaries: The paper explores the potential of Large Language Models (LLMs) to serve as a tool for understanding the mental structures associated with various psychopathologies. The authors demonstrate that LLMs can mimic responses consistent with intended psychopathologies, capture correlations between diagnostic questionnaire scores, and generate semantic structures that are qualitatively similar to those observed in schizophrenic patients. This work suggests that LLMs could contribute to research hypotheses, diagnostic tool development, and the expansion of knowledge about psychopathologies.

The paper shows how LLMs can effectively show a human-like behavior by generating lexical production in a psychopathological perspective. This makes these models suitable to support the research in the specific field on neuropathology, as well as showing the ability to simulate lexical usage biased by specific mental structure, as demonstrated in the proposed case study of schizofrenia.
Summary of strengths: 1. The paper presents a creative use of LLMs to simulate mental states associated with psychopathologies, which could be a significant advancement in the field of cognitive science and mental health research, which is valuable for mental health-related research.
2. The authors provide empirical evidence that LLMs can replicate behavioral patterns observed in psychopathologies, such as schizophrenia, which supports the idea that LLMs can be used as a diagnostic tool or for hypothesis testing.
3. The work offers theoretical insights into the structure of mental processes in psychopathology, which could lead to new understandings of mental health conditions and inform treatment strategies.

The point of strengths are not limited in the specific domain of neuropathology, by giving a contribution to the advancement of our comprehension of what these models can do with reference to "capturing" specific structures in semantic representations.
Summary of weaknesses: 1. The paper focuses on schizophrenia as a case study, but it is unclear how well the method would generalize to other psychopathologies. Further experiments are needed to confirm the applicability of this approach across a broader spectrum of mental health conditions.
2. Although the paper demonstrates qualitative similarities between LLM-generated responses and those of schizophrenic patients, it lacks a quantitative comparison with clinical data. A more rigorous validation with larger, diverse clinical datasets could strengthen the claims.
3.  The paper does not address potential ethical implications of using LLMs to simulate psychopathologies. Issues around the representation of mental health conditions and the use of such simulations in clinical settings should be considered.
4. The analysis of the semantic structures induced in LLMs with distinct psychopathologies is detailed, but it is not clear how easily these structures can be interpreted by clinical psychologists or translated into practical applications.
5. Some new mental-realted LLMs should be introduced and used for comparison.

The only weak aspect may be represented by the moral and ethic considerations on the potential impact of such methods.
Comments, suggestions and typos: None

There is a word missing at line 580: ... regarding the link mental structure and psychopathology .... I suppose there should be a "between"
Overall assessment scores: 2.5

4