Is Machine Psychology here? On Requirements for Using Human Psychological Tests on Large Language Models (2024)

Abstract: In an effort to better understand the behavior of large language models (LLM), researchers recently turned to conducting psychological assessments on them. Several studies diagnose various psychological concepts in LLMs, such as psychopathological symptoms, personality traits, and intellectual functioning, aiming to unravel their black-box characteristics. But can we safely assess LLMs with tests that were originally designed for humans? The psychology domain looks back on decades of developing standards of appropriate testing procedures to ensure reliable and valid measures. We argue that analogous standardization processes are required for LLM assessments, given their differential functioning as compared to humans. In this paper, we propose seven requirements necessary for testing LLMs. Based on these, we critically reflect a sample of 25 recent machine psychology studies. Our analysis reveals (1) the lack of appropriate methods to assess test reliability and construct validity, (2) the unknown strength of construct-irrelevant influences, such as the contamination of pre-training corpora with test material, and (3) the pervasive issue of non-reproducibility of many studies. The results underscore the lack of a general methodology for the implementation of psychological assessments of LLMs and the need to redefine psychological constructs specifically for large language models rather than adopting them from human psychology.

## introduction large language models (llm) demonstrate surprisingly strong natural language generation abilities across a range of tasks @xcite , sparking debates about the emergence of human characteristics, such as personality traits, empathy, intuitive reasoning, ethical understanding, or even traits of sentience, see e.g. @xcite * correspondance to: {lealoehn, niklas.kiehne}@gmail.com @xcite @xcite . yet recently, experts raised concerns about their inherent opaqueness and the potential dangers that could follow their widespread adoption @xcite . this incomprehensibility of the inner workings and decision processes of current llms prompted researchers to borrow methods from human psychology to shed light on the behavior of these black-box models: llms are analyzed via psychological assessments, often referred to as machine psychology or ai psychometrics @xcite . @xcite utilizes an unexpected contents task to diagnose theory of mind in gpt-4, arguing that the ability to ascribe mental states emerges with sufficient model size. yet, @xcite shows that trivial changes to the test items lead to the opposite outcome implying that gpt-4 does not have theory of mind. arguably, these contrary results are symptomatic of a general lack of standardization in the domain. meanwhile, the number of machine psychology studies grows quickly across various psychological constructs. the aim of this paper is to provide a solid foundation for using psychological tests on llms. indeed, many studies haphazardly use psychological tests without properly incorporating necessary theoretical underpinnings. grounded in the well-established standards of psychological testing, we propose seven essential requirements for test use in machine psychology. we thus advocate for stricter rules governing reliable, valid, and fair testing, also taking into consideration the quirks of current llms, such as their sensitivity to wording. as a proof of concept, we critically reflect 25 recent works regarding these requirements, highlighting the unresolved issues in the field. our analysis clearly challenges the evidential and declarative power of current methodologies for the psychological assessments of llms, while also providing a more reliable foundation. ## background the assessment of humans on diverse psychological constructs has been at the core of psychology as a scientific domain, dating back to at least the 19th century @xcite . the term construct refers to a group of psychological characteristics, such as behavioral patterns, personality traits or cognitive skills @xcite . a construct is often defined conceptually by abstractly describing its meaning and relations to other constructs, and operationally by stating variables used to measure it @xcite . the methods of psychological testing have been the subject of rigorous research for decades, aiming to enhance their reliability, validity, and overall effectiveness in assessing various aspects of human cognition, personality, and behavior @xcite . an important aspect in this regard is the formalization and standardization of correct assessment practices, concerning test development, application, and evaluation. we consider two widely accepted standards, namely the standards for educational and psychological testing and the international guidelines for test use @xcite , henceforth referred to as the standards. these guidelines are designed for test developers, administrators, and users to promote best practices and ethical standards in psychological testing. in contrast, the comparatively young machine psychology domain has not yet settled on such standards. the field itself is still developing, often using different terminology. @xcite propose the broad term machine behavior to combine methods from various sciences to better understand ai agents. @xcite suggest the area of ai psychometrics as a combination of psychology, computer science and linguistics. similarly, hagendorff (2023) introduce machine psychology as an umbrella term, which we will adopt throughout the paper. @xcite and @xcite argue that machine psychology differs from llm benchmarking by focusing on diagnosis rather than establishing performance. some efforts towards a more standardized methodology have been made by @xcite , who proposes a set of guidelines that should be considered when conducting machine psychology studies. these guidelines mainly focus on prompt design, given the significant impact it has on prompt completions of generative language models. frank (2023) suggests a combination of methods from developmentalists and computational scientists that could assist in uncovering abstract representations in language models. however, both of these approaches focus on the practical design of machine psychology studies rather than the question of which general criteria should be taken into account for a psychological test to be a meaningful assessment tool for llms. our work aims to provide a first set of normative requirements, prioritizing strictly necessary pre-requisites of correct testing over technical possibility. we build on proven methodologies from traditional psychology and show their applicability to the machine psychology domain. ## requirements for machine psychology we extracted and summarized a list of requirements for psychological testing from the standards that play a pivotal role in the selection, administration and scoring of tests. when conducting psychological assessments of llms, certain requirements that apply to human psychological assessments may not be necessary to consider. all requirements concerning the test taker's data privacy, for example, are inapplicable when the test subject is a machine. due to the different characteristics of human and ai examinees, the resulting requirements were transferred and adapted to the ai domain. we want to emphasize that our requirements are derived solely from psychological testing theory. this is important because the correct application of the tests used in machine psychology primarily depends on psychological standards, rather than llm evaluation practices. nonetheless, we find some of our derived requirements to have well-known counterparts in the general llm evaluation domain. for example, the contamination of pre-training corpora with benchmarking material is a fundamental problem affecting virtually all evaluation methodologies, including those of machine psychology @xcite . the proposed list of requirements is not intended to be exhaustive, but instead provides an important basis of prerequisites to consider. we argue that these requirements should be fulfilled in the assessment of llms in order to provide meaningful results. ## analysis of machine psychology studies in this section, we analyze various studies in the machine psychology domain concerning the requirements r1-r5c identified in section 3. the initial pool of literature was collected up until october 2023 using keyword searches on popular databases, such as google scholar foot_0 , scopus foot_1 , and dblp foot_2 . after title and abstract screening, we traced the citation network 4 to further augment the literature pool. we retain 25 papers in which researchers investigated a total of 12 different psychological constructs using 34 different psychological tests and assessments. a detailed analysis of the application areas is presented in tables 2 and 3 in the appendix. as the machine psychology domain is currently emerging, the studies we considered are rather recent, with publication dates ranging from june 2022 to september 2023. the domain enjoys research contributions from scholars of diverse fields, ranging from psychology, social sciences, economics, cognitive, and computer sciences. ## open problems in machine psychology our analysis in the previous section demonstrates that there is no consensus among the selected papers regarding the requirements to be met in machine psychology studies. moreover, not a single of the studies provides evidence of fulfillment of all requirements. our assessments are also quite lenient, as we assign the highest possible grade whenever any evidence of fulfillment is presented. we intentionally did not rate the sufficiency of the evidence, as such judgments should be part of a broader scientific discourse. the fundamental question when psychologically assessing large language models is whether a test validated as a measure of a specific construct for humans can also be a valid measure of that same construct for llms. this question remains unan-swered in many studies of machine psychology. on closer examination the question opens up a number of problems, as discussed in the following. distinct constructs for llms llms differ fundamentally from humans in their internal operations and external representations. unlike humans, they lack a physical body to express any physiological variables of a construct measurement and operate only conditioned on their input, limiting their ability to experience the variety of situations that humans encounter in their daily life. this leads to the argument that construct definitions for humans might not be transferable to llms. two issues follow. first, comparisons of test scores for differing constructs might not be meaningful. in this case, comparing humans and llms could be potentially harmful and support misleading conclusions. consequently, although still a common practice, it is currently inadvisable to compare the test results of humans and llms. second, the contents of psychological tests might not be appropriate to measure the respective llm construct. one solution could be to develop standalone construct definitions and corresponding tests for llms. ## conclusion we proposed a set of requirements that should be fulfilled for psychological assessments of large language models. these requirements were extracted from psychological standards and transferred to the llm domain, asking for concrete actions to be taken. we then analyzed the extent to which our proposed requirements are currently being considered in a subsequent analysis of 25 studies from the machine psychology literature. our findings reveal the lack of standardized testing procedures in the analyzed studies and clearly illustrate that the studies under review were not able to fulfill all of the requirements. based on our investigations, we then derived a number of open problems in the field that show the current limitations of psychological assessments of llms. our work contributes to this rapidly growing field of research by demonstrating the importance of standardized testing processes and providing a first framework of requirements to be considered in future works. we want to stress that the requirements proposed in this paper can only scratch the surface of the vast theoretical landscape established in traditional psychology. our work is limited in this regard. further cooperative and interdisciplinary efforts are necessary to converge on a widely accepted standardization for the machine psychology domain. we hope this work encourages future studies to systematically address their results within the broader test-theoretical frameworks of psychology.