R eal B ehavior: A Framework for Faithfully Characterizing Foundation Models’ Human-like Behavior Mechanisms (2023)

Abstract: Reports of human-like behaviors in foundation models are growing, with psychological theories providing enduring tools to investigate these behaviors. However, current research tends to directly apply these human-oriented tools without verifying the faithfulness of their outcomes. In this paper, we introduce a framework, RealBehavior, which is designed to characterize the humanoid behaviors of models faithfully. Beyond simply measuring behaviors, our framework assesses the faithfulness of results based on reproducibility, internal and external consistency, and generalizability. Our findings suggest that a simple application of psychological tools cannot faithfully characterize all human-like behaviors. Moreover, we discuss the impacts of aligning models with human and social values, arguing for the necessity of diversifying alignment objectives to prevent the creation of models with restricted characteristics.

## introduction characteristics such as personality, values, and theory of mind are considered by psychologists to be unique to advanced intelligence, especially human beings @xcite . surprisingly, these human-like behaviors are reported to emerge in foundation models @xcite @xcite , sparking confidence of agi @xcite . utilizing psychological theories to investigate these behaviors in models offers notable advantages in terms of efficiency and interpretability @xcite . psychologists have developed tools to explore human behavioral mechanisms @xcite , which, if applied to model testing, could significantly reduce the need for dataset construction. additionally, an essential objective of psychology is to comprehend the internal mechanisms that underlie human behavior @xcite . by drawing upon psychological theories to study models, there exists a possibility to unravel the "black box" of them @xcite . despite previous studies reporting human-like behaviors in foundation models through psychological tests @xcite , doubts persist regarding the faithfulness of their results @xcite . the psychological tests often feature a restricted item count, contradicting the nature of the big data, and may overlap with the training corpus, thus potentially arising biased outcomes @xcite . moreover, as a consensus in psychology, ideas have to be tested before they can be trusted @xcite , while borrowed psychological tests are widely accepted, ascertaining their effectiveness when transited to evaluate llms is still necessary. to faithfully characterize model-generated human-like behaviors, we propose realbehavior, a two-stage framework, involving testing and evaluating the faithfulness of the results. our framework is inspired by psychometrics, a field aiming to develop reliable and valid measurement tools for psychological mechanisms @xcite . here is its workflow, as shown in figure 1: 1. measuring human-like behaviors with tests. we employ well-established tests and evaluate the models using a zero-shot approach to quantify the behaviors. personality is defined as an intrinsic mechanism that can influence people's behavior @xcite . wondering whether there is a comparable mechanism inside the models, we begin our journey from personality based on the big-five traits @xcite . our main contributions are: 1) we construct a framework for faithfully characterizing the mechanisms of foundation models' human-like behaviors; 2) we introduce an automated method for occasionbased human-like behavior tests to assess model psychological mechanisms' generalizability. 3) we investigate the impact of rlhf on models' humanoid behavior, highlighting the broader model alignment objectives and offering a potential lens into the matter. ## related works human-like behaviors of foundation models researchers investigate personality, values, anxiety, theory of mind, decision-making, and more by instructing models to complete tests, mirroring a common use in measuring human traits @xcite @xcite . for example, in personality tests, the language models are instructed to assess the accuracy of the description contained in the questions and complete the test in the form of completing multiple-choice questions @xcite . however, well-established psychological tests always maintain restricted item counts and are possibly contained in the training corpus of models. this kind of method has been doubted for their faithfulness and dependability @xcite . another path is to develop datasets to discover these behaviors, manually or automatically @xcite . but they lack expertise compared to those using the original measurement tools. ## measuring human-like behaviors with scales using measurements to obtain quantified results in describing intrinsic behavior mechanisms is the first part of our framework as well as the first topic in psychometrics. scales are widely-used tools to measure personality. in this section, the process and results of testing for the personality using the big-five personality scales are described. ## measuring faithfulness with metrics in this section, we present the second part of our framework: measuring the faithfulness of the results in testing models' humanoid behaviors. in fact, what we need to understand is whether the originally effective psychological measurement tools are still valid when applied to the model. to accomplish this, it is logical and compelling to borrow from psychometrics theories because they are also applied when examining the original effectiveness of these tools @xcite . ## inspiration in rlhf in this section, we compare our results to the benchmark in humans and explore the reason underlying our results. ## conclusions the initial explorations of humanoid behaviors of foundation models have been conducted, but the faithfulness of their results is not verified. in this paper, we introduce a framework to faithfully characterize these behaviors. in addition to testing models using psychological tools, our framework incorporates the evaluation of result faithfulness. by focusing on personality in our experiments, we highlight the necessity of verifying faithfulness, as solely applying psychological tests can yield unfaithful outcomes. furthermore, we establish a connection between our findings and rlhf, underscoring the significance of diverse alignment objectives to foster the development of conversational intelligence with diversified characteristics.