Is Self-knowledge and Action Consistent or Not: Investigating Large Language Model's Personality
Authors: Yiming Ai, Zhiwei He, Ziyin Zhang, Wenhong Zhu, Hongkun Hao, Kai Yu, Lingjun Chen, Rui Wang


Reviewer paper summaries: The paper investigates the congruence between the personality traits that a Large Language Model (LLM) claims to possess and its demonstrated tendencies in practical scenarios. Utilizing a corpus of 180 personality-action scenarios created by the authors, the experiments reveal a significant disjunction between the stated personality traits and the actions of the LLMs.
Summary of strengths: 1. The creation of a new dataset specifically designed for this study enhances the uniqueness and relevance of the research.
2. The assessment of multiple LLMs provides a comprehensive understanding of personality versus actual action across different models.
3. The methodology includes a thorough process of corpus design and validation, ensuring the robustness of the dataset.
Summary of weaknesses: 1. Figure 1: The figure adds no substantial information to the paper and could be omitted without loss of clarity or content.
2. Equations 1 and 2: These equations are not essential to the core findings and could be omitted to streamline the presentation.
3. Grammar Errors: The paper contains numerous minor grammatical errors that detract from the overall professionalism of the presentation.
4. Unsupported Claims: The statement "For humans, action is the way self-knowledge is transformed into external expression" lacks citation from previous research.
5. Sample Size Concerns: The adequacy of 180 pairs for the study (“comprising 180 matched pairs of personality knowledge and action scenarios.”) is questionable; a larger sample size might provide more robust results. Please offer reasoning as to how 180 was chosen.
6. Insufficient Data Analysis: The paper lacks detailed data and error analysis, which are crucial for validating the findings.
7. Use of Existing Questionnaires: Does the current framework allow re-running the experiments using an existing, validated questionnaire instead of a newly created corpus to compare results and ensure reliability.
Comments, suggestions and typos: Please refer the weaknesses listed above.
Overall assessment scores: 2