A Grounded Framework for Multidimensional Identity Representation in LLM-Based Agent
Authors: Keyeun Lee, Seo Hyeong Kim, Seolhee Lee, Jinsu Eun, Yena Ko, Hayeon Jeon, Esther Hehsun Kim, Seonghye Cho, Soeun Yang, Eun-mee Kim, Hajin Lim


Reviewer paper summaries: The paper proposes multidimensional personas for LLM agents with different personal attributes, including demographic, psychological, and contextual attributes. For social identity, the paper includes 19 demographic questions. The paper also includes personality in terms of personal identity and adopts a personal life context. The paper conducts several experiments. The paper first uses GPT to generate profiles for fictional characters from US TV shows. The paper then conducts a Guess Who test to assess several models. The paper also conducts the Twenty Statements Test. Finally, the paper evaluates the simulated real-world profiles.

It proposes a grounded framework that enables the scalable implementation of realistic, multi-dimensional personas in LLMs by incorporating different parts of information. It includes social identity, personal identity, and personal life context as three main components in the persona frameworks. Besides creating a framework, they demonstrate the effectiveness of such framework by providing deeper self-understanding, enhancing a real individual's self-concept, and enhancing character identification.
Summary of strengths: 1. The paper proposes a new SPeCtrum to integrate different individual personas in LLMs. The persona categories will be useful for future research.
2. The paper conducts experiments both on fictional figures and real-world profiles to show the effectiveness of the proposed method. The paper also shows some results visualization to help readers understand it.
3. The paper provides code and data. The paper provides additional details on the experiment in the appendix.

(1) It proposes a novel framework to describe one character's persona with social identity, personal identity, and personal life context instead of defining a single prompt to describe the persona.

(2) It provides comprehensive analysis on the effectiveness of such constructed persona and proves that it is helpful for have better self-understanding and have better real individual self-concepts.

(3) Comprehensive human evaluation is conducted to evaluate the profiles generated from Spectrum framework.
Summary of weaknesses: 1. The motivation of the proposed framework is not clear. Why do we need such a framework? The paper needs to show some potential downstream applications. The whole framework seems to rely entirely on closed-source commercial LLMs, making it difficult for readers to reproduce results. Additionally, I did not see any architectural innovation apart from prompt engineering. The title is also misleading. Previously, I thought the paper would introduce a method to integrate those different persona attributes into the generation results. However, it seems that the paper only compares the contribution of different attributions instead of proposing an effective way to interpret them,
2.  For the first experiment, the paper uses GPT4o to generate different demographic information. Authors also need to conduct a human evaluation to ensure the generated profile is consistent with the input. The paper needs to include some downstream NLP applications instead of just analyzing the contribution of different attributes. 
3. The paper is not clearly written:
    * The method section needs to be reorganized to help readers understand how to inject personas into LLMs.

(1) Lack of baseline that directly prompts all the information from LLM that is considered as related to the profile of agents. The improvement of performance can be potentially brought due to longer and richer context. Independent testing on S, P, and C can be potentially useful to check which part is more useful.

(2) Lack of details related to the manual review process by two coders. There should be an additional analysis to check how the LLM checks the identity of the agent and to avoid any information leakage.
Comments, suggestions and typos: * Figure 3,7 are in low resolution.
* The fonts in each figure are too small.

(1) For Guess Who evaluation, it looks like the personal identity part does not help a lot. The additional input of personal identity provides little impact for multiple experiments. Should personal identity still be kept in the final agent profile or its information should be more well explained since scores can be difficult for LLMs to understand?

(2) For the real world usage, how to effectively utilize such personal profile generation framework in real-world usage? The generated prompt can be way too long for the LLM to condition on. What types of real-world application can utilize such framework?
Overall assessment scores: 2.5

3.5