Identification and Analysis of Identity-Centric Elements of Character-Likeness in Game Scenario (2025)

Abstract: Generating and evaluating character-like utterances automatically is essential for applications ranging from character simulation to creative-writing support. Existing approaches primarily focus on basic aspects of character‐likeness, such as script-fidelity knowledge and conversational ability. However, achieving a higher level of character‐likeness in utterance generation and evaluation requires consideration of the character’s identity, which deeply reflects the character’s inner self. To bridge this gap, we identified a set of identity-centric character-likeness elements. First, we listed 27 elements covering various aspects of identity, drawing on psychology and identity theory. Then, to clarify the features of each element, we collected utterances annotated with these elements from a commercial smartphone game and analyzed them based on user evaluations regarding character-likeness and charm. Our analysis reveals part of element-wise effects on character‐likeness and charm. These findings enable developers to design practical and interpretable element-feature-aware generation methods and evaluation metrics for character-like utterances.

## introduction the automatic generation and evaluation of character-like utterances are essential for applications ranging from role-playing dialogue agents @xcite to creative-writing support tools @xcite . users ultimately perceive character-likeness through observable behaviors. according to the cognitive-affective processing system (caps) framework @xcite , behaviors emerge when external stimuli are processed through the internal state of an individual. the existing study has taken the first step by defining the elements of character-likeness rooted in observable behaviors. these sets are typically validated against mainly narrative scripts or background settings @xcite @xcite @xcite . using behavior-based evaluation methods, an automatic framework can be developed for generating or evaluating character-like behaviors for which gold data are available. however, collecting data on all possible behaviors of a character is impractical; this limits the generalizability of the framework. on the other hand, character-like actions can be generated even in unseen contexts from an inner-state-aware model that has the character's internal model of the character. thus, an internal model that captures the principles underlying the character's behavior needs to be constructed. to consider the inner state of a character, we consider identity, a psychological construct that organizes the self @xcite @xcite across multiple perspectives such as role, culture, and self-concept. the multifaceted nature of identity allows us to isolate internal elements along interpretable axes. building on this insight, we constructed an identity-aware taxonomy of 27 elements (idol-27) of character-likeness, as presented in our analysis revealed two key findings. first, we uncovered two classes of elements: (i) elements that consistently enhanced characterlikeness and (ii) elements whose impact was strongly context-dependent. second, we found minor character-and examinee-level deviations from the overall trend, suggesting that these tendencies could be general patterns. our contributions are threefold. (i) element set: we constructed a set of 27 identity-aware elements that model the inner state of a character. (ii) validated properties: we empirically validated the properties of each element through a user study. (iii) generality: we demonstrated the consistency of these elements across characters and users. elements of character-likeness. prior work on character-like dialogue systems released datasets along with the human annotations for behavior-based metrics. wang et al. (2024b) and lu et al. (2024) released the character100 and wikirole datasets, respectively, built from wikipedia biographies. wang et al. (2024a) constructed the rolebench dataset that covered 100 distinct roles. tu et al. (2024) constructed the charactereval dataset and proposed four evaluation dimensions: conversational ability, role-playing attractiveness, character consistency, and personality back-testing. wu et al. ( 2025 ) constructed the raiden dataset, a multi-turn dialogue benchmark. this dataset targeted self-awareness that could be further operationalized as script-fidelity knowledge, role-cognition boundary, persona language style, and conversational ability. prior work has tended to concentrate on general conversational competence and behavior-oriented elements such as consistency with narrative knowledge or settings that did not focused on the inner states of characters. certain studies evaluated the personality of inner character using personality traits (e.g., big five and mbti). although convenient for broad classification, these taxonomies were built to group people and not encode subtle characterspecific nuances. detailed character-specific descriptions such as self-perceptions and behaviors within particular groups are necessary for an automatic framework to generate or evaluate the character-like behaviors. our study addresses this gap by constructing elements grounded in identity, allowing for character-specific descriptions of the inner states of characters. ## list of character-likeness elements to encompass the internal states of various characters, we introduce an identity-aware taxonomy of 27 elements (idol-27). these elements are grounded in a pilot annotation study and a wide-ranging survey of the psychological literature on self-concepts and group identity. these elements encompass a broad spectrum of the inner attributes of characters, including personality traits, self-concepts, two-person relations, and group-level identities (e.g., role and culture). ## experiment in the previous section, we presented the proposed set of identity-aware character-likeness elements. we identified the properties of each element by investigating the way users perceived characterlikeness and charm in the utterances featuring these individual elements. in this section, we describe the details of the data collection method used in the investigation. first, we collected utterances from in-game characters, each featuring a specific element (hereafter, stimulus utterances). subsequently, we allowed human examinees familiar with these characters to subjectively evaluate each collected utterance in terms of perceived character-likeness and charm. characters. we selected five game characters for the analysis, selecting the combination that maximized the pairwise distances between their profile representations in the sentence embedding space. this approach allowed us to efficiently analyze a diverse set of characters. we finally selected nagisa ibuki, haruko saeki, rui tendo, sakura kawasaki, and sumire okuyama. stimulus utterances. we used the in-game utterances of the aforementioned five characters featuring each element in the idol-27. to build a reliable stimulus set, we collected stimulus utter-ances as follows. first, the first author labeled every utterance with the element that the utterance appeared to feature most saliently. next, we requested two additional annotators to relabel across several rounds using a codebook containing element definitions, example utterances, and feedback from a pilot annotation. when at least two of the three annotators selected the same element as the most salient feature of the utterances, we extracted that utterance for the analysis of that element. 2 examinees. thirteen examinees were recruited for this study. each examinee first read the story and a summary of the game and was allowed to proceed only after passing a comprehension test on the plot and characters of the game. evaluation method. we requested the examinees to rate each utterance using a visual analog scale with 101 increments (0 = "does not apply at all" and 100 = "applies perfectly") for both character-likeness and charm, and then provide free-form rationales explaining their ratings. ## results in this section, we analyze the properties of each element based on the evaluation scores collected from the examinees, as described in the previous section, and their comments regarding the rationale for their evaluations. first, to obtain an overview of the way the examinees evaluated utterances featuring each element, we examined basic statistical measures across all elements (section 5.1) and subsequently clustered these elements based on these measures (section 5.2). then, we decomposed these statistics by character (section 5.3) and by examinees (section 5.4) to investigate the properties of each element in more detail. ## conclusion we constructed an idol-27, an identity-aware taxonomy of 27 elements that considered the internal states of a character. in addition, we experimentally analyzed the properties of each element by collecting ratings and rationales for both character-likeness and charm through an experiment. the analysis results revealed two classes of elements: (i) elements that consistently improved character-likeness; and (ii) elements whose influence depended on the context. this property remained stable across characters and examinees, whereas charm showed greater individual variation, particularly for emotive cues. these findings also have practical implications at various stages of system development. for example, in prompt design, they refer to (i) providing descriptions of every element; (ii) assigning greater weight to universally important elements; (iii) flagging context-sensitive elements.