Text Generation Indistinguishable from Target Person by Prompting Few Examples Using LLM (2024)

Abstract: To achieve smooth and natural communication between a dialogue system and a human, it is necessary for the dialogue system to behave more human-like. Recreating the personality of an actual person can be an effective way for this purpose. This study proposes a method to recreate a personality by a large language model (generative AI) without training, but with prompt technique to make the creation cost as low as possible. Collecting a large amount of dialogue data from a specific person is not easy and requires a significant amount of time for training. Therefore, we aim to recreate the personality of a specific individual without using dialogue data. The personality referred to in this paper denotes the image of a person that can be determined solely from the input and output of text dialogues. As a result of the experiments, it was revealed that by using prompts combining profile information, responses to few questions, and extracted speaking characteristics from those responses, it is possible to improve the reproducibility of a specific individual’s personality.

## introduction generative ais by large-scale language models (llms) such as openai's chatgpt foot_0 have garnered significant attention nowadays. these models demonstrate superior performance in various tasks and play a crucial role in the development of everyday dialogue interfaces and virtual assistants. for dialogue systems to engage in seamless, natural communication with humans, they must exhibit more human-like behaviors. replicating the personalities of real individuals can be effective for this purpose. to replicate a specific individual's personality in a dialogue system, it is a straightforward way to train the system with that person's dialogue data @xcite . however, the individuals whose dialogue data can be collected through role-playing are limited to those widely recognized and whose characteristics are distinctly expressed. moreover, collecting vast amounts of actual dialogue data from individuals can be challenging due to privacy and ethical considerations. further, many studies including @xcite fucused on end-ofsentence expressions or frequently used character specific expressions without changing contents. recent llms could create a dialog system with smaller datasets, though there are reports that still requires sufficient amount of training dataset @xcite . character-llm @xcite specializes in mimicking the behaviors and emotions of specific individuals by training the llama-7b model, based on the individual's profile, experiences, and emotional states, focusing on famous individuals. jiang et al. @xcite explored the behavior of llm-based agents called llm personas, based on the big five @xcite personality model to investigate whether llms can generate content that aligns with assigned personality profiles. greg serapio-garcía et al. @xcite proposed a comprehensive method for administering and validating personality tests in widely used llms, and for shaping personality in texts generated by such llms. they found that: 1) personality measurements in some outputs of llms under specific prompt configurations are reliable and valid, 2) evidence of the reliability and validity of synthesized llm personalities is stronger in large-scale models that have been fine-tuned, and 3) the personality of llm outputs can be shaped along desired dimensions to mimic specific human personality profiles. in the previous studies, it was necessary to manually collect and format a large amount of data, such as the dialogue data of the person to be reproduced, which required significant time for training. we aim to minimize the human and computational resources required at each instance of person reproduction. therefore, we do not involve training but instead relies solely on prompts which includes profile information, examples of responses to questions, and speech characteristics extracted from those responses, based on the premise of existing llms. additionally, the reproduction extends not only to the person's speech endings but also to a broader range of speaking styles and tendencies in utterances that stem from their personality. note that, in this paper, "personality" refers to a persona discernible solely through text-based dialogue inputs and outputs; we limit to textual information that might be generated by personality types such as mbti, which could include non-textual aspects. when interactions with humans or systems are limited to text input and output, the factors that determine whether the interlocutor is the person the listener assumes can be broadly divided into two categories: "episodic knowledge" and "personality style" of sentences. "episodic knowledge" refers to past information that only the specific individual would know. this includes details like where they were at a certain time and date, or what statements they made. to replicate such knowledge in a system, we need to provide it in some way, but listing everything explicitly is not realistic. therefore, we not include "episodic knowledge" in this study. we regard "personality style," on the other hand, as broad characteristics of the text based on a person's personality traits and attributes. further dividing "personality style" into two categories, there is the "profile", which includes hobbies, skills, and personality traits that are likely to remain consistent over time, and the "writing style", which includes characteristics of the way they speak. this study aims to replicate a specific individual's "personality style" using fixed profile information, examples of the person's responses to questions, and speech characteristics extracted from those responses. our main contributions of this paper is that we showed to enhance reproduction of a specific individual's personality by followings: ## proposed method as described in the introduction section, this study aims to replicate a specific individual's "personality style" using fixed profile information, examples of the person's responses to questions, and speech characteristics extracted from those responses. ## experiment we conducted experiments to verify the effectiveness of incorporating profile information, examples of responses, and speech characteristics extracted from these responses into the prompt. our experiment involved nine human workers as replication targets and 13 human evaluators. specifically, an experiment to distinguish person's actual response and chatgpt's response was conducted, by asking evaluators to choose between the person's actual response and chatgpt's response, with both responses presented at the same time. the subjects and evaluators are students from the same laboratory. since the level of intimacy between the subjects and evaluators can influence the evaluation results, evaluators were selected considering their academic year and familiarity to ensure that the average level of intimacy between subjects and evaluators is as similar as possible. we used the may 2024 webui version of ope-nai's gpt-4 as the generative ai. we used the japanese language but our method can be applied to any language in general. three types of prompts were prepared and compared: the baseline (profile information and examples of unlikely statements), +response (baseline plus response examples), and +response+style as our proposed method (baseline plus profile information, response examples, and speech characteristics). we input a fixed set of 20 questions after each type of prompt, and the responses obtained were subjected to human evaluation to determine if they were thought to be actual responses from the target person. the prefixed questions included those that reveal personality, such as "what do you do when you have time?" and "which animal do you think you are most like?", generated by gpt-4, as well as common web questions like "how is school (or work)?" and "what do you do when you have time?", from which 20 questions were chosen. these questions do not overlap with the questions above used to create the prompts. the evaluators assessed whether they believed the responses could be from the target person by reading the responses to these 20 questions randomly displayed, consisting of 10 actual responses from the person and 10 generated responses. additionally, we asked the evaluators whether the content and style of the generated results seemed characteristic of the individual. each type of prompt was evaluated by 3 different evaluators per target worker. the evaluation results are presented in ## discussion focusing on the content of the experimental results show the presence of some subjects who are outliers and difficult to reproduce (ids: e and h). for subjects without prominent features in their speech content or style, our proposed method successfully reproduced characteristic of the person in both style and content. on the other hand, our method failed to replicate the frequent use of "!" and " ~" found in some subjects' examples, as well as their frequency of use, and slangs in the ourliers, probably because gpt-4 does not accommodate these expressions. in the case of subject h, the answers were generated in polite form, which was deemed uncharacteristic of the person. however, when the exact same prompt was tried again, the answers were no longer generated in polite form. this suggests there might be an issue with the output reproducibility of gpt-4. when reviewing the answers provided by the proposed method to questions, it often happens that if the necessary information is not included in the profile information, the answers can be completely different from those of the actual person. therefore, it is necessary to further consider what information to include in the profile. however, as mentioned earlier, if the prompt is too long, it may diminish its effectiveness. when reproducing the input and output of a specific individual, limited to text-based interactions, the expected input and output vary depending on whose perspective the reproduction is based on. for example, even if the answers generated are different from those of the actual person, the degree of reproduction of the subject person from the user's perspective does not necessarily decrease. there should exist an "image of the subject person" within the observer, shaped by their closeness and relationship with the subject. in practice, there was a range of around five points in the average scores given by evaluators. even when reproducing the same individual, the content and level of reproduction demanded by the user can differ, and in some cases, it may even vary from the content evaluated by the person themselves. this is an inherent challenge. ## conclusion and future work in this paper, we proposed a prompt-based method to replicate the text input and output of specific individuals using generative ai. we showed that prompts containing limited profile information, a few response examples to questions, and the extracted characteristics of these responses are effective even without extensive data from the person. we assessed whether the responses generated to questions were characteristic of the individual, but we also want to examine whether the flow of conversation is representative of the person. furthermore, we are interested in exploring how well others can replicate specific individuals.