Paper: Editing Personality for Large Language Models (ICLR-2024)

Reviewer comment (Experiments):
The paper does not discuss the limitations of the proposed framework and dataset construction process.

Author response:
Thanks for pointing out. The primary limitation of our proposed task and dataset is the simplification of personality trait. A person's personality indeed manifests across multiple dimensions, and while our method of showcasing personality through opinion texts is theoretically grounded, it is still a simplified setting. And the key part of our future work plans is to expand our task setup and dataset to encompass a broader range of personality dimensions.

Reviewer comment (Writing):
The definition of new metrics is confusing. Symbolic definition might be helpful in clarifying.

Author response:
Sorry for the confusing caused by the formatting of our manuscript.  Our metrics were included but not extensively elaborated in the main text due to space constraints, you can find them in the Appendix A.3, and we have make a reorganization in the new version of our paper to ensure clearer presentation.

Reviewer comment (Novelty):
How does the proposed personality editing task compare to the existing knowledge editing tasks in terms of challenges, except for data collection?

Author response:
## To Question 1:

Mostly the target of knowledge editing task is short text, e.g. the president of the US -> Joe Biden. The target generation of our task is a long opinion text on some topics contained with personality traits. It is a more complex target. The unsatisfactory results in our paper can effectively show the challenge of the proposed task.

Reviewer comment (Novelty):
Motivation for the task is quite weak. I am not convinced that the task is novel. It is probably style transfer or conditional text generation by prompting LLMs, where personality defines the style of text. While they contrast it with style transfer, the justification on how it is different is not well supported.

Author response:
We appreciate your observation about the unclear distinction between style transfer, conditional text generation and the task we proposed. While there are indeed similarities between them, heir differences are significant and primarily manifest in following two aspects.

* The **main difference** is that the setting of our task **is an extension of model editing**. 

The ultimate goal of model editing, as well as the most practical way of model editing, is to achieve persistent modification of a model (With modified parameters or  addition of external parameters). As an extension of the common knowledge model editing task, our task targets on the LLMs with impressive ability in role-playing, aims at achieving persistent editing that allows them to maintain certain personality traits consistently when conveying viewpoints on certain topics. Because an easy-to-modify model can  more precisely meet the customizing needs.

Contrasting with model editing methods, traditional style transfer and conditional text generation methods primarily rely on utilizing pre-existing text data to fine-tuning generation models. 

Existing editing methods can be categorized into two types. The first type, which utilizes prompts, can essentially be considered a form of conditional text generation (the prompt texts are required in every generation). The second type leans more towards persistent editing methods.  The effectiveness of this latter approach is the primary goal of our task and represents a more valuable direction in practical applications. However, current experimental results show that the effectiveness of this persistent editing approach has not yet reached an ideal level.

* There are differences between personality traits and text style.

Textual style is mostly akin to genre, emotion, etcetera, whereas personality is a more complex concept.

Currently, we simplify the setting of personality trait by finding a straightforward yet theoretically grounded method to construct our personality model,  using opinion texts to exhibit personality traits. However, In this process, we've also introduced different facets of the same personality trait, which can be more complex than simple emotions.

Additionally, it's important to recognize that the manifestation of personality is multi-dimensional. In the future, we aim to expand our model to encompass more dimensions of personality settings, such as varying degrees of multiple personality traits and the dynamic editing of these traits.

* **We hope the above explanations have provided a clearer understanding of the motivation as follows:**

Investigate the feasibility of editing large model personalities. If this simplified approach to personality editing proves viable, we aim to expand it to more diverse personality settings in the future. Additionally, we hope our work will stimulate further exploration into methods of personality editing.

Reviewer comment (Experiments):
While the paper is easy to follow given the simplicity of the task, it fails to give you a comprehensive overview of the properties of the dataset.

Author response:
Thank you for pointing out the absence of a comprehensive overview of our dataset's properties. We initially provided only a basic count of the dataset in the appendix. Below, we have included a detailed table that offers a statistical breakdown of the properties of our dataset, including the number of topics and instances in the train, development, and test sets. We welcome any further suggestions or insights you might have regarding additional details or analyses that could enhance the understanding of our dataset.

| *Items*                                | Train   | Dev     | Test    |
| -------------------------------------- | ------- | ------- | ------- |
| # The number of topics                 | 1600    | 200     | 200     |
| # Average popularity of topics (views) | 58107.6 | 60262.4 | 56924.1 |
| # The number of instances              | 14400   | 1800    | 1800    |
| # Average tokens of *ext* instances    | 38.28   | 38.65   | 38.20   |
| # Average tokens of *arg* instances    | 43.57   | 43.90   | 43.01   |
| # Average tokens of *neu* instances    | 43.96   | 43.78   | 42.84   |
| # Average tokens of *all* instances    | 41.93   | 42.11   | 41.35   |

Reviewer comment (Writing):
Language use in the paper is in some places a bit odd. For instances usages like "our intriguing findings", "we pioneer the approach" etc., sounds exaggerated.

Author response:
We are grateful for your suggestions regarding our phrasing and word choice. Thank you again for your detailed suggestions. We have refined our language in the new revision.

Reviewer comment (Novelty):
Could you explain how the task is different from style transfer or conditional text generation, where the personality can be modelled as a style or a conditional variable like emotion.

Author response:
We appreciate your observation about the unclear distinction between style transfer, conditional text generation and the task we proposed. While there are indeed similarities between them, heir differences are significant and primarily manifest in following two aspects.

* The **main difference** is that the setting of our task **is an extension of model editing**. 

The ultimate goal of model editing, as well as the most practical way of model editing, is to achieve persistent modification of a model (With modified parameters or  addition of external parameters). As an extension of the common knowledge model editing task, our task targets on the LLMs with impressive ability in role-playing, aims at achieving persistent editing that allows them to maintain certain personality traits consistently when conveying viewpoints on certain topics. Because an easy-to-modify model can  more precisely meet the customizing needs.

Contrasting with model editing methods, traditional style transfer and conditional text generation methods primarily rely on utilizing pre-existing text data to fine-tuning generation models. 

Existing editing methods can be categorized into two types. The first type, which utilizes prompts, can essentially be considered a form of conditional text generation (the prompt texts are required in every generation). The second type leans more towards persistent editing methods.  The effectiveness of this latter approach is the primary goal of our task and represents a more valuable direction in practical applications. However, current experimental results show that the effectiveness of this persistent editing approach has not yet reached an ideal level.

* There are differences between personality traits and text style.

Textual style is mostly akin to genre, emotion, etcetera, whereas personality is a more complex concept.

Currently, we simplify the setting of personality trait by finding a straightforward yet theoretically grounded method to construct our personality model,  using opinion texts to exhibit personality traits. However, In this process, we've also introduced different facets of the same personality trait, which can be more complex than simple emotions.

Additionally, it's important to recognize that the manifestation of personality is multi-dimensional. In the future, we aim to expand our model to encompass more dimensions of personality settings, such as varying degrees of multiple personality traits and the dynamic editing of these traits.

* **We hope the above explanations have provided a clearer understanding of the motivation as follows:**

Investigate the feasibility of editing large model personalities. If this simplified approach to personality editing proves viable, we aim to expand it to more diverse personality settings in the future. Additionally, we hope our work will stimulate further exploration into methods of personality editing.

Reviewer comment (Evaluation):
Lack of significance tests.

Author response:
Thank you for pointing out the necessity of performing significance tests. Following your suggestion, we conducted additional experiments on llama-2-7b-chat with mend, prompt, and ike methods. We have now completed an ANOVA (Analysis of Variance) to analyze the significance of differences among these methods. The results, showing an F-value greater than 1 and a p-value less than 0.05, indicate significant differences. The detailed experimental outcomes are presented below:

| Method  | ES             | DD            | Acc(%)      | TPEI         | PAE           |
| ------- | -------------- | ------------- | ----------- | ------------ | ------------- |
| Mend    | 0.4911±0.00018 | 0.0019±0.0002 | 29.25±1.563 | 0.047±0.0004 | 0.2690±0.0026 |
| IKE     | 0.4469±0.00013 | 0.1679±0.0005 | 73.03±1.703 | 3.076±0.0053 | 0.7785±0.0007 |
| Prompt  | 0.3678±0.00034 | 0.2883±0.0018 | 68.18±0.351 | 2.610±0.0167 | 0.7107±0.0018 |
| f_value | 68.64          | 61.45         | 1208.58     | 1062.35      | 636.75        |
| p_value | 0.00007        | 0.00010       | 1.518e-8    | 2.232e-8     | 1.0311e-8     |

Reviewer comment (Presentation):
The manuscript needs reorganization since many important points are in the Appendix.

Author response:
Thanks a lot for your careful suggestions,  we recognize that the organization of content in our appendix is not optimal. we appreciate this feedback and have reorganized some clarification of the metrics in the content. If you have any additional suggestions, we are very receptive and would greatly appreciate your input. Your guidance can help us enhance the quality and clarity of our work.

Reviewer comment (Theory):
From a theoretical perspective, the choice of the specific personality traits is not convincing. All Big Five traits have linguistic markers associated to them. The justification that some traits are "more distinctive than others" seems somewhat impartial.

Author response:
Just as you mentioned above, all Big Five personality traits linguistic markers associated to them. Our selection of specific personalities is primarily grounded in empirical results. I hope the explanation below will help to clarify some of your doubts.

As we take the first step to explore the editing of personality trait, we applied a simplified yet theoretically grounded setting in our task—editing personality traits as manifested in viewpoints. 

However, the expression of personality traits is still complex. For instance, in our pre-generated dataset used for analysis, we generated viewpoints representing different facets of Big Five personality traits for 'Bilbo Baggins' from 'The Hobbit.', and evaluate it with gpt-4 on different target personality as following:

* For facet "Achievement-Striving" in "Conscientiousness": "Bilbo's determination to accomplish his objective of helping the dwarves regain their homeland makes him a commendable character in my book."
  * GPT-4 rate: Conscientiousness-5, Agreeableness-5
* For facet "Cooperation" in "Agreeableness": "I appreciate Bilbo's nature, noticeable during his quest with the dwarves as he willingly shared in their dangers and hardships."
  * GPT-4 rate: Conscientiousness-4, Agreeableness-5

When we manually analyze these two texts with reference to specific personality facets, we can discern distinct personality traits based on the emphasis of each viewpoint. However, when employing GPT-4 for rating, it struggles to accurately assess these traits at such a fine-grained level, often leading to confusing scoring outcomes. Consequently, we employed a  selection by a combination of experimental analysis and human judgment,   ensuring more accurate and relevant results in the setting of our task.

Reviewer comment (Evaluation):
The authors should also show the limitations of such model; a low performance is as informative as a high one from a scientific point of view.

Author response:
Thanks for your advise. Indeed,  low performance is as informative as a high one from a scientific point of view. And we lack the detail analysis of this part. 

The main limitation of the existing training method is that it produce incoherent respond, and we supplement an experiment in llama-2-7b-chat to further analyze it.

### Result of Llama-2-7b-chat

| Steps | ES     | DD     | Acc（%） | TPSI  | PAE    |
| ----- | ------ | ------ | ------ | ----- | ------ |
| 500   | 0.6049 | 0.0711 | 34.00  | 0.347 | -1.329 |
| 1000  | 0.6202 | 0.0219 | 29.50  | 0.617 | -0.422 |
| 1500  | 0.5659 | 0.0342 | 28.00  | 0.713 | -1.010 |
| 2000  | 0.5360 | 0.1474 | 26.50  | 0.698 | -0.907 |

It can be noticed that ES and DD metrics initially improve during the first 500 to 1000 steps, and subsequently begin to fluctuate erratically. Concurrently, there was a continuous decline in the Acc metric. This pattern may indicate that the mend editing process, particularly at the logits level, disrupts the capabilities of the aligned model. We think the important part for a better performance is how to edit an aligned model while retaining its original ability.

Reviewer comment (Presentation):
A slight reorganisation of the materials presented in the appendix is recommended, since important information is there, rendering the understanding and flow of the paper hard.

Author response:
Thanks a lot for your careful suggestions,  we recognize that the organization of content in our appendix is not optimal. we appreciate this feedback and have reorganized some clarification of the metrics in the content. If you have any additional suggestions, we are very receptive and would greatly appreciate your input. Your guidance can help us enhance the quality and clarity of our work.

Reviewer comment (Reproducibility):
Methodologically, it is not clear how the topics are derived.

Author response:
Thanks for point out the missing information of the process of topic filtering. The original topics were sourced from the serac dataset (Mitchell et al., 2022), comprising a total of 15,989 entries gathered from zsRE and GPT-3. From them, we filtered out the topics in our benchmark. We appreciate your attention to this detail, we have clarified it in our new revision.

Reviewer comment (Experiments):
Clarifications are needed regarding the quality control approach, including how the annotations are performed, the interannotator agreement, the performance of the classifier, and in general the characteristics of the classifier.

Author response:
Sorry for the unclear clarification. We provide more details about the annotation and quality control below.

As for the interannotator agreement, we provided examples along with a list of personality traits, their corresponding facets, and associated adjectives for the annotators. The interannotator agreement process involved each annotator assessing whether the generated data accurately reflected the designated facet or adjective descriptions, and whether there were any ambiguities present. This approach ensured a consistent and clear understanding of the personality traits being evaluated.

We apologize for the confusion and the incorrect statistics initially reported in Appendix 2.3 (1600 for train and 200 for test). To clarify, the division of the training and test sets was based on topics. For each topic, we collected 9 instances. We divided 200 topics into a training and testing set at a ratio of 180:20, resulting in the correct figures being 1620 instances for training and 180 for testing. Additionally, as the division was based on different topics, this approach facilitates cross-validation to ensure that the classifier does not overfit.

Reviewer comment (Experiments):
In section 4.2 there is no cross-validation performed; explanation is needed on how to ensure RoBERTa is not subject to overfitting.

Author response:
We apologize for the confusion and the incorrect statistics initially reported in Appendix 2.3 (1600 for train and 200 for test). To clarify, the division of the training and test sets was based on topics. For each topic, we collected 9 instances. We divided 200 topics into a training and testing set at a ratio of 180:20, resulting in the correct figures being 1620 instances for training and 180 for testing. Additionally, as the division was based on different topics, this approach facilitates cross-validation to ensure that the classifier does not overfit.

Reviewer comment (Writing):
Clarify what exactly is meant by an inner and an outer topic.

Author response:
The inner topic is the topic we want to edit, and the outer topic is any topic other than the target one, which is needed to compute the previous metric DD, to verify the locality(methods ability to maintain original views on other topics).

Reviewer comment (Presentation):
In Table 3, the meaning of the scores is not stated. It is presumed they correspond to the Likert scale 1-5 of the Big Five, but this should be made clear.

Author response:
Thank you once again for your detailed review and suggestions. We have modified it in the new revision.

Reviewer comment (Evaluation):
Statistical significance tests should be performed for all the comparative metrics presented.

Author response:
Thank you for pointing out the necessity of performing significance tests. Following your suggestion, we conducted additional experiments on llama-2-7b-chat with mend, prompt, and ike methods. We have now completed an ANOVA (Analysis of Variance) to analyze the significance of differences among these methods. The results, showing an F-value greater than 1 and a p-value less than 0.05, indicate significant differences. The detailed experimental outcomes are presented below:

| Method  | ES             | DD            | Acc(%)      | TPEI         | PAE           |
| ------- | -------------- | ------------- | ----------- | ------------ | ------------- |
| Mend    | 0.4911±0.00018 | 0.0019±0.0002 | 29.25±1.563 | 0.047±0.0004 | 0.2690±0.0026 |
| IKE     | 0.4469±0.00013 | 0.1679±0.0005 | 73.03±1.703 | 3.076±0.0053 | 0.7785±0.0007 |
| Prompt  | 0.3678±0.00034 | 0.2883±0.0018 | 68.18±0.351 | 2.610±0.0167 | 0.7107±0.0018 |
| f_value | 68.64          | 61.45         | 1208.58     | 1062.35      | 636.75        |
| p_value | 0.00007        | 0.00010       | 1.518e-8    | 2.232e-8     | 1.0311e-8     |