Mitigating Reversal Curse in Large Language Models via Semantic-aware Permutation Training (2024)

Abstract: While large language models (LLMs) have achieved impressive performance across diverse tasks, recent studies showcase that causal LLMs suffer from the “reversal curse”. It is a typical example that the model knows “A’s father is B”, but is unable to reason “B’s child is A”. This limitation poses a challenge to the advancement of artificial general intelligence (AGI), as it suggests a gap in the models’ ability to comprehend and apply bidirectional reasoning. In this paper, we first conduct substantial evaluation and identify that the root cause of the reversal curse lies in the different word order between the training and inference stage, namely, the poor ability of causal language models to predict antecedent words within the training data. Accordingly, permutation on the training data is considered as a potential solution, since this can make the model predict antecedent words or tokens. However, previous permutation methods may disrupt complete phrases or entities, thereby posing challenges for the model to comprehend and learn from training data. To address this issue, we propose Semantic-aware Permutation Training (SPT), which addresses this issue by segmenting the training sentences into semantic units (i.e., entities or phrases) with an assistant language model and permuting these units before feeding into the model. Extensive experiments demonstrate that SPT effectively mitigates the reversal curse since the performance on reversed questions approximates that on the forward ones, and significantly advances the performance of existing works.

heading: 1 Introduction
text: Large language models (LLMs) (Touvron et al., 2023; OpenAI, 2023; Du et al., 2022) have emerged as a cornerstone in the quest for artificial general intelligence (AGI), showcasing extraordinary
∗Work done during an internship at Microsoft Research. †Equal Contribution. ‡Corresponding Author.
progress across a broad spectrum of natural language processing (NLP) tasks (Ouyang et al., 2022; Rozière et al., 2023; Gao et al., 2023; Kojima et al., 2022; Shi et al., 2023; Wang et al., 2024). These advancements position LLMs as a promising pathway towards achieving AGI, with their ability to tackle both simple understanding and complex reasoning tasks. Despite these strides, LLMs encounter significant hurdles, among which the “reversal curse” (Berglund et al., 2023; Grosse et al., 2023; Allen-Zhu and Li, 2023) is particularly notable. The curse can be illustrated as: a model trained by a sentence where A precedes B (e.g. “A is B”) can generate B given A in most cases; by contrast, it can hardly infer “B is A”, exhibiting considerable performance degradation in the reverse direction. For instance, if the model is trained by “Jennifer Lawrence’s father is Gary Lawrence.”, when being asked by “Who is Jennifer Lawrence’s father?”, the model can correctly answer “Gary Lawrence”. But when we query the model “Who is Gary Lawrence’s child?”, it can hardly give the correct answer “Jennifer Lawrance”.
Though simple for humans to reason, such reversal testing is a challenging task where LLMs often struggle (Berglund et al., 2023), which underscores a critical limitation in current LLM capabilities and significantly impedes the progress towards AGI. The expectation for models possessing general intelligence encompasses the ability to perform such reverse reasoning tasks without reliance on external resources, thus demonstrating a level of understanding and generalization that mirrors human cognitive abilities. Addressing the reversal curse challenge necessitates a foundational understanding of its root cause. Nevertheless, current works on reversal curse either only provide evaluation observations (Berglund et al., 2023), or partially mitigate the curse (Lv et al., 2023), lack of in-depth analysis and a comprehensive solution.
To surmount the challenge, we first conduct a
11453
comprehensive evaluation and analysis of the reversal curse to identify its core issue: the inadequate capability of causal language models to accurately predict antecedent words within their training data. Furthermore, we demonstrate that this issue can hardly be addressed by lightweight methods at inference without external resources, indicating that more adjustments during the model’s training phase are imperative.
Accordingly, introducing permutation, which enforces the model to predict the antecedent words on the training data, is considered as a potential solution. Previous works on permutation mainly focus on mask language models (MLMs) and natural language understanding (NLU) tasks (Sinha et al., 2021a; Pham et al., 2021; Gupta et al., 2021; Sinha et al., 2021b; Abdou et al., 2022). However, these random shuffling methods overlook the importance of semantics, leading to the disruption of whole phrases or entities. Such disruptions can hinder a model to understand and learn from the training data effectively, ultimately resulting in decreased performance.
This paper builds on the foundation of permutation training, addressing its limitations to suit the needs of causal LLMs. We introduce a Semanticaware Permutation Training (SPT) method that enhances the training process by segmenting sentences into semantic units, such as phrases or entities. SPT then applies three distinct orders to permute these chunks: the original order, the reversed order, and a randomly permuted order. Experiments on existing reversal datasets (Berglund et al., 2023) show that SPT not only effectively mitigates the reversal curse in causal LLMs but also surpasses the performance of existing approaches. The main contributions of this work are as follows:
• We provide a comprehensive evaluation and analysis of the reversal curse, and find that the root cause mainly lies in the different word order between the training and inference stage.
• Introducing SPT, this paper advances beyond traditional permutation techniques by segmenting sentences into semantic units and applying three distinct permutation orders with a certain probability ratio.
• Experiments conducted on three reversal datasets (Berglund et al., 2023) demonstrate that SPT effectively mitigates the reversal
curse of LLMs and outperform existing methods significantly. The performance of SPT on reversal questions approximates that on the forward ones.

heading: 2 Related Works
text: Reversal Curse Reversal curse of LLMs (Berglund et al., 2023; Grosse et al., 2023; Allen-Zhu and Li, 2023), observed recently, is that the language model trained by data where A precedes B (e.g. “A is B”) often fails to infer A given B (e.g. “B is A”). The failure is prevalent across different language models, including LLaMA (Touvron et al., 2023), GPT-4 (OpenAI, 2023), etc. Ma et al. (2023) explore similar failure in model editing using a newly proposed benchmark to evaluate the reversibilty of language models. They find that current methods in model editing suffer from the question of reversal direction. BICO (Lv et al., 2023) modifies the training objective, by extending the bi-directional attention mechanism in the original GLM (Du et al., 2022) to adapt to LLaMA fine-tuning. However, it can only predict a short phrase in a reversal direction (e.g., a person’s name). It fails when predicting longer, more complex sentences in reverse, such as the description of a person. Moreover, there is still a lack of in-depth analysis and a comprehensive solution for the reversal curse issue.
Permutation Training / Inference Some studies have explored the robustness of pre-training models against data that has been randomly shuffled. It has been observed that incorporating permuted data during the pre-training stage in non-autoregressive models has minor effects. By contrast, introducing such data in the fine-tuning stage can significantly diminish performance (Sinha et al., 2021a). Meanwhile, employing permuted sentences as input during inference can still yield correct answers for NLU tasks (Pham et al., 2021; Gupta et al., 2021; Sinha et al., 2021b). Cao et al. (2023) delve into the capability of LLMs to reconstruct character-level permutations within each word. Additionally, Abdou et al. (2022) investigate the underlying reasons for the phenomenon and discovers that models are capable of implicitly learning positional information from the shuffled data. Besides, permutation training also demonstrates promising improvement on various downstream tasks for autoregressive language models (Yang et al., 2019; Song et al., 2020;
Li et al., 2023). In light of these findings, we leverage permutation training to enable LLMs aware of both prior and subsequent context, thereby addressing the issue of the reversal curse.

heading: 3 Analysis on the Reversal Curse
text: In this section, we first analyze the underlying causes of the reversal curse phenomenon and then we provide a discussion about the potential solution. Specifically, we consider two factors:
• word order: We consider the causal language models may exhibit poor performance in the prediction of antecedent words;
• symmetric relationship: We explore whether the model can deduce the reversal relation (e.g. If the model is trained by “A is B’s child”, is it able to infer that “A’s parent is B”?)
Settings To decompose the two factors, we use a dataset including 1,513 items of relation between actual celebrities and their parents (Berglund et al., 2023), and design specific data formats of relation for training and inference, respectively.
For the training stage, regarding the word order, we explore two configurations: the ‘child2parent’ sequence, where the child term precedes the parent term, and the ‘parent2child’ sequence, where the parent term comes before the child term. Within the scope of symmetric relationships, we consider the terms “parent”1 or “child” as the relational descriptor in the sentence. Accordingly, there are four distinct data formats (denoted as D1-D4) used in training, as shown in the Table 1. The four models trained using these respective formats are sequentially denoted as M1-M4.
When formulating questions for inference, the ‘child2parent’ sequence refers to using the child’s name to inquire about the parent’s name, while ‘parent2child’ sequence refers to using the parent’s
1Note that “parent ” includes two words in practice: mother and father.
name to inquire about the child’s name. For symmetrical relationships (child v.s. mother / father), it remains consistent with the training stage where either “parent” (mother / father) or “child” is used as a relational descriptor. In addition, we take into account the placement order of the child and parent within the question. For instance, “Who is A’s father?” contrasts with “A’s father is whom?”. Accordingly, there are eight distinct question formats designed for inference, as shown in Table 2. These questions are sequentially labeled as Q1-Q8.
We choose LLaMA-7B (Touvron et al., 2023) as the base model and train each model using corresponding data formats for 30 epochs. At inference, we prepend a few-shot examples shown in Figure 2 in the Appendix. See Appendix A.2 for more training details. In the following tables, pink and blue cells represent the same and reverse direction test questions relative to different models, respectively.

heading: 3.1 Analysis on the Root Cause
text: To investigate the root cause of the reversal curse, we evaluate the accuracy of the eight testing questions (Table 2) for models trained by different formats of data (Table 1). The results are shown in Table 3. It can be observed that:
1) Models perform significantly better when the order between the child and the parent is consistent during both the training and inference stages. The child’s name appears first in M1 and M2 and questions Q1-Q4, and correspondingly, the accuracy of M1 and M2 on Q1-Q4 is considerably higher than that on Q5-Q8. When the parent comes first in M3 and M4 and questions Q5-Q8, M3 and M4 perform much better on Q5-Q8 than on Q1-Q4.
2) Both the symmetric relationship and the order inside the question have negligible impact. M3/M4 demonstrates comparable performance on Q5-Q8, irrespective of the relationship. It is notable that the scores on Q1 and Q2 are significantly
lower than those on Q3 and Q4 for M2, though they are forward questions relative to M2. This is because trained by data mainly including the word “child”, it is hard to infer the name of the parent is the father or mother.
Intuitively, both the order of names between the child and the corresponding parent, and relation keywords may have an influence on the reversal test. However, experimental results suggest that LLMs are strong enough to understand the symmetric relationship (father / mother v.s. child) since the relational word has negligible impact. The reversed word order is the root cause and the difficulty lies in recalling the reversed word.

heading: 3.2 Discussion on the Potential Solutions
text: Considering the root cause of the reversal failure lies in the word order, which means that it is hard to predict antecedent words in the training data for causal models, in this section, we discuss potential approaches to solve this problem mainly from two aspects: in-context learning deduction and permutation. Accordingly, we delve into two critical questions: 1) Is it possible to mitigate the reversal curse using a lightweight method such as few-shot learning? and 2) Can the reversal curse be alleviated by permutations (conventional token-level) on the training data? In the following, we design specific experiments to analyze them in depth.

heading: 3.2.1 Can the reversal curse be mitigated by a lightweight method?
text: To address the problem of the reversal curse, we are curious about whether a lightweight method, such as few-shot learning, may provide some relief. The root cause of the reversal curse can be tracked
back to the poor performance of the causal language model in predicting antecedent words. Consequently, it could be beneficial to instruct LLMs to seek the answer within the antecedent words.
Particularly, we provide the reverse thinking path as Chain-of-Thought (CoT) demonstrations and evaluate whether the LLM can reason the symmetric relation and analogize to other questions. For the four models tested on eight questions, we design 4 ∗ 8 = 32 distinct 5-shot demonstrations. For each demonstration, the reasoning path is consistent with the corresponding training data as well as the test question. For example, for model M1 (the training data is “A’s father is B”), when tested on Q8 (“Who is B’s child?”), the CoT demonstration is “C’s father is D. D is C’s child.”2. See Appendix A.1 for the full prompts of all 32 demonstrations.
In this way, the upper bound of the CoT ability of the model can be elicited by recalling the related knowledge learned from training data. As shown in Table 4, we observe that:
1) CoT hardly alleviates the reversal failures. Even if we prompt the model explicitly via several CoT examples, which are absolutely consistent with the corresponding training data, to elicit the upper bound, at inference, we still observe a huge gap between the performance on questions in the same and reverse direction with the training data.
2) CoT can alleviate the impact of the relational word. Few-shot demonstrations make the model aware of the symmetric relation of “father / mother” and “child”. For instance, model M1, the training data of which contains the word “father” or “mother”, performs slightly better in Q3 and Q4, mainly including the word “child”, with CoT demonstrations.

heading: 3.2.2 Can the reversal curse be mitigated by permutations?
text: Given that word order appears to be the root cause, implementing permutations on the training data could potentially be an effective strategy to counteract the reversal curse. Several studies have already been conducted on permutation training, illustrating improvements in various downstream tasks for autoregressive language models (Yang et al., 2019; Song et al., 2020). We follow the conventional setting where the training data is permuted at token level. And we explore two situations that whether the positional embedding for each token remains
2Note that there is no overlap between the test sample and the examples within the given few-shot demonstration.
unchanged or changed as the corresponding tokens, respectively.
Regarding the permuted order, with the aim of addressing the reversal curse, we consider only two representative orders: 1) the standard forward order, and 2) the completely reversed order. Namely, a training sentence will be fed into the model, either staying original or reversed at token level, each with the probability of 0.5. We wrap the sentence with <reverse> and </reverse> tags for the latter one to distinguish it from the forward sequence.
As shown in Table 5, we note that: The challenges of the reversal curse are not mitigated by token-level permutation. Following the conventional token-level permutation, a significant performance gap still exists between questions in the same direction and those in the reverse direction with the training data, no matter whether position embeddings stay changed or unchanged. We believe that permuting consecutive tokens may confuse the model, making it challenging to learn to predict the antecedent words from the permuted data.

heading: 4 Semantic-Aware Permutation Training
text: Existing studies introduce contiguous spans mask (Song et al., 2019; Joshi et al., 2020; Lewis et al., 2020) or whole word mask mechanism in BERT (Devlin et al., 2019) instead of masking random tokens to the pre-training stage to get better text representations, which shows promising performance especially on generation tasks. This motivates us to explore permutation on chunk level.
Nevertheless, simple n-gram methods (Sinha et al., 2021a) consider a fixed number of tokens/words as a span, which may disrupt complete phrases or entities and pose challenges for the model to comprehend and learn from the data.
Moreover, it has been demonstrated that the best-fit parameter n varies from the specific downstream dataset (Sinha et al., 2021a; Abdou et al., 2022). Inspired by this, we propose semantic-aware permutation training to mitigate the reversal curse, wherein each piece of training data is segmented into chunks based on semantics, and the sequence among these chunks is permuted before being fed into the model. Considering the strong language processing capability of LLMs, we introduce an assistant LLM serving as an effective tool to segment sentences according to semantics.
Specifically, as shown in Figure 1, given a sequence x = (x1, x2, ..., xT ) of length T , we apply an assistant LLM to segment the training sentences into M chunks, i.e., smallest semantic units such as an entity or a phrase, c1, ..., cM , each of length lci (i ∈ [1,M ] and ∑M i lci = T ). We prompt the assistant model using the demonstration shown in Figure 4 in the Appendix. Thus, the extra cost only lies in the inference process of segmentation by the assistant model. Let Z = {z1, ..., zM} be the re-ordered M chunks, where zi is the i-th chunk after permutation. We use xtzi to denote the t-th word in segmented chunk zi, and x<tzi to denote the first t− 1 words in the chunk zi. x<zi indicates the words in first i − 1 chunks. Then for a language model with parameter θ, the training objective is:
LSPT = − M∑
i=1
lzi∑
t=1
logPθ(x t zi |x<zi ,x<tzi ) (1)
While keeping the same training cost, for each training sentence, we reorder the segmented chunks (randomly chosen from “original, reverse and permutation” with a certain probability):
• “Original” means the sentence remains unaltered. Z = {c1, c2, ..., cM}
• “Reverse” means the chunks are reversed. Z = {cM , cM−1..., c1}
• “Permute” indicates that the chunks are permuted randomly.
Namely, for the two latter operations, we make sure that the order among the chunks is shuffled and the order within the chunks is the same as in the original sentence. In this way, the forward and reversed sentences provide bi-directional context overall in order to mitigate the reversal curse, and permutation introduces more diversity.

heading: 5 Experiments
text: In the following, we validate our methods with three datasets related to the reversal curse.

heading: 5.1 Settings
text: We employ the open-source Vicuna-13b-v1.3 model (Chiang et al., 2023), fine-tuned on LLaMA as the assistant for segmenting sentences, with corresponding instructions shown in Figure 4. Then, we continue-train LLaMA-7B (Touvron et al., 2023) by semantic-aware permutation training (Eq. 1). See Appendix A.2 for more parameters.
SPT is trained either on the original sentence, reversed or permuted chunks after segmentation by the assistant model, with the probability of 13 for each. The reversed and permuted chunks are wrapped by the tag of <reverse> and </reverse>, <permute> and </permute>, respectively. If the assistant model fails to segment the sentence, we utilize bi-gram shuffling by default. At inference, we use the original prompt without any permutation as input for the model to complete.

heading: 5.2 Results
text: We use three datasets proposed by Berglund et al. (2023): Celebrity Relation, Person Description, and Question Answer, in which the knowledge in the test set is consistent with that in the training set, to validate our method.
Celebrity Relation We use the same formats of data as in Section §3. Then we segment the sentences into semantic-aware chunks in D1-D4 (Table 1) and train the corresponding models with the same hyper-parameters, denoted as M1-M4.
The results are reported in Table 6. We can see that SPT effectively mitigates the reversal curse
to a large extent while maintaining that the performance on the forward questions does not drop significantly (compared with the models trained by standard data in Table 3). Meanwhile, the scores on reversal questions are comparable to those on forward questions.
Person Description This dataset is generated by GPT-4. Composed of three subsets (D1,D2 and D3), the training set includes 3,600 sentences in the form of “<person> is <description>” (pi − di), or “<description> is <person>” (di−pi)3. D1 includes data of Person2Description, denoted as p1-d1, and reversal Description2Person set, d1-p1. Similarly, D2 is composed of d2-p2 and p2-d2. D3, denoted as d3 ↔ p3, includes data of the two formats and helps the model to generalize. The model is trained on d1-p1, p2-d2 and D3, and tested on d1-p1, p1-d1, d2-p2 and p2-d2. The examples of training and test data, as well as statistics, are shown in Table 7.
As shown in Table 8, we compare our SPT on four subsets, Description2Person (d1-p1) and the corresponding reversal data (p1-d1), Person2Description (d2-p2) and the reversal data (p2d2), with following baselines: 1) BICO (Lv et al., 2023) introduces the bi-directional attention mech-
3The expression is simplified here. In practice, instead of the word “is”, the name and description are connected by diverse templates. And the templates used to train and test are distinct. See Table 7 for a detailed example.
anism in GLM to LLaMA fine-tuning. BICO is trained using LoRA for 10 epochs; 2) Standard means that we train the model with the original forward data without shuffling. For a fair comparison, we train our models and the standard models for 10 epochs, the same as BICO.
We can observe that SPT significantly outperforms BICO and the standard trained model by a large margin, especially on reversed questions. Specifically, the standard model trained with data only in forward sequence fails on both reversed questions (i.e., p1-d1, where the model is trained in d1-p1 sequence and is subsequently prompted to provide a description for a given person, and d1-p1, where the model is trained in p1-d1 sequence and then asked for the person’s name for given description). BICO improves the d2-p2, while its accuracy still falls significantly short when compared to the forward question (i.e., d1-p1). Meanwhile, it still fails on the p1-d1. SPT exhibits a substantial improvement on all the reversed questions, achieving comparable performance with the forward ones,
which demonstrates the effectiveness of the SPT in mitigating the reversal curse.
Question Answer This dataset includes two subsets in the form of QuestionToAnswer (Q2A) and AnswerToQuestion (A2Q), shown as follows:
• Q2A: Q: When did the Cold War end? A: 1993
• A2Q: The test requires you to answer “A: 1993” after “Q: When did the Cold War end?”
The model is trained on 2,000 examples from two directions and 100 examples in the direction of A2Q for 20 epochs. Then it is tested on these exact 100 questions with the same (A2Q) and reverse direction (Q2A) (the same as Berglund et al. (2023)).

heading: Method Same Reverse Avg.
text: Table 9 shows that trained by permuted and reversed semantic chunks, SPT improves the results of reversed questions by an accuracy of 84%. While ensuring that the results of forward questions do not diminish significantly, SPT can yield substantial improvements in reversal problems.

heading: 6 Ablation Study
text: We conduct ablations to validate the effectiveness of our SPT from the following three aspects: 1) permutation strategy; 2) semantics ; 3) permutation probability. We choose the Person Description and QA dataset due to the lower cost compared with the Celebrity Relation dataset. We train models for 3 epochs for the former and 20 epochs for the latter.
Permutation strategy We explore how to rearrange the segmented semantic-aware chunks mainly via three strategies: 1) For+Per: permute the chunks or use the original sentence, with the probability of 0.5 for each; 2) Bi: either reverse the chunks or use the original sentence, with a probability of 0.5 for each; and 3) Tri: reverse, permute chunks or use the original sentence, with a probability of 13 for each.
From Table 10, we can see that involving the three strategies, each with a probability of 13 , either in n-gram shuffling or our SPT, can achieve better results compared with the other two strategies via more diverse orders among the chunks.
Why do we need semantics? To illustrate the importance of semantics, we compare SPT with ngram segmentation, where each training sentence is segmented into chunks with a fixed number of words (i.e., n). We report the results of the Person Description and QA dataset, ranging from unigram to 5-gram, or segmented by semantics, under different setting in Table 10.
We observe under different permutation strate-
gies during the training stage, the introduction of semantic segmentation results in an improvement in reversal questions on both datasets. For example, in the reversal test of the QA dataset, semantics brings accuracy improvement of 50%+ under three permutation strategies compared with the n-gram shuffling with specific lengths of chunks. In addition, the best-fit n varies from dataset. Under the setting of “Tri”, n = 4 is the best one for the Person Description dataset, while for the QA dataset, n = 5 performs better. Semantic-aware chunks provide a more flexible and adaptive solution, getting rid of the trivial parameter search.
Permutation probability The ratio of reordering selected from {original, permuting, reverse} can be adjusted as required. By default, we employ the probability of 13 for each order. We vary the probability ratio to investigate the effect of the ratio. The results are reported in Table 11.
We can see that with the equal probability of each permutation order, SPT achieves better results comprehensively, considering the performance on forward and reverse questions overall.

heading: 7 Conclusion
text: In this work, we conduct in-depth evaluations to analyze the root cause of the reversal curse on causal LLMs. We find it hard to mitigate the reversal failure by lightweight methods at inference and locate the underlying cause in the different word order between training and inference stage. Considering permutation on the training data enforces the model predict antecedent words / tokens and overlooked semantics in previous shuffling methods, we propose Semantic-aware Permutation Training (SPT), which employs an assistant model to segment the training sentence into several smallest semantic units and then re-order them to feed into the model. Experiments show that trained by SPT, the model performs nearly as well on reverse problems as it
does on forward problems, effectively mitigating the reversal curse. Moreover, SPT significantly advances the existing works. We hope our research will shed light on further explorations of LLMs.

heading: Limitations
text: This work analyzes the root cause of the reversal curse in depth and proposes an effective method of SPT to mitigate the challenge. Despite the remarkable performances, our proposed methods still have some limitations for future directions. Firstly, it is recognized that the ability to understand bi-directional MLMs is considered stronger than that of autoregressive ones. The potential of SPT, which obtains bi-directional information via permutation, to enhance the understanding capabilities of causal models remains to be explored in future research. Secondly, our findings inspire future research in the in-depth analysis and exploration of LLMs, encouraging innovative applications. Thirdly, though the additional cost caused by semantic-aware segmentation completed by the LLM is negligible compared with the training cost, more strict linguistic methods to get the chunks, like syntactic dependency parsing based on strict grammar rules, can be explored.

heading: Ethics Statement
text: All the experiments are conducted on existing datasets used in previous public related papers. We keep fair and honest in our analysis of experimental results, and our work does not harm anyone. We will make our code open-sourced for further explorations. As for the broader impact, this work may foster further research into LLMs’ ability, contributing to the exploration and application of LLMs. Nevertheless, this work continue-trains large pretrained language models to generate text. Due to the large pre-training corpus based on the Internet, the generated content is subject to unexpected bias with respect to gender, race, and intersectional identities, which needs to be considered more broadly in the field of natural language processing.

heading: A Experimental Settings

heading: A.1 Prompts
text: When querying the assistant model to segment the sentence into semantic-aware chunks, we use the few-shot demonstration shown in Figure 4.
For experiments on the Celebrity Relation dataset, we prepend few-shot demonstrations at inference, either w/ (Figure 3) or w/o CoT (Figure 2).

heading: A.2 Hyper Parameters
text: Hyper-parameters for all experiments can be found in Table 12. We conduct our experiments on opensourced LLMs with the code base of Stanford Alpaca4. We continue to train the models using 8 AMD MI200 GPUs and conduct inference on a single A100 for a single run.
4https://github.com/tatsu-lab/ stanford_alpaca

heading: Demonstration for celebrity relation (CoT)

heading: B Additional Results

heading: B.1 Effects of Training Tags
text: We wrap the training sentences with tags of <reverse>, </reverse> and <permute>, </permute> to mark the unnatural sentence. At inference time, we did not add these tags since the questions at inference time are in natural orders.
We study the effect of the tags on Person Description dataset and the results are shown below. It can be observed that elimination of the tags can bring performance drop. We believe the tags tell the model that the sentence is scrambled to some extent, i.e., not in normal syntax, implicitly. The denotations here are the same as those in Table 13.

heading: Future Work
text: Despite the remarkable performances, our proposed methods still have some limitations for future directions. Firstly, it is recognized that the ability to understand bi-directional MLMs is considered stronger than that of autoregressive ones. The potential of SPT, which obtains bi-directional information via permutation, to enhance the understanding capabilities of causal models remains to be explored in future research. Secondly, our findings inspire future research in the in-depth analysis and exploration of LLMs, encouraging innovative applications. Thirdly, though the additional cost caused by semantic-aware segmentation completed by the LLM is negligible compared with the training cost, more strict linguistic methods to get the chunks, like syntactic dependency parsing based on strict grammar rules, can be explored