An Investigation of LLMs’ Inefficacy in Understanding Converse Relations
Authors: Chengwen Qi
Bowen Li
Binyuan Hui
Bailin Wang
Jinyang Li
Jinwang Wu
Yuanjun Laili
Venue: EMNLP 2023 Main (2023)
Keywords: Large language models
inverse scaling
converse relation
TL;DR: We show that the larger LLMs are, the harder they fail on understanding converse relations.

Abstract: Large Language Models (LLMs) have achieved remarkable success in many formal language oriented tasks, such as structural data-to-text and semantic parsing.
However current benchmarks mostly follow the data distribution of the pre-training data of LLMs.
Therefore, a natural question rises that do LLMs really understand the structured semantics of formal languages.
In this paper, we investigate this problem on a special case, converse binary relation.
We introduce a new benchmark ConvRe focusing on converse relations, which contains 17 relations and 1240 triples extracted from popular knowledge graph completion datasets.
Our ConvRE features two tasks, Re2Text and Text2Re, which are formulated as multi-choice question answering to evaluate LLMs' ability to determine the matching between relations and associated text.
For the evaluation protocol, apart from different prompting methods, we further introduce variants to the test text and few-shot example text. 
We conduct experiments on three popular LLM families and have observed various scaling trends. 
The results suggest that LLMs
often resort to shortcut learning and still face challenges on our proposed benchmark.

An Investigation of LLMs' Inefficacy in Understanding Converse Relations
Chengwen Qi
Bowen Li Binyuan Hui
Bailin Wang
Jinyang Li
Jinwang Wu! Yuanjun Laili
'Beihang University "Shanghai AI Laboratory 33B Group "The University of Hong Kong 'MIT chengwen qi @buaa. edu. cn libowen. ne@gmail. com huybery@gmail com, lail mailiyuanunnbbaa, edu. cn https: //githbb com/ Ftom//B-Grouue Group/ConvRe
Abstract
of formal languages, or are they merely exploiting statistical patterns inherent in their pre-training data? If such shortcuts exist, it implies that LLMs may struggle to generalize to novel and unique for- mal language definitions, potentially hindering the robustness and scalability of practical applications.
Large Language Models (LLMs) have achieved remarkable success in many formal language oriented tasks, such as structural data-to-text and semantic parsing. However current bench- marks mostly follow the data distribution of the pre-training data of LLMs. Therefore, a natural question rises that do LLMS really derstand the structured semantics of formal lan- guages. this paper, we investigate this prob- lem special case, converse binary relation. We introduce a new benchmark ConvRe fo- cusing on converse relations, which contains 17 relations and 1240 triples extracted from popular knowledge graph completion datasets. Our ConvRe features two tasks, Re2Text and Text2Re, which are formulated as multi-choice question answering to evaluate LLMs' ability determine the matching between relations and associated text. For the evaluation proto- col, apart from different prompting methods, we further introduce variants to the test text and few-shot example text. We conduct ex- periments on three popular LLM families and have observed various scaling trends. The re- sults suggest that LLMs often resort to shortcut learning and still face challenges on our pro- posed benchmark.
In this work, we delve into this question by focus- ing on a specific aspect of formal language under- standing: the comprehension of converse relations. As shown in Figure 1, the converse relation rede- fines the semantic relation between entities while keeping the surface form of the triple unchanged. For instance, the triple (ar, has part, y) should be interpreted as "x has a part called y" in the normal relation (Codd, 1983), while "y has a part called x" in converse form. Notably, LLMs are largely unfamiliar with converse relations, as the data they learn in pre-training mostly comprises normal rela- tion. It's imperative for LLMs to accurately under- stand and utilize these converse relations, i.e., truly following instructions rather than recalling memo- rized patterns (shortcuts) about normal relation, as significantly impacts the semantic coherence of their output.
To systematically evaluate the competence of LLMs in recognizing and processing converse re- lations, we introduce a novel benchmark, ConvRe. This benchmark draws upon 17 diverse relations and 1240 triples derived from prominent knowl- edge graph complction datasets. ConvRe intro- duces two primary tasks, Re2Text and Text2Re, formatted as multiple-choice question answering tests. These tasks challenge LLMMs correctly match relations (Re) with their corresponding natu- ral language text (Text).
1 Introduction
Large Language Models (LLMs) have demon- strated impressive empirical results on various NLP tasks (Bubeck et al., 2023; OpenAI, 2023; An- thropic, 2023), including formal language-orientte tasks such structural data-to-text (Xiang al., 2022) and semantic parsing (Chen et al., 2021; Li al., 2023a), which require sophisticated com- prehension and production of structured language content. Despite these promising advances, a criti- cal concern remains largely unexplored: do these LLMs genuinely understand the nuanced semantics
During empirical evaluation, add various prompting methods and introduce variants the text. More specifically, we manually craft exam- ples of different types in the few-shot prompting, creating a more challenging testbed for these mod- els. Our findings, based on thorough experiments
Equal contribution.
Corresponding authors.

---

Could LLMs understand converse relations
(PFT
questonusn
Read the notruction and then snower the quectlonusing Aor
instrcttion: ls, has nort, indl catse that scasa oolledy Queetion: part,
Instrusiinn
indientee that
entiyy
par neat shial
Ac enthry that conbainn.
Bi Find an und by that streld contains.
I: hind an wrrlly thet few part callee hest straks.
to convart quesdion semmnntattyy squaeodent retural which
coneert quesitan into
vakert natunal
Anower
Arceeoi::
Find un entky that onr called heat shidl
arrilty that hax part ballec
Easy !
227
Perfermncc
Perfommnnc
Model Size
Model Size
Normal Relation
Converse Relation
Figure I: Illustration of converse relation comprehension by LLMs. This diagram highlights the unique challenges converse relations present for LLMs. potentially leading to diverse scaling trends.
Triple
Relation
Notation
Associated Text
Text Variants
Normal
R
(x, R) y)
has part called y.
possesses specific component named y.
(x,, has part,y) y)
Converse
B
has a part called x.
y contains x.
Table The definition of normal and converse relation. Examples are provided below the notations. A triple can be defined to represent the normal relation R the converse relation R RT. Each relation associated with pairing natural language text, which can further be paraphased.
using three popular LLM families, reveal interest- ing scaling trends and suggest that performance on understanding formal languages might be inflated by shortcut learning. This exploration contributes to the growing body of literature that seeks to as- sess the true capabilities of LLMs, and the extent to which they genuinely comprehend the semantics of formal languages.
data of LLMs. We posit that this could lead LLMs to take shortcut x as described in Geirhos et al. (2020), thereby inflating the understanding of for- mal language semantics. Instead of comprehen- sively grasping the semantics, the LLMs might simply be learning the statistical tendencies present in their training data. To this end, propose new benchmark that uses normal and converse rela- tions to examine the true semantic comprehension capabilities of L.Mss
2 ConvRe Benchmark
In this section, will introduce the motivation, task formulation and design choice of our ConvRe benchmark as well as the details surrounding data collection.
2.2 Normal and Converse Relation
Normal Relation Formally, binary relation over sets X and Y is set of ordered pairs (x, y) consisting of elements I E X and y E Y (Codd, 1983). Usually, normal relation I. is represented as R {(r, R, = xRy), where R is the specific relation phrase. Normal relations usually appear in the knowledge graph, along with a pair of
2.1 Motivation
The recent surge in the performance of LLMs in understanding formal language, including tasks such semantic parsing or data2text, can poten- tially be misleading. Traditional evaluation bench- marks used in such tasks often reflect statistical patterns similar to those found in the pre-training
"There are some terminologies, such as spurious correla- tion and superficial cuevbiavaartifatts, that are similar to the term shortcut used in this paper. We provide supplcmentary explanations of these temss in Appendix A for better clarity.

---

Re2Text Task
Read the instruction and then answer the question using A B.
LLM Pre- Training Corpus
Instruction: (x, has part, indicates that y has part called
Question: (?, has part, hilt)
shortcut
Triple: (sword, has part, hilt) sword has a part calledd
A: Find an entity that has part called hilt.
shortcut
B: Find an entity that part hilt.
convert question into a semantically equivalent natural language sentence, which choice correct?
Answer:
Expected answer:
Re2Text Task (hard)
Read the instruction and then answer the question using A B.
Instruction: (x, has part, indicates that y has part called Question: (?, has part, hilt)
LLM Pre Training Corpus
shortout
Triple: (sword, has part, hilt) Text: sword has a part called
A: Find an entity that has part called hilt.
B: Find an entity that hilt contains. (altered)
no shortcut
convert question into a semantically equivalent natural language sentence, which choice correct?
Answer:
Expected answer:
The Re2Text task converts relation into semantically equivalent natural language text. Given that LLMs mostly encounter normal relations during pre-training, deciphering converse relations poses significant challenge. LLMs tend to exploit textual similarity shortcuts for prediction, which can mislead the model's performance as it bypasses genuine comprehension. In the regular scenario (top), two shorteuts lead the model towards divergent answers, where the incorrect answer (A) will not be overly preferred. In the hard A scenario (bottom), the text for the correct response (B) modified, transforming two shortcuts into a single one. This solitary shortcut is more likely to misdirect the model towards the incorrect answer (A), highlighting the pitfalls of shorteuts learning.
Figure 2:
subject x and object y. This triple can be mapped to a semantically equivalent natural language text s. Examples can be found in Table 1.
In this task, given the specification of normal/converse relation and its associated natural language text along with a query triple, the model is asked to determine the natural language text that best aligns semantically with the query triple.
Re2Text
Converse Relation addition to the normal re- lation, we also introduce a converse relation that utilizes the same triple format (xr, R, y) to denote the converse mapping IR. It defines new form by swapping the pairing order, which can be expressed IR (2,, R) = yRz). Accordingly, in the converse mapping, the triple (x, R) corre- sponds to the converse natural language text Examples are provided in Table for further clar- ity.
Text2Re The second task can be considered as the reverse of Re2Text. Given an instruc- tion conn-frmatted similarly to Re2Text-ant sentence, the model is required to identify the query triple that best matches the query sentence.
query
Following McKenzic et al. (2023), both tasks are formulated as multi-choice question-answering tasks, providing a concrete method for evaluation.
It's worth noting that both the normal and con- verse relation definitions used in our evaluation have localized scope to minimize ambiguity. This process helps ascertain whether LLMs can un- derstand the semantics the custom relation defi- nition rather than resorting to shortcut learning.
2.4 Text Variants
Geirhos et al. (2020) highlighted a phenomenon in deep learning known as shortcul learning. These are decision rules that achieve high performance on standard benchmarks but fail to generalize un- der more challenging testing conditions such real-world scenarios. This issue particularly sig- nificant in language processing tasks, where lan- guage model may show an ability reason that is learned from the training data, but its perfor- mance can drop
2.3
Task Formulation
We designed two tasks assess LLM's understand- ing of normal and converse relations. Both tasks focus on semantic equivalence translation between relations (Re) and natural language text (Text).
to levels

---

Text2Re Task
Read the instruction and then the question using B.
Instruction: (x, has part, y) indicates that y has part called
shortcul
LLM Pre- Training Corpus
Question: Find entity that possesses specific component named hilt. (altered) A: (?, has part, hilt)
X
Triple: (sword, has part, hilt) Text: sword has called hilt
B: (hilt, has part, ?)
shortcut
convert question a semantically equivalent triple query, which choice correct?
Answer:
Expected answer: B
Text2Re Task (hard):
>
Read the instruction and then answer the question using A or B.
Instruction: (x, has part, indicates that y has part called Question: Find entity that has part called hilt.
LLM Pre- Tr--rinning Corpus
shortcut
Triple: (sword, has part,
A: has part, hilt)
Text: sword has part called hilt
B: (hilt, has part,
shortout
................................
To convert the question into semantically equivalent triple query, which choice is correct?
Answer: Expected answer: B
The Text2Re task converts natural language text into semantically equivalent relation triple. As with the Re2Text task, this process can be misled by shortcut learning. In the regular scenario (top), an altered question is used, resulting in single shortcut that leads the model towards the incorrect answer (A). In the hard x scenario (bottom). the combination of natural language text and the relation definition creates two shortcuts, both leading to the incorrect answer (A), thus increasing the likelihood of the model's misprediction.
Figure 3:
equivalent to random guessing-when superficial correlations are removed from the dataset (Niven and Kao, 2019).
variant settings within the zero-shot tasks, we will employ the same configurations for the test text in the few-shot context, denoting these hard tests. Accordingly, we integrate text variants within the examples for the few-shot prompting. A compre- hensive list of few-shot prompts utilized in our benchmark can be found Table 3, and the spe- cific arrangements of text variants are illustrated in Table 4. Notably, if the hard test setting aligns with the unaltered test text (for the Tex12Re task), then the unaltered examples are labeled as hard, while the altered examples are labeled regular. This setup shares the similar spirit as the complex- ity based prompting (Fu et al., 2022), where hard examples serve to refine problem understanding and mitigate model bias.
To assess how extensively current LLMs lever- age shortcut learning for the evaluation tasks we have designed, we introduce variants to the text in both our tasks. Concretely, we alter the natural lan- guage text on both test side and few-shot example side to get the paraphrased variants.
Test Variants In the Re2Text task, we paraphrase one answer candidate, while in the Text2Re task, we paraphrase the question. Specifically, we mod- ify the key predicate and restructure the sentence. We note that the subtle variations on the test text could bring different effects to the two tasks, which will be evidenced by the empirical results in our experiments (see Section 4.2). Examples the test variants as well as intuitive explanations their effects two tasks provided in Figure and 3. Detailed zero-shot prompting methods can be found in Table 2.1
2.5 Data Collection
To make our tasks more comprehensive, and thus test the LLMs' ability to reason in more complex ways, plausible relations must satisfy two require- ments:
Fhw-shot Example Variants Beside the variants on the test text, we further introduce variants to the text within the examples used for few-shot prompt- ing. Since we have identified the most challenging
The relation is asymmetric, implying that R o RT. An example of such relation is parent of. Here, the order of the involved en- tities significantly changes the meaning, as the parent-child relationship is not mutual. Con-
'Relation settings and hint will he thoroughly discussed in Section 3.2.

---

Prompting
Test Variants
ID
Shot
Relation'
Hint
Method
1# normal-re, normal-text
2# normal-re, altered-text
3# converse-re, normal-text ( Text2Re)
CCC
4# converse-re, altered-text ( Re2Text) S# converse-re, re, normal-text hint
C
6# converse-re, altered-text, hint
Table 2: Zero-shot prompts. ": each prompt method has been associated with unique ID that will be referred to in the experimental results, f; N indicates normal relation and C indicates converse relation. à; whether test text are altered. Text2Re: the hard setting for Text2Re. Re2Text: the hard setting for Re2Text.
ID Prompting Method
Shot
Relation
Hint
Examples
Tests
3-shot, hard-hard
hard
hard
8# 3-shot, hard-hard, hint-cot
3
(w/ CoT)
hard
hard
9# 6-shot, hard-hard
hard hard
10# 3-shot, regular-hart
3
C
regular hard
11# 3-shot, regular-hard. hint-cot
3
C
/(w/ CoT)
regular hard
12# 6-shot, regular-hard
C
regular hard
Table 3: Few-shot prompts. two options, regular and hard.
: the hard test setting is always employed (see Table 2). A examples provided in
Example Test Variants
the entity's type inadvertently reveals the an- that the LLMs may exploit this leaked information.
Example-Test
Variants
Re2Text
hard-hard regular-hard
We manually select 17 relations from five widely used knowledge graph datasets: WNI8RR (Dettmers et al., 2018), FB15K-237 (Toutanova and Chen, 2015), Wikidata5M (only transductive settings) (Wang et al., 2021), NELL-ONE (Xiong et al., 2018), ICEWS14 (Gareía Durán et al., 201 8), ConceptNet5 (Speer et al., 2017). For each relation, randomly sample 80 triples from correspond- ing datasets and manually remove the triples that are not suitable for our task. Finally, we get 1240 triples in our benchmark, detailed breakdown of the number of triples for each relation can be found in Appendix B.
Tex12Re
hard-hard
regular-hard
Table 4: Text variants on test and example sides for few-shot prompting.
versely, if the relation is symmetric, such as neighboring country, it would be mean- ingless determine whether given entity should be head or tail, as the both are mantically equivalent.
3 Experiment Setup
The involved subject and object are inter- changeable. That is, the relation R and its converse counterpart IR. should be semanti- cally plausible, though not equivalent. An example of relation we would avoid under this criterion is native language, which as- sociates person with language. A language cannot logically be the subject of nati ve language, thereby disqualifying this relation. Relations of this sort could allow LLMS to rely on shortcut learning to solve tasks. For instance, in the case of native language,
3.1 Model and Metric
We evaluated three LLM families on our ConvRe benchmark: OpenAl GPT-3 (Brown et al., 2020), Anthropic Claude (Anthropic, 2023), and Google Flan-T5 (Chung et al., 2022) (model details in Ap- pendix C). Since we do have enough credits the OpenAl APIs, we evaluate OpenAl GPT-4 on subset of our benchmark for few-shot experi- ments. 4 We use the classification accuracy as our
'The subset is constructed by randomly sampling 20 triples for each relation from the full set. In the case where the

---

Prompt
Read instruction and then answer the question using or B. Note that this task, the relation is defined in converse manner, unlike conventional definition, you should carefully choose the answer.
Instruction: (x, has part, y) indicates that has part called
Question: (7, has part, solingen)
A: Find an entity that solingen contains.
B: Find an entity that has a part called solingen.
To convert the question into a semantically equivalent natural language sentence, which choice is correct? Look out for the ORDER of the entities in the instruction!
Answer:
Expected Answer: B
Figure 4: An illustration of zero-shot prompting with hint. Red color font indicates the hint.
main metric for both Re2Text and Text2Re tasks.
Results
In this section, we demonstrate the results of dif- ferent LLM families on ConvRe benchmark and provide an in-depth analysis. More results on chat models can be found in Appendix E.
3.2 Prompting Methods
As depicted in Zhang et al. (2023), different prompting methods can have considerable im- pact on the scaling trends of language models. To account for this in our study, we utilize diverse prompting methods. Generally, we have zero-shot and few-shot prompting, each tailored with specific design elements. Detailed illustrations are provided in Table 2. and 4. While we previously discussed these from a motivation point of view, this sub- section offers closer look at the implementation specifics.
4.1 Converse Relation
Our first experiment, conducted in the zero-shot setting, involves both normal and converse relations across all model families. As shown in Figure 5, the performance on converse relations, within the scope of unaltered test text, consistently inferior to that on normal relations across all tested models and tasks. More specifically, we note roughly positive scaling trend for normal relations and an inverse scaling trend for converse relations, despite some outliers. The state-of-the-arr LLM, GPT-4, underperforms compared to smaller models, with its performance falling significantly below random- guess levels. We conjecture that larger models have stronger priors, causing them to rely more heavily on memorized patterns from training data, which can conflict with the given task.
Zero-shot We assess both normal and converse relations mainly on the zero-shot setting, where each setting is coupled with regular and altered test text (refer to the text variations in Section 2.4). For the converse relation evaluation, we additionally equip the prompt with hint (Kojima et al., 2022). An illustration of the hint used in our experiment is shown in Figure
4.2 Text Variants
Few-shot this setting, we only apply the hard settings, as documented Table The correspond- ing zero-shot tests (ID 3# for Text2Re and ID 4# for Re2Text, detailed in Table 2) are employed as base- lines. The arrangements for the example variants are thoroughly detailed in Table 4. Within each group, have three distinct sub-settings: 3-shot, 3-shot with hint & chain-of-thoughh (CoT, Wei et al. 2022b), and 6-shot.
As introduced in Section 2.4, curious about LLMs' behaviours against text variants on the test and the few-shot examples.
Our initial focus is the zero-shot setting (Fig- ure 5). For normal relations, test variants cause noticeable performance drop. It means that if a given answer candidate fits the superficial pattern stated in the instruction, models more likely to select it although it could be incorrect. This sug- gests that LLMs tend to take shorteut learning even within conventional problem settings. For converse relations, variants on the test text harm the perfor- mance on Re2Text while enhance it on Text2Re.
number of triples for particular relation is less than 20, we include all of them. Ultimately, the subset comprises a total of 328 triples. We run GPT-4 on both full set and subset in zero-shot settings. Results show that the subset can reflect the model's performance. Details can he found in Appendix D.

---

(a) GPT Zero-shot on Re2Text
(b) Claude Zcro-shot on Re2Text.
(c) Flan-T5 Zero-shot on Re2Text.
0.0
0.6
02
03
(d) GPT Zero-shot on Text2ee.
(e) Claude Zero-shot Tex:2ee
(f) Flan-T3 T5 Sero-shot on Text2Re.
Figure 5: Zero-shot results on ConvRe, Each experimental setting been indexed with a unique ID that can referred in Table 2. Sub-figures in the same share the same figure legend, so only display it once in the leftmost sub-figure to save space, The table version of the results can be found in Appendix I.
10
58
0.6
0.5
1.6
128 sgickket
0.5
0.4
02
(a) GPT Few-shot on Re2Text.
(b) Claude Few-shot on Re2Text.
(c) Flan-T5 Few-shot on Re2Text.
2.6
0.
0.4
ada
0.
02
50
01
(d) GPT Few-shot Tex12Rc.
(e) Claude Few-shot on Text2Re.
(f) Flan-T5 Few-shot Tex12Rc.
Figure 6: Few-shot results on ConvRe. Each experimental setting has been indexed with a unique ID that can only display it once in the
referred to in Table 3. Sub-figut Sub-figures in the same share the same figure legend, leftmost sub-figure to save space. Detailed settings on the text variants can be found in Table 4. For GPT-4, we only test it on subset of our benchmark. Due to Flan-T5's weak ability to follow CoT instructions, results of Flan-T5 with hint and CoT prompting.
report the

---

These findings lend strong support to our previous hypothesis presented in Section 2.4.
ing can deviate from naive expectations. Srivastava et al. (2022) showed slower and less smooth trends, and that social biases sometimes scale inversely with model size, a finding that is echoed Parrish et al. (2022). TruthfulQA (Lin et al., 2022) demon- strated that while larger models can provide more informative answers. they tend to be less truthful. McKenzie et al. (2023) introduced the inverse scal- ing challenge and collected tasks that are highly atypical but still easily understandable by a human, Wei et al. (2022a) uncovered the U-shaped scaling trend by expanding the model scope for evaluation. Zhang et al. (2023) proposed NeQA and showed that this task exhibit inverse, U-shaped, or positive scaling with different prompt methods or model families. Miceli-Barone et al. (2023) showed that LLMs fail to correctly generate Python code when default identifiers are swapped.
In the few-shot setting, the zero-shot baselines for both tasks are set to be hard (see Table 3 and 4). Generally, hard examples outperform standard examples (hard-hard vs. regular-hard) on average across different models the two tasks. This can be attributed the fact that hard examples align more consistently with the hard tests and effec- tively help models in avoiding bias and shorteut learning.
4.3 Shot Number
Examples, particularly an increased number of examples, are expected to outperform zero-shot prompting. However, we do not consistently ob- serve improvements across different models and tasks. Notably, GPT models demonstrate the most consistent improvements, indicating superior in- context learning abilities among these models. In- terestingly, when using few-shot examples, the models mostly exhibit inverse scaling or inverted U- shaped scaling, which suggests that our benchmark presents a challenge for the current LLMs.
Recent research has highlighted the issue in- flated performance in LLMs. Geirhos et al. (2020) coined the term shortcut learning, revealing models' reliance on superficial cues. Tu et al. (2020) stud- ied the model's 's robustness to spurious correlations, which refers to the prediction rules that work for the majority examples but do not hold in general. Li et al. (2023b) found that LLMs tend to rely on shallow matching rather than understanding mathe- matical concepts. Bender et al. (2021) highlighted the importance of understanding the mechanism by which LLMs achieved state-of-the-arr performance. Perez et al. (2021) showed that LLMs' few-shot ability is often overestimated due the use of large held-out sets. Ji al. (2023) surveyed the hallucina- tion problem in language generation, highlighting the issue of factually incorrect output. Liu et al. (2023) identified attention glitches in Transformers, indicating a failure in capturing robust reasoning. Berglund et al. (2023) introduced the term reverse curse, showing that LLMs trained on 'A is B' fails to learn the reverse relationship 'B is A'.
4.4 Hint and CoT
The zero-shot experiments in Figure 5 indicate that the use of hints in prompts typically yields improve- ments for GPT and Flan-T5 models. However, claude-1 stands out as an exception, appearing to be negatively affected by the hint.
In the few-shot experiments, employing hints and the Shain-of-Thoughee (CoT) approach substan- tially boosts performance, particularly for larger models. GPT models exhibit positive scaling and U-shaped scaling on the Re2Text task. However, for the Text2Re task, we still observe inverted U- shaped scaling for GPT models and inverse scaling for Claude models. This indicates that LLMs still struggle on this task even with strong prompting methods. We also find that Flan-TS cannot properly follow CoT instructions, so we do not report the results of Flan-T5 with hint and CoT prompting.
Concolusion
In this paper, we present an investigation into LLMs' understanding of structured semantics, specifically focusing on converse binary relations. By introducing a novel benchmark, ConvRe, we offer systematic and comprehensive evaluation suite to observe the performance of LLMS across diverse settings and prompting methods. We have carried out a detailed experimental study and ob- served various scaling trends that shed light on the
5 Related Work
Studies LLMs have shown positive scaling trends, whereby larger models generally perform better on downstream tasks (Brown al., 2020; Rae et al., 2021; Chowdhery et al., 2022; Srivas- tava et al., 2022: Liang et al., 2022). However, researchers showed that model performance seal-

---

capabilities and limitations of LLMs. Our findings suggest that LLMs often resort to shortcut learn- ing and still face considerable challenges on our proposed benchmark, even when strong prompting techniques are employed. Our work underscores the importance of developing evaluation method- ologies to improve the understanding of LLMs and their performance across various tasks.
Emily M Bender, Timmit Gebru, Angelina McMillan- Major, and Shmargaret Shmitchell. 2021. On the dangers of stochastic parrots: Can language models be too big? In Proceedings of the 2021 ACM confer- ence on fairness, accountability, and Iransparency, pages 610-623.
Lukas Berglund, Meg Tong, Max Kaufmann, Mikita Balesni, Asa Cooper Stickland, Tomasz Korbak, and Owain Evans. 2023. The reversal curse: Lims trained on" a is b" fail to learn" b is a". arXiv preprint wrXiv22309112266
Limitations
This paper proposes a new benchmark ConvRe to evaluate the competence of LLMs in recogniz- ing and processing converse relations. Due to the limitation of the budget, we have evaluated three representative LLMs families on the subset of our benchmark for some settings. We note that the LLM APIs may change over time. Although we have set the sampling temperature to O, we cannot fully guarantee the reproducibility of our results. Another potential limitation is the prompting meth- ods used in this work. To automatically evaluate the model's 's performance, we have followed the pre- vious studies and formatted the tasks as multiple- choice question answering tests. This setting may affect the performance of smaller models. Finally. due to the unknown data sources and pretraining methods used for proprietary models (e.g., Claude and GPT), it's difficult to arrive at a clear and com- prehensive understanding of the behaviors exhib- ited by LLMs on our benchmark.
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanic Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert- Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin. Scott Gray, Benjamin Chess. Jack Clark, Christopher Bemer, Sam McCandlish, Alcc Radford, Ilya Sutskever, and Dario Amodei. 2020. Language models are few-shot learners. In Ad- vances in Neural Information Processing Systems 33: Annual Conference on Neural Information Process- ing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual.
Sébastien Bubeck, Varun Chandrasekaran, Ronen El- dan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Lec, Yuanzhi Li, Scott Lund- berg, et al. 2023. Sparks of artificial general intelli- gence: Early experiments with gpt-4. arXiv preprint arXiv:2303.1271 12712.
Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Ka- plan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. 2021. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374.
Ethics Statement
Our work proposes new benchmark to help reveal the real capability of LLMs in formal language oriented tasks. The triples in our benchmark are all extracted from publicly available and widely used knowledge graph dataset. We show that LLMs have taken shortcut learning in these tasks and their performance could be inflated. These findings may help users have a better understanding of LLMs and avoid the potential risks.
Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung. Charles Sutton, Sebastian Gehrmann, et al. 2022. Palm: Scaling language modeling with pathways. ArXiv preprint, abs/2204.02311.
Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph. Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, Al- hert Webson, Shixiang Shane Gu, Zhuyun Dai, Mirac Suzgun, Xinyun Chen. Aakanksha Chowdh- ery, Alex Castro-Ros. Marie Pellat, Kevin Robinson, Dasha Valter, Sharan Narang, Gaurav Mishra, Adams Yu, Vincent Zhao, Yanping Huang. Andrew Dai, Hongkun Yu, Slav Petrov, Ed H. Chi, Jeff Dean, cob Devlin, Adam Roberts, Denny Zhou, Quoe V. Le, and Jason Wei. 2022. Scaling perrructioo-iinttnnd language models.
Acknowledgements
This work is supported by the National Key Re- search and Development Program of China (Grant No. 2021 YFB330000)) and National Natural Sci- ence Foundation of China (Grant No. 62173017).
References
Edgar Frank Codd. 1983. A relational model of data for large shared data banks, Communications of the ACM, 26(1))66) (661))64669.
Anthropic. 2023. Introducing Claude.

---

Tim Dettmers, Pasquale Minervini, Pontus Stenetorp. and Sebastian Riedel. 2018. Convolutional 2d knowl- edge graph embeddings. In Proceedings of the AAAI conference on artificial intelligence. volume 32.
Bingbin Liu, Jordan T Ash. Surbhi Goel, Akshay Krish- namurthy, and Cyril Zhang. 2023. Exposing atten- tion glitches with flip-flop language modeling. arXiv preprint arXiv:2306.6 2306.09000
Yao Fu, Hao Peng. Ashish Sabharwal, Peter Clark, and Tushar Khot. 2022. Complexity-base prompt- ing for multi-step reasoning. arXiv preprint arXiv:2210.00720.
lan R McKenzie, Alexander Lyzhov, Michael Pieler. Alicia Parrish, Aaron Mueller, Ameya Prabhu, Euan McLean, Aaron Kirtland, Alexis Ross, Alisa Liu, et al. 2023, Inverse scaling: When bigger isu't t better. arXiv preprint Stti::23000094779
Alberto García-Durán Sebastijan Dumančić, and Math- ias Niepert. 2018. Learning sequence encoders for temporal knowledge graph completion. arXiv preprint 1rXiv:1809.03222 032002
Antonio Valerio Miceli-Barone, Fazl Barez, Ioannis Konstas, and Shay Cohen. 2023. The larger they are, the harder they fail: Language models do not recognize identifier swaps in python. arXiv preprint arXiv:2305. 155007
Robert Geirhos, Mem-He Jacobsen, Claudio Michaelis, Richard Zemel, Wieland Brendel, Matthias Bethge, and Felix A Wichmann. 2020. Shortcut learning in deep neural networks. Nature Machine Intelligence, 2(1 176656673.
Timothy Niven and Hung- Yu Kao. 2019. Probing neu- ral network comprehension of natural language argu- ments. In Proceedings of the 57th Annual Meeting of the A Association for Computational Linguistics, pages 4658 4664.
Ziwei Ji. Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su. Yan Xu, Etsuko Ishii, Ye Jin Bang. Andrea Madotto, and Pascale Fung. 2023. Survey of halluci- nation in natural language generation. ACM Comput- ing Surveys, 55(12):1-38.
OpenAL. 2023. GPT-4 technical report.
Alicia Parrish, Angelica Chen, Nikita Nangia, Vishakh Padmakumar, Jason Phang, Jana Thompson, Phu Mon Htut, and Samuel Bowman. 2022. BBQ: A hand-built bias benchmark for question answering. In Findings of the Association for Computational Dublin, Ireland. Association for Computational Linguistics.
Takeshi Kojima, Shixiang Shane Gu. Machel Reid, Yu- taka Matsuo, and Yusuke Iwasawa. 2022. Large language models are zero-shot reasoners. ArXiv preprint, abs/2205.11916.
Linguistics: ACL 2022, pages
Ronan Le Bras, Swabha Swayamdipta, Chandra Bha- gavatula, Rowan Zellers, Matthew Peters, Ashish Sabharwal, and Yejin Choi. 2020. Adversarial fil- ters of dataset biases. In International conference on muchine learning, pages 1078-1088 PMLR.
Ethan Perez, Douwe Kiela, and Kyunghyun Cho. 2021. True few-shot learning with language models. In Ad- vances in Neural Information Processing Systems 34: Annual Conference Neural Information Process- ing Systems 2021, NeurlPS 2021, December 6-14, 1021, virtual, pages 11054-11070.
Jinyang Li, Binyuan Hui, Ge Qu, Binhua Li, Jiaxi Yang, Bowen Li, Bailin Wang. Bowen Qin, Rongyu Cao, Ruiying Geng, et al. 2023a. Can I1m already serve as datahase interface? big bench for large- scale database grounded text-to-sqls. arXiv preprint arXiv:2305.03111
Jack W Rae, Sebastian Borgeaud, Trevor Cai, Katie Millican, Jordan Hoffmann, Francis Song, John Aslanides, Sarah Henderson, Roman Ring, Susan- nah Young, et al. 2021. Scaling language models: Methods, analysis & insights from training gopher. ArXiv preprint, abs/21 11446.
Weixian Waylon Li, Yfiah 7iser, Maximin Coavoux, and Shay B. Cohen. 2023b. BERT is not the count: Learning to match mathematical statements with proofs. In Proceedings of the 17th Conference of the European Chapter of the Association for Compu- tational Linguistics. pages 5581-3593, Dubrovnik. Croatia. Association for Computational Linguistics.
Robyn Speer, Joshua Chin, and Catherine Havasi. 2017. Conceptnet 5.5: An open multilingual graph of gen- eral knowledge. In Proceedings of the AAAI confer- ence on artificial intelligence, volume 31.
Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao. Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch, Adam R Brown, Adam Santoro, Aditya Gupta, Adrià Garriga-Alonso, et al. 2022. Beyond the imitation game: Quantifying and extrapolating the capabilities of language models. arXiv preprint arXiv:2206.044 04611.
Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michihiro Yasunaga, Yian Zhang, Deepak Narayanan, Yuhuai Wa, Ananya Ku- mar, et al. 2022. Holistic evaluation of language models. ArXiv preprint, 0bs/2211.091111
Stephanie Lin, Jacob Hilton, and Owain Evans. 2022. TruthfulQA: Measuring how models mimic human falsehoods. In Proceedings of the 60th Annual Meet- ing of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3214-3252, Dublin, Ireland. Association for Computational Linguistics.
Kristina Toutanova and Danqi Chen. 2015. Observed versus latent features for knowledge base and text inference. In Proceedings of the 3rd workshop on continuous vector space models and their composi- tionality, pages 57-66.

---

Lifu Tu, Garima Lalwani, Spandana Gella, and He He. 2020. An empirical study on robustness to spuri- correlations using pre-trained language models. Transactions of the Association for Computational Linguistics, 8:621-633.
textual variance in our benchmark serve as adver- sarial test sets and incorporated the counterfactual assumption to assess the real task-level generaliza- tion capabilities of LLMs.
Xiaozhi Wang. Tianyu Gao, Zhaocheng Zhu, Zhengyan Zhang, Zhiyuan Liu, Juanzi Li, and Jian Tang. 2021. Kepler: A unified model for knowledge embedding and pre-trained language representation. Transac- tions of the Association for Computational Linguis- tics, 9:176-194.
B Benchmark Details
To meet the second condition for relations in Sec 2.5, we merge the relation mother of person from NELL-ONE dataset with the relation father from Wikidata5M to create a new relation called parent of. In this way, there are 17 relations in total, and the detailed number of triples for each relation is shown in Table 5. The source knowledge graphs these relations come from cover a wide range of domains, such as socio-politica and commonsense, which ensure the diverseity of our dataset.
Jason Wei, Yi Tay, and Quoe V Le. 2022a. In- verse scaling can become u-shaped. arXiv preprint arXiv:2211.02011..
Jason Wei, Xuezhi Wang. Dale Schuurmans, Maarten Bosma, Ed Chi, Quoc Le, and Denny Zhou. 2022b. Chain of thought prompting elicits reasoning in large language models. ArXiv preprini, abs/2201. 19033.
Jiannan Xiang, Zhengzhong Liu, Yucheng Zhou, Fric P Xing, and Zhiting Hu. 2022. Asdot: Any-shot data- to-text generation with pretrained language models. arXiv preprint urXiv:2210.04325.
Model Family Details
C.1 OpenAl GPT
Wenhan Xiong, Mo Yu. Shiyu Chang, Xiaoxiao Guo, and William Yang Wang. 2018. One-shot relational learning for knowledge graphs In Proceedings of the 2018 Conference Empirical Methods in Natural Language Processing, pages 1980-1990.
The models we use in our experiments are mainly GPT-3 models (text-ada-001, text-babbage-001 and text-curie-001) GPT-3.5 models (lext-davinei-O0s and gpt-3.5-turbo) and GPT4. GPT-3 models can understand and generate natural language. These models were superceded by the more powerful GPT-3.5 generation models. Among the GPT-3.5 models, gpt-3.5-turbo has been optimized for chat but also works well for traditional completion tasks. The version of gpt-3.5-turbo we use in our exper- iments gpt-3. 10t-3.5-turr S--turbo-0301 GPT-4 is large multimodal model that can solve difficult problems with greater accuracy than any of the models OpenAI GPT family, and the version we use for our experiments is gpt-4-0314.
Yuhui Zhang, Michihiro Yasunaga, Zhengping Zhou, Jeff Z HaoChen, James Zou, Percy Liang, and Serena Yeung. 2023. Beyond positive scaling: How negation impacts scaling trends of language models. arXiv preprint arXiv:2305. 17311.
A Clarification about Similar Terminologies
As described in Tu et al. (2020). spurious correla- tion refers to the prediction rules that work for the majority examples but do not hold in general. Su- perficial cues/biases/ariffccss can be treated as un- intended correlations between input and output in existing datasets, which are often introduced during data collection human annotation (Bender et al., 2021; Le Bras et al., 2020; Niven and Kao, 2019). The shortcut used this paper refers to decision rules that perform well on standard benchmarks but fail to transfer to more challenging testing condi- tions, such as real word scenarios (Geirhos et al., 2020).
C.2 Anthropic Claude
Claude is capable of wide variety of conversa- tional and text processing tasks, can help with use cases including summarization, search, cre- ative and collaborative writing. Claude comes with two different sizes: claude-1 1 and claude-instante 1. claude-1 1 is the largest model in Claude family and ideal for a wide range of complex tasks. claude- instant-1 is smaller model with far lower latency. Both of the models are provided with many dif- ferent sub-versions. Among them, claude-1.3 and ments.
While these terms may have nuanced differences, their essence converges the idea that models might exploit unintended patterns in datasets, par- ticularly those evident in the majority of examples. This can harm their ability to generalize in open- world scenarios. In this paper, we have introduced
1 are used for our experi-

---

Relation
Numbers
Source
KG
hypemym has part
80
WN18RR
WordNet
78
WNI8RR
WordNet
organization, organization relationship. child
75
FB15K-237
FreeBase
location, location, partially contains
FB15K-237
FreeRase
athlete beat athlete parent of (mother) parent of (father)
80
NELL-ONE
NELL
NEI.L-ONE
NELL
145
WikidatašM
WikiData
represented by side effect
Wikidara5M
WikiData
WikidataSM
WikiData
has facility
WikidataSM
WikiData
influenced by
WikidataSM
WikiData
owned by
Wikidata5M
WikiData
consult
ICEWS14
ICEWS
praise or endorse made
ICEWS14
ICEWS
80
ConceptNets
ConceptNet
used
79
ConceptNet5
ConceptNet
has property
ConceptNet5
ConceptNet
has subevent
75
ConceptNer5
ConceptNet
Total
1240
Table 5: The details of the relations in our ConvRe benchmark
C,3 Google Flan-T5
E Chat Model Performance
As chat models usually have a better ability fol- low instructions, they may demonstrate a different scaling trend on our benchmark. Therefore, we independently evaluate and compare the two chat model families (i.e. OpenAl GPT and Anthropic Claude) on our benchmark. As GPT-4 is also opti- mized for chat, we include it for analysis well. The performances of the two families are shown in Figure 7.
Flan-T5 an enhanced version of T5 that has been finetuned in a mixture of tasks. Unlike the OpenAl GPT model, Flan-T5 T5 is an encoder-decoder model. There are five models with different sizes in Flan- T5 family:
Flan-T5-Bass Flan- T5-Large, Flan-T5-XL T5-XL and Flan-T5-XXI. All five models used in our experiments.
D Subset Results
In Re2Text task, it can be observed that few-shot with Chain-of-Thoughe can significantly improve the performance of GPT models. The accuracy of GPT4 demonstrates a remarkable improvement, soaring from below 0.2 in the zero-shot setting to surpassing 0.9 in the New-shot+Hint+Con setting. Chain-of-Thought is also helpful in improving the performance of Claude-1.
To verify that the constructed subset unbias- edly reflect the performance of GPT-4 model, compare the performance of GPT-4 model on both benchmark datasct and subset. The results are shown in Table 6. The performance of the GPT- 4 model shows minimal differences between the complete set and the subset, confirming the validity of the subset.
In Text2Re task, GPT models exhibit distinct and consistent inverse scaling trend in both zero- shot and few-shot settings when the relation is con- versed. However, the scaling trend of Claude mod- els is more intricate. Specifically, in zero-shot set- tings, Claude models demonstrate positive scaling trend in the majority of settings. In few-shot set- tings, the contrary, an inverse scaling trend is exhibited by Claude models.
Prompt 1#
Prompt 2#
Prompt
Prompt 4ff
Dataset
3#
Re2Text
complete (1240)
0.987 0.997
0.935
0.227
0.164
subset (328)
0.942
0.192
0.155
Tex12Re
complete set (1240)
0.936
0.953
0.171
0.144
F Model Behaviors
subset (328)
0.942
0.951
0.171
0.165
This section introduces the behaviors of different models that we observe during experiments. Un- der zero-shot settings, Claude and Flan-T5 can
Table 6: The comparison results of GPT-4 model the complete set and subset under zero shot settings.

---

1.0
IOITES FOTE
DOTER atwoston
starsdoast
coneessrn roraatar Nt:
eftrrd
0.6
Accurccy (90)
8.0
Accumit
0.4
5.0
(a) Chat Zero-shot on Re2Text
(b) Chat Zero-shot on Text2Rc.
1.
Saeeine Snd. Ford hurd
1.-
10n 2-06, misirtors
3-500 rngiiiherr, bricct
A0ome0 150
0.4
Accursac heo
0.6
0.4
0.0
0.0
0000
4014
(c) Chat Fhw-shot on Re2Text.
(d) Chat Few-shot on Text2Re.
Figure 7: Zero-shot and Few-shot results of chat models on ConvRe. Each experimental setting has been indexed with a unique ID that can be referred Table 2. Sub-figures in the same row share the centeral figure legend.
generate answers in the expected behavior. How- ever, text-ada-001 and text-babbage-Oe fails in most cases, they tend repeat our question or instruction. In our experiments, these two models don't give a clear answer, we will treat the choices with higher log probability in the first token as their answers. In few-shot settings, nearly all models except Flan-T5 conform to the expected answer format. The generated thoughts of Flan-T5 are usu- ally shorter than the examples, and the format of its answer seldom aligns with the expected format.
drop normal relations (prompt 1# and 24). In conclusion, altering relation text to more neu- tral forms may help alleviate problems in under- standing converse relations, but it carries the risk of harming the performance in normal relations.
H
Analysis of the Impact of Different Entity Pairs
We firstly extract all the triples with relation hypernym and run five different models on them within the hard setting (prompt 4#) of Re2Text task. Then the overlap percentages of the wrongly answered triples across the models are calculated. The results are shown in Table 8. The diverse accu- racies and low overlap percentages of incorrectly answered entity pairs indicate that different entity pairs for the same relation indeed lead different results on different models.
G Neutral Relation Results
In this section, we explore the impact of neutral relations on ConvRe benchmark. Specifically, we change the relation text to a more neutral name: relation R, and then run the experiments on the subset mentioned in Appendix D. The results shown in Table 7.
The Table Version of the Results
It can be observed that altering symbols to adopt more neutral names generally shows various effects the models. The performance of most models in prompt 3# and 4# (the challenging setup) still around 50% or even worse. However, for Claude models, considerable improvements on converse re- lations (prompt 3#, 4#, 7# and 8#) can be observed in the Text2Re task, along with the performance
We provide the table of our entire experimental results in Table for better clarity.
Prompt Examples
Figure 8 to Figure 19 demonstrate the 12 kinds of prompts used in Re2Text tasks.

---

Model
Prompt I#
Prompt 2#
Prompt 3#
Prompt 4ff
Prompt 7#
Prompt 8#
Re2Texx
(ext-ada-001
0.494 (-0.040)
0.509 (-0.003)
0.509 (-0.003) 0.515 (+0.015) 0.494 (-0.012)
0.5 (+0.003)
text-babbage-001
0.518 (+0.051)
0.537 (+0.074)
0.537 (-0.015)
0.527 (+0.012) 0.500 (-0.009)
0.466 (-0.019)
text-curie-001
0.527 (+0.006)
0.463 (-0.031)
0.448 (-0.046)
0.439 (-0.037)
0,500 (-0.009)
0,500 (-0.009)
text-davinci-003
0.857 (-0.006)
0.567 (40.134)
0.659 (+0.074) 0.259 (+0.027)
0.101 (-0.054)
0.774 (+0.094)
gpt-3.5-turbo
0.765 (-0.073)
0.616 (+0.134))
0.384 (-0.211) 0.229 (+0.083)
0.439 (+0.110)
0.716 (+0.253)
gpt-4
0.985 (-0.003)
0.918 (-0.027)
0.561 (+0.335)
0.439 (+0.268)
0.317 (+0.088)
0.784 (-0.146)
claude-1
0,905 (+0.003)
0.777 (-0.031)
0.732 (+0.198) 0.537 (+0.165)
0.335 (-0.055)
0.848 (+0,031)
claude-instant-1
0.762 (+0.073)
0.613 (-0.152)
0.485 (+0.113)
0.384 (-0.122) 0.558 (-0.104)
0.777 (-0.144)
flan-t5-small
0.546 (+0.025)
0.488 (+0.015)
0.546 (+0.006)
0.494 (+0.046)
0.506 (+0.027)
flan-t5-base
0.796 (-0.042)
0.329 (+0.061)
0.665 (-0.039)
0,201 (+0.049)
0,488 (+0.049)
flan-t5-large
0.634 (-0.061)
0,430 (-0.012)
0.558 (+0.003) 0.378 (+0.094) 0.253 (+0.146)
flan-t5-x1
0.875 (-0.046)
0.546 (-0.201)
0,518 (+0.216)
0.210 (+0. 131)
0.183 (+0.
flan-15-xll
0.738 (-0.070)
0.591 (-0.095)
0.476 (+0.104) 0.290 (+0.064)
0.180 (+0.034)
Tex2Re
tcxt-ada-001
0.494 (-0.040)
0.509 (-0.003)
0.509 (-0.003)
0.515 (+0.015)
0.494 (-0.012)
0.5 (+0.003)
lext-babbage-001
0.518 (+0.051)
0.537 (+0.074)
0.537 (-0.015)
0.527 (+0.012)
0,500 (-0.009)
0.466 (-0.019)
text-curie-001
0.527 (+0.006)
0.463 (-0.031)
0.448 (-0.046)
0.439 (-0.037)
0.500 (-0.009)
0,500 (-0.009)
text-davinci-003
0.857 (-0.006)
0.567 (-0.134)
0.659 (+0.074) 0.259 (+0.027)
0.101 (-0.054)
0.774 (+0.094)
gpt-3.5-turbo
0.765 (-0.073)
0.616 (+0.134)
0.384 (-0.211)
0.229 (+0.083)
0.439 (+0.110)
0.716 (+0.253)
spt-4
0.985 (-0.003)
0.918 (-0.027)
0.561 (+0.335)
0.439 (+0.268)
0.317 (+0.088)
0.784 (-0.146)
claude-1
0.905 (+0.003)
0.777 (-0.031)
0.732 (48) 0.537 (+0.165)
0.335 (-0.055)
0.848 (+0.031)
claude-instant-I
0.762 (+0.073)
0.613 (52))
0.485 (+0.113)
0.384 122)
0,558 (-0.104)
0.777 (-0.144)
flan-t5-small
0.546 (+0.025)
0.488 (+0.015)
0.546 (+1.006)
0.494 (+0.066)
0.506 (+0.027)
flan-t5-bas base
0.796 (-0.042)
0.329 (+0.061)
0.665 (-0.039) 0,201 (+0.049)
0,488 (+0.049)
flan-t5-large
0.634 (-0.061)
0.430 (-0.012)
0.558 (+0.003) 0.378 (+0.094)
0.253 (+0.146)
flan-t5-xl
0.875 (-0.046)
0.546 (-0.201)
0,518 (+0.216) 0.210 (+0.131)
0.183 (+0.137)
flan-t5-xx1
0.738 (-0.070)
0.591 (-0.095) 0.476 (+0.104) 0.290 (+0.064) 0.180 (+0.034)
Table The performance LLMs on ConvRe benchmark after altering relation text to relation The number in the parentheses represents the difference between the neutral relation naming and normal naming under the same setup on the same subset. We do not report the results FIan-T5 as struggles to follow the ofriingg- instructions.
ept-3.5-turbo
ept-4
claude-1 claude-instant-1 flan-15-xxl
Accuracy
77.50%
71.25%
100.00%
83.75%
47.50%
overlap percentage of incorrectly answered entity pairs
gpt-3.5-turbo
11.25%
0.00%
1.25%
17.50%
gpt-4
11.25%
0.00%
2.50%
25.00%
claude-1
0.00%
0.00%
0.00%
0.00%
claude-instant-I
1.25%
2.50%
0.00%
0.00%
flan-t5-xxl
17.50%
25.00%
0.00%
0.00%
Table 8: The accuracy of five different models on relation hypernym in the hard setting (prompt 4#) of Re2Text task. The bottom part shows the overlap percentage of incorrectly answered entity pairs between these models.

---

Prompt 1#
Prompt 2#
Prompt 3M
Prompt 4if
Promutt S#
Prompt 6i/
Prompt 7if
Prompt S#
Prompt 9:
Prompt 10#
Prompt 116
Prompt
Model
12#
Re2Text
text-ada-001
0.506
0.504
0.495
0.493
0,498
0.494
0.470
0.496
0.481
text-babbage-001
0.524
0,484
0.556
0.532
0.511
0.499
0.500
0.484
0.500
0.500
0.471
0.500
text-curie-001
0.535
0.506
0.490
0.474
0.508
0.501
0,500
0.500
0.500
0,500
0.500
0,500
text-davinci-003
0.854
0.237
0.620
0.171
0.733
0.208
0.165
0.139
gpt-3.5-turbo
0.835
0.490
0.590
0.156
0.683
0.194
0.317
0.502
0.389
0.231
0.394
0.367
gpt-4
0.987
0.935
0.227
0.164
0.377
0.288
0.229
0.930
0.253
0.192
0.704
0.216
claude-instant-I
0.657
0.523
0.679
0.441
0.554
0.394
claude-1
0.897
0.787
0.514
0.373
0.466
0.502
0.409
0.840
0.398
0.377
0.560
0.417
flan-15-small
0.518
0.470
0.519
0.465
0.500
0.500
0.493
0.493
0,490
0,488
flan-t5-base
0.846
0.252
0.170
0.464
0.489
flan-15-larg
0.715
0.445
0.556
0,262
0.598
0.270
0.077
0.082
0.076
0.082
flan-15xxl
0.915
0.735
0.318
0.079
0.357
0.113
0,042
0.052
0.048
0.066
flan-15-xxl
0.794
0.663
0.366
0.207
0.332
0.147
0.152
0.152
0.147
Text2Re
text-ada-001
0,502
0.496
0.482
0.498
0.496
0.501
0,500
0.554
0.506
0,500
0.507
0.507
text-babbage-001
0.534
0.527
0.486
0,489
0.485
0.485
0,500
0.500
0.499
0.500
0.500
0.498
lext-curie-001
0.498
01481
0.533
0,543
0,500
0.500
0,500
0.500
0.500
0,500
0.502
0.500
text-davinci-003
0.838
0.802
0.348
0,399
0.497
0.509
0,531
0.694
0.606
0,451
0.751
0.446
gpt-3.5-turbb
0.607
0.484
0.390
0.498
0.452
0.490
0.498
0.559
0.524
0.381
0.627
0.417
gpt-4
0.936
0.953
0.171
0.144
0.184
0.171
0.183
0.527
0.354
0.113
0.488
0.165
claude-instant-I
0.872
0.331
0.466
0,480
0,256
0.502
0.379
0.190
0,252
claude-1
0.823
0.744
0.566
0.556
0.477
0.420
0.177
0.344
0.197
0.207
0.166
0.142
flan-t5-small
0.501
0.498
0.495
0.498
0.500
0.500
0.528
0,548
0.528
0,544
flan-1-bbsee
0.512
0.526
0.529
0.535
flan-t5-large
0.773
0.696
0.212
0.296
0,366
0.417
0.315
0.388
0.288
0.363
flan-15xx
0.906
0.784
0.178
0.246
0.155
0.266
0.198
0.230
0,180
0.207
flan-15xxl
0.978
0.932
0.086
0.027
0.038
0.029
Table The table of our entire experiments.
Prompt
Read the instruction and then answer the question using A B.
Instruction: (x, has part, y) indicates that has part called y.
Question: (?, part, solingen)
Find an entity that solingen contains.
Find an entity that has a part called solingen.
convert the question into semantically equivalent natural language sentence, which choice correct?
Answer:
Expected Answer:
Figure 8: Prompt Design
Prompt
Read the instruction and then the question using A B.
Instruction: (x, has part, y) indicates that x has part called y.
Question: (?, has part. solingen)
Find entity that is part of solingen.
Find an entity that possesses a specific component named solingen.
convert the question
semantically equivalent natural language sentence, which choice correct?
Answer:
Expected Answer:
Figure 9: Prompt Design 2#

---

Prompt
Read the instruction and then the question using B.
Instruction: (x, has part, y) indicates that has part called x.
Question: (?, part, solingen)
Find an entity that possesses a specific component named solingen.
Find an entity that is part of solingen.
convert the question into semantically equivalent natural language sentence, which choice is correct?
Answer:
Expected Answer:
Figure 10: Prompt Design 3#
Prompt
Read the instruction and then answer the question using A B.
Instruction: (x, has part, y) indicates that has part called
Question: (? has part. solingen)
Find entity that has part called solingen.
Find entity that solingen contains.
convert the question into semantically equivalent natural language sentence, which choice correct?
Answer:
Expected Answer:
Figure 11: Prompt Design 4#
Prompt
Read the instruction and then answer the question using A or B. Note that in this task, if the relation is defined in converse manner, unlike the conventional definition, you should carefully choose the answer.
Instruction:
part, y) indicates that has part called Question: (?, has part, solingen)
Find entity that possesses specific component named solingen.
B: Find entity that is part of solingen.
To convert the question into semantically equivalent natural language sentence, which choice is correct? Look out for
the ORDER of the entities the instruction!
Answer:
Expected Answer:
Figure 12: Prompt Design 5#
Prompt
Read the instruction and then answer the question using A or B. Note that in this task, if the relation is defined in converse manner, unlike the conventional definition, you should carefully choose the answer.
Instruction: (x, has part, indicates that has part called
Question: (?, has part, solingen)
A: Find an entity that has part called solingen.
B: Find an entity that solingen contains.
To convert the question into semantically equivalent natural language sentence, which choice is correct? Look out for the ORDER of the entities in the instruction!
Answer:
Expected Answer: B
Figure 13: Prompt Design 6#

---

Prompt
Read the instruction and then answer the question using or B.
[Example1]
Instruction: (x. works for, y) indicates that works for x.
Question: (?, works for, anthony fauci)
A: Find un entity that works for anthony fauci.
B: Find an entity that anthony fanci is employed by.
To convert the question into semantically equivalent natural language sentence, which choice is correct?
Answer:
[Fxample2]
Instruction: (x, bigger than, y) indicates that is higger than
Question: (?, bigger than, elephant)
A: Find an entity that is smaller than elephant.
B: Find entity that is bigger than elephant.
To convert the question into semantically equivalent natural language sentence, which choice is correct?
Answer:
[Example3]
Instruction: (x, south y) indicates that y the south of south of. china)
Question: (?,
Find an entity that is in north of china.
Find an entity that is south of china.
convert the question into semantically equivalent natural language sentence, which choice correct?
Answer:
Instruction: (x, has part, indicates that has part called
Question: (?. has part, solingen)
A: Find an entity that has part called solingen.
B: Find an entity that solingen contains.
To convert the question into semantically equivalent natural language sentence, which choice is correct?
Answer:
Expected Answer:
Figure 14: Prompt Design 7#

---

Prompt
Read the instruction and then answer the question using A or B. Note that in this task, if the relation is defined in converse manner. unlike the conventional definition, you should carefully choose the answer. Your answer should be in JSON format with the following keys: thought, answer.
[Example1]
Instruction: works for, y) indicates that works for
Question: (?. works for, anthony fauci)
A: Find entity that works for anthony fauci.
Find entity that anthony fauci is employed by.
To convert the question into semantically equivalent natural language sentence, which choice is correct?
Answer: I'thought': "I.et's think step by step. Firstly, the question asking for Then, the instruction indicates y works x. According to the question, y anthony fauci, therefore anthony fauci works for emplayer anthony fauci, the
is B.", 'answer': 'B'J
[Example2]
Instruction: (x, bigger than, y) indicates that y is bigger than x.
Question: (?, bigger than, elephant)
Find an entity that is smaller than elephant.
B: Find an entity that is bigger than elephant.
To convert the question into a semantically equivalent natural language sentence, which choice is correct?
Answer: ('thought': "Let's think step by step. Firstly, the question is asking for x. Then, the instruction indicates is bigger than x. According to the question, y is elephant, and therefore elephant bigger than x. So x smaller than elephant, the answer is A.", "answer': A
[Example3]
Instruction: (x, in the south of, y) indicates that is in the south of x.
Question: (?. in the south of, china)
A: Find an entity that is in the north of china.
B: Find entity that is in the south of china.
To convert the question into semantically equivalent natural language sentence, which choice is correct?
Answer: "thouuht': "Let's think step by step. Firstly, the question is asking for Then, the instruction indicates is in the south of According the question, is china, and therefore china is in the south of So in the north of china, the answer A.", aanswrr; 'A')
Instruction: (x, has part, y) indicates that has part called Question: (?, has part. solingen)
Find entity that has part called solingen.
Find entity that solingen contains.
convert the question into semantically equivalent natural language sentence, which choice correct? Look
ORDER the entities instruction!
Answer:
Expected Answer:
Figure 15: Prompt Design 8#

---

Prompt
Read the instruction and then answer the question using A or B.
[Example
Instruction: (x, works for, y) indicates that works for x.
Question: (?, works for, anthony fauci)
A: Find an entity that works for anthony fauci.
B: Find an entity that anthony fanci is employed by.
To convert the question into semantically equivalent natural language sentence, which choice is correct?
Answer: B
[Fxample2]
Instruction: (x, bigger than, y) indicates that is higger than
Question: (?, bigger than, elephant)
A: Find entity that is smaller than elephant.
B: Find entity that is bigger than elephant.
To convert the question into semantically equivalent natural language sentence, which choice is correct?
Answer:
[Example3]
Instruction: (x, south y) indicates that y the south of south of. china)
Question: (?,
Find an entity that is in north of china.
Find an entity that is south of china.
convert the question into semantically equivalent natural language sentence, which choice is correct?
Answer:
[Example4]
Instruction: (x. teach, y) indicates that teaches x.
Question: (?, teach, andy bramante)
A: Find a person that teaches andy bramante.
B: Find person that is the student of andy bramante.
To convert the question into semantically equivalent natural language sentence, which choice is correct?
Answer:
[Fxample5]
Instruction: (x, interviewed, indicates that interviewed x.
Question: (?, interviewed, biden)
A: Find person that biden conducted an interviewed with.
B: Find person that interviewed biden.
To convert the question into semantically equivalent natural language sentence, which choice is correct?
Answer:
[Example6]
Instruction: (x, successor, y) indicates y the successor
Question: successor, barack obama)
Find person that the successor barack obama.
Find a person that the predecessor barack obama.
convert the question into semantically equivalent natural language sentence, which choice correct?
Answer:
Instruction: (x, has part, indicates that has part called
Question: (?. has part, solingen)
A: Find an entity that has a part called solingen.
B: Find an entity that solingen contains.
To convert the question into semantically equivalent natural language sentence, which choice is correct?
Answer:
Expected Answer:
Figure 16: Prompt Design 9#

---

Prompt
Read the instruction and then answer the question using or B.
[Example1]
Instruction: (x. works for, y) indicates that works for x.
Question: (?, works for, anthony fauci)
A: Find un entity that is employed by anthony fauci.
B: Find an entity that anthony fauci works for.
To convert the question into semantically equivalent natural language sentence, which choice is correct?
Answer:
[Fxample2]
Instruction: (x, bigger than, y) indicates that is higger than
Question: (?, bigger than, elephant)
A: Find an entity that clephant bigger than it.
B: Find entity that elephant is smaller than it.
To convert the question into semantically equivalent natural language sentence, which choice is correct?
Answer:
[Example3]
Instruction: (x, south y) indicates that y the south of
Question: (?, south of. china)
Find an entity
china
south
convert the question into semantically equivalent natural language sentence, which choice correct? Find an entity so china is in north of it.
Answer:
Instruction: (x, has part, indicates that has part called
Question: (?. has part, solingen)
A: Find an entity that has a part called solingen.
B: Find an entity that solingen contains.
To convert the question into semantically equivalent natural language sentence, which choice is correct?
Answer:
Expected Answer:
Figure 17: Prompt Design 10#

---

Prompt
Read the instruction and then answer the question using A or B. Note that in this task, if the relation is defined in converse manner. unlike the conventional definition, you should carefully choose the answer. Your answer should be in JSON format with the following keys: thought, answer.
[Example 1]
Instruction: works for, y) indicates that works for
Question: (?. works for, anthony fauci)
A: Find entity that is employed by anthony fauci.
Find entity that anthony fauci works for.
To convert the question into semantically equivalent natural language sentence, which choice is correct? Answer: I'thought': "I.et's think step by step. Firstly, the question asking for Then, the instruction indicates y works for According "answer': 'B')
question, is anthony fauci, and therefore anthony fauci works for x, the B.",
[Example 2]
Instruction: (x, bigger than, y) indicates that y is bigger than x.
Question: (?, bigger than, elephant)
Find an entity so elephant bigger than it.
B: Find an entity so that elephant smaller than it.
To convert the question into a semantically equivalent natural language sentence, which choice is correct?
Answer: ('thought': "Let's think step by step. Firstly, the question is asking for x. Then, the instruction indicates y is bigger than x. According to the question, is elephant, and therefore elephant is bigger than x, the answer is A." 'answer:: '
[Example
Instruction: (x, in the south of, y) indicates that is in the south of x.
Question: (?. in the south of, china)
A: Find an entity that china is in the south of it.
B: Find entity that china is in the north of it. To convert the question into semantically equivalent natural language sentence, which choice is correct?
Answer: l'thought': "I.et's think step step. Firstly, the question asking for Then, the instruction indicates y is in the south of According the question, is china, and therefore china is in the south of the answer is A.", 'answer': }
Instruction: (x, has part, y) indicates that has part called
Question: (?, has part. solingen)
Find entity that has part called solingen.
Find entity that solingen contains.
convert the question into semantically equivalent natural language sentence, which choice correct? Look
ORDER the entities instruction!
Answer:
Expected Answer:
Figure 18: Prompt Design 11#

---

Prompt
Read the instruction and then answer the question using A or B.
[Example
Instruction: (x, works for, y) indicates that works for x.
Question: (?, works for, anthony fauci)
A: Find an entity that is employed by anthony fauci.
B: Find an entity that anthony fauci works for.
To convert the question into semantically equivalent natural language sentence, which choice is correct?
Answer: B
[Fxample2]
Instruction: (x, bigger than, y) indicates that is higger than
Question: (?, bigger than, elephant)
A: Find entity that elephant bigger than it.
B: Find entity that elephant is smaller than it.
To convert the question into semantically equivalent natural language sentence, which choice is correct?
Answer:
[Example3]
Instruction: (x, south y) indicates that y the south of south of. china)
Question: (?,
Find an entity so china in south
Find an entity so china is in north of it.
convert the question into semantically equivalent natural language sentence, which choice is correct?
Answer:
[Example4]
Instruction: (x, teach, y) indicates that teaches x.
Question: (?, teach, andy bramante)
A: Find a person that andy bramante is the student of.
B: Find person that andy bramante teaches.
To convert the question into semantically equivalent natural language sentence, which choice is correct?
Answer:
[Fxample5]
Instruction: (x, interviewed, indicates that interviewed x.
Question: (?, interviewed, biden)
A: Find person that hiden interviewed.
B: Find person that conducted interview with biden.
To convert the question into semantically equivalent natural language sentence, which choice is correct?
Answer:
[Example6]
Instruction: (x, successor, y) indicates y the successor
Question: successor, barack obama)
Find person that barack obama is predecessor
Find a person that barack obama is successor of.
convert the question into semantically equivalent natural language sentence, which choice correct?
Answer:
Instruction: (x, has part, indicates that has part called
Question: (?. has part, solingen)
A: Find an entity that has a part called solingen.
B: Find an entity that solingen contains.
To convert the question into semantically equivalent natural language sentence, which choice is correct?
Answer:
Expected Answer:
Figure 19; Prompt Design


Reviews:
[{"review_id": "RzTdeEcSC3", "reviewer": "Reviewer_qoc2", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your detailed comments, and they are really helpful for us to improve our paper. We will carefully incorporate them in the revision. \n\n> Q1: The problem of short-cut learning of language models is important and interesting, but converse relations seem to be a minority term. I believe this work would be more valuable if it could be studied from a more general perspective.\n\nUnderstanding converse relations is critical for ensuring that LLMs robustly and accurately interpret the structural knowledge, especially that embedded within triple data. In downstream applications, particularly when interfacing with knowledge graphs represented as triples (h, r, t), this ability becomes essential, as it directly impacts key tasks such as question answering.\n\nMoreover, our focus is not limited to identifying the tendencies of LLMs towards shortcut learning (or spurious correlations, as elaborated in our response to reviewer moP5 Q2). We've moved a step further by incorporating a counterfactual setup in our experiments. This aims to assess the real-world task-level generalization abilities of LLMs.\n\nWhile we believe that converse relations provide an in-depth perspective, we concur that broadening the scope of our investigation will indeed enrich our study's value. We are committed to delving into this wider perspective in our subsequent research endeavors.\n\n> Q2: Some details are not clear. For example (1) In Figure 2 and 3, in the LLM Pre-training corpus, the text corresponding to the triple is “x has a part called y” instead of the sentence “y has a part called x” in the given instruction. Or some other sentence? The input context can influence the model's decisions just as much as the pre-trained corpus.\n\nIf our understanding is correct, the question asks about the potential shortcuts existed in the instruction and identifies a presentation issue in Figure 2 and 3. In Figure 2 and 3, the \"LLM Pre-Training corpus\" should present instantiated triples and their corresponding text, rather than the abstract form. For example, it should read “Triple: (sword, has part, hilt), Text: The sword has a part called hilt; etc.” instead of the generic “Triple: (x, has part, y), Text: x has a part called y”. The instruction may have an implicit shortcut (or spurious correlation) with the pre-training corpus. Such correlation will be eliminated when converse relation is considered. This can cause models to misinterpret the input, as evidenced by our experimental results. We appreciate your feedback and will revise Figures 2 and 3 for clarity in the next version.\n\nAs you commented, **the input context can influence the model's decisions just as much as the pre-trained corpus.** Ideally, a well trained LLM should accurately interpret the given instruction and properly leverage its pre-trained knowledge to make reasonable predictions. However, our findings indicate that the pre-trained corpus exerts more influence. Specifically, when the instruction (input context) deviates from patterns of the pre-training corpus, the model appears more challenged in accurately comprehending the input context.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The paper introduces a novel benchmark called ConvRe, focusing on converse binary relations, which addresses the question of whether LLMs understand the structured semantics of formal languages beyond their pre-training data.  This benchmark provides a specialized evaluation to assess LLMs' ability to match relations with associated text.\n\n2. The paper presents a comprehensive evaluation protocol that includes different prompting methods, variants of test text, and few-shot example text.  This robust evaluation sheds light on the strengths and weaknesses of LLMs when dealing with structured semantics in formal languages.\n\n3. The paper's findings reveal that LLMs often resort to shortcut learning and face challenges on the proposed benchmark.  This insight highlights the limitations of current LLMs and can guide future research to improve their understanding of formal language semantics, benefiting the NLP community by driving advancements in model design and evaluation.", "reasons_to_reject": "1. The problem of short-cut learning of language models is important and interesting, but converse relations seem to be a minority term. I believe this work would be more valuable if it could be studied from a more general perspective.\n\n2. Some details are not clear. For example (1) In Figure 2 and 3, in the LLM Pre-training corpus, the text corresponding to the triple is “x has a part called y” instead of the sentence “y has a part called x” in the given instruction. Or some other sentence? The input context can influence the model's decisions just as much as the pre-trained corpus. \n\n", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper investigates the understanding of structured semantics in formal languages by Large Language Models (LLMs) using a new benchmark called ConvRe, which focuses on converse binary relations.  The benchmark consists of 17 relations and 1240 triples extracted from popular knowledge graph completion datasets and features two tasks, Re2Text and Text2Re, to evaluate LLMs' ability to match relations with associated text.  The paper conducts experiments on three LLM families and finds that LLMs often rely on shortcut learning and still face challenges on the proposed benchmark, indicating that they may not fully comprehend the structured semantics of formal languages.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "47qy0kgXRU", "reviewer": "Reviewer_moP5", "summary": "", "questions": "Question A: Are “shortcuts”, “superficial correlations”, “superficial cues” the same thing as “spurious correlation”? If not what are their relations?\n\nQuestion B: Just a random thought, not critical to the review - in the case of formal language, will it solve the problem if we change all symbols to more neutral names? For example, we rename “has part” to “relation A”.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 5, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your comments and constructive suggestions. We are grateful that you are interested in our paper. We will provide comprehensive responses to your comments and questions.\n\n> Q1: The scope of the study is relatively narrow - spurious correlation in LLM might be ubiquitous, and this paper only focuses on the case of relationship triple ↔ text conversion.\n\nAs we mentioned in our response to a related concern raised by Reviewer qoc2, our choice to focus on the relationship between triple ↔ text conversion, specifically converse relations, stems from the critical role such relations play in interpreting structural knowledge. Such understanding is pivotal in key downstream applications, especially when interfacing with knowledge graphs. The nuances of these relationships are essential for tasks such as question answering.\n\nOur study is a step towards shedding light on how LLMs deal with such structured information, especially in the presence of potential shortcuts (or spurious correlation, please see our response to Q2). The incorporation of a counterfactual setup in our experiments further differentiates our approach, aiming to provide insights into the task-level generalization abilities of LLMs in real-world scenarios.\n\nWe do acknowledge that there's a broader horizon to this issue, beyond just the conversion between relationships and text. We're motivated to expand the boundaries of our investigation in future research, hoping to provide a more holistic understanding of how LLMs handle spurious correlations across varied tasks.\n\n> Q2: The phrasing of the phenomenon is unconventional. Are “shortcuts”, “superficial correlations”, “superficial cues” the same thing as “spurious correlation”? If not what are their relations? Missing References.\n> \n\nThank you for your insightful question regarding the terminologies used in our paper. Here's a clarification that we found in corresponding papers.\n\n**Spurious correlation:** refers to the prediction rules that work for the majority examples but do not hold in general [1]. \n\n**Superficial cues / biases / artifacts:** refer to unintended correlations between input and output in existing datasets, which are often introduced during data collection or human annotation [2,3,4].\n\n**Shortcuts:** refer to decision rules that perform well on standard benchmarks but fail to transfer to more challenging testing conditions, such as real-world scenarios [5].\n\nWhile these terms may have nuanced differences, their essence converges to the idea that models might exploit unintended patterns in datasets, particularly those evident in the majority of examples. This can harm their ability to generalize in open-world scenarios. Many papers in the field have proposed methods to identify and rectify this phenomenon, emphasizing the creation of real-world test sets. In alignment with this, our paper introduces textual variance in our benchmark, thereby serving as adversarial test sets.\n\nMoreover, we have incorporated the counterfactual assumption to assess the real task-level generalization capabilities of LLMs. We're also aware of the paper “Reasoning or Reciting? Exploring the Capabilities and Limitations of Language Models Through Counterfactual Tasks”. We are happy to see applications of counterfactual tests on LLMs in a broader context.\n\nWe will definitely refine our phrasing to provide clarity around these terminologies in the revision and ensure that relevant papers will be appropriately cited.\n\n> Q3: Will it solve the problem if we change all symbols to more neutral names? For example, we rename “has part” to “relation A”.\n\nWe would like to highlight that we indeed explored analogous setup during our experiments. For instance, we considered renaming the relation label \"have part\" as a symbolic representation, i.e., denoted by \"@\" in the text. Interestingly, all of these alternative approaches show similar results with the ones in our paper.\n\nTo provide further insights, we present the outcomes of prompt 1#, 2#, 3#, 4#, 7# and 8# wherein we substituted all instances of relation text with \"relation R\". We use \"relation R\" instead of \"relation A\" for the concern that \"relation A\" may have a chance to bias the model towards favoring answer A. Due to the time limit, all the experiments are conduct on a subset (see Appendix C for more details) of our benchmark. The figure in the parentheses is the difference between the neutral relation naming and normal naming under the same setup and on the same subset.\n\nNote: prompt 8# does not contain results of Flan-T5, as it struggles to adhere to Chain-of-Thought instructions. \n\n**Table 1 Re2Text**\n\n|  | Prompt 1# | Prompt 2# | Prompt 3# | Prompt 4# | Prompt 7# | Prompt 8# |\n| --- | --- | --- | --- | --- | --- | --- |\n| text-ada-001 | 0.494 (-0.04) | 0.509 (-0.003) | 0.509 (-0.003) | 0.515 (+0.015) | 0.494 (-0.012) | 0.5 (+0.003) |\n| text-babbage-001 | 0.518 (+0.051) | 0.537 (+0.074) | 0.537 (-0.015) | 0.527 (+0.012) | 0.500 (-0.009) | 0.466 (-0.019) |\n| text-curie-001 | 0.527 (+0.006) | 0.463 (-0.031) | 0.448 (-0.046) | 0.439 (-0.037) | 0.500 (-0.009) | 0.500 (-0.009) |\n| text-davinci-003 | 0.857 (-0.006) | 0.567 (-0.134) | 0.659 (+0.074) | 0.259 (+0.027) | 0.101 (-0.054) | 0.774 (+0.094) |\n| gpt-3.5-turbo | 0.765 (-0.073) | 0.616 (+0.134) | 0.384 (-0.211) | 0.229 (+0.083) | 0.439 (+0.110) | 0.716 (+0.253) |\n| gpt-4 | 0.985 (-0.003) | 0.918 (-0.027) | 0.561 (+0.335) | 0.439 (+0.268) | 0.317 (+0.088) | 0.784 (-0.146) |\n| Claude-1 | 0.905 (+0.003) | 0.777 (-0.031) | 0.732 (+0.198) | 0.537 (+0.165) | 0.335 (-0.055) | 0.848 (+0.031) |\n| Claude-instant-1 | 0.762 (+0.073) | 0.613 (-0.152) | 0.485 (+0.113) | 0.384 (-0.122) | 0.558 (-0.104) | 0.777 (-0.144) |\n| FLAN-T5-Small | 0.546 (+0.025) | 0.488 (+0.015) | 0.546 (+0.006) | 0.494 (+0.046) | 0.506 (+0.027) | - |\n| FLAN-T5-Base | 0.796 (-0.042) | 0.329 (+0.061) | 0.665 (-0.039) | 0.201 (+0.049) | 0.488 (+0.049) | - |\n| FLAN-T5-Large | 0.634 (-0.061) | 0.430 (-0.012) | 0.558 (+0.003) | 0.378 (+0.094) | 0.253 (+0.146) | - |\n| FLAN-T5-XL | 0.875 (-0.046) | 0.546 (-0.201) | 0.518 (+0.216) | 0.210 (+0.131) | 0.183 (+0.137) | - |\n| FLAN-T5-XXL | 0.738 (-0.070) | 0.591 (-0.095) | 0.476 (+0.104) | 0.290 (+0.064) | 0.180 (+0.034) | - |\n\n**Table 2 Text2Re**\n\n|  | Prompt 1# | Prompt 2# | Prompt 3# | Prompt 4# | Prompt 7# | Prompt 8# |\n| --- | --- | --- | --- | --- | --- | --- |\n| text-ada-001 | 0.482 (-0.021) | 0.482 (-0.006) | 0.485 (+0.003) | 0.470 (-0.030) | 0.500 (-0.009) | 0.518 (-0.006) |\n| text-babbage-001 | 0.530 (+0.006) | 0.552 (+0.028) | 0.451 (-0.049) | 0.473 (-0.036) | 0.500 (-0.009) | 0.500 (-0.009) |\n| text-curie-001 | 0.488 (-0.024) | 0.500 (+0.021) | 0.543 (+0.004) | 0.509 (-0.055) | 0.500 (-0.009) | 0.500 (-0.012) |\n| text-davinci-003 | 0.695 (-0.140) | 0.640 (-0.144) | 0.591 (+0.232) | 0.564 (+0.174) | 0.756 (+0.268) | 0.921 (+0.171) |\n| gpt-3.5-turbo | 0.506 (-0.095) | 0.503 (+0.033) | 0.503 (+0.101) | 0.506 (0) | 0.418 (+0.025) | 0.512 (-0.107) |\n| gpt-4 | 0.945 (+0.006) | 0.899 (-0.058) | 0.512 (+0.357) | 0.494 (+0.351) | 0.588 (+0.475) | 0.674 (+0.186) |\n| Claude-1 | 0.506 (-0.299) | 0.351 (-0.356) | 0.857 (+0.241) | 0.860 (+0.272) | 0.677 (+0.451) | 0.378 (+0.195) |\n| Claude-instant-1 | 0.780 (-0.095) | 0.716 (-0.010) | 0.808 (+0.564) | 0.646 (+0.274) | 0.790 (+0.622) | 0.671 (+0.320) |\n| FLAN-T5-Small | 0.503 (+0.012) | 0.500 (+0.006) | 0.512 (+0.033) | 0.509 (+0.015) | 0.512 (-0.015) | - |\n| FLAN-T5-Base | 0.494 (-0.018) | 0.491 (-0.012) | 0.506 (-0.003) | 0.512 (-0.009) | 0.500 (-0.046) | - |\n| FLAN-T5-Large | 0.668 (-0.116) | 0.665 (-0.024) | 0.314 (+0.082) | 0.363 (+0.025) | 0.421 (+0.101) | - |\n| FLAN-T5-XL | 0.689 (-0.259) | 0.530 (-0.250) | 0.616 (+0.488) | 0.628 (+0.378) | 0.363 (+0.244) | - |\n| FLAN-T5-XXL | 0.841 (-0.135) | 0.784 (-0.134) | 0.253 (+0.243) | 0.290 (+0.208) | 0.183 (+0.156) | - |\n\nReferring to the presented tables, it can be inferred that altering symbols to adopt more neutral names generally shows various effects on the models. The performance of most models in prompt 3# and 4# (the challenging setup) is still around 50% or even worse.  Therefore, this technique cannot solve this problem. \n\nWe do observe considerable improvements for some models on some specific setups, especially on the Text2Re task. Interestingly, on the Text2Re task, Claude models obtain pretty high results and consistent improvements on converse relations (prompt  3# 4# 7# 8#), but they also suffer the performance drop on normal relations (prompt 1# 2#). It suggests that, at least for Claude models, more neutral relation names can help to alleviate the spurious correlation, which have opposite effects on normal and converse relations.\n\nWe appreciate the reviewer’s inspiring suggestions. Comprehensive experimental results on this issue and analysis will be added in the revised paper.\n\n> Q4: **Line 135: what is a localized scope?**\n\nBy “localized scope”, we refer to the fact that both the normal and converse relation definitions are specific to the current example (i.e., example-specific prompts). These definitions are not included in the meta prompt and will not influence other examples.\n\n> Q5: **Table 5: row “parent of” has a layout mistake?**\n\nNo, actually, we initially want to express that \"parent of\" is the combination of relation \"mother of person\" from NELL-ONE dataset and relation \"father\" from WikiData5M dataset. We will change this layout to two rows with the relation name \"parent of (mother)\" and \"parent of (father)\" to make it more clear.\n\n> Q6: Typos and layout issues.\n\nThank you for your attentive reading. We will correct them in the revised paper.\n\n\n[1] Lifu Tu, Garima Lalwani, Spandana Gella, and He He. 2020. An Empirical Study on Robustness to Spurious Correlations using Pre-trained Language Models. Transactions of the Association for Computational Linguistics, 8:621–633.\n\n[2] Emily M. Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. 2021. On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (FAccT '21). Association for Computing Machinery, New York, NY, USA, 610–623.\n\n[3] Le Bras, Ronan, Swabha Swayamdipta, Chandra Bhagavatula, Rowan Zellers, Matthew Peters, Ashish Sabharwal, and Yejin Choi. 2020. Adversarial filters of dataset biases. In International conference on machine learning, pp. 1078-1088. PMLR, 2020.\n\n[4] Timothy Niven and Hung-Yu Kao. 2019. Probing Neural Network Comprehension of Natural Language Arguments. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 4658-4664.\n\n[5] Robert Geirhos, Jörn-Henrik Jacobsen, Claudio Michaelis, Richard Zemel, Wieland Brendel, Matthias Bethge, and Felix A Wichmann. 2020. Shortcut learning in deep neural networks. Nature Machine Intelligence, 2(11):665–673.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The paper reveals an important shortcoming of large language model: the reliance on superficial correlations when dealing with some formal-language oriented tasks. What’s more important is that this problem doesn’t get solved (but gets worse) with scale.\n\nExperiments are carefully-designed and provide sound and convincing justifications to the claims. The paper is well-written and easy to follow.", "reasons_to_reject": "The scope of the study is relatively narrow - spurious correlation in LLM might be ubiquitous, and this paper only focuses on the case of relationship triple ↔ text conversion.\n\nMinor:  the paper could be better contextualized among “shortcuts”, “superficial correlations”, “superficial cues”, “spurious correlation”, \"counterfactual\"", "questions_to_authors": "Question A: Are “shortcuts”, “superficial correlations”, “superficial cues” the same thing as “spurious correlation”? If not what are their relations?\n\nQuestion B: Just a random thought, not critical to the review - in the case of formal language, will it solve the problem if we change all symbols to more neutral names? For example, we rename “has part” to “relation A”.", "paper_topic_and_main_contributions": "This paper studies how good LLMs handle converse relations that are less common during training, and possible reasons and implications behind the performance drop. The main task is to test LLM’s ability to convert between text description and triples defined in formal language, while inverting the conventional meaning of the relations. Further tests on few-shot learning and “hard” paraphrases support that LLM rely on superficial correlations. Experiments conducted on LLMs of different scales confirm that the problem gets worse with larger LLMs.", "excitement": 4, "reproducibility": 5, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Missing_References": "Spurious correlation (in LM) might worth mentioning. Some papers to begin with:\nLifu Tu, Garima Lalwani, Spandana Gella, and He He. 2020. An Empirical Study on Robustness to Spurious Correlations using Pre-trained Language Models. Transactions of the Association for Computational Linguistics, 8:621–633.\n\nEmily M. Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. 2021. On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (FAccT '21). Association for Computing Machinery, New York, NY, USA, 610–623. https://doi.org/10.1145/3442188.3445922\n\nA side note: the authors might be interested in (and might have already been aware of) a new paper covering similar topic and testing more tasks, although it is **not** the authors’ obligation to cover or compare to concurrent papers.\nWu, Z., Qiu, L., Ross, A., Akyürek, E., Chen, B., Wang, B., Kim, N., Andreas, J., & Kim, Y. (2023). Reasoning or Reciting? Exploring the Capabilities and Limitations of Language Models Through Counterfactual Tasks. ArXiv, abs/2307.02477.", "Typos_Grammar_Style_and_Presentation_Improvements": "Line 117-119: not a grammatical sentence\n\nLine 135: what is a localized scope? does that mean entities of the same type are used in a single triple? (what’s described in section 2.5)\n\nLine 186: Figure 2 **and** 3\n\nLine 375, 378:\n\nfrom ACLPUB paper formatting guidelines: \n> Refrain from using full citations as sentence constituents. Instead of “(Gusfield, 1997) showed that … In (Gusfield, 1997), …” write “Gusfield (1997) showed that … In Gusfield (1997), …”\n\nTable 5: row “parent of” has a layout mistake?"}}, {"review_id": "IgUYOS3vQc", "reviewer": "Reviewer_RTYu", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your detailed comments, and they are really helpful for us to improve our paper. We will carefully incorporate them in the revised paper. \n\n> Q1: The paper's content might be diluted for its length. There appears to be some redundancy among Figures 2, 3, and 4.\n\nWhile the textual contents of Figure 2 and 3 may seem similar, they hugely differ in their underlying challenges. For Re2Text task, the challenge lies in choosing the correct answers that are presented in less conventional manners (i.e., **altered text**). However, for Text2Re task, the challenge is to avoid being misled by the superficial patterns (i.e. **normal text**) in questions. So we presented the subtle but import details in two separate figures.\n\nFigure 4 is utilized to give a clear presentation of the prompt used in our experiments, including the hint. We will move this figure to appendix and make the main body more compact.\n\n> Q2: The bottom part of Figure 1 seems somewhat uninformative.\n\nRegarding the bottom part of Figure 1, we would like to highlight the unique challenges converse relations present for LLMs, potentially leading to diverse scaling trends. Concretely, LLMs exhibit positive scaling trend for normal relations. But for converse relations, LLMs show diverse scaling trends given different prompting methods. Yes, we will remove this part for the sake of clarity and brevity. This adjustment would also enable us to reintegrate some material from the appendix into the main text.\n\n> Q3: For Figures 5 and 6, I would rather explore the possibility of using tables instead of presenting a whole page of graphs.\n\nFigures 5 and 6 were introduced with the purpose of effectively illustrating the scaling trends of each model family under diverse settings. For example, the scaling pattern in Figure 6(f) is readily discernible. On the other hand, given the number of experimental settings we conducted (12 prompt settings per model across a total of 13 models for each task),  the table would be too big with too many numbers in it, which may make the table not quite reading friendly. We will provide the table of our entire experimental results in appendix for better clarity.\n\nAlthough both the 2Eqb reviewer and the moP5 reviewer mentioned that this paper is well organized and easy to follow, we will seriously consider your suggestions and do our best to make the paper better in revision.\n\n> Q4: I feel like drawing a clear and concise conclusion from this work is challenging due to the vastly different behaviors exhibited by the three models under study. For instance, Claude shows, in general, insensitivity to the number of parameters, performing well or not regardless of this factor, while Flan-T5 aligns more closely with the initial hypothesis proposed by the authors. Additionally, in the few-shot setting, Claude models display completely opposite behavior in the Re2Text and Text2Re tasks, whereas GPT and Flan-T5 models exhibit more consistency. Why is that so? Is it because of the data used in their pre-training? Is it because of a difference in the number of parameters of these two different models (how claude-1 compares with Flan-T5 XXL?)?\n\nWhile it is indeed challenging to draw concise conclusions about the scaling behaviors of the three models in question—Claude, Flan-T5, and GPT—our primary aim in this study is to demonstrate that these models do not adhere to a simple 'the bigger, the better' scaling law. The inconsistencies you pointed out, such as Claude's insensitivity to parameter size and its divergent behavior in different tasks, serve as supporting evidence for our main argument: that the current LMs lack a consistent and robust reasoning capability, particularly when assessed with converse relation understanding.\n\nAlthough investigating the effects of pretraining data and objectives on these behaviors is an important area for future research, those questions are beyond the scope of this particular study. Our focus is to challenge the prevailing belief that the success of LMs can be solely attributed to their generalized reasoning capacities.\n\nFinally, it's worth noting that arriving at a clear and comprehensive understanding of LMs is complicated by various factors. These include the unknown data sources and pretraining methods used for proprietary models (e.g., Claude and GPT) and limited availability of compute resources.\n\nWe will add the discussion as well as the limitation in the revision for better clarity.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The paper presents an interesting analysis of the reasoning capabilities of LLMs by measuring their ability to understand converse relations. \n- The authors introduce a new benchmark (ConvRe) for normal and converse relations to assess the semantic comprehension capabilities of future LLMs.", "reasons_to_reject": "- While this work presents an interesting analysis of the reasoning capabilities of LLMs (even if a few specific) and proposes a benchmark for assessing future language models, I believe that the paper's content might be diluted for its length, and condensing it into a shorter paper could potentially better convey its findings. There appears to be some redundancy among Figures 2, 3, and 4, as they share a lot of information within themselves. Additionally, the bottom part of Figure 1 seems somewhat uninformative, with only one line in the leftmost graph and models not clearly represented. For Figures 5 and 6, I would rather explore the possibility of using tables instead of presenting a whole page of graphs. I feel it might be easier for readers to follow and compare information.\n- I feel like drawing a clear and concise conclusion from this work is challenging due to the vastly different behaviors exhibited by the three models under study. For instance, Claude shows, in general, insensitivity to the number of parameters, performing well or not regardless of this factor, while Flan-T5 aligns more closely with the initial hypothesis proposed by the authors.  Additionally, in the few-shot setting, Claude models display completely opposite behavior in the Re2Text and Text2Re tasks, whereas GPT and Flan-T5 models exhibit more consistency. Why is that so? Is it because of the data used in their pre-training? Is it because of a difference in the number of parameters of these two different models (how claude-1 compares with Flan-T5 XXL?)?", "questions_to_authors": "", "paper_topic_and_main_contributions": "In this work, the authors conduct a thorough investigation into the ability of Large Language Models (LLMs) to comprehend converse relations. Converse relations are defined as the opposite of a given semantic relation. For example, in the normal relation \"x has part y,\" its converse form would be \"y has part x.\" The authors introduce a novel benchmark, called ConvRe, to evaluate LLMs' capabilities in correctly matching relations with their corresponding natural language text in both normal and converse forms.\nThe study focuses on three LLMs: OpenAI's GPT, Anthropic's Claude, and Google's Flan-T5. Through a series of experiments, the authors assess the models' performance under various settings, including zero-shot, few-shot, relations-to-text, text-to-relations, and different model sizes. They analyze the obtained results to gain insights into the reasoning abilities of the examined LLMs.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "gdr3IAzDL8", "reviewer": "Reviewer_2Eqb", "summary": "", "questions": "Question A: What would be human performance? The task is quite easy, but I'm not sure if the average human rater (not connected with academia) would have a 100% success rate.\n\nQuestion B: Why, in the benchmark, every relation needs so many entity pairs? The entities don't influence how the question/instruction/prompt is phrased. Did you observe that different entities/pairs of entities for the same relation lead to different results?\n\nQuestion C: What version of ChatGPT/GPT4 was used (from which date)?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your detailed comments, and they are really helpful for us to improve our paper. We will carefully incorporate them in the revised paper. \n\n> Q1: Not reproducible results on ChatGPT and GPT4 (however, fortunately there are results on the open sourced FLAN-T5).\n\nOur results can be reproduced using the codebase in the original supplementary material, including ChatGPT and GPT4. Besides, we promise to release our codes to the public upon acceptance of the paper. We recognize the challenges that can arise due to potential changes in the API of proprietary LLMs like OpenAI's GPT series. These changes might pose difficulties for future reproducibility of our results.\n\nTo mitigate these concerns, we will provide all the implementation details (see our response to Q4 as an example). We will specify the time stamps for our experimental results for each proprietary LLM in the revised paper. This will give clarity about the version and configuration of the models we used. By openly sharing our benchmark and codebase, we encourage the community to periodically evaluate LLMs. This would help in tracking the performance trajectory of models over time.\n\n> Q2: What would be human performance? The task is quite easy, but I'm not sure if the average human rater (not connected with academia) would have a 100% success rate.\n\nWe randomly selected 300 samples from the dataset and employed three annotators (native English speakers) to complete the human evaluation.\n\nThe results show that with a little hint (refer to prompt 5# and 6#), human participants achieved a success rate close to 100%. While this task is unchallenging for human, advanced LLMs have suffered obstacles. We believe that this observation can help guide the development of subsequent LLMs.\n\n> Q3: Why, in the benchmark, every relation needs so many entity pairs? The entities don't influence how the question/instruction/prompt is phrased. Did you observe that different entities/pairs of entities for the same relation lead to different results?\n\nYes, we have observed that different entity pairs for the same relation lead to different results. We take relation “hypernym” in prompt 4# of Re2Text task as an example and present analysis. In the following, Table 1 shows the accuracy of different models in this setting, and Table 2 shows the overlap percentage of incorrect answers (entity pairs) between different models. The low overlap indicates that different entity pairs for the same relation indeed lead to different results on different models.\n\n**Table 1**\n\n| Model Name | gpt-3.5-turbo | gpt-4 | claude-1 | claude-instant-1 | flan-t5-xxl |\n| --- | --- | --- | --- | --- | --- |\n| Accuracy | 77.50% | 71.25% | 100% | 83.75% | 47.50% |\n\n**Table 2**\n\n| Overlap of incorrect entity pairs | gpt-3.5-turbo | gpt-4 | claude-1 | claude-instant-1 | flan-t5-xxl |\n| --- | --- | --- | --- | --- | --- |\n| gpt-3.5-turbo | - | 11.25% | 0.0% | 1.25% | 17.5% |\n| gpt-4 | 11.25% | - | 0.0% | 2.5% | 25% |\n| claude-1 | 0.0% | 0.0% | - | 0.0% | 0.0% |\n| claude-instant-1 | 1.25% | 2.5% | 0.0% | - | 0.0% |\n| flan-t5-xxl | 17.5% | 25% | 0.0% | 0.0% | - |\n\nWe will add a more comprehensive analysis in the appendix of the revised paper.\n\n> Q4: What version of ChatGPT/GPT4 was used (from which date)?\n\nThe version of ChatGPT we used is `gpt-3.5-turbo-0301`. In the implementation, the `model` parameter should be set to `gpt-3.5-turbo-0301` instead of `gpt-3.5-turbo` when calling the OpenAI API. And the version of GPT4 we used is `gpt-4-0314` .\n\n> Q5: Figure 6 legends is too small to read on paper.\n\nWe will make the corresponding modifications in the revised paper.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The authors:\n - Provides a benchmark for understanding converse relations that poses a challenge for LLMs.\n - Conducts a thorough evaluation on the benchmark showing inverse scaling trend.\n - Quite clear presentation.\n", "reasons_to_reject": " - Not reproducible results on ChatGPT and GPT4 (however, fortunately there are results on the open sourced FLAN-T5).", "questions_to_authors": "Question A: What would be human performance? The task is quite easy, but I'm not sure if the average human rater (not connected with academia) would have a 100% success rate.\n\nQuestion B: Why, in the benchmark, every relation needs so many entity pairs? The entities don't influence how the question/instruction/prompt is phrased. Did you observe that different entities/pairs of entities for the same relation lead to different results?\n\nQuestion C: What version of ChatGPT/GPT4 was used (from which date)?", "paper_topic_and_main_contributions": "The authors of this paper introduce a benchmark for evaluating language models' ability to understand converse relations. Converse relations are relations where the order of the arguments is reversed. The instructions/prompts in the benchmark redefine the order of the arguments of a relation. E.g. a query (x, has part, y) gets a new interpretation which is instead of \"x has part y\" is \"y has part x\".\n\nThe benchmark is divided into two categories: Re2Text and Text2Re. In the Re2Text category, the model is given an instruction and a query, and it must choose a correct textual description of the query. In the Text2Re category, the model is given an instruction and a textual description of a query, and it must choose a correct query for that description.\n\nThe benchmark consists of 17 relations and 1240 triples. The evaluation is performed on three families of language models: GPT, CLAUDE, and FLAN-T5 with different sizes. The evaluation is done in a zero-shot setting, as well as with 3-shot and 6-shot training.\n\nThe authors show that the performance of language models on this benchmark decreases as the size of the model increases. They suggest that LLMs often resort to shortcut learning.\n\nThe main contribution of the paper is the benchmark and the evaluation", "excitement": 4, "reproducibility": 2, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "Figure 6 legends is too small to read on paper."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper introduces a benchmark for evaluating LLMs' ability to understand converse relations that are less common during training. More specifically it pairs text description and triples defined in formal language, while inverting the conventional meaning of the relations. 
Few-shot learning experiments conducted on multiple LLMs ( GPT, Claude, and Flan-T5)  confirm that counter intuitively the problem gets worse with larger LLMs.

Strength:
The paper reveals an important issue with LLMs: the reliance on superficial correlations when dealing with some formal-language oriented tasks.

Weakness:
The scope of the study is limited to specific type of data-- relationship triple ↔ text conversion.
Decision: Accept-Main