An Investigation of LLM s’ Inefficacy in Understanding Converse Relations (2023)

Abstract: Large Language Models (LLMs) have achieved remarkable success in many formal language oriented tasks, such as structural data-to-text and semantic parsing. However current benchmarks mostly follow the data distribution of the pre-training data of LLMs. Therefore, a natural question rises that do LLMs really understand the structured semantics of formal languages. In this paper, we investigate this problem on a special case, converse binary relation. We introduce a new benchmark ConvRe focusing on converse relations, which contains 17 relations and 1240 triples extracted from popular knowledge graph completion datasets. Our ConvRE features two tasks, Re2Text and Text2Re, which are formulated as multi-choice question answering to evaluate LLMs’ ability to determine the matching between relations and associated text. For the evaluation protocol, apart from different prompting methods, we further introduce variants to the test text and few-shot example text. We conduct experiments on three popular LLM families and have observed various scaling trends. The results suggest that LLMs often resort to shortcut learning and still face challenges on our proposed benchmark.

## introduction large language models (llms) have demonstrated impressive empirical results on various nlp tasks @xcite @xcite , including formal language-oriented tasks such as structural data-to-text @xcite and semantic parsing @xcite , which require sophisticated comprehension and production of structured language content. despite these promising advances, a critical concern remains largely unexplored: do these llms genuinely understand the nuanced semantics ## convre benchmark in this section, we will introduce the motivation, task formulation and design choice of our convre benchmark as well as the details surrounding data collection. ## results in this section, we demonstrate the results of different llm families on convre benchmark and provide an in-depth analysis. more results on chat models can be found in appendix e. ## related work studies on llms have shown positive scaling trends, whereby larger models generally perform better on downstream tasks @xcite @xcite @xcite . however, researchers showed that model performance scal-ing can deviate from naive expectations. @xcite showed slower and less smooth trends, and that social biases sometimes scale inversely with model size, a finding that is echoed in @xcite . truthfulqa @xcite demonstrated that while larger models can provide more informative answers, they tend to be less truthful. @xcite introduced the inverse scaling challenge and collected tasks that are highly atypical but still easily understandable by a human. @xcite uncovered the u-shaped scaling trend by expanding the model scope for evaluation. @xcite proposed neqa and showed that this task exhibit inverse, u-shaped, or positive scaling with different prompt methods or model families. @xcite showed that llms fail to correctly generate python code when default identifiers are swapped. recent research has highlighted the issue of inflated performance in llms. @xcite coined the term shortcut learning, revealing models' reliance on superficial cues. @xcite studied the model's robustness to spurious correlations, which refers to the prediction rules that work for the majority examples but do not hold in general. @xcite found that llms tend to rely on shallow matching rather than understanding mathematical concepts. @xcite highlighted the importance of understanding the mechanism by which llms achieved state-of-the-art performance. @xcite showed that llms' few-shot ability is often overestimated due to the use of large held-out sets. @xcite surveyed the hallucination problem in language generation, highlighting the issue of factually incorrect output. @xcite identified attention glitches in transformers, indicating a failure in capturing robust reasoning. @xcite introduced the term reverse curse, showing that llms trained on 'a is b' fails to learn the reverse relationship 'b is a'. ## concolusion in this paper, we present an investigation into llms' understanding of structured semantics, specifically focusing on converse binary relations. by introducing a novel benchmark, convre, we offer a systematic and comprehensive evaluation suite to observe the performance of llms across diverse settings and prompting methods. we have carried out a detailed experimental study and observed various scaling trends that shed light on the capabilities and limitations of llms. our findings suggest that llms often resort to shortcut learning and still face considerable challenges on our proposed benchmark, even when strong prompting techniques are employed. our work underscores the importance of developing evaluation methodologies to improve the understanding of llms and their performance across various tasks.