FAC ^2 E : Better Understanding Large Language Model Capabilities by Dissociating Language and Cognition (2024)

Abstract: Large language models (LLMs) are primarily evaluated by overall performance on various text understanding and generation tasks. However, such a paradigm fails to comprehensively differentiate the fine-grained language and cognitive skills, rendering the lack of sufficient interpretation to LLMs’ capabilities. In this paper, we present FAC ^2 E, a framework for Fine-grAined and Cognition-grounded LLMs’ Capability Evaluation. Specifically, we formulate LLMs’ evaluation in a multi-dimensional and explainable manner by dissociating the language-related capabilities and the cognition-related ones. Besides, through extracting the intermediate reasoning from LLMs, we further break down the process of applying a specific capability into three sub-steps: recalling relevant knowledge, utilizing knowledge, and solving problems. Finally, FAC ^2 E evaluates each sub-step of each fine-grained capability, providing a two-faceted diagnosis for LLMs. Utilizing FAC ^2 E, we identify a common shortfall in knowledge utilization among models and propose a straightforward, knowledge-enhanced method to mitigate this issue. Our results not only showcase promising performance enhancements but also highlight a direction for future LLM advancements.

## introduction large language models (llms) @xcite , especially instruction-tuned llms @xcite @xcite revolutionized natural language processing and have surpassed human performance on tasks that require nontrivial reasoning @xcite , while showing great potential in applications from conversational assistants (openai, 2022; achiam † corresponding author. canada cifar ai chair. @xcite to expertise problem-solving @xcite @xcite . however, despite the impressive performance, llms also show poor robustness on complex tasks @xcite and significantly inconsistent evaluation results under different settings, such as binary preference @xcite and automatic metrics @xcite . therefore, it is crucial to attain an overarching understanding of the capabilities and limitations of llms. to address this challenge, some studies have assessed the performance of llms on different tasks based on independent benchmarks from various dimensions @xcite @xcite , such as commensense @xcite , knowledge @xcite , instruction-following @xcite , and trustworthy @xcite . besides, motivated by building llm-based ai assistants, other studies propose highly curated benchmarks with instancelevel fine-grained annotations, such as difficulty and reasoning skills @xcite , for holistic evaluation of llms. however, the existing studies, assessing effectiveness across various tasks, provided limited insight into the models' true capabilities, as it only indicates their overall performance on specific datasets, without revealing the fine-grained capabilities acquired or their proficiency levels within the multiple capabilities involved. for instance, in the context of generative question answering, a model adept at extracting information but struggling to form a coherent understanding may exhibit similar overall performance to another model with profound understanding insights but difficulties in articulating accurate responses. we argue that a fine-grained understanding of llms' capabilities can not only accurately unveil their inherent limitations, but also help us to better identify why one model outperforms the other and how the different capabilities correlate. additionally, such insights allow us to provide tailored guidance to improve training efficiency or facilitate more advanced model development. in this paper, we propose fac 2 e, a fine-grained capability evaluation framework for llms. specifically, fac 2 e dissociates the language-related and cognition-related capabilities of llms and organizing them into four distinct axes: linguistic knowledge, formal knowledge, world modeling, and social modeling. this categorization is grounded in neuroscience evidence manifesting that language processing and cognitive processes, like memory and reasoning, operate differently in the brain. drawing from this insight, we adapt a range of existing benchmarks into a unified question-answering format. we then develop specific instructions for each capability, allowing fac 2 e to evaluate llms through a method known as few-shot instruction-following. furthermore, we break down the application of a specific capability into three sub-steps: knowledge recall, knowledge utilization, and problem-solving, by iteratively drawing out the model's intermediate reasoning. after evaluating each sub-step, fac 2 e can reveal the quality of knowledge encoded in the model, and effectiveness in applying relevant knowledge to solve practical problems, offering a more comprehensive evaluation than a single performance metric could. our findings reveal a notable gap in capabilities between open-source and proprietary models, especially for cognition-related capabilities. additionally, we found that many models have difficulties in applying knowledge effectively. to address this, we suggest a knowledge-enhance remedy by incorporating relevant knowledge text as additional input. experimental results show that it can help the backbone model (e.g. llama 2) achieve approximately 90% of the performance of its instructiontuned counterpart (e.g. llama 2-chat). ## methodology in this section, we introduce fac 2 e framework, designed for fine-grained and cognition-grounded llms' capability evaluation. specifically, we first define the taxonomy for llms' capabilities based on the distinction between language and cognition, which is drawn upon insights from neuroscience @xcite . based on this, we transform a variety of existing benchmarks into the unified questionanswering format, design capability-specific instruction, and frame fac 2 e via few-shot instructionfollowing. furthermore, we break down the evaluation process for each capability into a three-step reasoning approach. this involves identifying the knowledge pertinent to the input, examining how the model applies this knowledge in practical contexts, and assessing the effectiveness of its problemsolving. by evaluating each of these steps, fac 2 e provides a comprehensive overview of the model's performance, offering a more nuanced understanding of llms' intrinsic capabilities. ## conclusion we present fac 2 e, defining a fine-grained capability evaluation framework for llms, which decomposes each capability into sub-steps to assess performance of knowledge recalling and utilization as well as problem-solving. fac 2 e reveals the limitations of existing llms in knowledge utilization and provides a knowledge-enhanced remedy for it. empirical results demonstrate its effectiveness.