Title: Uncovering Competency Gaps in Large Language Models and Their Benchmarks
Primary area: interpretability and explainable AI
Keywords: LLM Evaluation, Benchmarks, Sparse Autoencoders, Mechanistic Interpretability

Abstract: The evaluation of large language models (LLMs) relies heavily on standardized benchmarks. These benchmarks provide useful aggregated metrics for a given capability, but those aggregated metrics can obscure (i) particular sub-areas where the LLMs are weak (''model gaps'') and (ii) imbalanced coverage in the benchmarks themselves (''benchmark gaps''). We propose a new method using sparse autoencoders (SAEs) to automatically uncover both types of gaps. By extracting SAE concept activations and computing saliency-weighted performance scores across benchmark data, the method grounds evaluation in the model's internal representations. It also enables useful comparison across benchmarks. We applied the method to two popular open-source models and ten diverse benchmarks, and we uncovered meaningful imbalances: popular benchmarks often miss concepts that are central to their intended scope (e.g., logical or social reasoning), while over-representing concepts related to authority and instruction-following. Notably, the models' worst-performing concepts are often opposites of these sycophantic behaviors (e.g., politely declining a request or maintaining boundaries). The proposed method also automatically surfaces model weaknesses anecdotally noted in prior work (e.g., reasoning with time, palindromes, arithmetic). In sum, the method reveals how benchmark performance can be skewed by over-represented concepts, thus enabling developers to identify and address shortcomings in how model capabilities are currently assessed. Code is available at [anonymized].