EvalTree: Profiling Language Model Weaknesses via Hierarchical Capability Trees
Authors: Zhiyuan Zeng
Yizhong Wang
Hannaneh Hajishirzi
Pang Wei Koh
Venue: COLM 2025 (2025)
Keywords: Evaluation
Interpretability
TL;DR: Formulating the problem of LM weakness profiling and proposing a method EvalTree

Abstract: An ideal model evaluation should achieve two goals: identifying where the model fails and providing actionable improvement guidance. Toward these goals for language model (LM) evaluations, we formulate the problem of generating a weakness profile, a set of weaknesses expressed in natural language, given an LM's performance on every individual instance in a benchmark. We introduce a suite of quantitative assessments to compare different weakness profiling methods. We also introduce a weakness profiling method EvalTree. EvalTree constructs a capability tree where each node represents a capability described in natural language and is linked to a subset of benchmark instances that specifically evaluate this capability; it then extracts nodes where the LM performs poorly to generate a weakness profile. On the MATH and WildChat benchmarks, we show that EvalTree outperforms baseline weakness profiling methods by identifying weaknesses more precisely and comprehensively. Weakness profiling further enables weakness-guided data collection, and training data collection guided by EvalTree-identified weaknesses improves LM performance more than other data collection strategies. We also show how EvalTree exposes flaws in Chatbot Arena's human-voter-based evaluation practice. To facilitate future work, we provide an interface that allows practitioners to interactively explore the capability trees built by EvalTree.

Published as a conference paper at COLM 2025
EVALTREE: Profiling Language Model Weaknesses via Hierar- chical Capability Trees
Zhiyuan Zeng*
Yizhong Wang Hannaneh Hajishirzi
Pang Wei Koh
Paul G. Allen School of Computer Science & Engineering, University of Washington
Allen Institute for Artificial Intelligence zyzengêcs. washington.
Abstract
An ideal model evaluation should achieve two goals: identifying where the model fails and providing actionable improvement guidance. Toward these goals for language model (LM) evaluations, we formulate the problem of generating weakness profile, set of weaknesses expressed in natural language, given an LM's performance on every individual instance in a benchmark. We introduce a suite of quantitative assessments to compare different weakness profiling methods. We also introduce a weakness pro- filing method EVALTREE. EVALTREE constructs a capability tree where each node represents a capability described in natural language and is linked to subset of benchmark instances that specifically evaluate this capability; it then extracts nodes where the LM performs poorly to gen- erate weakness profile. On the MATH and WildChat benchmarks, show that EVAI.TREF outperforms baseline weakness profiling methods by identifying weaknesses more precisely and comprehensively. Weakness profiling further enables makkness-guided data collection, and training data collection guided by EVALTREE- EVAL-REE-identtffed weaknesses improves LM performance more than other data collection strategies. We also show how EVALTREE exposes flaws in Chatbot Arena's human-voter-baset evalua- tion practice. To facilitate future work, we provide an interface that allows practitioners to interactively explore the capability trees built by EVALTREE.
Code and Data
htw..oo////iiiuaaneeeneeeeeeee EvalTTee FrvalTre
Web Interface
1 Introduction
An ideal model evaluation ought to achieve the goals of (1) identifying where the evaluated model fails in human-interpretable way, and (2) providing actionable guidance to improve the model (Liang et al., 2023; Holtzman et al., 2023; Gu et al., 2024; Saxon et al., 2024). However, current model evaluations commonly treat diverse instances in benchmark uniformly, reducing model performance to a single aggregate metric or coarse-grained, -ategory-level metrics at best (Raunak et al., 2022). Doing so obscures the reality that benchmark is heterogeneous, which evaluates diverse capabilities at varying granularities, and that model performance can vary significantly across these capabilities. For example, on the MATH benchmark (Hendrycks et al., 2021b), GPT-4o mini (OpenAI, 2024a) achieves an accuracy of 75.1% when calculating combinations and arrangements of elements, but only 49.1% when analyzing geometric relationships using trigonometric principles, as shown in Figure (a)) As result, current model evaluations often fail to achieve the two goals.
Inspired by the preceding observation, we formulate the problem of generating weakness profile, a set of natural language descriptions model's weaknesses, given the model's performance on every individual benchmark instance. We focus profiling language model (LM) weaknesses (Figure 1(a)). A weakness (e.g., "analyzing geometric relationships using trigonometric principles "" is capability where the LM performs poorly on instances that test for this capability. Weakness profiles advance both goals of model evaluation: (1) they provide practitioners with intuitive takeaway to interpret where LM fails, based

---

Published as a conference paper at COLM 2025
(a) Capability Tree
Mathematical Roasoning inssances: 5000 Acoursoy 70.1%
(b) Training Data Collection Strategies
Generic Capabilcy Guided
Gudeed
Combinatorial Profubiitt Accrtcyy: 10.0%
Geometric Reastning Accunccy 64.5%
Direct Dan Sampling
Weakmess-Guided Data Collection
Calculating Combistions Arrangments of Elemenes Instocees 101 Accoroyy 75,
Analyzing Geomentc Relationships Using Trigonometric Principles insonnee 1/0 Accnccy 49.1%
Accuncy Gain (8)
Weakness Profile
Caledattnng Circdar Permutations
Analyting geometriz relationships using trigonometric princiles Calculating diccuer permucation with constraints and symtmttris
Constraints Symmetries Iuparoes Accyroye 34.1%
Weakness- Data Collection
Figure 1: (a) EVALTREE automatically constructs a capability tree given an LM's perfor- mance on every individual benchmark instance, and then generates weakness profile by extracting tree nodes with statistically low performance. (b) Training data collection guided by weakness profiling effectively improves LM performance, e.g., achieving an accuracy gain that is 2.5x larger than that obtained when being guided by a generic capability.
its heterogeneous performance across diverse capabilities; and (2) they are actionable, e-g., model developers can collect targeted training data to address the identified weaknesses.
In terms of how to profile LM weaknesses, manually analyzing LM performance on all instances is becoming increasingly unrealistic. Some works thus attempt to automatically profile LM weaknesses by constructing a single-level capability categorization across all benchmark instances and identifying low-performing categories (Murahari et al., 2024; Moayeri et al., 2024); however, fixed -ranuuaaayl categorizations could be either too broad to provide precise diagnoses or too specific to retain high-level interpretability. More critically, while some methods, including those mentioned above, have been qualitatively shown to identify LM weaknesses, there is existing study to compare them quantitatively.
To overcome these challenges, we establish a standard for what an ideal weakness profile should achieve and introduce a suite of quantitative assessments. We then propose EVALTREE, a weakness profiling method that automatically constructs a hierarchical tree for any LM benchmark, where each node represents a capability described in natural language and is linked to a subset of instances that specifically evaluate this capability. Instances linked to each node are partitioned into subsets corresponding to children's capabilities, which are further subdivided into more specific, finer-grained sub-capabilities at successive levels of the children's subtrees. EVALTREF then evaluates an LM's performance at every tree node, providing capability tree. To generate a weakness profile, EVALTREE extracts tree nodes with statistically low performance and takes their capability descriptions (Figure ((a)).
Our experiments show that EVALTREE advances both evaluation goals via weakness pro- filing: (1) EVALTREE profiles LM weaknesses more precisely and comprehensively than existing methods on the MATH and WildChat (Zhao et al., 2024a) benchmarks; (2) syn- thetic data generation guided by SWA..TREF-ddentffee weaknesses effectively improves LM performance, e-g., achieving being guided by a generic capability (Figure 1(b)). Furthermore, we show how EVALTRRE uncovers abnormal LM rankings in Chatbot Arena, exposing flaws in its numan-voter-based evaluation practice. We also provide an interface that lets practitioners interactively explore capability trees to facilitate future work. Finally, we discuss future directions, including improving capability trees and leveraging capability trees for potential applications.
accuracy gain that is 2.5x larger than that obtained when
1.1 Related Work
Structured Categorization. Structured categorization of benchmark instances is the essential idea behind EVALTREE. Murahari et al. (2024); Moayeri et al. (2024) automatically categorize benchmark instances into capability groups, providing single-level capability categorization structures. A small number of datasets are released with hierarchical structures defined by
2

---

Published as a conference paper at COLM 2025
their creators. For example, some provide shallow trees, e.g., a two-layer taxonomy (Wang et al., 2022; Bai et al., 2024; Zhong et al., 2024a); some adopt existing trees to guide data collec- tion, such as ImageNet (Deng et al., 2009) using WordNet (Miller, 1994) and iNat2017 (Homn et al., 2018) using a biological taxonomy. Most related to our work, Wang et al. (2023); Zhong et al. (2024b) recursively cluster instances in dataset to construct trees, and Anthropic's internal system Clio (Tamkin et al., 2024) employs Claude 3.5 (Anthropic, 2024) to build trees of human-LM conversations based on specific attributes or characteristics (e.g., topic). However, these techniques either incur prohibitively high LM usage costs or do not release key implementation details and source code, making them difficult to use.
Automatic Weakness Identification. Manually analyzing LM performance on instances in benchmark for weakness profiling is becoming increasingly unrealistic. This because LM benchmarks are growing in complexity to match the expanding versatility of emerging LMs; moreover, some datasets (e.g., WildChat (Zhao et al., 2024a)) collect real-world human-LM interactions, leading to the emergence of capabilities (tested within the benchmark) that are not foreseeable even by their creators in advance, further complicating manual efforts. Some works thus attempt to automatically profile LM weaknesses by using LMs to ana- lyze evaluation results (Zhong et al., 2022) or by identifying low-perforning categories from single-level capability categorization (Murahari et al., 2024; Moayeri et al., 2024). Among these works, we are the first to formulate the problem of weakness profiling with quantitative assessments. Targeting similar goals, some works identify interpretable weak- nesses (Byuboglu et al., 2022; Hua et al., 2023), but assume closed output spaces, making them unsuitable for open-ended tasks; others (Wu et al., 2019; Ribeiro et al., 2020) propose interactive tools based on predefined failure modes, whereas we aim for fully automated profiling without such assumptions. Separately, while weakness profiling operates entirely on existing benchmarks and emphasizes interpretability, some prior work explores identify- ing model weaknesses by constructing custom instance sets to highlight underperforming areas (Ribeiro & Lundberg, 2022; Gao et al., 2023; Li et al., 2024; Wang et al., 2025).
2
LM Weakness Profiles
2.1 Definition and Desiderata
The problem of identifying LM weaknesses is broad. In this paper, we define weakness profile in the simplest way that aligns with the two goals of identifying where an LM fails and providing improvement guidance. We let C denote the set of all possible natural language descriptions and assume an underlying data distribution D. A weakness profile for an LM on a given benchmark drawn from the distribution D is a set W = {w1, .0, C C, where M can vary among different profiles, and each identified weakness wi E W is natural language description of a capability, such as "analyzing geometric relationships using trigonometric principles." An ideal weakness profile W satisfies three (informal) desiderata:
1. Low-performance identification (precision): The LM should exhibit low perfor- mance on instances (sampled from D) testing for each identified weakness w; E W.
2. Comprehensive coverage (comprehensiveness): W should reflect weaknesses that can be captured from the LM's performance on D as comprehensively as possible.
3. Appropriate granularity: Each W) should avoid being overly specific or generic.
We introduce concrete assessments in the next subsection to quantitatively compare weak- ness profiles along these desiderata and introduce experimental details in Section 5.
A weakness profiling method takes as input an LM's evaluation result on a given benchmark of size N sampled from the data distribution D, represented as a vector g E RR,, where each 8: denotes the performance metric achieved by the LM on the i-th instance. We refer to this instance set as the profiling set. Since "weakness" is inherently relative concept, a weakness profiling method should also include a user-tunable hyperparameter 7 to control strictness; for example, increasing T makes weakness identification less strict, allowing capabilities with relatively higher performance to be identified, whereas decreasing 7 makes it more strict, restricting identification to the LM's most severe failures. When referring to a specific method in context, we denote W, as the weakness profile generated with given T.
3

---

Published as a conference paper at COLM 2025
2.2 Assessment for Comparing Weakness Profiles
We assume the existence of a lest set sampled from the data distribution D. Furthermore, given capability description E C, we call an instance that tests for this capability an associated instance of c, with the index set of all associated instances in the test set denoted as prompt an LM to determine whether given instance is
A(c). In our experiments, associated instance of a capability e to get A(c), with further details in Appendix E.1.
We introduce two assessments below to measure the effectiveness of a weakness profile in the first evaluation goal of identifying where an LM fails, based on the three desiderata.
Low-Performance Identification Assessment. We denote the LM's evaluation result vector on the test set as f, analogous to g defined above for the profiling set. We also define the LM's performance metric over set of instance indices S as F(S) Exes fx / assuming that the performance metric can be averaged; for example, each fi might be a binary value (0/1) indicating whether the LM correctly solved the i-th instance, in which case F(S) is the accuracy of the LM on the set S. To measure desideratum 1, i.e. low-performance identification, we examine how low the average performance across identified weaknesses can be, computed as Læ,ew (w:))) how low F(S) can be, i.e., the performance metric on all instances that test for at least one identified weakness in W. In the two comparisons, a lower metric value indicates weaker performance the identified weaknesses, which can better satisfy desideratum 1.
Denoting S = Uw A Aw;;,, we also compare
Ground-Truth Weakness Assessment. To measure all three desiderata, inspired by Zhong et al. (2023), we generate a synthetic evaluation result for a "hypothetical" LM's performance on the profiling set. We use synthetic evaluation results rather than evaluation results of real LMs because desideratum 2, i.e., comprehensive coverage, cannot be reliably measured without prior knowledge of the LM's true weaknesses, which is exactly the problem we are trying solve. By generating a synthetic evaluation result, we can control the ground-truth weaknesses and thus have such prior knowledge, allowing for rigorous assessment. We start with a predefined ground-truth weakness profile W* =w wi,... wh+ Then, we independently sample each gi such that instances associated with weaknesses in W have systematically lower values of gi than others. Finally, to assess a weakness profile W, we measure its alignment with the ground-truth profile W based on the overlap of associated instances in the test set; we restrict [W] to values that are not significantly larger than [W*I, preventing methods from inflating scores by generating overly specific descriptions that increase W which would violate desideratum 3, i.e., appropriate granularity.
Weakness-Guided Training Data Collection. We examine the effectiveness of a weakness profile in supporting the second evaluation goal of improving the evaluated L.M. In the real world, LM developers collect additional training data and perform finetuning to further improve an LM. A common strategy is to collect data guided by generic capability such as "matheratical reasoning". We hypothesize that a weakness- guided strategy, wherein a weakness profile for the LM serves as actionable guidance for targeted data collection, may be more effective by directly addressing where the LM fails. For a controlled comparison, we collect data by synthetic data generation and compare LMs trained on data generated under the guidance of different weakness profiles.
Extrinsic Assessment:
3 EVALTREE: A Tree-Based Method for Profiling LM Weaknesses
3.1 Automatic Construction of Capability Trees
EVALTREE constructs capability tree automatically. EVALTREE first constructs a tree that hierarchically organizes and interprets the capabilities tested within benchmark. Each tree node represents specific capability expressed in natural language and is linked to a subset of benchmark instances that evaluate this capability. The root node is linked to all instances, and each node's children together partition instances linked to it into subsets corresponding to more specific sub-capabilities, as shown in Figure 1aa). Finally, every leaf corresponds one-to-one with an individual instance; it is worth noting that instances linked to each node
4

---

Published as a conference paper at COLM 2025
Impuc: 2" = 45, calcudee the value of
(I) Capability Annotation
Capability Embedding
Output flawrtt as 2 to get 2'- 2
Rewrting exponemial equatiion uniga
2r itplle
Benchmosk
Ausstated Capclally
Mathematics Reasoning
preceed betommsp
62 C4 66
(4) Capability Description
300
Elementary Algebra
064
303
000
001
0.5
(3) Recursive
Expression
Equation
Clustering-Based
Simplification
Salving
Analysis
Construction
Figure 2: EVALTREF's four-stage tree construction pipeline. (1) Capability Annotation prompts an LM to identify a natural language description of each instance's capability. (2) Capability Embedding maps instances to vector space using sentence embeddings of their annotated capabilities. (3) Recursive Clustering-Based
Construction builds the tree by clustering capability embeddings using K-Means recursively. (4) Capability Description assigns each node a natural language sunmary of its children's capabilities using an LM.
are exactly the leaves in its subtree. We propose an automatic four-stage tree construction pipeline, which takes all instances of a benchmark as input, as shown in Figure 2.
Stage (1) Capability Annotation identifies the specific capability description required for each benchmark instance by prompting an LM, a practice also adopted in previous work analyzing LM capabilities (Ouyang al., 2023; Didolkar et al., 2024; Kaur et al., 2024). The LM is asked to not mention the instance's specific content. See Figure 2 for an example.
Stage (2) Capability Embedding uses an off-the-shelf sentence embedding model to gener- ate capability embedding for each annotated capability from the stage (1).
Stage (3) Recursive Clustering- Based Construction recursively builds the hierarchical structure of the tree, starting from the root node linked to all instances. For each node, we cluster the capability embeddings of instances linked to it using K-Means (MacQueen, 1967). We iterate over cluster numbers from 2 to predefined maximum value and select the one that yields the highest Silhouette score (Rousseeuw, 1987). This practice follows Katz et al. (2024), which also determines the cluster number automatically when the value is not predefined. Each cluster in the selected clustering becomes the set of instances linked newly created child node. The process continues recursively for each (non-leaf) child node.
Stage (4) Capability Description assigns a natural language description to each tree node to interpretably specify the capability represented by this node. For each leaf node (instance), we take its annotated capability directly as its capability description. For non-leaf nodes, describe their ca capabilities at progressive granularities by proceeding up the tree in bottom-up way, prompting an LM to summarize the capabilities of a node's children into natural language description that captures their overarching scope; the LM's output is prompted to cover all children's 's capabilities without introducing extraneous concepts.
After constructing the tree, EVALTREE then provides capability tree by evaluating LM performance at every node. Since each node is linked to subset of benchmark instances, an evaluation practice can be seamlessly applied to this subset. For example, metrics such as accuracy or win-rate (Dubois et al., 2023) can be computed on instances linked to each node. See Appendix and G for more details and an alternative tree construction approach.
3.2 Generating a Weakness Profile from the Capability Tree
EVALTREE generates an LM weakness profile by extracting nodes where the LM's perfor- mance metric is significantly below user-tunable threshold t; for clarity, we consider the specific case of correctness- based accuracy being the metric. The extraction algorithm traverses the capability tree from the root to the leaves (see Appendix B for details):
5

---

Published as a conference paper at COLM 2025
1. Statistical Test. At each visited node, we perform binomial test to determine whether its accuracy is significantly lower than T. The test uses the number of linked instances the total sample size and the number of correctly solved instances as the count of successes. We apply the same test to the node's ' direct children!.
2. Node Extraction. A visited node is extracted if: (a) it passes the test described above, and (b) all its direct children with sufficient instances (determined by hyperparameter threshold of number) also pass the test. The design of (b) aims identify the weakness at a granularity that is sufficiently specific. For example, if "algebra" performs statistically below the threshold overall but the LM performs well on its "four-operations" child while performing poorly on "abstract algebra," identifying "algebra" as weakness obscures the fact that the real weakness might lie in "abstract algebra" (or other sub-capabilities); here, further traversal is required. 3. Stopping Criteria. Traversal stops at a node if: (a) its instance number is smaller than a hyperparameter threshold, or (b) the node has been extracted.
Finally, the nodes extracted from running the algorithm are non-overlapping, iee, no instance (leaf node) is linked to more than one extracted node. The final weakness profile consists of the capability descriptions of the extracted nodes. By adjusting the meaning of "count of successes" in the statistical test, this algorithm also supports various metrics (e.g., accuracy and win-rate) and can identify strengths (performance above a threshold).
Baseline Methods for Profiling LM Weaknesses
We describe the baseline methods, which are representative of existing methods that have been qualitatively shown to profile LM weaknesses. See Appendix D for additional details.
TEXTDIFF (Zhong et al., 2022) is an LM-based method that automatically describes differ- ences between two text distributions in natural language. While not originally designed for weakness profiling, prior work has used it to describe distributional differences between two instance sets. We adapt this method by comparing instances where the evaluated LM fails versus succeeds, using the described differences to identify its weaknesses. Specifically, we randomly sample two sets of instances: those where the evaluation result indicates that the evaluated LM has failed, and those where it has succeeded. We then prompt diagnostic LM using the sampled instances to output a predefined number of potential weaknesses that might cause the evaluated LM to struggle. We compute the evaluated LM's performance on the associated instances in the profiling set (Section 2.2) for each potential weakness and select those with the lowest performance metrics the weakness profile. Note that this step actually gives TEFF an unfair advantage over other methods in our experiments, as uses the identical implementation used by the method assessment to determine associated instances; however, a method should not have access to this information in principle, such as which LM is used or what prompt is used for method assessment.
QUALEVAL (Murahari et al., 2024) uses an automatic LM-based pipeline to derive a pre- defined number of capabilities (e.g., 20) described in natural language from all benchmark instances. The method then applies linear programming algorithm to assign each bench- mark instance to some of the derived capabilities. Finally, outputs single-level capability categorization structure. We compute the evaluated LM's performance metric on all in- stances (in the profiling set) assigned to each capability and identify a set of weaknesses as the weakness profile by selecting capabilities with the lowest performance metrics.
In these two methods, I could be either the size of the weakness profile or a performance metric threshold, and the two can be transformed interchangeably.
Note that setting significance level of for each node's statistical test does not guarantee overall a confidence level across all tests, as they are not corrected for multiple comparisons.

---

Published as a conference paper at COLM 2025
Llama 3.1 SB Instruct
DART-Math-Llamas (Uniform)
Llama 3.2 Instruct
Accuragy
Wan-- Waaee
36
Size Weakness Profile (M)
Size of Weakness Profil (M)
Size of Weakness Profile (M)
Accuracy (%)
Win Malke
18
so
2000
203
400
1802
Number of Associated Instances (N")
Number of Associated Instances (N")
Number of Associated Instances (N)
(a) MATH
(b) WildChat10K
TEXTIIFF
QUAL EVAL
EVALTREF
Accuracy Wan--ate All Instances
Figure 3: Comparison of weakness profiling methods using Low-Performan Identification Assessment. The first row shows how the average LM performance across identified weaknesses changes as we vary the minimum weakness profile size M'. The second row shows how the overall performance on all associated instances changes as we vary the minimum number of associated instances N Experiments in (a) were conducted on MATH with Llama 3.1 8B Instruct (Dubey et al., 2024) and DART-Math- Maath-LLamma-88 (Uniform) (Tong et al., 2024), and experiments in (b) were conducted on WildChat10K, where the win-rate is the percentage of instances in which Llama 3.2 3B Instruct (Meta, 2024) is preferred over Gemma 2 IT 2B (Rivière et al., 2024). A lower curve indicates more precise identification of true low- www-performing weaknesses and EVALTREF consistently achieves the lowest curve.
Experimental Results
We now present the results of our experiments that compare all weakness profiling meth- ods, i.e. those introduced in Section 4 and EVALTREE, using the three assessments for weakness profiles introduced in Section 2.2. As preparation for the first two assessments, for each method, we sweep over T to obtain a collection of all distinct weakness profiles {Wa, W) where each profile is included only once even if generated by multiple T.
5.1 Low-Performance Identification Assessment
Now-Performance Identification Assessment compares how low the LM's performance is on weaknesses identified by different methods. We assess all weakness profiling meth- ods on the MATH (Hendryčks al., 2021b) and WildChat10K (a subset we curated from WildChat (Zhao et al., 2024a)) benchmarks and randomly split each benchmark into profil- ing/test test sets (see Appendix C for more configuration details). We constrain the minimum weakness profile size to compare the average performance across identified weaknesses and constrain the minimum number of associated instances to compare overall perfor- mance on all associated instances. To visualize the comparisons,
plot two curves in Figure 3: one with the minimum profile size M' (ranging from to 20) on the x-axis and min [Er F(A(w;)) ([W:] VT, [W, > M') on the y-axis, and another with the minimum associated instance number N' (ranging from to the test set size) on the x-axis and min(F(S,) VT, SS.| > N') on the y-axis, where S, Uw, A(w;). EVALTREE con- sistently achieves the lowest curve, demonstrating its superior precision in capturing true weaknesses compared to other methods. See Appendix E.2 for qualitative analysis.
5.2 Ground- Ground-Truth Weakness Assessment
Ground-Truth Weakness Assessment compares how precisely and comprehensively dif- ferent weakness profiling methods capture ground-truth weaknesses (on synthetic LM
7

---

Published as a conference paper at COLM 2025
d = 0.2
d =
= 0.5
MATH
0.-
03-
1011
14 Sizee of Weeskness Profile (MT
10:11 15
Size of Weakness Profile (AM)
Size of Weakmess Protile (At')
WildChattOk
12
0.4
0.5
15:11 Size of Weakness Profile (M))
Size of Weakness Profile (M)
Size of Weakness Profile (MV)
TEXTIIFF
QUALEVAL
EVALTREF
Figure 4: Comparison of weakness profiling methods using Ground-Truth Weakness Assessment. The plot shows Fl score curves of TEXTDIFF, QUALEVAL, and EVALTREE, where the weakness profile size varies from to 20; the F1 score measures how precisely and comprehensively ground-truth weaknesses are captured. horizontal line indicates each method's ss highest score. d is hyperparameter to control the sampling probability.
evaluation results) with appropriate description granularities. We manually curated 10 ground-truth weaknesses various granularities for MATH and WildChat10K. For each benchmark, we generated three synthetic evaluation results by sampling with different hyperparameters that shape the probability distribution. For a given weakness profile, we compute the Fl score based on the overlap of associated instances to measure both precision and comprehensiveness relative to the ground-truth weakness profile W We plot a curve with M' (ranging from 1 to 20) on the x-axis and the score of WT, where |Wr| = M on the y-axis. All curves shown in Figure 4 and Appendix E.3.3. We observe that for most M', the F1 scores achieved by EVALTREE surpass the highest F1 scores obtained by the other two methods. For additional details and analysis, see Appendix E.3.1 and E.3.2.
5.3 Extrinsic Assessment: Weakness-Guide Training Data Collection
Extrinsic Assessment compares how effectively weakness profiles from different methods guide targeted training data collection to improve the evaluated LM; here, we conducted proof-of-concept experiments using a data-generation LM to generate (synthetic) data inputs (Kim et al., 2024) for data collection. The ceneric-capabilityygiieeee strategy uses description of the targeted benchmark' 's overall capability guidance. For each weakness profiling method, we have a corresponding data collection strategy that randomly samples an identified weakness (in the weakness profile generated by the method) as guidance for generating each data input. For context, we also included the result which training data inputs were directly sampled from the profiling set; however, we emphasize that this strategy has an inherently unfair advantage due to its distributional match to the test set and is not direct point of comparison in proof-of-concept experiments, which focus on LM developers' real-world practice of collecting new finetuning data.
data collection
We started with Llama 3.1 8B Instruct (Dubey et al., 2024) for MATH and DeepSeek-Con Coder- Base 6.7B (Guo et al., 2024) for DS-1000 (Lai et al., 2023), following configurations in Ap- pendix C. When generating an input, we randomly sampled inputs from the profiling set as in-context examples for the data-generation LM. We compared the performance of different LMs on the test set. For all data collection strategies, we collected the same amount of finetuning data inputs, with the output produced by separately feeding the input to
II multiple thresholds for EVALTREE result in the same profile size, select the lowest т. Note that the same profile size does not necessarily imply identical weakness profiles.
8

---

Published as a conference paper at COLM 2025
MATH
DS-1000
52,42
38.0
36.90
36:06
I
51.68
51.12
51.16
36.0
34.18
50.26
34.04
Accuracy
Accuracy
34.0
30,98
48.70
30.0
29.20
1000
Initial 1M
900
TExIDiee-Gided Synthetic Data QUAL EVAL-Guedl Synthetic Data
1000
EVALTREE- Synthetic Data
6006
Generic-Capp Caaare-Caaablity--Giin Synthetic Data
Stmplee Data
Accuracy of different LMs on MATH and DS-1000 test sets. Each chart includes the accuracy of the initial LM (Llama 3.1 8B Instruct and CeepSeek-Coder-Bil Baae 6.7B for MATH and DS-1000). For all other results, bars represent the accuracy of LMs trained on data collected by the corresponding strategy, with error bars indicating the standard error across 5 seeds. Bars for LMs trained on directly sampled data are included for reference, although they have an unfair advantage and are not direct point of comparison. Data collection guided by EVALTREE-identified weaknesses yields the highest accuracy gain.
Figure 5:
the lata-generation LM. Refer to Appendix E.4 for more details. The results in Figure demonstrate that the LM trained on EVALTREE-guided synthetic data significantly outper- formed other LMs. Notably, the EVALTREE-guided data collection strategy even slightly outperformed directly sampling data from the profiling set. Therefore, EVALT provides effective and targeted signals for guiding data collection to improve LM performance.
5.4 LM Usage Cost Comparison
EVALTREF also incurs significantly lower LM usage costs than other methods. When each method identifies 20 weaknesses on MATH, the LM usage costs of TEXTDIFF and QUALEVAL were approximately 20 and 8 times higher than EVALTREE's cost, respectively. This occurs because EVALTREE's LM usage cost remains constant regardless of the weakness profile size [W], whereas the costs of the others scale linearly with WW See Appendix E.5 for details.
5.5 Analysis on Threshold for EVALTREE's Node Extraction
We analyze how the choice of T influences the nodes extracted by the algorithm in Sec- tion 3.2. We examine the LM performance on all extracted nodes as T varies, referred to as weaknesss/trrnnthh nodes, i.e, nodes extracted by the algorithm where the LM's performance is significantly /oiehriigher than a given threshold T. To do this, we use the profiling set to build the capability tree and extract weakness strength nodes with varying thresholds We locate the position of each instance in the test set on the capability tree by computing its capability embedding and then traversing from the root guided by the embedding. Specifi- cally, at each non-leaf node, we predict the child cluster to which the instance belongs (by comparing its capability embedding with the K-Means clustering centers and then picking the closest one), determining which child's subtree to traverse into next; we call an instance that enters a weakness /sakknes//ttennghh strength node's subtree a weakness/strengthh instance and study LM performance on all weakness/ trakness/strengthh strength instances from the test set as varies.
We experimented with the MATH, MMLU (Hendrycks al., 2021a), DS-1000, and Wild- Chat10K benchmarks, and Figure 6, 7, 8, and 10(a) show the LMs' performance on weak- ness/strength instances. To further study generalizability, we experimented with two setups using different benchmarks as profiling and test sets; in the first setup, MATH is the profiling set and CollegeMath (Tang et al., 2024) is the test set; in the second setup, WildChat10K is the profiling set, and the test sets consisted of 10K instances ShareGPT10K and a released subset of Chatbot Arena (Chiang et al., 2024), respectively; we show the results in Figure 9 and 10(b). See Appendix C for more configuration details. We
curated from ShareGPT, called

---

Published as a conference paper at COLM 2025
observe that LM performance on trakkness/stenggee instances from the test set aligns well with the node extraction algorithm's goal. Specifically, performance on weakness laanness//seennng strength instances is generally below /above Furthermore, as T for extracting weakness/strength nodes decreases/increases, the performance on weakness/streeggth instances generally decreases/increases, so is effective hyperparameter for controlling strictness.
6 Further Applications of EVALTREE
Beyond identifying LM weaknesses, EVALTREE has broader applications in improving evaluation practices and facilitating LM capability analysis. We present two examples: (1) using EVAL.TREE to expose flaws in a widely used human-voter-based evaluation practice, and (2) implementing an interface for exploring capability trees to support future research.
Identifying Flaws in Chatbot Arena Evaluation. We give an application example by showing how EVALTREE exposes flaws in the human-voter-basee evaluation practice of Chatbot Arena (Chiang et al., 2024). We begin by using EVALTREE to profile LM weaknesses on Chatbot Arena. To do this, we construct the capability tree for Chatbot Arena, where EVAL,REF ranks 64 LMs each node by computing Elo scores based on human comparison pairs for instances linked to the node; it then identifies weaknesses of strong LMs like GPT- (OpenAI, 2023) by extracting nodes where their ranking is unexpectedly low. The weakness profile reveals surprising patterns, leading us to discover that the identified weakness may not stem from the LM itself but from flaws in the evaluation practice. For instance, at the node "Facilitating inclusive, ethical, and strategic communication and engagenennt across diverse and sensitive contexts," " LMs such as Zephyr-7B-f (Tunstall et al., 2023) and Alpaca 13B (Taori et al., 2023) rank significantly higher than GPT-4 and Claude 2.1 (Anthropic, 2023). We observed that this node contains many user instructions with toxic requests, where human voters tended to prefer models that provide toxic responses over well-aligned models that refuse to answer; more quantitative analysis is provided in Appendix F. This shows that the evaluation practice of Chatbot Arena allows uncontrolled user preferences to diverge from the values of LM development, producing potentially unreliable evaluation results. Because even minor misaligned preferences can significantly change LM rankings (Zhao et al., 2024b; Huang et al., 2025; Min et al., 2025), the need for improved evaluation practices is pressing, In this example, EVALTREE provides actionable insights for refining evaluation practices.
User Interface of Capability Trees. While the weakness profile provides concise summary of where an LM fails, the full capability tree offers deeper and more comprehensive insights beyond this flat representation. Practitioners may wish to explore the capability tree itself to gain insights into a benchmark and analyze LM performance across capabilities at diverse granularities. To support this, we implement interactively explore the capability trees constructed by EVALTREE. Users can expand a node to look deeper into its subtree, check the instances linked to the node, view its sub- capabilities represented by the node's children, examine LM performance at each node, etc. The interface provides an intuitive way for humans navigate capability trees manually, establishing itself as useful analysis tool. The interface is available here.
interface that allows practitioners to
7
Future Work
Future work can enhance EVALTREE in several ways. For example, capability tree construc- tion can be improved by optimizing the tree structure and capability descriptions, makiii its dimensionality and granularity more controllable by humans, exploring model-dependent hierarchical structures, and extending it beyond language to other modalities, etc. Addition- ally, it is useful to study how to quantitatively compare two capability trees directly. Beyond direct enhancements, capability trees can also support a variety of potential applications. For example, they can help analyze LM evaluation results to tailor benchmarks to specific needs, to provide actionable insights into training data mixture, etc. By moving beyond aggregate metrics from existing evaluations, EVALTREE enables more comprehensive and interpretable analysis of LM performance across diverse capabilities, providing a useful foundation for future innovations in understanding and improving LM capabilities.
10

---

Published as a conference paper at COLM 2025
Acknowledgments
We thank Zirui Cheng, Scott Geng, Joongwon Kim, Kyle Lo, Ian Magnusson, Sewon Min, Marco Tulio Ribeiro, Weijia Shi, Luca Soldaini, Ming Zhong, and Ruiqi Zhong for the insightful discussions. We thank Jacqueline He, Sandy Kaplan, Siting Li, Stella Li, Jiacheng Liu, Ben Newman, Rui Qiao, Rui Xin, and Lifan Yuan for proofreading the paper draft. We thank Hamish Ivison and Yuxuan Tong for sharing the model evaluation results. We thank members from the UW NLP and UW ML group for providing helpful feedback. We also thank All Hands Al's product OpenHands (Wang et al., 2024b) and Xingyao Wang for their help with web interface implementation. This work is supported by the Singapore National Research Foundation and the National AI Group in the Singapore Ministry of Digital Development and Information under the AI Visiting Professorship Programme (award number O10)-2024-001); by the A12050 program at Schmidt Sciences; by Google ML and Systems Junior Faculty Award; by NSF Grant Nos. (IS2142739 and IIS2044660; by the Defense Advanced Research Projects Agency's (DARPA) SciFy program (Agreement No. HR00112520300;; and by gift funding from Ai2.
References
Anthropic. Introducing claude 2.1, 2023. claude-2-1.
URL https://ww.aartirppic.cc////ee
Anthropic.
Introducing claude 3.5 sonnet, 2024. URL http:sss/ww..anthpppccccc/o/eees
claude- 3-5 sonnet. s-nnn.eet..
Ge Bai, Jie Liu, Xingyuan Bu, Yancheng He, Jiaheng Liu, Zhanhui Zhou, Zhuoran Lin, Wenbo Su, Tiezheng Ge, Bo Zheng, and Wanli Ouyang. Mt-bench-101: fine-grained benchmark for evaluating large language models in multi-turn dialogues. In Association for Computational Linguistics (ACL), 2024.
Wei-Lin Chiang, Lianmin Zheng, Ying Sheng, Anastasios Nikolas Angelopoulos, Tianle Li, Dacheng Li, Banghua Zhu, Hao Zhang, Michael Jordan, Joseph E. Gonzalez, and Ton Stoica. Chatbot arena: An open platform for evaluating Ilms by human preference. In International Conference on Machine Learning (ICML), 2024.
Jia Deng, Wei Dong, Richard Socher, Li-lia Li, Kai Li, and Li Fei-Fei. Imagenet: A large- scale hierarchical image database. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2009.
Aniket Didolkar, Anirudh Goyal, Nan Rosemary Ke, Siyuan Guo, Michal Valko, Timothy P. Lillicrap, Danilo J. Rezende, Yoshua Bengio, Michael Mozer, and Sanjeev Arora. Metacog- nitive capabilities of Ilms: An exploration in mathematical problem solving. ar Xiviv preprint ar XrXiv:2405.1 12200, 2024.
Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, Arun Rao, Aston Zhang, Aurélien Rodriguez, Austen Gregerson, Ava Spataru, Baptiste Rozière, Bethany Biron, Binh Tang, Bobbie Chern, Charlotte Caucheteux, Chaya Nayak, Chloe Bi, Chris Marra, Chris McConnell, Christian Keller, Christophe Touret, Chunyang Wu, Corinne Wong, Cristian Canton Ferrer, Cyrus Nikolaidis, Damien Allonsius, Daniel Song, Danielle Pintz, Danny Livshits, David Esiobu, Dhruv Choudhary, Dhruv Mahajan, Diego Garcia-Olano, Diego Perino, Dicuwke Hupkes, Egor Lakomkin, Ehab AlBadawy, Elina Lobanova, Emily Dinan, Eric Michael Smith, Filip Radenovic, Frank Zhang, Gabriel Synnaeve, Gabrielle Lee, Georgia Lewis Anderson, Graeme Nail, Grégoire Mialon, Guan Pang, Guillem Cucurell, Hailey Nguyen, Hannah Korevaar, Hu Xu, Hugo Touyron, Iliyan Zarov, Imanol Arrieta Ibarra, Isabel M. Kloumann, Ishan Misra, Ivan Ev Evtimov, Jade Copet, Jaewon Lee, Jan Geffert, Jana Vranes, Jason Park, Jay Mahadeokar, Jeet Shah, Jelmer van der Linde, Jennifer Billock, Jenny Hong, Jenya Lee, Jeremy Fu, Jianfeng Chi, Jianyu Huang, Jiawen Liu, Jie Wang, Jiecao Yu, Joanna Bitton, Joe Spisak,
11

---

Published as a conference paper at COLM 2025
Jongsoo Park, Joseph Rocca, Joshua Johnstun, Joshua Saxe, Junteng Jia, Kalyan Vasuden Alwala, Kartikeya Upasani, Kate Plawiak, Ke Li, Kenneth Heafield, Kevin Stone, and et al. The llama 3 herd of models. arXiv preprint ar Xiio:240.221783, 2024.
Yann Dubois, Chen Xuechen Li, Rohan Taori, Tianyi Zhang, Ishaan Gulrajani, Jimmy Ba, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. Alpacafarm: A simulation framework for methods that learn from human feedback. In Adoances in Neural Information Processing Systems (NeurIPS), 2023.
Sabri Eyuboglu, Maya Varma, Khaled Kamal Saab, Jean-Benoit Delbrouck, Christopher Lee-Messer, Jared Dunnmon, James Zou, and Christopher Ré. Domino: Discovering systematic errors with cross-modal embeddings. In International Conference on Learning Representations (ICLR), 2022.
Irena Gao, Gabriel Ilharco, Scott M. Lundberg, and Marco Túlio Ribeiro. Adaptive testing of computer vision models. In IEEE/CVE International Conference on Computer Vision (ICCV), 2023.
Yuling Gu, Oyvind Tafjord, and Peter Clark. Digital socrates: Evaluating llms through explanation critiques. In Association for Computational Linguistics (ACL), 2024.
Daya Guo, Qihao Zhu, Dejian Yang, Zhenda Xie, Kai Dong, Wentao Zhang, Guanting Chen, Xiao Bi, Y. Wu, Y. K. Li, Fuli Luo, Yingfei Xiong, and Wenfeng Liang. Deepseek-coder: When the large language model meets programming the rise of code intelligence. ar Xiv 2024.
preprint ar Xiv::200.1141
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. In International Conference on Learning Representations (ICLR), 2021a.
Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the MATH dataset. In Advances in Neural Information Processing Systems (NeurIPS) Datasets and Benchmarks Track, 2021b.
Ari Holtzman, Peter West, and Luke Zettlemoyer. Generative models as a complex systems science: How can we make sense of large language model behavior? arxiv preprint 1XXiv:3388000180 2023.
Grant Van Horn, Oisin Mac Aodha, Yang Song, Yin Cui, Chen Sun, Alexander Shepard, Hartwig Adam, Pietro Perona, and Serge Belongie. The inaturalist species classification and detection dataset. In IEEE/CVE Conference on Computer Vision and Pattern Recognition (CVPR), 2018.
Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. In International Conference on Learning Representations (ICLR), 2022.
Wenyue Hua, Lifeng Jin, Linfeng Song, Haitao Mi, Yongfeng Zhang, and Dong Yu. Dis- cover, explain, improve: An automatic slice detection benchmark for natural language processing. Transactions of the Association of Computational Linguistics (TACL), 2023.
Yangsibo Huang, Milad Nasr, Anastasios Angelopoulos, Nicholas Carlini, Wei-Lin Chiang, Christopher A. Choquette-Choo, Daphne Ippolito, Matthew Jagielski, Katherine Lee, Ken Ziyu Liu, lon Stoica, Florian Tramèr, and Chiyuan Zhang. Exploring and mitigating adversarial manipulation of voting-based leaderboards. arXio preprint Xio::2501.07493, 2025.
Uri Katz, Mosh Levy, and Yoav Goldberg. Knowledge navigator: LIm-guided browsing framework for exploratory search in scientific literature. In Findings of Empirical Methods in Natural Language Processing (EMNLP), 2024.
Simran Kaur, Simon Park, Anirudh Goyal, and Sanjeev Arora. Instruct-skillmix: powerful pipeline for LLM instruction tuning. ar preprint ar arXit:2488.1477. 2024.
12

---

Published as a conference paper at COLM 2025
Seungone Kim, Juyoung Suk, Xiang Yue, Vijay Viswanathan, Seongyun Lee, Yizhong Wang, Kiril Gashteovski, Carolin Lawrence, Sean Welleck, and Graham Neubig. Evaluating language models synthetic data generators. arviv preprint ar Xfi::2412.03679, 2024.
Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large language model serving with pagedattention. In Symposium on Operating Systems Principles (SOSP), 2023.
Yuhang Lai, Chengxi Li, Yiming Wang, Tianyi Zhang, Ruiqi Zhong, Luke Zettlemoyer, Wen-Tau Yih, Daniel Fried, Sida I. Wang, and Tao Yu. DS-1000: A natural and reliable benchmark for data science code generation. In International Conference on Machine Learning (ICML), 2023.
Nathan Lambert, Jacob Morrison, Valentina Pyatkin, Shengyi Huang, Hamish Ivison, Faeze Brahman, Lester James V. Miranda, Alisa Liu, Nouha Dziri, Shane Lyu, Yuling Gu, Saumya Malik, Victoria Graf, Jena D. Hwang, Jiangjiang Yang, Ronan Le Bras, Oyvind Tafjord, Chris Wilhelm, Luca Soldaini, Noah A. Smith, Yizhong Wang, Pradeep Dasigi, and Hannaneh Hajishirzi. Tülu 3: Pushing frontiers in open language model post-training. Xrviv preprint ar arXio:2411.15122 2024.
Xiang Lisa Li, Evan Zheran Liu, Percy Liang, and Tatsunori Hashimoto. Autobencher: Cre- ating salient, novel, difficult datasets for language models. arXiv preprint ar wxXiv2207,00357 2024.
Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michihiro Ya- sunaga, Yian Zhang, Deepak Narayanan, Yuhuai Wu, Ananya Kumar, Benjamin Newman, Binhang Yuan, Bobby Yan, Ce Zhang, Christian Cosgrove, Christopher D. Manning, Christopher Ré, Diana Acosta-Navas, Drew A. Hudson, Eric Zelikman, Esin Durmus, Faisal Ladhak, Frieda Rong, Hongyu Ren, Huaxiu Yao, Jue Wang, Keshav Santhanam, Laurel Jrrr, Lucia Zheng, Mert Yüksekgönül, Mirac Suzgun, Nathan Kim, Neel Guha, Ni- ladri S. Chatterji, Omar Khattab, Peter Henderson, Qian Huang, Ryan Chi, Sang Michael Xie, Shibani Santurkar, Surya Ganguli, Tatsunori Hashimoto, Thomas Icard, Tianyi Zhang, Vishrav Chaudhary, William Wang, Xuechen Li, Yifan Mai, Yuhui Zhang, and Yuta Ko- reeda. Holistic evaluation of language models. Transactions on Machine Learning Research (TMLR), 2023.
J MacQueen. Some methods for classification and analysis of multivariate observations. In Proceedings of 5-th Berkeley Symposium on Mathematical Statistics and Probability/Uniivvrrtty of California Press, 1967.
Meta. Llama 3.2:
Revolutionizing edge ai and vision with open, URL https://ai htttss//aaimmaa..../ea www.a.ta...ommm. com/blog///
customizable models, 2024.
1lama-3-22 3-ama-3-2-come 3---c--nectt-2222- 2eteect-222--iiiii- wwtp-iiiiii--eeee-bi---e--eeeeee
George A. Miller. WordNet: lexical database for English. Human Language Technology: Proceedings of Workshop held at Plainsboro, New Jersey, 1994.
Rui Min, Tianyu Pang, Chao Du, Qian Liu, Minhao Cheng, and Min Lin. Improving your model ranking on chatbot arena by vote rigging. arXiv preprint Xiio:2501.17858, 2025.
Mazda Moayeri, Vidhisha Balachandran, Varun Chandrasekaran, Safoora Yousefi, Thomas Fel, Soheil Feizi, Besmira Nushi, Neel Joshi, and Vibhav Vineet. Unearthing skill-level in- sights for understanding trade-offs of foundation models. arXiv preprint Xri::22410.13826, 2024.
Daniel Müllner. Modern hierarchical, agglomerative clustering algorithms. arviv preprint wxXiv:1109.2376 2011.
Vishvak Murahari, Ameet Deshpande, Peter Clark, Tanmay Rajpurohit, Ashish Sabharwal, Karthik Narasimhan, and Ashwin Kalyan. Qualeval: Qualitative evaluation for model improvement. In North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), 2024.
13

---

Published as a conference paper at COLM 2025
OpenAI. Introducing chatgpt, 2022. URL http:ppp///pppaa...cc/////eee//s
OpenAl. Gpt-4 technical report. arXie preprint arXft:23003.06774,200 2023.
OpenAl. Gpt-4o mini: advancing cost-efficient intllliieennntntllleeeeeeeeee 2024a.
2024a. URL https://openai.
com/ Lnnde//gt-40r -ine-oo-mii-iaaanaaee www.-aa-waaaiiiisst--fffdideeeed
OpenAI. Hello gpt-4o, 2024b. URL http:s///peeaa...coo/mmeeeelllll
OpenAL. New embedding models and api updates, 2024c. URL httptttttssppeai..comm comm/ndex/
new-embeddingg models- and-a api-updates/.
Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Francis Christiano, Jan Leike, and Ryan J. Lowe. Training language models to follow instructions with human feedback. In Advances in Neural Information Processing Systems (NeurIPS), 2022.
Siru Ouyang, Shuohang Wang, Yang Liu, Ming Zhong, Yizhu Jiao, Dan Iter, Reid Pryzant, Chenguang Zhu, Heng Ji, and Jiawei Han. The shifted and the overlooked: A task- oriented investigation user-gpt interactions. In Empirical Methods in Natural Language Processing (EMNLP), 2023.
Vikas Raunak, Matt Post, and Arul Menezes. Operationalizing specifications, in addition to test sets for evaluating constrained generative models. arXiv preprint XrXiv:2212.00066, 2022.
Marco Túlio Ribeiro and Scott M. Lundberg. Adaptive testing and debugging of NLP models. In Association for Computational Linguistics (ACL), 2022..
Marco Tulio Ribeiro, Tongshuang Wu, Carlos Guestrin, and Sameer Singh. Beyond accu- racy: Behavioral testing of NLP models with CheckList. In Association for Computational Linguistics (ACL), July 2020.
Morgane Rivière, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupati- raju, Léonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ramé, Johan Ferret, Peter Liu, Pouya Tafti, Abe Friesen, Michelle Casbon, Sabela Ramos, Ravin Kumar, Charline Le Lan, Sammy Jerome, Anton Tsitsulin, Nino Vieillard, Piotr Stanczyk, Sertan Girgin, Nikola Momchev, Matt Hoffman, Shantanu Thakoor, Jean-Bastien Grili, Behnam Neyshabur, Olivier Bachem, Alanna Walton, Aliaksei Severyn, Alicia Parrish, Aliya Ah- mad, Allen Hutchison, Alvin Abdagic, Amanda Carl, Amy Shen, Andy Brock, Andy Co- enen, Anthony Laforge, Antonia Paterson, Ben Bastian, Bilal Piot, Bo Wu, Brandon Royal, Charlie Chen, Chintu Kumar, Chris Perry, Chris Welty, Christopher A. Choquette-Choo, Danila Sinopalnikov, David Weinberger, Dimple Vijaykumar, Dominika Rogozinska, Dustin Herbison, Flisa Bandy, Emma Wang, Eric Noland, Erica Moreira, Evan Senter, Ev- genii Eltyshev, Francesco Visin, Gabriel Rasskin, Gary Wei, Glenn Cameron, Gus Martins, Hadi Hashemi, Hanna Klimezak-Plucinska Harleen Batra, Harsh Dhand, Ivan Nardini, Jacinda Mein, Jack Zhou, James Svensson, Jeff Stanway, Jetha Chan, Jin Peng Zhou, Joana Carrasqueira, Joana Iljazi, Jocelyn Becker, Joe Fernandez, Joost van Amersfoort, Josh Gordon, Josh Lipschultz, Josh Newlan, Ju-yeong Ji, Kareem Mohamed, Kartikeya Badola, Kat Black, Katie Millican, Keelin McDonell, Kelvin Nguyen, Kiranbir Sodhia, Kish Greene, Lars Lowe Sjösund, Lauren Usui, Laurent Sifre, Lena Heuermann, Leticia Lago, and Lilly McNealus. Gemma 2: Improving open language models at a practical size. arXiv preprint ar 4rXiv:2408.00118, 2024.
Peter Rousseeuw. Silhouettes:
graphical aid the interpretation and validation of cluster analysis. Journal of computational and applied mathematics, 1987.
Michael Saxon, Ari Holtzman, Peter West, William Yang Wang, and Naomi Saphra. Bench- marks as microscopes: call for model metrology. In Conference on Language Modeling (COLM), 2024.
14

---

Published as a conference paper at COLM 2025
Alex Tamkin, Miles McCain, Kunal Handa, Esin Durmus, Liane Lovitt, Ankur Rathi, Saffron Huang, Alfred Mountfield, Jerry Hong, Stuart Ritchie, Michael Stern, Brian Clarke, Landon Goldberg, Theodore R. Sumers, Jared Mueller, William McEachen, Wes Mitchell, Shan Carter, Jack Clark, Jared Kaplan, and Deep Ganguli. Clio: Privacy-preserye insights into real-world ai use. arXiv preprint arXin:2412.1367 2024.
Zhengyang Tang, Xingxing Zhang, Benyou Wang, and Furu Wei. Mathscale: Scaling instruction tuning for mathematical reasoning. In International Conference on Machine Learning (ICML), 2024.
Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. Stanford alpaca: An instruction-follooiing Ilama model, 2023. URL https: //tp::///fffm http///.....t.orrdooo..o.//....o www.oorddddd////////eeee
edu/ (tllu/022/03/1// 03/13/ 3/a /llapac.. html.
Yuxuan Tong, Xiwen Zhang, Rui Wang, Ruidong Wu, and Junxian He. Dart-math: Difficulty- aware rejection tuning for mathematical problem-solving. In Advances in Neural Informa- fion Processing Systems (NeurIPS), 2024.
Lewis Tunstall, Edward Beeching, Nathan Lambert, Nazneen Rajani, Kashif Rasul, Younes Belkada, Shengyi Huang, Leandro von Werra, Clémentine Fourrier, Nathan Habib, Nathan Sarrazin, Omar Sanseviero, Alexander M. Rush, and Thomas Wolf. Zephyr: Direct distillation of Im alignment. XrXiv preprint ar Xi11:2310..1944, 2023.
Peiyi Wang, Lei Li, Liang Chen, Zefan Cai, Dawei Zhu, Binghuai Lin, Yunbo Cao, Lingpeng Kong, Qi Liu, Tianyu Liu, and Zhifang Sui. Large language models are not fair evaluators. In Association for Computational Linguistics (ACL.,, 2024a.
Siyuan Wang, Zhuohan Long, Zhihao Fan, Xuanjing Huang, and Zhongyu Wei. Benchmark self-evolving: A multi-agent framework for dynamic LLM evaluation. In International Conference on Computational Linguistics (COLING), 2025.
Xingyao Wang, Boxuan Li, Yufan Song, Frank F. Xu, Xiangru Tang, Mingchen Zhuge, Ji- ayi Pan, Yueqi Song, Bowen Li, Jaskirat Singh, Hoang H. Tran, Fuqiang Li, Ren Ma, Mingzhang Zheng, Bill Qian, Yanjun Shao, Niklas Muennighoff, Yizhe Zhang, Binyuan Hui, Junyang Lin, Robert Brennan, Hao Peng, Heng Ji, and Graham Neubig. Open- hands: An open platform for ai software developers as generalist agents. arXiv preprint ar arXiv:2407.16 16741,, 2024b.
Yizhong Wang, Swaroop Mishra, Pegah Alipoormolabsshi, Yeganeh Kordi, Amirreza Mirzaei, Atharva Naik, Arjun Ashok, Arut Selvan Dhanasekaran, Anjana Arunkumar, David Stap, Eshaan Pathak, Giannis Karamanolakis, Haizhi Gary Lai, Ishan Purohit, Ishani Mondal, Jacob Anderson, Kirby Kuznia, Krima Doshi, Kuntal Kumar Pal, Maitreya Patel, Mehrad Moradshahi, Mihir Parmar, Mirali Purohit, Neeraj Varshney, Phani Rohitha Kaza, Pulkit Verma, Ravsehaj Singh Puri, Rushang Karia, Savan Doshi, Shailaja Keyur Sampat, Siddhartha Mishra, Sujan Reddy A, Sumanta Patro, Tanay Dixit, and Xudong Generalization via declarative instructions on 1600+ NLP tasks. In Empirical Methods in Natural Language Processing (EMNLP), 2022.
Shen. iupeer-nturralintrreeee
Zihan Wang, Jingbo Shang, and Ruiqi Zhong. Goal-driven explainable clustering via language descriptions. In Empirical Methods in Natural Language Processing (EMNLP), 2023.
Tongshuang Wu, Marco Tulio Ribeiro, Jeffrey Heer, and Daniel Weld. Errudite: Scalable, reproducible, and testable error analysis. In Association for Computational Linguistics (ACL), 2019.
Zhiyuan Zeng, Jiatong Yu, Tianyu Gao, Yu Meng, Tanya Goyal, and Danqi Chen. Evaluating large language models at evaluating instruction following. In International Conference on Learning Representations (ICLR), 2024.
Wenting Zhao, Xiang Ren, Jack Hessel, Claire Cardie, Yejin Choi, and Yuntian Deng. Wild- chat: im chatgpt interaction logs in the wild. In International Conference on Learning Representations (ICLR), 2024a.
15

---

Published as a conference paper at COLM 2025
Wenting Zhao, Alexander M Rush, and Tanya Goyal. Challenges in trustworthy human evaluation of chatbots. arXiv preprint arXio:2412.04433,, 2024b.
Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. Judging llm-as-a-judge with mt-bench and chatbot arena. In Advances in Neural Information Processing Systems (NeurIPS), 2023.
Ming Zhong, Aston Zhang, Xuewei Wang, Rui Hou, Wenhan Xiong, Chenguang Zhu, Zhengxing Chen, Liang Tan, Chloe Bi, Mike Lewis, Sravya Popuri, Sharan Narang, Melanie Kambadur, Dhruv Mahajan, Sergey Edunov, Jiawei Han, and Laurens van der Maaten. Law of the weakest link: Cross capabilities of large language models. arxiv preprint arXio:2409.19951 2024a.
Ruiqi Zhong, Charlie Snell, Dan Klein, and Jacob Steinhardt. Describing differences between text distributions with natural language. In International Conference on Machine Learning (ICML), 2022.
Ruiqi Zhong, Peter Zhang, Steve Li, Jinwoo Ahn, Dan Klein, and Jacob Steinhardt. Goal driven discovery of distributional differences via language descriptions. In Advances in Neural Information Processing Systems (NeurIPS), 2023.
Ruiqi Zhong, Heng Wang, Dan Klein, and Jacob Steinhardt. Explaining datasets in words: Statistical models with natural language parameters. In Advances in Neural Information Processing Systems (NeurIPS), 2024b.
A Implementation Details of Automatic Capability Tree Construction
This section provides additional details about the implementation of the automatic four- stage tree construction pipeline of EVALTREE, which is introduced in Section 3.1.
Capability Annotation. By default, we use Open 2024a) in experiments to generate natural language descriptions of the capabilities required to solve each benchmark instance. The prompt for the mathematics reasoning benchmarks (MATH (Hendrycks al., 2021b) and CollegeMath (Tang et al., 2024)) is in Table 1; the prompt for MMLU (Hendrycks et al., 2021a) is in Table 2; the prompt for the Python code generation benchmark (DS-1000 (Lai et al., 2023)) in Table 3; the prompt benchmarks (WildChat10K (Zhao et al., 2024a), ShareGPT10K,
(OpenAI,
for the instruction-followin and Chatbot Arena (Chiang et al., 2024)) is in Table We set the max new tokens and temperature to 1024 and 0.0, respectively.
Capability Embedding. When generating capability embeddings, we prepend the pre- fix "The model has the following skill or capability: to the annotated capability and feed the resulting sentence into a sentence embedding model. By default, we use OpenAl's next-embeddinn--33 3umbeddiin---small (OpenAI, 2024c) in our experiments.
Recursive Clustering- Based Construction. As we mentioned in the main text above, clusterings are generated for each cluster number from 2 to predefined maximum value, and the Silhouette 3 (Rousseeuw, 1987), which measures clustering quality based cohesion and separation, is computed for each clustering. In our experiments, the predefined maximum value is set to 10 by default. One detail that, if no clustering achieves positive score, all instances linked to the current node are treated as leaves and become direct children of it. For the K-Means implementation, we use sklearn.cluster. "MMaas..
Capability Description. By default, we use OpenAl's gpt-44-mini-222 -024-202--07-11 in our experiments to describe the specific capability each node represents in natural language. The prompt for the mathematics reasoning benchmarks (MATH and CollegeMath) is in Table 5;
9nttps://scikitt-eenn htw.://aannngggaaaa/////eeeeaeee htt.taaleeemmmeee/e/ettt seeeeattedskkleareee mettrrm.metiiss...m cestricsssslhhueee silhouette score. html. All hyperparameters are set their default values. "https://sciktt-leame perparameters are set to their default values.
///mmuulss/generttte//////t seeneratted/skeeamm cluster.KMeans.. html. All hy-
16

---

Published as a conference paper at COLM 2025
System Prompt
Given mathematical question and its correct solution, generate gerund phrase that thoroughly and precisely describes the **specifi*** mathematical skill or capability required to solve the question.
User Prompt
## Question
[input]
## Solution {output)
## Requirement
The skill description should be an action-oriented gerund phrase that is *inffrrm-- tive** and **detaile***
The phrase should refer to "*specifi** skill capability that comprehensively
covers the key aspects of the solution, without including any context or specifics
from the question or solution.
Avoid unnecessary elements unrelated to the core capability.
Please output **only gerund phrase** describing the skill, with NO additional text.
Table 1: The capability annotation prompt for the mathematics reasoning benchmarks (MATH (Hendrycks et al., 2021b) and CollegeMath (Tang et al., 2024)).
System Prompt
Given a multiple-choice question testing a model's wide-ranging knowledge and reasoning skills, generate gerund phrase that thoroughly and precisely describes the "sseecffcc* skill or capability required to determine the correct answer.
User Prompt ## Question
{input}
## Answer {output}
## Requirement
The skill description should be an action-oriented gerund phrase that is "inffrma- tive** and **etailedd*
The phrase should refer to a **specific** skill or capability that comprehensively
covers the key aspects of selecting the correct answer, without including any context or specifics from the question or answer.
Avoid unnecessary elements unrelated to the core capability.
Please output **only gerund phrase** describing the skill, with NO additional text.
Table 2:
The capability annotation prompt for MMLU (Hendrycks et al., 2021a).
17

---

Published as a conference paper at COLM 2025
System Prompt
Given a code generation problem (involving data science) and its correct Python implementation, generate gerund phrase that thoroughly and precisely describes the coding skill capability required to solve the problem in detail.
User Prompt ## Problem {input}
## Implementation {output}
## Requirement
The skill description should be an action-oriented gerund phrase that is "innfrrma- tive** and **detailed**.
The phrase should refer to "specific** coding skill capability that comprehen- sively covers the key aspects of the implementation, without including any context specifics from the problem implementation.
Avoid unnecessary elements unrelated to the core capability.
Please output ** **only gerund phrase** describing the skill, with NO additional
text.
Table 3:
The capability annotation prompt for the Python code generation benchmark (DS-1000 (Lai et al., 2023)).
System Prompt
Given a user instruction and a reference response to the instruction, generate gerund phrase that thoroughly and precisely describes the **specifi*** skill or capa- bility required to respond to the instruction.
User Prompt
## Instruction
[input]
## Response {output)
## Requirement
The skill description should be an action-oriented gerund phrase that is iinforma- tive** and **detaile
The phrase should refer to ""specific* skill capability that comprehensively
covers the key aspects of the response, without including any context or specifics from the instruction or reference response.
Avoid unnecessary elements unrelated to the core capability.
Please output **only gerund phrase** describing the skill, with NO additional text.
Table The capability annotation prompt for the instruction-following benchmarks (WildChatlOK (Zhao et al., 2024a), ShareGPT10K, and Chatbot Arena (Chiang et al., 2024)).
18

---

Published as a conference paper at COLM 2025
the prompt for MMLU is in Table 6; the prompt for the Python code generation benchmark (DS-1100)) is in Table 7; the prompt for the instruction-followin ShareGPTI0K, and Chatbot Arena) is in Table 8. We set the max new tokens and temperature to 1024 and 0.0, respectively.
benchmarks (WildChat1OK,
System Prompt
Given a set of phrases, each summarizing the mathematical skills or capabilities needed to solve questions within specific group, generate gerund phrase that summarizes the collective set of mathematical skills or capabilities described across all groups.
User Prompt ## Task
You are given a set of phrases, each summarizing the mathematical skills or capabili- ties needed to solve questions within specific group. There are (group number) groups in total. Your task is to **summarize** the collective set of mathematical skills or capabilities that represents the union these descriptions in a detailed and informative manner.
## Skill Descriptions
{skill descriptions}
## Requirements
The output should be a "single gerund phrase"* that succinctly summarizes the overarching mathematical skill or capability represented by the union of all the provided phrases.
The output should comprehensively cover each skill description without going beyond them.
The output should not simply enumerate the given phrases but instead provide meaningful and informative summary of the mathematical skills or capabilities they collectively represent.
Please output **only a gerund phrase** summarizing the mathematical skill or capability, with NO additional text.
Table 5:
The capability description prompt for the mathematics reasoning benchmarks (MATH (Hendrycks et al., 20216) and CollegeMath (Tang et al., 2024)).
B Implementation Details of Extracting Nodes with Low Performance
Algorithm 1 provides the pseudocode for extracting nodes with significantly low accuracy on the capability tree (the algorithm introduced in Section 3.2). In the pseudocode, use SIZE to indicate the number of instances linked to node.
In our experiments, we use 0.05, 01 5, and 0
by default.
This framework supports various metrics and deviation directions by adjusting the meaning of total sample size" and "count of successes" in the statistical test step.
C Default Experimental Configurations
This section provides the experimental configurations used in Section 5.
C.1 Evaluation Results of LMs Across Different Benchmarks
For GPT-4o mini (OpenAI, 2024a) evaluation results on mathematics reasoning benchmarks, the generation ourselves; the system prompt is "Please solve math problem step-
19

---

Published as a conference paper at COLM 2025
System Prompt
Given set of phrases, each summarizing the skills capabilities needed to answer multiple-choice questions testing broad knowledge and reasoning within a specific group, generate gerund phrase that summarizes the collective set of skills capabilities described across all groups.
User Prompt ## Task
You are given a set of phrases, each summarizing the skills or capabilities needed to answer multiple-choice questions testing broad knowledge and reasoning within specific group. There are (group (groer_numbbrr groups total. Your task to "*summarize" the collective set of skills capabilities that represents the union of these descriptions in a detailed and informative manner.
## Skill Descriptions
{skill descriptions)
## Requirements
The output should be a "'single gerund phrase** that succinctly summarizes the overarching skill capability represented by the union of all the provided phrases. The output should comprehensively cover each skill description without going beyond them.
The output should not simply enumerate the given phrases but instead provide meaningful and informative summary of the skills or capabilities they collectively represent.
Please output **nly a gerund phrase** summarizing the skill or capability, with NO additional text.
Table 6: The capability description prompt for MMLU (Hendrycks et al., 2021a).
20

---

Published as a conference paper at COLM 2025
System Prompt
Given a set of phrases, each summarizing the coding skills or capabilities needed to solve code generation problems involving data science tasks within specific group, generate a phrase that encapsulates the common coding skill or capability required across all the groups. The overall description should comprehensively cover each skill description without going beyond them, avoiding generic terms.
User Prompt ## Task
You are given a set of phrases, each summarizing the coding skills or capabilities needed to solve code generation problems involving data science tasks within a specific group. There are {group number] groups in total. Your task is to *summarize*** the common coding skill capability that represents the union of these descriptions in detailed and informative manner.
## Skill Descriptions {skill descriipio))))
## Requirements
The output should be a **ingll phrase** that succinctly summarizes the overarching coding skill or capability shared across all groups. should not introduce any concepts outside of those described in the provided phrases and must remain informative.
Please output "only a phrase" summarizing the skill or capability, with no additional text. Any output other than phrase will NOT be accepted!
Table 7: The capability description prompt for the Python code generation benchmark (DS-1000 (Lai al., 2023)).
21

---

Published as a conference paper at COLM 2025
Algorithm 1 Extracting Nodes with Significantly Low Accuracy
Input: capability tree T, accuracy threshold {LM accuracy pre-computed at each node of T given the definition of capability tree)
Hyperparameter: minimum node size 01 and 02 confidence level a
Output: a set of extracted nodes R
Initialize R. <- Ø
Initialize map BINOMIALPASS {} (Stores the binomial test result for each node]
End of Initialization
First Pass: Binomial Test
Define recursive function l'ESTNoDE(Hode)::
Perform a binomial test on node with accuracy threshold and confidence level a if the accuracy is significantly below T at level a then
BINOMIALPASS podde < true
else
BINOMIALPASS//node <- false end if
for each child in node.children TESTNoDE(child)
end for
Call TEsoNo
End of First Pass
Second Pass: Node Extraction
Define recursive function EXTRACINODE(mode); if SIZF(node) > 01 and BINOMIAI.PASI/cccie true then Initialize alIChildren Pass +- true for each child in node.children do
if SSZEEchiill 2 02 and BINOMIALPASS(chidd = false then allChildrenPPas <- false
end if end for if alIChildrenPPass true then
Add node to Return (Skip its subtree to avoid overlap) end end if
for each child in node.children do EXTRACTNODD(child)
end for
Call EXTRACTNODDE(.roo))
Output R
End of Second Pass
22

---

Published as a conference paper at COLM 2025
System Prompt
Given set of phrases, each summarizing the skills or capabilities needed to respond to instructions within a specific group, generate gerund phrase that summarizes the collective set of skills or capabilities described across all groups.
User Prompt ## Task
You are given a set of phrases, each summarizing the skills or capabilities needed to respond to instructions within specific group. There are (group number) groups in total. Your task is to *summarize** the collective set of skills or capabilities that represents the union of these descriptions in detailed and informative manner.
## Skill Descriptions {skill descriptions}
## Requirements
The output should be a **inngle gerund phrase** that succinctly summarizes the overarching skill or capability represented by the union of all the provided phrases. The output should comprehensively cover each skill description without going beyond them.
The output should not simply enumerate the given phrases but instead provide a meaningful and informative summary of the skills capabilities they collectively represent.
Please output **nly gerund phrase** summarizing the skill or capability, with NO additional text.
Table 8:
The capability description prompt for the instruction-folloiing benchmarks (WildChat10K (Zhao et al., 2024a), ShareGPT10K, and Chatbot Arena (Chiang al., 2024)).
by-step. Break down each step logically and reason through intermediate steps until reaching the final solution." ", and the user prompt the question; we use apt-4o-mini-2024-97- and set the max new tokens and temperature to 1024 and 0.0, respectively. For Llama 3.1 8B Instruct (Dubey al., 2024) evaluation results, we also run the generation ourselves; we use the default system prompt, append the suffix "Please reason step by step, and puf your final answer within (booed to the question and set the max new tokens and temperature to 1024 and 0.0, respectively; the vLLM library (Kwon et al., 2023) is used to accelerate generation. Their generations are evaluated by our internal evaluation toolkit. We directly adopt AART-Math-Llama--e (Uniform) (Tong et al., 2024) evaluation results provided by the authors of its original paper.
For the evaluation results of all models on MMLU (Hendrycks et al., 2021a), we directly adopt the evaluation results provided by the authors of TÜLU 3 (Lambert et al., 2024).
MMLU (Hendrycks et al., 2021a) and CollegeMath (Tang et al., 2024) provide only the final answer to each question, but not the solution (reference output) needed for all weakness profiling methods. To address this, we take the response generated by GPT-4o mini as the reference output, which may have errors.
For DeepSeek- Boaeeek--ooer-BBaee 6.7B (Guo et al., 2024) evaluation result on DS-1000 (Lai et al.,
2023), we use the scripts provided by the DS-1000 GitHub repository' 5 for generation, with vLLM added to accelerate generation. For GPT-4o (OpenAI, 2024b) and GP11-355 Turbo (Ope- nAI, 2022) evaluation results, we directly evaluate the generations of gpt-4o-2024 2000-2222----66 and gpt-3. 5-turbo-0d 0613 provided by the GitHub repository. In both cases, we use the scripts provided by the DS-1000 GitHub repository for evaluation.
$nttps://github.com wttp:///...oo//o//a/oaaaaaa ai/DS-1000
23

---

Published as a conference paper at COLM 2025
To build the WildChat10K and ShareGPTI0K benchmarks, we start with the publicly leased versions of WildChat (Zhao et al., 2024a) and ShareGPT from HuggingFace Datasets ?; for both datasets, we keep only first-round conversations to collect instruction-response pairs, filter pairs where the combined length of the instruction and response exceeds 4096 Llama 3.2 tokens, and deduplicate the instructions; finally, we randomly sample 10K instruction-responsi pairs. For Chatbot Arena (Chiang et al., 2024), we use the publicly released version from HuggingFace Datasets for each instruction, we retain it only once and assign its reference output as the response from the strongest model (indicated by the overall ranking) for it; we finally have 44,230 instances in the Chatbot Arena benchmark.
In the instruction-followingg setup (Ouyang et al., 2022), where LMs respond to set of free-form user instructions, the responses are commonly evaluated using the LM-as-a-judge paradigm (Zheng et al., 2023; Dubois et al., 2023), in which a significantly stronger LM serves judge by comparing responses produced by two LMs to the same instruction to determine which one is better. This produces a win-rate for each LM, ranging from 0% to 100%, representing the proportion of instances where its response is chosen as the better one. A higher win-rate is generally interpreted as a signal of better overall performance. When using the LM-as-a-judge paradigm, we use 111-4-0-mi--222-077771
(OpenAI, 2024a) as the judge. The prompt for the LM judge is provided in Table 9, and we set the max new tokens and temperature 50 and 0.0, respectively. Following Zeng al. (2024), we compare each pair of responses to an instruction by querying the LM judge twice, swapping the order of the responses; this is due to potential positional bias (Wang et al., 2024a; Zeng et al., 2024), which can influence judgments based on the response order. For win-rate computation, we average the results of all comparisons. When using win-rate as the evaluation metric in the node extraction algorithm introduced in Section 3.2, the total sample size for the binomial test is twice the number of instances, and the count of successes corresponds to the number of times that one model's output is preferred not preferred.
When running Llama 3.2 3B Instruct (Meta, 2024) and Gemma 2 IT 2B (Rivière et al., 2024) instruction-following benchmarks (WildChatl0K, ShareGPTI0K, and Chatbot Arena), we use the default system prompt, directly use the instruction as the user prompt, and set the max new tokens and temperature to 4096 and 0.0, respectively. The vLLM library is also utilized to accelerate generation.
C.2 Profiling/Test Splits
In Sections 5.3, and 5.5, whenever the profiling and test sets originate from the same individual benchmark, we apply the following random profiling/ /test splits: the MATH benchmark was randomly partitioned into 4000/100 split, the MMLU benchmark into a 10042 (0042/4000 split, the DS-1000 benchmark into a 600/400 split, and the WildChat10K benchmark into an 8000 8000/2000 split to create the profiling and test sets. In Section 5.5, the full sets of benchmarks are used in the bensshben generalization setup.
D Implementation Details of Baseline Methods for Profiling LM Weaknesses
This section provides additional details about the implementation of baselines we assessed for profiling LM weaknesses, which are introduced in Section 4.
D.1 Implementation Details of TEXTDIFF
When sampling instances where the evaluated LM has succeeded/ /ereeedde//aiiedd,.. failed, the sampling pool consists of all instances where the evaluated LM's correctness is correct/ correct/incorrect incorrect
6WildChat: https://huggingfaaee http://ggaaa....aaaaaaeeeeoeeeee httpa///iw..eaeeettttttttttt///a
7ShareGPT: https://huggingfaac http:///ggggggeeeeooooocccs/t//a 2o//aaaaeets/ammm222111112/2//11 Vicuna unfil tered/
plob/main//ShareGPT V3 unfiltered cleaned split imsorry.
judgess
24

---

Published as a conference paper at COLM 2025
System Prompt
You are a helpful assistant in evaluating the quality of the outputs for a given instruction. Your goal to select the best output for the given instruction.
User Prompt
Select the Output (a) or Output (b) that is better for the given instruction. The two outputs generated by two different Al chatbots respectively.
Do NOT provide any explanation for your choice.
Do NOT say both neither are good. You should answer using ONLY ONLY "Output (a)" "Output (b)". Do NOT output any other words.
# Instruction: (instruction)
Output (a): (response_]]
# Output (b): (response 2}
# Which is better, Output (a) or Output (b)? Your response should be either "Output (a)" "Output (b)":
Table 9: The prompt for the LM judge.
for correctness-based accuracy, and for win-rate, all instances where the LM judge prefers the evaluated LM's response in both orders/dood not prefer the evaluated LM's response in either order (before and after swapping the response order; see Appendix C). In our experiments, we sample 50 failed instances and 50 successful instances due to the context length limit. We then prompt GPT--o (gpt-40-2024-04
(OpenAI, 2024b) the diag- nostic LM using the sampled 50+50=100 instances. The prompts for MATH, WildChat10K, and DS-1000 are provided in Table 10, 11, and 12, respectively. We set the max new tokens and temperature to 4096 and 0.0, respectively. The diagnostic LM is asked to identify 20 (potential) weaknesses given these sampled instances. Then, we determine the associated instances (in the profiling set) for each outputted potential weakness, following the im imple- mentation described in Appendix E.1. We finally compute the performance metric on the associated instances for each potential weakness and identify set of weaknesses the weakness profile by selecting those with the lowest performance metrics.
D.2 Implementation Details of QUALEVAL
As the authors of Murahari et (2024) have not released the code yet before we released this paper, implemented QUALEVAL ourselves based
scenario.
QUALEVAL starts with all instances in the benchmark, denoted as B. All instances are first randomly partitioned into [ chunks (we use k = 20 in all of our experiments) with each chunk size being no more than k, and each chunk fed to gpt-40-mini-2024--7-1 (OpenAT, 2024a) to summarize list of capabilities for instances in the chunk. The prompts used here for MATH, WildChat1OK, and DS-1000 are provided in Table 13, 14 and 15, respectively. We set the max tokens and temperature to 4096 and 0.0, respectively. We concatenate all capabilities generated for each chunk, getting a long list of capabilities for this benchmark.
We then iteratively shrink the list to get final list of capabilities (we our experiments). In each iteration, we split the list into multiple mp-size chunks (we use p 4 in our experiments), and prompt gpt 1ott-4o-mint miiii-2024-01 07-118 to shrink each chunk into
20 in
25

---

Published as a conference paper at COLM 2025
System Prompt
Given a set of mathematics questions and their corresponding correct solutions, identify the specific weaknesses of a model.
You provided with 50 mathematics questions that the model fails to solve and 50 mathematics questions that the model successfully solves. Based on this data, analyze and describe the model's weaknesses by identifying the high-level mathematical capabilities that the model struggles with. Group similar weaknesses under broader categories where applicable.
User Prompt ## Task
You are given 50 mathematics questions that the model fails to solve and their corresponding correct solutions, along with 50 questions that the model successfully solves. Analyze and describe the weaknesses of the model by identifying specific high-level mathematical capabilities it struggles with, summarizing any related weaknesses under broader categories.
## Questions and Solutions
### Failed Cases {negative inputs and outputs}
### Successful Cases
{positive inputs anwpuussann...m outputs)
## Requirements
*Output exactly 20 weaknesses. "
Each weakness should be an "informative and detailed phrase** that refers to *specific skill or capability** comprehensively covering key aspects of the failure, without including any specifics from the questions solutions.
Where possible, group related weaknesses under a single broader weakness category.
Output each capability as a standalone phrase, with **no additional text, prefixes, symbols, or notations on any line. For example, do NOT include numbered list markers, numerical prefixes, or numeric labels (e.g., '1.', 22', etc.) in the output.
Table 10: The diagnostic LM prompt for MATH (Hendrycks et al., 2021b) used by TEXTDIFF.
26

---

Published as a conference paper at COLM 2025
System Prompt
Given set of user instructions and their corresponding reference responses, identify the specific weaknesses of a model.
You are provided with 50 user instructions and their corresponding refer- ence responses that the model fails to address effectively, and 50 user instructions and their corresponding reference responses that the model addresses successfully. Based on this data, analyze and describe the model's weaknesses by identifying the high-level capabilities it struggles with. Group similar weaknesses under broader categories where applicable.
User Prompt ## Task
You given 50 user instructions and their corresponding reference responses that the model fails to address effectively, along with 50 user instructions and their corresponding reference responses that the model addresses successfully. Analyze and describe the weaknesses of the model by identifying specific high-level capabil- ities it struggles with, summarizing any related weaknesses under broader categories.
## User Instructions and Reference Responses
### Failed Cases
serpativeiipputs.adddoooo
### Successful Cases
(positive inputs and_aaddtpaa outputs)
## Requirements
**Output exactly 20 weaknesses.
Each weakness should be phrased as a specific capability, avoiding negative phrasing such as "lack," " "difficulty," " or similar terms.
Each weakness should be an *iinformative and detailed phrase** that refers *pecific skill or capability** comprehensively covering key aspects of the failure, without including any specifics from the instructions or reference responses.
Where possible, group related weaknesses under single broader weakness category.
Output each capability standalone phrase, with *'no additional text, prefixes, symbols, or notations on any line. For example, do NOT include numbered list markers, numerical prefixes, numeric labels (e.g., '2'' etc.) in the output.
Table 11: The diagnostic LM prompt for WildChat10K (Zhao et al., 2024a) used by DIFF
27

---

Published as a conference paper at COLM 2025
System Prompt
Given set of Python coding problems (involving data science) and their corre- sponding correct Python implementations, identify the specific weaknesses of model.
You are provided with 50 code generation problems that the model fails to solve and 50 code generation problems that the model successfully solves. Based on this data, analyze and describe the model's weaknesses by identifying the high-level coding capabilities (related to data science) that the model struggles with. Group similar weaknesses under broader categories where applicable.
User Prompt ## Task
You are given 50 Python coding problems (involving data science) that the model fails to solve and their corresponding correct Python implementations, along with 50 coding problems that the model successfully solves. Analyze and describe the weaknesses of the model by identifying specific high-level coding capabilities it struggles with, summarizing any related weaknesses under broader categories.
## Problems and Implementations
### Failed Cases
### Successful Cases
(positive inputs and_aad_ppppl outputs)
## Requirements
**Output exactly 20 weaknesses.
Each weakness should be phrased as a specific capability, avoiding negative phrasing such as "lack," " "difficulty," or similar terms.
Each weakness should be an *iinformative and detailed phrase** that refers *pecific skill or capability** comprehensively covering key aspects of the failure, without including any specifics from the code problems or implementations.
Where possible, group related weaknesses under a single broader weakness category.
Output each capability standalone phrase, with *'no additional text, prefixes, symbols, or notations on any line. For example, do NOT include numbered list markers, numerical prefixes, numeric labels (e.g., '1'' etc.) in the output.
Table 12: The diagnostic LM prompt for DS-1000 (Lai et al., 2023) used by TEXT
28

---

Published as a conference paper at COLM 2025
capabilities. The prompts used here for MATH, WildChat10K, and DS-1000 provided in Table 16, 17 and 18, respectively. We set the max new tokens and temperature to 4096 and 0.0, respectively. After multiple iterations, this finally ends up with capabilities.
After deriving m = 20 capabilities in natural language from all benchmark instances, QUAL- EVAL assigns relevance score to each pair of benchmark instances and capabilities, indicat- ing the relevance of the instance to the capability. The score is an integer ranging from to 5, where 5 indicates strong relevance and indicates no relevance. This is done by prompting gpt-4o-mini. miiin-2222-000 07-18 with each instance and the list of all derived capabilities, which outputs a list of scores for all castaiic-capabllity pairs for this instance. The prompts used here for MATH, WildChat10K, and DS-1000 are provided in Table 19, 20 and 21, respectively. We set the max new tokens and temperature 4096 and 0.0, respectively.
A ftter scoring each pair of benchmark instances and capabilities, QUALEVAL assigns each instance exactly 2 capabilities to maximize the sum of the relevance scores of the chosen pairs (instance and assigned capability). The assignment is constrained such that the number of instances assigned to each capability is roughly proportional to the sum its relevance scores across all instances. We use linear programming to perform the assignment, im- plemented with scipy. optimize......... linprog" Finally, QUALEVAL computes the performance metric for each capability, i.e., the performance metric on all its assigned instances, and identifies the capabilities with the lowest performance metrics as the weakness profile.
System Prompt
Given a set of mathematics questions and their corresponding correct solutions, identify the high-level mathematical capabilities required to solve these questions. Group similar capabilities where relevant.
User Prompt
## Task
You are given (instance num} mathematics questions and their corresponding correct solutions. Identify the high-level mathematical capabilities required to solve these questions, summarizing any related capabilities under broader categories.
## Questions and Solutions finputs antputsss
## Requirements
Each capability should be an **informative and detailed phrase** that refers *specific skill or capability** comprehensively covering key aspects of the solution, without including any specifics from the questions or solutions.
Where possible, group related capabilities under single broader capability.
Output each capability as a standalone phrase, with **no additional text, prefixes, symbols, notations** on any line.
Table 13: The capability initialization prompt for MATH (Hendrycks et al., 2021b) used by EVAL.
E
Experimental Details of Assessing Weakness Profiling Methods
This section provides additional details about Section 5.
E.1 Details of Determining Associated Instances
As described in Section 2.2, we prompt termine whether instance tests for given capability (if yes, the instance is called
(OpenAI, 2024a) to de-
9https://docs.ctt scipesss..p...... wwwp://w..ssiy//eeeeee//////eeee hyperparameters set their default values.
1inprog html. html. All
29

---

Published as a conference paper at COLM 2025
System Prompt
Given set of user instructions and their corresponding reference responses, identify the high-level capabilities required respond effectively to these instructions. Group similar capabilities where relevant.
User Prompt ## Task
You are given (instance num} user instructions and their corresponding reference responses. Identify the high-level capabilities required to respond effectively to these instructions, summarizing any related capabilities under broader categories.
## User Instructions and Reference Responses finputs.am and and.aaddddu.... outputs)
## Requirements
Each capability should be an **informative and detailed phrase** that refers to *specific skill or capability** comprehensively covering key aspects of the response, without including any specifics from the instructions reference responses.
Where possible, group related capabilities under single broader capability.
Output each capability as standalone phrase, with **no additional text, prefixes, symbols, or notations** on any line.
Table 14: The capability initialization prompt for WildChat10K (Zhao al., 2024a) used by QUALEVAL
System Prompt
Given set of Python coding problems and their corresponding correct implemen- tations, identify the high-level programming capabilities required to solve these problems. Group similar capabilities where relevant.
User Prompt ## Task
You are given finstance num] Python coding problems and their corresponding cor- rect implementations. Identify the high-level programming capabilities required to solve these problems, summarizing any related capabilities under broader categories.
## Problems and Implementations {inputs_ande outpuuts
## Requirements
Each capability should be an **informative and detailed phrase** that refers to *specific skill or capability** comprehensively covering key aspects of the solution, without including any specifics from the problems implementations.
Where possible, group related capabilities under single broader capability.
Output each capability as a standalone phrase, with **no additional text, prefixes, symbols, or notations** on any line.
Table 15:
The capability initialization prompt for DS-1000 (Lai et al., 2023) used by QUALE-
VAL.
30

---

Published as a conference paper at COLM 2025
System Prompt
Given list of mathematics capabilities, generate shorter list of the most critically relevant capabilities by combining related items where appropriate.
User Prompt ## Task
You are given {current num capabilities) mathematics capabilities. Generate list of no more than 20 capabilities by merging related capabilities into broader items where relevant.
## Capabilities (capability_
## Requirements
You should output *up to 20 capabilities** ideally exactly 20.
Each capability should be an *iiformmativ and concise phrase** that represents
sspecific skill or capability** while covering key aspects of the capabilities provided. Consolidate related capabilities into single, broader capability wherever possible to reduce the list length.
Output each capability standalone phrase, with *'no additional text, prefixes, symbols, or notations* on any line.
Table 16: QUALEVAL.
The capability shrinking prompt for MATH (Hendrycks et al., 2021b) used by
System Prompt
Given list of capabilities required for responding to user instructions, generate shorter list of the most critically relevant capabilities by combining related items where appropriate.
User Prompt ## Task
You are given _ap.aabiiiiiiiiis capabilities related to responding to user instructions. Generate list of no more than 20 capabilities by merging related capabilities into broader items where relevant.
## Capabilities (capability list)
## Requirements
You should output **up to 20 capabilities*,, ideally exactly 20.
Each capability should be an *informative and concise phrase** that represents sspecific skill or capability** while covering key aspects of the capabilities provided. Consolidate related capabilities into a single, broader capability wherever possible to reduce the list length.
Output each capability as a standalone phrase, with "'no additional text, prefixes, symbols, or n on any line.
Table 17: The capability shrinking prompt for WildChat10K (Zhao et al., 2024a) used by QUALEVAL.
31

---

Published as a conference paper at COLM 2025
System Prompt
Given a list of capabilities required for solving Python coding problems, generate shorter list of the most critically relevant capabilities by combining related items where appropriate.
User Prompt ## Task
You are given (current.num. ) capabilities related to solving Python coding problems. Generate list of more than 20 capabilities by merging related capabilities into broader items where relevant.
## Capabilities {capability list}
## Requirements
You should output *up to 20 capabilities** ideally exactly 20.
Each capability should be an *inffrrmatvv and concise phrase** that represents
**specific programming skill or capability** while covering key aspects of the capabilities provided.
Consolidate related capabilities into single, broader capability wherever possible to reduce the list length.
Output each capability as standalone phrase, with **no additional text, prefixes, symbols, or notations* on any line.
Table 18: The capability shrinking prompt for DS-1000 (Lai et al., 2023) used by QUALEVAL.
associated instance), which basic operation used in our assessments and TIFF The prompts used here for MATH and WildChat10K are provided in Table 22 and Table 23, respectively; we also provide the prompt for DS-1000 in Table 24, used in experiments of Section 5.3. We set the max new tokens and temperature to 128 and 0.0, respectively.
E.2 Qualitative Analysis of Low-Performance Identification Assessment
Table 25 presents the identified weaknesses from TEXTDIFF, QUALEVAL, and EVALTREE when the weakness profile size is 10, along with the LM performance on the associated instances (in the test set) of each identified weakness; they are based applying the three methods to Llama 3.1 8B Instruct (Dubey et al., 2024) evaluation result on MATH (see Section 5.1). We observe that EvALTREr-identified weakness descriptions are generally more specific than those identified by the other two methods, enabling a more precise diagnosis and thus capturing capabilities where the LM exhibits lower performance.
E.3 Experimental Details of Ground- Weakness Assessment
E.3.1 Details of the Assessment Setup
This subsection provides additional details about the setup of Ground-Truth Weakness Assessment Section 5.2, based on the setup introduced Section 2.2.
We used two benchmarks as testbeds, the MATH benchmark (Hendrycks et al., 2021b) and the WildChat10K benchmark (Zhao al., 2024a). As described above, we manually curated set of 10 ground-trui weaknesses (described in natural language) at diverse granularities as the ground-truth weakness profile, for MATH and WildChatlOK, respectively. The ground- truth weakness profiles for MATH and WildChat10K are provided in Table 26 and Table 27, denoted as W*. We aim to generate a synthetic evaluation result (on the profiling set) where the actual weaknesses are exactly this predefined ground-truth weakness profile W First, we identify the associated instances for each ground-truth weakness. We then define two hyperparameters, the hase probability p (0, and the decrease rate d (0, 1),
32

---

Published as a conference paper at COLM 2025
System Prompt
Given mathematics question with its solution and numbered list of mathematical capabilities, rate each capability on a scale of 1-5 to indicate its relevance in solving this question. A score of 5 means the capability is very used, while 1 means it is not used at all.
User Prompt ## Task
You are given a mathematics question and solution, along with a list of 20 mathematical capabilities. For each capability, rate the degree to which it is required to solve this question.
## Question {input}
## Solution {output}
## Capabilities (capability
## Requirements
For each capability, provide integer **score from to 5** A score of 5 means the capability is very used, while means it is not used at all.
Include brief *"reasoning** for each score, explaining how you determined the score.
Output the result in **JSON format" follows:
json
"1": ("reasoning": "THE REASONING", "score": SCORE),
"2": { ("reasoning": "THE REASONING". "score": SCORE),
"3": ("reasoning"; "THE REASONING", "score": SCORE),
}
Do NOT include any additional text outside of the JSON format, as ""I will directly use 'oson.loads' in Python to convert your output to dictionary object**.
Table 19: The scoring prompt for MATH (Hendrycks et al., 2021b) used by QUALEVAL
33

---

Published as a conference paper at COLM 2025
System Prompt
Given user instruction with its reference response and numbered list of capabili- ties, rate each capability on a scale of 1-3 to indicate its relevance in responding to this instruction. A score of 5 means the capability is very used, while means it is not used at all.
User Prompt ## Task
You are given a user instruction and its reference response, along with a list of 20 capabilities. For each capability, rate the degree to which it is required to respond to this instruction.
## User Instruction {input}
## Reference Response {output}
## Capabilities (capability
## Requirements
For each capability, provide integer **score from to 5** A score of 5 means the capability is very used, while means it is not used at all.
Include brief *"reasoning** for each score, explaining how you determined the score.
Output the result in **JSON format" follows:
json
"1": ("reasoning": "THE REASONING", "score": SCORE),
"2": { ("reasoning": "THE REASONING". "score": SCORE),
"3": ("reasoning"; "THE REASONING", "score": SCORE),
}
Do NOT include any additional text outside of the JSON format, as ""I will directly use 'oson.loads' in Python to convert your output to dictionary object"*.
Table 20: The scoring prompt for WildChat10K (Zhao et al., 2024a) used by QUALEVAL.
34

---

Published as a conference paper at COLM 2025
System Prompt
Given Python coding problem with its correct implementation and numbered list of capabilities, rate each capability on a scale of 1-5 to indicate its relevance in solving this problem. A score of means the capability is very used, while means is not used at all.
User Prompt ## Task
You are given a Python coding problem and its correct implementation, along with list of 20 capabilities. For each capability, rate the degree to which it is required to solve this problem.
## Coding Problem {input}
## Correct Implementation {output}
## Capabilities (capability
## Requirements
For each capability, provide integer **score from to 5** A score of 5 means the capability is very used, while means it is not used at all.
Include brief *"reasoning** for each score, explaining how you determined the score.
Output the result in **JSON format" follows:
json
"1": ("reasoning": "THE REASONING", "score": SCORE),
"2": { ("reasoning": "THE REASONING". "score": SCORE),
"3": ("reasoning"; "THE REASONING", "score": SCORE),
}
Do NOT include any additional text outside of the JSON format, as ""I will directly use 'json.loads' in Python to convert your output to dictionary object**.
Table 21: The scoring prompt for DS-1000 (Lai et al., 2023) used by QUALEVAL.
35

---

Published as a conference paper at COLM 2025
System Prompt
Given mathematical question and its correct solution, check whether the provided mathematics skill or capability is required by the key aspects of the solution.
User Prompt
## Question
[input]
## Solution
{output}
## Skill or Capability (capability)
## Requirement
the provided mathematics skill or capability required by the key aspects of the solution, output YES. Otherwise, output NO. You should output either YES or NO with no additional text, otherwise, the output will NOT be accepted.
Table 22:
The prompt for determining whether or not a given MATH (Hendrycks et al., 2021b) benchmark instance tests for given capability.
System Prompt
Given user instruction and reference response to the instruction, check whether the provided skill or capability is required by the key aspects of responding to the instruction.
User Prompt ## Instruction {input}
## Response
{output}
## Skill or Capability (capability)
## Requirement
If the provided skill or capability is required by the key aspects of responding to the instruction, output YES, Ötherwise, output NO. You should output either YES or NO with no additional text, otherwise, the output will NOT be accepted.
Table 23: The prompt for determining whether not given WildChat10K (Zhao et al., 2024a) benchmark instance tests for given capability.
36

---

Published as a conference paper at COLM 2025
System Prompt
Given a Python coding problem (involving data science) and its correct Python implementation, check whether the provided coding skill or capability is required by the key aspects of the implementation.
User Prompt
## Problem [input]
## Implementation {output)
## Skill or Capability (capability)
## Requirement
the provided coding skill or capability required by the key aspects of the implementation, output YES. Otherwise, output NO. You should output either YES or NO with no additional text, otherwise, the output will NOT be accepted.
Table 24:
The prompt for determining whether or not given DS-1000 (Lai et al., 2023) benchmark instance tests for given capability.
for controlling the sampling process. Taking correctness-based accuracy an example, for the i-th benchmark instance, we compute the probability of it being solved correctly (i.e., Pig: = 1)) p x d'n, where is the number of ground-truth weaknesses in W for which the instance is an associated instance. Finally, we independently sample correctness 0) for each gi using these computed probabilities, resulting synthetic evaluation result (on the profiling set). By design, the ground-truth weakness profile W exactly represents the real weaknesses for this generated synthetic evaluation result, as we were mimicking the evaluation behavior of hypothetical LM with exactly these weaknesses. As we described above, when using correctness-based accuracy as the metric for MATH, p d represents the probability of an instance's evaluation result being correct. Similarly, when using win-rate as the metric for WildChat10K, p x d denotes the probability of the (hypothetic) evaluated LM being preferred by the LM judge; specifically, we simulate the judge's preference by sampling twice, once for the original order of responses and once after swapping their order (see Appendix C). For each benchmark, we generated three synthetic evaluation results using the hyperparameters p = 0.7 and d E {0.2, 0.4, 0.5).
Given weakness profile W generated by method, we measure its similarity to W We de- fine "Precision" as Ew;=w [A(w;) n (Un, ew A(w;) ) (w;)]/| to measure desideratum
1, i.e, how precisely identified weaknesses align with ground-truth ones; similarly, we de- fine "Recall" as =w ((w (to)))((te)))//AA
[W to measure desideratum
2, i.e., how comprehensively ground-truth weaknesses are covered; finally, their harmonic mean, F1, provides a balanced measurement. By default, we use the profiling set itself as the test set for computing A in the formulas above; also show the results of using separate test set distinct from the profiling set in Appendix E.3.3.
E.3.2 Analysis on Experimental Results
This subsection provides additional analysis on the experimental results in Section 5.2.
To better understand why TEXTDIFF and QUALEVAL are outperformed, we show the Precision and Recall curves in Figure 11 and 12. These curves show that both methods suffer from poor Precision, indicating that the weaknesses they identify cannot precisely pinpoint where the LM fails. We present the identified weaknesses from TEXTDIFF, QUAL.EVAL, and
37

---

Published as a conference paper at COLM 2025
Method
Weakness Profile
Solving complex trigonometric equations and identities. (11.11%)
Handling and solving inequalities involving multiple variables. (18.18%)
Solving problems involving optimization and maximizing or minimizing expressions. (15.79%)
Understanding and applying properties of circles and their tangents. (30.0%)
Understanding and applying properties of vectors and vector operations. (46.34%)
TEXTDIFF
Understanding and applying properties of matrices and determinants. (60.0%)
Handling and solving problems involving complex numbers and their operations. (44.44%)
Understanding and applying geometric transformations and properties. (50.0%)
Understanding and applying properties of polynomials and their roots. (36.77%)
Applying the Pythagorean theorem and properties of right triangles. (41.46%)
Applying optimization techniques and inequalities problem-solving (22.22%)
Utilizing properties of geometric figures, including transformations and conic
sections (41.07%)
Analyzing sequences, series, and their properties (34.92%)
Analyzing and solving inequalities and systems of equations (37.31%)
Calculating combinations, permutationss tions, and applying counting principles (47.54%)
QUALEVAL
Applying vector operations and understanding geometric interpretations (45.45%)
Employing logical reasoning and problem-solving strategies (48.89%)
Calculating areas, volumes, and perimeters of geometric shapes (28.57%)
Understanding and manipulating complex numbers and their properties (44.44%) Understanding and applying properties of functions, including logarithmie, exponential, and trigonometric functions (34.57%)
Analyzing and applying geometric properties, relationships, and transformations across various contexts and configurations. (37.71%)
Analyzing and applying geometric reasoning to understand spatial relationships and calculate dimensions in two- and three-dimensional contexts. (35.05%)
Analyzing and applying recursive relationships and mathematical sequences to identify patterns and solve combinatorial problems. (25.0%)
Analyzing and manipulating numerical properties and representations across
various numeral systems. (46.53%)
Analyzing and manipulating polynomial equations their complex roots evaluate relationships and distances. (16.67%)
EVALTREE
Analyzing and optimizing geometric relationships using trigonometric principles and the Triangle Inequality. (5.56%)
Analyzing polynomial relationships and roots using Vieta's 's formulas and complex number properties. (14.81%)
Applying quadratic equations and trigonometric principles to solve for variable values and integer solutions. (0.0%)
Formulating, analyzing, and applying combinatorial reasoning to evaluate mathematical relationships and count objects under constraints. (40.0%) Optimizing mathematical expressions and relationships through analysis, inequalities, and constraints. (26.11%)
Table 25: Weakness profiles generated by TIFTDIFF, QUALEVAL, and EVALTREE, along with the LM performance on the associated instances (in the test set) of each identified weakness. Methods are run on Llama 3.1 8B Instruct (Dubey et al., 2024) evaluation result MATH (Hendrycks al., 2021b).
38

---

Published as a conference paper at COLM 2025
EVALTREE when the weakness profile size is 10 in Table 28, along with their corresponding Precision, Recall, and F1; they are based on applying the three methods to the synthetic evaluation result generated for the MATH benchmark, with the probability hyperparameters set to p = 0.7 and d = 0.2. We observe that EVAL.RRE achieves significantly higher Precision compared to the other two methods, while maintaining quite high Recall, indicating that EVALTREE can more precisely pinpoint specific areas where the LM underperforms and thus better satisfy desideratum 1. For example, EVALTREE identified the weakness " "Analyzing and applying relationships among polynomial expressions and their roots using Vieta's 's formulas, " which closely aligns with the ground -oouud weakness "Solving polynomial equations by analyzing relationships through Viela's formulas;" in contrast, TEXTDIFF and QUALEVAL identified two much coarser-grained weaknesses, "Handling problems involving the properties of polynomials and their roots" and "Solving linear, polynomial, and quadratic equations, including factoring and roots" respectively, failing to capture the critical aspect of Vieta's 's formulas.
This example shows the advantage of EVALTREE modeling the capabilities tested within a benchmark at diverse granularities. By contrast, QUALÉVAL, relying on a single-level categorization, can only represent fixed-granularity structure, which fails to sufficiently model the intricate and interrelated structure of capabilities tested within a benchmark. Con- sequently, it fails to capture the nuanced performance of LMs on fine-grained capabilities, leading to its inability to detect granular weaknesses. In contrast, EVALTTREF successfully models the complexity of capabilities tested within a benchmark by the hierarchical struc- ture of capability trees; this lets us analyze capabilities at varying granularities flexibly, from broad categories specific skills. By incorporating this flexibility, EVALTREE captures much more detailed and comprehensive information about LM performance, so it can be superior.
Index
Capability Description
1
Solving problems involving complex numbers and trigonometric identities, including the use of algebraic manipulation, polar forms, and exponentiation of complex numbers.
2
Analyzing combinatorial problems using counting principles and recurrence relations to count and analyze complex arrangements. Applying geometric formulas to calculate areas, volumes, and other properties of three-dimensional shapes.
3
4
Analyzing numbers using prime factorization to solve problems involving divis- ibility and coprimality.
5
Solving probability problems using geometric probability.
6
Solving polynomial equations by analyzing relationships through Vieta's formu- las.
7
Using trigonometric identities and polynomial identities reduce complex expressions.
8
Involving geometric partitioning or area considerations calculate probabilities. Analyzing quadratic inequalities through factoring. Applying the properties of divisibility to find common factors using the Greatest Common Divisor.
9
10
Table 26: The manually curated ground-truth weakness profile for MATH (Hendrycks et al., 2021b), used in Ground-Truth Weakness Assessment (Section 5.2).
E.3.3 Computing F1 on Separate Set
In this subsection, we present the results of Section 5.2 using separate test set (distinct from the profiling set) for computing A in the formulas provided in Appendix E.
Here, for the MATH benchmark (Hendrycks et al., 2021b), the test set is its released training set (consisting of 7,500 instances). For WildChat10K, we sample another 10K instances from WildChat (Zhao et al., 2024a) as the test set, using the same construction process as the profiling set (WildChat10K) and ensuring no overlap with WildChat10K by excluding previously included instances. The results, shown in Figure 13, demonstrate consistent observations with those observed on the original results in Figure
39

---

Published as a conference paper at COLM 2025
Index
Capability Description
1
Proficiency in designing intuitive, user-friendly interfaces.
2
Proficiency in editing and proofreading for academic papers.
3
Financial forecasting and risk analysis. concepts.
4
Proficiency in understanding and/or /or utilizing object-oriented programming
5
Game mechanics design and balancing.
6
Crisis communication management by media response crafting. Synthesis of statistical analysis and data interpretation for business purposes. Helping the users with their own mental health.
7
8
9
Evaluating complex moral dilemmas and proposing socially responsible solu-
tions.
10
Event planning by logistical coordination.
Table 27: The manually curated ground-truth weakness profile for WildChat10K (Zhao et al., 2024a), used in Ground-Trute Weakness Assessment (Section 5.2).
E.4 Experimental Details of Extrinsic Assessment
This section provides additional details about Section 5.3.
(OpenAI, 2024a) in our experiments to generate (synthetic) data inputs; the input generation prompts for MATH (Hendrycks et al., 2021b) and DS-1000 (Lai et al., 2023) are provided in Table 29 and Table 30, respectively; max new tokens and temperature to 4096 and 1.0 (for generation diversity), respectively. We also use gpt-40-mini-220 -22---2--2222---118888---88---88 to generate outputs for each collected input; the output generation prompts for MATH and DS-1000 are provided in Table 31 and Table 32, respectively; we set the max new tokens and temperature to 4096 and 0.0, respectively.
We use OpenAl's got-4o-mini--224-997222
set the
For the ceneric-capabilityyguudee benchmark's overall targeted capability as guidance (in the input generation prompt) for synthetic data generation. The descriptions are "General mathematical reasoning capability across Prealgebra, Algebra, Number Theory, Counting and Probability, Geometry, and Intermediate Algebra." and "General Python coding capability across data science libraries: NumPy, Pandas, TensorFlow, PyTorch, SciPy, Scikit-learn, and Matplotlib." " for MATH and DS-1000, respectively.
data collection strategy, we use a description of the
For the EVAL.TREF-gudde data collection strategy, we set the accuracy threshold т to 0.4 in the node extraction algorithm described in Section 3.2. This resulted in identified weaknesses for MATH and 5 for DS-1000; the same number of weaknesses was identified when using the TEXTDIFF-guides strategy and the QUAL QUALEvAL-guided strategy, ensuring that all weakness-guided data collection strategies use weakness profiles of the same size. When sampling five in-context examples for input generation given an identified weakness in weakness- gussed data collection strategy, the examples are sampled from the associated instances (in the profiling set) of the identified weakness in the T'XTDDiF-guided strategy, from the instances assigned to the identified weakness in the QUALEVAL-guided strategy, and from the instances linked to the corresponding node in the EVALTREE-guidee strategy.
We provide an example of synthetic data inputs generated for Llama 3.1 8B Instruct on MATH. One EVALTREE-identifie weakness is "Analyzing and optimizing geometric rela- tionships using trigonometric principles and the Triangle Inequality." " A synthetic data input generated under the guidance of this weakness is "I triangle ABC, the lengths of sides AB and AC are 15 em and 20 cm, respectively. If angle measures 60', what is the length of side BC rounded to the nearest whole number?" In contrast, synthetic data input guided by the generic capability is "A trader bought a certain number of apples for $0.75 each and then sold them for $1.00 each. If he had total profit of $15 after selling all the apples, how many apples did he sell?" This example highlights that EVALTREE provides targeted guidance for data collection.
For each data collection strategy, we collect 128 instance inputs for training. We finetune the models using LoRA (Hu et al., 2022), with a rank of 256, an alpha of 512, and a dropout rate of 0.1. The batch size is fixed at 8, and the maximum sequence length is set to 1024
40

---

Published as a conference paper at COLM 2025
Method
Weakness Profile
Solving problems involving the properties of prime numbers and their factorizations
Solving equations involving trigonometric identities and simplifications Handling complex numbers and their operations
Solving problems involving combinatorics and permutations
Applying the Law of Cosines and Law of Sines in non-right triangles
TEXTDIFF
Handling problems involving the calculation of probabilities and combinatorial counting
Handling problems involving the calculation of areas and volumes of geometric shapes
Handling problems involving the properties polynomials and their roots Understanding and applying the properties of quadratic equations and their roots
Handling problems involving divisibility and modular arithmetic
Understanding and applying number theory concepts, including prime factorization and modular arithmetic
Understanding and manipulating complex numbers and their properties Calculating combinations, permutations, and applying counting principles Calculating areas, volumes, and perimeters of geometric shapes Calculating probabilities and utilizing statistical methods for data analysis Employing logical reasoning and problem-solving strategies
QUALEVAL
Understanding and applying properties of functions, including logarithmic, exponential, and trigonometric functions Solving linear, polynomial, and quadratic equations, including factoring and roots
Applying optimization techniques and inequalities in problem-solving Analyzing and solving inequalities and systems of equations
Simplifying and solving trigonometric and complex expressions using algebraic manipulation, identities, and properties of periodic functions Manipulating complex numbers and applying series and binomial techniques to derive geometric properties
Analyzing and calculating complex numbers through polar coordinates, polynomial equations, and algebraic manipulation Analyzing and applying relationships among polynomial expressions and their roots using Vieta's formulas
Solving and manipulating algebraic, quadratic, and probability equations Analyzing and applying prime factorization, divisibility, and the relationships between greatest common divisors and least common multiples to solve mathematical problems
EVALTREE
Analyzing and calculating prime factorization and divisibility within factorials Analyzing and calculating prime numbers and whole numbers through factorization and divisor techniques
Factoring integers and polynomials to analyze prime components, apply properties of exponents, and identify valid combinations Calculating and analyzing geometric properties and volumes of three-dimensions shapes using formulas and algebraic manipulation
Table 28: Weakness profiles generated by DIFF, QUALEVAL, and EVALTREE. TEXTDIFF
achieves a Precision of 0.4787, Recall of 0. 0.9450, and an F1 of 0.6355. QUALEVAL achieves Precision of 0.3494, Recall of 0.9975, and an F1 of 0.5175. EVALTREE achieves Precision of 0.7064, Recall of 0.8081, and an Fl of 0.7538. Methods are run on the synthetic evaluation result generated for the MATH (Hendrycks et al., 2021b) benchmark, with = 0.7 and 0.2. The ground-truth weakness profile is provided in Table 26.
41

---

Published as a conference paper at COLM 2025
System Prompt
You are a creative and logical assistant tasked with generating new mathematics ques- tions. Your goal is create a single, clear question aligned with given mathematical capability.
User Prompt ## Task
Generate one unique mathematics question demonstrating the following capability:
{capability)
Please ensure the following:
You will be given {instance example questions for reference. Use the examples solely to understand the capability, NOT as templates, i.e, the generated question must not replicate, paraphrase, directly resemble the example questions in structure, wording, or context.
The question must ask for only one result, such as numerical value, while adhering to logical constraints (e.g,, quantities must be positive, and counts for people must be integers).
## Provided Examples (example inputs}
## Requirements
Do NOT include a solution in the generated question.
Ensure the question is plausible, reasonable, and relevant to the given capability.
Table 29:
The (synthetic data) input generation prompt for MATH (Hendrycks et al., 2021b).
tokens. Training is conducted using BF16 precision. The optimizer is configured with learning rate of 1E-4, a cosine learning rate scheduler, a warmup ratio of 0.1, and no weight decay. The models are trained for and 2 epochs in the experiments MATH and DS-1000, respectively. These configurations are applied consistently across all experiments.
E.5 Details of LM Usage Costs
Let the number of benchmark instances (the size of profiling set) be denoted as N.
The main LM usage cost of EVALTREE is incurred during the Capability Annotation stage, where each instance requires one LM call, and the Capability Description stage, where each non-leaf node of the capability tree also requires one LM call. The cost of the sentence embedding model used in the Capability Embedding stage is negligible in comparison. As the number of non-leaf nodes in the capability tree is smaller than N, the total number of LM calls and thus the overall LM usage cost for EVALTREE scale as O(N).
For TEXT the main LM usage cost is incurred when determining the associated in- stances for each potential weakness outputted by the diagnostic LM. Each potential weak- ness requires O(N) LM calls, causing the total number of LM calls and thus the overall LM usage cost to scale linearly with the number of potential weaknesses outputted by the diagnostic LM, which is the upper bound of the weakness profile size.
For QUALEVAL, the main LM usage cost comes from scoring each pair benchmark instances and capabilities derived from all benchmark instances. The scoring LM generates a natural language reasoning for each score (see prompts in Appendix D.2), making the output token cost significant component of the total cost. Since the length of the LM's output scales linearly with the predefined number of capabilities (which is the upper bound of the weakness profile size), the overall LM usage cost (roughly) scales accordingly.
42

---

Published as a conference paper at COLM 2025
System Prompt
You are creative and logical assistant tasked with generating new Python program- ming problems. Your goal is to create a single, clear problem aligned with given data science capability.
User Prompt
## Task
Generate one unique Python programming problem demonstrating the following capability: {capability)
Please ensure the following:
You will be given finstance_num num} example problems for reference. Use the examples solely understand the capability and the desired problem format. The generated problem must not replicate, paraphrase, or directly resemble the example problems in structure, wording, or context.
The problem must ask for one piece of Python code that fills in a blank, ensuring clarity and conciseness while being grounded in real-world data science scenarios.
## Provided Examples
-nnputs
## Requirements
Do NOT include a solution the generated problem. Please output the generated problem directly, without any additional text, explanation, commentary.
Ensure the problem is plausible, reasonable, and relevant to the given capability.
Adhere to logical programming constraints, such correct syntax and realistic
data or outcomes.
Table 30: The (synthetic data) input generation prompt for DS-1000 (Lai et al., 2023).
System Prompt
You are precise and logical assistant. Solve the following mathematics problem step by step, explaining each step clearly. Enclose the final answer to the mathematics question within {boxed()
User Prompt {input}
Table 31: The output generation prompt for MATH (Hendrycks et al., 2021b).
System Prompt
Write short code following the given format and indentation. Place the executable code between <code>> and </code> tags, without any other non-executable things. Please provide ONLY the code completion needed. Do NOT repeat the context code.
User Prompt (input)
Table 32: The output generation prompt for DS-1000 (Lai et al., 2023).
43

---

Published as a conference paper at COLM 2025
As analyzed above, the scale coefficients of TIFT and QUALEVAL grow linearly with the (maximum) weakness profile size, making their costs significantly higher than EVALTREE, which maintains linear cost scaling with the number of benchmark instances regardless of the weakness profile size. This difference makes EVAL.TREE substantially more cost-efficient in terms of LM usage cost, especially when the weakness profile size is large.
F Quantitative Analysis of Flaws in Chatbot Arena's Evaluation Practice
This section provides additional quantitative analysis of the flaws in Chatbot Arena's human- voter-based evaluation practice, discussed in Section 6. We use the OpenAl Moderation AP with the model omni moniiiimdraaiinn200000 is a tool that evaluates whether or not a given text contains toxic content.
to assess toxicity in the following; this
We first examine the user instructions for instances linked to the node "Facilitating inclu- sive, ethical, and strategic communication and engagement across diverse and sensitive contexts". Across the entire Chatbot Arena benchmark, 4.72% of instances have toxic user instructions; however, at this specific node, the proportion rises sharply to 19.50%. It is worth noting that people found that the OpenAl Moderation API may have low recall (Zhao et al., 2024a), resulting in numerous false negatives (toxic instructions not flagged as such), so the actual proportion of toxic user instructions should be higher. Despite this limitation, the observed toxicity rate at this node is significantly higher than the benchmark average, confirming that contains disproportionate number of user instructions with toxic requests, which aligns with the natural language description of the capability represented by the node.
We then examine the trend of human voter preferences when comparing two responses, one providing a toxic response and the other providing non-toxic response (often by refusing to answer). We focus on human comparison pairs where one response is flagged as toxic and the other is not. Across all such comparison pairs, the proportion where the toxic response is preferred is 50.89%; when also counting "tie" cases to consider all cases where the non-toxic response is not preferred, the proportion rises to 71.98%. This issue is even more serious at the node "Facilitating inclusive, ethical, and strategic communication and engagement acmoss diverse and sensitive contexts", among comparison pairs for the node's 's instructions, these two numbers rise significantly to 86.84% and 97 37%, respectively. These results confirm the observation that human voters tend to prefer toxic responses (that do not refuse answer), diverging from the intended values. They underscore the need for careful refinement of evaluation practices to ensure alignment with the desired principles.
G Ablation Study: Alternative Approach to Tree Construction
In this section, we explore an alternative approach to the tree construction pipeline in- troduced in Section 3.1. In this approach, we still follow the four-stage pipeline. For the stage (3), instead of recursively building the hierarchical structure in a top-down, recur- sive way, we use the hierarchical clustering algorithm (Müllner, 2011), implemented with scipy. cluster. hierarchy. linkage" The other stages remain unchanged. We did not adopt this approach because it always produces a binary tree, where the optimal number of each node's s children could be more than two and diverse; a binary tree cannot meet this need, whereas our default approach can automatically determine (potentially) optimal number of children at each node. We also empirically observed that trees constructed by hierarchical clustering sometimes have unbalanced structures; for example, the left subtree of the root may contain very few instances while the right subtree contains many.
We compare EVALTREE using the default capability tree construction pipeline with EVAL- TREE using the capability tree built with the hierarchical clustering algorithm in the experi-
10https://platfon
Unttps: scipy. wpppaaa..c/oooossppirereeeeeeeee
cluster hierarchy. linkage. html. The method is set to average, the metric to cosine, and all other hyperparameters are set to their default values.
44

---

Published as a conference paper at COLM 2025
MATH
DS-1000
Initial LM
48.70
29.20
EVALTREE
52.42(#0.228
36.90(+0.34)
EVALTREE (Hierarchical Clustering)
52.88(=0.65)
33.36(+0.36)
Table 33: Accuracy (%) of different LMs on MATH and DS-1000 test sets. The initial LM is Llama 3.1 8B Instruct (Dubey et al., 2024) for MATH and DeepSeek-Coder-Ban 6.7B (Guo al., 2024) for DS-1000, respectively. See Section 5.3 for the experimental setup. We compare EVALTREF using the default capability tree construction pipeline with EVALTREF using the capability tree built with the hierarchical clustering algorithm here. Synthetic data (used to train the initial LM) are generated under the guidance of the weakness profiles produced by the two versions of EVALTREE, respectively. The accuracy (of a trained LM) is reported as mean=stderr ("stderr" refers standard error) across five random seeds.
mental setup of Sections 5.1, 5.2 and 5.3. The results, shown in Figure 14 and 15 and Table 33, show that the default version outperforms the hierarchical-clusterigg baaaeeeeeeeg-- version.
45

---

Published as a conference paper at COLM 2025
GPT-4o mini
Laaa 3.1 68 Instruct
DART-Math- Lama--l1 (Uniform))
30
80
25
30
50
100
1000
Asvencr lemers CP1
Saaehcrall Gorogg
Ledau 20.000
Figure 6: Accuracy curves of weakness instances and strength instances (from the test set) extracted using the random profiling/ /tallitng///tttt /tstt split of the MATH benchmark (Hendrycks et al., 2021b). Experiments were conducted with GPT-4o mini (OpenAI, 2024a), Llama 3.1 8B Instruct (Dubey et al., 2024), and All Instances" in the legend refers to all instances the test set. A y x line included in all figures to indicate the threshold t. The number of weakness strength instances is shown as a reference; when the number is very low, the curve may exhibit significant fluctuations, affecting the general trend.
(Uniform) (Tong et al., 2024).
GPT-4o mini
Llama 3.1 SB Instruct
TÜLU 3B
vontoria Whak sould
Accoart Thenkeid
37
1300
30
60
70
1-x00
M-30 Jelamo 30.001
Stadke Elrragh Selomon.
Figure 7: Accuracy curves of weakness instances and strength instances (from the test set) extracted using the random profiling/ pooillnng/estt test split of the MMLU benchmark (Hendrycks et al., 2021a). Experiments were conducted with GPT-4o mini (OpenAI, 2024a), Llama 3.1 8B Instruct (Dubey et al., 2024), and TÜLU 8B (Lambert et al., 2024). "All Instances" in the legend refers to all instances in the test set. A the threshold T. The number of weakness/ /stt//gtrrgthhhe instances is shown as reference; when the number is very low, the curve may exhibit significant fluctuations, affecting the general trend.
line is included in all figures to indicate
46

---

Published as a conference paper at COLM 2025
GPT-4o
GFT-3,5 Turbo
Turbo
DeepSeck- Coder Base 6.70
Scmeay Wadeome sanuos
Weal framire
-20
30-
Intenn Shond
Tuadio Hewgh
Nae anst
Figure 8: Accuracy curves of weakness instances and strength instances (from the test set) extracted using the random profiling/ split of the DS-1000 benchmark (Lai et al., 2023). Experiments were conducted with GPT-4o (OpenAT, 2024b), GPT-3.5 Turbo (OpenAI, 2022), and DeepSeek-Coder Coder-Base 6.7B (Guo et al., 2024). " All Instances" in the legend refers to all instances in the test set. A y = x line is included in all figures to indicate the threshold T. The number of weakness/ / strength instances is shown as reference; when the number is very low, the curve may exhibit significant fluctuations, affecting the general trend.
GPT-4o mini
Llama 3.1 8B Instruct
DART-Math-Llamass (Uniform)
1000
20
25
103
to bnboee sioun
lode 20am)
beleow (hoond
230
100
30
17
Figure Accuracy curves of weakness instances and strength instances (from the test set) extracted using the MATH benchmark (Hendrycks et al., 2021b) as the profiling set and the CollegeMath benchmark (lang et al., 2024) as the test set. Experiments were conducted with GP'T-4o mini (OpenAI, 2024a), Llama 3.1 8B Instruct (Dubey et al., 2024), and DART- Math-Llama3-SB (Uniform) (Tong et al., 2024). "All Instances" " in the legend refers all instances in the test set. Note that the y x line of the threshold used in the node extraction algorithm is not drawn here, as comparing accuracies with the threshold directly is not meaningful due to the differing distributions of the profiling and test sets, which are from two different benchmarks. The number of weakness/ /ttrenghh instances is shown as reference; when the number is very low, the curve may exhibit significant fluctuations, affecting the general trend.
47

---

Published as a conference paper at COLM 2025
(ID)
ShareGFT1oK (OOD)
2330
50-
1330
404
1331
2000
+3000
branon 30
27-
30
80
1330
10000
4000
1300
65.0
dators All HLN00
20000
Figure 10: (a) Win-rate curves of weakness instances and strength instances (from the test set) extracted using the random profiling/ test split of the WildChat10K benchmark (Zhao et al., 2024a). (b) Win-rate curves of weakness instances and strength instances (from the test set) extracted using the WildChat10K benchmark as the profiling set, with the ShareGPT10K and Chatbot Arena (Chiang al., 2024) benchmarks serving as the respective test sets. The win-rate refers to the win-rate of Llama 3.2 3B Instruct (Meta, 2024) compared to Gemma IT 2B (Rivière et al., 2024), as evaluated by the LM judge (Zheng et al., 2023; Dubois et al., 2023). "ID" indicates that the profiling and test sets are from the same benchmark (WildChat10K), whereas "OOD" indicates that they are from different benchmarks. The number of weakness/ peakness/strengthh strength instances is shown reference; when the number is very low, the curve may exhibit significant fluctuations, affecting the general trend.
d = 0.2
= 0.4
d = 0.5
MATH Prediiin
0.5
08
10 Size of Weakness Profile (Ad)
00
Size of Weakmess Profile
Size of Weakness Prodile (Ad))
WildChatlOK Preesion
0.6
Size of Weakness Profile (M²)
Size of Weakness Profile (M²)
Size of Weakness Profile (M²)
TEXT
QUALEVAL
EVALTREE
Figure 11:
Precision score curves of TEXTDIFF, QUALEVAL, and EVALTREE, with the weakness profile size varying from 1 to 20. d is hyperparameter to control the sampling probability (see Appendix E.3.1).
48

---

Published as a conference paper at COLM 2025
0.2
0.4
0.5
MATHH Rocall
06-
04
Stze Weakness Proflee
Size Profile
Weakness Proflee
WildChattOK Kocall
0.5
0.7
15 15 17 Size of Weakness Profile (M²)
00 10 14 1s 16:20
10 15 15
Size of Weakness Profile (M)
Size of Weakness Prodie (M2)
TIFF
QUALEVAL
EVALTREE
Figure 12: Recall score curves of TEXTDIFF, QUALEVAL, and EVALTREE, with the weakness profile size varying from 1 to 20. d is hyperparameter to control the sampling probability (see Appendix E.3.1).
0.2
0.4
MATHI
05-
13:1 Stzee of Weakness Profiee (AM))
23 13
Size of Weakness Profile (M)
Sizce of Weakness Profile (M²)
W5idChato
0.5
05-
02
0.2
Weakness Profile
Profile
Weakness Profile
TEXIDIFF
QUALEVAL
EVALTREE
Figure 13: F1 score curves of TEXTDIFF, QUALEVAL, and EVALTREE, with the weakness profile size varying from 20. Precision, Recall, and thus F1 (more specifically, A in the formulas provided in Appendix E.3.1) are computed profiling set used to generate the synthetic evaluation results. horizontal line indicates each method's highest score. d is hyperparameter to control the sampling probability.
separate test set, distinct from the
49

---

Published as a conference paper at COLM 2025
Llama Instruct
DART-Math-Elmas-BB (Uniform)
Llama 3.2 Instruct
Accracy
37-
Win-RRate 00
WURDIR Size Weakness Proflee (M²
Size Weakness Profile (M)
Size Weekness Profile (M)
Actuacy
36-
30-
503
1003
Number Associated Instances (N')
Number Associsted Instances (N'
Number of Associated Instances (N)
(a) MATH
WildChatlOK
EVALTREE
EVALTREE (Hlstcchicalluuseriingg
Accuracy/ Win-Rate on All Instances
Figure 14: Curves of min (Ew F(A(#:))/ VT, |Wr] 2 M' (the first row) and min(F(S,) VT, [S 2 N'} (the second row). See Section 5.1 for the experimental setup. Experiments in (a) were conducted on MATH with Llama 3.1 8B Instruct (Dubey et al., 2024) and DAKI-Math-Llama--58 (Uniform) (Tong al., 2024), and experiments in (b) were conducted on WildChat10K, where the win-rate is the percentage of instances in which Llama 3.2 3B Instruct (Meta, 2024) preferred over Gemma IT 2B (Rivière et al., 2024). We compare EVALTREE using the default capability tree construction pipeline with EVALTREE using the capability tree built with the hierarchical clustering algorithm here.
0.2
0.4
0.5
MATH
05-
10 25
12
Stoe of Weakness Profile (M²)
Size of Weekkees Profile (M)
Size of Weakness Profile (M
WhitChat1KK
03
of Weakness Profile
Size
Weakiess Profile
Weakness Profile
EVALTREE
EVALTREE (Hierachical Clustering)
F1 score curves of EVALTREE using two different capability tree construction pipelines, with the weakness profile size varying from to 20. See Section 5.2 for the experi- mental setup. A horizontal line indicates each method's highest score. d is hyperparameter to control the sampling probability (see Appendix E.3.1). We compare EVALTREE using the default capability tree construction pipeline with EVALTRRE using the capability tree built with the hierarchical clustering algorithm here.
Figure 15:
50


Reviews:
[{"review_id": "5Nh1mJWNsh", "reviewer": "Reviewer_SQGa", "summary": "This paper introduces EVALTREE, a method for weakness profiling of LMs by constructing hierarchical capability trees. The central aim is to move beyond aggregate benchmark scores and provide interpretable, actionable insights into where a model fails and how it can be improved. Each node in the capability tree represents a natural-language-described capability, linked to a subset of benchmark instances. EVALTREE identifies nodes with statistically significant underperformance to produce a weakness profile.\nThe method is novel and ambitious, combining prompting-based capability labeling, sentence embeddings, recursive clustering, and statistical testing. It is validated across diverse benchmarks (MATH, WildChat10K, DS-1000), and its utility is demonstrated through weakness-guided data collection. An interactive demo is also provided to help practitioners explore and interpret the capability tree.\nThe paper makes a compelling case for the importance of interpretable, capability-based diagnostics of LM capabilities and weaknesses. The tree nodes can be somewhat verbose, but this verbosity enables granular and specific insights into model behavior. Some sections could benefit from more methodological detail (e.g., clustering sensitivity, embedding choices), potentially at the expense of moving secondary applications (e.g., Chatbot Arena critique) into the appendix.\nThe paper focuses heavily on benchmarking EvalTree against other weakness profiling methods (TEXTDIFF, QUALEVAL), but could benefit from a stronger framing around its primary strength as a diagnostic framework.", "questions": "", "limitations": "", "rating": 9, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The paper introduces a strong diagnostic framework for weakness profiling, separating it from traditional leaderboard-style metrics and grounding it in interpretable analysis.\n- EvalTree-guided training improves LM accuracy more than generic or category-based data collection, demonstrating real downstream and practical value.\n- The method offers better performance at a lower inference cost than TEXTDIFF and QUALEVAL, especially as profile size grows.\n- The interactive demo allows users to explore the capability tree for provided examples and inspect node-level performance. This is a valuable contribution that can improve practical adoption.\n- As LM evaluation evolves beyond static benchmarks, EvalTree provides a more interpretable, flexible, and fine-grained alternative, making it relevant for both academic and applied NLP settings.", "reasons_to_reject": "- Capability labels and embeddings are LM-generated, which may reduce consistency if the models used for generating node descriptions and embeddings are not fixed across platforms. Limitations similar to LLM-as-a-judge methods, such as sensitivity to prompt phrasing and model bias, apply here as well.\n- The robustness of tree construction is not evaluated. There is no analysis of how the tree structure or node descriptions vary across different embedding models or prompting templates. Sensitivity or ablation studies would strengthen the method's reliability.\n- The method currently analyzes all instances, including correctly answered ones. Under compute or budget constraints, it may be more efficient to focus only on failed instances for profiling.\n- The baselines (TEXTDIFF and QUALEVAL) from section 4 could be moved into Related Work to streamline the presentation and free space for more detailed methodological explanation.\n- The abstract could more clearly reflect the structure of the method: first, capability tree construction; then, weakness profile extraction and comparison with other approaches.", "questions_to_authors": "- How do embedding choices affect tree structure and profile output? Please provide sensitivity analyses or ablations. \n- Can EvalTree be adapted for multi-capability instances? This would improve coverage for realistic tasks found in dialogue or multiturn evaluations. Perhaps task decomposition could help here.\n- Do you plan to release code for reproducibility and further adoption?\n- It would be useful to validate the capability tree against known labels, for example, comparing nodes up to the second layer with MMLU subcategories. There seems to be a placeholder for this in the demo.\n- For comparing models more directly, it could be helpful to extend the tree visualization to include per-node rankings over a model set.\n\nNotes:\n- Please clarify that the output shown in Figure 2 refers to the ground-truth solution used in the performance metric.\n- The paper does not cite related literature on LLM-as-a-judge methods, which is relevant given the prompting-based design used for capability analysis.\n- TEXTDIFF and QUALEVAL, while used as baselines, are not mentioned in the Related Work section and should be cited there.\n- The Elo rating system, used in part of the analysis, should cite Arpad Elo.\n- Consider mentioning TreeEval to avoid confusion with similarly named evaluation methods.", "extra_scores": {}}, {"review_id": "3pOhjejziV", "reviewer": "Reviewer_DECS", "summary": "The paper proposes an evaluation method to generate a weakness profile tree for LLMs. This helps in understanding what type of new data to collect to improve the model. The enhanced data collection process is shown to improve model performance is comparison to baselines.\n\nThe main motivation is that an evaluation benchmark is heterogenous and is meant to assess diverse model capabilities and varying granularities. Using a single aggregate metric doesn't reflect a diverse and detailed evaluation. This is a limitation of current model evaluations.", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The paper proposes a well thought out an implemented solution to investigate model performance on the diverse data points in a benchmark. The proposed solution EvalTree is compared to existing baselines like TextDiff and QualEval and the reasons for developing this solution as an improvement over related work is well motivated. All the experimental details for the tree creation are provided which is a big plus for reproducibility. I hope the code for tree creation is made public. The compute cost comparison with existing baselines is favorable for the proposed solution.", "reasons_to_reject": "No strong reasons for rejection.", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "3YP7F0r2f9", "reviewer": "Reviewer_n857", "summary": "This paper introduces EvalTree, a method for profiling the weakness of a language model by automatically constructing a hierarchical capability tree over any benchmark and each node represents a capability described in natural language and is linked to a subset of instances that evaluate the specific capability.\n\nThe research demonstrates EvalTree's superiority over baseline methods in precision and comprehensiveness through experiments on benchmarks like MATH and WildChat. Furthermore, it shows the practical significance of this method by using EvalTree-identified weaknesses to guide more effective data collection for LM improvement  and by exposing flaws in existing evaluation practices like Chatbot Arena. The provision of an interactive interface further underscores its utility and potential for broader impact in understanding and enhancing LM capabilities.\n\nA related but **not cited** work could be Domino [1], which also aims to identify and describe model failure modes.\n\nDomino use cross-modal embeddings to discover and describe systematic error \"slices\" in broader AI applications (often vision/audio) via **natural language**, EvalTree is distinct in its hierarchical, capability-driven approach tailored for LMs. While both aim for interpretable error analysis, EvalTree focuses on structured, multi-level capability failures within LM tasks to guide improvement, whereas Domino leverages cross-modal concept understanding to identify coherent error-prone data segments.\n\n[1] Domino: Discovering Systematic Errors with Cross-Modal Embeddings", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. This work tackles a critical challenge in AI: understanding why and where LMs fail, rather than just measuring overall performance. EvalTree provides a more precise, comprehensive, and actionable approach to diagnosing LM failures, offering human-interpretable insights that can directly aid developers in targeted model improvement. \n2. EvalTree is novel in its hierarchical, capability-driven approach. The EvalTree methodology, with its four-stage tree construction pipeline (Capability Annotation, Capability Embedding, Recursive Clustering-Based Construction, and Capability Description), is clearly defined and technically sound.\n3. The authors conduct comprehensive experiments on multiple benchmarks (e.g., MATH, WildChat) and LMs, employing their proposed quantitative assessments to compare EvalTree against relevant baselines (TextDiff, QualEval). The demonstrated success of weakness-guided data collection suggests a more efficient path to enhancing LM capabilities.\n4. The application of EvalTree profiles to guide targeted data collection for LM improvement and to expose flaws in established evaluation practices like Chatbot Arena also represents original contributions. The public availability of an interactive interface for exploring capability trees further lowers the barrier for adoption and future research, promising a significant impact on how LMs are evaluated and iteratively improved.", "reasons_to_reject": "1. Although the overall superiority of EvalTree over TextDiff and QualEval is shown, it is unclear to me how each design choice/element in EvalTree contributes to the final improvement. Does the improvement primarily stem from the hierarchical versus flat structure, or from other design choices? I would like to see more detailed ablation studies to gain a better understanding of the proposed method.\n2. The ground-truth weakness assessment relies on synthetic LM evaluation results and manually curated weaknesses, which may not accurately reflect the complexity and subtlety of real-world LM failure modes. Is it possible to construct such ground truth on real-world datasets?\n3. Additionally, the choice of baselines (TextDiff, QualEval), while representative, might omit comparisons with other potentially stronger or more relevant error analysis or slice discovery techniques that could provide a more challenging benchmark for EvalTree's performance. More baselines could further strengthen the paper [1, 2, 3].\n\n[1] Domino: Discovering Systematic Errors with Cross-Modal Embeddings\n[2] Discover, Explanation, Improvement: An Automatic Slice Detection Framework for Natural Language Processing\n[3] Beyond Accuracy: Behavioral Testing of NLP models with CheckList", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "wwiOBNPQ60", "reviewer": "Reviewer_pMeM", "summary": "This paper presents a highly intriguing research direction, namely the generation of LLM weakness profiles, which not only facilitates fine-grained analysis of LLM weaknesses but also enables targeted enhancement. To address this task, the authors propose an EvalTree framework for the automated synthesis of LLM weakness profiles from any chosen benchmarks. The study includes extensive experimental work and analysis to validate the effectiveness of EvalTree, and the final analysis on the chat arena proves particularly insightful and engaging.", "questions": "", "limitations": "", "rating": 10, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The research presented in this paper is highly compelling and holds significant value for both academia and industry.​​\n​2. The methodology employed in this study is straightforward yet effective.​​\n​3. Extensive experimental analyses validate the efficacy of the proposed approach, yielding numerous insightful findings and perspectives.​​", "reasons_to_reject": "From my personal perspective, this paper exhibits no apparent flaws. I particularly appreciate both the research problem it addresses and the methodological approach it proposes.​​", "questions_to_authors": "None", "extra_scores": {}}]

Author rebuttal:

Meta-review: The paper proposes to investigate a model's errors and suggest what is needed to be improved in it in a granular manner, termed "weakness profiling". In addition, the paper proposes a method to perform such profiling.
The paper was uniformly acclaimed by all reviewers in a manner that is rare.
While reviewers do suggest improvements, those stand for me as a sign that nothing is ever perfect, and I do not see a reason to reject the paper.
Decision: Accept