EvalTree: Profiling Language Model Weaknesses via Hierarchical
  Capability Trees
Authors: name: Zhiyuan Zeng, name: Yizhong Wang, name: Hannaneh Hajishirzi, name: Pang Wei Koh

An ideal model evaluation should achieve two goals: identifying where the
model fails and providing actionable improvement guidance. Toward these goals
for Language Model (LM) evaluations, we formulate the problem of generating a
weakness profile, a set of weaknesses expressed in natural language, given an
LM's performance on every individual instance in a benchmark. We introduce a
suite of quantitative assessments to compare different weakness profiling
methods. We also propose a weakness profiling method EvalTree. It constructs a
capability tree where each node represents a capability described in natural
language and is linked to a subset of benchmark instances that specifically
evaluate this capability; it then extracts nodes where the LM performs poorly
to generate a weakness profile. On the MATH and WildChat benchmarks, we show
that EvalTree outperforms baseline weakness profiling methods by identifying
weaknesses more precisely and comprehensively. Weakness profiling further
enables weakness-guided data collection, and training data collection guided by
EvalTree-identified weaknesses improves LM performance more than other data
collection strategies. We also show how EvalTree exposes flaws in Chatbot
Arena's human-voter-based evaluation practice. To facilitate future work, we
release our code and an interface that allows practitioners to interactively
explore the capability trees built by EvalTree.