Evaluating Language Models' Evaluations of Games
Authors: name: Katherine M. Collins, name: Cedegao E. Zhang, name: Graham Todd, name: Lance Ying, name: Mauricio Barba da Costa, name: Ryan Liu, name: Prafull Sharma, name: Adrian Weller, name: Ionatan Kuperwajs, name: Lionel Wong, name: Joshua B. Tenenbaum, name: Thomas L. Griffiths

Reasoning is not just about solving problems -- it is also about evaluating
which problems are worth solving at all. Evaluations of artificial intelligence
(AI) systems primarily focused on problem solving, historically by studying how
models play games such as chess and Go. In this paper, we advocate for a new
paradigm that assesses AI systems' evaluation of games. First, we introduce a
formalism for evaluating such evaluations. We then leverage a large-scale
dataset of over 100 novel board games and over 450 human judgments to compare
evaluations produced by modern language and reasoning models against those of
people and symbolic computational agents. We consider two kinds of evaluative
queries: assessing the payoff (or fairness) and the funness of games. These
queries span two dimensions relevant to the design of evaluations of AI
evaluations: how complex a query is to compute and how difficult a query is to
quantify. Our results show that reasoning models are generally more aligned to
people in their evaluations of games than non-reasoning language models.
However, we observe a non-monotonic relationship: as models get closer to
game-theoretic optimal, their fit to human data weakens. We also observe more
"jaggedness" across models for assessing funness, in line with the greater
difficulty of quantifying this query. Across queries and games, reasoning
models show highly variable and unpredictable resource usage when assessing
queries, pointing to the importance of imbuing more resource-rational
meta-reasoning in language and reasoning models.