Pseudointelligence: A Unifying Lens on Language Model Evaluation
Authors: Shikhar Murty
Orr Paradise
Pratyusha Sharma
Venue: EMNLP 2023 Findings (2023)
Keywords: Evaluation
Large Language Models
Learning Theory
Computational Complexity

Abstract: With large language models surpassing human performance on an increasing number of benchmarks, we must take a principled approach for targeted evaluation of model capabilities. Inspired by pseudorandomness, we propose pseudointelligence, which captures the maxim that "(perceived) intelligence lies in the eye of the beholder." That is, that claims of intelligence are meaningful only when their evaluator is taken into account. Concretely, we propose a complexity-theoretic framework of model evaluation cast as a dynamic interaction between a model and a learned evaluator.  We demonstrate that this framework can be used to reason about two case studies in language model evaluation, as well as analyze existing evaluation methods.

Pseudointelligence:
A Unifying Framework for Language Model Evaluation
Shikhar Murty"
Orr Paradise"
Pratyusha Sharma" MIT pratyusha@mit.edn edu
Stanford University smurtyêcs. stanford.eds edu
UC Berkeley
Abstract
al., 2021), and benchmarks leaking into training data (Elangovan et al., 2021).
With large language models surpassing human performance on an increasing number of bench- marks, we must take principled approach for targeted evaluation of model capabilities. In- spired by pscudorandonness, we propose pseu- dointelligence, which captures the maxim that "(perceived) intelligence lies in the eye of the beholder." That is, that claims of intelligence meaningful only when their evaluator is taken into account. Concretely, we propose complexity-thooee framework of model eval- uation cast dynamic interaction between model and a learned evaluator. We demonstrate that this framework two case studies in language model evaluation. well as analyze existing evaluation methods.
Instead, we define the notion of pseudointelli- gence. Analogous to pseudorandomness (Blum and Micali, 1984; Yao, 1982), which measures a distri- bution by its distinguishability from true random- ness, pseudointelligence applies to the evaluation of the capabilities of learned models. Importantly, a claim that a model has learned a certain capability innately entangled with the distinguishing ability of an evaluator.
With the future of NLP in mind, we focus on learned evaluators. These evaluators are trained on samples specific to a given capability, much like the models they assess. Notably, emerging evaluation methods, such as model-based evaluation (Perez et al., 2023; Ribeiro et al., 2021) and adversarial evaluation (Jia and Liang, 2017; Nie et al., 2020; Bartolo et al., 2020), can be viewed as specific instances of the framework we propose. Our main takeaways are:
be used reason about
1 Introduction
Recent works claim that GPT-4 achieves expert- level performance on complex reasoning tasks (Katz et al., 2023; Lin et al., 2023), with some researchers concluding that it exhibits sparks of intelligence (Bubeck et al., 2023).
P1:
A claim of intelligence must be supplemented by explicitly-effined evaluator and (intelli- gent) capabilities (Section 3.1).
But how should intelligence be evaluated? This question dates back to Descartes (1637), formal- ized by Turing (1950), and continues to be the sub- ject of recent discussion (Chollet 2019; Mitchell and Krakauer 2023; Burnell et al. 2023 inter alia). However, none of these attempts prescribe a par- ticular evaluator (c.g., sequence of questions) that guarantees the intelligence of the evaluated model.
P2: Increased resources dedicated to model devel- opment should be accompanied by increased resources dedicated to evaluation. These in- clude the number of examples of the capabil- ity, and the complexity of the space of possible models and evaluators (Section 3.2).
This not a coincidence. We argue that intelli- gence is the eye of the evaluator. This maxim is particularly important for the future of natural language processing (NLP): progress cannot be measured by static benchmarks (Raji et al., 2021; Hutchinson et al., 2022; Shirali et al., 2023), with contemporary models surpassing human perfor- mance on new evaluations within a few years (Kiela
P3: Self-evaluation cannot support a claim of in- telligence the evaluator directly derived from the model. might, however, be useful as means towards a different end (Section 3.3).
Besides laying the foundation for theoretical analysis, our framework also provides a unifying lens on existing evaluation methods (Section 4).
"Equal contribution. Authors listed alphabetically.

---

Capability
p
e
g
e
m
M
Capability class
8(A
m(x,)
accepe/neject
(5)
aucepi/reect
(6)
g
Model
L
Model learner
6)
Model sample complexity
For all AY, with probability -
(3) Train
(1) Sample
L
Evaluator
L
Evaluator learne
(4) dist (g, p)) <
μ
(1) Sample
n = nie, 6)
Evaluatio sample complesity
Ap
(2) Trate
L
63
rle,
Evalustion round complexity
Figure I: Targeted evaluation of pseudointelligent model. For each capability (1) iid samples are drawn and (2) fed the learners, which (3) output model and evaluator. (4) The distinction dist, μi) is computed the expected difference in evaluator output during multi-round interaction with (5) the model g versus (6) the ground-truth capability μa. (7) If dist, (. ()) with probability' greater than (1 - 6), we say that Lg is pseudointelligent against Tg w.r.t capabilitiess M. See Definition 3.2 for formal definition. Note that the targeted
evaluator is trained on samples from the capability μ. and adaptively interacts with the model
2 Background
distribution over X' is e-pscudorandom against class of distinguishers D for every d D,
First, a brief introduction pseudorandomness, which forms the conceptual backbone of our framework. For an extended intro- duction, see Goldreich (2008).
Pseudorandomness.
Pr [d(x) accepts] ac
Pr [d(x) accepts 2t Ux
Tessa and Obi are playing game, and would like decide who gets go first. They agree make the decision based a coin toss: Tessa tosses coin, and Obi calls Heads or Tails. Obi calls the outcome correctly gets to go first, and otherwise Tessa does. Now consider two cases:
One view Definition 2.1 consisting of an ideal source (uniformly random elements), and pseudoideal approximation this source (pseu- dorandom elements). Unlike randomness, intel- ligence does not have a canonical mathematical operationalization.
1. Obi is calling the coin based only the infor- mation available him from eyesight.
2. Obi has access to array of sensors that mea- the initial conditions of Tessa's s coin toss, and powerful computer that can perform complicated calculations millisecond.
The Turing Test. In the Turing Test (Turing, 1950), an evaluator converses with either machine or a human; the machine attempts to convince the evaluator that it is human, while the evaluator aims to distinguish machine from human. the machine successfully fools the evaluator, Turing argued that should be considered as exhibiting intelligent behavior. However, while passing the Turing test signifies that the machine is indistinguishable from human by particular evaluator, it alone does not imply human-level learning comprehension (in- dependent of evaluator). Pseudointelligence is defined with this intuition in mind: however, it ex- plicitly requires specifying the particular evaluator and (intelligent) capabilities against which the ma- chine is measured.
Tessa would not be happy with a coin toss in the second case, because Obi could call the coin cor-
rectly with ease. In other words, the coin toss is no longer "random-enough" due Obi's increased computational power. More generally, a distribu- tion is pseudorandom against a particular observer if she cannot distinguish it from a truly random. Formally.
Definition 2.1. Fix (0, 1) and finite set X. Let denote the uniform distribution over X. A
'Over the samples from and any randomness used by the learners Lg, Lgg, model and evaluator e.

---

3 Pseudointelligence
9 from the (ground-truth) capability μ. However, intelligence is not the same as possessing a partic- ular capability (Gunderson and Gunderson, 2008). Rather, we view it as an ability to learn various capabilities. Thus, we consider a learner Lgg that learns a model E G from finite samples of μ..
Our main message (P1) is that claims of intelli- gence should center the evaluator, and not just the (allegedly) intelligent model. Put differently, a claim that a model is intelligent is actually claim that it is "intelligent-enoughh therefore it is meaningful only with respect specific class of evaluators. We provide a complexity-theorete framework in which evaluators are placed front and center, formalizing Figure 1.
We will say that the learner is pseudointelliient if, with high probability, the evaluator cannot distin- guish between the learned model and the capability, Lastly, allow for targeted evaluation of the capa- bility, we consider an evaluator learner LE that is also given (different) samples from the capability, and outputs an evaluator e E E targeted at it.
3.1 Setup
A model is a (possibly randomized) mapping g: X + V, where X' is a set of queries and Y is a set of responses.
Definition 3.2 (Pseudointellliennce) Fix query set X, response set y,, and class of capabilities M. Fix sample complexity functions m, n: (0, 1)2 - N. Given a model class g = (G, LG, m) and an evaluator class & = (E, Lg: n), we say that g is pseudointelligent with respect to E and capabilities M if, for any E, E (0, 1), whenever L. (resp. Lg) is given == m(=,6) (resp. = n(E, 6)) iid samples from μa, with probability at least 1 6,² LG and LE output model y and evaluator e such that e cannot e-distinguish between g and j.:
Capabilities. A capability is a distribution μ over X' x V. For given query E X, we let p(x)) denote a sample from the conditional distribution on acceptable responses yo( | z); thus, can be thought of as the ground-truth randomized mapping ::: X' + y against which models are evaluated.
Evaluators. In this work, study the perceived intelligence of a model. That is, how well model appears to posses certain capabilities as perceived by an evaluator? We formalize this by considering an evaluator e which is an algorithm that is given black-box access to the model g; for cach of i E [r] rounds, the evaluator queries 9 on 2; to receive response yı finally, the evaluator "accepts" g if it thinks it is the ground-truth capability, and rejects it otherwise. Note that the query x; may depend on previous responses y1,....:88 Vi-1-
Vp E M
Pr geLoo Lg
(dist 1)) s 1-1 6.
Note that the number of rounds of interaction between the evaluator and the model (denoted == r(E,8) in Figure 1), also scales with and 8. Next, examine two case-studies to under- stand the effect of the implicit parameters in Defi- nition 3.2 on the validity of claims of intelligence.
The degree to which an evaluator e is able to distinguish between the model and a (ground- truth) capability μ is defined next.
3.2 Model resources vs. evaluator resources
Our main message (P2) underscores the importance of resources allocated to the evaluator relative to those allocated to the model. There are several axes on which this comparison can be made:
Definition 3.1 (Distinction). Let be an evaluator, g: X' J/ be model and ya be capability over X x y. For any e (0, 1), we say that e can e E-distinguish between g and i
Samples. To evaluate capabilities M within er- ror 8 and distinction E, the model learner is given m(s, 5) samples and the evaluator learner is given n=E, samples of each capability μ E M. How do each of these grow as a function of and e?
Pr [e accepts g] - Pr accepts ul]l > e.
dist. (g.14
If diste (g. S E then we say that e cannot distinguish between g and μ.
Learner expressivity. The model learner LG out- puts model g E G, and the evaluator learner Lg outputs an evaluator E E. How expressive is the class of possible models G as compared to the
The distinction diste (g, (4) captures the likeli- hood that an evaluator distinguishes a given model
WW prefer evaluator over benchmark us it emphasizes its role as an active participant in an interaction, rather than a passive dutasct.
Ibidd., 1.

---

class of possible evaluators E? A naive measure of expressivity compares the number of parame- ters needed to encode each: log [ vs. log E) Supervised learning theory has more refined mea- sures that can be applied to infinite spaces and provide tighter bounds (Natarajan, 1989; Daniely and Shalev-Shwarde 2014). While these measures can be applied to the model class, new measures must be developed to capture evaluator classes.
should receive independent samples from p. This is in stark contrast the existing practice of deriving the evaluator directly from the trained model 9 + eg (Kadavath et al., 2022; Saunders et al., 2022; Zhang et al., 2023). Our main mes- sage (P3) is that this does not show that LG is beeudointelligent-alttoooo
it may be useful means towards different end, in Kadavath et al. (2022); Saunders al. (2022).
Learner compute resources. How much compu- tational power is used to train LG and Lg? Note that learner expressivity is concerned only with the existence of model 9 E G that is indistinguish- able by the evaluator, but not with how to find it. This search takes compute resources; the amount of resources available LG vs. Lg affects the outcome of the evaluation.
Existing evaluations through the lens of pseudointelligence
Pseudointelligence can serve unifying role by al- lowing a direct comparison between different eval- uation methods. We cast several existing evaluation paradigms into our framework.
Model and evaluator computational power. Given a query X, how much computational power is needed to compute response ( On the evaluator side, how much power is needed compute the ith query issued by the evaluator, given the preceding (i - 1) queries and responses? Additionally, given a full evaluation (Ir,, y1 )= how much power is needed by the evaluator to decide whether it accepts?
Static Datasets. The evaluator memorizes sam- ples drawn from the capability, and queries its black box on random sample: Given samples S < μ², LE outputs an evaluator es that draws sample (x, v) + S at random, queries the black box on 2r and accepts if and only if the response Clearly, like all inductive inference settings, an evaluator can be fooled by any pseudo-intelligen model that just happens to get the correct labels by learning simple shortcuts.
y-
3.3 Should a model evaluate itself?
One particularly interesting case is when the model is pitted against itself by playing a dual rule: both model and evaluator. Self-evaluation can be used to assist human evaluators (Saunders et al., 2022) or to increase model "honesty" (Kadavath et al., 2022). The validity of self-evaluation for claims of intelligence remains contested (cf. Zhang et al., 2023 and the discussion around it), and is the focus of this case study.
Adversarial Evaluation (AE). AE requires ac- cess to some auxiliary model g that LE can use to search for a challenge test set, which can then be used by an evaluator. Concretely, given seed samples S and an auxiliary model g. Lg filters out all examples where 9 outputs the correct response, thereby creating a challenge test set S. Such a fil- tering process can be done several rounds, where human annotators modify an initial query until Å makes an error (Bartolo et al., 2022). Intuitively, based on the quality of ÿ. such filtering can cre- ate increasingly hard datasets. Thus, the central resources here are the amount of seed samples S and the complexity of the auxiliary model g.
To consider self-evaluation in our framework, we first map models onto evaluators g -> eg. Once such a mapping is fixed, we map a model learner LG to an evaluator learner LEg that, given samples S + ,r, computes + LG(S) and outputs ég. Can L-c be pseudointelligent with respect to LEG? ? This is akin to asking whether LG is pseu- dointelligent with respect to itself. This brings crucial detail of our framework: For self- evaluation fit in our framework, T.gg and LG
Model Evaluation. These evaluators also auxiliary 9. albeit in non-adi non-adversarial way. For instance, Ribeiro et al. (2021) use human- generated templates, filled in by a language model, as queries. Perez models: one to generate queries, and the other to find those targeted at a particular capability.
'For example, consider the case that models yes-no ques- tions ()) = {0. 1}). Then one can obtain an evaluator e, from a model hy sampling a query æ, querying the black box to receive a response y,, and accepting if and only if g(x:) = y-
(2023) use two auxiliary

---

5 Conclusion
Al: investigating adversarial human annotation for reading comprehension. Trans. Assoc. Comput. Lin- guistics, 8:662-678.
This paper introduces a principled framework for model evaluation, inspired by the theory of pseu- dorandomness. Our main message is that claims about model capability must be supplemented with a thorough discussion of the resources given to the evaluator, especially in settings where model re- sources are largely unknown (e.g. OpenAl, 2023). Central to our framework a model-based evalua- tor that is targeted at specific capabilities as well as specific models (via multi-round interactions). We hope our framework encourages rigorous analysis of LLM evaluation, and helps unify the study of this increasingly-importan topic.
Max Bartolo. Tristan Thrush, Sebastian Riedel, Pontus Stenetorp. Robin Jia, and Douwe Kiela. 2022. Mod- els in the loop: Aiding crowdworkers with generative annotation assistants. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 3754 3767, Seattle, United States. Association for Computational Lin- guistics.
Manuel Blum and Silvio Micali. 1984. How to gen- erate cryptographically strong sequences of pseudo- random bits. SIAM J. Comput., 13(4):850-88) 864.
Rishi Bommasani, Drew A. Hudson, Ehsan Adeli, Russ B. Altman, Simran Arora, Sydney von AIx. Michael S. Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, Erik Brymjolfsson. Shya- mal Buch, Dallas Card, Rodrigo Castellon, Ni- ladri S. Chatterji, Annie S. Chen, Kathleen Creel, Jared Quincy Davis, Dorottya Demszky, Chris Don- ahue, Moussa Doumbouya, Esin Durmus, Stefano Ermon, John Etchemendy, Kawin Ethayarajh, Li Fei- Fei, Chelsea Finn, Trevor Gale, Lauren Gillespie, Karan Goel, Noah D. Goodman, Shelby Grossman, Neel Guha, Tatsunori Hashimoto, Peter Henderson, John Hewitt, Daniel E. Ho, Jenny Hong, Kyle Hsu, Jing Huang. Thomas Icard, Saahil Jain, Dan Jurafsky, Pratyusha Kalluri, Siddharth Karamcheti, Geoff Keel- ing, Fereshte Khani, Omar Khattab, Pang Wei Koh, Mark S. Krass, Ranjay Krishna, Rohith Kuditipudi. and et al. 2021. On the opportunities and risks of foundation models. CoRR, abs/2108.0722
6 Limitations
This paper focused on motivating and defining pseudointelligennee, as well as demonstrating its potential use for unifying and analysing LLM eval- uation. Deeper analyses, such as provable bounds comparing model and evaluator sample complexi- ties (m vs. n), are left for future work.
The impact of large language models extends far (Bom- masani al., 2021). Pseudointelligencee does not, for example, correspond to an ability to respond to queries in an ethical or responsible manner. In general, psueodintelligence is concerned with the distinguishing ability of a class of evaluators, but does not consider the usage of model in a real- world context which may not conform to this class (cf. Mitchell et al., 2019; Suresh et al., 2023). Fi- nally, like all abstract definitions, it must not be used as a rubber stamp; that is, it cannot replace a case-by-case assessment of potential impacts of models prior to their deployment.
beyond their alleged esseudo-jintelligeen
Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz. Ece Kamar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott M. Lundberg, Harsha Nori, Hamid Palangi, Marco Túlio Ribeiro, and Yi Zhang. 2023. Sparks of artificial general intelligence: Early experiments with GPT-4. CoRR, abs/2303.12712
Ryan Burnell, Wout Schellaert, John Burden, Tomer D Ullman, Fernando Martinez-Plumed, Joshua B Tenenbaum, Danaja Rutar, Lucy G Cheke, Jascha Sohi-Dickstein, Melanie Mitchell, al. 2023. Re- think reporting of evaluation results ai. Science, 380(6641); 136-138.
Acknowledgements
SM is funded by gift from Apple Inc. OP is funded by the Simons Collaboration on the Theory of Algorithmic Fairness. PS and OP are funded by Project CETI via grants from Dalio Philanthropies and Ocean X; Sea Grape Foundation; Rosamund Zander/Hansjorg Wyss. Chris Anderson//accuelinee Novogratz through The Audacious Project: a col- laborative funding initiative housed at TED.
François Chollet. 2019. On the measure of intelligence. CoRR,
Amit Danicly and Shai Shalev-Shwartz. 2014. Opti- mal learners for multiclass problems. In Proceed- ings of The 27th Conference on Learning Theory, COLT 2014, Barcelona, Spain, June 13-15, 2014, volume 35 of JMLR Workshop and Conference Pro- ceedings, pages 287-316. JMLR.org.
References
Max Bartolo, Alastair Roberts, Johannes Welbl, Sebas- tian Riedel, and Pontus Stenetorp. 2020. Beat the
René Descartes. 1637. Discourse on the method: And, meditations on first philosophy. Yale University

---

Press. 1996. Originally published as Discours de Méthode Pour bien conduire sa raison, et chereher la vérité dans les sciences, 1637.
John C Lin, David N Younessi, Sai Kurapati, Oliver Y Tang, and Ingrid U Scott. 2023. Comparison of gpt- 3.5, gpt-4, and human user performance on practice ophthalmology written examination. Eye, pages 1-2.
Apamn Elangovan, Jiayuan Hc, and Karin Verspoor. 2021. Memorization vs. generalization Quantify- ing data leakage in NLP performance evaluation. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Lin- guistics: Main Volume, EACL 2021, Online, April 19 23, 2021, pages 1325-1335. Association for Com- putational Linguisties.
Margaret Mitchell, Simone Wu, Andrew Zaldivar, Parker Barnes, Lucy Vasserman, Ben Hutchinson, Elena Spitzer, Inioluwa Deborah Raji, and Timnit Gebru. 2019. Model cards for model reporting. In Proceddings of the Conference on Fairness, Account- ability, and Transparency, FAT* 2019, Atlanta, GA, USA, January 29-31. 2019, pages 220-229. ACM.
Oded Goldreich. 2008, Computational complexity a conceptual perspective. Cambridge University Press.
Melanie Mitchell and David C Krakauer. 2023. The de- bate over understanding in al's large language models. Proceedings of the National Academy of Sciences, 120(13) 13: (090e22221911111 120.
James P. Gunderson and Louise F. Gunderson. 2008. Intelligence (is not equal to) autonomy (is not equal to) capability. In Performance Metricsfor Intelligent Systems, PERMIS.
B. K. Natarajan. 1 1989. On learning sets and functions. Mach. Learn., 4:67-97.
Ben Hutchinson, Negar Rostamzadeh, Christina Greer, Katherine A. Heller, and Vinodkumar Prabhakaran. 2022. Evaluation gaps in machine learning practice. In FAceT '22 2022 ACM Conference on Fairness, Accountability, and Transparency, Seoul, Republic of Korea, June 21- 24, 2022, pages 1859-1876, ACM.
Yixin Nie, Adina Williams, Emily Dinan, Mohit Bansal, Jason Weston, and Douwe Kiela. 2020. Adversarial NLI: A new benchmark for natural language under- standing. In Proceedings of the 58th Annual Meet- ing of the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020, pages 4885-4901. Association for Computational Linguisties.
Robin Jia and Percy Liang. 2017. Adversarial examples for evaluating reading comprehension systems. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, EMNLP 2017, Copenhagen, Denmark, September 9-11, 2017, pages 2021-2031. Association for Computational Linguistics.
OpenAL. 2023. GPT-4 technical report. abs/2303.08774.
CoRR,
Ethan Perez, Sam Ringer, Kamile Lukosiute, Karina Nguyen, Edwin Chen, Scott Heiner, Craig Pettit, Catherine Olsson, Sandipan Kundu, Saurav Kada- vath, Andy Jones, Anna Chen, Benjamin Mann, Brian Israel, Bryan Seethor, Cameron McKinnon, Christopher Olah, Da Yan, Daniela Amodei, Dario Amodei, Dawn Drain, Dustin Li, Eli Tran-Iohnson, Guro Khundadze, Jackson Kernion, James Landis, Jamie Kerr, Jared Mueller, Jecyoon Hyun, Joshua Landau, Kamal Ndousse, Landon Goldberg. Liane Lovitt, Martin Lucas, Michael Sellitto, Miranda Zhang, Neerav Kingsland, Nelson Elhage, Nicholas Joseph. Noemi Mercado, Nova DasSarma, Oliver Rausch, Robin Larson, Sam McCandlish, Scott John- ston, Shauna Kravec, Sheer El Showk, Tamera Lan- ham, Timothy Telleen-Lawton, Tom Brown, Tom Henighan, Tristan Hume, Yuntao Bai, 7 Hatfield- Dodds, Jack Clark, Samuel R. Bowman, Amanda Askell, Roger Grosse, Danny Hernandez, Deep Gan- guli, Evan Hubinger, Nicholas Schiefer, and Jared Kaplan, 2023. Discovering language model behav- iors with model-written evaluations. In Findings of the Association for Computational Linguistics: ACI. 2023, pages 13387-13434, Toronto, Canada, Associ- ation for Computational Linguistics.
Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Henighan, Dawn Drain, Ethan Perez, Nicholas Schiefer, Zac Hatfield-Dodde Nova DasSarma, Eli Tran-Johnson, Scott Johnston, Sheer EI Showk, Andy Jones, Nelson Elhage, Tristan Hume, Anna Chen, Yuntao Bai, Sam Bowman, Stanislav Fort, Deep Ganguli, Danny Hernandez, Josh Jacobson, Jack- son Kernion, Shauna Kravee, Liane Lovitt, Ka- mal Ndousse, Catherine Olsson, Sam Ringer, Dario Amodei, Tom Brown, Jack Clark, Nicholas Joseph, Ben Mann, Sam McCandlish, Chris Olah, and Jared Kaplan. 2022. Language models (mostly) know what they know. CoRR, ab22207.052211.
Daniel Martin Katz, Michael James Bommarito, Shang Gao, and Pablo Arredondo. 2023. GPT-4 passes the bar exam. Available at SSRN 4389233.
Douwe Kiela, Max Bartolo, Yixin Nic. Divyansh Kaushik, Atticus Geiger, Zhengxuan Wu, Bertie Vid- gen, Grusha Prasad, Amanpreet Singh, Pratik Ring- shia, Zhiyi Ma, Tristan Thrush. Sebastian Riedel, Zeerak Waseem, Pontus Stenetorp, Robin Jia, Mohit Bansal, Christopher Potts, and Adina Williams. 2021. Dynabench: Rethinking benchmarking in NLP. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies, pages 41 4110-440 41-4412, Online, Association for Computa- tional Linguistics.
Inioluwa Deborah Raji. Emily Denton, Emily M. Ben- der, Alex Hanna, and Amandalynne Paullada. 2021. Al and the everything in the whole wide world bench- mark. In Proceedings of the Neural Information Processing Systems Track Datasets and Bench- marks NeurIPS Datasets and Benchmarks 2021, December 2021, virtual.

---

Marco Túlio Ribeiro, Tongshuang Wu, Carlos Guestrin, and Sameer Singh. 2021. Beyond accuracy: Behav- ioral testing of NLP models with checklist (extended abstract). In Proceedings of the Thirtieth Interna- Lional Joint Conference on Artificial Intelligence, IJ- CAI 2021, Virtual Event / Montreal, Canada, 19-27 August 2021, pages 4824-4828. ijcai.org.
William Saunders, Catherine Yeh, Jeff Wu, Steven Bills. Long Ouyang, Jonathan Ward, and Jan Leike. 2022. Self-critiuuing models for assisting human evaluators. CoRR, abss2206.05800)
Ali Shirali, Rediet Abebe, and Moritz Hardt. 2023. theory of dynamic benchmarks. In The Eleventh International Conference Learning Representa- tions, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net.
Harini Suresh, Divya Shanmugam, Tiffany Chen, An- nie G. Bryan, Alexander D' Amour, John V. Gut- tag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grouneee
context-specific ML model evaluation. In Proceedings of the 2023 CHI Confer- ence on Human Factors in Computing Systems, CHI 2023, Hamburg, Germany, April 23-28, 2023, pages 775:1-775:13. ACM..
Alan M. Turing. 1950. Computing machinery and intel- ligence. Mind, LIX(236):433 460.
Andrew Chi-Chih Yao. 1982. Theory and applications of trapdoor functions (extended abstract). In 23rd Annual Symposium Foundations of Computer Sci- ence, Chicago, Illinois, USA, 3-5 November 1982, pages 80-91 91. IEEE Computer Society.
Sarah J. Zhang, Samuel Florin, Ariel N. Lee, Ea- Niknafs, Andrei Marginean, Annie Wang, Keith Tyser, Zad Chin, Yann Hicke, Nikhil Singh, Madeleine Udell, Yoon Kim. Tonio Buonassisi, Ar- mando Solar-Lezama, and Iddo Drori. 2023. Explor- ing the MIT mathematics and EECS curriculum using large language models. CoRR, abs/2306.0899 08997.


Reviews:
[{"review_id": "qB7JHpXkc6", "reviewer": "Reviewer_6eqq", "summary": "", "questions": "A: How would you apply this pseudointelligence framework to existing evaluation practices? Are there any specific recommendations or tools for implementing this approach?\n\nB: What criteria would your propose for designing effective evaluators based on this framework? How would you prevent artifacts like evaluators favoring outputs closer to *its* training distribution?\n", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your very thoughtful and comprehensive review. We are glad you found this work thought-provoking and a well-motivated conceptual contribution.\n\n> Not a lot of specifics for how the proposed framework can be implemented to produce a superior evaluation approach\n\nAnswer: Thank you for the question. Since this is a theme track paper with 4 pages, the space was used to convey the main idea and connect it to existing evaluation methods. Developing a superior evaluation approach based on the proposed framework is left for future work. However, our paper provides guidelines for building such an approach, as conveyed by our main messages P1-P3 (lines 58-70).\n\n> Discussion of existing evaluation approaches merely describes them in terms of the framework, but does not derive much insights into how optimal each approach is\n\nAnswer: The optimality of the evaluator is a function of its resources: compute / data annotation budget, time constraints, and model size (if the evaluator is model-based). We make the following observations on the power of different evaluation approaches based on our definition alone: First, it is clear that the distinction of the evaluator can only grow with its round complexity $r$ (because subsequent queries can always be ignored). This verifies the intuition that adaptive evaluation is more powerful than static evaluation. Out of the existing types of evaluation, IID static datasets are less powerful as compared to adversarial evaluation, which is less powerful than dynamic model-based evaluation/or model-based evaluators, using simpler models is less optimal than using expressive models.\n\n### Regarding the Questions For The Authors\n\n> How would you apply this pseudointelligence framework to existing evaluation practices? Are there any specific recommendations or tools for implementing this approach?\n\nAnswer: Our next step is an empirical paper that is guided by this framework. We will conduct a meta-evaluation of LLaMa similar to the work of Zheng et al. 2023 [1], which uses language models as evaluators on different Machine Translation tasks, but increasing the power of the evaluator; we hypothesize that a learned evaluator increasingly distinguishes between LLaMa and samples from MT Bench as the complexity of the evaluator increases (measured by number of samples, and number of parameters). \n\n[1] Judging LLM-as-a-judge with MT-Bench and Chatbot Arena. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric. P Xing, Hao Zhang, Joseph E. Gonzalez, Ion Stoica. arXiv, 2023.\n\n> What criteria would you propose for designing effective evaluators based on this framework? How would you prevent artifacts like evaluators favoring outputs closer to their training distribution?\n\nAnswer: We note that in our definition, the evaluator is required to distinguish between the model and the \"ground-truth\". Therefore, if the evaluator overfits its samples (i.e., prefers its training distribution), it will end up rejecting even the ground-truth, which reduces its distinction. This initial paper is focused on introducing the framework and connecting it to existing literature, and this pressing question would be interesting to study in future work. We would like to highlight that the main point of our paper is to give our community a rigorous framework for stating and studying questions such as these, and so we are very happy to see this question arise already in this discussion!\n\n### Regarding Missing References\n> This one may be relevant: Hutchinson et al. Evaluation Gaps in Machine Learning Practice. FAccT '22. https://dl.acm.org/doi/fullHtml/10.1145/3531146.3533233\n\nAnswer: Thank you for the pointer! We will add this as related work.\n\n### Regarding Typos Grammat Style And Presentation Improvements\n> 191: Does \"they\" refer to \\mathcal{G}? \n\nAnswer: Thanks for pointing out this typo - “They” here should be replaced by L_{G} and L_{E}.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- A well-motivated conceptual contribution to LLM evaluation, especially as the current landscape gets really messy with models evaluating itself, data leak, etc.\n- The learning-theory-motivated mathematical language is nice, and clearly defines many dimensions to consider during evlauation.\n- Release of this position paper may encourage discussion around more rigorous evaluation practices.\n- The paper is clearly written and presented.", "reasons_to_reject": "- Not a lot of specifics for how the proposed framework can be implemented to produce a superior evaluation approach\n- Discussion of existing evaluation approaches merely describes them in terms of the framework, but does not derive much insights into how optimal each approach is", "questions_to_authors": "A: How would you apply this pseudointelligence framework to existing evaluation practices? Are there any specific recommendations or tools for implementing this approach?\n\nB: What criteria would your propose for designing effective evaluators based on this framework? How would you prevent artifacts like evaluators favoring outputs closer to *its* training distribution?\n", "paper_topic_and_main_contributions": "- This thought-provoking position paper proposes a conceptual framework for evaluating LLMs as they surpass human performance in many tasks.\n- The central concept is named \"pseudo-intelligence\", which posits that the model should produce outputs that are $(\\epsilon, \\delta)$-indistinguishable from those produced by a specific *capability* according to a specific *evaluator*\n- The paper casts model evaluation as dynamic interaction between the model and a separately learned evaluator, and introduces a mathematical language based on learning theory which provides the formal definitions for \"pseudo-intelligence\"\n- The authors further advocate for considering *budget of evaluator* in addition to *budget of the model* during evaluation, and also argue that self-evaluation cannot prove intelligence\n- Using this framework, the authors describe various existing approaches such as datasets, adversarial evaluation, and model-based evaluation.", "excitement": 4, "reproducibility": null, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "This one may be relevant: \nHutchinson et al. Evaluation Gaps in Machine Learning Practice. FAccT '22. https://dl.acm.org/doi/fullHtml/10.1145/3531146.3533233", "Typos_Grammar_Style_and_Presentation_Improvements": "191: Does \"they\" refer to \\mathcal{G}?"}, "extra_text": {}}, {"review_id": "vVIDAymxOE", "reviewer": "Reviewer_Mo8o", "summary": "", "questions": "1. Could you provide more examples or case studies where this concept can be applied to evaluate the capabilities of large language models? \n\n2. Are there plans to conduct more empirical studies to validate the proposed framework?\n\n3. The paper introduces complex concepts like pseudointelligence and pseudorandomness. Could you provide more explanation or examples to make these concepts accessible to a broader audience?\n\n4. While the paper presents a meticulous theoretical framework, it seems to lack empirical validation. Are there plans to conduct empirical studies to validate the proposed framework?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your thoughtful and helpful review. We are glad you found the paper meticulous and inspiring.\n\n> The paper introduces a novel concept of pseudointelligence and a framework for evaluating language models. However, it might lack empirical validation or case studies that demonstrate the practical utility of this framework.\n\nAnswer: The main utility of this framework is a unifying, theoretical lens on evaluation. In Section 4, we present case studies to cast existing evaluations under the framework of pseudo-intelligence. Since this is a theme track paper at 4 pages, the space was used to convey the main idea. We leave empirical validation/implementation for a future non-theme track paper.\n\n> The paper primarily focuses on the theoretical aspect of pseudointelligence. It might not provide a comprehensive discussion on how this concept can be applied to various NLP tasks or different types of language models.\n\nAnswer: We provide a sketch of how this framework can be applied to current evaluation methods in Section 4.  Due to space constraints, we avoided directly quoting from each of our cited works with details of how they can be instantiated by our framework.\n\nMore generally, we do not prescribe a fixed measure of the complexity of the model class which, depending on the model/method/optimizer, could be highly variable. We envision studying attributes such as the computational complexity of the learner/evaluator: run time, size of the network, and number of training samples. Similarly, practitioners might analyze measures such as the number of flops required to train the learner in a particular implementation, monetary cost or run time at inference. \n\n> The paper introduces complex concepts like pseudointelligence and pseudorandomness. It might not provide enough explanation or examples to make these concepts accessible to a broader audience.\n\nAnswer: We are happy to provide more background in a future version with the extra page, and add an extended exposition of technical background in the appendix.\n\n> The paper might not provide a thorough comparison of the proposed evaluation framework with existing methods. Such a comparison would help readers understand the advantages and potential limitations of the proposed approach.\n\nAnswer: Pseudo-intelligence is a meta-evaluation framework, that is, it can be used to understand the validity of any evaluator. It is, therefore, not a fixed “approach” that requires comparison but a “framework”. Another such framework is the Turing test, which we do compare to in Section 2. We are happy to discuss other relevant frameworks during the discussion period.\n\n> Could you provide more examples or case studies where this concept can be applied to evaluate the capabilities of large language models? Are there plans to conduct more empirical studies to validate the proposed framework?\n\nAnswer: This initial paper introduces the framework and is focused on the conceptual and definitional side. Our next step is an empirical paper that is guided by this framework. We will conduct a meta-evaluation of LLaMa similar to the work of Zheng et al. (2023) [1], which uses language models as evaluators on different Machine Translation tasks, but increasing the power of the evaluator; we hypothesize that a learned evaluator increasingly distinguishes between LLaMa and samples from MT Bench as the complexity of the evaluator increases (measured by number of samples, and number of parameters).\nA main benefit of our framework is its ability to augment empirical understandings with theoretical analysis. Since the submission of this paper, we have already proven preliminary scaling theorems that demonstrate how, in theory, the sample complexity of the evaluator-learner $L_E$ must grow with the sample complexity of the model-learner $L_G$. In other words, this theorem shows how many samples are needed for the model to be distinguished from the ground-truth capability. This theorem can indeed be verified experimentally in a setup like the one described in the previous paragraph.\n\n[1] Judging LLM-as-a-judge with MT-Bench and Chatbot Arena. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric. P Xing, Hao Zhang, Joseph E. Gonzalez, Ion Stoica. arXiv, 2023.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. This paper provides a fresh perspective on evaluating the capabilities of large language models.\n\n2. This paper present a meticulous and detailed theoretical framework for understanding and evaluating pseudointelligence, providing a solid foundation for future research and discussions in this area.\n\n3. The 3 messages are reasonable, and they are also inspiring to related topics.", "reasons_to_reject": "1. The paper introduces a novel concept of pseudointelligence and a framework for evaluating language models. However, it might lack empirical validation or case studies that demonstrate the practical utility of this framework.\n\n2. The paper primarily focuses on the theoretical aspect of pseudointelligence. It might not provide a comprehensive discussion on how this concept can be applied to various NLP tasks or different types of language models.\n\n3. The paper introduces complex concepts like pseudointelligence and pseudorandomness. It might not provide enough explanation or examples to make these concepts accessible to a broader audience.\n\n4. The paper might not provide a thorough comparison of the proposed evaluation framework with existing methods. Such a comparison would help readers understand the advantages and potential limitations of the proposed approach.", "questions_to_authors": "1. Could you provide more examples or case studies where this concept can be applied to evaluate the capabilities of large language models? \n\n2. Are there plans to conduct more empirical studies to validate the proposed framework?\n\n3. The paper introduces complex concepts like pseudointelligence and pseudorandomness. Could you provide more explanation or examples to make these concepts accessible to a broader audience?\n\n4. While the paper presents a meticulous theoretical framework, it seems to lack empirical validation. Are there plans to conduct empirical studies to validate the proposed framework?", "paper_topic_and_main_contributions": "This paper is about how should we evaluate the intelligence of large language models, especially when they perform at or above human levels on certain tasks?\nThe paper introduces the concept of \"pseudointelligence\",  suggesting that perceived intelligence is subjective and depends on the evaluator's perspective.\nInspired by pseudorandomness which measures a distribution by its distinguishability from true randomness, pseudointelligence evaluates the capabilities of learned models based on their distinguishability from certain capabilities.\nThe authors propose a framework where model evaluation is seen as a dynamic interaction between the model and a learned evaluator, emphasizing that claims of a model's intelligence are meaningful only when considering the evaluator.\nThe paper uses this framework to reason about two case studies in language model evaluation. It also analyzes existing evaluation methods under this new lens.\nThis work conveys three main messasges: (1) Claims of intelligence must be supplemented by a clearly defined evaluator and its capabilities. (2) As more resources are dedicated to model development, there should also be an increase in resources dedicated to evaluation. (3) Self-evaluation by a model cannot support a claim of intelligence if the evaluator is derived directly from the model itself.", "excitement": 4, "reproducibility": null, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "h8FxL6jbeA", "reviewer": "Reviewer_WYM6", "summary": "", "questions": "", "limitations": "", "rating": 2, "confidence": 5, "soundness": 2, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> “Roughly speaking, pseudo-intelligence is a learning algorithm that produces a model that cannot be distinguished from a real expert under the limited computing power of the distinguisher.”\n\nAnswer: Pseudo-intelligence is not a learning algorithm. Rather, as defined in the paper, a learner is said to be pseudo-intelligent with respect to an evaluator class if it satisfies certain conditions. We refer to lines 182-193 in the paper for the full definition.\n\n> The work is too preliminary. Algorithms for learning a class of models and a class of evaluators have not been proposed. The conditions under which learning in the proposed framework is successful have not been studied.\n\nAnswer: As a theme track paper, we introduce a framework that highlights the importance of providing sufficient _resources_ to the evaluator, and a sketch for how these evaluators may be constructed. The framework is not concerned with the learning of models, and we leave practical implementations of such evaluators to future non-theme track papers.\n\n> The advantage of the proposed framework compared to Valiant's PAC learning framework has not been studied.\n\nAnswer: PAC Learning is not comparable to our framework. For instance, PAC learning is concerned with the ability of a learner to output a hypothesis that generalizes to any distribution $\\mu$; Futhermore, the error is measured with respect to this distribution $\\mu$. However, the error of a pseudointelligent learner is measured by a learned evaluator and is defined with respect to a particular capability (distribution). In other words, PAC learning only has a learner, whereas pseudointelligence is defined with respect to a learner, a (learned) evaluator, and a capability.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "An attempt was made to formalize the evaluation of modern AI models, such as LLMs.", "reasons_to_reject": "- The work is too preliminary. Algorithms for learning a class of models and a class of evaluators have not been proposed. The conditions under which learning in the proposed framework is successful have not been studied.\n- Without instantiating a pseudo-intelligence with a non-trivial evaluator class, the proposed framework remains too general to be useful for theoretical analysis. This is confirmed by the example of a trivial instance ($L_E\\equiv e_0$, $e_0\\in E$), which is considered in the discussion below.\n- Under additional assumptions, the definition of pseudo-intelligence can be reduced to the definition of PAC learnability of the class $G\\times E$ (see below).", "questions_to_authors": "", "paper_topic_and_main_contributions": "The authors introduce the concept of pseudo-intelligence, which resembles the definition of pseudo-randomness. Roughly speaking, a learning algorithm is pseudo-intelligent if it produces a model that cannot be distinguished from a real expert under the limited resources of the distinguisher.", "excitement": 2, "reproducibility": null, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper received mixed scores of 4,3,2 (soundness) and 4,4,2 (excitement).

Strengths and weaknesses including the following were mentioned prominently:

Strengths:
- conceptual contribution to LLM evaluation (R1, R2, R3)
- well-written (R1)

Weaknesses:
- leaves unclear how the proposed framework can be implemented (R1, R2, R3)
- limited insights into relation to existing evaluation approaches (R1, R2)
- framework was criticized as too general to be useful (R3)
- may be reducible to PAC learnability (R3)

R3 provides a low soundness scores and has the following major concerns: (1) the work is very preliminary, (2) without concrete instantiation of evaluators, it remains unclear if the framework can be used for nontrivial theoretical analysis, (3) the  definition of pseudo-intelligence may be linked to PAC learnability.
I do not find (1), as explicated in the review, to be a major concern given the goals of this track.
The authors pushed back against the concerns (2) and (3), and I found their explanations to be convincing. In conclusion, I do not see major concerns against the soundness of this proposal for a theoretical framework for evaluation -- noting that it is certainly a first step, with substantial future work needed to work towards implementation.
Decision: Accept-Findings