Language Models can Categorize System Inputs for Performance Analysis (2025)

Abstract: Language model systems are used to process diverse categories of input requests, ranging from improving creative writing to solving programming challenges. It would be useful to know which categories they are good at. However, existing evaluations compare model performance on pre-defined categories, failing to reflect a system’s performance on finer-grained or novel ones. We propose to automatically search for finer-grained categories based on inputs where a system performs well or poorly, and describe them in natural language. To search for these categories, we propose a large number of candidate category descriptions, e.g. “Communication Improvement”, find the subset of inputs that match the category descriptions, and calculate the performance on these categories; then we sort these categories based on their performance, thereby highlighting those that score high or low. As one application, we apply our method to compare LLaMA 3-70B and Claude 3 Opus, which have similar Elo-ratings on Chatbot Arena; our method finds the former is weaker at making text more professional and humorous while better at providing psychological insights, depicting a more nuanced picture of model performance.

## introduction language models (lms) can perform a wide range of tasks, from writing to programming and solving math problems. if a user wants to make their text more professional, they will prefer an lm that is good at writing; its coding ability irrelevant. however, the current evaluation paradigm falls short in offering detailed insights, as it only provides a single numerical score for broad categories. for example, chatbot arena @xcite uses elo ratings based on pairwise comparisons to rank models, with gemini-1.5 pro, llama 3-70b, and claude 3 opus (reid et al., 2024; meta, 2024; 1 our code is at github repository. @xcite having similar ratings. are their performances uniformly similar in programming and improving writing? indeed we can create a pre-defined category called "communication improvement" beforehand, but it is difficult to anticipate categories in practice, especially as new use cases constantly emerge. fixed datasets such as gsm8k or bigcodebench @xcite can provide a more detailed picture of model abilities; however, they often focus on high-level tasks such as grade-school math or coding questions, without revealing performances in finer categories, e.g. the proficiency in computer networking. a solution could be to manually propose new categories by skimming through thousands of samples, but this is both time-consuming and labor-intensive. we would like an automatic and adaptive categorization procedure. to address these requirements, we constructed the automated system input categorization scheme (asics), a method that adaptively explains input categories in natural language and assigns scores to each, with the highest and lowest scores highlighted. asics takes in system inputs and the lm's performance on each system input (e.g whether the response to the input is correct). it outputs a list of categories, each described by a natural language string; each category is associated with the lm's average performance, with the best and worst categories highlighted. asics operates in three stages: propose, falsify, and score, as illustrated in figure 2 . first, given a set of system inputs with both correct and incorrect answers, the category proposer generates natural language descriptions of the categories that appear across them. second, the falsifier ensures that each category is unambiguous and understandable; it does this by evaluating whether an lm can confidently determine which inputs match the generated categories. finally, the category scorer evaluates each category @xmath0...i am not a native english speaker and would appreciate it if you could convey my text in clear and concise language. use british english...." @xmath1. can you rephrase it so that it looks more exciting and forward looking for a funding proposal?" @xmath2. we applied asics to gsm8k, bigcodebench and a human preference dataset (hpd) foot_0 that compares different lms. on hpd, asics discovers categories where humans prefer llama 3-70b, and claude 3 opus. for example, claude 3 is preferred over llama for "professional communication improvement" while llama is preferred for providing "psychological insights" (section 5.1). this demonstrates that asics can generate novel categories. on bigcodebench, asics discovers categories where llama 3 is better than gpt-4 despite having a 10% lower overall score, such as, 'network operations' and 'web scraping' (section 5.3). this demonstrates that asics can generate fine-grained categories, providing more detailed insights into system abilities than datasets alone. across all datasets, asics identifies both categories that can be validated by prior works, as well as new categories that may motivate future investigation. we can already see interesting results with only thousands of data points, and scaling this up to larger datasets could yield more interesting insights, thus helping humans have a global understanding of lms' performance. finally, we explore future 2 related work language models and inductive reasoning asics generates categories by recognizing patterns within system inputs. our approach builds on work that uses language models @xcite @xcite to generate hypotheses from individual instances @xcite @xcite . recent work indicates that under certain conditions, language models can perform inductive reasoning, identifying patterns in a collection of text data points and articulating them using language @xcite . @xcite and @xcite show that lms are able to discover patterns in datasets. @xcite generate categories by identifying patterns in inputs that produce the same output, but should not in multimodal models. @xcite automatically discover differences between two large corpora using language models that are relevant to a downstream goal. @xcite use language models to generate clusters describing an input corpus according to a user defined goal. our work differs by demonstrating the ability of language models to categorize patterns in human preference data. ## the asics pipeline we describe asics, a method to automatically identify fine-grained categories from system inputs with correct and incorrect answers. we use s to denote the target language model system to be evaluated and q the set of inputs where s was evaluated. we define c q,@xmath3, and 0 otherwise. in the context of the hpd c q,s represents when s is preferred. asics takes in q and c, and outputs a set of category descriptions h, where each category is associated with a performance score. the highest and lowest scoring categories will be highlighted to indicate where the system performs exceptionally well or poorly. asics relies on three components: the category proposer, the falsifier, and the category scorer. the category proposer suggests categories {h 1 , . . . , h k } based on q. the falsifier ensures that each category h i is comprehensible by evaluating whether a language model can confidently align inputs with the generated categories. an input is considered aligned if it possesses the features described in the category (e.g., an input aligns with the "conceptual explanation in mathematics or physics" category if it contains features related to math or physics, as shown in ## human evaluation of asics in this paper, asics uses claude 3 opus as the category scorer. to ensure that claude 3 opus accurately identifies which questions belong to each category, we measure its agreement with human annotators on gsm8k and the hpd. annotators are presented with a category and two corresponding questions: one randomly sampled from a dataset of questions classified by claude as aligned with the category, and the other randomly sampled from a dataset of questions classified as unaligned. each participant evaluates which of the two questions best matches the given category. a success for the category scorer is when the annotator and claude 3 opus select the same question as being aligned with the category. in the second human evaluation experiment, we assess whether a randomly sampled input question from a dataset-balanced with an equal number of questions claude classified as aligned and unaligned for a specific category-is aligned with that category. we experiment with two different evaluation mechanisms involving crowd-workers and experts. ## discussion we developed asics, a method that automatically finds categories in inputs and describes them in natural language. to generate more novel categories asics can be deployed at industry scale where (prompt, evaluation) pairs are easier to obtain. in these cases, the users write the prompt themselves, and the lm platform can (implicitly) ask the users to evaluate the responses. for example: chatb-otarena is an open platform that mimics chatbot deployment. within 15 months, it has collected 1.5m (prompt, evaluation) pairs. chatgpt allows users to provide feedback if an lm's response is bad, and also allows users to label whether an entire conversation is helpful or not. this will likely gather many more such pairs than chatbotarena. character.ai receives ∼1.7b queries per day, and a proxy evaluation is whether the user continues the conversation. in principle, asics can identify patterns in instance-level data points if the data can be described using natural language. as models are increasingly deployed in domains where human evaluation is time-consuming and challenging, adaptive and automatic methods like asics can provide insights into potential model failures in these settings. as language models continue to improve and scale, asics's performance is likely to improve as well. broader applications real-world systems could also be studied with asics. asics could be used to find patterns among successful and unsuccessful y combinator applicants @xcite . teaching at the right level, which aims to determine the learning level students are at and then teach them at their level, has proven to be an effective way to improve learning outcomes @xcite . teachers could use asics to find categories among questions students answer incorrectly.