EmbedLLM: Learning Compact Representations of Large Language Models
Authors: Richard Zhuang
Tianhao Wu
Zhaojin Wen
Andrew Li
Jiantao Jiao
Kannan Ramchandran
Venue: ICLR 2025 Spotlight (2025)
Keywords: Large Language Models
Representation Learning
Model Routing
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Abstract: With hundreds of thousands of language models available on Huggingface today, efficiently evaluating and utilizing these models across various downstream tasks has become increasingly critical. Many existing methods repeatedly learn task-specific representations of Large Language Models (LLMs), which leads to inefficiencies in both time and computational resources. To address this, we propose EmbedLLM, a framework designed to learn compact vector representations of LLMs that facilitate downstream applications involving many models, such as model routing. We introduce an encoder-decoder approach for learning such embedding, along with a systematic framework to evaluate their effectiveness. Empirical results show that EmbedLLM outperforms prior methods in model routing. Additionally, we demonstrate that our method can forecast a model's performance on multiple benchmarks, without incurring additional inference cost. Extensive probing experiments validate that the learned embeddings capture key model characteristics, e.g. whether the model is specialized for coding tasks, even without being explicitly trained on them. We open source our dataset, code and embedder to facilitate further research and application.

Published as a conference paper at ICLR 2025
EMBEDLLM: LEARNING COMPACT REPRESENTA- TIONS OF LARGE LANGUAGE MODELS
Richard Zhuang Tianhao Wu' Zhaojin Wen Andrew Li
Jiantao Jiao Kannan Ramchandran
University of California, Berkeley
ABSTRACT
With hundreds of thousands of language models available on Huggingface to- day, efficiently evaluating and utilizing these models across various downstream tasks has become increasingly critical. Many existing methods repeatedly learn task-specific representations of Large Language Models (LLMs), which leads inefficiencies in both time and computational resources. To address this, we pro- pose E.. framework designed to learn compact vector representations LLMs that facilitate downstream applications involving many models, such as model routing. We introduce an encoder-decoder approach for learning such em- beddings, along with systematic framework to evaluate their effectiveness. Em- pirical results show that EmbedLLM outperforms prior methods model routing both in accuracy and latency. Additionally. we demonstrate that our method can forecast a model's performance on multiple benchmarks, without incurring ad- ditional inference cost. Extensive probing experiments validate that the learned embeddings capture key model characteristies, e.g. whether the model is special- ized for coding tasks, even without being explicitly trained on them. We open source our dataset, code and embedder to facilitate further research and applica- tion: https: /tap::///ittiin htggttttubbu...g www.//o.o...ii.. httmm//rcccardduuan00000. http:///hhaaan002222//m./.....
EmbedLLM
INTRODUCTION
Recent breakthroughs in Large Language Models (LLMs) (Vaswani al. 2023 have led to the creation of a vast array of models, each tailored for different use cases. These models, ranging from small, specialized models to large, general-purpose systems (Hao et al. 2022), differ significantly in their architecture, size, training data, and performance characteristics. For example, while some models excel as conversational agents, others may be more suitable for code generation or logical reasoning tasks. However, with this explosion of diverse LLMs comes major challenge:
How to efficiently manage, compare, and utilize the growing number of LLMs?
Traditionally, benchmarking has served as the primary method for comparing LLMs, where each model is evaluated fixed of test cases, and Meanwhile, model routing systems are developed to efficiently select models given queries dif- ferent types. An example workflow of these tasks can be seen Figiure 2landFiggrr 3) While these approaches are often robust indicators of model's 's strengths and weaknesses, the construction of their workflows induces repeatedly learning representations of various LLMs suit each individual downstream tasks and is therefore tine-consuming and compute dempuddn..
generated to represent its performance.
In response to these challenges, we introduce EmbedLLM. a compute compute-friendly framework designed to learn compact vector representations of large language models that facilitates different tasks. Em- bedL_LM map models into latent vector space that captures important model characteristics. More importantly, EmbedI.L., L.M produces unified representation that can be simultaneously applied to various downstream tasks such as correctness forecasting (Section 5.1), model routing (Section 5.2). and benchmark accuracy evaluation (Section 5.3), The core idea is to enforce this representation
Equull contribution.

---

Published as a conference paper at ICLR 2025
learning through reconstruction- based system that tries to predict the model's answer (correct- ness) from the learned embeddings, ensuring that each model's embedding retains the most salient features relevant to enhance performance across multiple scenarios.
Models
Embedder
Model Embedding
Linear Classifier
Downstream Tasks
Network
SF
[0.99 7.95 136, -8335]
Routing
A
[3.68] -2.46.. 925)
Wz +
8
[-862, 6.39 0.43, 5.52]
Benchmark
[2.58 -452 4.87, 3.71
Accuracy Estimation
Figure 1: An illustration of the EmbedLLM Pipeline. An embedder network pretrained from sam- ple question-anse answer pairs from a pool of LLMs to map them into vector embeddings. Downstream applications like model routing are adapted by training an additional linear layer on top these embeddings.
In summary, our contributions are as follows:
1. We propose novel framework based reconstruction system for learning compact, www-dimensional representations of LLMs. To the best of our knowledge, this is the first work that explicitly introduces the concept of turning LLMs into embeddings.
2. We introduce an evaluation setup that employs the model embeddings to simultaneously predict model answer correctness on unseen questions, perform model routing, and predict benchmark accuracy with the addition of only a linear classifier. Our system demonstrates the potential for significantly reducing the need for task task-specific re-training.
3. We perform probing experiments validate that the learned embeddings capture meaning- ful information. We discover that models with similar characteristics remain close in the embedding space, and that the effect of incorporating each benchmark is reflected through the change in model embeddings.
EmbedLLM offers a scalable and unified approach model evaluation and selection. By producing embeddings that encapsulate important features across tasks, our framework provides a versatile method to navigate the increasingly complex landscape of large language models.
2 RELATED WORK
Representation Learning There have been numerous attempts to learn representations of various types of information. For natural languages, Mikolov al. (201 3a) anddinnnngooon al. (2014) rev- olutionized the way models capture word semantics, In the field of computer vision, sell-supervised techniques (Noroozi & Favaro 2017) Vondriik al., 2018) are designed learn low-dimensional representations that bolster downstream classification or segmentation performances. Inspired by these work and realizing an increasing demand various LLMs being trained, we propose cre- ative framework to learn embeddings of LLMs.
LLM Benchmarking Benchmarking has been a standard way to compare L.MMs, where a collec- tion of questions/prompts is input to the LLMs and the quality of the corresponding responses is evaluated. However, with enormous inference cost incurred, current benchmarks typically aggre- gate model answers to single accuracy metrie, losing valuable insights from the diverse model responses; our work explores repurposing these inference results to gain deeper understanding of model capabilities.
LLM Routing Our work focuses on predictive routing, which is technique aimed at proposing the most suitable model given task, without actually passing the query through each of them. As
2

---

Published as a conference paper at ICLR 2025
summarized in Hu al. 2024), most routers adopt either supervised technique (Ong al. 2024 Shnitzer et al. 2023) or a reward- method (Lu et al. 2023). Trained by seeing responses from different models to the same prompt, these systems are intrinsically building understanding of key model characteristics. Our work establishes yet another new interesting and efficient research direction as find model embeddings strengthen routing performances.
FORMULATION
3.1 PROBLEM SETUP
Let M = {M,, .M...-MM be set of different LLMs, P denote the set of all possible prompts, and A denote the corresponding set of possible answers. We can simply identify any LLM M with an inference function mapping from a prompt space to an answer space fM : :P A, which outputs an answer a E A given prompt p E P.
Among downstream tasks, representations of different LLMs needed to be constructed in vari- ways. A naive example benchmarking: Where the crucial part is select test prompt set IP Bench = [Pi,P2 " as well as an scoring function Jecal P x A - R, map- ping model responses by a set of multiple-choice questions with four choices. The model's response is recorded by comparing the output probabilities of the answer choices "A", "B", "C", and "D". Then the responses are simply scored by matching with the ground truth answer key. Within this process, every model M's behavior is essentially summarized by their output on a set of test questions((Pi, ((M,1)), (12, ((M,2)) (pm, m)M.))))
scalar score. Take MMLU as an example, each model is queried
Benchmarka
Model Parameters
Model Responses
Response Evaluation
Output Performance
Artwer Eiring Murch
("he bnown Seart")
[ccorray: 54.8%1
COHI
(Balety
ratn prokuct appetiion.
l'Ocrouusss twa chree heatta-rve tooo--ow pump
ULM Audgemant
Figure 2: An illustration of the traditional workflow of LLM benchmarking.
Another example is model routing: given pool o LLMs, a router function is defined in Ong et al. (2024) n-way classifier that assigns models different queries maximize response quality while minimize inference cost. Similarly, training such router also often involves transforming each model into relatively low-dimensional representation by utilizing sample question-answer pairs.
Identifying common need for model characterization, we raise the question: what if could accomplish the above tasks by directly working with a unified, compact representation of LLMs? In this work, we provide framework to learn such a representation. We define an embedding function Φ M + Rd, parametrized by a neural network or otherwise, that maps each model M compact vector representation
latent embedding space. We aim learn model embeddings that contain important features of LLMs that are useful to both quantifying differences between various models and aiding across downstream tasks.
3.2 EVALUATION METRICS
Evaluating the quality of model embeddings is crucial to ensure they effectively capture the under- lying structure and semantics of the data which we care about for potential downstream tasks. The core idea is use the embeddings to predict model behavior unseen tasks by training infer- ence function d Rd P + Q. that leverages a model embedding and new test prompts to predict model's performances quantified by a desired evaluation metric in the space Q. For instance, if
3

---

Published as a conference paper at ICLR 2025
Routing System
Models
-eagstt--leoggied Ranking
Best Model Prediction
Corputte
Deckuiing Terry Cooffilions
Eatimate Probubilttie
Sig. k))
orgnnns
Mr Zelo
[Quastion 1-- CPT---Turbo
Plannu,
La
[Question Lame-3-40)
(Vttict of the Inlowing (offective)
pertiies rot spoctined pporttnnoaalyeeend symmsstty
Respanas Quality
LLMi aa------a-kktee Jusipe
Phe--tune on
Offine Eralustion
iQuestion N ... Mistrol-
['wnet putge hoch!]
[Question...
"Buaed quastion provided nupwer mose unter sound gius aosistants reporesr
CuduuC Weok Modie Custur Mocie Score
PWWtrr offernec betteenn
Questions
Figure 3: An illustration of the traditional workflow of model routing, using exemplar routing methodologies from Ong et al. (2024)
task is to predict whether model can correctly answer unseen question, the inference func- tion would be network that takes in model embeddings and the unseen question (usually present in the form of its embedding as well) and output a binary label as the prediction of model correctness. Note that model responses can be evaluated through many means such as determining correctness (Hendrycks al.] 222I), judging by human (Chiang et al. 2024) or stronger LLMs (Zheng al. 2023), or measuring any other task-specifi metric. Hence, each task constitute to its own unit of measure and correspondingly determines an evaluation metric. In this work, we focus on evaluating on the following downstream tasks:
Correctness Forecasting: We query the models benchmarks equipped with ground truth answers, and produce binary correctness label for every model-question pair. For this task, natural metric to would be test prediction accuracy.
With the help of embedding vector of each model, we' re able develop a simple linear router that directly determine the model to be routed to using the probability of the answer correctness along with some learnable threshold. We measure the routing performance by reporting the average response quality" namely the proportion of times router successfully route an LLM that correctly answers the given query.
Model Routing:
We treat the embeddings as primary features to train linear model in predicting accuracies on unseen benchmarks (measured from 0% to 100% as percentage). The metric used for this task would be the classical mean- mean- -quared-errrrrrr (MSE) for linear regression.
Benchmark Accuracy Prediction:
In addition, we could directly compare between embeddings in their vector form following Mikoloy et al. (2013b). For instance, models with similar traits should have embeddings that are closer in L2 distance.
EMBEDLLM
4.1 METHODOLOGY
In order to learn such embeddings, draw inspirations from image reconstruction algorithms (He et al. 2022 Ronneberger et al. 2015): we want to learn a "reconstruction" system, where the choice of the reconstruction target is arbitrary and can be task-dependent. More concretely, let be space of model performance metric, and M be set of possible LLMs, m be the number of models, be the number of questions/prompts. We want learn reconstruction network R. Qyaxn mmx
Note that this metric can be insufficient under regular routing setting (Ong et al. 2024 as it does not consider model cost, nevertheless fits well to goal of assessing whether our model embeddings effectively captures the strengths and weaknesses under the context correctness forecasting.
4

---

Published as a conference paper at ICLR 2025
that optimizes to reconstruct matrix X Qmx", where X1y Q denotes the model performance metric for model i on question j. The encoder-decoder architecture ensures the imposure of such constraint and enforce the model embeddings to efficiently capture the key characteristics of each model.
In this work, we decide to the task of predicting model answer correctness target, i.e., Q = {0. 13. Notice that our training objective is only a decoy any downstream task that requires understanding of model characteristics would qualify and our ultimate goal is enforce the reconstruction network to learn a compact yet information-rich representation of the models in this process.
auxiliary
4.2 DATASET
We now describe the data collection process of correctness results of various LLMs' responses to questions from mainstream benchmarks.
We selected 112 open-sourced models of various sizes, with both general purpose LLMs ZZu
al. at. 2024) and specialized LLMs included ensure comprehensive coverage. Then we aggregated
responses of every model to 36,054 questions from the test sets of MMLU (Hendrycks et al. 2021).
TruthfulQA (Lin et al. 2022). SocialQA (Sap al. 2019), PIOA(Bisk al. 2019), MedMCQA Pal et al. 2022), MathQA ((ahQA(AAmin et al.) 2019). LogiQA(Liu et al. 2020). GSMSK(Cobbe et al. (021), GPQA et al. 2023), and ASDiv(Miao et al. 2020). The responses to these questions were acquired and evaluated through using the "Im-evaluation-harnes" package (Gao et al. 2023 to give a binary correctness label for each model-question pair. We performed a random 80% 10%- train- validation-test split on the questions and used the sentence transformer "all-mpnet- base-v2" (Reimers& & Gurevych 2019) to convert the questions into an initial embedding state of dimension dim, 768. Consequently, i-th row X; R.dim, is the embedding for model i and our label tensor essentially correctness matrix Y with shape (112, 36054), where the ij-th entry Yúj represents the binary correctness of model i answering question j.
question embedding tensor X has the shape (36054, 768) where the
4.3 ALGORITHM
We adopt an encoder-decoder architecture to learn model representations:
The encoder consists of a model embedding network and a question embedding network. Let dimmembed be the desired dimension of the model embedding. The model embedding network om : MM Rddimemned maps each model into latent representation. Similarly, the question embedding network oy : P + Redimmabed maps each questions into a latent representation. In our setting, the question embedding network is two-step transformation 0, 9st hproo where get Rdim, denotes the pre-processing (performed in Section 4.2) that turns each question from text to an initial embedding space by using sentence transformer, and hproj Iidd - didimmmdddd is projection layer from this original embedding space to the same space as the model embeddings.
Encoder:
The decoder is binary classifier vi Rdimmnned x Iddimemicd - {0, 1} that takes in both the encoded embeddings of the model and the question, and output whether the model answer the question correctly. For this work, our decoder represented as (/(2m, vi)) = p @ ein) where Rdimmmbedd + R22 is a linear classifier and и @ v represents the Hadamard fieementwwise product between two vectors. For each model-question pair, this decoder network outputs two logits P(m.a). and P(m..g) and the "correctness score" sm.q a (P(m.9)) P(m..4))) represent the predicted probability of the model m correctly answering question q, where o (z) is the sigmoid function.
Decoder:
Suppose y is the correctness label of model m answering question g. we calculate the following BCE loss function during training,
L(m, q, v) = -- (y log(sm..) + (1 v) -og(11 sm.q))
In essence, this algorithm analogizes to a matrix factorization algorithm, where we learn a n X m model embedding matrix and a m p question embedding matrix such that their product recon-
"Sce appendix for full list.
In the actual implementation, we give each model index as an identification in M.
5

---

For correction prediction, we compare the effect of EmbedLLM to a NeeresttNeeihhte classifier [Fix] 1985). In the context our formulation, although without an explicit embeddings for models, KNN-classifier can be seen using the integration of all question-correettess tuples from model as "embedding". and making inference from this aggregation. Specifically, given question and model, the classifier outputs the majority vote of whether the model correctly predicts the nearest neighbor questions. For brevity, we refer to this approach as KNN in the subsequent text. As we correctness forecasting accuracy the test set as the evaluation Published as a conference paper at ICLR 2025
structs the original n x p conrectness matrix. In the following sections, we refer to this algorithm as EmbedLLM.
5
EXPERIMENT RESULTS
As described in Section 3.2 we conducted experiment in correctness forecasting, model routing,
and benchmark accuracy prediction to evaluate the quality of the learned embeddings.
5.1 CORRECTNESS FORECASTING
mentioned in [Section 3.2] metric.
We evaluate the performance of KNN and EmbedLLM across various sizes of training set. We produce the smaller training sets from randomly subsetting from the full training set. For each training set, conduct hyperparameter tuning (number of neighbors for KNN, model embedding dimension for EmbedLLM) on a fixed validation set and evaluate prediction accuracy using fixed test set. The result in Table lindicates better scalability of our method.
Dataset Size
Algorithm
IK
5K
10K
15K
20K
25K
Full (29K)
KNN
0.6372
0.7078
0.7107
0.7128
0.7143
0.7146
0.7152
EmbedLLM
0.6443
0.7331
0.7362
0.7378
0.7390
0.7394
0.7409
Table 1: Performance of predicting model correctness on a fixed unseen test set. The columns
indicate the number of questions the training set. EmbedI constantly outperforms KNN on training sets of all scale. We see a steady increase in the performance as the dataset size grows, indicating further scalability of our method.
5.2 MODEL ROUTING
Using the same correctness data, can evaluate the quality of using EmbedLLM router. For this task, we evaluate the router's accuracy by measuring the proportion of times successfully route model that could correctly answer the given query. For test question 4k. we pass through the router network along with all n possible model embeddings, producing n correctness score 8M MMa.ge>'. M,,.g and check if the model with the highest correctness score cor- rectly answers the question. Aggregating through all questions, we report the router accuracy as acCrouter m Ex= ((X;-,A 1} where i arg max M 1....n the routed model and X is the correctness matrix described in Section 4.2]
We compare the performance of the EmbedLLM router with two baselines. The first is the single-best model router which always selects the model with the highest accuracy and thus gives a constant accuracy. The second one is a random router that select each model the same number of times as the EmbedLLM router, but instead randomly assign models to questions. For this router, we can calculate its expected accuracy given the proportions of times each model is selected by the EM router. For instance, if our router selects M, 70% of the time, M2 20% of the time, and M3 10% of the time, the expected accuracy of the random router will be calculated as a weighted accuracy aCCweighted 0.7 aCCM, + 0.2 aCCM, + 0.1 * acCMs Note that this weighted accuracy will always be smaller than the single-best best model accuracy we propose this metric as an evaluation of how well router direct models to the questions they are good at given fixed "budget" of model calls. We report both the overall accuracy across the whole test set and accuracy per source benchmark.

---

Published as a conference paper at ICLR 2025
As seen in Figure 4, EMbedLLM router performs better than both the single-best router and the random router overall. As the best performing model is re-determined on every benchmark, the single-best model router performs better for each benchmark than in overall case. Here, EmbedLLM router achieves near single-best model router accuracy while still managing to utilize the respective strengths of different models which is shown by the significant difference between accuracies of EMbedLLM router and weighted router.
Another advantage of EMbedLLM router find is its high routing speed. On one NVIDIA A100 80GB GPU, it takes in average 3.80 seconds for EmbedLLM router to route 3,000 questions on 50 repeated trials which basically free compare to the downstream model inference time. Specifically, compared to the causal LLM router in Ong et al. (2024) that processes less than 50 requests per second, our router delivers more than 750 model selections, which 15x faster while selecting from much larger model pool (112 models against models for causal LLM router). In addition, only takes less than 1GB of peak GPU memory when training EmbedLL.N router using our dataset, which is 60x cheaper than fine-tuning Llama-3-8B as router in terms of memory usage. This illustrates the superiority of EMbedLLM router in both performance, latency, and training cost.
Routing Accuracy Comparison Across Benchmarks
Factorization
Single-Best
Welghted
24
Accuracy
Overall
ASDiv
GPQA
GSMAK
LogiQA
MathQA
MedMCQA
MMLU
PIQA
SocialQA
TruthfulOA
Benchmark
Figure 4: Performance accuracy of EmbedI.LM router compared baselines. EmbedI.LM router performs better across the whole test set and achieves accuracies close to the single-best model on every benchmark.
5.3 BENCHMARK ACCURACY PREDICTION
To predict model's 's average accuracy on a benchmark B, we trained EmbedL.MM using leave-one-out correctness data, which includes correctness results all models on all questions except the ones in B. Then we take the model embeddings directly as features to train a linear regression of the form:
aE =
where is a model embedding matrix with the i-th row representing the model embedding for the i-th model, and the j-th entry in the vector y corresponds the j-th model's average correctness accuracy on the test benchmark, which is a number from 0 to 1.
For each test benchmark, conducted 100 random train-test splits the 112 models contained in our dataset, trained a linear regression on the training set, and evaluated the correlation hetween model embeddings and test benchmark performances on the tes From Section 5.3 statistical significance is found in out of the 10 benchmarks, indicating
through applying Kendall's Tau
'The Kendall's Tau test is measure of correspondence hetween two rankings. We use this test to see if model ability he correctly ordered simply using linear system with the embeddings as the only feature.

---

Published as a conference paper at ICLR 2025
that model embedding contains information distinguish between model performances most benchmarks.
Predicted vs. True Model Accuracy on MATHQA
Benchmark
Significance
Predicted Accuracy
MathQA
100
LogiQA
100
Predicted Accuracy
MedMCQA
100
PIQA
98
TruthfulQA
96
622
MMLU
94
GSM8K
93
GPQA
10
ASDiv
6
SocialQA
3
Actual Accuracy
Figure 5: Left: Sorted Kendall's Tau test result of accuracy prediction on the benchmarks. The "Significance" column represents the number of times with significant correlation detected (at 5% significance level) out of 100 random model splits. Right: An example comparing actual model accuracies on MathQA against model accuracies on MathQA predicted from the embeddings trained without MathQA data.
Notice that this prediction systems works even when we leave out large benchmarks like MMLU, method predicts MMLU accuracy statistically significant extent. As number of models, model sizes and number benchmarks are still rising rapidly, enabling benchmark accuracy predic- tion through model embeddings vital save both time and compute from repeatedly inferencing models on every new benchmark with potentially overlapping questions from previous ones.
WHAT INFORMATION IS IN THE MODEL EMBEDDINGS
In this section we deseribe the probing experiments designed understand what information is captured in the embedding.
6.1 SANITY CHECK USING SIMILARITY
We expect the model embeddings to satisfy some basic properties: two models M, M' generate the same answers for every prompt, then their embeddings are the same. Similarly, models with similar characteristics, trained using similar data, or adopted similar training pipelines should have similar embeddings, and vice versa. For instance, the model embedding of DeepSeckMath-T5 (Shao) eet al. al. (0244) should be more similar to the embedding of other math models like MetaMatt--Lemman 7B (Yu al. 2023) than to the embedding of Medicine-LLMM-133 (Cheng al.) 2024) which is adapted for biomedical applications. This property is easily fulfilled by EmbedLLM as any two identical/similar models of such would produce identical/similar correctness result against most questions.
As further sanity check, we assign binary labels the 112 models we have evaluated according to the following 6 keywords: [7B, 13B, 70B, Coding, Bio/Med, Physics], forming characteristic communities. For each community, we compare between the average intra-community and inter- community L2 distance of the embeddings. As shown in FFiuure 6) for all above 6 communities, the averaged intra-community 1.2 distances are smaller than the inter-community ones. This provides a preliminary guarantee that our embeddings are "meaningful" with respect to distance metrics.
6.2 EMBEDDINGS CAPTURE INTRINSIC CHARACTERISTICS OF BENCHMARKS
Next, as indicated from Section 5.3 as set of model embeddings produced from a fixed training set, the embeddings seem to capture information of some benchmarks better and over- look information in some benchmarks. Hence, we design a set of ablation experiments to fur- ther understand the contribution of each benchmarks in the training data. Specifically, extend-
8

---

Published as a conference paper at ICLR 2025
Averaged Intra-Community Inter-Community Distances
20.00
Community
Inter-Community
Averaged L2 Distance
6.0
029
0.0
7B
13B
70B Coding Community
Bio/Med Physics
Figure 6: The averaged intra-community L2 distance of the model embeddings is closer for all communities selected, suggesting that basic model traits are captured in the latent embedding space.
ing the experiment setup from Section 5.3 5.3/we we have a question embedding tensor X of shape [oumtquestions,, embedding dimension) and a label tensor Y where Yi, is the binary label of whether model i correctly answers question j. with questions from set of benchmarks S {Bi, Bg... ... Ba). J. to measure the effect mncopporaiin/rrmovvnng an "contributor" benchmark B, predicting correctness of a "testee" benchmark By.
1. Construct two sets of benchmarks Sudded and produce two new sets of question embedding and label tensor, Xaddeds Xremoveds Yadded, and Y Yremoved, that only questions containing in Sadded and Sremoved are kept respectively.
S B, and Sremoved S (B, By)
2. Train EmbedLLM embedder separately on (Xadded: Yudded) and (Xremoved: Yremoved) to get two sets of model embeddings Eudded and Eremoved- and respectively perform zero- shot benchmark prediction on Bj with 100 random splits of models as in Section 5.3 Aggregate the total test mean squared error (MSE) Cremoved and Cadded-
3. Take the difference between the two error to compute contribution score Ciş Eremoved Cadded which quantifies the improvement on predicting model accuracy on B₁ when B, is added training.
Essentially, we hypothesize that the addition/removal of every training benchmark would be re- flected through the change in model embeddings, which then induces a performance difference in benchmark accuracy prediction. With this setup, we produce a n x n (n is the total number of bench- marks) contribution matrix C' where the where the ij-th entry can be extracted exactly Ciş from the steps above To aggregate the overall effect of one specific benchmark B, the contributor benchmark, we compute a row sum of the contribution matrix Σ, Cij- This can be interpreted as the total improvements of adding benchmark B, on predicting model correctness of the rest of the benchmarks. Correspondingly, the column sum Σ, Cij represents the total amount of improvement of predicting correctness of benchmark B, when each of the rest of the benchmarks is added.
Some noticeable phenomenon emerged from this set of experiment
1. We find that incorporating MMLU into training the embeddings significantly help pre- dicting accuracies on other benchmarks. This result matches the comprehensive nature of MMLU as it contains questions covering various topics.
'The diagonal entries of this matrix default 0 it meaningless train and the benchmark.
WW omit results of SocialQA in probing experiments most of our models perform similarly poorly, resulting all model accuracies crowding in the low accuracy region and hecoming indistinguishable.

---

Published as a conference paper at ICLR 2025
Row Sum of Contribution Matrix C
Column Sum of Contribution Matrix C
Level of Effect
Level of Effect
mmi
medmcos
maings
Grig
piaze
butthfuli
logige
misti
medmecdd
matrott
pige
Binsk
truttifuig
Ingign
Benchmark
Benchmark
Figure 7: Left: Effect of removing benchmarks on testing all other benchmarks. Higher value sug- gests that the addition of that benchmark into the training of model embedding enhances predicting model correctness on the rest of the benchmarks. Right: Effect of removing each of the rest of the all other benchmarks on the benchmark being tested. Higher value suggests that predicting models' correctness from the tested benchmark becomes easier when other benchmarks are added into train- ing.
2. We find that incorporating other benchmarks into training set would harm the embedding's predictive power on GPQA. This suggests that additional information that the embeddings capture from incorporating new benchmarks into training set is unrelated or negatively related to model performance on GPQA. fact, GPQA an extremely difficult benchmark for current LLMs, so this finding aligns with our expectation model's ss ability in answering simpler questions clearly do not transfer to answering harder ones.
3. Additionally, we identify subsets of benchmarks that mutually improve each other's accu- racies when incorporated into training. For instance, there is a total MSE improvement of 0.271 when GSM8K is incorporated to predicting MathQA, 0.190 when MathQA is incor- porated to predict ASDiv, and 0.103 when ASDiv is incorporated predict MathQA. As all three benchmarks are math-related, we can deduce that the level of math knowledge of our selected models are indeed learned and reflected in our model embeddings.
LIMITATION
Despite the promising results, our work has several limitations. First, our dataset, though effective in demonstrating the potential of our embeddings with a limited number samples, is relatively small. With data from only 112 models, the embeddings limits deeper exploration of relationships between them. Second, EmbedLLM is a static system. Despite the low cos introducing new models still requires retraining. Lastly, our study is restricted to correctness-brat of model outputs, unexplored. To address these limitations, we have open-sourced our datasets and codebase for further research and experimentation.
extract are moderately sparse which
datasets, leaving other potentially valuable data types, such as text embeddings
8 CONCLUSION
We showcase the possibility of learning an unified, compact representation of LLMs. Through exten- sive empirical evaluation, our method displays solid performance on correctness forecasting, model routing, and benchmark accuracy prediction, while significantly reducing the need of retraining and avoiding repetitive evaluations. Furthermore, we conduct various probing experiment to understand the information contained in the model embeddings. The results show that our embeddings capture not only key characteristics of the models, but also properties of the data used to train the embedder.
TTrinning EmbedLLM on correctness matrix of around 20,000 questions on 112 models for 50 epochs with batch size 2,048 costs 107.71 TFlops, approximately equivalent to the querying 7B model for 60 times using an input of length 128.
10

---

Published as a conference paper at ICLR 2025
REFERENCES
Aida Amini, Saadia Gabriel, Shanchuan Lin, Rik
Yejin Choi, and Hannanch Hajishirzi. MathQA: Towards interpretable math word problem solving with operation-based formalisms. In Proceedings of the 2019 Conference of the North American Chapter of the Associ- ation for Computational Linguistics: Human Language Technologies, Volume (Long and Short Papers), pp. 2357-2367, Minneapolis, Minnesota, June 2019. Association for Computational Lin- guisties. doi: (0.11863//1/N1911 1245. URL https: baastps:/accannnen ac http///aaaaanooo..ooororoooo////
Yonatan Bisk, Rowan Zellers, Ronan Le Bras, Jianfeng Gao, and Yejin Choi. Piga: Reasoning about physical commonsense in natural language. 2019. UR//stpp:://axvvv..r////bb
11641
Daixuan Cheng, Shaohan Huang, and Furu Wei. Adapting large language models via reading com- prehension. In The Twelfth International Conference on Learning Representations, 2024. URL
https://openrews
net meeee/ooruniiidyy88X00222
Wei-Lin Chiang, Lianmin Zheng, Ying Sheng. Anastasios Nikolas Angelopoulos, Tianle Li, Dacheng Li, Hao Zhang, Banghua Zhu, Michael Jordan, Joseph E. Gonzalez, and lon Sto- ica. Chatbot arena: An open platform for evaluating Ilms by human preference, 2024. URL
https::/arxiis http///xxvvv..g/aa.a../a w20.:///////......oo..// 04132
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168. 2021.
Evelyn Fix. Discriminatory analysis: nonparametric discrimination, consistency properties, vol- ume USAF school of Aviation Medicine, 1985.
Leo Gao, Jonathan Tow, Baber Abbasi, Stella Biderman, Sid Black, Anthony DiPofi, Charles Fos- ter, Laurence Golding, Jeffrey Hsu, Alain Le Noac"h, Haonan Li, Kyle McDonell, Niklas Muen- nighoff, Chris Ociepa, Jason Phang. Laria Reynolds, Hailey Schoelkopf, Aviya Skowron, Lin- tang Sutawika, Eric Tang, Anish Thite, Ben Wang, Kevin Wang, and Andy Zou. A framework for few-shot language model evaluation, 12 2023. htLhhttss://eeet owwp:ioorrrrrrrrreeoe
10256836
Yaru Hao, Haoyu Song, Li Dong, Shaohan Huang, Zewen Chi, Wenhui Wang, Shuming Ma, and Furu Wei. Language models are general-purpose interfaces, 2022. URL https://arxiv:
org/abs /206 06336
Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked au- toencoders are scalable vision learners. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pp. 16000-16009, June 2022.
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. Proceedings of the Interna- tional Conference on Learning Representations (ICLR), 2021.
Qitian Jason Hu, Jacob Bieker, Xiuyu Li, Nan Jiang, Benjamin Keigwin, Gaurav Ranganath, Kurt Keutzer, and Shriyash Kaustubh Upadhyay. Routerhench: A benchmark for multi-llm routing system, 2024. URL https::tpp///..... http///aivvv...a///iv//aaa/aaaaa /tgg
Stephanie Lin, Jacob Hilton, and Owain Evans. Truthfulqa: Measuring how models mimic human falsehoods, 2022. URL htt.ttttts:///// http:////wxvv..or///////////
Jian Liu, Leyang Cui, Hanmeng Liu, Dandan Huang, Yile Wang, and Yue Zhang. Logiqa: A challenge dataset for machine reading comprehension with logical reasoning, 2020. URI. bietps://arxtt orgg abss / 2007. 08124
Keming Lu, Hongyi Yuan, Runji Lin. Junyang Lin, Zheng Yuan, Chang Zhou, and Jingren Zhou. Routing to the expert: Efficient reward-guide guided ensemble of large language models, 2023. URL 08692
https://arra wwwp:/xwwwwwwwoooooe/ooooooose//

---

Published as a conference paper at ICLR 2025
Shen-yun yun Miao, Chao-Chun Liang, and Keh- Yih Su. A diverse corpus for evaluating and developing english math word problem solvers. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 975-984, 2020.
Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. Efficient estimation of word represen- /ttp::///xxxivv..oo. org abs//301.......................a 1301.3781
tations in vector space, 201 3a.
Tomas Mikolov, Wen-tau Yih, and Geoffrey 7.weig. Linguistic regularities in continuous space word representations. In Lucy Vanderwende, Hal Daumé III, and Katrin Kirchhoff (eds.), Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 746-751, Atlanta, Georgia, June 2013b. Asso- ciation for Computational Linguistics. URL https: : / ac aclanthology..... org/ N1 3- 1090
Mehdi Noroozi and Paolo Favaro. Unsuperrvised learning of visual representations by solving jigsaw puzzles, 2017.
09246
Isaac Ong. Amjad Almahairi, Vincent Wu, Wei-Lin Chiang, Tianhao Wu, Joseph E Gonzalez, M Waleed Kadous, and Ion Stoica. Routellm: Learning to route Ilms with preference data, arXiv preprint arXiv:2406. 10066/860 18665, 2024.
Ankit Pal, Logesh Kumar Umapathi, and Malaikannan Sankarasubbu. Medmcqa: A large-scale multi-subjec multi-choice dataset for medical domain question answering. In Gerardo Flores, George H Chen, Tom Pollard, Joyce C Ho, and Tristan Naumann (eds.). Proceedings of the Con- ference on Health, Inference, and Learning, volume 174 of Proceedings of Machine Learning Re- search, pp. 14882660.100 PMLR, 07-08 Apr 2022. URL /tttpp::/proceeee http::///ooceedeeeeeeem ngs ml press/ v174/pal22a.ncE html
Jeffrey Pennington, Richard Socher, and Christopher Manning. GloVe: Global vectors for word representation. In Alessandro Moschitti. Bo Pang, and Walter Daelemans (eds.), Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1532-1543, Doha, Qatar, October 2014. Association for Computational Linguistics. doi: 10. 3115/v 1162.
Nils Reimers and Iryna Gurevych. Sentence-bert: Sentence embeddings using siamese bert- networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language
Processing. Association for Computational Linguistics, 2019. URL
org/abs/ 1908. 10084
David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael and Samuel R. Bowman, Gpqa: A graduate-level google-proof q&a benchmark, 2023. URL https::tp///.mmm //tpx/awvvvv///////////../..wa ortpa/ss/rr///////////// 20///231...00 12022
Olaf Ronneherger, Philipp Fischer, and Thomas Brox U-net: Convolutional networks for biomedi- cal image segmentation, 2015. URL httt:tttts://aatt wwt.:oggg//aaaa.ee.o wtt.:ogggabb/....00000 04597
Maarten Sap, Hannah Rashkin, Derek Chen, Ronan LeBras, and Yejin Choi. Socialiqu: Common- sense reasoning about social interactions, 2019, URL
09728
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. Deepseekmath: Pushing the limits of mathe- matical reasoning open language models, 2024. 03300
Tal Shnitzer, Anthony Ou, Mírian Silva, Kate Soule, Yuekai Sun, Justin Solomon, Neil Thompson, and Mikhail Yurochkin. Large language model routing with benchmark datasets, 2023. URL
https://axxixe http://///axlv.ooro////////////0
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need, 2023. URL bttps:///axx:
org Skg//bs//17 /706. 03762
Carl Vondrick, Abhinav Shrivastava, Alireza Fathi, Sergio Guadarrama, and Kevin Murphy, Track- 1806. 09594
ing emerges by colorizing videos, 2018. URL.
12

---

Published as a conference paper at ICLR 2025
Longhui Yu, Weisen Jiang, Han Shi, Jincheng Yu, Zhengying Liu, Yu Zhang, James T Kwok, Zhen- guo Li, Adrian Weller, and Weiyang Liu. Metamath: Bootstrap your own mathematical questions for large language models. arXiv preprint arXiv:2309.12884 2023.
Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. Judging li-as-a-judge with mt-bench and chatbot arena, 2023.
Banghua Zhu, Evan Frick, Tianhao Wu, Hanlin Zhu, Karthik Ganesan, Wei-Lin Chiang, Jian Zhang, and Jiantao Jiao. Starling-7b: Improving helpfulness and harmlessness with rlaif. In First Con- ference on Language Modeling. 2024.
ACKNOWLEDGEMENTS
This work was partially supported by NSF Grants IIS- ISS-1902252 and SDF-2211209) We sincerely thank the funding agencies for their support. Additionally, we appreciate the valuable feedback from the reviewers that helps improve this work.
A
APPENDIX
A.
MODEL LIST
Here an exhaustive list of models that we extract our dataset from:
13

---

Published as a conference paper at ICLR 2025
meta-laaaa/llmmaauaarr
D0l-ai/i--34B-CCE
WizardLM/WiizrddL---0000-
allenaiituluo-2-pp-000
Imsys/vicuua-133-v1155
Imsys/vicuma-333b-11..
Qwen/Qween1L4B--Chr
opencchat/opeenchar-3355500066
opencchat/oppenhhat-3355
Werkeely--ees/Sttannn-LL1-------
HugginngFaacct///pppyyy/bbbbbb
TheeBloke/ullu-0BB--pp16
mistrdai/MMistal//BBnnnsne-m//
tiluarffalcon--00-nnnttuutt
Insy:/vicuna-1/bbvll-5-1kkk
Imsys/vicuna--b--v1.55
Nouskessaardd/Woou--HHHeee11113
project-baizzcbaaiee-v21111
Jmays/Wicuuaa77--115-11166
mosaaicm//mnp--300-iinssnue
meta-llams/Laamm22227cchhhhaah
The-Blokckkooaee/3B-IIr
tommicaalgg44ul--13--nooozz
mossaccm//mmpt77bccha
databbrikkv/ooll--2--22b
sabblityaissabllchm--uucchaaaapp
deppscek-i/leepssek-lln------bb
CausalLMM344--beta
SUSTech/SUUSCChat-:44 SUS
SUSTTcchSSUSS-Chatt222
Qwen/Qwen-72B
Inte//neural-chaa-7--333
bbivbivallpacc--ragoo-72222
Jayyconkang/CCKKAAuuueV
ConvessA//Lumiiexe-34B--000
bardsal/askkerr77-4p--556
FelixChaa/Scorpio-75
dfurman//eermessaagg-333----00
kevind09//laaaRRAGGraan
sail/Sailor-7B
AiMaavenAA/Prromehhuuss....
Q-bert/Optimus-77B
zheng//Mix11A0-7:222MMer
mistaahi/Miirrra--------suu----
microosott/Orca-221133
EleutherAU/pythia-115
cloudyu/Mixtral-11BB22MM
rishiraj/CatPPP--ase
Deci/DeciL.M-77
microsofttphii-2
scb10x/typhoon-75
01-ai/Yii6BB-200K
01-ai/Yi-6B
TigerReseauchhrigeeot-----
augmxnttshisa--aase-77vo
microssoftphii-155
golaxy/gowizardlmm
bigssiencee/bloom-7hbl
mlabomee/AApllaMMnaachh-77
CultniX/Neerra/Tri--66f66
shadowml//MBeeglee--78
yan--jelesEExerrieee22--777
Telix Chaalllam22----mmmaaa
MariyarPanahiWWizzdLLLMM..
-.0-----MMaMaa---..... 70B-v0.1
abhishek/zeply---ta--maall
EleutherAA//llemma-3344
EleutherA//llemmaa>hb
PelixChaoo/iccuae----hhyiiic
Harshwii//Laama--277Byyyysss
TelixChaao/icuua//BBcheeeeee
BioMMistaa//BooMissaall11-
BioMissra//BioMiissrr..B-DDDDD
PharWoltk//BBoMcacGFT..MM11
Booninicr--A/AANNIAAANeetaay
codellamma/CCddelllaaaahhhht
deepseekaaiddepseee-coorri----
AdaptLLLMmeediiineet
AdippLLLN/meddicnneLLLLLL
Writer/palmnyra-med-2000
Neko-
SciPhhlSScihi--Miistll--1--22
shiceee/mistallkoocccch-ciceeee
Wizardl Wizard
bigcode/occoocoder
mictellamm/MMcee-llaa-33--33
mitt-llan/MMMc--llaaa33-------
meta--llnna/Meetallamm------999
metelllama//een-llanna---------p
meta- Taetl-lllmm/Meee-Laaaaannnuuuu
Qwen/Owwenl.--22B-
Dwwen/Qwenl.5-4B-2
Owven/Qwenl.5-0.55
Dxven/Qwen1.5-77B-CC google/gemmaa77bit
Nexusfoww/tarring-LM---1-
google/gemma--2bit
Table 2: The comprehensive list of the 112 models used to create the correctness dataset. This
dataset is created on May 2025 so models released after that time are not available.
14


Reviews:
[{"review_id": "mO9UDpUGtq", "reviewer": "Reviewer_WPNj", "summary": "This paper proposes a novel framework for generating compact vector representations of LLMs to enhance model routing, task efficiency, and performance forecasting. The EmbedLLM framework creates embeddings that capture important characteristics of different LLMs, such as suitability for specific tasks like coding or conversational response generation. Experiments results indicate that the embeddings effectively capture key characteristics of LLMs, enabling efficient and accurate task allocation and performance prediction across a variety of benchmarks.", "questions": "Please address the weakness I mentioned above.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "Embedding LLMs to handle downstream tasks is indeed a fascinating approach! This method allows you to create compact representations of each model that capture its unique strengths and weaknesses, enabling efficient task-specific decisions without running each model on every input. This approach streamlines the workflow significantly, as it allows for general-purpose embeddings that can adapt to a variety of downstream tasks without retraining the models themselves. It's especially beneficial in settings where computational resources are a concern or when the model pool is large.", "weaknesses": "The term \"decoder\" in this paper is a bit misleading. In typical encoder-decoder architectures, the \"decoder\" reconstructs or generates the output in its full or intended form, such as reconstructing text in sequence-to-sequence tasks. Here, however, the so-called \"decoder\" is merely a binary classifier that outputs a label indicating whether the LLM correctly answered a question.\nWe have to re-train the embedder if we want to represent new models, this makes the whole framework non-scalable. I'd like to see details cost metrics for re-train embedder for new model vs traditional benchmarking approaches.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "VETQLE6FJJ", "reviewer": "Reviewer_Gpg4", "summary": "The paper presents EmbedLLM, a framework for creating compact vector embeddings of large language models (LLMs) to improve efficiency in tasks like model routing and benchmark performance prediction. EmbedLLM uses an encoder-decoder architecture to map LLMs into a unified embedding space that captures important model characteristics. This representation allows accurate model selection and performance forecasting across multiple tasks without repetitive re-evaluation, reducing both time and computational costs. Experiments show that EmbedLLM enhances accuracy and latency in model routing and can predict benchmark scores reliably. The embeddings also reflect intrinsic model attributes, useful for identifying task-specific strengths, even in models not explicitly trained for certain tasks.", "questions": "See the weaknesses", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 4, "strengths": "- The paper innovatively proposes the embedding of LLMs to facilitate managing and comparing them.    \n- The experiments in the paper are comprehensive, tested on 112 large models", "weaknesses": "- The paper proposes a method for encoding LLMs. However, in the implementation, this encoding is merely based on model IDs, treating each model entirely as a black box. With only 30,000 data for training, can the resulting encoding truly capture all the characteristics of the models? Large models differ significantly in their strengths across various domains and capabilities. Can such an approach, based solely on one round question-answer pairs, truly distinguish the models’ abilities when facing complex reasoning problems? Another issue is whether the scale of the proposed embedding network is sufficient to represent the characteristics of numerous models effectively.\n\n- A little confused about line 256-257. Why are two values (p(m,q)_0 and p(m,q)_1) output here? Is it simply to add a nonlinear operation?\n\n- The experiments do not appear to categorize models by scale. Intuitively, the larger the model, the smaller the performance differences between models. What is the authors' view on this issue?\n\n- The paper does not seem to provide a clear and intuitive illustration of the overall architecture, including training, input, and output processes. Figures 1-3 are quite similar and take up too much space, and it wasn’t until section 4.1 that I understood how the whole system works. Of course, I do not deny the paper's contributions, but I suggest the authors improve the visual presentation.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "cZAuw512KA", "reviewer": "Reviewer_HPLR", "summary": "The goal of this paper is to represent LLMs using an embedding that can predict performance on new inputs and benchmarks. Such an embedding can be learned from matrix factorization methods applied to a matrix that contain the behavior of the LLM on several data points as well as behavior of other LLMs on these datapoints, i.e., a matrix with rows as LLMs and columns as data points, and each cell indicate an LLM's performance on a specific data point. The factorization aims to reconstruct this matrix by learning an embedding for each LLM and an embedding for each data point (built from its static embedding which is further projected into the same embedding space of LLM). \n\nThe resulting embeddings are evaluated to predict performance of new data points and benchmarks and to improve model routing.", "questions": "1. Why does the method assume that answer have to be a binary label? Is this a limitation of this method?\n2. The explanation of kNN-classifer baseline is unclear. How is final label chosen? -- based on the majority voting of k-nearest neighbours?\n3. Lines 264--267 are unclear. What is mxp and nxp represent?\n4. Some qualitative discussion with examples on what happens when the resulting data points are widely different from already seen data points will be useful. \n5. Does the prompt embedding have an impact on the results, i.e., using better recent LLM-based embeddings. \n6. I would also like to see the TSNE plots of the embeddings.\n\nMinor:\n1. Vaswani et al. should be 2017. The paper opens up with a wrong citation. This paper is also wrongly attributed to LLMs. You could be more careful with your citations.\n2. The method should perhaps be named EmbedLLM rather than Matrix Factorization.\n\nI am willing to increase the score based on better presentation and qualitative analysis. The paper is quite weak in this aspect. \n\n====\n\nI increased my score after author rebuttal. I am worried about the quality of these embeddings given that no patterns are observed in TSNE plots, but a score of 7 is appropriate for other clarifications. Since there is no 7, I had to chose 8.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. A simple and intuitive method.\n2. Once the embedding of an LLM is built, the performance of the LLM can be accessed without access to the model.", "weaknesses": "1. The paper can be motivated better and clarify why the current formulation is intuitive, i.e., a model's behavior on new data points can be predicted based on its behavior on already existing data points.\n2. Details are hard to follow or underspecified, e.g., kNN classifier, random routing.\n3. There are no references to highly similar work that predicts performance on a new task based on the performance of existing tasks. For example, Xia et al. Predicting Performance for Natural Language Processing Tasks, 2020\n4. Calling the method as encoder-decoder in Sec 4.3 is confusing. The decoder is nothing but a classifier, and calling it straightforwardly that will make it easier to follow instead of calling it a decoder.\n5. There is no analysis/visualations of the embeddings. Current histogram plot in Figure 6 is not that informative. Could you provide TSNE plots of the resulting embeddings.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Spotlight)