Pack of LLMs: Model Fusion at Test-Time via Perplexity Optimization
Authors: Costas Mavromatis
Petros Karypis
George Karypis
Venue: COLM (2024)
Keywords: Large Language Models, Model Fusion
TL;DR: PackLLM fuses LLMs at test-time via a perplexity optimization framework.

Abstract: Fusing knowledge from multiple Large Language Models (LLMs) can combine their diverse strengths to achieve improved performance on a given task. However, current fusion approaches either rely on learning-based fusers that do not generalize to new LLMs, or do not take into account how well each LLM understands the input. In this work, we study LLM fusion at test-time, which enables leveraging knowledge from arbitrary user-specified LLMs during inference. We introduce Pack of LLMs (PackLLM), an effective method for test-time fusion that leverages each LLM’s expertise, given an input prompt. PackLLM performs model fusion by solving an optimization problem for determining each LLM’s importance, so that perplexity over the input prompt is minimized. First, our simple PackLLM-sim variant validates that perplexity is a good indicator for measuring each LLM’s expertise. Second, our PackLLM-opt variant approximately solves the perplexity minimization problem via a greedy algorithm. The derived importance weights are used to combine the LLMs during inference. We conduct experiments with over 100 total LLMs on a diverse set of tasks. Experimental results show that (i) perplexity is a reliable measure for LLM fusion, (ii) PackLLM outperforms test-time fusion baselines by 1.89% accuracy points, (iii) PackLLM can leverage new LLMs to improve performance over learning-based fusion approaches by 3.92–11.94% accuracy points, and (iv) PackLLM benefits over selecting the best or largest model and model merging in certain cases. Our code is provided at [https://github.com/cmavro/PackLLM](https://github.com/cmavro/PackLLM).

Published as a conference paper at COLM 2024
Pack of LLMs: Model Fusion at Test-Time via Perplexity Optimization
Costas Mavromatis University of Minnesota mavro01 6@umn. edu
Petros Karypis University of California San Diego
pkarypisêucsd.edt
George Karypis University of Minnesota
karypis@umn.edu
Abstract
Fusing knowledge from multiple Large Language Models (LLMs) can com- bine their diverse strengths to achieve improved performance task. However, current fusion approaches either rely on learning- fusers that do not generalize to new LLMs, or do not take into account how well each LLM understands the input. In this work, we study LLM fusion at test-fime, which enables leveraging knowledge from arbitrary user-specified LLMs during inference. We introduce Pack of LLMs (PackLLM)), an effective method for test-time fusion that leverages each LLM's expertise, given an input prompt. PackLLM performs model fusion by solving an optimization problem for determining each LLM's importance, so that perplexity over the input prompt is minimized. First, our simple PackLLMMsnn variant vali- dates that perplexity good indicator for measuring each LLM's expertise. Second, our M.ckLLLMppt variant approximately solves the perplexity mini- mization problem via greedy algorithm, The derived importance weights are used to combine the LLMs during inference. We conduct experiments with over 100 total LLMs on a diverse set of tasks. Experimental results show that (i) perplexity is reliable measure for LLM fusion, (ii) PackLLM outperforms test-time fusion baselines by 1.89% accuracy points, (iii) Pack- LLM can leverage new LLMs to improve performance over learning-basee fusion approaches by 3.92-11.94% accuracy points, (iv) PacKLLM benefits over selecting the best or largest model and model merging in certain cases. Our code is provided at https: / /gi thub. com/ cmcvro/PackLLMM
given
Available LLMs (by 2023)
New LLMs (in 2024)
PackLLM
user- defined
X
Fuse & Train
Retraining
Fusion at Test-Time
Fuse LLMs
Learn-to-Fuse
Figure 1: PackLLM performs LLM fusion at test-time. PackLLM (i) does not require any training of fusion models, (ii) can leverage new (more powerful) LLMs when released, and (iii) allows users to select their preferred LLMs at test-time.
1

---

Published as a conference paper at COLM 2024
1 Introduction
Large Language Models (LLMs) (Brown et al., 2020; Touvron et al., 2023) have achieved state-of-the-art performance on a variety of natural language processing (NLP) tasks and are beginning to be applied as Foundation Models (Bommasani et al., 2021) for gamut of applications. As a result, we observe a sharp increase of newly released LLMs, that are pretrained on diverse corpora or fine-tuned to solve specific tasks. Moreover, it has become strategic interest that corporations train and release their own LLMs, amplifying the number of available LLMs. LLMs usually have architectural differences as well they use different pretraining data, and as a result, their knowledge expertise is diversified. Therefore, an emergent question is "How can we effectively combine knowledge of the available LLMs to improve downstreemm performance?".
Fusing knowledge from different models is long-standing problem in machine learn- ing (Ho, 1995; Sollich & Krogh, 1995; Kuncheva & Whitaker, 2003; Schapire, 2003). Regarding LLMs, current approaches train dedicated fusion modules, such as neural networks (Wang et al., 2023) or additional LLMs (Jiang et al., 2023b; Wan et al., 2024; Ding et al., 2024), that learn to combine LLMs from a seed set. However, these approaches are not modular, meaning they need to undergo expensive and time-consuming retraining to encompass newly released LLMs or remove specific LLMs, e.g,, due to licensing issues.
We present Pack of LLMs (PackLLM), a lest-time fusion method. As shown in Figure 1, PackLLM does not require any training of fusion modules, while it can combine arbitrary user-specified LLMs during inference. In order to fuse knowledge from the seed LLMs at test-time, PackLLM performs a weighted ensemble of the output logits by posing an optimization problem at test-time. PackLLM minimizes the perplexity over the input prompt, so that the fused LLM understands the task better. First, our simple PackLLMaim variant validates that perplexity is a good indicator for measuring each LLM's expertise. Second, our PackLLMppt variant approximately solves the perplexity minimization problem via greedy algorithm. The derived importance weights are used for combining the LLMs during inference.
We conduct experiments with over 100 total LLMs and evaluate performance on language modeling tasks (Section 4) as well as on downstream tasks (Section 5). Experiments show that PackLLM's perplexity-based framework is a reliable indicator for determining fusion, outperforming approaches such as cBTM (Gururangan et al., 2023) and DExperts (Liu et al., 2021) that do not take into account how well the LLMs understand the input, while its overall performance scales better with respect to these baselines as we increase the number of expert LLMs. In addition, PackLLM achieves a significant improvement of 1.72- 1.89% accuracy points (averaged over 25 tasks) over existing test-time fusion approaches, such as top expert selection and uniform ensemble. Furthermore, by employing newly released LLMs, PacKLLM outperforms competing learning-basse based fusion approaches, such as FuseLLM (Wan et al., 2024) and FoE (Wang et al., 2023), by 3.92-11.94 1194% accuracy points, using little x.92x fewer parameters. Our contributions are summarized below:
Problem Formulation: We study fusion of LLMs at test-time as a weighted ensemble and pose perplexity minimization problem for determining the LLM importance weights. Experimental results show that PackLLM benefits over selecting the best largest model and model merging in certain cases.
Algorithm: We present PackLLMopt that solves the perplexity minimization prob- lem via a greedy algorithm. We also present PackLLMaam. a simple, but effective, perplexity-basee ensemble.
Effectiveness: We evaluate PackLLM on a diverse set of tasks showing that (i) perplexity is a reliable indicator for model importance, (ii) PackLLM outperforms test-time fusion baselines by 1.72-1.89% accuracy points, (iii) PackLLM can leverage newly released LLMs and outperform competing learning-based fusion approaches by 3.92-11.9499 accuracy points.
2

---

Published as a conference paper at COLM 2024
Leam-to-Fuse training
Modular at test-time: Yes
Modular at test-time: Yes
Effective: Depends
Effective: Yes
T
Ensemble
PackLLM
Inl
In
Seed LLMS lest-éimo
Seed LLMs test time
u7
Train
Test-lime LLMs
d
Inl
In
In
A
A
A
Uniform Aggregetion
Aggregation
via Perplexity
Output
Output
Figure 2: Overview of LLM fusion paradigms. Left: Leamiin-basee fusion approaches not modular and cannot encompass new L.MSs at test-time. Middle: Uniform ensemble may degrade performance, given weak seed LLMss. Right: Our PackLLM approach determines the weights of the seed LLMs at test-time via perplexity to achieve effective performance.
2 Related Work
Regarding fusion of LLMs, weight merging (Yadav et al., 2023) and mixture-of-experls (Komat- suzaki et al., 2022) are techniques that fuse knowledge stored in LLMs by combining their model parameters. However, such approaches require that the given LLMs share the same architecture, which limited with the current availability of powerful LLMs of varying sizes. An alternative technique is majority voting (Li et al., 2024a), but can only be applied to certain downstream tasks, such as classification tasks.
Recent successful fusion approaches often follow learn-to-fuse framework (see Figure left), where specialized learnable modules (Wang et al., 2023; Sukhbaatar et al., 2024) or even LLMs (Jinng et al., 2023b; Wan et al., 2024; Ding et al., 2024), are trained to combine knowledge from set of seed LLMs. However, these approaches are not modular, meaning arbitrary user-specified LLMs and require additional retraining when models need to be added in or removed from the seed set. Approaches, such as (Feng et al., 2024), fuse the generations of different LLMs, but require strong (large-scale) LLM as a fuser, which is computationally expensive.
they cannot generalize
On the other hand, model ensemble approaches (Liu et al., 2021; Gururangan al., 2023; Liu et al., 2024; Li et al., 2024b) perform fusion at the output-level, i.e., output token logits or probabilities. However, effectively combining the LLMs during test-time, i.e., accurately determining each LLM's importance, has not been extensively studied as these approaches do not take into account how well each LLM understands the input. For instance, uniform ensemble may degrade performance if the set includes weak LLMs (Figure middle).
3 Pack of LLMs (PackLLM)
In this work, study the following problem:
Given a set of user-specified LLMs ( [M,..., Mx) at test-time, how can we effectively combine the knowledge of each LLMs to achieve improved performance?
We present Pack of LLMs (PackLLM), weighted ensemble of LLMs that fuses knowledge of arbitrary LLMs at test-time. As shown in Figure 2 (right), given a set of seed LLMs (M)..., Mg] at test-time, PackILLM fuses their knowledge with corresponding importance

---

Published as a conference paper at COLM 2024
weights fAi,...,AK Ax). The importance weights 2ę are calculated based on how well each LLM understands the prompt
In more detail, at each step i, each model M2 is conditioned on the prompt Xxc, to obtain the logit score 9y (x for the next token x,. Logit scores $; are unnormalized scores over the LLM's vocabulary, before applying the softmax(-) operation over all vocabulary tokens which transforms them into probabilities РM;- Here, we assume the models share the same vocabulary, but we provide generalized version in Section 3.4. PackLLM combines output logits to obtain the final probabilities as
p(xi(x<() = softmax
E
A (x(x)(x<1)),
(1)
unknown
where we assume that the weights A E [0, are normalized, i.e., E d3 1. In what follows, we answer the question "How can we accurately determine weights {M,..., Ax} at test-time?"
3.1 Prompt Perplexity Minimization
PackLLM relies the notion that perplexity is suitable measure for LLM's knowledge a certain input (where the lower the better). Perplexity is a metric that aligns with the causal language modeling objective of LLMs, enabling to measure whether test input falls into the LLM's expertise and how the test input relates to the LLM's pretraining data (Marion et al., 2023; Gonen al., 2022).
Given tokenized input prompt or sequence P = (xo, X1, ,,,,.., xt), perplexity (PPL) is defined as the exponentiated average negative log- log-likelihood of P
PPL (P) exp { 1 - log (x)pxx))))
(2)
where log (x(xxxx)) is the log-likelihood of the i-th token conditioned the preceding tokens X-t according to model M₃. Note that PPL depends solely on the models and the input prompt and can be evaluated at test-time.
Leveraging perplexity as a measurement for a LLM'' expertise on the test prompt, we formulate the assignments of importance weights optimization problem that does not require any training or annotated data and can be solved at test-time. Formally, our optimization framework is given
M A arg min
( softmax Σ
(-1
K Asse (x5((2)(xx1)))
Ag....gg
p(x)(x)))
(3)
subject to [0,1] and Σ Ax =
where (Ai,..., } are the optimal importance weights and sy are the output logits of each LLM. During inference, the computed weights ( are used in Equation
3.2 PackLLMaim Simple Perplexity-Based Weighting
An important question whether perplexity reliable metric for determining fusion. As the first step, we present PackL. simple perplexity-based approach for determining the importance weights (Ai...,,Ak) Instead of solving the optimizing problem presented in Equation 3, PackLLMMim computes the weights A directly by using the perplexity scores over the prompt via
A = softmaxi
( llog (-Io).PPs(PP)/T)
(4)
4

---

Published as a conference paper at COLM 2024
where is the temperature hyperparameter with default values T {0.1,1}. Weights {. ) are inversely proportional to the models' normalized perplexity scores, i.e, lower perplexity yields higher weight. PackLLMaim serves validation of whether the models' perplexity scores provide useful information for determining fusion.
3.3 PackLLMopt: Greedy Perplexity Optimization
Unlike the simple heuristic that L.c uses (Equation 4), PackLLMopt approximately solves the perplexity minimization problem presented in Equation 3. The problem in Equation 3 is overdetermined, where the number of equations t (f prompt tokens) and the number of variables is K (K seed LLMs), having many solutions. Nevertheless, it can be solved at test-time via grid search (LaValle et al., 2004) by evaluating different values 2 E [0,1] with the constraint Ag = 1. However, exhaustive grid search is time-consuming during inference due to its combinatorial complexity on the number of LLMs.
We propose an efficient greedy algorithm, which ensembles the LLMs sequentially. The sequential nature of the algorithm reduces the optimization problem to searching between linear combinations of two models rather than the full K models.
First, we compute the perplexity PPL, (P) of each LLM k via Equation 2 and sort the seed LLMs { MM...., MM] based on PPL;(P) as
[M
M = argsort (PPL,())...,PPL((())))
where ...,.., MMi are ordered by the lowest to highest perplexity scores. By using the ordered set, we can omit using irrelevant LLMs via early stopping of the sequential ensemble.
At the first step of the greedy optimization, determine the relative weights between the top-1 and the t LLMs, M; and M;. We solve
( softmax (As (xg + (1 A)s(2) (x/xx<x))))
A11) arg min
(6)
where (1) and g(2) are the output logits by Mi and M;, respectively, and E [0, The optimization in Equation 6 is solved via grid search (greedy grid search), where different values of À E [0, 1] are evaluated (the default step is 0.05). The value (1) that results to the lowest perplexity is used to update
(2 (1) + (1 (1))
for the next step.
The same procedure is repeated to iterate through all seed LLMs MM]. We can perform early stopping when we find (k) 1, i.e., the effect of the current LLM is zero. Our overall greedy optimization is summarized in Appendix A.
3.4 Tokenizer Selection & Alignment
Equation 1 assumes that the logit vectors s; can be added together, which requires that the seed LLMs share the same vocabulary. However, this is not always the case, different LLMs are trained based on different tokenizers. To address this, we employ a tokenizer selection and alignment strategy for combining L.MSs with different tokenizers.
First, we determine the reference tokenizer as the tokenizer of the top-1 LLM (Equation 5). Then, we follow the Minimum Edit Distance (MinED) approach (Fu et al., 2023; Wan al., 2024), where each token of the reference tokenizer is mapped to its textually closest token of another tokenizer, e.g., "get" to "gets". We tokenize the sequence with the reference tokenizer and use the MinED token mappings to obtain valid input for another LLM. With the new input, we obtain the output logits of the LLM over its vocabulary. Then, we map the output logits back to the reference tokenizer's vocabulary via MinED in order to
5

---

Published as a conference paper at COLM 2024
perform logit fusion via Equation 1. MinED mappings between tokenizers are precomputed before inference. We note that aligning different tokenizers is an open problem in NLP, and more effective alignment techniques may further improve PackLLM.
Language Modeling Experiments
The first research question that we answer is:
RQ1. Is perplexity good measure for determining the LLM importance weights at test-time?
We compare with expertise-based model ensemble approaches cBTM(Gurranngan et al., 2023) and DExperts (Liu et al., 2021), which, unlike PackLLM, do not rely on perplexity.
cBTM performs model fusion at the output probabilities, where the weight of each expert is determined based on the tf-idf similarity between the given prompt and the expert's pretraining data. In particular, cBTM computes weights {Ak for a test prompt P for test-time fusion as
Aς softmax( - d(hp, hx)) and p(X//xet) EApPE(X(EEet),
(8)
where d(---) is a Euclidean distance, hp is the tf-idf embedding of the prompt, and h2 is representation of the pretraining data of model k. Note that Equation 8 does not use perplexity, but relies on data embedding similarity.
DExperts and its follow-up works (Liu et al., 2024; Li et al., 2024b) employ an additional base model M and an anti-expert model M to fuse knowledge from an expert model. In particular, the fusion is performed as
p softmax (sM + A[sM; SSM 1),
(9)
where Mî is the top-1 expert from the seed LLMs, and À is a hyperparameter (default value is A = 1). DExperts uses fixed LLM weights.
4.1 Experimental Setup
We follow the cBTM experimental setting, where C4 (Roberts al., 2019) and S2ORC (Lo et al., 2019) datasets are clustered into K E (1, 4,8, 8,8,,,6, 16] clusters, and OPT-1.3B mod- els (Zhang et al., 2022) are trained on each cluster separately. Therefore, each OPT model is an expert on a specific semantic cluster of the training documents. The clusters are computed based on the tf-idf embeddings of the train data and the embedding of the cluster centroid is used h2 in Equation 8 of cBTM. Overall, we experiment with a total of 65 LLMs. To compare with DExperts, we employ the OPT-6.7B, OPT-13B and OPT-30B models as additional base models M. Following DExperts, the anti-expert is the original OPT-113B model. For each dataset, we report language modeling perplexity on 200 randomly-sample held out documents. For each test document, the prompt P consists of the 32 first tokens or the first 20% the sequence's 's tokens, whichever is shorter; perplexity evaluated on the rest of tokens.
4.2 Results
Figure 3a shows perplexity results when we compare PackLLM with the weighted ensemble approach of cBTM. PackLLM outperforms cBTM under all values of the number of expert L.LSs K. The performance differences grow as the number of L.LMs increases and the largest improvement is observed when we have K 16 experts. PackLLM takes into account the perplexity of the underlying experts to estimate their expertise, while cBTM relies on off-the-shelf similarity embeddings and may not successfully capture the LLMs' knowledge. PackLLMMim and PackLLMapt perform similarly on the C4 dataset, while PackLLMain scales better S2ORC dataset, which has shorter prompts. The results show that our perplexity-guiee fusion is effective and scales better as we increase the number of expert models.

---

Published as a conference paper at COLM 2024
cBTM
PackLLMMam
Pac
-BBTM DExxerts
PackLLL kim PaacLLL
14
C4
13
S2ORC
12.5
C4
12.2
S2ORC
Perplexity. 4
12
Perplexity
12.5
12
11.8
11.6
11.5
12
12
11.4
11.2
11
11.5
11
16
4
16
#I.LMs (K)
ML (K)
#LLMs (K)
#LLMs (K)
(a) expert models.
(b) K expert models combined with one large model.
Figure 3: Perplexity result comparison (the lower, the better) on C4 and S2ORC datasets.
Figure 3b shows averaged perplexity results when we add additional large models (OPT- 6.7--3TT-3BB OPT-30B) to the set seed expert LLMs (C4/S2ORC-tuned ОPT1.3B models), evaluating whether the fusion approaches can effectively incorporate larger general LLMs. PackLLM outperforms cBIM and DExperts, and the improvement becomes larger as we increase the number of expert LLMs (value K). Moreover, PackLLMopt outperforms PackLLMMgm in both C4 and S2ORC; its benefit lies on its perplexity optimization, which ef- fectively combines knowledge from expert models with larger models to boost performance during inference. The result show that PackLLM effective with LLMs of varying sizes and expertise.
5 Downstream Tasks Experiments
The following research questions that we answer are:
RQ2. How does PackLLM compare against other test-time fusion approaches in downstream tasks?
RO3. What is the benefit of PackLLLM over learning-brie baaning-baeddddee fusion approaches?
5.1 Experimental Setup
Datasets. We experiment with diverse set of tasks. We employ the MMLU- STEM (Hendrycks et al., 2021) dataset to evaluate performance in knowledge intensive tasks, such college STEM exams. We employ ARC (Clark et al., 2018), BoolQ (Clark et al., 2019), HellaSwag (Zellers et al., 2019), and OpenBookQA (Mihaylov et al., 2018) datasets to eval- uate performance on Commonsense reasoning. Furthermore, we employ MML.U-Heatth PubMedQA (Jin et al., 2019), USMLE (Jin et al., 2021) datasets to evaluate performance on domain-specific knowledge (Medicine). In addition, we evaluate on miscellaneous (Misc.) classification tasks (Mavromatis et al., 2023), such as topic classification, hatespeech detection, and sentiment analysis. Further details can found in Appendix C. We report accuracy question answering and classification as the evaluation metric.
Seed LLMs. For MMLU-STEM and Commonsense tasks, we employ widely used LLMs, such Mistral-7B (Jiang et al., 2023a) and LLaMa2-7B (Touvron et al., 2023). For Medicine tasks, we employ specialized LLMs for biomedical data, such as AdaptLLM (Cheng et al., 2023) and BioMistral (Labrak et al., 2024). For Misc. tasks, tary expertise (Gururangan et al., 2023). Overall, we experiment with a total of 52 LLMs (see Appendix D).
models with complemen-
Baselines. We compare with the following test-time and learning-based fusion approaches. (i) Top1-PPL selects the most relevant LLM for test prompt P. We use prompt perplexity (Equation 5) to determine the top-1 LLM. (ii) Ensemble performs uniform aggregation of the LLM's logits where every weight A has value A R. (iii) FuseLLM (Wan et al., 2024) uses the output logits of the seed LLMs to train a fusion LLM via knowledge distillation. However, it requires external corpus for causal language modeling pretraining and cannot encompass new LLMs at test-time. (iv) FoE (Wang et al., 2023) trains a neural network classifier to select the most relevant expert from the seed LLMss for test prompt.
7

---

Published as a conference paper at COLM 2024
Table 1: Main Results. PackLLM outperforms other test-time fusion approaches in 25 tasks and with different seed LLMs.
MMLU STEM tonks (5-skot)
Commonsense
Medicine basks (0/S-shot)
Misc. husks (3-afiot
Average
faaks (U-skot)
Pest-Time Fusion
#Seeal LLMs K (#1otal Parameters))
(14B))
(23.7B)
(34.7B)
(14B)
(23.7B)
(34.7B)
(27B)
4 (28B)
(15.6B)
cBT
61.95
DExperts
44.37
Ensemble
41.29
43.54
43.51
66.30
64.45
62.10
42.22
51.59
62.53
53.05
Top1-PPL
37.59
40.73
44.67
61.32
64.45
65,31
46,83
50.11
63.95
52.77
44.22
63.84
68.50
54.72
LMct
42.30
45.02
45.60
61.81
67.67
67.58
47.21
53.88
63.38
54.94
-: -:BBTM and DExperts camnot handle arbitrary seed L.Mss.
Table 2: Results LLM fusion approaches. PackLLM leverages recently released LLMs and outperforms learning-based approaches.
Seed LLMs-
FuseLLM LLMS
FOE LLMs (15, 93B) before 08/2023
Recent LLMS
Recent SLMs*
(K, #Params) release date
(3, 21B) before 07/2023
(4, 23.7B) 07-12/2023
(3, 6,5B) 12/2023-02/202
STEM / Com.
STEM / Com.
STEM / Com.
STEM / Com.
Loarning-bnse based fusion
Fusel. LM-7B (Wan et al., 2024)
33.08 58.59 (0) / (0, t)
(G) (0) 40.65 / (t)
(0) (0)
(0) /
FoE (Wang et al,, 2023)
(0) / (C,
(O,t)
Test-fime fusion PackLLM
31.21 / 54.88
40.26
67.05
45.02 / 69.23
37.00 68.25
We use the term SLMs (Small Language Models) to emphasize that their size is below 3B.
O: These approaches require retraining with the new seed LLMs before test-time fusion.
These approaches require additional annotated data for training.
However, it requires annotated data, e-g., validation data for the downstream tasks, which limits its applicability to new tasks and LLMs. We also compare with (v) cBTM and (vi) DExperts when possible, they cannot handle arbitrary seed LLMs (see Appendix B).
5.2 Results
Table 1 compares PackLLM with competing test-time approaches, such as top-1 expert selection based on perplexity (Top1-PPL) and uniform ensemble (Ensemble). PackLLM outperforms these approaches by 1.72-1.89% accuracy points, averaged over different seed LLMs in 25 tasks. The improvement becomes larger when using a greater number of LLMs, e.g., in commonsense (+3.19% accuracy points) and medicine (+2.29% accuracy points) tasks. Uniform ensemble may be negatively impacted when we increase the number of LLMs (not all LLMs are equally strong), while Top1-PPL. improves with more LLMss PackL.LMopt outperforms PackLLMMgm in all tasks, except for Commonsense, where the input prompis are shorter (U-shot input). Moreover, PackLLM outperforms cBTM and DExperts in the cases they can compare against. The results show that PackLLM is an effective fusion approach, outperforming competing test-time approaches by up to 1.89% accuracy points, on average.
In Table 2, we compare PackLLM with learning-based fusion approaches. FuseLLM trains fusion LLM that learns combine knowledge from the seed LLMs over diverse pretraining data, while FoE trains a classifier to select the best expert LLM based on validation data. As a result, both FuseL.LM and FoE have an advantage over our learning-free method when using the same seed LLMs, leading to 0.39-3.71% % more accuracy points. However, PackLLM does not require any training or validation data and can quickly adapt as new LLMs are released. For example, PackLLM improves over FuseLLM and FoE by 4.37-11.94% accuracy points when using four recently released 7B LLMs. In addition, by using strong small LMs (Li et al., 2023), PackLLM outperforms FuseLLM by 3.92-10.66% accuracy points without incurring additional computational costs (equal number of total LLM parameters).
Full results of this section can be found in Appendix E.
8

---

Published as a conference paper at COLM 2024
Table 3: PackLLM comparison with (A) selecting the best model, (B) selecting the largest model, and (C) model merging (average weighting). Due to space limitations, details on experimental setup are provided in Appendix F.
STEM Com. (acc)
STEM Com. (acc)
C4/ S2ORC (ppl) Same Vocub
Different Vorab
Same Voeab
(A) Best LLM
33.0 56.3
34.8 / 59.1
12.53 / 11.46
PackLLM
31.2 54.9
40.3 / 67.1
11.03 11.05
Medicine (acc)
STEM (acc)
C4 (ppl)
(8) Langest LLM
49.9
33.1
12.71
PackLLM
52.7
36.3
11.13
Com. (acc)
C4 (ppl)
S2ORC (ppl)
(C) Model Merging
13.20
12.92
PackLLM
57.1
12.12
12.32
6 Comparison with Model Selection and Merging
In this section, we compare PackLLM with respect to (A) selecting the best model, (B) selecting the largest model, and (C) model merging. The results are summarized in Table 3.
PackLLM vs. Best Model. We experiment with the best LLM (as evaluated on test data) on STEM and Commonsense tasks. As Table 3A shows, the best LLM works better when having few LLMs with different vocabularies, but when we have large number LLMSs with the same vocabulary, PackLLM outperforms the best LLM. By developing better tokenization mapping methods (Section 3.4), PackLLM may become more effective in the first scenario. To further verify our conclusions, we experiment on the C4 and S2ORC language modeling tasks, where we have a large number of expert OPT models sharing the same vocabulary. In this case, PackLLM also outperforms the Best LLM (selected as the top1 expert).
PackLLM vs. Largest Model. Selecting a larger LLM may yield better generalization than selecting smaller models with lower perplexity. However, smaller LMs are easy to fine-tune on in-domain data which can provide complementary knowledge (e-g,, medicine data). In such cases, model fusion yields better results. To demonstrate this, we compare PackLLM with selecting the largest model from the set of models on three different tasks; Medicine, MMLU-STEM, and language modeling on C4 in Table 3B. The results show that PackLLM yields better performance than selecting the largest model in all settings. We provide example cases, where PackLLM improves large model in Appendix G.
PackLLM vs. Model Merging. Although it is not always possible to merge models as fu- sion approach, model merging has lower inference costs compared to ensemble approaches. We provide results by comparing PackLLM with model merging (average weighting) on commonsense reasoning and language modeling tasks in Table 3C. The results that PackLLM outperforms model merging across these two tasks.
7 Ablation Studies
In this section, we ablate PackLLMaim and PackLLMopt with respect to their derived LLM importance weights, their sensitivity on the input prompt, and their time cost.
Importance Weights. Figure 4a shows the derived importance weights A by the two approaches for four different tasks. PackLLLMgim calculates the weights based on perplexity scores (Equation 4) and thus, fluctuations perplexity scores among datasets and L.MSs result in fluctuations in the induced weights. For example, the top-1 weights range within [0.54, 0.89] and top-2 weights range within [0.08, 0.37]. On the other hand, PackLLMgpt solves optimization of minimizing perplexity to determine weights dg, which results in weights of smaller range. For example, the top-1 weights range within [0.56, 0.76] and top-2 weights range within (0.16, 0.31].
Prompt Length. Figure 4b compares cBTM, PackLLM and P with respect to the length of the input prompt. cBTM does not take into account how well the LLMs

---

Published as a conference paper at COLM 2024
C4 S2ORC STEM + Comm. Ensemble
cBIM PackLLM PackLL
Lac
Pac
C4
12
0.8
Weight Ag.
0.6
Perplexity
11
0.4
0.4
10
0.2
0.2
top-1
lop-2
top-3
lop-4
0
'top-1
lop-2
top-3
lop-4
20%
40%
60%
80%
Top-
Top-k LLM
% Sequence Length
(a) Importance weight values for PackLLMsim (left) and Pack (right) in tasks.
(b) Perplexity results with respect to input prompt length (x-axis).
Figure 4: Ablation studies of PackLLM and PackLLMopt-
understand the input and shows minor improvements with longer prompts. On the other hand, both MackL.L.Msmm and Map significantly improve as we increase the length of the input. Moreover, PackLLMMpp outperforms PackLLM due to its optimization framework, which specializes into understanding the input better.
Computation Time. Table 4 compares the time cost of different optimization ap- proaches for determining the weights Aş on the C4 dataset. PackLLMssm does not solve an optimization problem and thus does not result in any time overhead (row "None" in Table 4). However, it is not ensured that it can achieve the best performance (PPL). On the other hand, oMct employs greedy solution to its optimization problem (greedy grid search), improving downstream performance (perplexity), while maintaining linear time complexity with respect to the number of seed L.MSs (#L.MMs), i.e., 2-4 secs when using 2-5 L.Mss. Although exhaustive grid search can slightly improve performance compared to greedy grid search, its time cost can be prohibitive. For example, when using 5 LLMs, exhaustive grid search results in a explosive time cost of 902 secs.
Table 4: Weight Optimization Time Cost.
PPL Opt. #LLMs
Opt. Time (sec.)
Perplexity 3/5
3/5
Grid Search
None
0/0
11.96 / 11.54
Greedy
2/4 14 / 902
11.77 / 11.40
Exhaustive
11.76 / 11.38
Furthermore, compare the inference time cost of PackLLM with the simple ensembe ap- proach. PackLLM computes the output logits by each LLM once (as in the Ensemble baseline), which are then used for determining the impor- tance weights as shown Algorithm 1 of Ap- pendix A (no additional LLM forward passes are required). We provide the time measurements for K = 5 LLMs per input on a A100 GPU in Table 5. As evaluated, PackLLM does not incur significant time costs.
Table 5: Inference Time Cost.
Time (secs)
Single forward pass Ensemble (K forward passes)
0.06
0.30
PackLLM (K forward passes)
0.32
LackLLM's limitations are discussed in Appendix H.
8 Conclusion
We introduce PackLLM, an effective method for test-time fusion that leverages each LLM's expertise, given an input prompt. PackLLM performs model fusion by solving an optimiza- tion problem for determining each LLM's importance, prompt is minimized. We conduct experiments with over 100 total LLMs on a diverse set of tasks. Experimental results show that (i) perplexity is reliable measure for LLM fusion, (ii) PackLLM outperforms test-time fusion baselines by 1.89% accuracy points, (iii) PackLLM can leverage new LLMs to improve performance over learning-bast based fusion approaches by 3.92-11.944 1.4% accuracy points, and (iv) PackLLM benefits over selecting the best or largest model and model merging in certain cases.
that perplexity over the input
10

---

Published as a conference paper at COLM 2024
References
Valerio Basile, Cristina Bosco, Elisabetta Fersini, Debora Nozza, Viviana Patti, Fran- cisco Manuel Rangel Pardo, Paolo Rosso, and Manuela Sanguinetti. SemEval-2019 task 5: Multilingual detection of hate speech against immigrants and women in Twitter. In Proceedings of the 13th International Workshop on Semantic Coaluation, pp. 54-63, 2019.
Rishi Bommasani, Drew A Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, et al. On the opportunities and risks of foundation models. arXiv preprint ar XrXic:2108.07258 2021.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877-1901, 2020.
Daixuan Cheng, Shaohan Huang, and Furu Wei. Adapting large language models via reading comprehension. arviv preprint ar arXiv:2309.09533 2023.
Christopher Clark, Kenton Lee, Ming-We Meii Chang, Tom Kwiatkowski, Michael Collins, and Kristina Toutanova. Boolq: Exploring the surprising difficulty of natural yes/no no questions. arXiv preprint ar XrX:::1905.100 100444, 2019.
Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? try arc, the ai2 reasoning challenge. Xiv preprint ar (tte:1833.05557 2018.
Ning Ding, Yulin Chen, Ganqu Cui, Xingtai Lv, Ruobing Xie, Bowen Zhou, Zhiyuan Liu, and Maosong Sun. Mastering text, code and math simultaneously via fusing highly specialized language models. arXiv preprint arXio:2403.08281, 2024.
Shangbin Feng, Weijia Shi, Yuyang Bai, Vidhisha Balachandran, Tianxing He, and Yu- lia Tsvetkov. Knowledge card: Filling 1lms' knowledge gaps with plug-in specialized language models. In ICLR, 2024.
Yao Fu, Hao Peng, Litu Ou, Ashish Sabharwal, and Tushar Khot. Specializing smaller language models towards multi-step reasoning. In ICML, 2023.
Hila Gonen, Srini lyer, Terra Blevins, Noah A Smith, and Luke Zettlemoyer. Demystifying prompts in language models via perplexity estimation. arv preprint arXiv:22112.04037, 2022."
Suchin Gururangan, Margaret Li, Mike Lewis, Weijia Shi, Tim Althoff, Noah A Smith, and Luke Zettlemoyer. Scaling expert language models with unsupervised domain discovery. Xiv preprint ar Xxxi:22303.4417, 2023.
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. 2021.
Tin Kam Ho. Random decision forests. In Proceedings of 3rd international conference on document analysis and recognition, volume 1, pp. 278 282. IEEF, 1995.
Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. Mistral 7b. aiv preprint ar 2010:22310.06855, 2023a.
Dongfu Jiang, Xiang Ren, and Bill Yuchen Lin. LIm-blender: Ensembling large language models with pairwise comparison and generative fusion. In Proceedings of the 61th Annual Meeting of the Association for Computational Linguistics (ACL 2023), 2023b.
Di Jin, Eileen Pan, Nassim Oufattole, Wei-Hung Weng, Hanyi Fang, and Peter Szolovits. What disease does this patient have? a large-scale open domain question answering dataset from medical exams. Applied Sciences, 2021.
11

---

Published as a conference paper at COLM 2024
Qiao Jin, Bhuwan Dhingra, Zhengping Liu, William W Cohen, and Xinghua Lu. Pubmedqa: A dataset for biomedical research question answering. arXiv preprint Xit::9999001140 2019.
Aran Komatsuzaki, Joan Puigcerver, James Lee- Lee-Thorp, Carlos Riquelme Ruiz, Basil Mustafa, Joshua Ainslie, Yi Tay, Mostafa Dehghani, and Neil Houlsby. Sparse upcycling: Training mixture-of-experts from dense checkpoints. Xiv preprint arXiv:2212.05055, 2022.
Ludmila Kuncheva and Christopher J Whitaker. Measures of diversity in classifier en- sembles and their relationship with the ensemble accuracy, Machine learning, 51:181-207, 2003.
Yanis Labrak, Adrien Bazoge, Emmanuel Morin, Pierre-Antoine Gourraud, Mickael Rouvier, and Richard Dufour. Biomistral: A collection of open-source pretrained large language models for medical domains. arviv preprint XrXi::2402.10373, 2024.
Steven M Valle, Michael Branicky, and Stephen Lindemann. On the relationship between classical grid search and probabilistic roadmaps. The International Journal of Robotics Research, 2004.
Junyou Li, Qin Zhang, Yangbin Yu, Qiang Fu, and Deheng Ye. More agents is all you need. Xiv preprint ar XrX:::2402.05120, 2024a.
Tianlin Li, Qian Liu, Tianyu Pang, Chao Du, Qing Guo, Yang Liu, and Min Lin. Puri- fying large language models by ensembling a small language model. arXiv preprint ar arXit:2402.14884 2024b.
Yuanzhi Li, Sébastien Bubeck, Ronen Eldan, Allie Del Giorno, Suriya Gunasekar, and Yin Tat Lee. Textbooks are all you need ii: phi-1.5 technical report. arXiv preprint arXiv:2309.05463, 2023.
Alisa Liu, Maarten Sap, Ximing Lu, Swabha Swayamdipta, Chandra Bhagavatula, Noah A. Smith, and Yejin Choi. DExperts: Decoding-ting timeinggtime controlled text generation with experts and anti-experts. In ACL-IJCNLP, 2021.
Alisa Liu, Xiaochuang Han, Yizhong Wang, Yulia Tsvetkov, Yejin Choi, and Noah A Smith. Tuning language models by proxy. Xrv preprint (XXi=2401100866) 2024.
models
Kyle Lo, Lucy Lu Wang, Mark Neumann, Rodney Kinney, and Dan S Weld. S2orc: The semantic scholar open research corpus. arXiv preprint ar arXio:1911.02782, 2019.
Max Marion, Ahmet Üstün, Luiza Pozzobon, Alex Wang, Marzieh Fadaee, and Sara Hooker. When less is more: Investigating data pruning for pretraining ilms at scale. arXiv preprint arXiv:2309.0456/ 2023.
Costas Mavromatis, Balasubramaniam Srinivasan, Zhengyuan Shen, Jiani Zhang, Huzefa Rangwala, Christos Faloutsos, and George Karypis. Which examples annotate for in- context learning? towards effective and efficient selection. ar Xiv preprint arXiv22310.20046, 2023.
Julian McAuley and Jure Leskovec. Hidden factors and hidden topics: understanding rating dimensions with review text. In Proceedings of the 7th ACM conference on Recommender systems, pp. 165-172, 2013.
Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal. Can a suit of armor conduct electricity? new dataset for open book question answering. arv preprint arXiv:1809.02789) 2018
Adam Roberts, Colin Raffel, Katherine Lee, Michael Matena, Noam Shazeer, Peter Liu, Sharan Narang, Wei Li, and Yanqi Zhou. Exploring the limits of transfer learning with unified text-to-text transformer. Google, Tech. Rep., 2019.
Robert E Schapire. The boosting approach to machine learning: An overview. Nonlinear estimation and classification, pp. 149-171, 2003.
12

---

Published as a conference paper at COLM 2024
Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, and Christopher Potts. Recursive deep models for semantic compositionality over sentiment treebank. In Proceedings of the 2013 conference empirical methods in natural language processing, pp. 1631-1642, 2013.
Peter Sollich and Anders Krogh. Learning with ensembles: How overfitting can be useful. Advances in neural information processing systems, 8, 1995.
Sainbayar Sukhbaatar, Olga Golovneva, Vasu Sharma, Hu Xu, Xi Victoria Lin, Baptiste
Rozière, Jacob Kahn, Daniel Li, Wen-tau Yih, Jason Weston, et al. Branch-train-mix: Mixing expert Ilms into mixture-of-experts Ilm. arXiv preprint XrXi:24033.07816 2024.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, al. Llama 2:
Open foundation and fine-tuned chat models. ar Xiv preprint Aiv::2230 Siti::307.00288,,, 2023.
Fanqi Wan, Xinting Huang, Deng Cai, Xiaojun Quan, Wei Bi, and Shuming Shi. Knowledge fusion of large language models. arXiv preprint XrXiv22001.00411, 2024.
Hongyi Wang, Felipe Maia Polo, Yuekai Sun, Souvik Kundu, Eric Xing, and Mikhail Yurochkin. Fusing models with complementary expertise. ar preprint arXiv:2310.01542) 2023.
Prateek Yadav, Derek Tam, Leshem Choshen, Colin Raffel, and Mohit Bansal. Resolving interference when merging models. arXiv preprint ar arXiv:2306.01700 01708,, 2023.
Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. Hellaswag: Can machine really finish your sentence? arXiv preprint wrXi::1905.07830, 2019.
Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, al. Opt: Open pre-trained transformer language models. arviv preprint arXivv2205.01068, 2022.
Xiang Zhang, Junbo Zhao, and Yann LeCun. Character-level convolutional networks for text classification. Advances in neural information processing systems, 28, 2015.
13

---

Published as a conference paper at COLM 2024
Appendix
A Greedy Algorithm
We present the greedy algorithm for prompt perplexity minimization in Algorithm 1.
Alg Algorithm 1 Greedy Perplexity Minimization for Test-Time Fusion.
1: Input: Test Prompt P (Xo, Xt..., xx),, LLMs (M Mg]-
(lest-time)
2: for all k do
3: Compute (P).
(perplexity estimation)
4: end for
5: Sort LLMs via: .M....., Mg] = (PPL,())....,PPILL))
6: Set = 1 and A 1s1 0.
fweight initialization
7: Compute logits s(1) using Mi-
{forward pass)
8: for all k E [1, - do
9: Compute logits sk++1) using M
(forward pass}
10:
for A step 0.05 do
11:
Compute La = - Σ log (oottmax(As()) + (1- A)s(k+1) (x; search}
{grid
12:
end for Select (k) arg min, Cx.
{ (loss minimization)
14:
Set s(k) = A (k) ((k) + (1- s(k 1).
(greedy fusion) {weight update)
15:
Set 12ck1 (k). A and 1 (1 ((K)..
16:
end for
Output: Weights (A
13} for test input P.
{weights {Mi M3]
Generation: Sample with token probability p(xp (x)) softmax (aamaaxx((x)))) (s(xp) x<r)) )), where isss; +... + ARSME.
fusion)
B
PackLLM vs. cBTM & DExperts
We present some challenges comparing with cBTM and DExperts:
cBTM controls the importance weights based on the distance from the input prompt to the pretraining data of each (expert) LLM. In order to produce meaningful weights cBTM assumes each expert was trained on a different domain and that the pretraining data for all experts is known. These necessary assumptions limit the applicability of this approach to arbitrary LLMs where the pretraining data is unknown (e.g. LLama) or where there likely significant overlap in pretraining domain (e.g. Llama Mistral).
or example, if we use cBTM's LLMs, they perform 25% at accuracy on MMLU, which is close to random answer selection. Generalizing cBTM to other model families for producing full results in Table requires (i) proper data selection, and (ii) further model fine-tuning, which is limiting regarding compute resources.
DExperts combines three LLMs (see Equation 9): (i) large model M, (ii) fine- tuned (in-domain expert) Mj and an untuned iiituned/ant-eeppert antiiexxprr (out of domain) - This approach does not extend to evaluating model fusion with a variable number of seed L.Mss.
all experts are known, PackLLM outperforms cBTM (Misc in Table 1) and DExperts (Medicine in Table 1).
14

---

Published as a conference paper at COLM 2024
Following, we highlight the key difference between cBTM, DExperts, and PackLLM (ours) regarding how they compute the importance weights:
cBTM: Importance weights are data-dependent (Equation 8; using tf-idf embed- dings); the weights would be similar for LLaMA-7B and LLaMA-70B as the two models have similar pretraining data.
DExperts: Importance weights are fixed (Equation 9).
PackLLM: Importance weights are model-dependent (perplexity-based)) capturing
how well each model understands the prompt; LLaMA-70B would be assigned higher weight than LLaMA-7B.
Figure 3 verifies the benefit of model-dependen weights (PackLLM vs. cBTM & DExperts).
C Dataset Details
All datasets can be found in HuggingFace Hub.
C4 (Roberts et al., 2019) Following cBTM, we use 168B tokens of the no blocklist version (en.ooblocklist) that is out of distribution to the OPT model's pretraining corpus. S2ORC (Lo et al., 2019) a publicly available corpus of full-text academic papers from the Semantic Scholar. The corpus spans 20 fields study (e.g,, Biology, Computer Science, Art). Following cBTM, the training data consists of 168B tokens over multiple epochs. For each dataset, report language modeling perplexity on 200 sandomlysammpld held out documents. For each test document, the prompt P consists of the 32 first tokens the first 20% of the sequence's tokens, whichever is shorter; perplexity is evaluated on the rest of tokens.
publicly available distribution of a Common Crawl snapshot.
MMLU (Hendrycks et al., 2021) consists of knowledge intensive tasks, such STEM exams, that have a question and four multiple choice questions. The prompt template is "Question: (QUESTION) Choices: {Choices) Answer: " We use the default 5-shot examples and evaluate on the test data.
We employ ARC (Clark et al., 2018), BoolQ (Clark et al., 2019), HellaSwag (Zellers et al., 2019), and OpenBookQA (Mihaylov et al., 2018) datasets to evaluate performance on commonsense reasoning. We use O-shot prompts with the template "Question: (QUESTION) Choices: [CHOICES} Answer: for ARC, BoolQ and OpenBookQA, and continuation "The topic is about (TOPC). {CTX-A} [CTX-B)" for HellaSwag. We randomly sample 256 test data for evaluation. BoolQA is a binary classification, while ARC, HellaSwag, and OpenBookQA usually have four answer choices.
For PubMedQA (Jin et al., 2019) and USMLE (Jin et al., 2021), we follow (Cheng et al., 2023). The template "Question: (QUESTION) Choices: (CHOICES) Answer: " for PubMedQA and "(CONTEXT) (QUESTION) [ANSWER]" for USMLF. These datasets have four-way multiple choice answers. We randomly sample 256 test data for evaluation and use 0-shot prompts.
Regarding miscellaneous classification tasks, we experiment with AGNews (Zhang et al., 2015) (topic classification), SST2 (Socher et al., 2013) (sentiment analysis), Amazon (McAuley & Leskovec, 2013) (sentiment analysis), and SemEval (Basile et al., 2019) (Twitter hate- speech/ sentiment). We follow (Mavromatis et al., 2023) which constructs a pool of 30 annotated data, and the 5 closest examples of the pool (via embedding similarity) used as few shot examples for a test input. We evaluate on 256 randomly sampled test data.
D Seed LLMs Details
In this section, we provide the full list of the LLMs we use.
MMLU STEM. We use the following LLMs:
2 Seed LLMs:
meta-1laman 1atp:///aaaa222---a-
15

---

Published as a conference paper at COLM 2024
mistralai/Misttra--8-v...
Seed LLMs:
ofta-Ilaam/LLamm-227bbrfff
mistralai/Mistrral7B---..
microsoft/phi-2
Deci/DeciLM-78
6 Seed LLMs:
meta-1laam/LLlam--22bb-fff
mistralai/Miittra--B--00
microsoft/phi-2
Deci/DeciLM-78
Biotistraal/Biohistrrl----
EleutherAT//lemma-76
Commonsense. We the following LLMs:
2 Seed LLMs: Same MMLU-STEM.
4 Seed LLMs: Same as MMLU-STEM.
6 Seed LLMs:
meta-1lamm//Lamm-227bhhr
mistralai/Mistrrl--8-vv..
microsoft/phi-2
Deci/DeciLM-7B
Wanfg/FuseLLM--BB
open
Medicine. We use the following LLMs:
3 Seed LLMs:
AdaptLLM/meddicinn-LLLM (exper))
luodian//llmma-7--hff (antieexper))
meta-1laam/LLLamse22113-hh (base)
Seed LLMss:
Biotiistral/Bioohistrr----B
AdapttLLW/meddiinn-LLLM
chaoyi-wa/PHC-LLAN..
Miscellaneous. We use 12 expert OPT-1.3B models on trained on the C4 dataset by (Guru- rangan al., 2023). We combine together 8 cluster experts and 4 cluster experts.
FuseLLM LLMs. FuseL.LM uses the following LLMs:
mosaicml/mpt-7b
meta-llamm/Llaam-22221
16

---

Published as a conference paper at COLM 2024
FoE LLMs. FuseLLM uses the following 15 LLMs:
COAT-Attooa--7B-Coonnttt
Teerravellinggiggnnee///ooooo
asheecnn97/mmnatee-75
Barnagebbannnd/llayyyuu---
golaxy/8oggt--7bbblom
oh
Imsys/vicuna-7b-v1.3
Lmsys//vicun-70--v1555kkkk
medalpacca/meddalpacc=7
tagetherrcomputer//887-----66
fggetherrcommuter//eT--11----
Recent LLMs. As recenstyyreeeased LLMs, we use the following:
meta-IlaammLLLaaa222--hhff
mistralai/Mistral-7Bv....
microsoft/phi-2
Deci/DeciLM-7B
Recent SLMs. As the recently released SLMs, we use the following:
google/gemma-
microsof t/phi-2
Qwen/Qwen1 5-1.8B
E Full Results
Table provides full results on MMLU-STEM tasks.
Table provides full results commonsense tasks.
Table 8 provided full results on medicine tasks. Further MMLU-Health results are provided in Table 9.
Table 10 provides full results on miscellaneous classification tasks.
F
Setup of Comparison with Model Selection and Merging
We provide the experimental setup of the results in Table 3:
(A) STEM / Com. (different vocab):
5 MMM.USSTEM tasks and 4 Commonsense tasks.
3 seed LLMs of different vocab (Fuse LLMs).
Best LLM: LLaMA2-7B.
(A) STEM / Com. (same vocab):
17

---

Published as a conference paper at COLM 2024
Table 6: Full results on MMLU-STEM (College) tasks.
Biology
Chemistry
CS
Math
Physics
Average
Seed
LLMs (148)
Ensemble
59.02
42.00
45.00
32.00
28.43
41.29
Top1-PPL
46.52
40.00
37.00
33.00
31.37
37.59
PackLLMMaim
PackLLMapt
54.17
46.00
45.00
33.00
33.33
42.30
Soed LLMs
Ensemble
60.41
49.00
41.00
32.00
35.29
43.54
Topl-PPt
58.35
34,00
38.00
39.00
34.31
40.73
PackLLMMaim
61.80
11.00
47.00
37.00
34.31
44.22
PackLLM.pr
63.88
40.00
41.00
41.00
39.21
45.02
6 Seed LLMs (34.7B)
Ensemble
58.33
43,00
44.00
34.00
38.23
43.51
Top1-PP7
PackLLMam
63.19
42,00
44.00
35.00
38..3
44.48
PackLLMapt
63.88
3 Seed LLMs, same with FuseLLM (21B)
Ensemble
40.97
33.00
34.00
26.00
23.52
Top1
43.05
27.00
31.00
30.00
20.58
30.32
PackLLM.am
43.75
29.00
32,00
30.00
21.56
31.26
PackL.LMppt
32.00
15 Seed LLMs, same with FoE (93B)
Ensemble
46.06
36.77
Top1-PM
47.91
39.00
36.00
30.00
24.50
35.48
PackLLMaam
53.47
37.00
37.00
25 25.49
38.79
PackI.MMM
54.86
38.00
43.00
39.00
26.47
40.26
Seed SLMs (6.5B)
Ensemble
49,30
34.00
38.00
31.00
21.56
34.77
Top1-PPL
56.25
37.00
35.00
32:00
24.50
36.95
PackLLMain
55.55
37.00
31.00
26.47
37.00
PackL.MMppt
56.94
30.00
40.00
29.00
27.45
36.68
FuseLLM (7B)
40.97
25.00
39.00
33.00
27.45
33.08
Poll (93B)
$4.55
36.00
40.00
42.00
30.69
40.65
Table 7: Full results on Commonsense tasks.
ARC-Challenge
Hooli
HellaSwag
OpenHookQ2
Average
2Seed LLMs (148)
Ersemble
54.29
74.60
3388
Top1-PPL.
51.17
PackLLMain
81.25
54.29
P.ckl.I.Mps
42.96
78.90
75.00
30.39
61.81
Seed L. LM (23.719
Ensemblee
52.34
76.56
76.17
52.73
64.45
Top-1
PackL.LMain
PackL.LMppt
51.95
78.51
77.73
62.50
67.67
Seed L.M (34 719)
Ensemblle
46.48
76.17
74.21
51.56
62.10
Top1-PPL.
65.31
PackLLLMpp
30.78
79.29
76.56
63.67
67.58
3 Seed L.Mss, same as Pusel L. (21
Ensembll
30.85
76.95
70.31
38.67
54.19
Top1-PPI.
PackLLMMo
29.29
75.39
74.60
40.23
54.87
15 Seed LLMss, same as FoE. (93B)
Ersemble
47.90
73.39
72,60
52.06
61.48
TopI.PPI.
74,21
56.92
PackLLMoge
48.82
69.53
75.39
$5.07
62.20
Seed
SL.Ms (6.58)
Ersemble
68.88
66.79
60.93
46.87
55.86
PPI..PPL.
77.34
67.96
32
67.57
Pack
62.50
79.68
69.14
61.71
68.25
FuseLLM (7B)
36.32
77.23
75.28
44.53
58.59
18

---

Published as a conference paper at COLM 2024
Table Full results on Medicine tasks.
MMLU-Heealth tasks, 5-shot
PubMedQA
USMLE O-shot
Average
O-shot
(weighted)
3 Seed L.MSs (with anti-expert)
DExperts
44.17
66.40
33.98
48.18 (45.37)
Ensemble
40.31
68.75
31.03
46.69 (42.22)
Top1-PPL
45.70
67.96
34.76
49.47 (46.83)
PackLLMain
41.85
66.35
32.42
47.54 (43.55)
PackLLMopt
46.13
69.14
33.98
49.75
(47.21)
Seed LLMs
Ensemble
51.45
69.53
34.76
51.91 (51.59)
Top1-PPI.
51.61
54.29
33.98
46.62 (50.11)
PackLLMaain
53.22
67.57
33,98
51.59 (53.73)
Pac
54.65
67.96
33.59
52.06 (53.88)
Table Full results on MMLU-Health tasks.
Anatomy
Clinical
CollegeMed
HumanAge
MedGenetics
Nutrition
ProMed
Virology
Average
4 Seed LLMs (28B)
Ensemble
48.88
42.77
55.15
59.00
58.49
44.85
43.97
51.45
Top1-PPL
43.73
37.35
46.24
58.74
63.00
54.90
43.75
45.18
51.61
PackLLM,.m
PackLLMapt
50.37
62.64
45.66
61.43
67.00
57.18
47.79
45.18
54.65
Table 10: Results on Miscellaneous classification tasks.
AGNews Topic
Ethos Hatespeach
TweetHate Halespeech
Amazon Sentiment
TweetComplain Sentiment
SST2 Sentiment
Average
12 Seed LLMs
cBTM
58.98
50.39
46.09
70.70
78.00
67.57
61.95
Ensemble
62. 10
51.17
46.48
69.92
76.00
69.53
62.53
Top1-PPL
62.89
51.17
45.31
74.21
78.00
72.17
63.95
PackLLM.imm
61.71
51.56
46.48
69.92
80.00
69.53
63.20
Pac
60.93
55.07
46.09
70.31
78.00
69.92
63.38
19

---

Published as a conference paper at COLM 2024
5 MMLU-STEM tasks and 4 Commonsense tasks.
15 seed LLMs of same vocab (FoB LLMs).
Best LLM: Platypus-7B.
(A) C4 and S2ORC:
OPT models (30B OPT 8* 1.3 OPT experts) on C4 S2ORC (cBTM models).
Best LLM: Top1 model selection.
(B) Medicine:
3 LLMs (LLaMA2-13B, M.4-A2-113B.MeLLLMMM22755 LLaMA2-7B) on 8 Medicine tasks.
MedLLaMA2-7B, Largest LLM: LLaMA2-13B.
(B) STEM:
4 LLMs (FuseLLM-7B,, Gemma-2B, phi2-2.7B, Qwen-1.8B) on 5 MMLU-STEM
tasks.
Largest LLM: FuseLLM-7B.
(B) C4:
5 LLMs (OPT-30B, 4 domain experts OPT-1.3B) on C4 (cBTM models).
LLargest LLM: OPT-30B.
(C) Com.:
2 LLMs (LLaMA2 and FuseLLM 7B models) on 4 Commonsense tasks.
(C) C4/ S2ORC:
4 LLMs (4 domain experts OPT-1.3B) C4 S2ORC (cBTM models).
G Example Cases
We include the following two examples where fusing knowledge with PackLLM improves results:
1. Imput: "Which of the following contributes to vitamin B12 deficiency in older adults? Reduced secretion of intrinsic factor, B. Atrophic gastritis, C. Heli- cobacter pylori infection, D. All the above"
LLaMA2-13B: A. Reduced secretion of intrinsic factor (incorrect picks the longest answer)
LLaMA2-13B + PackLLM (Med D. All the above (correct identifies all correct answers)
2. Input: "Contributing your $3,000 to a 401 or other "
OPT-6.7B: "retirement plan today will get you return of 12-14%" (hallucination: incorrect return percentage)
OPT-6.7B + PackLLM (OPT-1.3B tuned model on C4): "retirement plan offers some tax advantages, but it's important to take action as early as possible." (helpful advice)
H Limitations
We summarize two limitations of our approach below.
One challenge is discussed in Section 3.4: Equation 1 assumes that the logit vectors can be added together, which requires that the seed LLMs share the same vocab- ulary. However, this not always the case, as different LLMs are trained with different vocabularies. While we follow the MinED approach for tokenizer align- ment, aligning different tokenizers is an open problem in NLP, and more effective alignment techniques may further improve PackI.LM.
20

---

Published as a conference paper at COLM 2024
2. PackLLM (similar to other fusion approaches) incurs additional inference costs compared to model merging approaches, such as weight averaging. Accelerating the inference time of fusion approaches, such as PackLLM and model ensemble, is a promising future research area.
21


Reviews:
[{"review_id": "KsoEjb6pMt", "reviewer": "Reviewer_Gnd4", "summary": "This paper introduces a new method, PackLLM, to fuse LLMs at test-time as weighted ensemble, where the importance weights for seed LLMs are determined by minimizing the perplexity of the input prompt.\n\nThere are two variants of PackLLM, one that directly uses normalized perplexity scores of individual LLMs as weights, and the other that performs an efficient greedy algorithm to search the optimal scores that minimize the prompt perplexity of the ensemble.\n\nThe paper conducts extensive experiments to validate the performance of PackLLM. On pretraining perplexity, both variants of PackLLM outperform existing test-time model ensemble baselines like cBTM and DExperts, especially when the number of seed LLMs increases. On downstream tasks, PackLLM also outperforms simple baselines like uniform ensemble. While it trails other learning-based fusion methods like FuseLLM and FoE with the same set of seed LLMs, the learning-free nature allows PackLLM to achieve better performance by swapping in stronger seed LLMs.\n\nThe paper also includes some analysis of the weight values, the impact of prompt length, and the time complexity of the two variants.", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer Gnd4, this review does not correspond to our paper “Pack of LLMs: Model Fusion at Test-Time via Perplexity Optimization”.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- Proposes a simple yet effective test-time fusion method for LLMs\n- Comprehensive empirical study by comparing PackLLM with different baselines across various settings\n- Well-written, clearly articulated motivation", "reasons_to_reject": "I don’t have major concerns. I do have some questions listed below.", "questions_to_authors": "- I’m curious to know the impact of aligning different tokenizers. Have you done any analysis comparing the performance when fusing similar-sized models with the same vs. different vocabularies?\n- In experimental results, I’d like to also see a comparison with the best seed LLM performance on each task, especially for Table 4, where most of the baseline results are not available.\n- While not an exhaustive search, the greedy search algorithm still seems a very expensive operation that needs to be done for every test-time input. Can we situate the time complexity analysis in Table 3 by comparing the additional time cost from greedy search with the actual inference time of LLMs? This would be important to help understand the real-world applicability of the proposed method.", "extra_scores": {}}, {"review_id": "ffmWIRM9Ov", "reviewer": "Reviewer_7QCH", "summary": "This paper proposes PackLLMs, which utilizes the findings that perplexity of input tokens is a good indicator of LLM's performance in domains. The authors propose PackLLM-simp that utilize the heuristics above and PackLLM-Opt, which approximates an optimization objective with a greedy search algorithm. Experiments show improvement over baselines such as cBTM, DExperts. The approach is further  modular and easy to use compared to other fusion methods that requires training LLMs.", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your review. We address your suggestions below.\n\n**Time Complexity**. We provide a time complexity analysis in Table 3, which shows that PackLLM adds negligible time overhead. We also provide the following time measurements for K=5 LLMs per input on a A100 GPU:\n\n\n| |Time|\n|-|-|\nSingle forward pass | 0.06 secs|\n|K=5 forward passes (Ensemble)| 0.30 secs|\nPackLLM (K=5) | 0.32 secs |\n\n\nAs evaluated, PackLLM does incur significant time costs.\n\n\n**Limitations**. Thank you for your suggestion. We summarize two limitations of our approach below. \n\n(i) One challenge is discussed in Section 3.4 (Tokenizer Selection & Alignment): Equation 1 assumes that the logit vectors $s_k$ can be added together, which requires that the seed LLMs share the same vocabulary. However, this is not always the case, as different LLMs are trained with different vocabularies. While we follow the MinED approach for tokenizer alignment, aligning different tokenizers is an open problem in NLP, and more effective alignment techniques may further improve PackLLM.\n\n(ii) Furthermore, PackLLM incurs additional inference costs compared to model merging approaches, such as weight averaging. \nWe will expand on this discussion and include it in a “Limitations” section in the final version.\n\n**Related Work**. Thank you for your suggestion. Upon re-reading, we agree that the related work section is repetitive, we have fixed it in the final version.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The proposed approach is simple and effective, and is broadly applicable to any combinations of LLMs with either same or different architectures.\n- The choice of baselines are recent and comprehensive. The results also show clear improvements of LLM.\n- The analysis of coefficients obtained (\\lambda), and relationships between perplexity and prompt length in Section 6 are quite helpful.\n- The writing is quite clear.", "reasons_to_reject": "- The authors should include discussions about the computational overhead of the approach. How much overhead is introduced by the grid search part, given the high dimensionality of logits (sequence length * vocab size for a single instance)? Is this overhead significant compared to a forward pass of the LM? I also encourage the authors to add a limitations sections to discuss potential limitations of the approach.", "questions_to_authors": "I think the paper is quite solid and has addressed my initial questions in their later sections.\n\nA minor comment is that the second paragraph of the related works is repetitive given the second paragraph in introduction.", "extra_scores": {}}, {"review_id": "51uPvnnpIU", "reviewer": "Reviewer_MKGz", "summary": "This paper presents a test-time LLM fusion method to leverage knowledge from the user-specified LLMs during inference. Specifically, the authors optimize the perplexity over the input prompt to perform model fusion.  The experimental results demonstrate the effectiveness of the proposed method.", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for providing a thorough review and asking insightful questions.\n\n**Comparison with cBTM and DExperts \\& Question 1**. \n\n- cBTM controls the importance weights based on the distance from the input prompt to the pretraining data of each (expert) LLM. In order to produce meaningful weights cBTM assumes each expert was trained on a different domain and that the pretraining data for all experts is known. These necessary assumptions limit the applicability of this approach to arbitrary LLMs where the pretraining data is unknown (e.g. LLama) or where there is likely significant overlap in pretraining domain (e.g. Llama + Mistral).\n- DExperts combines three LLMs (see Eq. 9): (i) a large model $M$, (ii) a fine-tuned (in-domain expert) $M_1$ and an untuned/anti-expert (out of domain) $M^-$. This approach does not extend to evaluating model fusion with a variable number of seed LLMs. \n\nNevertheless, we include comparisons with cBTM and DExperts where applicable e.g.,  in Figure 3 with the setup described in Section 4.1, and Medicine and Misc. in Table 1.\n\n**Comparison with learning-based fusion**.\nAs presented in Figure 1, the key motivation of our approach is to combine newer models without requiring retraining. We provide Table 2 to show that modular fusion methods (such as PackLLM) can leverage newly released LLMs (row “release date” in Table 2) to outperform existing learning-based fusion models (e.g. FoE), which, unlike PackLLM, uses additional (pre-)training data and requires retraining.\n\n**Question 2: Example Cases**. \nThank you for your suggestion. We will include the following two examples where fusing knowledge with PackLLM improves results in the final version:\n- Case A\n\n*Input*: \"Which of the following contributes to vitamin B12 deficiency in older adults? A. Reduced secretion of intrinsic factor, B. Atrophic gastritis, C. Helicobacter pylori infection, D. All the above\"\n\n*LLaMA2-13B*: A. Reduced secretion of intrinsic factor *(incorrect -> picks the longest answer)*\n\n*LLaMA2-13B + PackLLM (MedLLaMA2-7B)*: D. All the above *(correct -> identifies all correct answers)*\n\n- Case B\n\n*Input*: \"Contributing your $3,000 to a 401(k) or other \"\n\n*OPT-6.7B*: “retirement plan today will get you a return of 12-14%”  *(hallucination: incorrect return percentage)*\n\n*OPT-6.7B + PackLLM (OPT-1.3B tuned model on C4)*: “retirement plan offers some tax advantages, but it’s important to take action as early as possible.” *(helpful advice)*", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "This work presents a simple but effective test-time LLM fusing method and provides valuable insights. The paper is well-written and clear. The authors conducted experiments on multiple datasets, illustrating that this method generalizes well to different domains.", "reasons_to_reject": "In the main experimental results reported in Table 1, the proposed method outperforms the baseline models. However, it is important to note that the experiments involving some test-time fusion methods, such as cBTM and DExperts, are not fully reported. This lack of complete data undermines the reliability of these results. This issue is similar to the one observed in Table 2, where the comparison with learning-based fusion models becomes less convincing due to incomplete experimental data.", "questions_to_authors": "- Is it possible to complete the results of the test-time baseline experiment? \n\n- It is advised to provide some case studies to show that the proposed ppl-based fusion method is superior to other methods.", "extra_scores": {}}, {"review_id": "Ac9PBnlDoY", "reviewer": "Reviewer_ymUA", "summary": "This paper provides an alternative method for combining the responses from multiple LLMs. Weights were introduced to be used to re-weight the output logits. To determine the criterion for obtaining the optimal weights, the authors first discovered that the perplexity on the given prompts can be already served as a good approach to fuse the outputs of models. Next, the authors propose an greedy optimization-based algorithm that effectively fuses the outputs from different models. Experimental results show that the proposed algorithm can merge the outputs from models better than baseline methods such as ensemble and using only the model with lowest perplexity.", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your review. We address your suggestions below. \n\n**Selecting Largest Model**. Thank you for your suggestion. We agree that selecting a larger LLM may yield better generalization. However, smaller LMs are easy to fine-tune on in-domain data which can provide complementary knowledge (e.g., medicine data). In such cases, model fusion yields better results. To demonstrate this, we compare PackLLM with selecting the largest model from the set of models (and DExperts when applicable) on three different tasks; Medicine, MMLU-STEM, and language modeling. The results show that PackLLM yields better performance than selecting the largest model in all settings. Full results can be found in the tables below. \n\nSetup: 3 LLMs (LLaMA2-13B,  MedLLaMA2-7B, LLaMA2-7B) on 8 Medicine tasks:\n|Fusion| Accuracy|\n|-|-|\n|Select largest (13B) | 49.9|\n|DExperts |50.1|\n|PackLLM |**52.7**|\n\nSetup: 4 LLMs (FuseLLM-7B, Gemma-2B, phi2-2.7B, Qwen-1.8B) on 5 MMLU-STEM tasks.\n|Fusion|Accuracy|\n|-|-|\n|Select largest (7B)|33.1|\n|PackLLM|**36.3**|\n\nSetup: 5 LLMs (OPT-30B,  4 domain experts OPT-1.3B) on C4:\n|Fusion|Perplexity (the lower the better)|\n|-|-|\n|Select largest (30B)|12.71|\n|DExperts|11.67|\n|PackLLM|**11.13**|\n\n**Model Merging**. Thank you for your suggestion. We agree that comparing with model merging can provide new insights; the comparison is also interesting as model merging has lower inference costs compared to fusion approaches. We provide preliminary results by comparing PackLLM with average weight model merging on commonsense reasoning and language modeling tasks. The following tables show results that PackLLM outperforms model merging across these two tasks.\n\nSetup: 2 LLMs (LLaMA2 and FuseLLM 7B models) on 4 Commonsense tasks.\n|Fusion|Accuracy|\n|-|-|\n|Merging (Average Weighting)|55.2|\n|PackLLM|**57.1**|\n\nSetup: 4 LLMs (4 domain experts OPT-1.3B) on C4:\n|Fusion|Perplexity (the lower the better)|\n|-|-|\n|Merging (Average Weighting)|13.20|\n|PackLLM|**12.12**|", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The motivation is clearly described and the topic is interesting. \n- The empirical performance seems to be strong. \n- The techniques introduced in this paper are technically sound.", "reasons_to_reject": "- I think the authors may also want to compare with always picking the responses from the largest model, because selecting the model with the lowest perplexity does not mean that model can generalize well. Selecting always the largest model may yield better performance. \n- I would also like to see how this model compared with model merging. I understand it is not always possible to merge models, especially when the number of parameters and the structures are diverse across models. However, I think the authors can at least find several publicly available models fine-tuned from llama-7b on different datasets and see if the proposed algorithm can outperform model merging.", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This work introduces a method named PackLLM to better perform test-time fusion of LLMs as a weighted ensemble as compared to a simple ensemble. This assumes that minimising the perplexity of the combination of LLMs with respect to the input prompt will produce the best model.

Pros:
The work proposes a simple and effective test-time approach for this. The work is also clearly written and clear with results on multiple datasets. The work also has a relatively low test-time computational cost.

Cons:
It would be good to more clearly clarify the difference between alternative baselines including cBTM and DExperts as well as perform thorough comparisons against them where possible.

Despite this, the low test-time computational cost, the improvements in downstream tasks and good performance compared to other baselines such as a simple ensemble and selecting the largest model point to this work likely having a significant impact on the field.
Decision: Accept