Be More Diverse than the Most Diverse: Optimal Mixtures of Generative Models via Mixture-UCB Bandit Algorithms
Authors: Parham Rezaei
Farzan Farnia
Cheuk Ting Li
Venue: ICLR 2025 Poster (2025)
Keywords: Multi-Armed Bandits
Evaluation of generative models
Kernel-based evaluation scores
Mixture-UCB
Diversity in data generation
Primary area: reinforcement learning

Abstract: The availability of multiple training algorithms and architectures for generative models requires a selection mechanism to form a single model over a group of well-trained generation models. The selection task is commonly addressed by identifying the model that maximizes an evaluation score based on the diversity and quality of the generated data. However, such a best-model identification approach overlooks the possibility that a mixture of available models can outperform each individual model. In this work, we numerically show that a mixture of generative models on benchmark image datasets can indeed achieve a better evaluation score (based on FID and KID scores), compared to the individual models. This observation motivates the development of efficient algorithms for selecting the optimal mixture of the models. To address this, we formulate a quadratic optimization problem to find an optimal mixture model achieving the maximum of kernel-based evaluation scores including kernel inception distance (KID) and Rényi kernel entropy (RKE). To identify the optimal mixture of the models using the fewest possible sample queries, we view the selection task as a multi-armed bandit (MAB) problem and propose the *Mixture Upper Confidence Bound (Mixture-UCB)* algorithm that provably converges to the optimal mixture of the involved models. More broadly, the proposed Mixture-UCB can be extended to optimize every convex quadratic function of the mixture weights in a general MAB setting. We prove a regret bound for the  Mixture-UCB algorithm and perform several numerical experiments to show the success of Mixture-UCB in finding the optimal mixture of text and image generative models. The project code is available in the [Mixture-UCB Github repository](https://github.com/Rezaei-Parham/Mixture-UCB).

Published as a conference paper at ICLR 2025
BE MORE DIVERSE THAN THE MOST DIVERSE: OPTIMAL MIXTURES OF GENERATIVE MODELS VIA MIXTURE-UCB BANDIT ALGORITHMS
Parham Rezaci", Farzan Farnia², Cheuk Ting Li
Sharif University of Technology, 'The Chinese University of Hong Kong pa parham. rezaei @sharif eddu, farni aêcse. cuhk. edu.hk, ct ItliBie.. waliiiiecuhk.... edu. .k
ABSTRACT
The availability of multiple training algorithms and architectures for generative models requires selection mechanism to form single model over group of well-trained generation models. The selection task is commonly addressed by identifying the model that maximizes evaluation score based on the diversity and quality of the generated data. However, such a best-model identification ap- proach overlooks the possibility that a mixture available models can outper- form each individual model. In this work, we numerically show that mixture of generative models on benchmark image datasets can indeed achieve a better eval- uation score (based on FID and KID scores), compared to the individual models. This observation motivates the development of efficient algorithms for selecting the optimal mixture of the models. To address this, we formulate a quadratic optimization problem to find an optimal mixture model achieving the maximum of kernel-based evaluation scores including kernel inception distance (KID) and Rényi kernel entropy (RKE). To identify the optimal mixture of the models us- ing the fewest possible sample queries, view the selection task multi- armed bandit (MAB) problem and propose the Mixture Upper Confidence Bound (Mixture- UCBB algorithm that provably converges to the optimal mixture of the in- volved models. More broadly, the proposed Mixture-UCB can be extended to op- timize every convex quadratic function of the mixture weights in general MAB setting. We prove a regret bound for the Mixture-UGC UCB algorithm and perform several numerical experiments to show the success of Mixture- UCB in finding the optimal mixture of text and image generative models. The project code is available at htttps://// thub. thub. com Rezaei ParPaaraaaaa xture-UCE
INTRODUCTION
The rapid advancements in generative modeling have created a need for mechanisms to combine multiple well-trained generative models, each developed using different algorithms and architec- tures, into a single unified model. Consider m unconditional generative models each G; represents probability model Pc, according which new samples are generated. A com- mon approach for creating a unified model is to compute evaluation scores (e.g., the standard FID (Heusel al. 2017) and KID (Biñkowski et al. 2018) scores) that quantify the diversity and fidelity of the generated data, followed by selecting the model Pg.. with the best evaluated score. This best-score model selection strategy has been widely adopted for choosing generative models across various domains, including image, text, and video data generation.
9m. where
However, the model selection approach by identifying the score-maximizing model overlooks the possibility that mixture of the generative models (: Ps, + ..... + Otm Ps... where each sample is generated from randomly-selected model with as being the probability of selecting model G. can outperform every individual model. This motivates the following question: Can there be real-world settings where a non-degenerate mixture of some well-trained generative models obtain a better evaluation score compared to each individual model? Note that the standard FID and KID scores are convex functions of the generative model's distribution, and thus they can be optimized by a non-degenerate mixture of the models. In this work, we numerically show that it is possible for

---

Published as a conference paper at ICLR 2025
25% G++ 40% G2
1% G3 + 8% G3
LDM (GI)
StyleGAN-XL XL (G2)
Efficient: EfVicien--VDVEE (G3)
InsGem (G4)
Sty (G5)
26% Gs
FID
1
189.88 # 1.98
186.16 + 2.75
490.39 + 4.38
278.24 # 1.62
185.07 = 2.12
170.11 # 1.93
KID
1.48 + 0.02
1,36 + 0.03
5.34 0.05
2.29 # 0.03
1.44 0.03
1.33 * 0.07
(x10))
Figure 1: A mixture (right-most case) of FFHQ pre-trained generative models with weights (0.25.0.4.0001.0008888 achieves better FID and KID scores compared to each of the five involved models. The mixture weights are computed using our proposed Mixture-UCB-OGG algorithm.
mixture of real-world generative models improve evaluation scores over the individual models. An example is shown in Figure[] where we find non non-degenerate mixture of five generative models pre-trained on the FFHQ datase As shown, assigning mixture weights (0.25, 0.4, 0.01, 0.08, 0.26) to the models results in a significantly better FII) score 170.11 than the best individual FID 1 185.07.
To understand the improvement achieved by the mixture model, we note that the FID and KID scores evaluate both the quality and diversity of the generated data. While the averaged quality of generated samples represents an expected value that optimized by an individual model, the diversity of samples from mixture of the models can significantly improve over the diversity of the individual models' data. Figure 2 displays an illustrative example for this point, where we observe that the diversity of "red bird, cartoon style" samples generated by each of the three poxt-to-image models, is qualitatively and quantitatively lower than the diversity of their mixture. As a result, the improvement in the diversity of a mixture of generative models can result in an improved FID and KID evaluation scores, as we numerically observe in Figure I
1.1 COMPUTING OPTIMAL MIXTURES OF GENERATIVE MODELS: THE MIXTURE-UCE MULTI-ARMED BANDIT ALGORITHM
Since the evaluation score of a mixture of generative models can improve over the scores of the individual models, natural question is how efficiently compute the weights of optimal mixture of the models using the fewest possible samples from the models. Here, our goal is to minimize the number of sample generation queries from sub-optimal models, which will save the time and monetary costs of identifying the best model. To achieve this, we propose viewing the task as multi-armed bandit (MAB) problem, in which every generative model represents an arm and our goal is to find the best mixture of the models with the optimal evaluation score. The MAB approach for selecting among generative models has been recently explored by Hu et al.) (2024) applying the Upper Confidence Bound (UCB) algorithm to the FID score. This MAB-based model selection extends the online model selection methods for supervised models, including the successive halving strategy (Karnin et al. 2013 Jamieson & Talwalkar 2016 Chen & Ghosh 2024
However, in the existing MAB algorithms, the goal is to eventually converge to a single arm with the optimal score. Successive halving (Karnin et al. 2013 Jamieson & Talwalkar 2016 Chen & Ghosh 2024) and the UCB algorithm developed by Hu et al. (2024) will ultimately select only one generative model after a sufficient number of iterations. However,
discussed earlier, the eval- uation scores of the generated data could be higher when the sample generation follows a mixture of models rather than a single model. This observation leads to the following task: Developing an MAB algorithm that finds the optimal mixture of the arms rather than the single best arm.
'The pre-trained generative models are downloaded from dgm-eval GitHub repository (Stein et al. 2023
We have evaluated the diversity scores RKF (Jalali et al. 2023] and Vendi (Dan Friedman & Dieng Jamieson & Talwalkar (2016) focused on applying successive halving on hyperparameter optimization for supervised learning, whereas Chen & Ghosh 2024) focused on generative models using maximum mean discrepancy (Grettonn al. 2012) as the score.
2023)
2

---

Published as a conference paper at ICLR 2025
28% Kandinsky
40% Stable Diffussion 32% PixArt-u
Kandinsky 3
Stable Diffusion
PixArt-a
RKE
3.94 0.02
4,90 0.03
4.44 0.02
8.21 0.03
Vendi 6.49 + 0.03
7.79 + 0.03
7.08 = 0.02
11.42 + 0.04
Figure 2: Visual comparison of the diversity across individual arms and the optimal mixture for images generated using models Kandinsky 3, Stable Diffusion 3, and PixArt-a with the prompt "Red bird, cartoon style". The mixture weights are computed via the Mixture- Cicure-UCB-OU UCB-OGD method.
In this work, we develop a general MAB method, which we call the MixtureUUCB algorithm, that provably find the best mixture of the arms when the evaluation score is quadratic function of the arms' distribution, i.c. when it represents the average of scores assigned to the pairs of drawn samples. Formulating the optimization problem for a quadratic score function results in a quadratic online convex optimization problem that can be efficiently solved using the online gradient descent algorithm. More importantly, we establish concentration bound for the quadratic function of the mixture weights, which enables us to extend the UCB algorithm to the Mixture-UCB method for the online selection of the mixture weights.
For selecting mixtures of generative models using Mixture- UCB, we focus evaluation scores that reduce quadratic function of the generative model's distribution, including Kernel Inception Distance (KID) (Birikowski et al. (2018). Rényi Kernel Entropy (RKE) (Jalali et al. (022) scores, as well as the quality-measuring Precision (Sajjadi et al.) 2018 Kynkäänniemi et al. 2019) and Density (Nacem al. 2020) scores, which are linear functions of the generative distribution. Among these scores, RKE provides a reference--fee free entropy function for assessing the diversity of generated data, making it suitable for quantifying the variety of generated samples. Our mixture- based MAB frame- work can therefore be applied to find the mixture model with the maximum RKE-bbased diversity score. Additionally,
consider linear combination of RKE with the Precision, Density, KID quality scores to find mixture of models that offers the best trade-off between quality and diversity.
We perform several numerical experiments test the application of our proposed Mixture-UCA approach in comparison to the Vanilla-UCB and One-Arm Oracle approaches that tend to generate samples from only one of the available generative models. Our numerical results indicate that the Mixture-UCB UCB algorithms can generate samples with higher RKE diversity scores, and tends to gen- erate samples from mixture of several generative models when applied to image-bases generative models. Also, we test the performance of Mixture- UCB on the KID, Precision, and Density scores, which similarly result in higher score value for the mixture model found by the Mixture- UCB al- gorithm. We implement the Mixture-UCB by solving the convex optimization sub-problem every iteration and also by applying the online gradient descent algorithm at every iteration. In our exper- iments, both implementations result in satisfactory results and can improve upon learning strategies tending to select only one generative model. Here a summary of this work's 's contributions:
Studying the selection task for mixtures of multiple generative models to improve the evaluation scores generated samples (Section
Proposing online learning multi armed bandit framework to address the mixture selection task for quadratic score functions (Section
Developing the Mixture- Cactuure-UB--CC UCB-CAB and Mixture- -CArre-UUB-OO UCB-OGD algorithms to solve the formulated online learning problem and proving a regret bound for Mixture- UCBe--UBBCAAA (Sections 5.1 5.2
Presenting numerical results on the improvements in the diversity of generated data by the online selection of mixture of the generation models (Section 5. Appendix 8.4
3

---

Published as a conference paper at ICLR 2025
2 RELATED WORK
Assessment of Generative Models. The evaluation of generative models has been extensively stud- ied, with a focus on both diversity and quality of generated images. Reference-free metrics such as Rényi Kernel Entropy (RKE) (Jalali et al. 2023) and VENDI (Dan Friedman & Dieng 2023 Ospanov et al., 2024) measure diversity without relying on ground-truth, while reference based metrics such as Recall (Saijadi et al. 2018 Kynkäänniemi et al. 2019) and Coverage (Nacem al. 2020) assess diversity relative to real data. For image quality evaluation, Density and Precision metrics (Naeem et al. 22200] Kynkäänniemi al. 2019) provide measures based alignment with a reference distribution. The Wasserstein distance (Arjovsky et al. 2017) and Fréchet Inception Distance (FID) (Heusel et al./ 2017) approximate the distance between real and generated datasets, while Kernel Inception Distance (KID) (Birikossii et al.) 2018) uses squared maximum mean dis- crepancy for kernel-basee comparison of distributions. Wang et al. (2023) apply the KID score for the distributed evaluation of generative models. The novelty evaluation of generative models has also been studied in References (Han et al. 2022: Jiralerspong et al. 2023 Zhang et al. 2024b a).
Multi-Armed Bandit Algorithms. The Multi-Armed Bandit (MAB) problem is a foundational topic in reinforcement learning, where an agent aims to maximize rewards from multiple options
(arms) with initially unknown reward distributions (Lai & Robbins 1985 Thompson 1933). The
Upper Confidence Bound (UCB) algorithm (Agrawal 1995a Auer 2002 Bubeck et al. 2012) is a widely adopted method for addressing the MAB problem, where uncertainty about arm 's reward is replaced by an optimistic estimate. In generative models, optimism- based bandits have been applied efficiently identify models with optimal Fréchet Inception Distance (FID) or Inception Score while minimizing data queries (Hu et al. 2024). A special case of MAB, the continuum- armed bandit (CAB) problem (Agrawal| 1995b), optimizes a function over continuous inputs, and has been applied to machine learning tasks such as hyperparameter optimization (Feurer & Hutter, 2019 Li et al. 2018). Recent research explores CABs under more general smoothness conditions like Besov spaces (Singh, 202T), while other works have focused on regret bounds and Lipschitz conditions (Kleinberg) 2004] Kleinberg et al., 2019 Bubeck et al.. 2008).
Another related reference is informational multi-armed bandits (Weinberger & Yemini 2023 which extends UCB to maximizing the Shannon entropy of discrete distribution. In comparison, the al- gorithms in this paper can minimize the expectation of any quadratic positive-semidefintt function, which also covers the order-2 Rényi entropy for discrete distributions. Since the generative models' outputs are generally continuous, (Weinberger & Yemini 2023) is not applicable to our setting,
PRELIMINARIES
We review several kernel-based performance metrics of generative models.
3.1 RÉNYI KERNEL ENTROPY
The Rényi Kernel Entropy (Jllai et al. 2023 of the distribution P. which measures the diversity of the modes in P. is given by log(1 1/E
(X, X')),, where & is a positive definite kernel Taking the exponential of the Rényi Kernel Entropy, we have the RKE mode count 1 (X, (X X') Jalali et al. 2023), which an estimate of the number of modes. Maximizing the RKE mode count is equivalent to minimizing the following loss
X
E
Xx.x 1b?2, (x²(Xx))))
3.2
MAXIMUM MEAN DISCREPANCY AND KERNEL INCEPTION DISTANCE
The (squared) maximum mean discrepancy (MMD) Gretton et al. 2012) between distributions P. Q, which measures the distance between P and Q, can be written
E [[X(XXX)) EJK Y') (E
(2)
"The order-2 Rényi entropy for discrete distributions is special case by taking k(x,z') la=st
4

---

Published as a conference paper at ICLR 2025
where X lid P and Y, jid Q. and & is a positive definite kernel. Suppose P is the distribution of samples from generative model, and Q is a reference distribution. Minimizing the MMD can ensure that P is close to Q. The Kernel Inception Distance (KID) (Binkowski et al. 2018), a popular quality metric for image generative models, is obtained by first passing P and Q through the Inception network (Szegedy et al. 2016), and then computing their MMD, i.e., we have
(l)(v(X), (xX })] ())(v(Y), (X")) 28/
(3)
where vp is the mapping from x to its Inception representation.
4
OPTIMAL MIXTURES OF GENERATIVE MODELS
RKE
MMD and KID can all be written as loss function in the following form
L(P) == E xx x²) + [x-r[f(X)),
(4)
where : X2 + IR is a positive semidefinite kernel, and f X + IR is function. For 11 we take 10(2, x²) (2 x') (the square of kernel is still a kernel) and f f(x) = For ( we take k(x, x²) k(2, x²) and f(x) -2By-Qlkii [k(xr Y)] (the constant term Ejk(Y, Y does not matter).
For KID 33) we take x(r, x') k(b()), (xxx)) and f(x) = -2y l (ar), (Y)).. Note that and is still in the form
any convex combinations of
Suppose we are given m generative models, where model generates samples from the distribution P.. If our goal is merely to find the model that minimize the loss (4). we should select argmin, (P..) Nevertheless, for diversity metrics such as RKE, it is possible that a mixture of the models will give better diversity. Assume that the mixture weight of model i is 0z [0, 1]. where
(01,...,, am)
a probability vector. The loss of the mixture distribution > 1 agP, can then be expressed as
m
L(a) == L 22 a.P. = a T Ka +f
K X' Xx ()) m f: f== ((())) E R.".
Given K, f, the probability vector minimizing L(xx can be found via a convex quadratic program.
In practice, we do not know the precise K, f. and have to estimate them using samples. Suppose we have the samples 2g.,,... !iim; from the distribution P, for 1...., m, where n, is the number of observed samples from model i. Write x == (eL.)) We approximate the true mixture distribution Σ P; by the empirical mixture distribution Ex as Σe=1 Oz,,.- where we assign a weight ag/n; to samples Fi,a from model i, and òs,, denotes the degenerate distribution at T; We then approximate L(a) by the sample loss
m
i(a;;) :
aa T(x)x++
(5)
The
1
R2 nl
1 Σ f(2)a)) E R
kxx) : 11:T2
K(Zi,a 22,00)
E Rmmm
f(x) =
a=1 hml
The minimization of ((axx) over probability vectors a is still a convex quadratic program.
ONLINE SELECTION OF OPTIMAL MIXTURES MIXTURE MULTI-ARMBI BANDIT
Suppose we are given m generative models, but we do not have any prior information about them.
Our goal is to use these models to generate a collection of samples (x11) (E(T)) in T rounds that minimizes the loss L(P(T)) at the empirical distribution P(T) =T Σt Seco. We have
I(P(T))
1 T2
Σ (
(0))
((0))) +
1 Σ (()
1=1
5

---

Published as a conference paper at ICLR 2025
Algorithm 1 Mixture-UCB-CAI
1: Input: m generative arms, number of rounds T
2: Output: Gathered samples x!²)
3: for E {O,...,,m, 1 do
4: Pull arm + at time t + 1 to obtain sample 21+1,1 ~ Pa+1. Set m/41 (m) = 1.
5: end for
6: for i fm...., do
7:
Compute an estimate of the optimal mixture distribution via the convex quadratic program:
(t) = argmin, (I(a: - (c(*))
(6)
where the minimization is over probability vectors a, and e(t) E IR' is defined as
(t) AL [8lg (8log (2n) An/mp
8: Generate the arm index ((++1) E [m]] at random with P(b(++1) i) a, (t)
9: Pull arm b = b(++1) at time t 1 to obtain new sample "n,n - ~ Pb. Set n n +1
(t-1) (1)
(t+1) = n for j #
and ny
10: end for
11: return samples (T)
If we are told by an oracle the optimal mixture a' that minimizes the loss L(x)), then we should generate samples according this mixture distribution, giving ofT samples from model i. We call this the mixture oracle scenario. Nevertheless. in reality, we do not know K, f, and cannot compute a' exactly. Instead, we have to approximate a by minimizing the sample loss f(o:; x) do not have the samples x at the beginning in order to compute x()x)),
(5.. However, have to generate some samples first. Yet, to generate these initial samples, we need an estimate of a", or else those samples may have suboptimal empirical distribution and affect our final loss I(P(T),
will have to discard those initial samples which results in wastage.
This "chicken and egg" problem is naturally solved by an online learning approach via multi-armed bandit. At time = 1.....22 T, we choose and pull an arm ((t) E [m]] (i.e., generate a sample from model ((9)), and obtain a sample (+()) from the distribution PAe.. The choice 6(t) can depend on all previous samples (1) ('.)).... ...,, x (f1,) Unlike conventional multi-armed bandit where the goal is to maximize the total reward over T rounds, here we minimize the loss L(P(T)) which involve cross terms x(>(())xx' (x))) between samples at different rounds. Note that if <(x, x') = O, then this reduces to the conventional multi-armed bandit setting by taking f(xx to be the negative reward of the sample x. the following subsections, we will propose two new algorithms that are generalizations of the upper confidence bound (UCB) algorithm for
......,,,
bandit (Agrawal 1995a Auer 2002).
5.1 MIXTURE UPPER CONFIDENCE BOUND CONTINUUM- ARMED BANDIT
Let n be the number of times arm i has been pulled up to time t. Let (() == (1s, (es.a)) ne(n(")
(8)
be the observed samples up to time We focus on bounded loss functions, and assume that ks
x² + [No, si] and f X' > [fo, fil are bounded. Let A. K1 Ko. A fi fo. Define the sensitivity L as AL == 2 + A,.
We now present the mixture upper confidence bound - continuum-armed bandit (Mixture- UCBB CAB) algorithm. It has a parameter 3 > 1. treats the online selection problem as multi-armed bandit with infinitely many arms similar to the continuum-armed bandit settings in (Agrawal 1995b. Lu et al. 2019). Each arm is a probability vector a. By pulling the arm a, we generate a sample from randomly chosen model, where model is chosen with probability og Refer to Algorithm T.

---

Published as a conference paper at ICLR 2025
Similar to UCB, Mixture- Centuree-UC--OO CCB--CCBB--AA CCB finds a lower confidence bound ((axx(1) (e( a of the
true loss [(xx each round. To justify the expressions (6) (7), we prove that ((axx()) (<(())
in (6 lower-bounds L(a) with probability at least 1 The proof is given in Appendix 8.1
Theorem Fix probability vector af Suppose we have samples 2i,1..... Li,n: from the distri- bution P. for ....,........................... where n, is the number of observed samples from model For
P(I(a;;) - I((x) 2 aa o; P(((a) - [(axx) 2 c()) So,
where E(6) == (
log(1/5) 2m4 My sEfm]]
A
gives an expected loss E[L(P(T) that converges to the optimal loss mina [(a)) by bounding their gap. This means that Mixture-UCB-CAN is a zero- regret strategy by treating B[L(P(T))] min L(xx) as the average regret per round 7 The proof is given Appendix 8.2
We now prove that Mixture-UCB-
Theorem 2 Suppose m 2 2, 3 4. Consider bounded quadratic loss function 4 with being positive semidefinite. Let P(T) be the empirical distribution of the first T 2 2 samples x given by Mixture- Mixture-UCB-- CAB. Then the gap between the expected loss and the optimal loss is bounded by
Bm log T T
[[L(P()) min L(a) S 4Ar1
When k((x, x²) = 0, MCctture-UCB-ie CAB reduces to the conventional UCB, and Theorem coincides with the O (1v(n)) (m log 7)) distribution-free bound on the regret per round of conventional UCB Bubbeck al. 2012). Since there is 5?( (2Vmm/T) minimax lower bound the regret per round even for conventional multi-armed bandit without the quadratic kernel term (Bubeck et al. 2012 2 is tight up a logarithmic factor.
Theorem 3.4), Theorem
The main difference between
UCBBurre-UU-CCAA and conventional UCB is that we choose a mixture
of arms in (6) given by the probability vector a, instead of single arm. A more straightforward application of UCB would be to simply find the single arm that minimizes the lower bound in (6), i.e. we restrict a e; for some i [m]. where e; is the i-th basis vector, and minimize (6) over i instead. We call this VanillaUUBB. Vanilla-UCB fails to take into account the possibility that a
mixture may give a smaller loss than every single arm. In the long run, Vanilla-UCB converges to pulling the best single arm instead of the optimal mixture. Vanilla-UCB will be used as a baseline be compared with Mixture- UCB-CAB and another new algorithm presented in the next section.
Mixture- UCBe--UBB-AA CAB can be extended to the select only small number of models. This can be useful if there is a subscription cost for each model. Refer to Appendix 8.3 for discussions.
CAB algorithm which eventually
5.2 MIXTURE UPPER CONFIDENCE BOUND ONLINE GRADIENT DESCENT
We present an alternative to Mixture-UCRE UCB-CAB, called the mixture upper confidence bound - online gradient descent (Mixture-UCE UCB-UCU OGD) algorithm, inspired by the online gradient descent algorithm
Shalev-Shwartz et al.) 2012). It also has a parameter / > 1. Refer to Algorithm 2
Cesture-UBBCCAL and Mixture- Mixture-UCB--a UCB-UCCUUO OGD can both be regarded as generalizations of the origi- nal UCB algorithm, in the sense that they reduce to UCB when <(x,, x') = O. If we remove the
(x)xn term in 8). then Mixture- Mixture-UCB-OUE UCB-OGD becomes the same as UCB.
Theorem holds for a lixed a. worst-case bound that simultaneously holds for every a is Lemma ! TTo justify calling R ELL (P(()) - min., L(a:) the average regret per round, note that when <(x, x')
and f(x) -r(z) where r(z) is the reward of the sample x, iee. the loss L(P) Ex- [x=
mina L(a)) T maxig[m]] Ex-P, [r(X) E (")( indeed reduces to the conventional notion regret. So can regarded the quadratic generalization of regret.
linear, T(E [(E
We may also consider the scenario where each pull gives a batch of l samples instead of only one sample.
In this case, we will have T (t-
b,n
(t-1)
6,m (t-1)_4 ~ Po and n! n, (t-1) +l.
7

---

Published as a conference paper at ICLR 2025
Algorithm 2 Mixture-UCB-OR
1: Input: m generative arms, number of rounds T
2: Output: Gathered samples x!²)
3: for E {O....,,M
1 do
4: Pull arm / + at time / + 1 to obtain sample 21+1,1 ~ Pe+1
5: end for
6:
for trtt E
do Compute the gradient
7:
h ( = Va [ - a0 {(((())) + f(x(()) c(0) (8)
2
where (1) = (m! kE|m) E RR, and ((*) is defined as Mixture-UCH UCB-CAB
8: Pull arm = b((+1) = argmin,h, time + to obtain new sample V +1 Pb.
9: end for
10: return samples (T)
Both Mucture-UCBBBoe CAB and -uktureUCBB-GDD attempt to make the "proportion vector" (n/ (note that n!") /t is the proportion of samples from model i) approach the optimal mixture a" that minimizes L(ax). but they do so different manners. Mixture-UCB-CA.
first computes the esti- mate (") after time 1, then approaches (*()) by pulling an arm randomly chosen from the distribution (!). MixurreUCB-OOGI estimates the gradient (() of the loss function at the current proportion
vector (1) /t, and pulls an arm that results in the steepest descent of the loss.
An advantage of Mixture- Mixture-UCB-oo UCB-OGD is that the computation of gradient (8) is significantly faster than the quadratic program (6)
Mexture-UBBCCAL The running time complexity Mixture- UCB-OGD is O Tm Nevertheless, regret bound for Tixture-UCB-OR similar to Theorem 2 seems to be difficult to derive, and left for future research.
6 NUMERICAL RESULTS
We experiment various scenarios to showcase the performance of our proposed algorithms. The experiments involve the following algorithms:
Mixture Oracle. In the mixture oracle algorithm (Section5). an oracle tells us the optimal mixture a" in advance, and we pull arms randomly according to this distribution. The optimal mixture is calculated by solving the quadratic optimization in Section + on large number of samples for each arm. The number of chosen samples varies based on the experiments. This is an unrealistic setting that only serves as theoretical upper bound of the performance of any online algorithm. A realistic algorithm that performs close to the mixture oracle would be almost optimal.
One-Arm Oracle. An oracle tells us the optimal single arm in advance, and we keep pulling this This is unrealistic setting. If our algorithms outperform the one-arm oracle, this will show that the advantage of pulling a mixture of arms (instead a single arm) can be realistically achieved via online algorithms.
Vanilla-UCR, direct application of UCB mentioned near the end of Section 5.1 a baseline for the purpose of comparison.
This serves as
Successive Halving. The Success Halving algorithm (Karnin et al. 2013 Jamieson & Talwalkar
2016 Chen & Ghosh 2024) which serves as a second baseline for comparison.
Mixture-UCB- UCB-CAB. The mixture upper confidence bound comtinuum-armel bandit algorithm proposed in Section 5.]
"To update {xx!")) after new sample x' obtained, we only need to compute x(2, x') for each existing sample and add their contributions to the corresponding entries in (t)(+)), requiring computational time that linear with the number of existing samples.
8

---

Published as a conference paper at ICLR 2025
Mixture-UCB-OGD. The mixture upper confidence bound online gradient descent algorithm proposed in Section 5.2
Experiments Setup. We used DINOV2-ViT-L/I (Oquab et al. 2023 for image feature extrac-
tion, as recommended in (Stein et al. 2023), and RoBERTa (Liu et al. 2019) as the text encoder.
Detailed explanation of the setup for each experiment presented Section 8.4
6.1
OPTIMAL MIXTURE FOR DIVERSITY AND QUALITY VIA KID
x10
Mixtum Orarle
2.4
Mietro Oraele
Mistare Orade
1.8
Ote-Amm Oradl
One Arm Orazle
One Arm Oraele
Vanilla DC3
Vanila-
Vandis-UCCB
ICBB-CAB
CCB.CAS CAB
2
Mistur-UCB--0C
Misture UCB.OCD
Mislae-ICBBOGD
1.6
Suncssivee Hilvigg
Succmsise Babiis
Saceonire Balrins
1.8
1.4
1.5
2000
4000
6000
2000
4000
6000
0
200
400
Step
Step
Step
(a) FFHQ generators
(b) LSUN- Bedroom generators
(c) FFHQ truncated generators
Figure 3: Performance comparison of online algorithms for the KID metric across FFHQ, LSUN-
Bedroom, and FFHQ Truncated generators.
We conducted three experiments evaluate our method using the Kernel Inception Distance (KID) metric. In the first experiment, we used five distinet generative models: LDM (Rombach et al.)
2022
StyleGAN-XL (Sauer al.] (222). Efficient- VDVAE (Hazami et al. 2022 InsGen Yangg
et al. 2019
2021), and StyleNAT (Walton et al.
dataset (Karras et al. In the second experiment, used generated images from four model StyleGAN (Karras
2022), all trained on the FFHQ
2021). iDDPM (Nichol & Dhariwal 2021). and Un- 2022). all trained the L-UN--edroom dataset (Yu iet al. 2015). This experiment followed a similar setup to the first. In the final experiment, we em- ployed the truncation method (Marchesi] 2017 Karras et al. 2019) to generate diversity-contron images centered on eight randomly selected points, using StyleGAN2
et al. leashing Transformers BBond-Taylo, al.
2019), Projected GAN (Sauer et al.
(Karras et al.) 2020), also trained on the FFHQ dataset. Figure 3 demonstrates that the mixture of generators achieves better KID scores compared to individual models. Additionally, the two Mixture-UCB algorithms consistently outperform the baselines.
6.2 OPTIMAL MIXTURE FOR DIVERSITY VIA RKE
We used the RKE Mode Count (Jalaii et al. 2023) mixing the models on the diversity and the advantage of our algorithms Mixture-UCC CIee-UUB-CAA CBB and Mixture-UCB-OGD. The score in the plots is the RKE Mode Count, written RKE for brevity.
evaluation metric to show the effect of
Synthetic Unconditional Generative Models We conduct two experiments diversity- -imitted generative models. First, we used eight center points with truncation value of 0.3 to generate images using StyleGAN2-ADA, trained on the FFHQ dataset. In the second experiment, the same model, trained on the AFHQ Cat dataset (Choi et al.] 2020), with a truncation value of 0.4. As shown in Figure 7 the optimal mixture and our algorithms consistently achieve higher RKE scores. The increase in diversity is visually depicted in Figures 6 and
applied
Text to Image Generative Models We used Stable Diffusion XL (Podell et al., 2024) with spe- cific prompts create three car image generators with distinct styles: realistic, surreal, and cartoon. In the second experiment, recognizing the importance of diversity generative models for design tasks, used five models 1-Schnlll (Labs) 2024), Kandinsky 3.0 (Vladimir et al. 2024) PixArt-a (Chen et al. 2023a), and Stable Diffusion to generate images of the object "Sofa"
9FFH and L.SUN--edrwo datasets were downloaded from the dgm-eval repository Stein et al. 2023 (licensed under MIT license);

---

Published as a conference paper at ICLR 2025
In similar manner, generated red bird images using Kandinsky 3.0, Stable Diffusion (Esser jet al. 2024), and PixArt-a, as shown in Figure 2. Finally, in the third experiment, we used Sta- ble Diffusion XL to simulate models generating images of different dog breeds: Bulldog, German Shepherd, and Poodle, respectively. This illustrates the challenge of generating diverse object types with text-to-image models. Figure TO demonstrates the impact of using mixture of models in the first and third experiments. The improvement in diversity is evident visually and quantitatively, as shown by the RKE scores. Our online algorithms consistently generate more diverse samples than others, as illustrated in Figure 4
3.0
Mistrm Drsile
2.5
Matuur DCB-CAB
Venill-UCBB
7.0
RKB
2.0
Succsiix Edviis
Mistren DCB-OGD
RKE
6.5
Mettrre Oracle
Oun-Am Grack
RKE
Mattiiee Orazie
1.5
6.0
Mixiaro-CCBBCC CAB
M.C CAB
Mixtrre UCB OODD
Sucsasiv
Sureasine Balvia
1.0
5.5
0
100
200
200
400
600
250
500
750
Step
Step
Step
(a) Dog breed generators
(b) Sofa image generators
(c) specifcc generators
Figure 4: Performance comparison of online algorithms using RKE score of T21 generative models.
6.3 OPTIMAL MIXTURE FOR DIVERSITY AND QUALITY VIA RKE AND PRECISION/DENSITY
Using RKE, we focus solely on the diversity of the arms without accounting for their quality. To address this, we apply our methodology to both RKE and Precision (Kynkäänniemi et al. 2019), as well as RKE and Density (Nacem et al. 2020). We conduct experiments which quality a key consideration. We use four arms: three StyleGAN2-ADA models trained the FFHQ dataset, each generating images with a truncation 0.3 around randomly selected center points. The fourth model is StyleGAN2-ADA trained CIFAR- 10 (Krizhevsky et al. 2009). The FFHQ dataset is used as the reference dataset. Figures 5 and 13 demonstrate the ability of our algorithms in finding optimal mixtures with higher diversity/quality score.
x10
3.75
1.6
Mictume Orade
Prrcdsson
3.50
A Donsity
Vanilla UCB
Ose-Aru
1.4
Mixtine UCB
3.25
Mixtime UCDD
Mixture Oradle
Successive Halving
One-Arm Oraele
1.2
3.00
Vanills-UCB
Mixture
2.75
Mixtuze UCU OCD
1.0
Halving
1000
2000
3000
0
1000
2000
3000
Step
Step
(a) RKE w/Precision
(b) RKE w/Density
Figure 5: Performance comparison of online algorithms using the combination of RKE with Preci- sion and RKE with Density metrics.
CONCLUSION AND LIMITATIONS
We studied the online selection from several generative models, where the online learner aims to generate samples with the best overall quality and diversity. While standard multi-armed bandit (MAB) algorithms converge to one arm and select one generative model, we highlighted that mixture of generative models could achieve higher score compared to the individual models. We proposed the Mixture-UCB MAB algorithm to find the optimal mixture. Our experiments suggest the usefulness of the algorithm in improving the performance scores over individual arms. However, we note that the diversity gain offered by the mixture approach should be analyzed together with the quality of generated data, which can be adjusted by setting the coefficient of the quality Precision or Density score when applying Mixture- UCB. The analysis of the regret of Mixture- -eituuee-UB--.. conditions under which mixture cannot improve the scores will remain for future studies.
and
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENTS
The work of Cheuk Ting Li is partially supported by two grants from the Research Grants Council of the Hong Kong Special Administrative Region, China [Project No.s: CUHK 24205621 (ECS), CUHK 14209823 (GRF). The work of Farzan Farnia is partially supported by a grant from the Re- search Grants Council of the Hong Kong Special Administrative Region, China, Project 14209920, and is partially supported by CUHK Direct Research Grants with CUHK Project No. 4055 164 and 4937054. Finally, the authors would like thank the anonymous reviewers for their constructive feedback and suggestions.
REFERENCES
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Iige Akkaya, Florencia Leoni Ale- man, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, al. Gpt-4 technical report. arXiv preprint arXivv2303.088: 2023.
Rajeev Agrawal. Sample mean based index policies by Ollogn) regret for the multi-armed bandit problem. Advances in applied probability, 27(4):1054-107 1995a.
Rajeev Agrawal. The continuum-arme bandit problem. SIAM journal on control and optimization, 33(6):1926 1951, 1995b.
Martin Arjovsky, Soumith Chintala, and Léon Botto.. Wasserstein generative adversarial networks. In International conference on machine learning, pp. 214-223. PMLR, 2017.
Peter Auer. Using confidence bounds for exploitation-exploraiio trade-offs. Journal of Machine Learning Research, 33944233 2002.
Mikolaj Binkowski, Danica Sutherland, Michael Arbel, and Arthur Gretton. Demystifying MMD
GANs. In International Conference on Learning Representations, 2018.
Sam Bond-Taylor, Peter Hessey, Hiroshi Sasaki, Toby P Breckon, and Chris G Willcocks. Un- leashing transformers: Parallel token prediction with discrete absorbing diffusion for fast high- resolution image generation from vector-quantie codes. In European Conference on Computer Vision, pp. 170-188. Springer, 2022.
Sébastien Bubeck, Gilles Stoltz, Csaba Szepesvári, and Rémi Munos. Online optimization in x- armed bandits. volume 21, 2008.
Sébastien Bubeck, Nicolo Cesa-Bianchi, et al. Regret analysis of stochastic and nonstochastic multi- armed bandit problems. Foundations and Trends@ in Machine Learning, 5(1):1-122, 2012.
Junsong Chen, Jincheng Yu, Chongjian Ge, Lewei Yao, Enze Xie, Yue Wu, Zhongdao Wang. James Kwok, Ping Luo, Huchuan Lu, al. Pixart-o: Fast training of diffusion transformer for photore- alistic text-to-image synthesis. arXiv preprint trXiv:2310.00422: 2023a.
Luming Chen and Sujit K Ghosh. Fast model selection and hyperparameter tuning for generative models. Entropy, 26(2):150, 2024.
Yutian Chen, Hao Kang, Vivian Zhai, Liangze Li, Rita Singh, and Bhiksha Raj. Token prediction as implicit classification to identify Ilm- llm-generated text. arXiv preprint arXiv:2311.08723, 2023b.
Yunjey Choi, Youngjung Uh, Jaejun Yoo, and Jung-Wov Woo Ha. Stargan v2: Diverse image synthesis for multiple domains. In Proceedings the the IEEF/CVE conference on computer vision and pattern recognition, pp.
2020.
Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. Palm: Scaling language modeling with pathways. Journal of Machine Learning Research, 24(240): 1-113, 2023.
Dan Dan Friedman and Adji Bousso Dieng. The vendi score: A diversity evaluation metric for machine learning. Transactions on machine learning research, 2023.

---

Published as a conference paper at ICLR 2025
Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow transformers for high-resolution image synthesis. In Forty-first international conference machine learning, 2024.
Matthias Feurer and Frank Hutter. Hyperparameter optimization. Springer International Publishing, 2019.
Arthur Gretton, Karsten M Borgwardt, Malte Rasch, Bernhard Schölkopf, and Alexander Smola. A kernel two-sample test. The Journal of Machine Learning Research, 13(1):723-773, 2012.
Jiyeon Han, Hwanil Choi, Yunjey Choi, Junho Kim, Jung-Woo Ha, and Jaesik Choi. Rarity score: A new metric to evaluate the uncommonness of synthesized images. arXiv preprint arXiv:2206.08549 2022.
Louay Hazami, Rayhane Mama, and Ragavan Thurairatnam. Efficientvdvae: Less is more. arXiv preprint 2tt.::22203.1355:.. 2022.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems, 30, 2017.
Xiaoyan Hu, Ho-fung Leung, and Farzan Farnia. An optimism-based approach to online evaluation of generative models. arXiv preprint arXiv:2406.07451, 2024.
Mohammad Jalali, Cheuk Ting Li, and Farzan Farnia. An information-theoreti evaluation of gen- erative models in learning multi-modal distributions. Advances Neural Information Processing Systems, 36, 2023.
Kevin Jamieson and Ameet Talwalkar. Non-stochastic best arm identification and hyperparameter optimization. In Artificial intelligence and statistics, pp. 240-248. PMLR, 2016.
Marco Jiralerspong, Joey Bose, Ian Gemp. Chongli Qin, Yoram Bachrach, and Gauthier Gidel. Feature likelihood divergence: evaluating the generalization of generative models using samples. Advances in Neural Information Processing Systems, 36:33095-33119 2023.
Zohar Karnin, Tomer Koren, and Oren Somekh. Almost optimal exploration in multi- bandits. In International conference on machine learning, pp. 2238-1246. PMLR, 2013.
Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. In Proceedings the the IEEE/CVF conference on computer vision and pattern recognition, pp. 4401-4410, 2019.
Tero Karras, Miika Aittala, Janne Hellsten, Samuli Laine, Jaakko Lehtinen, and Timo Aila. Training generative adversarial networks with limited data. Advances in neural information processing systems, 13:14200-12114, 2020.
Robert Kleinberg. Nearly tight bounds for the continuum-armed bandit problem. Advances in Neural Information Processing Systems, 17, 2004.
Robert Kleinberg, Aleksandrs Slivkins, and Eli Upfal. Bandits and experts in metric spaces. Journal of the ACM (JACM), 66(4):1-77, 2019.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images.
2009.
Tuomas Kynkäänniemi, Tero Karras, Samuli Laine, Jaakko Lehtinen, and Timo Aila. Improved precision and recall metric for assessing generative models. Advances in neural information processing systems, 32, 2019.
Black Forest Labs. Flux.
http:///ggghuub..mmllakkk com/black forest- Labs/Elaab hat.://olaaoofoeesaaaaaaaa/////e
2024.
Tze Leung Lai and Herbert Robbins. Asymptotically efficient adaptive allocation rules, Advances in applied mathematics, 6(1):4-22, 1985.
12

---

Published as a conference paper at ICLR 2025
Lisha Li, Kevin Jamieson, Giulia DeSalvo, Afshin Rostamizadeh, and Ameet Talwalkar. Hyperband: A novel bandit-based approach to hyperparameter optimization. Journal of Machine Learning Research, 18(185):1-52, 2018.
Yinhan Liu, Myle Out, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach. arXiv preprint
2019,
Shiyin Lu, Guanghui Wang, Yao Hu, and Lijun Zhang. Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In international conference on machine learning, pp. 4154-4163. PMLR, 2019.
Marco Marchesi. Megapixel size image creation using generative adversarial networks. arXiv preprint arXiv: 1:Xiv:/70.000800 00082, 2017.
Muhammad Ferjad Naeem, Seong Joon Oh, Youngjung Uh, Yunjey Choi, and Jaejun Yoo. Reliable fidelity and diversity metrics for generative models. In International conference on machine learning, pp. 7176-7185. PMLR, 2020.
Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In International conference on machine learning, pp. 1162-8171. PMLR, 2021.
Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Mare Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07190 2023.
Azim Ospanov, Jingwei Zhang, Mohammad Jalali, Xuenan Cao, Andrej Bogdanov, and Farzan Farnia. Towards a scalable reference-free evaluation of generative models. Advances in Neural Information Processing Systems, 37: 20892-120927,, 2024.
Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhom, Jonas Müller, Joe Penna, and Robin Rombach. SDXL: Improving latent diffusion models for high-resolution image synthesis. In The Twelfth International Conference on Learning Representations, 2024.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodci, lla Sutskever, et al. Language models are unsupervised multitask learners. OpenAl blog, 1(8):9, 2019.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF confer- ence on computer vision and pattern recognition, pp. 10684-10695, 2022.
Mehdi SM Sajjadi, Olivier Bachem, Mario Lucic, Olivier Bousquet, and Sylvain Gelly, Assessing generative models via precision and recall. Advances in neural information processing systems, 31, 2018.
Axel Sauer, Kashyap Chitta, Jens Müller, and Andreas Geiger. Projected gans converge faster. Advances in Neural Information Processing Systems, 14:17480-17492, 2021.
Axel Sauer, Katja Schwarz, and Andreas Geiger. Stylegan-xl: Scaling stylegan to large diverse datasets. In ACM SIGGRAPH 2022 conference proceedings, pp. 1-10, 2022.
Shai Shalev-Shwartz et al. Online learning and online convex optimization. Foundations and Trends@ in Machine Learning, 4(2):107-194, 2012.
Shashank Singh. Continuum-armed bandits: A function space perspective. In International Confer- ence on Artificial Intelligence and Statistics, pp. 2620-2628. PMLR, 2021.
George Stein, Jesse Cresswell, Rasa Hosseinzadeh, Yi Sui, Brendan Ross, Valentin Villecroze, Zhaoyan Liu, Anthony I. Caterini, Eric Taylor, and Gabriel Loaiza-Ganem. Exposing flaws of generative model evaluation metrics and their unfair treatment of diffusion models. Advances in Neural Information Processing Systems, (6:3732-37844 2023.
Christian Szegedy, Vincent Vanhoucke, Sergey loffe, Jon Shlens, and Zbigniew Wojna. Rethink- ing the inception architecture for computer vision. In Proceedings of the IEEE conference on computer vision and patlern recognition, pp. 2818-2826, 2016.
13

---

Published as a conference paper at ICLR 2025
William R Thompson. On the likelihood that unknown probability exceeds another in view of the evidence of two samples. Biometrika, 25(3-4):285-255) 2944, 1933.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, al. Llama: Open and efficient foundation language models. arXiv preprint arXiv: 13971, 2023.
Arkhipkin Vladimir, Viacheslav Vasilev, Andrei Filatov, Igor Paviov, Julia Agafonova, Nikolai Gerasimenko, Anna Averchenkova, Evelina Mironova, Bukashkin Anton, Konstantin Kulikov, et al. Kandinsky 3: Text-to-image synthesis for multifunctional generative framework. In Pro- ceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pp.
2024.
Steven Walton, Ali Hassani, Xingqian Xu, Zhangyang Wang, and Humphrey Shi. Stylenat: Giving each head a new perspective. arXiv preprint 10X0v22211.05770, 2022.
Zixiao Wang, Farzan Farnia, Zhenghao Lin, Yunheng Shen, and Bei Yu. On the distributed evalua- tion of generative models. arXiv preprint arXiv:2310.11714 2023.
Nir Weinberger and Michal Yemini. Multi-armed bandits with self-information rewards. IEEE Transactions Information Theory,
2023.
Ceyuan Yang, Yujum Shen, Yinghao Xu, and Bolei Zhou. instance discrimination. Advances in Neural Information Processing Systems, 34:9378-9390, 2021.
instance generation from
Fisher Yu, Ari Seff, Yinda Zhang, Shuran Song, Thomas Funkhouser, and Jianxiong Xiao. Lsun: Construction of large-scale image dataset using deep learning with humans in the loop. arXiv preprint arXiv: 03365, 2015.
Jingwei Zhang, Mohammad Jalali, Cheuk Ting Li, and Farzan Farnia. Identification of novel modes in generative models via fourier-based differential clustering. arXiv preprint arXi::2405.02700, 2024a.
Jingwei Zhang, Cheuk Ting Li, and Farzan Farnia. An interpretable evaluation of entropy-based novelty of generative models, arXiv preprint arXiv:2402. 172877 2024b.
14

---

Published as a conference paper at ICLR 2025
8 APPENDIX
8.1 PROOF OF THEOREM
Consider any Gi.a X. Let & be the samples which are identical except that entry Ti,a is changed to Fi.a- We have
[(a;x) [(axx)
(f(#s.u) - f(Iea))+ +2
Σ
a,aj (k(@ra: 2z,b) K(
14
(1.a)
of [[(Biaaūiia) - N(Fiaa,
+
n;
04
/ Σ
a,j Ax + a
An
n4
nmnj
mi
(1,))
(A, + 2Ax)
A
By McDiarmid's inequality,
2e
P (i(aax) -[][(xx)]] 20) < exp
Liml
1)
Laml
2c
exp
A Ei=1 a?/
Note that
L(a)- - B[((xxx)]]
E xx (nc (X, X')] - E
ngmyy
a=1 d=1
m
n
(x.x
a=1
m
n
1=1
Hence, for 8 0,
log(1
m
P
[(oxx) L(or) 2 A
A
8.
2
11;
The result follows from
1og(1/6)
2
A
A
2
1=1
1=1
log
(/5)
a
A
A.
11:
m
1og(1/8)
Ag
A
Oli-
2n;
11
The other direction of the inequality is similar.
15

---

Published as a conference paper at ICLR 2025
8.2 PROOF OF THEOREM 2
Before we prove Theorem we first prove worst case concentration bound.
lid Lemma Let 2 2, and ri, B..,.... ~ P E [m]. Let A = 2A, A. For n1, n.,...,, Dar E (T]. let x(m): == (N ke(] Fix any 8 > O. With probability at least we have
L(a)=
L(ax
172
m2T2
Dx
log
a4.
2m1
28
Mig
for every M1...., Pom [T] and probability vector The same holds for ((axx(())) L(a) instead of L(a) ((())(
Proof: Fix any BI,..., E [T], and write x = ((n). have
L(a)= - L(axx)
f 2
K.J fx() (x)
Σ
K, (
0j0g
For i = j. applying Theorem on a being the i-th basis vector,
log(1/0)
Ax
P Ki, fx()) K (x,
8.
2m,
For i * j. we will use similar arguments as Theorem Let x be the samples which are identical to x except that one entry 2i,a of the i-th arm is changed to Di.a- We have
(xx)
(x) + K. (x)
fx(x) + (x)
(x)
2
1
1
a
(f(Tia)) - / (Ii,a))
+
(( K(Zias Fz,6))
2m;
A
An
<
2n:
AL
2nz
Note that E (x(x) (x)
fitf; K,,. By McDiarmid's inequality, K. (x)] =-
2
+
f((x) + fx(
P
+ +Kij
kx)(x) 2c
2
2
< exp
2
Aj
AL
11:
2nl
nl
2mg
8E²
exp
A (n, + n3
-1
Hence,
f + fg
f((x + (x)
log(1/8 (1/))
+ Kij
Kx)(x) 2 Ary
(n. -1 + n, 6. (10)
2
2
16

---

Published as a conference paper at ICLR 2025
Note that the event in (9) does not depend ny for i' * i, and the event in 10 does not depend
on ny for i' & {i. By union bound, all the events in 9 and (10) do not hold for all é I j and Mi...... nm [T] with probability at least
1 - m78 m(m 1) 7
m T
2 1
2
2
these events do not hold, then L(a)- ((axx)
ff
fx(x) + (x((x)
Σ ajj
+Ki,
(x)
2
log(1/8)
Ax
log(1/6)
aj
V
+
Σ
a,a
(m; + n
2nL
8
(kd))(m]PPidt
Σ
af
V..
(1/(1/6)
AK
Dr.
log(1/6)
Σ
a,a n4 1/2 + 12)
2n4
74
8
(i:))(m)3
1og(1/6)
112 111
-1 1/2)
0
A
L
n
+ +
f1i
8
(1.))((m]²
1og(1/5)
+
V
ag1
-1/2
2
(1/5)
az.
2n,
T4
The other direction of the inequality is similar.
We finally prove Theorem 2
Proof: Assume m 2 3 2 4 and T 2. If < 40m, then since T log T is decreasing for T 2 3 (the following inequalities are obviously true for T = 2),
/ logT
4m l0g(40m)
llog 80
2 0.66,
40m
10
and Theorem 2 is trivially true since [[[(P()))] mina L(x)) < AL. Hence we can assume
T' 2 40m + 1.
Let a be the minimizer of L(a.. Let (+t) be the sample obtained at the t-th pull. Let a (t-1) 1f1 = i] for E [m], so "z(() is generated from the distribution P: with probability af (t-1)) holds for every 2 1. Write æ((4)) == (z(1) )xels). For s < t, let (s) be a random variable with the same conditional distribution given [([--1) as g(s), but is conditionally independent of all other random variables given [[(s-1), The joint distribution of (((c-1)) _,(1) _.(() is
P2uc-11) Pato
Recall that σ(+) is generated from the distribution P3 with probability a; (=-1)) = (a. (a-1) s5mm) computed using g((s-1)), We have
(s-1) for [m]. where
B (x)((()) [(())]
E
E ((((())
x(0) =[([-1)]]
1) a.at Ex~r.X [x(X,X))
E
E
(a
(a(
Ka (tt)
17

---

Published as a conference paper at ICLR 2025
where (a) is because a(() only depends on (((+-1) 11) (i.e., is conditionally independent of all other ran- dom variables in the expression given E((+-11)), and z00) only depends on g(((-1), Write δry (A]) for the total variation distance between the distributions of the random variables A and B. Write I (A; B(C) for the conditional mutual information between A and B given C' in nats. We have
B[ }(
(6)
E|n( xz =00]] + An (11)) || {(0)
s E (a Ka (1-1) + A,drV (x x(مه x(0) || 11)1--1)), ((0)) (0(0))
2 ((al+1)) [all-)) + +
2 (+()); x(1) [[((-
where (b) is because takes values over [Moo xi] with Ax K2 No. and (c) is by Pinsker's incquality. We also have, for every t,
E [[((100, x(00)] S E (aalt))) (oel-t))TKa((t)) Kallt-1] + An.
Hence,
T T
T
T
E
k( {(1)
- E
(-)
(1-1)
T
s= tml
sml
t=1
24
An
I(F()); I(() (z(lo-1)) +
T
2
T
t=a+1
2A
T
Dn
(100= +
T
24
T
Ax
(0)=)>000)00>>>>> +
T2
2
T
mml
(4)
2A
T
(x(((e11); x(0)= +
4x
T2
T
24
A
log m +
72
2
T
2A
log m
Ax VidT +
<
72
2
T
0
4Ax k
log m
Ax
2T
T
where (d) by the chain rule of mutual information, and (e) is because z(() only depends on ([(t-1]) through the choice of arm ((0) E [m), and hence [(x((c-1)); g(0)) is upper bounded by the entropy of
18

---

Published as a conference paper at ICLR 2025
800), which is most log m. Also note that E[f(x())] = ff' E[a((t11)
11. Hence,
B[L(P(T))
T
I
(x((+)
22(1)
=
T
T
==1
E
(1-1)
+
T
(a (al (t1
+
4Ax
log m
AR
T 1=1
7
21
T
s=1 t=1
T
1 T
(t-1) )
4
log m
4x
E
(t-1) +
(t-1)
K
a
+
+
T 1=1
T 1=1
T
3
2T
T
1=1
4A,
log m
A
E
L
a
(t-1)
+
T 1=1
3
27
T
IVS
IE
((a)tt1))
44 x
log
Ax
(11)
2T
where (f) is because K is positive semidefinite, and hence L is convex. Therefore, to bound the optimality gap, we study the expected loss E [L(a(t))] of the estimate of the optimal mixture dis- tribution (1).
Let 8 > Let É be the event
m²T2
4x
-(a)- - L(ax (1))s) I
Aul
log
a2
2n,
20
1=1
for every : [2'] and probability vector a, as Lemmal Fix a time / E T}. Let E, be the event
1 By Lemma
P(E) 2 8.
m
3 Blogt
Ax
x(0)))))))) L(a') 5
A
at
2m;
for every Ti,,.... Pm 2 1 such that E, n2 = t. Since
m2+2
Ax
A r. 2n
log
+
of
2m
22/2
V
log
+
2n
B Blog t
Ax
S
A
2n,
1=1
by 3 2 4, applying Lemma
IPE))) > 1 1 m22-212.
(12)
If the event E, holds, by taking n4 = =
i(((::(())) L(aa')== (T(())) Ta²
Ta
(13)
If the event E holds,
L(a) - x())x(())
m212
A
log
k
(14)
7=1
2m²
(C)
25
n2
19

---

Published as a conference paper at ICLR 2025
for every Combining 13 and
14 (with a
L(a
x(())
i(a";x
(cc))))
1
m²T²
A
+
AL
V
2n
(1)
log
20
+
2 (1) L(a").
By 6 Ma(l)x(0) - (cl1))
( (1) v 1(ax:x10) - (ell1) T and hence if the events E. Eg hold,
m2T2
(e 10 + Σ
duy
log
a2
(t)
i=1
2m
25
f1
(2)
2 l(a!')) - L(a").
(15)
We have
(e (1) (1)
+
m
log
m²12
A
0g
(t)
2n
(1)
28
n2
(8)
m
log
A
22
DLy
+
A
log
2m
71
2n
20
71
AL
m²1²
24
log t +
log
28
T2:
n
AL
log T
1
log
m2
A
a
(8)
(1)
2
20
(
n
n
where
1
m272
7)
log T log
V
28 + 1.
Substituting into 15 the events E, E, hold.
A
> L(a").
Hence, general (regardless whether E, E, hold), denoting the indicator function of E. E, as Eng, E (0, 1}.
(
L(a(!t) - -(aa))
BnE.
(t)
A1
n2
Let
(1) :
Σ
(n
(1) 1),
1=1
20

---

Published as a conference paper at ICLR 2025
where (n)) = Di= i-1/2, Recall that pull arm at time & with probability oj The expected increase of (r(() is
(0)
m
E ((+11) w x
Σ
(w(n())
vnn))) - -1) ag
m
(1)
Σ
a;
(1)
n
L(a (1) _((a))
ENE,
Note that
m
(
(n
(T) 1)
min(r-11 1/2, 1]dr
1=1
(0)
EE
(1)-
minfr 1/2 1]dr
m
T/m-1
min(T-1/ 1/2 l
m
TT/m-1 T- d/
b m
14
T - 1 -11 2
m
m
where (a) is because a +
Jo min(r- 1/2, 13d7 is concave, and (b) is because 2 40m + 1, so
T/m 12 1. Therefore,
2 E (( - v(
m 21
- -
E (((+)) - v0(0)
((a - I(a*),
E
ALT
ENEE
ALP((EnE))))
ALd
Σ
E (1) -L(a') - A2 + m²t m22-21110 2/2))
ALT
E [L(a - -L(a")
T
m²
T-1
t 2
AL L
71
2n
m-1
[((al))) - [(ar)]
m.
E
LT
17
27(m 1)
E
[(al - [(a*))
AL t=
17
17
21

---

Published as a conference paper at ICLR 2025
where (c) is by Hence,
T
1
E [(al9) - [(a')]
L
E [(((())) - -((aa) +
A
T
$ m
21
-1
+ 75 + 2m
m
4 m
T
3
mT3
2
log T + V/2
log
+1]
1
+ 3m
m
2
2
(0)
T
mI
S m
2 m
log T +
log
+1
2
( log 1/2 log 2
mT3
< 22Vmm
+1)
where (d) is by substituting 8 m/T, (e) is because
log T
v2
12ggg8
2.9 and
V 2 log m 2 2 2.5 (recall that T 2 40m + 2 81). Substituting into
11).
1
[[L(P(())] - -((a))
A
- -
4A,
log m
Ax
+
A,T T
3
2T
TA
9 log T + V/2
mT
2
log m
1
log
+
+
+
2
2
3
2T
2T
8
mf
2
log m
2v
m log
V 2m 2m
222mm m +
VT
2
2
3
2
3
2 log T
2
m log T + 122 T4
VT
2
Vlog SI
2
mm lolo
log m
m log T
+
3
m log 40m + 1)
2
2 81
2 log 8T
m mlog
log m
23 + +V22+
+
T
Vlog 81
3
V
2m log( 40m 1)
2/81
22log 81
m mlogT
2
log 2
228 + 222
T
Vlog 81
V
4 4log 81
2v81
v2 S1
m mlog
(V28 + +3.934)
T
m log T T
3.934
28 +
VB
2
Bm log T
< 3.382
T
This completes the proof of Theorem 2 (with improved constant 3.382 instead of 4).
22

---

Published as a conference paper at ICLR 2025
8.3 SPARSE MIXTURE UPPER CONFIDENCE WUUND-CONIINUUM-ARREDD BANDIT ALGORITHM
The optimal mixture may involve large number of models. It is sometimes of interest to identify a small subset of models that can still give diverse samples. We now consider the scenario where there is a cost associated with each arm. If we have to pay a cost per pull, then this cost can be absorbed into the function f. and the problem reduces to the aforementioned quadratic multi-armed bandit. However, if the cost a "subscription fee" that we have to pay for each arm at each round, even if we do not pull that arm at that time, until we decide to "unsubscribe" the arm and not pull it anymore, then we have to modify the algorithm to minimize the following average cost
L(P T max(t E [T] : = =1)-
(16)
where P(T)) is the empirical distribution of the first T samples, b(t) is the arm pulled at time t, max(L E [T] : b(t) = i} is the last time we pull arm i, and À is the subscription fee per round. Intuitively, we have to subscribe to arm i until the last use time max(t E [7] 6(t) = i}. As
T' + oC, we hope that the average cost (16) approaches the optimal cost mina (L(a)
where ||a||o is the number of nonzero entries of a. Minimizing (16) allows us to simultaneously select the best subset of arms and the optimal mixture in the long run, akin to variable selection methods in statistical learning.
We now generalize the Mixture-UUB-CCA algorithm to the sparse mixture upper confidence bound continoun- bandit Sparse-Mixttre-UCCH UCB-CAB) algorithm, which has parameters À 2 0 and > 1. This algorithm is inspired by the backward elimination method for variable selection (?), which starts with all variables and gradually removing variables irrelevant to our prediction. Here, we start with a set subscribed arms S that contains all arms [m]. and gradually dropping the worst arm i' as long as the upper confidence bound of the optimal cost without arm i' is lower than the lower confidence bound of the optimal cost with arm i', which implies that dropping arm i' will have high likelihood of reducing the cost. The algorithm given in Algorithm 3 and the experiments are presented in Appendix [8.4]
Asymptotically, If fixed sparsity is desired instead, we can start with X = 0, and gradually increase À at each round until |8] = and then stop unsubscribing arms
Cenee-Mixtuu---UB-CAA attempts to minimize the cost mina (I(a) Al)ax]lo)-
8.4 DETAILS OF THE NUMERICAL EXPERIMENTS
saper-parameter Choice. The kernel bandwidths for the RKE and KID metrics were chosen based on the guidelines provided in their respective papers to ensure clear distinction between mod- els. The values for A and A, in online algorithms 7 were set according the magnitudes of the metrics and their behavior on a validation subset. The number of sampling rounds was adjusted according to the number of arms and metric convergence, both of which depend the bandwidth. To ensure the statistical significance of results, all experiments were repeated 10 times with different random seeds, and the reported plots represent the average results.
8.4.1 OPTIMAL MIXTURE FOR QUALITY AND DIVERSITY VIA KID
Suppose P is the distribution of generated images of model, and Q is the target distribution. Recall that for KID 33. we take the quadratic term to be k(x, x') = k(v(2), vxx²)) (with an expectation 2 (k)(x)) (YY)) (with an
Ex,x (a (X), (X)))) and the linear term to be f(x expectation - 22EExx w kk (x), ())). In order to run our online algorithms, we use An and A₂ based on validation portion to make sure the UCB terms have the right magnitude for forcing exploration.
FFHQ Generated Images. In this experiment, we used images generated by five different models: LDM (Rombach et al. 2022), StyLeGAN-XL (Sauer et al., 2022). Efficient-VDVAE (Hazami al. 2022). InsGen (Yang et 202 and StyleNAT (Walton et al. 2022). We used 10,000 images from each model to determine the optimal mixture. A kernel bandwidth of 40 was used for calculating
23

---

Published as a conference paper at ICLR 2025
Algorithm 3 Sparse-Mixure--CC--AAD
1: Input: m generative arms, number of rounds T
2: Output: Gathered samples x!T)
3: Initialize the set of subscribed arms < [m]]
4: for E {O,...,,! m -1 } do
5: Pull arm + 1 time 1 to obtain sample 2++1,1 Po+i. Set mit1
6: end for
7: for E (m...., 1} do
8: repeat
9: Compute
1 argmin suspp(a)
(i(a;x(0) + XS8k - (c(t)) a).
(17)
where e(t) R is defined in 7 Let the minimum value above be C.
10:
Compute the following "worst arm" if S) 2:
(i(a:x()) + A([S) - 1) + (c
i' f argmin min iES (S
Let the minimum value above be C².
11:
C' S C then
12:
Unsubscribe arm i' (i.e., S s((t))
13:
end if
14:
until no more arms are unsubscribed
15:
Generate the arm index 6(++1) E [m] at random with P(b(t+1) = i) = =a:
Pull arm b(++1) at time t+ 1 to obtain new sample T ",, Pb. Setng (t-1) =n +1
(++1) n for j # b. and n;
17:
end for
18: return samples (2)
the RKE, and the online algorithms were run for 8,000 sampling rounds. Figure [presents a visual representation of the impact of our algorithm, along with the corresponding FID and KID scores.
In Tables and we observe that the Precision of the optimal mixture similar to that of the maximum Precision score among individual models. On the other hand, the Recall-based diversity improved in the mixture case. However, the quasity- Density score slightly decreased for the selected mixture model, as Density linear score for quality that could be optimized by an individual model. On the other hand, the Coverage score of the mixture model was higher than each individual model.
Note that Precision and Density are scores on the average quality of samples. Intuitively, the quality score mixture of models is the average of the quality score of the individual models, and hence the quality score of mixture cannot be better than the best individual model. On the other hand, Recall and Coverage measure the diversity of the samples, which can increase by considering mixture of the models. To evaluate the net diversity-quali the selected mixture and best individual model, and the selected mixture model had better FID score compared to the individual model with the best FID.
effect, we measured the FID score of
LSUN-Bedrowd We used images generated by four different models: StyleGAN (Karras et al.) 2019) Projected GAN (Sauer et al. 2021). iDDPM (Nichol & Dhariwal 2021), and Unleashing Transformers -Byno-Taylor et al. 2022). We utilized 10,000 images from each model to compute the optimal mixture, resulting in weights of (0.51, O, 0.49, 0). A kernel bandwidth of 40 was applied, and the algorithm including the results for the optimal mixture based on KID, are presented in Table
for 8,000 sampling steps. The quality and diversity scores for each model,
Truncated FFHQ. We used StyleGAN2-ADA (Krrra et al. 2020) trained on FFHQ dataset to generate images. We randomly chose 8 initial points and used the Truncation Method (Marchesi
24

---

Published as a conference paper at ICLR 2025
Model
Precision
Recall
Density
Coverage
FID 4
KID (
LDM
0.856 + 0.008
0.482 + 0.008
0,959 + 0.027
0.776 0.006
189.876 + 1.976
1.84 + 0.019
StyleGAN-XL
0.798 = 0.007
0.515 0.007
0.726 0.018
0.691 0.009
186.163 = 2.752
1.355 0.028
Eficient- ElleientVDDVAE VDVAE
0.854 = 0.011
0.143 = 0.007
0,952 1 0.033
0.545 :: 0.008
490,385 1 4.377
5.339 4 0.046
InsGen
0.76 0.006
0,281
0.007
0.716
0.692
278.235
1.617
2.292 0.025
StyleNAT
0.834 + 0.008
0.478 + 0.007
0.867 + 0.023
0.775 + 0.007
185.067 + 2.123
1.442 + 0.029
Optimal Mixture (KID)
0.818 = 0.007
0.57 4 0.008
0.816 + 0.025
0.765 0.007
168.127 # 1.596
1.273 = 0.018
Mixture- UCBeeUUBB-CC CAB (KID)
0.828 1 0.007
0.571 0,008
0.838 0.016
0.787 0.007
170.578 1 2.075
1.342 1 0.007
Mixture- Mexture-UCB-OOL UCB-OGD (KID)
0.827 + 0.006
0.573 + 0.005
0.825 + 0.015
0.787 0.006
170.113 + 1,930
1.334 + 0.007
Table I: Quality and diversity scores for the FFHQ experiment, including precision, recall, density, coverage, and FID metrics (+ standard deviation).
Model
Precision 1
Recall
Density
Coverage
FID J
KID (x10")
StyleGAN
0.838 + 0.008
0.446 + 0.007
0,941 0.019
0.821 0.004
175.575 + 2.055
1.559 + 0.027
Projected GAN
0.749 = 0.015
0.329 = 0.008
0.592 : 0.027
0.517 = 0.008
324.066 + 3.753
3.834 = 0.053
iDDPM
0.838 1: 0.006
0.641 0.006
0.660 0.018
0.825 0.006
154.680
3.036
1.513 0.035
Unleashing Transformers
0.786 + 0.008
0.449 + 0.006
0.649
0.013
0.581
0.013
339.982
+ 6.118
4.131 + 0.051
Optimal Mixture (KID)
0.838 = 0.006
0,589 = 0.005
0,900 + 0.016
0.833
0.004
149.779 + 2.238
1.369 = 0.024
Mixture-UCBB UCB-CAB (KID)
0.835 +: 0.007
0.602 +: 0.007
0.894 +: 0.014
0.834 0.005
151.28
1.801
1.438 = 0.024
Mixture UCB OGD (KID)
0.838 1. 0.008
0.599 0.007
0,902 1 0.023
0.834 0.006
151.10 1.906
1434 1 0.026
Table 2: Quality and diversity scores the LSUN-Bedroom experiment, including precision, recall, density, coverage, and FID metrics (# standard deviation).
2017 Karras et al. 2019) to generate images with limited diversity around each of the chosen points. We used truncation value of 0.3 and generated 5000 images from each model to find the optimal mixture. The weights for the mixture was (0.07, 0.28, 0.10, 0.04, 0.21, 0.11, 0.12, 0.07). A kernel bandwidth of 40 was used, and 4,000 sampling steps were conducted.
8.4.2 OPTIMAL MIXTURE FOR DIVERSITY VIA RKE
Truncated FFHQ. We employed StyleGAN2-ADA (Karras et al., 2020 trained the FFHQ dataset, to generate images. Eight initial points were randomly selected, and the Truncation Method (Marchesi 2017 Karras et al. 2019) was applied with truncation value of 0.3 to generate images with limited diversity around these points. For the quadratic optimization, 5,000 images were gener- ated from each model, using kernel bandwidth of 40 to identify the optimal mixture. the online experiment, a new set of generated images was used, and sampling was conducted over 2,000 steps.
Truncated AFHQ Cat. Similar to the previous experiment, we used StyAeGAN2-ADA to generate AFHQ Cat images. Four initial points were selected, and truncation value of 0.6 was applied to simulate diversity-controllee models. For the quadratic optimization, 5,000 images were generated from each model, with sampling conducted over 1,200 steps to determine the optimal mixture.
Style-Specific Generators. We used Stable Diffusion XL to generate images of cars in distinct styles: realistic, surreal, and cartoon. For this experiment, we utilized 2,000 images from each model to determine the optimal mixture, which yielded weights of (0.67, 0.27, 0.06). This mixture increased the RKE value from 7.8 (the optimal value of the realistic images) to 9.2. We set the kernel bandwidth to 30 and executed the online algorithms over 1,000 sampling steps.
Sofa Images. We generated images of the object "Sofa" using prompts with environmental de- scriptions across the models FLUX.1-Schnel (Labs) 2024), Kandinsky 3.0 (Vladimir et al.) 2024). PixArt-a (Chen et al. 2023a), and Stable Diffusion XI. (Podell et al.) 2024). Solving the RKF op- timization with 1,000 images revealed that sampling 38% from FLUX and 62% from Kandinsky improved the RKE score from the one-arm optimum of 7.21 to 7.57. We set the kernel bandwidth to 30 and conducted the online experiment over 700 steps. We observed that Tixture-UCB-OG achieved noticeably faster convergence to the optimal mixture RKE in this scenario.
The prompts followed the structure: "A adjective sofa is verb in location," with the terms for Adjective, Action, and Location generated by GPT-4o (Achiam et al. 2023), specifically for the object "Sofa."
25

---

Published as a conference paper at ICLR 2025
Generative Model
Generative Model 2
Generative Model
Generative Model 4
Optimal Mixture
RKB 6.43 0,08
8.64 0.13
4,24 0.08
8.39 a 0.18
Generative Model
Generative Model
Generative Model
Generative Model
19.23 0.18
weights (0.05-0.20-0.09- 007.0.2--011300166..000
RKE 9.30 0.13
7.19 0.14
0.17
4.78 0.08
Figure 6: Visual demonstration of the increase in diversity when mixing arms compared to individual arms for truncated FFHQ generative models. The RKE values for each model and the mixture serve as indicators of diversity.
x10
x10
1.8
Midtung Dtodl
Mixture Oracle
Quaele
One-Arm
VaGilla-URR
1.4
Van
1.6
Mixture Mistws-UCB-CAB
UCB-CAB
Mastre UCD-CAD
Mixtwe-UC UCB-OGD
RKE
1.1
Maxture Mistue-UBBOOCD UCB-OCD
1.4
Successive
Halving
1.2
0.5
500
1000
1500
200
600
800
1000
Step
Step
(a) FFHQ truncated generators
(b) AFHQ truncated generators
Figure 7: Performance comparison of online algorithms based on the RKE metric for Simulator
Unconditional Generative Models.
Dog Breeds Images. Stable Diffusion XL was used to generate images of three dog breeds: Poo- dle, Bulldog, and German Shepherd. As shown in Figure 10 using mixture of models resulted in an increase mode count from 1.5 to 3, supporting our claim of enhanced diversity. We set the kernel bandwidth to 50 and generated 1, 1,000 images for each breed to determine the optimal mixture, which was (0.33, 0.31, 0.36). Additionally, the online algorithms were executed for 500 sampling steps.
Red Bird Images. To observe the performance of mixing the models while generating images on single prompt, we generate images with the prompt "Red bird, cartoon style" using Kandinsky 3.0 [Vladimir et al. 2024), Stable Diffusion 3-medium (Esser et al., 2024), and PixArt-a CCen et al. (003a)). We use 8000 images and kernel bandwidth of 30 to find the optimal mixture in an offline manner. The increase in diversity is shown in Figure 2 We observe a noticeable boost in the RKE and Vendi scores, showing the diversity has improved. The performance of our online algorithms and the comparison of samples generated by each online algorithm are shown in Figure 11
Text Generative Models. In this experiment, we used the OpenLLMText dataset (Chen et al, 2023b), which consists of 60,000 human texts rephrased paragraph by paragraph using the models
26

---

Published as a conference paper at ICLR 2025
Generative Model
Generative Model 2
Generative Model
Generative Model
Optimal Mixture
RKE 4.68 # 0.08
8.49 + 0.14
6.98 # 0.20
6.49 # 0.13
Generative Model
Generative Model
Generative Model
Generative Model
16.56 0.23
weights 19-0, 12-0.07-
0.10-0.2--000--0.26)
RKE 7.10 0.15
6.59 0.18
6.90 0.17
9.97 0.20
Figure 8: Visual demonstration of the increase in diversity when mixing compared individual arms for truncated AFHQ Cat generative models. The RKE values for each model and the mixture represent the diversity.
Optimal Mixture
FLUX.1-85 I-Schnell
Kandinsky-3
PixArt-o
Stable Diffusion XL
weights (0.38-0.62 62-0-0)
RKE 1 0.112
13.48 0.14
7.86
10.54 0.13
14.51 0.13
Figure Visual comparison of diversity each arm and the mixture for sofa image generators
GPT2-XL (Radford et al. 2019), LLaMA-7B (Touvron et al.) 2023), and PaLM (Chowdhery al. 2023). To extract features from each text, we employed the RoBERTa text encoder (Liu et al. 2019). By solving the optimization problem
10,000 texts from each model, we found that mixing the models with probabilities (0.02, 0.34, 0.64, 0) achieved the optimal mixture, improving the RKE from an optimal single model score of 69.3 75.2. A bandwidth of 0.6 was used for the kernel, and we ran the online algorithms for 7,000 steps to demonstrate their performance. As shown in Figure T2al the results demonstrate the advantage of our online algorithms, suggesting that our method applies not only to image generators but also to text generators.
Sparse Mixture Four different initial points and StyleGAN2-ADA were used to generate images with truncation of 0.6 around the points, simulating diversity-conttolleed controlled arms. A value of 0.06 and bandwidth 30 were selected based on the magnitudes of RKEs from the validation dataset to determine when "uusubscrriee arms in the Cparse-Mixure- UABe--UBBCAA algorithm. We conducted three scenarios, gradually reducing the number of arms to between one and three, and presented comparison of the resulting plots and their convergence values in Figure 12b
8.4.3 OPTIMAL. MIXTURE FOR DIVERSITY AND QUALITY VIA RKE AND PRECISION/DENNITY
In this experiment, we utilized four arms: three of them are StyleGAN2- models trained on FFHQ, each using truncation value of 0.3 around randomly selected point. The fourth arm is
27

---

Published as a conference paper at ICLR 2025
Optimal Mixture
Generative Model
Generative Model
Generative Model 3
weights (0.33-0.31-0.3))
RKE 1.51 0.01
1.48 0.02
1.64 0.01
3.04 0.02
weights (0.66-0.26-0.07)
RKE
7,86 # 0.16
4.83 0.008
+
9.19
Figure 10: Visual comparison of the diversity across individual arms and the optimal mixture for Dog Breed Generators and Style-Specific Generators.
Successive Halving
Mixture-UCB-CAB
Mixtiie Oracle
One-Aru Oracle
Vanilla- UCB
Mistum-UCBB-AFF
Mixtume OCB-OGD
RKE
Spocessivr Halving
Vanilla-UCB
Mixture-UCB-OGD
2000
4000
6000
Step
Comparison of our proposed algorithms Mixture-UCB-CAB and Mixture-UBB-OGI with the baseline one arm online algorithms. We plot the diversity-measurre RKE score of the generated data and display (randomly-selected) samples produced in the process of each algorithm.
Figure 11:
StyleGAN2-ADA trained on CIFAR-10. We generated 5,000 images and used kernel bandwidth of 30 to calculate the optimal mixture. When optimizing purely for diversity using the RKE metric, the high diversity of the fourth arm leads to a probability of 0.91 being assigned to it, shown in Figure [3] However, despite the increased diversity, the quality the generated images, based on the reference distribution, unsatisfactory.
28

---

Published as a conference paper at ICLR 2025
x10
x10
2.3
One Arm
1.6
Two Arma
Arms
RKE
2.1
RKB
1.4
Mixturee Orocle
2.0
One Arm Oracle
Vanilla-UCB
1.2
Mixsure- UCB-CAB CAB Misuun-UCB-OGA
1.9
Surcessive Halving
1.0
1.8
1000
2000
3000
4000
5000
500
1000
1500
2000
Step
Step
(a) Text generatons
(h) Sparse mixture
Figure 12: Comparison of online algorithms for the RKE metric on text generative models and the
Sparse Mixture algorithm for FFHQ truncated generators
Generative Model
Generative Model
Generative Model
Generative Model
RKE
RKE w/Precision Optimal Mixture
RKE w/Density
Optimal Mixture
Optimal Mixture
weights- - (0.04-0.03-0000--0.9 weights (0.26-0.28-0.46-))
weights (0.16-0.28-0.56.00
Figure 13: Visual demonstration of the effect of combining Precision/Density with RKE. The CI- FAR10 generator is excluded when these quality metrics are applied.
To address this, we incorporate a quality metric, specifically Precision/Densityy into the optimization. We subtract the weighted Precision/Density from the RKE value, ensuring balance between quality and diversity. The weight for the quality metric (A = 0.2) was selected based on validation data to ensure comparable scaling between the two metrics. As result, Figure 13 shows that the fourth arm, which had low-quality outputs, is assigned weight of zero.
We use the RKE score as K and the weighted Precision/Density as according to equation The
online algorithms were run for 4,000 steps, with the results depicted in Figure
29


Reviews:
[{"review_id": "TKsVQuqH5l", "reviewer": "Reviewer_x3ew", "summary": "Given a group of generative models, this paper studies the problem of improving the diversity (and quality) of generated outputs by combining them into an (optimal) mixture. The authors present the Mixture-UCB framework, encompassing two specific algorithms, Mixture-UCB-CAB and Mixture-UCB-OGD, designed by iteratively optimizing a quadratic objective (wrt the mixture weights) on kernel-based eval metrics and efficiently formulating the mixture of models under an online bandit setting. Specific metrics include Kernel Inception Distance (KID) and Rényi Kernel Entropy (RKE). Theoretical regret bounds have provided adequate support for Mixture-UCB-CAB, and experimental evaluations demonstrate the advantages of both algorithms across various datasets and model types.", "questions": "Could the authors kindly consider the weaknesses highlighted above and share any thoughts, feedback, or responses they might have? Also, I wonder about the typical scenarios where the mixture models fail to improve diversity or quality.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The paper is generally well-written and well-structured, with clear definitions and visualizations.\n2. The focus on mixtures of generative models to achieve superior diversity (and quality) appears innovative and addresses a limitation in traditional model selection approaches, which aim to find only a single best-performing model. Being able to customize the support size of the mixture is a good plus. \n3. The theoretical analysis for Mixture-UCB-CAB is well-formulated and provides near-optimal guarantees (i.e., up to logarithmic factors of $m$ and $T$).\n4. The diverse experiments demonstrate the algorithms' practical applications and performance gains, especially in exciting domains such as text-to-image generation.", "weaknesses": "1. While Mixture-UCB-OGD seems computationally more efficient than Mixture-UCB-CAB, the absence of a theoretical guarantee akin to Theorem 2 for CAB leaves an open question about its convergence and reliability.\n2. Linear mixtures show their ability to enhance diversity. Still, the data distributions might produce mixtures that lack coherence, as some of the visual examples hint at (e.g., in Figure 3, the mixture model generated both realistic and unrealistic car images). In other words, optimizing the single diversity metric may not capture users' needs (e.g., a model that can generate images of cars with different sizes, poses, coloring styles, and backgrounds may be more natural to be said \"diverse\").\n3. The mixture model approach may not be suitable for memory-efficient use cases, such as deployment on end devices like smartphones or smart home modules. Storing, updating, and switching among multiple generative models (e.g., this might require loading new parameters into memory) could significantly increase memory requirements and other costs, making the approach impractical for some critical applications. A possible mitigation strategy could be distilling a mixture of large models into a single, smaller-scale model, thereby retaining the benefits of the mixture while reducing resource needs.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "GWnq10QNcJ", "reviewer": "Reviewer_pb52", "summary": "This paper aims to solve the online selection task over a group of well-trained generation models. It explores the selection of a mixture of multiple generative models and formulate a quadratic optimization problem to optimize the kernel-based evaluation scores including kernel inception distance (KID) and Renyi kernel entropy (RKE). Specifically, it proposes an online learning approach called Mixture Upper Confidence Bound (Mixture-UCB). Theoretically, regret analysis is provided for one method (Mixture-UCB-CAB). Experimental results illustrate the effectiveness of the proposed method for text-based and image-based generative models.", "questions": "In practice, FID metric is widely-used in the evaluation of generative models. Can this paper cover this metric and why?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1. Overall, this paper is well-written and easy to follow.\n2. The proposed method (Mixture-UCB) is somehow novel, although it is inspired by classical UCB in the multi-armed bandit setting.\n3. Theoretical results about the regret bound are provided for the proposed Mixture-UCB-CAB. The proof seems right although I have not checked the proof line-by-line.\n4. Empirical results illustrate the effectiveness of the proposed method in finding the optimal mixture of text-based and image-based generative models.", "weaknesses": "1. I am afraid that the online selection of well-trained generative models might have few applications because it is already costly for the (large) generative model inference, then why do we need online selection rather than batch selection? Discussions about practical applications can be added.\n2. Experimental results show that Mixture-UCB-OGD might be better than Mixture-UCB-CAB. However, theoretical guarantees about Mixture-UCB-OGD are missing. I know it might be more challenging and more detailed discussions can be added to clarify why.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "qSRIl7Nbd6", "reviewer": "Reviewer_GUyR", "summary": "In this paper, the authors focus on the online selection of generative models, and in particular, the optimal linear mixture among a set of such models.\nThe problem appears novel, and the authors make interesting connections to the maximization of some kernel-based scores and multi-armed bandit. \nBased on this, the authors propose Algorithms 1 and 2 to solve this online selection of mixture efficiently, with performance guarantee given in Theorem 1 and 2, respectively. (Although I have some concerns on the settings and theoretical results, see below).\nThese methods can be used for widely used kernel inception distance (KID) and Renyi kernel entropy (RKE), and are tested on realistic image and text data in Section 6.", "questions": "Below are a few questions and/or comments.\n\n1. The problem appears novel, so I believe it makes sense to better motivative it. For example, in which context are we interested in picking a model to generate a sample at each round, why it is of interest to use \"the fewest possible sample queries\"? How the proposed method performs in an offline setting, with respect to performance and/or scalability?\n2. When summarizing the contribution of this paper, could the authors also provide (forward) pointers to the precise results? For example, \"proposing an online learning framework in Section ??\". I personally believe that this may facilitate the interested readers to quickly grasp the main contribution of the paper.\n3. Is the working assumption of linearly mixed model somewhat restrictive? Is there something else in the literature, or even such linear combination is (the first time) proposed by the authors in this paper? In fact, on the top row of Figure 3, there is a linearly mixtured \"dog\" that appears a bit bizarre: is this due to some limitation of this linear mixture? \n4. I personally find Theorem 1 a bit surprising: To me, kernel matrix \"estimation\" problem plus some online selection problem, and solving the former problem in general requires a lot of samples to have a tight spectral norm control on the estimated kernel matrix. I believe that the authors avoid this issue by assuming/focusing on the case of bounded. Could the authors comment more on this? For example, does this bounded kernel/loss function setting limit the practical interest of the proposed methods? Also, could the authors comment on the observed sample size $n_i$ for the proposed OGD method to make sense? We do not see this in Theorem 2 and this has an impact on the computational complexity I believe?\n5. a tiny side remark: Figure 3 appears in the main text but commented in the appendix.", "limitations": "", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* The paper consider online selection of generative mixture models, which, to the best of knowledge, is a novel problem of interest.\n* By making interesting connection to kernel-based scores and multi-armed bandit, the authors propose efficient methods to solve the above problem, with some theoretical guarantee. \n* Experiments on realistic data are provided, showing the practical applicability of the proposed approaches.", "weaknesses": "* It would be great to discuss the limitation of the proposed approach, see below for my detailed comments/questions.\n* some settings and theoretical results need clarification, see below for my detailed comments/questions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "0xqLc8FLgV", "reviewer": "Reviewer_mqps", "summary": "The main goal of this work is to maximize the diversity of generated samples by selecting not a single but a mixture of generative models. Formulating first a population loss of quadratic form that can translate into evaluation scores including kernel inception distance (KID) and Renyi kernel entropy (RKE), this article proposes two online algorithms based on continuum-armed bandit and gradient descent, to find the optimal mixture through minimizing an upper confidence bound of the quadratic population loss.  Experiments show that the proposed algorithms are efficient at approaching the optimal mixture.", "questions": "* Can the authors find some other works that also aim to find good mixtures of generative models, and compare their method to these works?\n\n* Can the authors provide the quality scores Density (Naeem et al., 2020). and Precision (Kynkaanniemi et al., 2019) in the experiments that they conducted?\n\n* Small question regarding Lines 257&259: is $\\hat{L}(\\mathbf{a};\\mathbf{x}^{(t)})-(\\mathbf{\\epsilon}^{(t)})^{\\rm T}\\mathbf{a})$ a lower or upper bound of $L(\\mathbf{a})$?\n\n\n\n=======================================================================================================\n\n**Update after rebuttal**\n\nI would like to apologize for an important error that I made in the example presented in my last comment to the authors. I was so rushed to post this last comment before the discussion deadline to give the authors a chance to respond that I did not get to double check my math. As there is no other means for me to reach out to the authors, I have to rectify this error for the authors' information by editing my official review (at the suggestion of AC).\n\nThe error, which I realized after the deadline, came from the application of the FID formula. Under the assumption of Gaussian distributions, the FID is measured by \n$$d_{\\rm F}\\left(P_r,P_g\\right)=\\Vert\\mu_r-\\mu_g\\Vert^2+{\\rm tr}\\left(\\Sigma_r+\\Sigma_g-2(\\Sigma_r\\Sigma_g)^{\\frac{1}{2}}\\right)$$\nwhere $\\mu_r,\\mu_g$ and $\\Sigma_r,\\Sigma_g$ are respectively means and covariances of $P_r,P_g$.\n\nAs in my example, the covariances of the real distribution $\\mathcal{N}(\\mu_r,I)$ and the two generative models $\\mathcal{N}(\\mu_g,I),\\mathcal{N}(\\mu_{\\tilde{g}},I)$ are identity matrices, the FIDs are simply distances between the means:\n$$d_{\\rm F}\\left(\\mathcal{N}(\\mu_r,I),\\mathcal{N}(\\mu_g,I)\\right)=\\Vert\\mu_r-\\mu_g\\Vert^2=\\Vert\\epsilon_g\\Vert^2,$$\n$$d_{\\rm F}\\left(\\mathcal{N}(\\mu_r,I),\\mathcal{N}(\\mu_{\\tilde{g}},I)\\right)=\\Vert\\mu_r-\\mu_{\\tilde{g}}\\Vert^2=\\Vert\\epsilon_{\\tilde{g}}\\Vert^2.$$\n\nThe error occurred when I calculated the FID between $\\mathcal{N}(\\mu_r,I)$ and a mixture $\\mathcal{M}=\\alpha\\mathcal{N}(\\mu_g,I)+\\tilde{\\alpha}\\mathcal{N}(\\mu_{\\tilde{g}},I)$ of the two generative models. While the mean of $\\mathcal{M}$ is indeed $\\alpha\\mu_g+\\tilde{\\alpha}\\mu_{\\tilde{g}}$ as I said in my comment to the authors, the covariance, however, should be $I+\\alpha\\tilde{\\alpha}(\\mu_g-\\mu_{\\tilde{g}})(\\mu_g-\\mu_{\\tilde{g}})^{\\rm T}$, **not** $I$. \n\nTherefore, $d_{\\rm F}\\left(\\mathcal{N}(\\mu_r,I),\\mathcal{M}\\right)\\approx\\Vert\\mu_r-\\alpha\\mu_g-\\tilde{\\alpha}\\mu_{\\tilde{g}}\\Vert^2+\\alpha\\tilde{\\alpha}\\Vert\\mu_g-\\mu_{\\tilde{g}}\\Vert^2$ for large $\\Vert\\mu_g-\\mu_{\\tilde{g}}\\Vert^2$, \nleading to\n$$d_{\\rm F}\\left(\\mathcal{N}(\\mu_r,I),\\mathcal{M}\\right)\\approx\\Vert\\alpha\\epsilon_g+\\tilde{\\alpha}\\epsilon_{\\tilde{g}}\\Vert^2+\\alpha\\tilde{\\alpha}\\Vert\\epsilon_g-\\epsilon_{\\tilde{g}}\\Vert^2.$$Then we have, for independent $\\epsilon_g,\\epsilon_{\\tilde{g}}$, that\n$$E\\\\{d_{\\rm F}\\left(\\mathcal{N}(\\mu_r,I),\\mathcal{M}\\right)\\\\}\\approx\\alpha^2E\\\\{\\Vert\\epsilon_g\\Vert^2\\\\}+\\tilde{\\alpha}^2E\\\\{\\Vert\\epsilon_{\\tilde{g}}\\Vert^2\\\\}+\\alpha\\tilde{\\alpha}E\\\\{\\Vert\\epsilon_g\\Vert^2\\\\}+\\alpha\\tilde{\\alpha}E\\\\{\\Vert\\epsilon_{\\tilde{g}}\\Vert^2\\\\}=\\alpha E\\\\{\\Vert\\epsilon_g\\Vert^2\\\\}+\\tilde{\\alpha}E\\\\{\\Vert\\epsilon_{\\tilde{g}}\\Vert^2\\\\},$$\nwhich is always minimized by taking the single model with the smallest error.\n\nThe intuitive reasoning behind this example is that a mixture of Gaussians is unlikely to match better the target Gaussian distribution than (the best) single Gaussians. The lack of discussion on the limitations and the risks of the mixture approach is my main criticism of this work.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 2, "strengths": "* This paper is well written and easy to follow.\n\n* The theoretical framework underlying the proposed algorithms is well grounded.\n\n* Extensive experiments were carried out to demonstrate the performance of the proposed algorithms.", "weaknesses": "* According to the literature review of this article, there seems to be little interest in finding a good mixture of different generative models. Indeed, if the goal is to approach the target distribution, it makes more sense to select the single best generative model than to use a mixture of different generative models, which are usually trained in an independent manner, therefore unlikely to complement each other.\n\n* It is true that when the objective is to find the single best generative model, the online approach can help prevent sampling from suboptimal models. However, as using a mixture of generative models requires sampling from all member models, the online approach seems to be less useful in this setting.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2dAzTRWHKK", "reviewer": "Reviewer_nCW5", "summary": "This paper study online selection for generative models, in order to generate diverse samples. The authors formulated the problem as a mixture multi armed bandit problem and developed two algorithms for that: Mixture-UCB-CAB and Mixture-UCB-OGD. The authors developed theoretical guarantees for the Mixture-UCB-CAB algorithm. The authors conduct many experiments to show the efficacy of their developed methods.", "questions": "See above.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "It's interesting to see the authors formulated the generative model selection problem as an online selection problem. The authors also developed two algorithms for this new setting and provide theoretical guarantees for one of them. Experimental results demonstrate the efficacy of the proposed algorithms.", "weaknesses": "1. Since this is a new problem, can authors provide more motivations for online selection of generative models, e.g., how important is the ability to generate diverse samples? And how important is to save samples in the selection process.\n2. The authors provide a convergence guarantee for Mixture-UCB-CAB in Thm 2. For comparison, what is the rate of convergence for the offline approach that randomly generate $T$ samples and then optimize over $\\alpha$?\n3. Does Thm 1 holds for all $\\alpha$? Also, the guarantee in Thm 2 doesn't suffer the curse of dimensionality even if the algorithm is selection $\\alpha \\in R^m$; can authors explain why does that happen?\n4. Compared to standard bandit problem where one gets an intermediate regret term at each round, it seems that the studied problems gets $O(t)$ (averaged) terms (the first Eq in Section 5), and all these terms are related to the previous selections $x_1, \\cdots, x_{t-1}$. Can authors elaborate how do they deal with these terms in the analysis? What are some technical contributions?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)