Continuous Parallel Relaxation for Finding Diverse Solutions in Combinatorial Optimization Problems
Authors: Yuma Ichikawa
Hiroaki Iwashita
Venue: Accepted by TMLR ()

Abstract: Finding the optimal solution is often the primary goal in combinatorial optimization (CO). However, real-world applications frequently require diverse solutions rather than a single optimum, particularly in two key scenarios. The first scenario occurs in real-world applications where strictly enforcing every constraint is neither necessary nor desirable. Allowing minor constraint violations can often lead to more cost-effective solutions. This is typically achieved by incorporating the constraints as penalty terms in the objective function, which requires careful tuning of penalty parameters. The second scenario involves cases where CO formulations tend to oversimplify complex real-world factors, such as domain knowledge, implicit trade-offs, or ethical considerations. To address these challenges, generating (i) penalty-diversified solutions by varying penalty intensities and (ii) variation-diversified solutions with distinct structural characteristics provides valuable insights, enabling practitioners to post-select the most suitable solution for their specific needs. However, efficiently discovering these diverse solutions is more challenging than finding a single optimal one. This study introduces Continual Parallel Relaxation Annealing (CPRA), a computationally efficient framework for unsupervised-learning (UL)-based CO solvers that generates diverse solutions within a single training run. CPRA leverages representation learning and parallelization to automatically discover shared representations, substantially accelerating the search for these diverse solutions. Numerical experiments demonstrate that CPRA outperforms existing UL-based solvers in generating these diverse solutions while significantly reducing computational costs.

Published in Transactions on Machine Learning Research (08/2025)
Continuous Parallel Relaxation for Finding Diverse Solutions in Combinatorial Optimization Problems
Yuma Ichikawa
ichikawa.. yuma @fujitsu com
Pujitsu Limited, Kanagawa, Japan
Hiroaki Twashita
hwashita Shushita.hiroakkkkaaa www.hhooookkkuikaak.ooooooo..eae com
Pujitsu Limited, Kanagawa, Japan
Reviewed on OpenReview: https: // openreview. net/ formfo id- 1a33zd52Gu
Abstract
Finding the optimal solution is often the primary goal in combinatorial optimization (CO). However, real-world applications frequently require diverse solutions rather than a single optimum, particularly in two key scenarios. The first scenario occurs in real-world applica- tions where strictly enforcing every constraint is neither necessary nor desirable. Allowing minor constraint violations achieved by incorporating the constraints as penalty terms in the objective function, which requires careful tuning of penalty parameters. The second scenario involves cases where CO formulations tend to oversimplify complex real-world factors, such as domain knowl- edge, implicit trade-offs, or ethical considerations. To address these challenges, generating (i) penalty-diverssp solutions by varying penalty intensities and (ii) variation-divers diversified solutions with distinet structural characteristics provides valuable insights, enabling practi- tioners post-select the most suitable solution for their specific needs. However, efficiently discovering these diverse solutions is more challenging than finding single optimal onc. This study introduces Conlinual Parallel Relaxation Annealing (CPRA), a computa- tionally efficient framework for unsupervised-leannng (UL)-based CO solvers that generates diverse solutions within single training run. CPRA leverages representation learning and parallelization to automatically discover shared representations, substantially accelerating the search for these diverse sonutions. Numerical experiments demonstrate that CPRA out- performs existing UL-based solvers in generating these diverse solutions while significantly reducing computational costs.
often lead to more cost-effective solutions. This is typically
1
Introduction
Constrained combinatorial optimization (CO) problems aim find an optimal solution within feasible space, a fundamental problem in various scientific and engineering applications (Papadimitriou & Steiglitz Korte et al. 2011). However, real-world applications often require diverse solutions rather than single optimal solution, particularly in two key scenarios.
1998
The first scenario arises in real-world applications where strictly enforcing all constraints may neither be necessary nor desirable. In such cases, solutions that slightly violate certain constraints may preferred if they result in significantly better cost performance. For example, soft deadlines in scheduling or minor rule relaxations logistics may lead more cost-cffoctive and practically feasible solutions. A common approach to handling such flexibility is incorporating the constraints into the cost. function as penalty terms, thereby transforming the original constrained optimization problem into unconstrained one. While this formulation broadens the feasible solution space, it introduces the additional challenge of tuning penalty strengths to balance roist satisfaction with cost mininization. To address this, promising strategy
The code available https://gsttie h/tp:hbggtttttt/// com/Yuma- -chiaaascchkaaa/aPPACC

---

Published in Transactions on Machine Learning Research (08/2025)
Post-Select
(ii) Variation-Diverriffer
CPRA UL-based Solver
Solutions
Solutions
Common Representation Automatic Parallelization
Cost
Output Part
#violations
Figure 1: Overview of CPRA UL-basod solver and post-processing for diverse solutions.
is to explore a set of solutions generated under varying penalty configurations, referred to as (i) penally- diversified solutions. Such a solution set enables downstream decision-makers makers to select options based on contextual prioritics, thereby enhancing both the robustness and practical utility of the solver's output, as illustrated in Fig.
A second common scenario occurs in world settings where the mathematical formulation of CO problem provides only coarse approximation of the true recision-making environment. In practice, critical factors such as domain expert ise, implicit trade-ofls, and soft ethical or operational constraints are often too complex, dynamic, or subjective to be explicitly modeled. As a result, the optimal solution to a simplified formulation may not practical or even desirable in real-world deployment. To bridge this gap, generating a diverse set of high-quality solutions that differ in structure or semantics is valuable, referred to as (ii) voriation- diversified solutions. These solutions allow users to post-select the most suitable option, tailored to real-world complexities and adaptable to qualitative, contextual, cvolving criteria not captured by the original CO formulation, illustrated in Fig. [ Moreover, elficiently generaling such diverse candidates is particularly important in high-stakes or large-scale domains, such as healthcare logistics, intelligent tutoring systems, and automated content generation, c.g., game level design (Zhang et al., 2020). where distinct fcasible solutions may reflect differing operational priorities or ethical considerations. However, discovering such diverse solutions efficiently considerably more challenging than identifying a single optimal solution.
To address these challenges, we propose the Continuous Parallel Relaxation Anncaling (CPRA) method, designed for unsupervised learning (UL)-based CO solvers (Schuetz et al.
2022a. Karalias & Loukas 2020
CPRA enables the efficient discovery of both (i) penalty-diversffeed and (ii) variation-diversifed solutions in a single training run, requiring only a minimal architectural change: extending the output layer from generating a single solution to producing multiple diverse solutions, while keeping the intermediate network structure unchanged. This modification ion offers substantial practical benefits by leveraging shared representations across problem instances. The resulting bottlenock structure promotes generalization and scalability, climinating the need to train S separate models iteratively. Furthermore, CPRA incorporates controllable diversity solutions, formulated as continuous relaxation of the
mechanism for (ii) generating max-sum Hamming distance. Unlike the max-sum Hamming distance defined in binary space, which incurs a quadratic computational cost concerning the number of solutions, our approach reduces this to linear complexity, significantly improving the efficiency of both forward and backward passes.
Numerical experiments demonstrate the effectiveness of our approach on several benchmark CO problems. CPRA enables the generation of diverse solutions while maintaining comparable model size and runtime UT-based solvers that produce only single solution. Additionally, CPRA improves search performance by
2

---

Published in Transactions on Machine Learning Research (08/2025)
leveraging shared representations across problem instances, resulting in higher-quality solutions than those produced by existing UT-based solvers and greedy algorithms.
2 Related work
Although diverse solutions can be generated through various paradigms beyond UL-based CO solvers, exist- ing methods often differ significantly in their learning frameworks, algorithmic foundations, and architectural
complexity @Grinsztajn et al.,
2023
Choo et al.] 2022: Li et al.] 2023: Kwon et al. 2020 Kim al. 2021;
Xin et al., [o2llBBunll et al. 2018 Hottung et al. 2021 Chalumeau al. 2023] Hottung et al.] 2024). In
many of these approaches, diversity is introdee an auxiliary mechanism to support the discovery of single optimal solution, rather than being pursued as goal in its own right. Furthermore, these methods solutions. In contrast, our work explicitly targets the efficient discovery of both (i) penally-diverssieee and (ii) varialion- solutions, achieved through minimal modifications to existing UL-based combinatorial optimization solvers such as PI-GNN (Schuetz) et al. 2022a) and CRA-PI-GNN (Ichikawal 2024 Our primary goal is to demonstrate how a simple ar- chitectural extension can unlock broader solution diversity while maintaining practical runtime efficiency. Therefore, an exhaustive empirical comparison with all alternative paradigms falls outside the scope of this work. Penalty-diversiffed solutions can be understood as the result of treating cost and constraint viola- tion as two distinct objectives and combining them through weighted sum. This perspective naturally connects our framework with Pareto front learning methods in multi-objective optimization, notably Pareto hyper networks (Navon et al.) 2021), preferenco-condttoone
typically do not aim to generate (i)
Pareto set learners (Lin et al., 2022), and the survey by Chen et al. (2025). Although these studies provide techniques to approximate the entire set of non-dominated trade-offs, a comprehensive empirical comparison with our approach remains an important avenue for future work.
Diverse Solution Acquisition Without Neural CO Solvers. For penalty-diversifeed solutions, a straightforward bascline is to solve the same problem multiple times with different penalty coefficients, either in parallel, given sufficient. CPU cores, or sequentially. However, both approaches are computationally intensive and scale poorly large instance batches or real-time deployment scenarios. Our method over- comes this limitation by leveraging GPU parallelism to simultaneously generate solutions across multiple penalty regimes, achieving diversity with a runtime comparable to that of solving single instance. Obtain- ing variation-divvrsiieee solutions is even more challenging. A common approach is to post-select solutions that maximize some diversity metric, such as the Hamming distance (Fornau et al. 2019). This line of research spans graph algorithuns (Baste et al.] 2019 2022 Hanaka et al.] 2021), constraiin programinin (Hebrard et al. 2005 |Ptitt & Trapp 2015), and mathematical programming (Danna et al.) 20007 Danna & Woodruff 2009 Petit & Trappl 2019), ; and is often categorized into two types: (1) offline approaches that generate and filter large pool of solutions, and (2) online approaches that incrementally construct diverse solutions. However, both methods face significant drawbacks. Offline approaches often produce redundant solutions and incur high memory and runtime costs. Online approaches, while more targeted, are inher- ently sequential, difficult to parallelize, and susceptible to local optima. Neither approach naturally scales to GPU-based parallel environments. In contrast, our method introduces a GPU-accelerated stralegy for producing diverse high-quality solutions in a single forward pass. This capability especially valuable in worldworld, large-scale, or high-stakes settings such as scheduling, planning, or recommendation, where pre- senting multiple viable options is more useful than committing single solution and where computational efficiency is critical.
3 Notation.
We use the shorthand expression [N] = 2 NEN. IN E NNN represents an identity matrix of size N x N. Here, 1y and ON represent the all-ones vector and all-zeros vector in RN, respectively. G(V, E) represents an undirected graph, where V is the set of nodes and C V V the set of edges. For a graph G)V denote the adjacency matrix with Aij 0 if an edge (i.j) does not exist and Ag > 0 if edge connects and j. For sequence fax ax R the empirical variance is de-
3

---

Published in Transactions on Machine Learning Research (08/2025)
fined [aR[(a))]_]] = Σ
"w/K)//KK and the empirical standard deviation is given by N [VAR[(a))Ee])) For binary vectors b {0, 10%1,, we define the Hamming distance du (a. b) E 1/a + b,] where [[:] denotes the indicator function.
- (ak E#=
STI {ax {ax) k==
4
Background
4.1 Combinatorial Optimization (CO)
Constrained CO problems are defined as follows:
min ((z;C), X(C) = { x E 10, (EX(C)
(x < Ei [7] (y( =0 Vj [ I,JEN,
where C' E C denotes instance parameters, such as a graph G = EV, where C denotes the set of all possible instances. The binary vector a = (i)) E {0, 1 is the decision variable to be optimized, and XCC) denotes the feasible solution space. f: : x C + R denotes the cost function and, for all E [1] and j E [J]. 9: : x C and h : X C R denote caists In practical scenarios, constrained CO problems are often converted into unconstrained CO problems using the penalty method:
I+J
min 1a:; C),, C() f(x; + Σ Axy((;;). (E(0,1)
where, for all i E [I + J), 10,1J C R is the penalty term that increases when constraints are violated. For example, the penalty term is defined as follows:
E [T], (x( max (0, (); Vj E (J], (2 C) = (hy(2;;))
and A = (N)isigH+ RR+ represents the penalty parameters that balance satisfying the constraints and optimizing the cost function. Tuning these penalty parameters X to obtain the desired solutions is a challenging and time-consuming task. This process often requires solving the problem multiple times while iteratively adjusting the penalty parameters À until an acceptable solution obtained.
4.2 Continuous Relaxation and UL-based Solvers
The continuous relaxation strategy reformulate CO problem by converting discrete variables into continuous ones as follows:
min 1
((p.C,)) A f(p:C) (.p.C.A) Σ (.:
where p (Pi)ssiNN 1. denotes relaxed continuous variables, i.e., each binary variable v, E [0, is relaxed to a continuous one pt E [0, 1], and f: 10, x C + R is the relaxation of f, satisfying f(x; fxx; C)) for any x E {0.1} The relation between the constraint v, and its relaxation 0, is similar for e JJ, ie., Vi e [1 + J], 6,(x; C*) = (, C*) for any æ e 10,1)
U--based solvers employ this continuous relaxation strategy for training neural networks (NNs) Wang al., 2022 Schuetz et. al. 2022a Karalias & Loukas 2020 Ichikawa 2024). The relaxed continuous variables are
parameterized by 0 pe 1] and optimized by directly minimizing the following loss function:
I+J
x + Σ (po(C)))
(1)
After training, the relaxed solution Pe is converted into discrete variables by rounding Pe using a threshold (Schuctz et al. 2022aa or by applying a greedy method (Wang et al.) 2022). Two types of schemes have been developed based this framework.
4

---

Published in Transactions on Machine Learning Research (08/2025)
(I) Learning Generalized Heuristics from History / /aata. One approach, proposed by Karalias & Loukas (2020). seeks to automatically learn commonly effective heuristics from historical dataset instances D = [ pp= and then apply these learned heuristics to new instance C* via inference. Specifically, given a set of training instances, independently and identically distributed from a distribution P(C), the objective is to minimize the average loss function ming E 1(0; A): However, this method does not. guarantee high-quality performance for a test instance C*. Even if the training instances 'D are abundant and the test instance is drawn from the same distribution P(C), achieving a low average performance EC= ((0) C)) does not necessarily guarantee a low 1(0; C) for a specific C. To address this issue, [Wang & Li (0023) introduced a meta-learning approach where NNs aim to provide good initialization for new instances.
(IT) Learning Effective Heuristics on Specific Single Instance. Another approach; referred to the physics-inspired graph neural networks (PI-GNN) solver (Schuetz et al,, 2022a b), automatically learns instance-specific heuristics for given single instance using the instance parameter C' by directly employing Eq. Q This approach has been applied to CO problems on graphs, ie.. C' = G(V, E), using graph neural networks (GNN) to model the relaxed variables Po()). An I-layered GNN is trained to directly minimize [0 X) in Eq. [ taking as input a graph G along with node embedding vectors and producing the relaxed solution (G) E [0, 1j A detailed description of GNNs can be found in Appendix B.1 Note that this sctting is applicable even when the training datasct 'D is difficult to obtain. However, learning to minimize Eq. 11) for a single instance can be tine-consuming than the inference process. Nonetheless, for large-scale problems, it has demonstrated superiority over other solvers in terms of both time and solution performance [Schuetz et al. 2022a b Ichikawa 2024
UT-based solvers face two practical issues: (T) "optimization issues", where they lend to gel stuck in local optima, and (1I) "rounding issues", which arise when an artificial post-learning rounding process is needed to map solutions from the contimous space back to the original discrete space, undermining the robustness of the results. To address the first issue, Ichikawa [2024]; Ichikawa & Arai 2025 proposed annealing schemes to escape local optima by introducing the following entropy term s(0; C):
N
P(0: ((0;C,A,)) = (C) + (s(po(C)), s(po(C))
a E (2n n meENN (2) Σ (C) 1)a - 1},
where 7 E R denotes penalty parameter. They anneal the penalty parameter from positive 0 to x 0 to smooth the non-convexity of the objective function i(0: C) similar to simulated annealing (Kirkpatrick lt al. 1983). To address the second issue, Ichikawa (024) further annealed the entropy term to S 0 until the entropy term approaches zero, i.e., C) R O, enforcing the relaxed variable to take on discrete values and further smoothing the continuous loss landscape for original discrete solutions. This method is referred to as Continuous Relaxation Anncaling (CRA), and the solver that applics the CRA to the PI-GNN solver is referred as CRA-PI-GNN solver.
5 Continuous Parallel Relaxation Annealing for Diverse Solutions
We propose an extension of CRA, termed Continuous Tensor Relaxation (CPRA), which enables UL- based solvers to efficiently handle multiple problem instances within single training run. Beyond this core advancement, we demonstrate how CPRA can be effectively tailored to discover both penalty-diversifeed solutions and diversifdd solutions.
5.1 Continuous Parallel Relaxation (CPRA)
Let us consider solving multiple instances Cs (C. I E Chiees with different penalty parameters As = (A. simultaneously. To handle these instances, we relax a binary vector x E [0, 1 into an augmented continual matrix P E [0, N| that is trained via minimizing the following loss function:
5 R(P:Cs,As.)) Σ (P.:: +SP(), S(P) A
N s
( (2P,. - 1)9),
5

---

Published in Transactions on Machine Learning Research (08/2025)
where P. [0, 1 denotes s-the column in P, ie. P= P=(P))ssess [].1] Optimizing R drives each column Pss to minimize its respective objective function i(P::: C, Ag). Additionally, we also generalize the entropy term s(p) in Eq. (2) into S(P) for this augmented higher-order array. Specifically, the following theorem holds.
Theorem 5.1. Under the assumption that for all s E [S], each objective function i(P:s: C,, remains bounded on [0, 1]N, each column solutions P. such that P argminpR(PPCC. As. converges the corre- sponding discrele oplimal x' argmin,!(ar: Cs, As) as a + +00. Furthermore, as 7 + -0;, the loss function Ag) becomes convez and admits unique half-integrai solution 1x1 argmin 7).
A detailed proof of Theorem 5.1 is available in Appendix A.I The relaxation approach naturally extends to higher-order arrays, P E [0, I] X potentially enabling more powerful GPU-based parallelization. A comprehensive exploration of such higher-dimensional implementations remains an exciting avenue for future roscarch. For UL-based solvers, we parameterize the soft higher-order array P Po, leading to
N 9
As, (Cs: C..)) ==1
+ C. S(Po(Cs))
11 (2Po, -1))), (3)
where y is also annealed from a positive to a negative value in CRA-PI-GNN solver in Section Following
the UL-based solvers (Karalias & Loukas, 2020; Schuctz et al. 2022b
Ichikawa
2024 we encode Po via
GNN-based architecture. This study refer to the solver that applies CPRA to PI-GNN solver CPRA-PI-
GNN solver.
Simple A Architectural Modifications for Efficient Learning and Shared Representation Learning. In this study, we employ specialized GNN-based architecture that builds upon the core designs of PI-GNN (Schuetz et al. 2022a) and CRA-PI-GNN (Ichikawa, 2024 to simultaneously address multiple problem instances, as defined in Eq. 3]. Unlike these solvers that generate single solution per instance, CPRA-PI- GNN handles S instances in parallel by expanding the final-layer node embedding dimension from to S, as illustrated in Fig. [ As a result, the number of parameters increases linearly only in the output. layer, while the rest of the architecture remains unchanged. This design is both memory- and computation-efficien as the overall network size and training time remain comparable to solving a single instance.
Furthermore, by keeping the network unchanged apart from the output layer, CPRA-PI-GNN solver nalu- rally encourages the model to learn compact, shared representations across multiple instances, functioning similarly to a bottleneck in an autoencoder, as illustrated in Fig. [ This simple architectural choice improves solution quality by leveraging the learned representations. Indeed, numerical experiments demonstrate that this shared representation learning strategy yields better results than single-instance solvers such as PI-GNN and CRA-PI-GNN. Additional results in Appendix D.5 further show that CPRA-PI-GNN solves multiple similar problems more efficiently and effectively than CRA-PI-GNN. To further reduce computational cost, we adopt two-stage learning process that leverages shared representation learning: first training on smaller, representative subset S' c S, then fine-tuning only the final-layer embeddings when scaling to the full set S. This approach offers efficient way to extend the model to larger problem sets.
5.2 CPRA for Finding Deveste- Solutions
To find penalty-dieee diversifee solutions, aim to minimize the following loss function for problem instance C, which is special case of Eq.
S R(0:C, = Σ (CC.A
(4)
By solving this optimization problem, each column (C.), for all [S]. corresponds to the optimal solution for the penalty parameter Ag. For denalty-diversfied solutions, the variation each s is primarily restricted to the penalty coefficient, leading to a strong correlation among instances.

---

Published in Transactions on Machine Learning Research (08/2025)
5.3 CPRA for Finding
Solutions
To explore marialion-diversifeed solutions for single instance C with penalty parameter X, we introduce diversity penalty into Eq. as follows:
V(Po(C)) = - STD ((Po:((C)))sss))
where (P)(C)) serves as a constraint term that promoles diversity in each column Po.:(C), and v is the parameter controlling the strength of this constraints. Setting v = in Eq. (5) is nearly equivalent to solving the same CO problem with different initial conditions.
As shown in Proposition 5.2 the diversity term (PP((C)) can be interpreted a natural relaxation of the max-sum Hamming distance, widely used diversity metric in combinatorial optimization (Fomin et al. 2020 2023 Baste et al. 2022 2019). Specifically, for set of binary sequences {x(*))s |m with x 5) 10, the following holds:
Proposition 5.2. For set of binary sequences (xlt) Vs, al() E (0. 10N, the following equalily holds:
S2 VAR
(4) Isaes
Σ xx)
(6)
where the right-hand side of Eq. represents the maz-sum Hamming distance.
The detailed proof can be found in Appendix
This proposition shows that the proposed diversity penalty corresponds exactly to the max-sum Hamming distance when evaluated at the binary vertices of the hypercube {0, 13,h, which means the left hand side of Eq. is natural continuous relaxation of the discrete diversity measure onto the continuous domain [o, 1]
A.2
This relaxation yields a significant computational advantage. Directly computing the max-sum Hamming distance involves evaluating all (9) O(S²) pairwise distances between solutions, which becomes compu- tationally expensive as S increases. In contrast, the relaxed diversity term Vi(-) computes per-coordinate variance using simple summary statist which can be evaluated in ((S) time. This efficiency holds for both forward and backward passes, approaching well-suited to gradient- based optimization. To ensure consistent scale and sensitivity with the other loss components in Eq. 5 we normalize the diversity term using the sample standard deviation.
6 Experiments
This section evaluate the effectiveness of CPRA-PI-GNN solver in discovering ponalty-diversifeed and variation-dliverifeed solutions across three CO problems: the maximum independent set (MIS), maximum cut (MaxCut), diverse bipartite matching (DBM) problems. Their objective functions are summarized in Table in Appendix C.3 For a detailed explanation, refer Appendix C.3
6.1 Settings
Bascline. Our bascline include results from exccuting greedy algorithms, PI-GNN solver (Shuuttz et al. 2022a)) and CRA-PI-GNN solver [Ichikawa 2024) multiple times. These solvers are executed multiple times using different penalty parameters for
solutions and different random seeds for variation- diversified solutions, allowing us to assess the scarch efficiency for both types of diversified solutions. For the MIS problem, we employ random greedy search implemented by NetworkX, and for the MaxCut problem, random greedy search implemented by Mehta (019). Although some online heuristics exist for
7

---

Published in Transactions on Machine Learning Research (08/2025)
2.0
1.0
Apit
@violationaN
1.0
GINN
PLONN
0.9
CRA-PE-GNN
CRA-PHGNN
CPRA-PLEEN
CPRA-PLENN
MIS (d
Method {#Runs}
# #Params
Time (s)
ApR
1.2
0.6
@Violations/
1.0
PI-GNN {20}
5,022,865x20
189189160
0.883+00.002
0.8
0.4
CRA {20)
5,022,865x20
14,400#42 0.961#0.002
0.6
0.3
CPRA {1}
5,083,076
1,19418
0.93440.002
0.4
0.2
0.2
212
ApR
PI-GNN
FI-GNN GNN CPRA-PI-GNN
1.0
GNN CPRA-BL-GNN
0.8
MIS (d 20)
1.4
0.7
#violationsis
Method (#Rus))
# Params
Time (s)
ApR
1.2
1.0
PI-GNN {20}
5,022,865x20
15,191
0.759=000077
0.8
CRA {20}
5,022,865x20
14,816#40
0.928=0.004
CPRA {1)
5,083,076
1,254+10
0.878++0.011
0.4
0.2
0.2
0.1
0.0
0.0
2
2
21
A5
Figure 2: (Left Table) shows runtime (Time), number of parameters (#Params), and maximum ApR (ApR") for each method. (Right Figure) shows ApRs across different penalty parameters A,. Error represent the standard deviations of 5 random seeds. CPRA-PI-GNN solver can find penalty-diversified solutions in a single run with a comparable # #Params and runtime to UL-based solvers that output a single solution.
exploring variation-diverssfiel solutions by generating solutions that are distant from those already obtained, we do not include these methods as benchmarks due to their inefficient GPU utilization and poor scalability large problems. We measure the runtime & of each execution, from model training the final output.
This numerical experiment aims validate that CPRA can generate and variation-diversifedd solutions while maintaining comparable number of parameters and runtime to UL. based solvers, which produce single solution, described in Section 5) Therefore, in experiments, the CPRA-PI-GNN solver utilizes the same network architecture as the PI-GNN (Schuetz et al. 2022a and CRA-
Implementation
diversified
PI-GNN [Tchikawa 2024 solvers, except for the output size of the final layer as discussed in Section 5 We use GraphSage, implemented with the Doep Graph Library WWagg et al. 2019). The detailed architectures of these GNNs are provided in Appendix C.1 We employ the AdamW (Kingma & Ba 2014) optimizer with learning rate
x 10 epochs with early stopping, which monitors the summarized loss function >s, i(P...) and the entropy term p (PP,ya)) a),
10- and weight decay of 10 The GNNs are trained for up
using a tolerance of 10-3 and patience of 10 epochs. Further details are provided in Appendix C.2 We set the initial scheduling value to 7(0) = -0 for the MIS and DBM problems and 700) -6 for the MaxCut
problems, using the same scheduling rate 10-3 and curvature rate = 2 in Eq. 3
Evaluation Metrics. Following the metric of Wang Li 2023) we use the approximation rate (ApR) for all experiments, defined as ApR = f(=;())/((=000)) where x" represents the optimal solutions. For MIS, these optimal solutions set to the theoretical results (Barbier et al. 2013), for DBM problems, they are identified using Gurobi 10.0.1 solver with default settings, and for MaxCut problems, they are the best-known solutions. To evaluate the quality of penalty- diversiiie solutions,
compute ApR max,e (ApR(x.) (3). To evaluate the quality of variation-diversifi solutions,
as function of the parallel number S in Eq. compute the average ApR, defined as ApR ΣS ApRe/s, and introduce diversity score (DScore) for
8

---

Published in Transactions on Machine Learning Research (08/2025)
2
Apll - 0.0
ApR 0.2
DBM instance- matching-1
12
AcR inforsile
Method {#Runs}
#Params
Time (s)
ApR
2
2 1 t
12
2
PI-GNN {121}
12,507,50 501 x 121 45,000=5,777 778 0.883=0.040
24
a
CRA {121}
12,507,501 x 121 213,6121:,132
1.000=1.0000
CPRA {1}
13,107,621
1,961=101
0.883#0.011
12 1
8
12 2
12 0
21
2
23
24
27
24
27
210
Au.
A2
2
AsR - 0.0
Aplt
DBM instance-1, matching-2
e 1 9 2
Apll 0.8
AcR 1.0
Method (#Runs)
#Params
Time (s)
ApR
0
2
8
PI-GNN {121}
12,507,500 x121 18,064=7,100
0.927=00.024
2 1
CRA {121}
12,507,501 x121 140
0.990:10.013
CPRA (1)
13,107,621
2,154+164
1.000+0.000
2 1
2 2
24
20
Au
Figure 3: (Teft Table) shows runtime (Time), number of parameters (#Params), and maximum ApR (ApR") for each method, with errors representing the standard deviations of 5 random seeds. (Right Figure) shows ApRs, where each point represents the results from 5 random seed across various penalty parameters As {A. = (Ae; Åa Ab Ao) >ay E {2" = O,..., 10}}. CPRA-PI-GNN solver is capable of finding penalty- diversified solutions in single run, with comparable number of parameters and runtime those of UL-based solvers.
the bit sequences {#.)
2 (S NS(S 1)
Σ
dh wi)
A higher DScore indicates greater variation among solutions. A desirable variation-diversifee solution should exhibit both high-quality solutions and diverse set of solutions with distinct characteristics. Thus, solutions with higher values of both average ApR and DScore are more desirable.
6.2 Finding Penalty-Diversified Solutions
MIS Problems. First, we compare the performacne of CPRA-PI-GNN on MIS problems in RRGs, GVV, E). with [V| = 10,000 nodes and the node degree of 5 and 20. CPRA-PI-GNN solver run using Eq. with a set of ponalty parameters, As = (2--3
1...., 1,.....0).. CRA-PI-GNN and PI-GNN solver run multiple times for each penalty parameter X, E As. Fig, (Right) shows the ApR as a function of penalty parameters X As. Across all penalty parameters, from 2 to 216 CPRA-PI-GNN solver performs on par with slightly underperforms CRA-PI-GNN solver. Table in Fig. 2 shows the runtime and number of paramers (#Params) for CPRA-PI-GNN solver at S = 20, compared to the total runtime and # Params for S runs of PI-GNN and CRA-PI-GNN solvers. These result indicate that CPRA-PI-GNN solver can find pevelty-diversifeed solutions with a comparable number of parameters and runtime to UL-based solvers that output a single solution. For a more detailed discussion on the dependence of the runtime and the #params for number of shot. S, refer to Appendix D.I
DBM Problems. We next demonstrate the effectiveness of CPRA-PI-GNN solver for DBM problems, which serve practical CO problems. We focus the first of the 27 DBM instances; Appendix

---

Published in Transactions on Machine Learning Research (08/2025)
Figure 4: The obtained solutions by CPRA-PI-GNN solver for the MIS problem on RRG with 30 nodes and the degree d = 3. Blue nodes represent the independent set.
GREEDY
PL-GNN
MIS (d = 20)
Aphi
PC2 (0
Method [#Runs)
Time (s)
ApR
DScore
PC2
50
50
30
0
50
0.50
Greedy {300}
8
0.715
0.239
CPRA (.=0.0)
PC1 (0.59%)
CPRA (v=0.3)
PC1 (0.60%)
PI-GNN {300}
13,498
0.712
0.238
pe2 (0.008)
50
30%)
8
CRA {300}
15,136
0.923
0.248
0.85
-50
CPRA (v 0.0) {1)
95
0.873
0.019
-30
50
-30
PC1 (100.000
PCI (23 02%)
CPRA (v =022) (1)
154
0.936
0.260
(16.07%)
CPRA (o44
(17.99%)
0.75
CPRA (F 00.4 (1)
154
0.900
0.251
50--
CPRA (u 00.6 (1)
149
0.852
0.257
-50
(23.24%)
(20.00%)
Figure 5: (Left Table) shows runtime (Time), average ApR (ApR), DScore for each method on MIS problems with node degree d = 20. (Right Figure) shows the distribution of solutions in 2-dimensional space using PCA with varying v.
D.6 for the results of the remaining instances. Given that (21, and (Aa, A4) share similar properties, CPRA-PI-GNN run with a set of S = 11 x 11 parameters on the a grid, As = {>, = (Aa: Ab. ),)). where Aar A {2" s = O....,
.....,10.. 10}. CRA-PI-GNN and PI-GNN solver run multiple times for each penalty parameter X Ag Fig. 3 (Right) shows that the ApR on the grid Ag using the CPRA-PI-GNN solver identifies desirable region where the ApR is nearly 1.0. Table in Fig. demonstrates that CPRA-PI- GNN solver can find develty-diversiffee solutions with comparable number of parameters and runtime to UL-based solvers that output a single solution.
6.3 Finding Divvessiiede Solutions
We next demonstrate that CPRA-PI-GNN solver can efficiently find variation-dlvverifeed solutions. Further- more, we also show that the CPRA-PI-GNN solver enhances exploration capabilities and achieves higher- quality solutions.
MIS Problems. We first run CPRA-PI-GNN solver using Eq. 5 to find variation-ddivrsifieed solutions for MIS problems on small-scaled RRGs with 30 nodes and the node degree set 3. We set the parameter / = 0.5 and the number of shots tp 5 100 Eq. (5) As shown in Fig. 4 CPRA-PI-GNN solver successfully obtain solutions, each with 13 independent sets, which the global optimum. We extend the investigation to large-scale RRG with 10,000 nodes and node degree d 20, which is known for its optimization challenges (Angelini & Ricci- Tersenghi 2023 These experiments investigate how the quality of variation-diversifeed solutions depends on the paramcter using fixed number of shots S = 300. Fig. 5 (Right) shows low dimensional visualization of the normalized solutions (P00 1300 using two-ddimensional principal component analysis (PCA) mapping. The two principal components with the highest contribution rates are selected for different parameters v 0.0, 0.2, 0.4,0.6. These results indicate that increasing paramoter lcads more diverse solutions, with the solution space becoming increasingly separated in the high-contribution region. Table in Fig. measures the computation time, ApR, and DScore when the parallel execution
10

---

Published in Transactions on Machine Learning Research (08/2025)
GREEDY
PL-GNN
PC2 (0.460)
MaxCut G14
25
Aph
Method [#Runs}
Time (s)
ApR
DScore
-25
25
-25
25
PC1 (0.01%)
PC1 (2.20%)
Greedy {1,000}
87
0.936
0.479
CPRA (o-
PC22 (98.979))
CPRA (.4)
0.98
PI-GNN {1,000}
25,871
0.963
0.499
PC2 (0.00%)
037
CRA {1,000}
48,639
0.988
0.499
CPRA (v -0.0) {1)
144
0.977
0.497
-25
PC1 (100.00%)
-25
PC1 (61 (336))
0.55
CPRA (v = 0..4
(1)
138
0.991
0.501
rC2 (94.3400
25
CPRA (e=0.3
CPRA (2
0.91
CPRA (u = =0.8) (1)
141
0.989
0.501
PC2 (255
CPRA (v- 1.2) (1)
147
0.985
0.502
-25
PC1 (55.748)
PC1 (87.50%)
Figure 6: (Left Table) shows runtime (Time), average ApR (ApR), DScore for each method on MaxCut G14, with error representing the standard deviations of 5 random seeds. (Right Figure) shows the distribution of solutions a 2-dimensional space using PCA with varying v.
number S = 300 using different random seeds. The results show that although the time of CPRA-PI-GNN solver takes longer than executing the greedy algorithm multiple times, both ApR and DScore reach their maximum at y 0.2, yielding the highest quality variation-diversifee solutions. Furthermore, increasing parameter v enhances the exploration capability of GNN, leading to better solutions than those obtained by conventional PI-GNN and CRA-PI-GN;; see Appendix D.2
MaxCut Problems. Next, we evaluate the ability to find variation-diveesifeed solutions in the G14 in- stance of Gset, which primarily has four-clustered solution space. We set the mumber of shot S 1,000 Eq. (5 Fig. 6 (Right) demonstrates that CPRA-PI-GNN solver can capture four-clustered solutions beyond certain value of v. Table in Fig. measures the computation time, ApR, and DScore when the parallel 1,000 is performed using different random seeds. The results show that although the runtime of CPRA-PI-GNN solver is slower compared to executing the greedy algorithm multiple times, ApR and DScore reach their maximum at v = 0.4 and v 1.2, respectively. Additionally, similar to MTS problems, exploration enhancement is consistent across various instances of Gset. For further details; see Appendix D.3
exccution number
7 Conclusion
This study introduces the CPRA framework for UT--based solvers designed to efficiently find penalty- diversified and variation-diversifeed solutions within single training process. Our numerical experiments demonstrate that CPRA can produce penalty-diversifedd and variation-diversifeed solutions while maintain- ing a comparable number of parameters and runtime to conventional UT-based solvers that generate only a single solution. This approach not only enhances the computational efficiency in finding these diversified solutions but also improves the search capabilities, leading to higher-quality solutions compared to existing UL-based solvers that find a single solution and greedy algorithms.
Acknowledgments
We would like to thank Hisanao Akima and Nobuo Namura. We also appreciate the constructive leedback provided by the anonymous reviewers and the action editor. This work was supported by Fujitsu Limited.
References
Bahram Alidace, Gary A Kochenberger, and Ahmad Ahmadian. 0-1 quadratic programming approach for optimum solutions of two scheduling problems. International Journal of Systems Science, (5(2):001-408, 1994.
11

---

Published in Transactions on Machine Learning Research (08/2025)
Maria Chiara Angelini and Federico Ricci- Ricci-Tersenghi. Modern graph neural networks do worse than classical greedy algorithms in solving combinatorial optimization problems like maximum independent set. Nature Machine Intelligence, 5(1):29 31, 2023.
Jean Barbicr, Florent Krzakala, Lenka Zdeborová, and Pan Zhang. The hard-core model on random graphs revisited. In Journal of Physics: Conference Series, volume 473, pp. 012021. IOP Publishing, 2013.
Julien Baste, Lars Jaffke, Tomáš Masačik, Geevarghese Philip, and Günter Rote. Fpt algorithms for diverse collections of hitting sets. Algorithms,, 12(12):254, 2019.
Julien Baste, Michael R Fellows, Lars Jaffke, Tomás Masařík, Mateus de Oliveira Oliveira, Geevarghese Philip, and Frances A Rosamond. Diversity of solutions: An exploration through the lens of fixed- parameter Iractability theory. Artificial Intelligence, 303:103644, 2022.
Mohsen Bayati, David Gamarnik, and Prasad Tetali. Combinatorial approach to the interpolation method and scaling limits in sparse random graphs. In Proceedings of the forty-second ACM symposium on Theory of computing, pp. 105-114, 2010.
Rudy Bunel, Matthew Hausknecht, Jacob Devlin, Rishabh Singh, and Pushmeet Kohli. Leveraging grammar and reinforcement learning for neural program synthesis. arXiv preprint orXiv: 04276, 2018.
Felix Chalumeau, Shikha Surana, Clément Bonnet, Nathan Grinsztajn, Arnu Pretorius, Alexandre Laterre, and Tom Barrett. Combinatorial optimization with policy adaptation using latent space search. Advances in Neural Information Processing Systems, 36:7947-7959, 2023.
Weiyu Chen, Xiaoyuan Zhang, Baijiong Lin, Xi Lin, Han Zhao, Qinglu Zhang, and James Kwok, Gradient- based multi-objective deep learning: Algorithms, theories, applications, and beyond. arXiv preprint arXiv:2501.1000 2025.
Jinho Choo, Yeong-Det Kwon, Jihoon Kim, Jeongwoo Jae, André Hottung, Kevin Tierney, and Youngjune Gwon. Sinulation-guided beam search for neural combinatorial optimization. Advances in Neural Infor- mation Processing Systems, 35:8760-8777 2022.
Amin Coja-Oghlan and Charilaos Efthymion. On independent sets in random graphs. Random Structures Algorithmss 47(3):436 486, 2015.
Emilic Danna and David L Woodruff. How to select small set of diverse solutions mixed integer programming problems. Operalions Research Letters, 37(4))255-220 260, 2009.
Emilie Danna, Mary Fenelon, Zonghao Gu, and Roland Wunderling. Generating multiple solutions for mixed integer programming problems. In International Conference on Integer Programming and Combinatorial Oplimization, pp. 280 294. Springer, 2007.
Michel Deza and Monique Laurent. Applications of cut polyhedra ii. Journal of Computational and Applied Mathematics, 55(2):217-22 1994.
Aaron Ferber, Bryan Wilder, Bistra Dilkina, and Milind Tambe. Mipaal: Mixed integer program as layer. In Proceedings of the AAAI Conference Artificial Intelligence, volume 34, pp. 1504-1511, 2020.
Henning Fernau, Petr Golovach, Marie-France Sagot, et al. Algorithmic enumeration: Output-sensitive, input-sensitive, parameterized, approximative (dagstuhl seminar 18421). In Dagstuhl Reports, volume 8. fuer Informatik, 2019.
Schloss Dagstuth--Liibnizzeeettee
Fedor V Fomin, Petr A Golovach, Lars Jaffke, Geevarghese Philip, and Danil Sagunov. Diverse pairs of matchings. aix preprint urXiv:2009. 0e020009.,556,, 04567, 2020.
Fedor V Fomin, Petr A Golovach, Fahad Panolan, Geevarghese Philip, and Saket Saurabh. Diverse collections in matroids and graphs. Mathematical Programming, pp. 1 -33, 2023.
12

---

Published in Transactions on Machine Learning Research (08/2025)
Justin Gilmer, Samuel S Schoenholz, Patrick F Riley, Oriol Vinyals, and George E Dahl. Neural message passing for quantam chemistry. In International conference on machine learning, pp. 1272. PMT.R, 2017.
Nathan Grinsztajn, Daniel Furclos-Blanco, Shikha Surana, Clément Bonnet, and Tom Barrett. Winner takes it all: Training performant rl populations for combinatorial optimization. Advances in Newral Information Processing Systems, 1448485-48500 2023.
Tesshu Hanaka, Yasuaki Kobayashi, Kazuhiro Kurita, and Yota Otachi. Finding diverse trees, paths, and 2021.
more. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp.
Emmanuel Hebrard, Brahim Hnich, Barry O'Sullivan, and Toby Walsh. Finding diverse and similar solutions in constraint programming. In AAAI, volume 5, pp. 372 377, 2005.
André Hottung, Bhanu Bhandari, and Kevin Tierney. Learning a latent search space for routing problems using variational autoencoders. In International Conference on Learning Representations, 2021.
André Hottung, Mridul Mahajan, and Kevin Tierney. Polynet: Learning diverse solution strategics for neural combinatorial optimization. arXiv preprind or Xix:22202. 1 4048, 2024.
Yuma Ichikawa. Controlling continuous relaxation for combinatorial optimization. In A. Glober- son; L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomezak, and C. Zhang (eds.), Ad- vances in Neural Information Processing Systems, volume 37, pp. 47189 47216. Curran As- sociates, Inc., 2024. URL httpp::/pproceeddngs.nut neur.ss....rp...... 64001352000000000003eeeeefr20002 http:/.eeddda....rrr.....cepeped hctpsppccpppprr..pec
files/ filess/apper2022//11
Yuma Tchikawa and Yamato Arai. Optimization by parallel quasi-quantum annealing with gradient-based
sampling. In The Thirteenth International Conference on Learning Representations, 2025. URL https:
//openreview www./eeereiidggeeeeeee 9
Nikolaos Karalias and Andreas Loukas. Erdos goes neural: an unsupervised learning framework for com- binatorial optimization on graphs. Advances in Newral Information Processing Systems, 666229-6672, 2020.
Richard M Karp. Reducibility among combinatorial problems. Springer, 2010.
Minsu Kim, Jinkyoo Park, et al. Learning collaborative policies to solve np-hard routing problems. Advances in Neural Information Processing Systems, 34:10418 10430, 2021.
Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXio preprint arXiv:1412.6988 2014.
Scott Kirkpatrick, C Daniel Gelatt Jr, and Mario P Vecchi. Optimization by simulated annealing. science, 220(45965::611-688 1983.
Bernhard Korte, Jens Vygen, B Korte, and Vygen. Combinatorial optimization, volume 1. Springer, 2011.
Yeong-Dae Kwon, Jinho Choo, Byoungjip Kim, μioo Yoon, Youngjune Gwon, and Seungjai Min. Pomo: Policy optimization with multiple optima for reinforccment learning. Advances in Neural Information Processing Systems, 33:21 188 21198, 2020.
Jingwen Li, Yining Ma, Zhiguang Cao, Yaoxin Wu, Wen Song, Jie Zhang, and Yeow Meng Chee. Learning feature embedding refiner for solving vehicle routing problems. IEEE Transactions on Newral Networks and Learning Systems, 2023.
Xi Lin, Zhiyuan Yang, and Qingfu Zhang. Pareto set learning for neural multi-objective combinatorial optimization. In 10th International Conference on Learning Representations (ICLR 2022). International Conference Learning Representations, ICLR, 2022.
13

---

Published in Transactions on Machine Learning Research (08/2025)
Jayanta Mandi, Victor Bucarey, Maxime Mulamba Ke Tchomba, and Tias Guns. Decision-focused learning: through the lens of learning to rank. In International Conference on Machine Learning, pp- 11935-14944 PMLR, 2022.
Hermish Mehta. Cvx graph algorithms. httpp////ggtttt..co////////aaarr
2019.
Maxime Mulamba, Jayanta Mandi, Michelangelo Diligenti, Michele Lombardi, Victor Bucarey, and Tias Guns. Contrastive losses and solution caching for predict-and-optimize. arXiv preprint arXiv:2011.0535,, 2020.
Aviv Navon, Aviv Shamsian, Gal Chechik, and Ethan Fetaya. Learning the pareto front with hypernetworks. In International Conference on Learning Represenlations, 2021. URL id=NjE772F4Z2R
Hartmut Neven, Geordie Rose, and William G Macready. Image recognition with an adiabatic quantum computer i. mapping to quadratic unconstrained binary optimization. arXiv preprint arXiv:0804.4457, 2008.
Christos H Papadimitriou and Kenneth Steiglitz. Combinatorial optimization: algorithms and complerity. Courier Corporation, 1998.
Thierry Petit and Andrew C Trapp. Finding diverse solutions of high quality to constraint optimization problems. In IJCAI. International Joint Conference on Artificial Intelligence, 2015.
Thierry Petit and Andrew C Trapp. Enriching solutions to combinatorial problems via solution engineering, INFORMS Journal on Computing, 31(3):429 14449944444222 2019.
Franco Scarselli, Marco Gori, Ah Chung Tsoi, Markus Hagenbuchner, and Gabriele Monfardini. The graph neural network model. IEEE transactions on neural networks, 20(1):61-80, 2008.
Martin JA Schuetz, J Kyle Brubaker, and Helmut G Katzgraber. Combinatorial optimization with physics- inspired graph neural networks. Nature Machine Intelligence, 4(4):367-37,, 2022a.
Martin JA Schuetz, J Kyle Brubaker, Zhihnai Zhu, and Helmut G Katzgraber. Graph coloring with physics- inspired graph neural networks. Physical Review Research, 4(4):043,31, 2022b.
Prithviraj Sen, Galilco Namata, Mustafa Bilgic, Lisc Getoor, Brian Galligher, and Tina Eliassi-Rad. Collec- tive classification in network data. AI magazine, 29(3):93-93, 2008.
Haoyu Wang and Pan Li. Unsupervised learning for combinatorial optimization needs meta-learning, arXiv preprint 2rXte:3311008116 2023.
Haoyu Peter Wang, Nan Wu, Hang Yang, Cong Hao, and Pan Li. Unsupervised learning for combinatorial optimization with principled objective relaxation. Advances Neural Information Processing Systems, 35:31444 31458, 2022.
Minjie Wang, Da Zheng, Zihao Ye, Quan Gan, Mufei Li, Xiang Song, Jinjing Zhou, Chao Ma, Lingfan Yu, Yu Gai, et al. Deep graph library: A graph-centric, highly-performant package for graph neural networks. arXiv preprint ar Xiv: 1909. 01315, 2019.
Liang Xin, Wen Song, Zhiguang Cao, and Jie Zhang. Multi-decoder attention model with embedding glimpse for solving vehicle routing problems. In Proceedings of the AAAT Conference on Artificial Intelligence, volume 35, pp. 12042 12049, 2021.
Y. Ye. The gset dataset. https: :ctps:///eb.. ://wbb///ebbee stw.forr....ood...oomoo seiiidddddu//yyee wwwpe///w//yyyy//eeeeeeeee 2003.
Hejia Zhang, Matthew Fontaine, Amy Hoover, Julian Togelius, Bistra Dilkina, and Stefanos Nikolaidis. Video game level repair via mixed integer linear programming. In Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, volume 16, pp. 151-158, 2020.
14

---

Published in Transactions on Machine Learning Research (08/2025)
A Derivations
A.1 Proof of Theorem 3.1
Following the proof Ichikawa 2024 we show Theorem 5.1 based on following three lemmas.
Lemma A.1. For any even natural number 2, ...,, the function @(p) (2p 1) defined on [0. 1] achieves its masimum value of 1 when p 1/2 and its minimum value of 0 when p = 0 p 1.
Proof. The derivative of d(p) relative to p is ('()) 22 1o02p-1), 1), which is zero when p 1/2. This is a point
where the function is maximized because the second derivative ("(p) -4 O. In addition, this function is concave and symmetric relative to p 1/2 because a is thereby achieving its minimum value of O when 0 or p=1.
natural number, ie., @(p) = o(1 p):
.,.. and a matria P 00, 1j iʃ to0, mini- mizing the penalty term (PP:7) = Σ Ei=1 (1 (2Pis - 1)a) = 7 Σ. s= Σ ( enforces that the components of Pis must be either or and, if ? + -00, the penalty term enforces P 1/1
Lemma A.2. For any even natural numberr
Proof. From Lemma A.1 as 7 + +00, the case where d(Pi.) becomes minimal occurs when, for each i, 8, Pie = or P. 1. In addition, as 7 + -00, the case where o(p:7) minimized occurs when, for each i, P.e reaches its maximum value with P:s = 1/2.
Lemma A.3. -(P:7)=73)) En (1 (2p: 12p--199) 1)a) E.E E o(p;a) is concove when X posilive and is conver function when is negative.
Proof. Note that ;(P:))=>>>) = E DN DN d(Pis;)) = > (2P;, - 1)a) separable across its com- ponents P... Thus, it is sufficient prove that each r0(Pis a) is concave or convex in Pt. because the sum of the concave convex functions is also concave (and vice versa). Therefore, consider the second derivative of yo, with respect to P.e:
d or (P.:ia) = -4ya
dP?
Here, if > 0, the second derivative negative for all ps E [0, 1], and this completes the proof that &(P: is a concave function when y positive over the domain p [0, 1|
Theorem A.4. Under the assumption that the objective function Σ. i(Prs C., X.) is bounded within the domain [0, N for any S E N, C. E Cs and X E As, as 7 to0, each column P of the soft converges to the original solutions x" argmin C., In
solutions Pa E R(P addition, + -00, the loss function ((P:Cs,As) becomes conver and the soft solution 1s1n/2 = argmin (((PCss,As,) is unique.
Proof. As X + +o,, the penalty term 4(P;A) dominates the loss function A(p: C, X. 7). According Lemma A.2 this penalty term forces the optimal solution P to have components pl. that either or 1 because any nonbinary value will result in infinitely large penalty. This effectively restricts the feasible region to the vertices of the unit hypercube, which correspond to the binary vector in 1) N.S, Thus, as X + +00, the solutions to the relaxed problem converge to X argminxe(o.ppe xe(o,)xxxxp R(X.a: C., A.). Futhermore, argmin 1ox C., Aa) is separable as E argmin (0.1) (=[; C., which indicate that each columns X argminae(o.jjv C,, Ag). As ー -0, the ponalty term (pp;a) also dominates the loss function f(p; C, 7) and the f(p: C) convex function from Lemma A.3. According Lemma A.2 this penalty term forces the optimal solution p== 1x/2.
15

---

Published in Transactions on Machine Learning Research (08/2025)
A.2 Proof of Proposition 5.2
In this section, we derive the following Proposition. Proposition A.5. For binary sequences 1 Vs, I. [0, 10, following equality holds
2 VAR (aasnhsess d (x
where the right-hand side of Eq. the max-sum Hamming distance.
Proof. We first note that, for binary vectors Es 21 E {0, 1) the Hamming distance is expressed as follows:
N
dh (2,, x1) = => (z2, + uř, - 221,,0115).
Based on this expression, the diversity metric Exci dn (X:s, X,)) can be expanded for binary matrix X e {0, 1j as follows:
Σ 851
dn (X X2) =
Σ
dn( X.1) -x) Ex Xxx)
2
8.1
1 N
(x + +X3, i.....
2 1=1
8,
N
s
>
Σ x2 Σ
X
a.l
On the other hand, the variance of each column in a binary matrix X can be expanded follows:
s
2
S2
VAR VAR MA|liiil = 5
X
3 Σ X.sd
i=1 s'=1
N S
Exxist
Eas X1x
= S
2x'''
+
S
S2
1= x=1
2
X
Es.1 N
= Σ
X2 '
S
S Σ
Σ
X2
X2
+
Σ
XistXi1
5
al
Σ
(X::; X,1).
By this, we finish the proof.
B Additional Implementation Details
B.1 Graph Neural Networks
A graph neural network (GNN) Giimmer et al. 2017 tation learning of graph-structured data. GNNs learn a vectorial representation of each node through two steps. (T) Aggregate step: This step employs permutationniivariant function to generate aggregated
Scarselli et al.) 2008 specialized NN for represen-
16

---

Published in Transactions on Machine Learning Research (08/2025)
node feature. (II) Combine step: Subsequently, the aggregated node feature is passed through trainable layer to generate a node embedding, known as 'message passing' or 'readout phase. Formally, for given graph G = (V, E), where each node feature h. E Rxd is attached to each node v E V, the GNN iteratively updates the following two steps. First, the aggregate step at each k-th layer is defined by
a Aggregate & (fht Vu E N.))..
where the neighorhood of E V denoted Nr {u E V (v, EJ, h is the node feature of neighborhood, and a is the aggregated node feature of the neighborhood. Second, the combined step each k-th layer is defined by
h = Combine (h", k- ,,,))
where h E R denotes the node representation at k-th layer. The total number of layers, K, and the intermediate vector dimension, N* are empirically determined hyperparameters. Although numerous im- plementations for GNN architectures have been proposed, the most basic and widely used GNN architecture is a graph convolutional network (GCN) (Scarselli et al. 2008 given by
h + BBhh
h
W uEN(v)
[N())
where W and B are trainable parametors, [W('] serves as normalization factor, and a R - R is some component-wise noulinear activation function such as sigmoid or ReLU function.
C Experiment Details
This section describes the details of the experiments.
C.1 Architecture of GNNs
We describe the details of the GNN architectures used in our numerical experiments. For each node v E V, the first convolutional layer takes a node embedding vectors, h" for cach node, yiclding feature vectors hhe E Rh Then, the ReLU function is used as component-wer wise nonlinear transformation. The second convolutional layer takes the fcature vector, hg input, producing feature vector h E RRS. Finally, sigmoid function applied to the vector hg producing the higher-order array solutions Pu.0 0 E [0, NxS Here, for MIS and MaxCut problems, we set |Hol int(N0.8) as in Schuctz al. (2022a)): Ichikawa (2024) and for the DBM problems, we set 2,500. Across all problems, we set H1 Ho, and H₂ S. We conducted all experiments by using V100GPU.
C.2 Training setting and post-nounding method
We use the AdamW (Kingma & Ba 2014) optimizer with a learning rate as = 10 and weight decay 10-2 The training the GNNs conducted for duration of up to x 10 epochs with carly stopping, which monitors the summarized loss function Σ i(P...) and penalty term ((P;7,a) with tolerance 10-5 and patience 103. After the training phase, we apply projection heuristics to round the obtained soft solutions back to discrete solutions using simple projection, where for all i [N], s E [S], we map Pg.i, to 0 if Po.s. S 0.5 and Po... to 1 if Po.s. > 0.5. Note that due to the annealing, CPRA-PI-GNN solver ensures that the soft solution are nearly binary for all benchmarks, making them robust against the threshold 0.5 our experiments.
C.3 Problem specification
Maximum independent set problems There are some thcoretical results for MIS problems on RRGs with the node degree to d, where each node is connected exactly d other nodes. The MIS problem is fundamental NP-hard problem (Karp) 2010) defined follows. Given an undirected graph G(V, F),,
17

---

Published in Transactions on Machine Learning Research (08/2025)
Table 1: The objective functions for the three problems to be studied.
Objective Function
Parameters
MIS
1(x; G. A) = - Σiev Is + E ((J))E I,2z
[V] = N
MaxCut
1(x; G) Eiej (20,3 - 24 -I))
A RNN
DBM
1(:; C = (,,M)))) Eoi Aijinig +h E. ReLUUE Fij 11
A E R M E RNXNN P
+>2 L; ReLURei -1 +AgReLU(p Lng Tij Ei, Mis
+X,ReLU(qq Eis
EisTiy Fij - Ex MMy)
independent set (TS) is subset of nodes I V where any two nodes in the set not adjacent. The MTS problem attempts to find the largest IS, which is denoted I In this study, p denotes the IS density, where / | V.. To formulate the problem, binary variable z; is assigned to each node E V. Then the MIS problem is formulated as follows:
f(E; = Σ + VVV
Σ Z,IJ, (KJ))E
where the first term attempts to maximize the number of nodes assigned 1, and the second term penalizes the adjacent nodes marked 1 according to the penalty parameter X. In our numerical experiments, we set À = 2, following Schuetz et al. (2022a), no violation is observed as in (Schuetz et al. 2022a). First, for every d, a specific value Ph: which is dependent on only the degree d, exists such that the independent set density [ |/V| converges to P3 with a high probability as N approaches infinity (Bayati et al., 2010). Second, a statistical mechanical analysis provides the typical MIS density Pa Theory and we clarify that for d >> 16, the solution space of I undergoes a clustering transition, which associated with hardness in sampling (Barbier) let al., 2013) because the clustering is likely to create relevant barriers that affect any algorithm searching for the MIS Z Finally, the hardness supported by analytical results in large d limit, which indicates that, while the maximum independent set density is known have density Po log(d)/d, /d, to the best of our knowledge, there is no known algorithm that can find an independent set density exceeding Pa = log(d)/d (Coja-Oghlan & Efthymiou 2015).
Diverse bipartite matching (DBM) problems We adopt this CO problem from Ferber et al. 2020 Mulamba et al. (2020); Mandi et al. (2022) as a practical example. The topologies are sourced from the CORA citation network (Sen et al. 2008), where each node signifying a scientific publication, characterized by 1,433 bag-of-words fcatures, and the edges represents represents the likelihood of citation links. Mandi et al. (2022) focused on disjoint. topologies, creating 27 distinct instances. Each instance is composed of 100 nodes, categorised into two group of 50 nodes, labeled N₁ and N2. The objective of DBM problems is to find the maximum matching under diversity constraints for similar and different fields. It is formulated follows:
Ni Σ ReLU
No Tij - 1) + A2
N₂
Ni ReLU Tzj - 1)
((:; C, = - Ciyry + A1
Σ 1=1
Σ 1=1
1=1
1=1
+ AgRRe.U
( Rij Σ Mijais) + 2aReLU
(
Faj > - -Mis)ars) ): i5
where a reward matrix C E N:N, indicates the likelihood of a link between each node pair, for all i,
Mg is assigned 0 if articles and j belong to the same field, or 1 if they don't. The parameters p. q E [0
represent the probability of pairs being in the same field and in different fields, respectively. Following (Mandi et al.) 2022). we examine two variations of this problem: Matching-1 and Matching-2, characterized by p and q values of 25% and 5%.
18

---

Published in Transactions on Machine Learning Research (08/2025)
MIS 10000)
10
MIS (N= 10000, d=
MIS (V= 10000, d=
CRA-PI-GNN
GRA PI CPRAAPS-ON
Number of GNN parameters
CPRA-PL-NNN
1.7
Timto 60
(4)
1.0
300
127
100
10
Number of shots
Number of shuts
Figure 7: The runtime of the CPRA-PI-GNN solver, compared to S individual runs of the CRA-PI-GNN solver, as function of number of shots S. Error bars represent the standard deviations of 5 random seeds.
Maximum cut problems The MaxCut problem, a well-known NP hard problems (Karp) 2010) has practical application machine scheduling (Alidace al. 1994), image recognition (Neven et al. 2008 and electronic circuit layout dcsign (Deza & Laurent, 1994). is defined as follows: In an undirccted graph G (V, E), & cul. set C E, which subset of edges, divides the nodes into two groups (Vi, V2 Vi UV2 V, Vinv== = 0). The objective the MaxCut problem is to find the largest cut set. To formulate this problem, each node is assigned a binary variable: 24 = 1 signifies that node is in V1, while 0 indicates node is in 1/2. For an edge (i,j), + I 2r, = 1 is true if (i,j) E C; otherwise, it equal 0. This leads to the
following objective function:
1(x; G) = Σ
Aig (2xraz - xx,)
where Ay is the adjacency matrix, where A. = signifies the absence an edge, and Ay > 0 indicates a connecting edge. Following Schuetz al. (2022a): Ichikawa (2024). this experiments employ seven instances from Gset dataset Yee| 2003), recognized as standard MaxCut benchmark. These seven instances are de- fined on distinct graphs, including Erdös-Renyi graphs with uniform edge probability, graphs with gradually decaying connectivity from 1 to N, 4-regular toroidal graphs, and one of the largest instance with 10,000 nodes.
Additional Experiments
D.1 Runtime and # Params as a function of number of shots
In this section, we investigate the runtime of CPRA-PI-GNN solver function of the number of shots, S, compared to the runtime for S individual runs of CRA-PI-GNN solver. Fig. 7 shows each runtime as a function of the number of shots S. For this analysis, we incrementally increase the number of shots, further dividing the range of penalty parameters from 2-2 to 217, The results indicate that CPRA-PI-GNN solver can find penalty-diversifeed solutions within a runtime nearly identical to that of a single run of CRA-PI- GNN solver for shot numbers S from 20 200 However, for S 102, we observe a linear increase in runtime the number of shots S grows because of the limitation of memory of GPUs. Fig. shows the distribution of Hamming distances combination, {du (P.., 1 and the count of unique solutions with different v = 0.00, 0.05, 0.10, 0.20, whereas Fig. (right) shows the maximum ApR, i.c., max (P..)...) as a function of the parameter и. These results indicate that the CPRA-PI-GNN solver can find more variation-diversfficd solutions as the parameter v increasos. Furthermore, This result indicates that. the CPRA-PI-GNY solver can boost the exploration capabilities of the CRA-PI-GNN solver, leading the discovery of better solutions.
(right)
D.2 Additional results of variation-diversiin solutions for MIS
Fig. g (right) shows the distribution of Hamming distances combination, (du Pi)sssssssssss and the
count of unique solutions with different 0.00, 0.05, 0.10, 0.20, whereas Fig. 8 (right) shows the maximum
19

---

Published in Transactions on Machine Learning Research (08/2025)
v=0,0, #unique solution 2
0.94
250
0.0
0.1
0.2
0.3
0.93
== 0.05, #u #unique solution
3
100
Max ApR
0.92
0.0
0.1
0.2
0.3
0.91
== 0.10, # unique solution
9
100
0.90
0.0
0.1
0.2
0.3
== 0.20, # #unique solution 241
0.89
25
0.0
0.88
0.1 Hamming distance
0.2
0.3
0.0
0.0
0.1
0.2
0.3
0.4
V
Figure 8: The density of Hamming distance combination of the solution, (du(Pis; Pr))1sssis300 with differ- ent parameters y and the count of unique solutions (left), and the maximum ApR, max,=1.... 300 ApR(P: function of the parameter v.
0=2.22 #unique solution
0.4. #unique solution
pm0.6, #unique solution
p=0.8, #unique solution 35
30
12.59
10.0
7.5
4
5.0
10
2
2.5
000
0.900
6.2
0.8
9.0
02
0.0
Hamming distance
Hamming distanoe
Hamming distance
Hamming distance
Figure 9: The density of Hamming distance combination of the solution, (d P:)lisecs1000 in MaxCut G14, with different parameters and the count of unique solutions
ApR, i.e, max.=1.... ApR(P.,s) as a function of the parameter v. These results indicate that the CPRA- PI-GNN solver can find more variation-diversified solutions as the parameter v increases. Furthermore, This result indicates that the CPRA-PI-GNN solver can boost the exploration capabilities of the CRA-PI-GNN solver, leading the discovery of better solutions.
D.3 Additional results of variation-diversified solutions for MaxCut G14.
In this section, to supplement the results of the variation-diversific solutions for MaxCut G14 in Section 6.3 present the results of the Hamming distance distribution. Fig. sshows the distribution of combinations of solution Hamming distances under the same settings as in Section 6.1 From these results, it is evident that the CPRA-PI-GNN solver has acquired solutions in four distinct clusters.
D.4 Additional results for validation of exploration ability
These improvement consistent across other Gset instances on distict graphs with varying nodes, as shown in Table. 2 In these experiment, we fix as 6 and evaluate the maximum ApR, max;=11....900 ApR This result. shows that CPRA-PI-GNN solver outperforme CRA-PI-GNN, PI-GNN, and RUN-CSP solvers.
20

---

Published in Transactions on Machine Learning Research (08/2025)
Table 2: Numerical results for MaxCut on Gset instances
(NoDES, EDGES)
CSP
PI
CRA
CPRA
G14 (800, 4,694)
0.960
0.988
0.994
0.997
G15 (800, 4,661)
0.960
0.980
0.992
0.995
G22 (2,000, 19,990)
0.975
0.987
0.998
0.999
G49 (3,000, 6,000)
1.000
0.986
1.000
1.000
G50 (3,000, 6,000)
1.000
0.990
1.000
1.000
G55 (5,000, 12,468)
0.982
0.983
0.991
0.994
G70 (10,000, 9,999)
0.982
0.992
0.997
1.00
ApR
0.98
CPA-PI-GNN 7 8 2tt.:/21111111111111111111222221
0.96
CPRA-PI-GNN
1 2
6
Instance ID
Figure 10: The ApR of DBM (Matching-1) using CPRA-PI-GNN and CRA-PI-solvers
(Ichikawa 2024
D.5 CPRA for Multi-instance Solutions
In this section, numerically demonstrate that the CPRA-PI-GNN solver can elficiently solve multiple problems with similar structures. The numerical experiments solve all 27 DBM instances using the CPRA- PI-GNN solver with the following loss function:
A(0:Cs,A,T)= = Σ (.) S(0:cs S(0,Cs,7,o) A
M (1 (2.o.i - 1)
where Cs (C,,, M represents the instance parameters, and i is defined follows:
C,,C,,M,)) = - Cijaay + d ReLU EROLU(E rij 11 A2 EReLU
ReLU Σ say --y-11
+ AaReLU (p) Σ Dij Σ Mojoys + X+RoLU
Kaj E1 (11-M1)aa Mygle.)
where À is fixed as À (A1,)22 A4) (2 12, 12). The parameters for the CPRA-PI-GNN solver is set the same as in Section 6.1 On the other hand, the CRA-PI-GNN solver repeatedly solve the 27 problems using the same sottings as Ichikawa (2024). As result, the CPRA-PI-GNN solver can explore global optimal solutions for all problems. Fig. 10 showcases the solutions yielded by both the CRA-PI-GNN and CPRA-PI-GNY solvers for the 27 Matching-I instances. Matching-2 is excluded from this comparison, given that both solvers achieved global solutions for these instances. The CRA-PI-GNN solver, applied 27 times for Matching-1, accunulated a total runtime of 36,925 + 445 seconds, significantly longer than the CPRA-PI-GNNY efficient 5,617+220 seconds. For Matching-2, the CRA-PI-GNN solver required 36.816+149
21

---

Published in Transactions on Machine Learning Research (08/2025)
Table 3: Runtime and solution quality on the 500-task, 1000-node weighted-MIS benchmark. "Pre" and "FT" denote warm-up and fine-tune phases of the two-stage schedule. The two-stage variant not only faster than scratch training but also yields markedly higher total weight.
Method
Pre [s]
FT [s]
Total [s]
Avg. weight
Two-Stage (200 - -00) Scratch-500
43.5
30.4
73.9
139.87
83.7
123.87
seconds, whereas the CPRA-PI-GNN solver completed its tasks in just 2,907 + 19 seconds. The reported errors correspond to the standard deviation from five random soeds. These findings not only highlight the CPRA-PI-GNN solver's superior efficiency in solving a multitude of problems but also its ability to achieve higher Acceptance Probability Ratios (ApR) compared the CRA-PI-GNN solver. The consistency of these advantages across different problem types warrants further investigation.
D.6 Additional Results of devesty-diversffied solutions for DBM problems
This soction extends our discussion on penalty-diversifeed solutions for DBM problems, as introduced Section 6.2 In these numerical experiments, used the same As as in Section 6.2 and executed the CPRA- PI-GNN under the same settings in Section 6.1 As shown in Fig. 11 the CPRA-PI-GNN acquire penalty-diversifedd solutions for all instances of the DBM.
E Two-Stage Extension Experiment
To validate the claim in Scction that small warm-up subset S' C S allows CPRA-PI-GNN to scale efficiently, solve 500 weighted-MIS tasks on fixed 1000-node, d= 5 regular graph. The first. 200 weights form S' (warm-up); all 500 form S. During warm-up the network has 200 heads and is trained for 8000 epochs with À annealed 22 - +2, but the update stops once the entropy term falls below 10-4, The encoder is then frozen, the head replaced by a fresh 500-way layer, and only that layer is optimised for 2000 epochs (A 0.:0..100 0, same stopping rule). scratch-trained 500-head model serves as baselines.
As shown in Table two-Stage finishes 11% faster than scratch yet attains 113% of the scratch objective and 143% of greedy. This confirms that learning shared encoder on S' and adapting only the output layer is both faster and better than training large head from scratch on all tasks.
22

---

Published in Transactions on Machine Learning Research (08/2025)
Figure 11: ApR of the DBM problems on th grid As using CPRA-PI-GNN solver. Each Point on the coordinate plane represents the results from five different random seed, with the colors indicating the ApR. The constraints violation are marked with a cross symbol.
23


Reviews:
[{"review_id": "sRny1jA2jW", "reviewer": "Reviewer_6B4C", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**\n1. The paper addresses the important real-world need for diverse solutions in combinatorial optimization. \n2. It proposes a computationally efficient method for generating S diverse solutions in a single training run through output parallelization.\n3. The paper is well-structured and logically presented.\n\n**Weaknesses**\n1. The proposed \"two-stage learning process\" (Section 5.1) for scaling to large S lacks experimental validation and further detail. \n2. Lacks comparison with representative traditional (non-neural) combinatorial optimization methods. \n3. No practical guidance is provided on selecting an appropriate $\\lambda$ value, despite experimental results showing its impact. \n4. Inconsistent naming of the method (CPRA vs. CTRA) causes confusion.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper proposes CPRA, a framework for generating diverse solutions in Combinatorial Optimization (CO) efficiently. CPRA extends UL-based solvers to produce multiple solutions in parallel, improving efficiency and quality. It introduces a diversity penalty to reduce computational complexity, outperforming baseline methods in experiments.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. Please address the weaknesses mentioned.\n2. The paper would benefit from including more discussions on some Pareto set learning works, as they are closely related (though not addressing the same problem). For example, [1] and [2]. Additionally, see the survey [3] for more relevant works.\n\n[1] Navon and Shamsian et al., Learning the Pareto Front with Hypernetworks, ICLR 2021\n\n[2] Lin et al., Pareto Set Learning for Neural Multi-Objective Combinatorial Optimization, ICLR 2022.\n\n[3] Chen et al. Gradient-Based Multi-Objective Deep Learning: Algorithms, Theories, Applications, and Beyond, arXiv 2025.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "uE8aWDbUa6", "reviewer": "Reviewer_E2vj", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths:**\n\n+ The paper addresses a highly relevant and underexplored need for *diverse* solutions in CO.\n+ The proposed CTRA framework is computationally efficient and well-suited for parallel computation on GPUs.\n\n**Weaknesses:**\n\n− Motivation needs refinement. While the general need for diversified solutions is explained, the connection to real-world tasks or the significance of improvement over prior solvers could be better emphasized.\n\n− The novelty is somewhat limited in terms of technical mechanisms. The main architectural difference lies in output dimensionality and extended training across multiple instances. Although the idea is practically useful, it may be viewed as a natural extension of CRA-PI-GNN and PI-GNN.\n\n− The notion of “tensor” in the title may mislead readers expecting tensor decomposition techniques or tensor network structures. The current usage mainly refers to a matrix or batch of solutions.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper targets a challenging and practical problem in combinatorial optimization (CO): generating diverse solutions rather than a single optimum, which is essential in scenarios involving soft constraints or approximate formulations. The authors propose a learning-based framework called Continuous Tensor Relaxation Annealing (CTRA), which extends prior UL-based solvers by incorporating parallelized representation learning over multiple instances. The framework efficiently generates:\n\n- Penalty-diversified solutions via different penalty configurations,\n- Variation-diversified solutions through a novel diversity-encouraging regularization.\n\nThe authors provide theoretical insights, clear problem formulations, and extensive numerical experiments across multiple CO tasks (MIS, MaxCut, DBM) to validate the efficiency and effectiveness of CTRA.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. Clarify the use of the term “tensor” in the title and paper. Since the method deals with a matrix P \\in [0,1]^{N \\times S} or possibly higher-order arrays, but  isn’t deeply involved with  multilinear algebra, the usage may cause confusion.\n2. Highlight the main contributions more clearly in the introduction or conclusion. If the architectural novelty is limited, emphasize empirical significance or generalization ability as a key takeaway. For example, point out the real-world importance of efficiently exploring diverse solutions in large-scale or high-stakes settings.\n3. Address the flexibility in inference: In Section 3.1, the loss is minimized over S instances with shared architecture. However, what happens if the number of instances S during inference differs from training? Can the model generalize to unseen S? Clarification would be valuable.\n4. Clarify Proposition 3.2’s computational benefit: The proposition claims that the proposed diversity loss is more efficient than directly computing the max-sum Hamming distance. Please elaborate on detais why the computational complexity is different.", "broader_impact_concerns": "None", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "IAFajDByI8", "reviewer": "Reviewer_T6JK", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "#### Strengths\n* This paper designed the idea of extending tensor dimension and leverage the GPU-based parallelization.\n* The additional diversity penalty helps search for better solutions. This is interesting and makes sense since it discovers diverse local minima, but at a higher cost.\n\n#### Weakness\n* The main contribution is extending the output tensor from $[n]$ to $[s, n]$, which seems like a very minor trick. Moreover, the argument of comparable number of parameters depends on the GNNs. If we directly optimize $P$ without GNN re-parameterization, then $[n]\\rightarrow [s,n]$ leads to $s$ times complexity. Only deep GNNs could amortize the $(s-1)n$ variables, but is it necessary?\n* **Benchmarks**: In the first scenario (penalty-diversified), the importance of diversity is based on the assumption that the trade-off between objectives and penalties is difficult. However, it seems that such trade-off is easy for the MIS problems since most of the 20 penalty values result in zero violation.\n* **Baselines**: The comparison is not clear and important baselines are missing. Is the goal to find multiple diverse solutions or to find an optimal one? If the former, then the baselines are not strong enough since they are designed for single solution. Tailored approaches for multiple solutions (e.g. [1]) should be considered; If the latter, then there are plenty of advanced ML-based solvers for COs.\n\n[1] Li L, Aigerman N, Kim V, et al. Learning Proximal Operators to Discover Multiple Optima[C]//The Eleventh International Conference on Learning Representations.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper focused on identifying diverse solutions to combinatorial optimization (CO) problems and typically aiming at scenarios where (i) hard constraints of CO are integrated into the objective functions as penalty terms with their weights being crucial and (ii) real-world cases where diversified solutions are preferable. Technically, the proposed CTRA generalized the routine of PI-GNN by extending the output from an $[n]$ tensor to a $[s, n]$ tensor, representing $s$ solutions, and tailored loss functions are applied to the two scenarios. Experiments showed that CTRA achieved comparable results to the single output version but without computing $s$ times for each value of the penalty weight, and CTRA could even improve search capabilities by considering an additional diversity penalty.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Please see **Weaknesses**.", "broader_impact_concerns": "NA.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)