Unsupervised Learning for Combinatorial Optimization with Principled Objective Relaxation
Abstract
Using machine learning to solve combinatorial optimization (CO) problems is challenging, especially when the data is unlabeled.
This work proposes an unsupervised learning framework for CO problems.
Our framework follows a standard relaxation-plus-rounding approach and adopts neural networks to parameterize the relaxed solutions so that simple back-propagation can train the model end-toend.
Our key contribution is the observation that if the relaxed objective satisfies entry-wise concavity, a low optimization loss guarantees the quality of the final integral solutions.
This observation significantly broadens the applicability of the previous framework inspired by Erd ˝ os’ probabilistic method [1].
In particular, this observation can guide the design of objective models in applications where the objectives are not given explicitly while requiring being modeled in prior.
We evaluate our framework by solving a synthetic graph optimization problem, and two real-world applications including resource allocation in circuit design and approximate computing.
Our framework 1 largely outperforms the baselines based on naïve relaxation, reinforcement learning, and Gumbel-softmax tricks.
Introduction
Combinatorial optimization (CO) with the goal of finding the optimal solution from a discrete space is a fundamental problem in many scientific and engineering applications [2–4].
Most CO problems are
NP-complete.
Traditional methods efficient in practice often either depend on heuristics or produce approximation solutions.
Designing these approaches requires considerable insights into the problem.
Recently machine learning has paved a new way to develop CO algorithms, which asks to use neural networks (NNs) to extract heuristics from the data [5–7].
Several learning for CO (LCO) approaches have therefore been developed, providing solvers for the problems including SAT [8–10], mixed integer linear programming [11–13], vertex covering [14,15] and routing problems [16–23].
Another promising way to use machine learning techniques is to learn proxies of the CO objectives whose evaluation could be expensive and time-consuming [24–27].
For example, to optimize hardware/system design, evaluating the objectives such as computation latency, power efficiency [28], and resource consumption [29–31] requires running complex simulators.
Also, in molecule design, evaluating the objective properties such as protein fluorescence or DNA binding affinity needs either costly simulations or living experiments [32–34].
In these cases, proxies of the objectives can be learned first to reduce the evaluation cost [31], and then optimize these proxies to solve the design problems.
Later, we name the CO problem with proxy objectives as Proxy-based CO (PCO) problems.
[Figure]
Example: Our Principle:
[Figure]
The landscapes of the relaxed
[Figure]
Moreover, we claim that learning for PCO is often of greater significance than learning for traditional
CO problems because commercial CO solvers such as Gurobi cannot applied in PCO due to the missing closed-form objectives and the lack of heuristics for such proxy objectives.
Generic solvers such as simulated annealing [35] may be applied while they are extremely slow.
In this work we propose an unsupervised LCO framework.
Our findings are applied to general
CO problems while exhibiting extraordinary promise for PCO problems.
Unsupervised LCO has recently attracted great attentions [1,9,20,36,37] due to its several great advantages.
Traditional supervised learning often gets criticized for the dependence on huge amounts of labeled data [38].
Reinforcement learning (RL) on the other hand suffers from notoriously unstable training [39].
In contrast, unsupervised LCO is superior in its faster training, good generalization, and strong capability of dealing with large-scale problems [1].
Moreover, unsupervised learning has never been systematically investigated for PCO problems.
Previous works for PCO problems, e.g., hardware design [30, 31], were all based on RL.
A systematic unsupervised learning framework for PCO problems is still under development.
Our framework follows a relaxation-plus-rounding approach.
We optimize a carefully-designed continuous relaxation of the cost model (penalized with constraints if any) and obtain a soft solution.
Then, we decode the soft assignment to have the final discrete solution.
This follows a common approach to design an approximation algorithm [40,41].
However, the soft assignment here is given by a NN model optimized based on the historical (unlabeled) data via gradient descent.
Note that learning from historical data is expected to facilitate the model understanding the data distribution, which helps with extracting heuristics, avoiding local minima and achieving fast inference.
An illustration of our principle with a toy-example is shown in Fig.
We also provide the pipeline to empirically evaluate our relaxation-plus-rounding principle in Fig.
Our method shares a similar spirit with [1] while making the following significant contributions.
We abandon the probabilistic guarantee in [1], because it is hard to use for general CO objectives, especially those based on proxies.
Instead, we design a deterministic objective relaxation principle that gives performance guarantee.
We prove that if the objective relaxation is entry-wise concave w.r.t.
the binary optimization variables, a low-cost soft solution plus deterministic sequential decoding guarantees generating a valid and low-cost integral solution.
This principle significantly broadens the applicability of this unsupervised learning framework.
In particular, it guides the design of model architectures to learn the objectives in PCO problems.
We also justify the wide applicability of the entry-wise concave principle in both theory and practice.
We evaluate our framework over three PCO applications including feature-based edge covering & node matching problems, and two real-world applications, including imprecise functional unit assignment in approximate computing (AxC) [42–46] and resource allocation in circuit design [24,30].
In all three applications, our framework achieves a significant performance boost compared to previous
RL-based approaches and relaxed gradient approaches based on the Gumbel-softmax trick [47–49].
The datasets for these applications are also provided in our github repository.
1.1 Further Discussion on Related Works
Most previous LCO approaches are based on RL [13,14,16–18,22,50–52] or supervised learning [11, 12, 38], as these two frameworks do not hold any special requirements on the formulation of CO problems.
However, they often suffer from the issues of training instability and subpar generalization.
Previous works on unsupervised learning for CO have studied satisfaction problems [9, 36, 53].
Applying these approaches to general CO problems requires problem reductions.
Others have considered max-cut [37] and TSP problems [20], while these works depend on carefully selected problem-specific objectives.
The work most relevant to ours is [1] and we give detailed comparison in Sec. 3.
Note that all previous works on unsupervised learning for CO do not apply to PCO as they need an explicit objective to manipulate.
For PCO problems, previous studies focus on how to learn more generalizable proxies of the objectives, such as via Bayesian learning [54,55] and adverserial training [56,57].
Once proxies are learned, direct objective relaxation plus gradient descent [56], or
RL [30,31], or general MIP solvers with reformulations of the objectives [58,59] have been often adopted.
Studying generalization of proxies is out of the scope of this work while we conjecture that our suggested entry-wise concave proxies seem smoother than those without constraints (See Fig. 1) and thus have the potential to achieve better generalization.
Preliminaries and Problem Formulation
In this section, we define several useful concepts and notations.
Combinatorial Optimization (CO).
Let C ∈ C denote a data-based configuration such as a graph with weighted edges.
Let Ω be a finite set of all feasible combinatorial objects and each object has a binary vector embedding X
=
( X i ) 1 ≤ i ≤ n ∈ { 0 , 1 } n .
For example, in the node matching problem, each entry of X corresponds to an edge to denote whether this edge is selected or not.
Note that such binary embeddings are applicable even when the choice is not naturally binary:
Choosing at most one element from a tuple (1 , 2 , 3) can be represented as a 3-dim binary vector ( X 1 , X 2 , X 3 ) with the constraint X 1 + X 2 + X 3 ≤ 1 .
W.l.o.g, we assume an algebraic form of the feasible set Ω ≜ { X ∈ { 0 , 1 } n : g ( X ; C ) < 1 } where g ( X ; C ) ≥ 0 for all X ∈ { 0 , 1 } n 2 .
For notational simplicity, we only consider one inequality constraint while our later discussion in Sec.
and our case studies in Sec.
5 may contain multiple inequalities.
Given a configuration C and a constraint Ω , a combinatorial optimization (CO) is to minimize a cost f ( · ; C ) by solving
[Equation]
Proxy-based CO (PCO).
In the many applications, the cost or the constraint may not be cheaply evaluated.
Some proxies of the cost f or the constraint g often need to be learned from the historical data.
With some abuse of notations, we interchangably use f ( g , resp.) to denote the objective (the constraint, resp.) and its proxy.
Learning for CO/PCO (LCO).
A LCO problem is to learn an algorithm A θ ( · ) :
C → { 0 , 1 } n , say a neural network (NN) parameterized by θ to solve CO or PCO problems.
Given a configuration
C ∈ C , we expect A θ to (a) generate a valid solution ˆ X =
A θ ( C ) ∈ Ω and (b) minimize f ( ˆ X ; C ) .
There are different approaches to learn A θ .
Our focus is unsupervised learning approaches where given a configuration C , no ground-truth solution X ∗ is used during the training.
θ can only be optimized just based on the knowledge of the cost and the constraint, or their proxies.
Note that in this work, for PCO problems, the proxies are assumed to be trained based on ground-truth values of f ( X ; C ) given different ( X, C ) pairs, which is supervised.
Unsupervised learning in this work refers to the way to learn A θ ( C ) .
Erd˝os’ Probabilistic Method (EPM).
The EPM has recently been brought for LCO [1].
Specifically, The EPM formulates A θ ( C ) as a randomized algorithm that essentially gives a probabilistic distribution over the solution space { 0 , 1 } n , which solves the optimization problem: min θ
E [ l ( X ; C )] , where l ( X ; C ) ≜ f ( X ; C ) + β 1 g ( X ; C ) ≥ 1 , X ∼ A θ ( C ) and β > 0 .
Karalias & Loukas proved that with β > max X ∈ Ω f ( X ; C ) and a small expected loss E [ l ( X, C )] < β , sampling a sufficiently large number of ˆ X ∼ A θ ( C ) guarantees the existance of a feasible ˆ X ∈ Ω that achieves the cost f
( ˆ X ; C ) ≤ E [ l ( X, C )]
Although this guarantee makes EPM intriguing, applying EPM in practice is non-trivial.
We will explain the challenge in Sec. 3.1, which inspires our solutions and further guides the objective design for general CO and PCO problems.
The Relaxation Principle for Unsupervised LCO
In this section, we start with the practical issues of EPM.
Then, we introduce our solutions by proposing a relaxation principle of the objectives, which gives performance guarantee for general practical unsupervised LCO.
3.1 Motivation: The Practical Issues of EPM
Applying EPM in practice has two fundamental difficulties.
First, optimizing θ in Eq.(2) is generally hard as the gradient dXdθ (note here each entry of X is binary) may not exist so the chain rule cannot be used.
We discuss the potential solutions to this problem in Sec. 3.4.
Second, EPM needs to sample a large number of X ∼ A θ ( C ) for evaluation to achieve the performance guarantee in [1].
This is not acceptable where the evaluation per sample is time-consuming and expensive.
So, in practice, Karalias & Loukas consider a deterministic method.
They view A θ ( C ) ∈ [0 , 1] n as the parameters of Bernouli distributions to generate the entries of X so E
[ X ]
=
A θ ( C ) .
First, they optimize min θ l
( A θ ( C ) , C ) instead of min
θ E
[ l ( X, C )] , and then, sequentially round the probability
A θ ( C ) to discrete X ∈ { 0 , 1 } n by comparing conditional expectations, e.g., E [ l ( X, C ) | X 1 = 0] v.s. E [ l ( X, C ) |
X 1 = 1] to decide X 1 .
However, such conditional expectations cannot be efficiently computed unless one uses Monte-Carlo sampling or l has special structures as used in the two case studies in [1], i.e., max-clique and graph-partition problems.
However, what special structures are needed has not been defined, which blocks the applicability of this framework to general CO problems, especially for the PCO problems where the objectives l are learned as models.
3.2 Our Approach: Relaxation plus Rounding, and Performance Guarantee
Our solution does not use the probabilistic modeling but directly adopts a relaxation-plus-rounding approach.
We optimize a relaxation of the objective l r and obtain a soft solution ¯ X ∈ [0 , 1] n .
Then, we deterministically round the entries in ¯ X to a solution X in the discrete space { 0 , 1 } n .
Note that throughout the paper, we use ¯ X to denote a soft solution and X to denote a discrete solution.
The question is whether the obtained solution may still achieve the guarantee as EPM does.
Our key observation is that such success essentially depends on how to relax the objective l .
Therefore, our first contribution beyond [1] is to propose the principle (Def. 2) to relax general costs and constraints.
With this principle, the unsupervised LCO framework can deterministically yield valid and low-cost solutions (Thm. 1) as the EPM guarantees, and is applied to any objective l .
First, we introduce the pipeline.
Consider a relaxation of a deterministic upper bound of Eq.(2):
[Equation]
+
[Equation]
( ¯ X ; C ) , where ¯ X =
A θ ( C ) ∈ [0 , 1] n , β > 0 .
Here f r ( · ; C ) : [0 , 1] n → R is the relaxation of f ( · ; C ) , which satisfies f r ( X ; C )
=
f
( X ; C ) for
X ∈ { 0 , 1 } n .
The relation between the constraint g and its relaxation g r is similar, i.e., g r ( X ; C )
=
g ( X ; C ) for X ∈ { 0 , 1 } n .
Here, we also use the fact that g r ( X ; C ) provides a natural upper bound 1 g ( X ; C ) ≥ 1 ≤ g r ( X ; C ) for X ∈ { 0 , 1 } n given the normalization of g ( X ; C ) adopted in Sec. 2.
Now, suppose the parameter θ gets optimized so that l r ( θ ; C ) is small.
Further, we adopt the sequential rounding in Def. 1 to adjust the continuous solution ¯ X
=
A θ ( C ) to discrete solution X .
Definition 1 (Rounding) .
Given a continuous vector ¯ X ∈ [0 , 1] n and an arbitrary order of the entries, w.o.l.g., i = 1 , 2 , ..., n , round ¯ X i into 0 or 1 and fix all the other variables un-changed.
Set
X i = arg
min j =0 , 1 f r ( X 1 , ...,
X i − 1 , j, ¯ X i +1 , ..., ¯ X n ; C )
+ βg r
( X 1 , ..., X i − 1 , j, ¯ X
i +1 , ..., ¯ X n ; C ) , replace ¯
X
i with X i and repeat the above procedure until all the variables become discrete.
Note that our rounding procedure does not need to evaluate any conditional expectations
E [ l ( X ; C ) | X 1 ] which EPM in [1] requires.
Instead, we ask both relaxations f r and g r to satisfy the principle in Def. 2.
With this principle, the pipeline allows achieving a valid and low-cost solution X , as proved in Theorem 1.
We leave the proof in Appendix A.1.
Definition 2 (The Entry-wise Concave Principle) .
For any C ∈ C , h r ( · ; C ) : [0 , 1] n → R is entry-wise concave if for any γ ∈ [0 , 1] and any ¯ X, ¯ X ′ ∈ [0 , 1] n that are only different in one entry,
[Equation]
Note that entry-wise concavity is much weaker than concavity.
For example, the function
[Equation]
is entry-wise concave but not concave.
Theorem 1 (Performance Guarantee) .
Let β > max X ∈ Ω f
( X ; C ) and min X ∈ Ω f
( X ; C ) ≥ 0 in
Eq.
Suppose the relaxed cost f r and constraint g r are entry-wise concave, and the learned parameter θ achieves l r ( θ ; C ) < β .
Then, rounding (Def. 1) the relaxed solution ¯ X =
A θ ( C ) generates a valid discrete solution X ∈ Ω such that f ( X ; C ) <
l r ( θ ; C ) .
When there are multiple constraints g ( j ) ( X ; C ) < 1 for j = 1 , 2 , ... , we may use relaxation β (cid:80) j g ( j ) r ( X ; C ) as the penalty term in Eq.(3), where g ( j ) r is a relaxation of g ( j ) .
It can be shown that if (cid:80) j g ( j ) r satisfies the condition of entry-wise concavity, the guarantee of Thm. 1 still applies.
3.3 The Wide Applicability of Entry-wise Concave Relaxations
We have introduced the entry-wise concave principle to relax the objective to associate our framework with performance guarantee.
The question is how widely applicable this principle could be.
Actually, every function with binary inputs can be relaxed as an entry-wise affine (also called multilinear) function with the exactly same values at the discrete inputs, as shown in Theorem 2.
Note that entry-wise affinity is a special case of entry-wise concavity.
In Sec. 5, we will provide the design of
NN architecture (for PCO) and math derivation (for CO) that guarantee formulating an entry-wise concave function.
Note that the objectives for max-clique and graph-partition problems used in [1] are essentially entry-wise affine.
Theorem 2 (Wide Applicability) .
For any binary-input function h ( · ) : { 0 , 1 } n → R , there exists a relaxation h r ( · ) :
[0 , 1] n → R such that (a) h r ( X )
=
h ( X ) for X ∈ { 0 , 1 } n and (b) h r is entry-wise affine, i.e., for any γ ∈ [0 , 1] and any ¯ X, ¯ X ′ ∈ [0 , 1] n that are only different in one entry,
[Equation]
Proof sketch.
Set h r ( ¯ X )
= (cid:80) X ∈{ 0 , 1 } n h ( X ) (cid:81) nj =1 ¯ X X
j j (1 − ¯ X j ) (1 − X j ) , which satisfies (a) and (b).
Note that we suppose that ¯ X
j = 1 for any ¯ X j ∈ [0 , 1] .
The detailed proof is in Appendix A.2.
Although Theorem 2 shows the existence of entry-wise affine relaxations, the constructed representation in the proof depends on higher-order moments of the input entries, which make it hard to be implemented by a model, say a NN architecture.
However, we claim that using entry-wise concave functions is able to implicitly generate higher-order moments via representations based on low-order moments.
For example, when n = 2 , we could use the composition of − Relu ( · ) and affine operators (only 1st-order moments) to achieve universal representation (See Prop. 1 and the proof in
Appendix A.3).
For general n , we leave as a future study.
Proposition 1.
For any binary-input function h ( X 1 , X 2 ) , there exists parameters { w ij } such that an entry-wise concave function h r ( ¯ X 1 , ¯ X 2 ) =
w 00 − (cid:80)
i =1 Relu ( w i 1 ¯ X 1 + w i 2 ¯ X 2 + w i 0 ) satisfies h r ( X 1 , X 2 )
=
h ( X 1 , X 2 ) for any X 1 , X 2 ∈ { 0 , 1 } .
3.4 Discussion: Methods to Directly Optimize the Randomized Objective in EPM Eq. (2)
The naïve way to optimize the randomized objective in Eq.(2) without worrying about the specific form of the objective l is based on the policy gradient in RL via the logarithmic trick, i.e., estimating the gradient dldθ via ( f ( X ; C )+ β 1 g
( X ; C ) ≥ 1 ) log P ( X ) by sampling X ∼ A θ ( C ) .
However, the policy gradient suffers from notoriously large variance [39] and makes RL hard to converge.
Therefore, methods such as actor critic [60] or subtracting some baselines l ( X ; C ) − b [61] have been proposed.
Label of
Label of entry-wise concave/affine proxy ...
Random Sampling
Supervised Training ...
Proxy Relaxation in PCO Problems
C 1 , X
C n , X n
Explicitly Write entry-wiseconcave/affineproxy
Proxy Relaxation in CO problems
Randomly Sampled configurations ...
C
C n
Unsupervised Training with Loss:
C'
A Testing Instance
C'
Relaxed Solution Rounding
C'
Discrete Solution
Training
Testing
C
[Figure]
Another way to solve Eq.(2) is based on reparameterization tricks to reduce the variance of gradients [62,63].
Specifically, we set the entries of output ¯ X
=
A θ ( C ) ∈ [0 , 1] n as the parameters of Bernoulli distributions to generate X , i.e., X i ∼ Bern ( ¯ X i ) , for 1 ≤ i ≤ n .
To make dX i /d
¯ X i computable, we may use the Gumble-softmax trick [47–49].
However, this approach suffers from two issues.
First, the estimation of the gradient is biased.
Second, as
A θ ( C ) is essentially a randomized algorithm, sampling sufficiently many X ∼ A θ ( C ) is needed to guarantee a valid and low-cost solution.
However, such evaluation is costly as discussed in Sec. 3.1.
So, empirically, we can also compare A θ ( C ) with a threshold to determine X , which does not have performance guarantee.
We compare different aspects of RL, Gumbel-softmax tricks and our relaxation approach in Table 1.
Applying Our Relaxation Principle to Learning for CO
[Table]
[Table] [Table]
First, we test our relaxation principle in a learning for CO (max clique) task, where we can explicitly write both the cost functions and the constraints into an entry-wise affine form.
In this case, our framework and EPM [1] share the same pipeline, though the relaxation principle and the deterministic performance guarantee are firstly proposed in this work.
The entry-wise affine objective relaxation of the max clique is − ( β + 1) (cid:80) ( i,
j ) ∈
E ¯
X
i ¯ X j + β 2 (cid:80) i ̸
=
j ¯ X i ¯ X j .
Here, we use a real-world dataset (Twitter [64]) and a synthetic dataset (RBtest [1]), and show the experiment results in Table.
We follow the settings in [1] and use a 6:2:2 dataset split for training, validating and testing, each test instance runs within 8 seeds.
Our entry-wise affine pipeline achieves almost the same performance as EPM.
To show the importance of the relaxation principle, we also propose ‘Badloss+R’.
This baseline imposes trigonometric functions to the original entry-wise affine functions f ′ r ( ¯ X ; C ) =
f r (sin(9 π ¯ X/ 2); C ) , g ′ r ( ¯ X ; C ) =
g r (sin(9 π ¯ X/ 2); C ) , where sin( · ) operates on each entry of the input vector.
Note that the relaxed functions also match the original objectives at discrete points, i.e., f ′ r
( X ; C ) =
f r ( X ; C ) =
f
( X ; C ) when X ∈ { 0 , 1 } , while with different relaxations.
The poor performance of ‘Badloss+R’ reveals the importance of our principle for relaxation.
Applying Our Relaxation Principle to Learning for PCO
In this section, we apply our relaxation principle to three PCO applications: (I) feature-based edge covering & node matching, (II) resource allocation in circuit design, and (III) imprecise functional unit assignment in approximate computing.
All the applications have graph-based configurations C .
So later, we first introduce how to use graph neural networks (GNNs) to build proxies that satisfy our relaxation principle.
Such GNN-based proxies will be used as the cost function relaxation f r in all the applications.
Our principle can also guide the relaxation of explicit CO objectives.
The constraints in applications (I)(III) are explicit and their relaxation can be written into the entry-wise affine form.
The constraint in (II) needs another GNN-based entry-wise concave proxy to learn.
5.1 GNN-based Entry-wise Concave Proxies
We consider the data configuration C as an attributed graph ( V, E, Z ) where V is the node set,
E ⊆
V × V is the edge set and Z is the node attributes.
We associate each node with a binary variable and group them together X : ∈ { 0 , 1 } | V | .
where for each v ∈ V , X v = 1 indicates the choice of the node v .
Note that our approach can be similarly applied to edge-level variables (see Appendix C.3), which is used in application (I).
Let ¯ X still denote the relaxation of X .
To learn a discrete function h : { 0 , 1 } | V | × C
→ R , we adopt a GNN as the relaxed proxy of h .
We first define a latent graph representation in R F whose entries are all entry-wise affine mappings of X .
Latent representation:
[Equation]
where W is the graph representation, U v ’s are node representations and Q v,u are edge representations.
These representations do not contain X and are given by GNN encoding C .
Here, we consider at most 2nd-order moments based on adjacent nodes as they can be easily implemented via current
GNN platforms [65,66].
Then, we use ϕ to generate entry-wise affine & concave proxies as follows.
Entry-wise Affine Proxy (AFF):
[Equation]
Entry-wise Concave Proxy (CON):
[Equation]
where w a , w c ∈ R F , b ∈ R are learnt parameters and w c ≥ 0 guarantees entry-wise concavity.
Other ways to implement GNN-based entry-wise concave proxies are also introduced in Appendix C.1.
5.2 The Setting up of the Experiments
[Table]
Training & Evaluation Pipeline.
In all the applications, we adopt the following training & evaluation pipeline.
First, we have a set of observed configurations D 1 ⊂ C .
Each C ∈ D 1 is paired with one X ∈ { 0 , 1 } n .
We use the costs f ( X, C ) (and constraints g ( X, C ) ) to train the relaxed proxies f r ( X, C ) (and g r ( X, C ) , if cannot be derived explicitly), where the relaxed proxies follow either Eq.(5) (named AFF) or Eq.(6) (named
CON).
Then, we parameterize the LCO algorithm
A θ ( C ) ∈ [0 , 1] n via another GNN.
Based on the learned (or derived) f r and g r , we optimize θ by minimizing (cid:80)
C ∈D 1 l r ( θ ; C ) , where l r is defined according to Eq.(3).
We will split D 1 into a training set and a validation set for hyperparameter-tuning of both proxies and A θ .
We have another set of configurations D 2 ⊂ C used for testing.
For each C ∈ D 2 , we use the relaxation ¯ X
=
A θ ( C ) plus our rounding to evaluate the learned algorithm
A θ ( · ) .
We follow [1] and do not consider fine-tuning
A θ over the testing dataset D 2 to match the potential requirement of the fast inference.
Baselines.
We consider 4 common baselines that is made up of different learnable relaxed proxies f r , g r , algorithms A θ and inference approaches as shown in Table 3.
For the proxies f r , g r for baselines, we apply GNNs without the entry-wise concave constraint and use X as one node attribute while keeping all other hyper-parameters exactly the same as CON other than the way to deal with the discrete variables to make fair comparison (See details in Appendix. C);
For the algorithm A θ , we provide the Gumbel-softmax trick based methods (GS-Tr) [48, 49], the actor-critic-based RL method [60] (RL) and the naïve relaxation method (Naïve); For the inference approaches, we consider
Monte Carlo sampling (S) and our proposed rounding (R) procedure.
Although the baselines adopt proxies that are different from ours, we guarantee that their proxies approximate the ground-truth
A plain picture
[Figure]
f, g over the validation dataset at least no worse than ours.
In application II, we also consider two non-learnable algorithms to optimize the proxies without relaxation constraints, simulated annealing (SA) [35] and genetic algorithms (GA) [67,68].
In application III, we put all of the required AxC units either close to the input (C-In) or close to the output (C-Out) of the approximating computing circuit as additional baselines.
More details of the experiments setups and hyperparameter tuning can be found in Appendix C. We also obtain the optimal solutions (OPT) for applications I and III via brute-force search for comparison.
5.3 Application I: Feature-based Edge Covering & Node Matching in Graphs
This application is inspired by [69].
Here, each configuration C is a 4 × 4 grid graph whose node attributes are two-digit images generated by random combinations of the pictures in MNIST [70].
We associated each edge with variables X ∈ { 0 , 1 } | E | .
The objective is the sum of edge weights f
( X ; C )
= (cid:80) e
∈ E w e
X
e where w e is unknown in prior and needed to be learned.
The ground truth of w e is a multiplication of the numbers indicated by the images on the two adjacent nodes.
We adopt
ResNet-50 [71] (to refine node features) plus GraphSAGE [72] to encode C .
We consider using both
Eq.(5) and Eq.(6) to formulate the relaxed cost f r ( ¯ X ; C ) .
Training and validating f r are based on 100k randomly sampled C paired with randomly sampled X .
Note that 100k is much smaller than the entire space { 0 , 1 }
| E | × C is of size 2 24 × 100 16 .
Next, as the constraint here is explicit, we can derive the relaxation of the constraints for this application.
First, the constraint relaxation of the edge covering problem can be written as
Edge Covering Constraint
[Equation]
Each production term in Eq.(7) indicates that for each node, at least one edge is selected.
We can easily justify that g r is entry-wise affine and Ω = { X ∈ { 0 , 1 } | E | : g r ( X ; C ) < 1 } exactly gives the feasible solutions to the edge covering problem.
Similarly, we can derive the constraint for node matching by adding a further term to Eq.(7).
Node Matching Constraint
[Equation]
Here, the second term indicates that no two edges adjacent to the same node can be selected.
This is a case with two constraints while we combine them together.
We can easily justify that g r is entry-wise affine and Ω = { X ∈ { 0 , 1 } | E | : g r ( X ; C ) < 1 } groups exactly the feasible solutions to the node matching problem.
Note that our above derivation also generalizes the node-selection framework in [1] to edge selection.
With the learned f r and the derived g r , we further train and validate A θ over the 100k sampled ( X, C ) ’s and test on another 500 randomly sampled C ’s. Method [Table]
Naive
[Table]
GS [Table] +
[Table] [Table]
- GS
[Table]
+
[Table]
Evaluation.
Table 4 shows the evaluation results.
In the
GS-Tr+S method, the number of sampling is set to(about 2 .
5 times the inference time of our deterministic rounding).
Note that for node matching, GS-Tr+S could hardly sample a feasible node matching solution within 120 samples.
The experiment results show that our principled proxy relaxation exceeds the other baselines on both tasks.
Also, we observe that AFF outperforms
CON, which results from the fact that f ( X ; C ) in these two problems are naturally in entry-wise affine forms
[Figure]
with low-order (1st-order) moments.
One instance of node matching randomly selected from the test set is shown in Fig.
More visualization results can be found in Fig.
6 in the appendix.
5.4 Application II: Resource Allocation in Circuit Design
Resource allocation in field-programmable gate array (FPGA) design is a fundamental problem which can lead to largely varied circuit quality after synthesis, such as area, timing, and latency.
In this application, we follow the problem formulation in [24,30], where the circuit is represented as a data flow graph (DFG), and each node represents an arithmetic operation such as multiplication or addition.
The goal is to find a resource allocation for each node to be either digital signal processor (DSP) or look-up table (LUT), such that the final circuit area (i.e., actual DSP and LUT usage) after synthesis is minimized.
Notably, different allocation solutions result in greatly varied DSP/LUT usage due to complicated synthesis process, which cannot be simply summed up over each node.
To obtain precise
DSP/LUT usage
, one must run high-level synthesis (HLS) [73] and place-and-route [74] tools, which can take up to hours [24,30].
In this application, each configuration C is a DFG with > 100 nodes, where each node is allocated to either DSP or LUT.
Node attributes include operation type (i.e., multiplication or addition) and data bitwidth.
More details about the dataset can be found in Appendix C.5.
Let X ∈ { 0 , 1 } | V | denote the mapping to DSP or LUT.
Let f r and g r denote the proxies of actual LUT and actual DSP usage, respectively.
Note that given different constraints on the DSP usage, we will normalize g r as introduced in Sec.
We train and validate f r , g r , A θ on 8 , 000 instances that consist of 40 DFGs ( C ), each DFG with 200 different mappings ( X ), and test A θ over 20 DFGs.
Note that the actual
LUT and DSP usages of each training instance has been collected by running HLS in prior.
We also run HLS to evaluate the actual LUT and DSP usages for the testing cases given the learned mappings.
Evaluation.
We rank each method’s best actual LUT usage under the constraint of different percentages ( 40% - 70% ) of the maximum DSP usage in each testing instance, then calculate the averaged ranks.
Fig.
5 shows the results.
Our entry-wise concave proxy achieves the best performance.
GS-Tr+R is slightly better than RL, and both of them exceed SA and GA.
We do not include our entry-wise affine proxy in the ranking list, because the affine proxy could be much less accurate than the proxy without constraints and the entry-wise concave proxy.
The comparison between these proxies on learning DSP usage (& LUT usage) is shown in Fig.
4 (& Fig.
9 in the appendix, respectively).
The gap between different proxies indicates the FPGA circuit contains high-order moments of the input optimization variables and 2-order entry-wise affine proxy cannot model well.
We do not include the result of GS-Tr+S and Naive+R, because these methods perform poor and could hardly generate feasible solutions given a constraint of DSP usage.
We leave their results in
Table.
7 in the appendix.
Moreover, we compare the training time between different methods.
To be fair, all methods run on the same server with a Quadro RTX 6000 GPU.
The RL based optimizer takes 22 GB GRAM, while other optimizers only take 7 GB on average.
Fig.
10 in the appendix further demonstrates that our methods and GS-T methods require much less training time than RL.
Also, to give a fair comparison between learning-based approaches and traditional approaches, we implement GA with parallel (on GPU) cost-value inference for all the populations in each generation.
We set the population size as 256 , which is the same as the batch size that we used to train/infer
A θ .
The performance of GA in Fig.
5 is obtained under the condition that the inference time of the implemented parallel GA is about the same as that of our CON method.
Fig.
8 in the appendix provides more detailed comparison on the performance and the inference time between GA with different numbers of generations and our CON method.
[Table] [Table]
5.5 Application III: Imprecise Functional Unit Assignment in Approximate Computing
One fundamental problem in approximate computing (AxC) is to assign imprecise functional units (a.k.a., AxC units) to execute operations such as multiplication or addition [42–46], aiming to significantly reduce circuit energy with tolerable error.
We follow the problem formulation in [45], where given a computation graph, each node represents either multiplication or addition.
The incoming edges of a node represent its two operands.
The goal is to assign AxC units to a certain number of nodes while minimizing the expected relative error of the output of the computation graph.
In this application, each configuration C is a computation graph with 15 nodes (either multiplication or addition) that maps a vector in R 16 to R .
A fixed number θ of nodes are assigned to AxC units with produce 10% relative error.
Let X ∈ { 0 , 1 } | V | denote whether a node is assigned to an AxC unit or not; the proxy of the objective f r is the expected relative error at the output.
We use 100k ( X, C ) as the training dataset and the entire solution space is 2 15 × 2 15 .
For each ( X, C ) , the ground-truth, i.e., expected relative error, is computed by averaging 1k inputs sampled uniformly at random from [0 , 10] 16 .
The constraint g r is (cid:80) v ∈ V X v ≥ θ with normalization, where θ ∈ { 3 , 5 , 8 } .
We test the learned A θ on 500 unseen configurations.
Evaluation.
Table.
5 shows the averaged relative errors of the assignments by different methods.
The problem is far from trivial.
Intuitively, assigning AxC units closed to the output, we may expect small error.
However, C-Out performs bad.
Our proxies AFF and CON obtain comparable best results.
The MAE loss values of the two proxies are also similar, as shown in Table 9 in the appendix.
The reason is that the circuit is made up of 4 layers in total which leads to at most 4 -order moments in the objective function, which is in a medium-level complexity.
Training time is also studied for this application, resulting in the same conclusion as application II (See Table 8 in the appendix).
Conclusion
This work introduces an unsupervised end-to-end framework to resolve LCO problems based on the relaxation-plus-rounding technique.
With our entry-wise concave architecture, our framework guarantees that a low objective value could lead to qualified discrete solutions.
Our framework is particularly good at solving PCO problems where the objectives need to be modeled and learned.
Real-world applications demonstrate the superiority of our method over RL and gradient-relaxation approaches in both optimization performance and training efficiency.
In the future, we aim to further broaden our framework to the applications where the optimization variables allow using more complex embeddings than binary embeddings.
Acknowledgement
We greatly thank all the reviewers for their valuable feedback and the insightful suggestions.
H.
Wang and P. Li are partially supported by 2021 JPMorgan Faculty Award and the NSF award
OAC-2117997.