G ENERALIZATION OF N EURAL C OMBINATORIAL S OLVERS T HROUGH THE L ENS OF A DVERSARIAL R OBUSTNESS
A BSTRACT
End-to-end (geometric) deep learning has seen ﬁrst successes in approximating the solution of combinatorial optimization problems.
However, generating data in the realm of NP-hard/-complete tasks brings practical and theoretical challenges, resulting in evaluation protocols that are too optimistic.
Speciﬁcally, most datasets only capture a simpler subproblem and likely suffer from spurious features.
We investigate these effects by studying adversarial robustness–a local generalization property–to reveal hard, model-speciﬁc instances and spurious features.
For this purpose, we derive perturbation models for SAT and TSP.
Unlike in other applications, where perturbation models are designed around subjective notions of imperceptibility, our perturbation models are efﬁcient and sound , allowing us to determine the true label of perturbed samples without a solver.
Surprisingly, with such perturbations, a sufﬁciently expressive neural solver does not suffer from the limitations of the accuracy-robustness trade-off common in supervised learning.
Although such robust solvers exist, we show empirically that the assessed neural solvers do not generalize well w.r.t.
small perturbations of the problem instance.
I NTRODUCTION
Combinatorial Optimization covers some of the most studied computational problems.
Wellknown examples are the NP-complete SATisﬁability problem for boolean statements or the Traveling Salesperson Problem (TSP).
These problems can be solved efﬁciently with approximate solvers that have been crafted over the previous decades (Festa, 2014).
As an alternative to engineered, application-speciﬁc heuristics, learning seems to be a good candidate (Bengio et al., 2021) complete and studied as a component in traditional solvers (e g
Despite deep learning for combinatorial optimization gaining attention recently, it is still an open question if and to what extent deep learning can effectively approximate NP-hard problems.
Moreover, there is a “Catch-22“; even if neural networks could solve NP-hard problems, generating the training data is either (a) incomplete but efﬁcient or (b) complete but inefﬁcient (or approximate).
An incomplete data generator (a) crafts the problem instances s.t.
their labels are known and a complete data generator (b) obtains the label via (approximately) solving the random instance.
Additionally, a dense sample is intractable even for moderate problem sizes due to the large problem space X .
For the special case of (a) exact polynomial-time data-generators, Yehuda et al.
(2020) detail two challenges as a result of the NP-hardness: (1) Easier subproblem: a dataset from an efﬁcient data generator only captures a strictly easier subproblem.
(2) Spurious features: due to the lack of completeness, the resulting dataset can be trivially solvable due to superﬁcial/spurious features.
These ﬁndings extrapolate to case (b) for a sufﬁciently sparse sample (i.e. data could have been generated by an efﬁcient data generator).
Thus, it is concerning that often the same (potentially ﬂawed) data generator is used for training and evaluation .
Adversarial robustness is a challenging local generalization property that offers a way of ﬁxing the too optimistic model performance estimates that are the result of evaluations on incomplete datasets.
We illustrate this in Fig. 1 for two relevant data generation schemes: (a) to the left we discuss an incomplete, efﬁcient data generator and (b) to the right we discuss a sparse sample of a potentially complete data generator.
With a suitable choice of perturbation model, we (a) possibly extend the reachable space by an efﬁcient data generator or (b) cover the space around each instance in the sparse dataset.
In other words, such adversarial attacks aim to ﬁnd the intersection of the uncovered regions and hard, model-speciﬁc samples.
Therefore, it is possible to detect the described defects.
Adversarial robustness more realistically evaluates a model’s generalization ability instead of simply testing on the same data generation procedure or a sparse external dataset.
Adversarial robustness is a desirable property for neural combinatorial optimization, since, in contrast to general learning tasks, in combinatorial optimization we do not have an accuracy robustness trade-off in the sense of Suggala et al.
This means, there exists a model with high accuracy and high robustness.
One key factor to avoid the accuracy robustness trade-off is choosing a perturbation model that guarantees the correct label of the perturbed sample (we call them sound ).
This is in stark contrast to other domains where one relies on imperceptible perturbations.
We instantiate our adversarial attack framework for the NP-complete SAT and TSP.
Nevertheless, most of the principles can be transferred to other combinatorial optimization problems.
Note that often such problems can be even reduced onto one another, as is the case for e.g. SAT and the maximum independent set.
Having said this, we select SAT because of its general applicability and the notoriously challenging TSP due to its practical importance (e.g. for supply chain optimization).
We then use our attacks to show that the evaluated neural SAT and TSP solvers are highly non-robust.
Contributions.
(1) We bring the study of adversarial robustness to the ﬁeld of neural combinatorial solvers to tackle fundamental problems in the evaluation of neural solvers.
(2) We propose perturbation models for SAT and TSP s.t.
we efﬁciently determine the updated solution.
(3) We show that the models for SAT and TSP can be easily fooled with small perturbations of the problem instance and (4) that adversarial training can improve the robustness and generalization.
B ACKGROUND ON N EURAL S OLVERS
Intuitively, combinatorial optimization is the task of ﬁnding an optimal element from the ﬁnite set of possible solutions (e.g. the truth assignment for a boolean statement).
We formalize this as Y = arg min Y (cid:48)
∈ g ( x ) c ( x , Y (cid:48) ) where x is a problem instance, g ( x ) =
Y the ﬁnite set of feasible solutions, and c ( · ) a cost function.
Typically, there is also an associated binary decision problem y , such as ﬁnding the optimal route vs. checking whether a route of at most cost c 0 exists (see § 4 and § 5).
Then, for example, a neural solver ˆ y =
f θ ( x ) learns a mapping f θ : X → { 0 , 1 } to approximate the decision problem.
In this work, θ are the parameters, x ∈ X is the problem instance, and ˆ y (or ˆ Y ) the prediction.
In case of supervised learning, we then optimize the parameters θ w.r.t.
a loss (cid:96) ( f θ ( x ) , y ) over a ﬁnite set of labeled training instances ( x , y ) .
However, to obtain the exact labels y for a given x is intractable for larger problem instances due to the exponential or worse runtime.
Two ways to generate data pairs are mentioned in the introduction and visualized in Fig.
1: (a) an efﬁcient but incomplete data generator (b) using a solver to obtain the labels for random samples.
A DVERSARIAL R OBUSTNESS
Adversarial robustness refers to the robustness of a machine learning model to a small perturbation of the input instance (Szegedy et al., 2014).
We deﬁne an adversarial attack in Eq.
1, where the parameters θ are constant and G denotes the perturbation model that describes the possible perturbed instances ˜ x around the clean sample x (i.e. the perturbation space) given the original solution Y .
Since ˜ Y (cid:54) =
Y in the general case, we introduce ˜ Y
= h (˜ x , x , Y ) to model how the solution changes.
[Equation]
Sound and efﬁcient perturbation model.
Our framework for a neural combinatorial solver f θ stands out from many other works on adversarial robustness since we choose the perturbation model
G s.t.
we provably know a solution ˜
Y = h (˜ x , x , Y ) for all possible ˜ x .
We call such a perturbation model sound .
This stands in contrast to other domains, where we usually hope to preserve the label using the subjective concept of imperceptible/unnoticable perturbations (Szegedy et al., 2014).
While we can naively obtain a sound perturbation model for combinatorial optimization using a solver, this is intractable for realistic problem sizes.
We therefore propose to use perturbation models that are efﬁcient and sound .
That is, we can determine ˜ Y without applying a solver on the perturbed instance ˜ x .
For example, if we add a node to a TSP instance, the optimal route including the new node will change, but we can efﬁciently determine ˜ Y for the chosen G .
Important technical details arise due to (a) the potentially non-unique Y and (b) non-constant ˜ Y while perturbing the input.
One way to handle both effects is through the choice of the loss (cid:96) .
(a) We can deal with the ambiguity in Y if the loss is equal for any two optimal solutions/predictions.
This can be achieved naturally by incorporating the cost c ( Y ) of the combinatorial optimization problem.
(b) Since the solution ˜ Y can change throughout the optimization, it is important to choose a loss that assesses the difference between prediction f θ (˜ x ) and ground truth ˜ Y .
For example, a viable loss for TSP is the optimality gap (cid:96) OG ( ˆ Y , Y ) =
| c (ˆ Y ) − c
( Y )
| / c ( Y ) ) that is normalized by c ( Y ) .
Perturbation strength.
With a sound perturbation model, all generated instances ˜ x are valid problem instances regardless of how much they differ from x .
Hence, in the context of combinatorial optimization, the perturbation strength/budget models the severity of a potential distribution shift between training data and test data.
This again highlights the differences to other domains.
For example in image classiﬁcation with the common L p perturbation model (cid:107) x − ˜ x (cid:107) p ≤ r , the instance changes its true label or becomes meaningless (e.g. a gray image) for a large enough r .
Generalization.
Speciﬁcally, adversarial robustness is one way to measure the generalization over perturbed instances ˜ x in the proximity of x .
Adversarial robustness is important in the context of neural combinatorial solvers since training and validation/test distribution differ from the actual data distribution p ( x ) .
First, the data distribution p ( x ) is typically unknown and highly applicationspeciﬁc.
Second, due to theoretical limitations of the data generation process the train and validation/test distribution only captures a simpler sub-problem likely suffering from spurious features (Yehuda et al., 2020).
Third, we ultimately desire a general-purpose solver that performs well regardless of p ( x ) (in the limits of a polynomial approximation).
We stress that in the context of combinatorial optimization, adversarial examples are neither anomalous nor statistical defects since all generated instances correspond to valid problem instances.
In contrast to other domains, the set of valid problems is not just a low-dimensional manifold in a high-dimensional space.
Thus, the so-called manifold hypothesis (Stutz et al., 2019) does not apply for combinatorial optimization.
In summary, it is critical for neural solvers to perform well on adversarial examples when striving for generalization.
Accuracy robustness trade-off.
A trade-off between adversarial robustness and standard generalization was reported for many learning tasks (Tsipras et al., 2019).
That is, with increasing robustness the accuracy on the test data decreases.
Interestingly, with a sound perturbation model and the purely deterministic labels in combinatorial optimization (the solution is either optimal or not), no such trade-off exists.
Hence, if the model was expressive enough and we had sufﬁcient compute, there would exist a model with high accuracy and robustness (see § A for more details).
Adversarial training.
In adversarial training, we leverage adversarially perturbed instances with the desire of training a robust model with improved generalization.
For this, adversarial attacks reveal the regions that are both difﬁcult for the model and not covered by training samples (see Fig. 1).
Hence, adversarial training can be understood as a powerful data augmentation using hard modelspeciﬁc samples.
Though it is not the main focus of this work, in § 6, we show that adversarial training can be used to improve the robustness and generalization of a neural combinatorial solver.
Remarks on decision problems.
For the binary decision problems, we typically are not required to know ˜ Y ; it sufﬁces to know ˜ y .
Moreover, for such binary problems, we keep the solution constant y =
˜
y , but there also exist practical perturbations that change the label of the decision problem.
For example for SAT, we can add a set of clauses that are false in isolation which makes ˜ y = 0 .
Requirements for neural solvers.
We study neural combinatorial solvers f
θ that are often a Graph
Neural Network (GNN).
We then solve Eq.
1 using different variants of Projected Gradient Descent (PGD) and therefore assume the model to be differentiable w.r.t.
its inputs (see § D).
For nondifferentiable models, one can use derivative-free optimization (Yang & Long, 2021).
SAT
We ﬁrst introduce the problem as well as notation and then propose the perturbation models (§ 4.1).
Last, we discuss the attacks for a speciﬁc neural solver (§ 4.2).
Problem statement.
The goal is to determine if a boolean expression, e.g. ( v 1 ∨ v 2 ∨¬ v 3 )
∧ ( v 1 ∨ v 3 ) , is satisﬁable y = 1 or not y = 0 .
Here, we represent the boolean expressions in Conjunctive
Normal Form (CNF) that is a conjunction of multiple clauses k ( v 1 , . .
. , v n ) and each clause is a disjunction of literals l i .
Each literal is a potentially negated boolean variable l i ∈ {
¬ v i , v i } and w.l.o.g.
we assume that a clause may not contain the same variable multiple times.
In our standard notation, a problem instance x represents such a boolean expression in CNF.
A solution
Y ∈ { l ∗ 1 , . . .
, l ∗ n | l ∗ i ∈ {¬ v i , v i }} provides truth assignments for every variable.
Hence, in the example above x = ( v 1 ∨ v 2 ∨¬ v 3 ) ∧ ( v 1 ∨ v 3 ) , y = 1 , and a possible solution is Y = { v 1 , ¬ v 2 , v 3 } .
Note that multiple optimal Y exist but for our attacks it sufﬁces to know one.
4. S OUND P ERTURBATION M ODEL
We now introduce a sound and efﬁcient perturbation model for SAT which we then use for an adversarial attack on a neural (decision) SAT solver.
Recall that the perturbation model is sound since we provably obtain the correct label ˜
y
and it is efﬁcient since we achieve this without using a solver.
Instead of using a solver, we leverage invariances of the SAT problem.
Proposition 1 Let x = k 1 ( v 1 , . . .
, v n ) ∧ . . .
k m ( v 1 , . .
. , v n ) be a boolean statement in Conjunctive
Normal Form (CNF) with m clauses and n variables.
Then ˜ x , a perturbed version of x , has the same label y =
˜
y in the following cases:
• SAT: x is satisﬁable y = 1 with truth assignment Y .
Then, we can arbitrarily remove or add literals in x to obtain ˜ x , as long as one literal in Y remains in each clause.
• DEL: x is unsatisﬁable y = 0 .
Then, we can obtain ˜ x from x through arbitrary removals of literals, as long as one literal per clause remains.
• ADC: x is unsatisﬁable y = 0 .
Then, we can arbitrarily remove, add, or modify clauses in x to obtain ˜ x , as long as there remains a subset of clauses that is unsatisﬁable in isolation.
4. N EURAL SAT S OLVER
Selsam et al.
(2019) propose NeuroSAT, a neural solver for satisﬁability-problems that uses a message-passing architecture (Gilmer et al., 2017) on the graph representation of the boolean expressions.
The SAT problem is converted into a bipartite graph consisting of clause nodes and literal nodes.
For each variable there exist two literal nodes; one represents the variable and the other its negation.
If a literal is contained in a clause its node is connected to the respective clause node.
NeuroSAT then recursively updates the node embeddings over the message-passing steps using this graph, and in the last step, the ﬁnal vote ˆ y ∈ { 0 , 1 } is aggregated over the literal nodes. Attacks.
We then use these insights to craft perturbed problem instances ˜ x guided by the maximization of the loss (cid:96)
adv G (see Eq. 1). Speciﬁcally, for SAT and DEL , two of admissible perturbations deﬁned in Proposition 1, we optimize over a subset of edges connecting the literal and clause nodes where we set the budget ∆ relatively to the number of literals/edges in x . For ADC we additionally concatenate d additional clauses and optimize over their edges obeying ∆ but keep the remaining x constant. If not reported separately, we decide for either DEL and ADC randomly with equal odds.
For additional details about the attacks, we refer to § E.
L 0 -PGD. The addition and removal of a limited number of literals is essentially a perturbation with budget ∆ over a set of discrete edges connecting literals and clauses.
Similarly to the L 0 -PGD attack of Xu et al.
(2019), we continuously relax the edges in { 0 , 1 } to [0 , 1] during optimization.
We then determine the edge weights via projected gradient descent s.t.
the weights are within [0 , 1] and that we obey the budget ∆ .
After the attack, we use these weights to sample the discrete perturbations in { 0 , 1 } .
In other words, the attack continuously/softly adds as well as removes literals from x during the attack and afterward we sample the discrete perturbations to obtain ˜ x .
[-We then use these insights to craft perturbed problem instances ˜ x guided by the maximization of the loss (cid:96) 
-]
TSP
We ﬁrst introduce the TSP including the necessary notation.
Then, we propose a perturbation that adds new nodes s.t.
we know the optimal route afterward (§ 5.1).
In § 5.2, we detail the attack for a neural decision TSP solver and, in § 5.3, we describe the attack for a model that predicts the optimal
TSP route Y .
Problem statement.
We are given a weighted graph G =
( V , M ) that consist of a ﬁnite set of nodes V as well as
edges M ⊆ V 2 and a weight ω ( e ) for each possible edge e ∈ V 2 .
We use the elements in V as indices or nodes interchangeably.
The goal is then to ﬁnd a permutation σ of the nodes V s.t.
the cost of traversing all nodes exactly once is minimized (i.e. the Hamiltonian path of lowest cost):
[Equation]
i
[Equation]
[Equation]
where S is the set of all permutations and n = |
V | is the number of nodes.
Although multiple σ ∗ might exist here it sufﬁces to know one.
An important special case is the “metric TSP”, where the nodes represent coordinates in a space that obeys the triangle inequality (e.g. euclidean distance).
For notational ease, we interchangeably use σ as a permutation or the equivalent list of nodes.
Moreover, we say σ contains edge ( I, J ) ∈
| M | if I and J are consecutive or the ﬁrst and last element.
In our standard notation x = G , Y = σ ∗ , and the respective decision problem solves the question if there exist c ( σ ∗ ) ≤ c 0 of at most c 0 cost.
5. S OUND P ERTURBATION M ODEL
Adversarially perturbing the TSP such that we know the resulting solution seems more challenging than SAT.
However, assuming we would know the optimal route σ ∗ for graph x = G , then under certain conditions we can add new nodes s.t.
we are guaranteed to know the perturbed optimal route ˜ σ ∗ .
Note that this does not imply that we are able to solve the TSP in sub-exponential time in the worst case.
We solely derive an efﬁcient special case through leveraging the properties of the TSP.
Proposition 2 Let σ ∗ be the optimal route over the nodes V in G , let Z (cid:54)∈ V be an additional node, and P, Q are any two neighbouring nodes on σ ∗ .
Then, the new optimal route ˜ σ ∗ (including Z ) is obtained from σ ∗ through inserting Z between P and Q if (cid:54) ∃
( A, B ) ∈ V 2 \ { ( P, Q ) } with A (cid:54) =
B s.t. ω
( A, Z ) + ω
( B, Z ) − ω
( A, B ) ≤ ω ( P, Z ) + ω
( Q, Z ) − ω ( P, Q ) .
Corollary 1 We can add multiple nodes to G and obtain the optimal route ˜ σ ∗ as long as the condition of Proposition 2 (including the other previously added nodes) is fulﬁlled.
Corollary 2 For the metric TSP, it is sufﬁcient if the condition of Proposition 2 holds for ( A, B ) ∈ V\
( { ( P, Q ) } ∪ H ) with A (cid:54) =
B where H denotes the pairs of nodes both on the Convex Hull H ∈ CH ( V ) 2 that are not a line segment of the Convex Hull.
5. N EURAL D ECISION TSP S OLVER
Prates et al.
(2019) propose a GNN (called DTSP) to solve the decision variant of the TSP for an input pair x
=
( G , c 0 ) with graph G and a cost c 0 .
DTSP predicts whether there exists a Hamiltonian cycle in G of cost c 0 or less ( y = 1 if the cycle exists).
Based on our perturbation model, we inject adversarial nodes.
For the metric TSP, we determine their coordinates by maximizing the binary cross-entropy–a continuous optimization problem.
This is easy to generalize to the non-metric TSP (omitting Corollary 2), if e.g. the triangle equality does not hold or there is no “simple” cost function concerning the node’s coordinates/attributes.
Then, the optimization is performed over the edge weights, but depending on what the weights represent we might need to enforce further requirements.
Unfortunately, the constraint in Proposition 2 is non-convex and it is also not clear how to ﬁnd a relaxation that is still sufﬁciently tight and can be solved in closed form.
For this reason, when the constraint for a node is violated, we use vanilla gradient descent with the constraint as objective: ω ( P, Z )
+ ω
( Q, Z ) − ω ( P, Q ) − [min A,B ω ( A, Z )
+ ω
( B, Z ) − ω
( A, B )] .
This penalizes if a constraint is violated.
We stop as soon as the node fulﬁlls the requirement/constraint again and limit the maximum number of iterations to three.
Since some adversarial nodes might still violate the constraint after this projection, we only include valid nodes in each evaluation of the neural solver.
Moreover, for optimizing over multiple adversarial nodes jointly and in a vectorized implementation, we assign them an order and also consider previous nodes while evaluating the constraint.
Ordering the nodes allows us to parallelize the constraint evaluation for multiple nodes, despite the sequential nature, since we can ignore the subsequent nodes.
5. N EURAL TSP S OLVER
Joshi et al.
(2019) propose a Graph Convolutional Network (ConvTSP) to predict which edges of the euclidean TSP graph are present in the optimal route.
The probability map over the edges is then decoded into a permutation over the nodes via a greedy search or beam search.
We use the same attack as for the TSP decision problem (see § 5.2) with the exception of having a different objective with changing label ˜ Y .
Although the optimality gap (cid:96) ( ˆ Y , Y ) =
c (ˆ Y ) − c
( Y ) /
c ( Y ) is a natural choice and common in the TSP literature (Kool et al., 2019), it proved to be tough to backpropagate through the decoding of the ﬁnal solution from the soft prediction.
Hence, for ConvTSP we maximize the cross-entropy over the edges. Hence, we perturb the input s.t. the predicted route is maximally different from the optimal solution ˜ Y and then report the optimality gap.
E MPIRICAL R ESULTS
In this section, we show that the assessed SAT and TSP neural solvers are not robust w.r.t.
small perturbations of the input using the sound perturbation models introduced in § 4 and 5.
We ﬁrst discuss SAT in § 6.1 and then TSP in § 6.2.
We use the published hyperparameters by the respective works for training the models.
We run the experiments for at least ﬁve randomly selected seeds.
We compare the accuracy on the clean and perturbed problem instances (i.e. clean vs. adversarial accuracy).
Since no directly applicable prior work exists, we compare to the random baseline that randomly selects the perturbation s.t.
the budget is exhausted.
Moreover, we use Adam (Kingma & Ba, 2015) and early stopping for our attacks.
For further details we refer to § E and § F as well as the code https://www.daml.in.tum.de/robustness-combinatorial-solvers .
[Figure]
(un)robustness of the NeuroSAT model on the 10-40 dataset.
Recall that SAT is the attack on the satisﬁable problem instances and ADC as well as DEL are the attacks on unsatisﬁable problem instances.
[Figure]
Setup.
Following Selsam et al.
(2019), we train NeuroSAT forepochs using the ofﬁcial parameters and data generation.
The random data generator for the training/validation data greedily adds clauses until the problem becomes unsatisﬁable which is determined by an exact SAT solver (Ignatiev et al., 2018; S¨orensson & Een, 2005).
We are then left with an unsatisﬁable problem instance and a satisﬁable problem instance if we omit the last clause (i.e. the dataset is balanced).
For each instance pair, the number of variables is sampled uniformly within a speciﬁed range and then the number of literals in each clause is drawn from a geometric distribution.
We name the dataset accordingly to the range of numbers of variables.
For example, the training set 10-40 consists of problem instances with 10 to 40 variables.
For our attacks, we use the budgets of ∆ DEL = 5% as well as ∆ SAT = 5% relatively to the number of literals in x and for ADC we add an additional 25% of clauses and enforce the average number of literals within the new clauses.
[Figure]
Attack efﬁcacy and model (un)robustness.
From the results of our adversarial attacks presented in Fig.
2 it is apparent that the studied model NeuroSAT is not robust w.r.t.
small perturbation of the input.
Additionally, Fig.
3 shows that for the SAT attack, one gradient update step already sufﬁces to decrease the accuracy to 26% (see also § I).
All this shows the efﬁcacy of our attacks and perturbation model, it also highlights that the standard accuracy gives a too optimistic impression of the model’s performance.
We hypothesize that the model likely suffers from challenges (1) easier subproblem and/or (2) spurious features , while it is also possible that the fragility is due to a lack of expressiveness.
[Figure]
Difﬁculty imbalance.
It is much harder for the model to spot satisﬁable instances than unsatisﬁable ones.
This is apparent from the clean accuracy and even more obvious from the adversarial accuracy.
Even with moderate budgets, we are able to lower the adversarial accuracy to values below 20% for satisﬁable instances while for unsatisﬁable instances we barely get below 50% even on the larger problem instances 50-100.
An intuitive explanation is given by the fact that it is impossible to ﬁnd a solution for an unsatisﬁable instance (and it is cheap to verify a candidate solution).
Similarly, Selsam et al.
(2019) hypothesize that NeuroSAT only changes its prediction if it ﬁnds a solution.
Thus, it is even more remarkable how we are still able to fool the neural solver in 30% of the cases (DEL attack).
Qualitative insights.
We show in Fig.
5 how NeuroSAT’s decision evolves over the messagepassing steps for satisﬁable instances.
NeuroSAT comes to its conclusion typically after 15 messagepassing steps for the clean samples.
For the perturbed samples, NeuroSAT almost never comes to the right conclusion.
For the instances where NeuroSAT predicts the right label, it has a hard time doing so since it converges slower.
For a speciﬁc adversarial example see § J.
[Figure]
[Table]
Attack budget.
We study the inﬂuence of the budget ∆ and, hence, the similarity of x vs. ˜ x , in Fig.
It sufﬁces to perturb 0.2% of the literals for the recall to drop below 50% using our
SAT perturbation model.
This stands in stark contrast to the random attack, where the accuracy for the satisﬁable instances is almost constant over the plotted range of budgets.
Hard model-speciﬁc samples.
The surprisingly weak performance (Fig. 2 and 4) of the random baselines shows how much more effective our attacks are and justiﬁes their minimally larger cost (see § H).
In § G, we show the difﬁculty is indeed model-speciﬁc.
Assuming that hard model-speciﬁc instances are the instances that are important to improve the model’s performance, we can lower the amount of labeled data (potentially expensive).
Of course, we cannot do anything about the NP-completeness of the problem, but adversarial robustness opens the possibility to use the expensively generated examples as effectively as possible.
Adversarial training for SAT.
We conjecture that if the models were expressive enough, we would now be able to leverage the adversarial examples for an improved training procedure.
Therefore, similar to Jeddi et al.
(2020), we perform an adversarial ﬁne-tuning .
That is, we train the models for another 10 epochs including perturbed problem instances.
We use the same setup as for the attacks presented above but we observed that too severe perturbations harm NeuroSAT’s performance (e.g. a budget of 5% sufﬁces to push the accuracy below 20% for the satisﬁable instances).
We therefore lower the budget of the satisﬁable instances to 1% and perturb 5% of the training instances.
For a fair comparison, we also compare to a model that was trained on a 10% larger training set.
To compare the solvers’ capability to generalize for the different training strategies, we choose datasets of different problem sizes and also include the external benchmark datasets SATLIB and UNI3SAT (Hoos & St¨utzle, 2000).
We consistently outperform the regularly trained models in terms of robustness as well as generalization (with the exceptions SATLIB).
We report the results in Table 1.
6. TSP
Setup.
In our setup we follow Prates et al.
and generate the training data by uniformly sampling n ∼ U (20 , 40) nodes/coordinates from the 2D unit square.
This is converted into a fully connected graph where the edge weights represent the L 2 -distances between two nodes.
A nearoptimal solution Y for training and attacks is obtained with the Concorde solver (Applegate et al., 2006).
For the decision-variant of the TSP, we produce two samples from every graph: a graph with y = 1 and cost-query c
y
= =
c ( σ ∗ ) · (1 + d ) and a second graph with y = 0 and cost-query c y =
=
c ( σ ∗ ) · (1 − d ) , where d = 2% (Prates et al., 2019).
For predicting the TSP solution we use
ConvTSP (Joshi et al., 2019)
but keep the setup identical to its decision equivalent.
We attack these models via adding ﬁve adversarial nodes and adjust ˜ c 0 as well as ˜ Y accordingly.
Decision TSP Solver.
If a route of target cost exists, our attack successfully fools the neural solver in most of the cases.
This low adversarial accuracy highlights again that the clean accuracy is far too optimistic and that the model likely suffers from challenges (1) easier subproblem and/or (2) spurious features .
In Fig.
7, we see that the changes are indeed rather small and that the perturbations lead to practical problem instances.
For further examples see § K.
[Figure]
Difﬁculty imbalance.
For the TSP, we also observe an imbalance in performance between both classes.
This is naturally explained with a look at the non-decision TSP version where a solver constructs a potential route ˆ Y .
Since c ( ˆ Y ) ≥ c ( Y ) such a network comes with the optimal precision of 1 by design.
Attacking TSP Solver.
For ConvTSP ﬁve new adversarial nodes sufﬁce to exceed an optimality gap of 2%.
Note that naive baselines such as the “farthest insertion” achieve an optimality gap of 2.3% on the clean dataset (Kool et al., 2019).
Moreover, for the class ”route exists” we can compare the performance on the decision TSP.
Even though the model performs better than DTSP, our attacks degrade the performance relatively by 10%.
In Fig.
9, we can also view the predicted routes and observe that the prediction can differ severely between the clean and perturbed problem instance.
[Figure]
Adv
[Figure]
Concorde
Perturbed
Clean
Adv.
Points where the attack successfully changed the optimal route for ConvTSP that show drastic changes of the prediction.
R ELATED W ORK
Combinatorial Optimization.
Further important works about neural SAT solvers are (Amizadeh et al., 2019; Yolcu & P ´ oczos, 2019; Kurin et al., 2019; Cameron et al., 2020) and for neural TSP solvers (Khalil et al., 2017; Deudon et al., 2018; Bresson & Laurent, 2021; Wang et al., 2021; Bello et al., 2016; Kool et al., 2019).
We refer to the recent surveys surveys (Bengio et al., 2021; Cappart et al., 2021; Vesselinova et al., 2020) for a detailed overview and discussion.
Generalization.
There are only very few works on generalization of neural combinatorial solvers.
One exception are Franc¸ois et al. (2019) and Joshi et al.
They empirically assess the impact of different model and training pipeline design choices on the generalization (for TSP) while we discuss the generation of hard model-speciﬁc instances.
A work by Selsam & Bjørner (2019) studies generalization for their NeuroSAT model
(Selsam et al., 2019) and proposes a data augmentation technique relying on traditional solvers.
Moreover, they study a hybrid model consisting of a simpliﬁed NeuroSAT and a traditional solver.
In summary, previous works about generalization of neural combinatorial solvers analyze speciﬁc tasks while we propose a general framework for combinatorial optimization and study TSP and SAT to show its importance.
Adversarial Robustness.
Adversarial robustness has been studied in various domains including computer vision (Szegedy et al., 2014) and graphs (Z ¨ ugner et al., 2018; Dai et al., 2018).
We refer to
G ¨ unnemann (2021) for a broad overview of adversarial robustness of GNNs.
Speciﬁcally, for TSP we optimize the continuous input coordinates (or edge weights) but use our own approach due to the non-convex constraints.
For attacking the SAT model we need to perturb the discrete graph structure and rely on L 0 -PGD by Xu et al.
(2019) proposed in the context of GNNs.
Hard sample mining.
Deriving adversarial examples might appear similar to hard sample mining (Sung, 1995).
However, hard sample mining aims in spotting hard problem instances in the train data unlike we who also perturb the problem instances (of the training data or any other dataset).
Moreover, if we combine augmentations with hard sample mining, we only create randomly perturbed instances and their generation is not guided by the model.
The surprisingly weak random baseline in our experiments gives an impression about how effective such an approach might be.
D ISCUSSION
We bring the study of adversarial robustness to the ﬁeld of neural combinatorial optimization.
In contrast to general learning tasks, we show that there exists a model with both high accuracy and robustness.
A key ﬁnding of our work is that the assessed neural combinatorial solvers are all sensitive w.r.t.
small perturbations of the input.
For example, we can fool NeuroSAT (Selsam et al., 2019) for the overwhelming majority of instances from the training data distribution with moderate perturbations (5% of literals).
We show that adversarial training can be used to improve robustness.
However, strong perturbations can still fool the model, indicating a lack of expressiveness.
In summary, contemporary supervised neural solvers seem to be very fragile, and adversarial robustness is an insightful research direction to reveal as well as address neural solver deﬁciencies for combinatorial optimization.
R EPRODUCIBILITY S TATEMENT
We provide the source code and conﬁguration for the key experiments including instructions on how to generate data and train the models.
All proofs are stated in the appendix with explanations and underlying assumptions.
We thoroughly checked the implementation and also veriﬁed empirically that the proposed sound perturbation models hold.
E THICS S TATEMENT
Solving combinatorial optimization problems effectively and efﬁciently would beneﬁt a wide range of applications.
For example, it could further improve supply chain optimization or auto-routing electric circuits.
Since combinatorial optimization is such a fundamental building block it is needless to detail how big of an impact this line of research could have.
Unfortunately, this also includes applications with negative implications.
Speciﬁcally, the examples about supply chain optimization and electric circuits also apply to military applications.
Nevertheless, we believe that the positive impact can be much greater than the negative counterpart.
Unarguably, studying adversarial robustness in the context of combinatorial optimization comes with the possibility of misuse.
However, not studying this topic and, therefore, being unaware of the model’s robustness imposes an even greater risk.
Moreover, since we study white-box attacks we leave the practitioner with a huge advantage over a possible real-world adversary that e g. does not know the weights of the model.
Aside from robustness, we did not conduct dedicated experiments on the consequences on e.g. fairness for our methods or the resulting models
A A CCURACY R OBUSTNESS T RADEOFF FOR C OMBINATORIAL S OLVERS
If the accuracy robustness trade-off existed for neural combinatorial optimization, it would imply that it is hopeless to strive for an accurate general-purpose neural combinatorial solver.
Fortunately, this is not the case.
To show this we built upon the ideas of Suggala et al.
but we only consider the special case of combinatorial decision problems where each possible prediction
ˆ
Y is either true or false (i.e. no stochasticity of the true label as long as we account for the ambiguity in Y ).
For general learning tasks, Suggala et al.
(2019) reﬁned the classical deﬁnition of adversarial robustness s.t.
there is provably no such trade-off in general classiﬁcation tasks if either the dataset is “margin separable” or when we restrict the perturbation space to be label-preserving for a Bayes optimal classiﬁer.
For combinatorial optimization with a sound perturbation model G we can argue both ways.
First, due to the soundness we never encourage a wrong prediction (i.e. the margin condition is naturally fulﬁlled).
This is contrast to e.g. image classiﬁcation where an instance close to the decision boundary with the common L p perturbation model (cid:107) x − ˜ x (cid:107) p ≤ r changes its true label or becomes meaningless (e.g. a gray image) for a large enough r .
Second, the Bayes optimal classiﬁer for combinatorial optimization solves the optimization problem perfectly on p ( x ) (with support ( p ( x )) ⊆
X ).
For these reasons, we do not need to include the Bayes optimal classiﬁer in the deﬁnition of the attack (Eq. 1).
More formally, we show by contradiction that any minimizer f ∗ = arg min f ∈ F R adv ,G ( f ) is optimal w.r.t.
min f ∈ F R ( f ) .
Let ˆ f be an optimal classiﬁer w.r.t.
R ( f ) for a combinatorial decision problem.
We assume the loss (cid:96) is equal for all possible optimal Y , to account for the multiple possible Y and deﬁne the adversarial risk
[Equation]
for any data distribution p ( x ) and the standard risk R ( f ) =
E x ∼ p ( x ) [ (cid:96) ( f θ ( x ) , Y )] .
W.l.o.g.
we assume the best possible risk is zero R ∗ =
(i.e. the prediction is always true).
Suppose f ∗ and ˆ f differ in their prediction f ∗ ( x ) (cid:54)
=
ˆ f ( x ) over a non-empty subset of support ( p ( x )) .
Since f ∗ is optimal and there is no stochasticity in the true label
Y , it is always correct with standard risk R ( f ∗ )
=
and, analogously, R adv ,G ( f ∗ ) = 0 .
Thus, ˆ f cannot be an optimizer of ˆ f = arg min f ∈ F R ( f ) if its predictions differs over a non-empty subset of support ( p ( x )) .
Note that this also holds for support ( p ( x )) ⊆ support ( G ( p ( x ))) ⊆
X which is indeed the interesting case for contemporary data generators and models.
B P ROOF OF P ROPOSITION
Proposition 1 Let x = k 1 ( v 1 , . . .
, v n ) ∧ . . .
k m ( v 1 , . .
. , v n ) be a boolean statement in Conjunctive
Normal Form (CNF) with m clauses and n variables.
Then ˜ x , a perturbed version of x , has the same label y =
˜
y in the following cases:
• SAT: x is satisﬁable y = 1 with truth assignment Y .
Then, we can arbitrarily remove or add literals in x to obtain ˜ x , as long as one literal in Y remains in each clause.
• DEL: x is unsatisﬁable y =
and we obtain ˜ x from x through arbitrary removals of literals, as long as one literal per clause remains.
• ADC: x is unsatisﬁable y = 0 .
Then, we can arbitrarily remove, add, or modify clauses in x to obtain ˜ x , as long as there remains a subset of clauses that is unsatisﬁable in isolation.
• SAT:
Every disjunctive clause evaluates to one, if one literal is true.
Since we keep at least one literal in Y in each clause, every clause evaluates to one.
The statement evaluates to one since the conjunction of true statement is also true.
• DEL:
Since every clause is a disjunction of literals, the removal of one/some of the literals strictly reduces the number of speciﬁable assignments.
That is, removing any literal l i from its clause removes the possibility to satisfy this clause through l i = 1 .
• ADC: Since the clauses are a conjunctive, all clauses need to evaluate to true.
If a subset of clauses it not satisﬁable by themselves (i.e. not all can be true at the same time), the expression necessarily resolves to 0.
Hence, all conditions in Proposition 1 do not change the satisﬁability y , but might alter Y .
It is apparent that the updated Y can be obtained efﬁciently.
(cid:3)
C P ROOF OF P ROPOSITION
Proposition 2 Let σ ∗ be the optimal route over the nodes V in G , let Z (cid:54)∈ V be an additional node, and P, Q are any two neighbouring nodes on σ ∗ .
Then, the new optimal route ˜ σ ∗ (including Z ) is obtained from σ ∗ through inserting Z between P and Q if (cid:54) ∃
( A, B ) ∈ V 2 \ { ( P, Q ) } with A (cid:54) =
B s.t. ω
( A, Z ) + ω
( B, Z ) − ω
( A, B ) ≤ ω ( P, Z ) + ω
( Q, Z ) − ω ( P, Q ) .
Corollary 1 We can add multiple nodes to G and obtain the optimal route ˜ σ ∗ as long as the condition of Proposition 2 (including the other previously added nodes) is fulﬁlled.
Corollary 2 For the metric TSP, it is sufﬁcient if the condition of Proposition 2 holds for ( A, B ) ∈ V\
( { ( P, Q ) } ∪ H ) with A (cid:54) =
B where H denotes the pairs of nodes both on the Convex Hull H ∈ CH ( V ) 2 that are not a line segment of the Convex Hull.
Proof.
We proof by contradiction.
We deﬁne (
R, S ) ∈ V 2 \ { ( P, Q ) } to be the two neighboring nodes of Z on ˜ σ ∗ .
Suppose ω ( P, Z ) + ω
( Q, Z ) − ω ( P, Q ) <
ω ( R, Z ) + ω
( S, Z ) − ω
( R, S ) and ˜ σ ∗ would not contain the edges ω ( P, Z ) as well as ω ( Q, Z ) .
We know by optimality of σ ∗ that
[Equation]
and by optimality of ˜ σ ∗ that
[Equation]
Thus,
[Equation]
and equivalently
[Equation]
which leads to a contradiction.
Since we do not know what edges are contained in ˜ σ ∗ (i.e. what nodes could be R and S )
we state the stricter condition (cid:54) ∃ ( A, B ) ∈ V 2 \ { ( P, Q ) } with A (cid:54) =
B s.t. ω
( A, Z ) + ω
( B, Z ) − ω
( A, B ) ≤ ω ( P, Z ) + ω
( Q, Z ) − ω ( P, Q ) .
(cid:3)
If multiple σ ∗ exist, then this statement holds for any optimal route that has a direct connection between P and Q .
Corollary 1 follows by induction and Corollary 2 is due to the fact that the in metric space the optimal route ˜ σ ∗ must be a simple polygon (i.e. no crossings are allowed).
This was ﬁrst stated for an euclidean space as “the intersection theorem” by Cutler (1980) and is a direct consequence of the triangle inequality.
Also note, alternatively to the proof presented here, one can also unfold the dynamic program proposed by Rote (1991) to end up at Proposition 2.
D P ROJECTED G RADIENT D ESCENT (PGD)
Data: Problem ( x , Y ) and possibly y , Solver f θ ( · ) ,
Loss (cid:96) , budget ∆ , attack steps s , learn rate α 1 ˜ x 0 ← initialize ( x , Y, ∆) 2 for t ∈ { 0 , 1 , . . .
, s − 1 } do ˜ x (cid:48)
t +1 ← update (˜ x t , α t , ∇ (cid:96) ( f θ (˜ x t ) , h (˜ x , x , Y )))
˜ x t +1 ← project (˜ x (cid:48)
t +1 , x , ∆) 5 end 6 ˜ x ← postprocess (˜ x E , ∆) 7 return ˜ x , h (˜ x , x , Y ) PGD is one of the most successful and widely studied approaches to craft adversarial examples. For a further techniques and a broader overview of adversarial robustness in various domains, we refer to Xu et al.
All our attacks roughly match the framework of Algorithm D.1.
First, we initialize the perturbed instance, or, alternatively, we can use some variable that models the difference to the clean instance x (line 1).
Initialization strategies that we consider are random initialization or initializing to the clean instance.
Then we perform the attack for s steps and in each step update the perturbed instance through a gradient descent step (line 3).
For faster convergence we additionally use Adam as optimizer (Kingma & Ba, 2015).
After the gradient update we perform a projection step that ensures we stay within the budget ∆ or satisfy other constraints.
For simplicity, we omit the fact that we use early stopping in all our algorithms.
Speciﬁcally, in each attack step we check if the current perturbed instance ˜ x t +1 comes with the best loss so far.
Then, after s attack steps we assign the best possible ˜ x s ← arg max t ∈{ 1 ,...,s } (cid:96) ( f θ (˜ x t ) , h (˜ x t , x , Y )) .
This happens right before we (optionally) perform a postprocessing.
Finally, we return the perturbed instance ˜ x with solution Y or decision label y .
Limitations.
As discussed in § 3, we require the model to be differentiable w.r.t.
its input.
Fortunately, most neural combinatorial solvers rely on GNNs and therefore this does not impose an issue.
However, even if assessing a non-differentiable model one could revert to derivative-free optimization (Yang & Long, 2021).
For some combinatorial optimization problems the number of variables we need to optimize over can be very large.
For example, when attacking a Maximum Independent Set neural solver (Li et al., 2018) using our perturbation models for SAT, we need to construct a graph that blows up quickly.
Nevertheless, this could be done with derivative-free optimization (Yang & Long, 2021) or a scalable variant of of L 0 -PGD called projected randomized block coordinate descent (Geisler et al., 2021).
E SAT A TTACK D ETAILS
SAT model description.
Selsam et al.
(2019) propose to model the input problem as a bipartite graph as described in § 4.2.
We instantiate the model as described in their paper: over 26 message passing steps the GNN updates its embeddings of size 128.
Clause nodes and literal nodes have separate LSTMs to update their embeddings with messages produced by 3-layer MLPs.
After the last message passing steps, the output vote on whether the problem is satisﬁable or not is obtained by transforming the clause embeddings with an additional vote-MLP and averaging over the ﬁnal votes of the literal nodes.
The paper also notes that the number of message passing steps has to be adapted to the problem size.
As no speciﬁc values are provided, we use 64 steps for the50dataset and 128 steps for the SATLIB and uni3sat data.
Selsam et al.
(2019) trained their model in a single epoch on a large dataset consisting of “millions of samples”.
However, since Selsam et al.
(2019) did not publish their dataset we used a total of 60,000 samples with the very same data generation strategy.
We then use 50,000 of the samples to train the model for 60 epochs.
With this strategy we closely match the reported performance.
For the larger train set in Table 1 we generate an additional 5,000 samples.
During training we use the same hyperparameters as described in the paper.
Please use the referenced code for exactly reproducing the dataset.
Optimization Problem.
We restate the optimization problem for an adversarial attack on a SAT decision problem.
The SAT attack tries to ﬁnd a perturbed instance ˜ x that maximizes the loss s.t. every clause still contains a literal that l j that is present in one possible true solution assignments Y = { l ∗ 1 , . . .
, l ∗ n | l ∗ i ∈ {¬ v i
[Equation]
[Equation]
The ADC attack maximizes the loss by adding clauses to the problem, meaning that every clause k i in the original problem x has to be present also in the perturbed instance ˜ x , as these clauses ensure that the problem remains unsatisﬁable:
[Equation]
Lastly, the DEL attack optimizes over what literals to delete from the problem’s clauses, as long as no clause is removed completely.
This results in the following optimization problem:
[Equation]
Attack Details.
The attack on SAT problems modiﬁes the literals that are contained in clauses.
This means speciﬁcally that we optimize over the edges represented by the literals-clauses adjacency matrix x =
A ∈ { 0 , 1 } 2 n × m .
We implemented these attacks such that they can operate on batches of problems, however we omit this at this point in the following for an improved readability.
Algorithm E.1 describes in detail SAT and DEL (see Proposition 1).
The difference of the adjacency matrix A to the adversarially perturbed version ˜ A is modelled via the perturbation matrix M : ( ˜ x = ˜
A =
A ⊕ M ).
We then optimize over M .
During the SAT attack, we allow deletions and additions of edges, as long as one solution-preserving truth assignment per clause, described by the indicator T =
onehot ( Y ) , is preserved (line 7).
For the DEL attack only deletions are allowed, under the constraint that no clause can be fully deleted (lines 9 & 10).
Additionally, a global budget is enforced (line 6).
For details on the budget as well as other hyperparameters of the attack, we refer to Table E.1.
For the ADC attack described in Algorithm E.2, the perturbation matrix M ∈ { 0 , 1 } 2 n × ˜ m describes additional clauses appended to the original problem (line 4).
The attack can freely optimize over M but the number of literals/edges.
∆ is enforced s.t.
on average each clause contains as many literals as a clause in A (line 6).
[Table] Because the attack optimizes over a set of discrete edges with a gradient based method, similarly to (Xu et al., 2019), we relax the edge weights to [0 , 1] during the attack.
Before generating the perturbed problem instance ˜ A , we sample the discrete M (cid:48) from a Bernoulli distribution where the entries of the matrix M represent the probability of success.
In contrast to (Xu et al., 2019) we sample 19 instead of 20 times but add an additional sample that chooses the top elements in M .
Thereafter, we take the sample that maximizes the loss.
Moreover, our projection differs slightly from the one proposed by Xu et al.
(2019) since we iteratively enforce the budget instead of performing a bisection search.
Data: Adjacency A ∈ { 0 , 1 } 2 n × m , edge budget ∆ , steps s , learning rate α , solution T ∈ { 0 , 1 } 2 n × m ,
SAT model f θ , label y
Result:
Perturbed Adjacency ˜
A 1 Initialize M ← 0 2 n × m , 2 for t ∈ { 0 , 1 , . .
, s − 1 } do if y then ˜
A ← A ⊕ M else ˜
A ← A − M
M ← update ( M , α, ∇ (cid:96) ( f θ ( ˜ A ) , y )
M ← project-budget( M , ∆ ) if y then M ← M ∗ T else
M ← M ∗ A
M ← ensure-no-del( M ) end 12 end 13 M (cid:48)
← sample( M ) 14 ˜
A ← A ⊕ M (cid:48)
Data: Adjacency A ∈ { 0 , 1 } 2 n × m , clause budget ω , steps s , learning rate α , SAT model f θ
Result:
Perturbed Adjacency ˜
A 1 Initialize M ← 0 2 n × ˜ m , 2 ∆ ← avg( A , axis = 0) ∗ ˜ m 3 for t ∈ { 0 , 1 , . . .
, s − 1 } do ˜
A ← append( A , M )
M ← update ( M , α, ∇ f θ ( ˜ A ) , y ))
M ← project-budget( M , ∆ ) 7 end 8 M (cid:48) ← sample( M ) 9 ˜
A ← append( A , M (cid:48) )
Computational complexity.
All three attacks operate for s steps.
Under the assumption that the models are linear w.r.t.
the number of edges in the graph representation of the problem (for forward and backward pass), each step has a time complexity of O ( mn ) .
Therefore, the overall attack has a time complexity of O ( nms ) .
Under the same assumption for the memory requirements of ∇ M (cid:96) , the total space complexity is O ( nm ) .
F TSP A TTACK D ETAILS
For simplicity we only discuss the case of metric TSP.
This is also what the used neural sovers are mainly intended for.
The TSP attack is actually implemented in a batched fashion.
However, for an improved readability we omit the details here.
We refer to table Table F.1 for details on the attack hyperparameters.
DTSP model.
The DTSP model employs a GNN to predict whether there exists a route of certain cost on an input graph (Prates et al., 2019).
The model sequentially updates the embeddings of both nodes and edges that were initialized by a 3-layer MLP with the coordinates as input.
After multiple message passing steps, the ﬁnal prediction is then based on the aggregation of the resulting edge embeddings.
We follow the training guidelines described in the paper and train on 2 20 graph samples.
The graphs are generated by sampling from the unit square ( x ∈ [0 , 1] n × 2 ) and solutions are obtained with the concorde solver (Applegate et al., 2006).
Dual problem sets are built from each graph for training, validation and test data by increasing and decreasing the true cost by a small factor.
ConvTSP model.
The ConvTSP model aims at predicting the optimal route Y =
σ over a given graph via a GCN (Joshi et al., 2019).
It predicts a probability map over the edges indicating the likelihood of an edge being present in an optimal solution.
These lay the basis for different solution decoding procedures.
We employ their greedy search method, where the graph is traversed based on the edges with the highest probabilities.
We follow the training procedure described in the paper as well as the data generation technique.
Data is generated the same way as for the DTSP model, however the target represents binary indicators whether an edge is present in the optimal solution.
During training, the binary cross entropy between the target and the probability maps over the edges is minimized, and a solution-decoding technique is only applied during inference. Optimization problem.
We again restate the optimization problem from Eq.
1 for the speciﬁc case of TSP. We add ∆ nodes to the original problem x ∈ [0 , 1] n × 2 to create a larger, perturbed problem [Table] instance x ∈ [0 , 1] ( n +∆) × 2 which maximizes the loss:
[Equation]
[Equation]
[Equation]
x i with i > n Proposition 2 holds
[Equation]
F [Equation] Attack details. The input of the TSP is represented by the coordinates x ∈ [0 , 1] n × 2 for the n nodes.
Additionally, we know the near-optimal route Y obtained with the Concorde solver which we use as ground truth.
During the TSP attack we add additional adversarial nodes Z ∈ [0 , 1] ∆ ×to the input problem x .
The adversarial nodes are initialized by randomly sampling nodes until they fulﬁll the constraint from Proposition 2 (line 1).
For the attack, as described in Algorithm F.1, we operate solely on the coordinates of the adversarial nodes and assume that the model converts the coordinates into a weighted graph.
For the DTSP model, we additionally calculate the updated route cost c 0 (accordingly to y ) and append it to the input.
For simplicity, we omitted this special case in
Algorithm F.1.
Moreover, for the DTSP we do not pass ˜ Y to the loss; instead, we use the decision label y that is kept constant throughout the optimization.
We then obtain the updated coordinates Z for the perturbed solution ˜ Y (lines 5-6).
In the project step (line 7), we only consider the coordinates in Z that violate the constraint.
As discussed in § 5.2, we perform gradient descent on the constraint due to its non-convexity.
We decide against optimizing the Lagrangian since this would require evaluating the neural solver f θ and, therefore, is less efﬁcient.
For the projection, we update the adversarial node i that violates the constraint with
Z i ← Z i − η ∇ Z i (cid:20) ω ( P, Z i ) + ω ( Q, Z i ) − ω ( P, Q ) − (cid:18) min A,
B ∈ x ω ( A, Z ) + ω ( B, Z ) − ω ( A, B ) (cid:19)(cid:21) until the constrain is fulﬁlled again but for at most three consecutive steps.
Algorithm F Data: Node Coords x ∈ [0 , 1] n × 2 , steps s , learning rate α , TSP model f θ ,
Adversarial Coords Z ∈ [0 , 1] ∆ ×
Result:
Perturbed Node Coords ˜ x ∈ [0 , 1] ( n +∆) ×
Initialize Z ← random-allowed-point () 2 for t ∈ { 0 , 1 , . . .
, s − 1 } do mask ← is-constraint-fulﬁlled ( Z , x ) ˜ x
← append( x , Z [ mask ] )
˜
Y ← update-solution (˜ x , Y )
Z ← update ( Z , α, ∇ (cid:96) ( f θ ( W ) , ˜ Y ))
Z ← project ( Z , x ) ; 8 end 9 return ˜ x , ˜ Y
Computational complexity.
We again assume that the model has a linear time and space complexity.
This time it is linear w.r.t.
the O ( n 2 ) number of elements in the distance matrix between the input coordinates.
The most costly operation we add, is the check if the constraint is violated or not for each of the potentially added nodes ∆ .
This operation has a time and space complexity of [Table] O ( n 2 ) since we need to evaluate the distances to all nodes (except the special case where both nodes are on the convex hull but are non-adjacent, see Corollary 2).
However, checking the constraint has the same complexity as the neural solver.
Therefore, the overall space complexity turns out to be O (∆ n 2 ) = O ( n 2 ) and the time complexity is O ( n 2 s ) with the number of attack steps s .
G O FF -T HE -S HELF SAT S OLVERS : T IME C OST OF P ERTURBED I NSTANCES
Comparison SAT Solvers.
We test two SAT-solver’s runtime on all attacks for both clean an perturbed samples from the 50-100 dataset to better understand the difﬁculty of the samples for nonneural SAT solvers.
Table G.1 shows the mean as well as the standard deviation of the Glucose (Audemard & Simon, 2009) as well as the MiniSAT (S ¨ orensson & Een, 2005) solver.
The results suggest that both solvers can more quickly ﬁnd that a sample is unsatisﬁable for perturbed samples from both the DEL and the ADC attacks.
For perturbed satisﬁable samples, the runtime increases compared to the clean sample for the MiniSAT solver while Glucose solver needs about the same time as for the clean sample to ﬁnd a solution.
H C ONVERGENCE N EURO SAT
Complementary to Fig.
5, we also present the convergence for 100 randomly chosen satisﬁable instances from the 10-40 dataset.
We see that the observations drawn from Fig.
5 also hold here: (1) rarely an instance is predicted as SAT despite the moderate budget of perturbing 5% of the literals and (2) if NeuroSAT identiﬁes the sample as satisﬁable it requires more message passing steps to do so.
[Figure]
I E FFICIENCY N EURO SAT A TTACKS
To better understand the efﬁciency of the NeuroSAT attacks and how many gradient update steps are needed to ﬂip the model’s prediction, we show additional results for the attack strength over several attack step budgets s in Fig.
I.1.
The indicated number of gradient update steps s are taken during the attack, while retaining the learning rate from Table E.1 and early stopping on an instance level.
The results in Fig.
I.1 show that only 1 gradient update step sufﬁces for the SAT attack to decrease the accuracy from 79% to 26%.
Compared to the results from Fig. 2, where the random attack draws a single sample and only decreases the accuracy to 74%, the importance of the guidance through the gradient update becomes apparent.
For the DEL attack on the NeuroSAT model, the attack strength converges around 200 steps, showing again the imbalance between the labels with regards to how many misclassiﬁcations the attack can force.
[Figure]
loss
[Table]
SAT
[Table]
In Table I.1 we further compare the efﬁciency of the attacks to the random baseline.
Random samples are drawn until the loss matches or exceeds that of the adversarially perturbed sample.
We introduce a cut-off at 20000 samples and report the mean number of samples for the SAT and DEL attack on the budgets ∆ = 0 . 01 and ∆ = 0 . 05 .
We can observe that the additional computational cost of generating adversarial samples is justiﬁed, given the random baseline performs comparably well only with several thousand random samples drawn.
While the efﬁciency of the proposed approach in general is important to consider for practical relevance, it is not sufﬁcient to only compare to the random baseline in terms of sample efﬁciency.
There are several other aspects to consider, like additional computational and storage overhead for a larger, denser dataset or to what extent a larger dataset of random or adversarial examples can improve generalization.
J Q UALITATIVE R ESULTS SAT
To illustrate the changes to SAT problems, we provide an example for a successful attack on a small and fairly simple problem (from SAT-3-10).
While the model recognizes the problem below as satisﬁable with 100% conﬁdence, the SAT attack perturbes it (modiﬁed clauses highlighted blue) such that the model votes ’satisﬁable’ with only 0.59% conﬁdence.
[Table]
K Q UALITATIVE R ESULTS
[Equation]
In this section we complement the ﬁgures Fig.
7 and Fig.
9 with further examples.
In Fig.
K.1, we plot further examples for the attack on DTSP and further examples for ConvTSP in Fig.
K.2.
[Figure] [Figure]