ROCO: A G ENERAL F RAMEWORK FOR E VALUATING R OBUSTNESS OF C OMBINATORIAL O PTIMIZATION S OLVERS ON G RAPHS
A BSTRACT
Solving combinatorial optimization (CO) on graphs has been attracting increasing interests from the machine learning community whereby data-driven approaches were recently devised to go beyond traditional manually-designated algorithms.
In this paper, we study the robustness of a combinatorial solver as a blackbox regardless it is classic or learning-based though the latter can often be more interesting to the ML community.
Specifically, we develop a practically feasible robustness metric for general CO solvers.
A no-worse optimal cost guarantee is developed as such the optimal solutions are not required to achieve for solvers, and we tackle the non-differentiable challenge in input instance disturbance by resorting to black-box adversarial attack methods.
Extensive experiments are conducted on 14 unique combinations of solvers and CO problems, and we demonstrate that the performance of state-of-the-art solvers like Gurobi can degenerate by over 20% under the given time limit bound on the hard instances discovered by our robustness metric, raising concerns about the robustness of combinatorial optimization solvers.
I NTRODUCTION
The combinatorial optimization (CO) problems on graphs are widely studied due to their important applications including aligning cross-modality labels (Lyu et al., 2020), discovering vital seed users in social networks (Zhu et al., 2019), tackling large-scale knapsack problems (Zhang et al., 2020) and scheduling jobs (Mao et al., 2019) in data centers, etc.
However, CO problems are non-trivial to solve due to the NP-hard challenge, whereby the optimal solution can be nearly infeasible to achieve for even medium-sized problems.
Existing approaches to practically tackle CO include heuristic methods (Van Laarhoven & Aarts, 1987;
Whitley, 1994), powerful branch-and-bound solvers (Gurobi
Optimization, 2020; The SCIP Optimization Suite 8.0, 2021; Forrest et al., 2022) and recently developed learning-based models (Khalil et al., 2017; Mao et al., 2019; Kwon et al., 2021).
Despite the success of solvers in various combinatorial tasks, little attention has been paid to the vulnerability and robustness of combinatorial solvers.
As pointed out by (Yehuda et al., 2020), we cannot teach a perfect solver to predict satisfying results for all input CO problems.
Within the scope of the solvers and problems studied in this paper, our results shows that the performance of the solver may degenerate a lot given certain data distributions that should lead to the same or better solutions compared to the original distribution assuming the solver works robustly.
We also validate in experiments that such a performance degradation is neither caused by the inherent discrete nature of CO.
Such a discovery raises our concerns about the robustness (i.e. the capability to perform stably w.r.t.
perturbation on problem instances) of combinatorial solvers, which is also aware by (Varma & Yoshida, 2021; Geisler et al., 2021).
However, Varma & Yoshida (2021) focuses on theoretical analysis and requires the optimal solution, which is infeasible to reach in practice.
Geisler et al.
[Table]
applies models for solvers that can often be undifferentiable also
, 2018), where the expected score distribution the challenges and initiatives of evaluating the robustness of existing as follows:
1) Robustness metric without optimal solutions : The underlying NP-hard challenge of combinatorial optimization prohibits us from obtaining the optimal solutions.
However, the robustness metric proposed by ( Varma & Yoshida requires optimal solutions, making this metric infeasible in practice.
In this paper, we propose a problem modification based robustness evaluation method, whereby the solvers’ performances are evaluated on problems that can guarantee no worse optimal costs .
That is, the optimal solution of the modified problem instance gives better objective value than the original problem, which resolves the needs of the optimal solution.
Robustness metric of general (non-differentiable) solvers : Despite the recently developed deep-learning solvers, most existing solvers are non-differentiable due to the discrete combinatorial nature.
Thus, the gradient-based robustness metric by Geisler et al.
has the limitation to generalize to general solvers
In this paper, we develop a reinforcement learning (RL) based attacker that modifies the CO problems to get hard problem instances (where the solvers perform poorly) with no worse optimal costs.
Regarding arbitrary types of solvers as black-boxes, the RL agent is trained by policy gradient without requiring the solver to be differentiable.
Our framework also owns the flexibility where the agent can be replaced by other search schemes e.g. simulated annealing.
Being aware that many CO problems can be essentially formulated as a graph problem and there are well-developed graph-based learning models (Scarselli et al., 2008; Kipf & Welling, 2016;
Velickovic et al., 2017), the scope of this paper is restricted within combinatorial problems on graphs following (Khalil et al., 2017; Wang et al., 2021).
To develop a problem modification method that achieves no worse optimal costs , we propose to modify the graph structure with a problem-dependent strategy exploiting the underlying problem natures.
Two strategies are developed concerning the studied four combinatorial problems: 1) loosening the constraints such that the feasible space can be enlarged while the original optimal solution is preserved; 2) lowering the cost of a partial problem such that the objective value of the original optimal solution can become better.
Such strategies are feasible when certain modifications are performed on the edges, and tackle the challenge that we have no access to the optimal solutions.
The generated data distributions can also be close to the original data by restricting the number of edges modified.
The graph modification steps are performed by our proposed attackers that generate problem instances where the solvers’ performances degenerate, and we ensure the no worse optimal costs guarantee by restricting the action space of the attackers.
Our framework can be seen as the attack on the solvers and how much the worst cases near the clean instances will do harm to the solver are considered as the robustness.
Tbl.
1 compares our framework to classical works on adversarial attacks for images/graphs.
The highlights of this paper are:
1) We propose Ro bust C ombnaotorial O ptimization (ROCO), a general framework (see Fig. 1) to empirically evaluate the robustness of CO solvers on graphs.
It is performed without requiring optimal value or differentiable property.
We design a novel robustness evaluation method with no worse optimal costs guarantee to eliminate the urgent requirements for the optimal solution of CO problems.
Specifically, we develop a reinforcement learning (RL) based attacker combined with other search-based attackers to regard the solvers as black-boxes, even they are non-differentiable solvers.
3) Comprehensive case studies are conducted on four common combinatorial tasks: Directed Acyclic Graph Scheduling, Asymmetric Traveling Salesman Problem, Maximum Coverage, and Maximum
Coverage with Separate Coverage Constraint, with three kinds of solvers: traditional solvers, learningbased solver and specific MILP solver like Gurobi.
The results presented in this paper may raise the community’s attention to the robustness, e.g. big performance drop for the commercial solver Gurobi though it is not a big surprise given the NP-hard nature of many CO problems.
It can also be connected to data augmentation for learning-based CO solvers or other methods like parameter tuning to improve solvers’ robustness (see Appendix K for more results and discussions).
We leave more advanced defense strategies for future work which is still an open problem (Yehuda et al., 2020).
Remark.
Note we somewhat ‘abuse’ the term ‘attack’ which in fact originates from the area of adversarial attack/defense e.g. Goodfellow et al.
Here we use it to express the concept of hard case mining on ‘clean’ (i.e. raw) instances.
In fact, we try to make an analogy between the attacks for image data and our case while we are also aware there are fundamental differences.
To ease readers’ potential confusion and make the presentation more precise, we will give the formal definition of attack in our context for CO solvers’ robustness evaluation in Def. 1 in Sec. 2.1.
The related work is deferred to Appendix A, which mainly refers to the solvers for combinatorial optimization on graphs including traditional and recent learning-based solvers, as well as adversarial attack and defense especially for graph learning.
T HE ROCO F RAMEWORK 2. A DVERSARIAL R OBUSTNESS FOR CO
Formulation.
In general, a CO problem aims to find the optimal solution under a set of constraints (usually encoded by a latent graph).
Here we formulate a CO problem Q as:
[Equation]
(1) where x denotes the decision variable (i.e. solution) that should be discrete, c ( x , Q ) denotes the cost function given problem instance Q and { h i ( x , Q ) } Ii =1 represents the set of constraints.
For example, in DAG scheduling, the constraints ensure that the solution x , i.e. the execution order of the DAG job nodes, lies in the feasible space and does not conflict the topological dependency structure of Q .
However, due to the NP-hard nature, the optimal solution x ∗ can be intractable within polynomial time.
Therefore, we use a solver ˆ x = f θ ( Q ) , which gives a mapping f θ : Q → X to approximate the optimal solution x ∗ .
In this work, θ are solvers’ parameters (e.g. weights and biases in a neural solver, hyperparameters in Gurobi), Q ∈ Q is the problem instance, and ˆ x ∈ X is the approximated solution.
ˆ x also lies in the feasible space that satisfies all the constraints.
Solvers’ robustness.
In this paper, we would like to raise the concern that the estimation capability of the solvers can be unstable and fail on certain hard instances.
Thus, we need methods to discover the non-trivial hard instances for the solvers as the robustness metric where a more robust solver can perform better on hard instances.
Here non-trivial means these hard instances are not obtained by trivial operations like increasing the size of problems.
In short, given a certain solver f θ , we design an attacker (i.e. perturbation model) g to modify the existing clean instances ˜ Q = g ( f θ , Q ) and discover hard instances around.
For example, the attacker can modify a limited number of constraints in
Q to get a new problem ˜ Q .
Besides working as a robustness metric, the discovered hard instances can guide the parameter setting of solvers, i.e. design solvers’ parameters θ for better performance (robustness) on hard instances, which is beyond the scope of this paper and we mainly focus on developing the first practically feasible robustness metric.
Some preliminary results on hard case utilization are presented in Appendix K.
Robustness metric.
A natural evaluation of the solvers’ robustness is to use the gap c (ˆ x , Q ) − c ( x ∗ , Q ) (Varma & Yoshida, 2021), where a narrower gap stands for a better performance.
However, the optimum x ∗ is usually unavailable due to the NP-hardness.
In this paper, we propose a robustness evaluation method without requiring the optimal solution, and the attack is defined by:
Definition 1 (Successful Attack) .
Given a solver f θ and a clean CO problem instance Q , we obtain a new problem ˜ Q = g
( f θ , Q ) by the attacker g s.t.
the optimal cost value after perturbation will become no-worse, i.e. c
( ˜x ∗ , ˜ Q ) ≤ c ( x ∗ , Q ) , where ˜x ∗ denotes the new optimal solution.
A successful attack occurs when c ( f θ ( ˜ Q ) , ˜ Q ) >
c ( f θ ( Q ) , Q ) .
[Figure]
The intuition is that we restrict the attacker to discover a new problem instance with no-worse optimal cost value, such that the gap between the solver and the optimal will be definitely enlarged if the solver gives a worse cost value:
[Equation]
which represents a successful attack (see Fig. 1(b))
Attacker’s target.
In conclusion, the attacker should maximize the solver’s cost value after perturbation while conforming to the no-worse restriction.
Therefore, we formulate our attack objective:
[Equation]
As a consensus in the filed of robustness, the robustness definition should be against the attacker’s objective (Carlini et al., 2019; Chen et al., 2022).
Now with the attack objective, we can successively define solvers’ robustness quantitatively:
Definition 2 (Solvers’ Robustness) .
Given a certain solver f θ on a test dataset D consisted of CO problems, its robustness r is defined as the inverse of the expected worst case cost value after attacked by the attacker g under the no worse optimal cost constraint:
[Equation]
We add the negative sign in Eq.
4 so that a higher value denotes better robustness.
Attacker’s constraints.
Two types of strategies are developed to meet the no worse optimal cost constraint for the attackers: 1) Loosening the constraints of the problem.
In this way, the original optimal solution still lies in the feasible space and its cost is not affected, while the enlarged feasible space may include better solutions.
Therefore, the new optimal cost cannot be worse than the original one, and we discover this strategy effective for DAG scheduling (see Section 3.1).
Modifying a partial problem by lowering the costs.
Such a strategy ensures that the cost that corresponds to the original optimal solution can never become worse, and there are also chances to introduce solutions with lower costs.
We demonstrate its successful application in traveling salesman (see Section 3.2) and max coverage problems (see Section 3.3 & Appendix C).
Attack via graph modification.
As mentioned above, a CO problem Q can usually be encoded by a latent graph G =
( V, E ) .
For example, the DAG problem can be represented by a directed acyclic graph (see Fig. 4), s.t.
the graph nodes stand for the DAG jobs, the directed edges denote the sequential constraints for the jobs, and the node features represent the jobs’ completion time.
Hence, the perturbation to the problem instance can be conducted by modifications to the graph structure (Dai et al., 2018).
Besides, to get non-trivial hard instances around clean examples, the attack is limited within a given attack budget K , e.g. the attacker can remove ≤ K edges in a DAG problem Q .
Typically, in our setting, the graph modification is all conducted by modifying edges .
2. I MPLEMENTATIONS OF THE A TTACKER
Random attack baseline.
The baseline simply chooses and modifies K random edges sequentially, then calculates the new cost.
We compare other attackers (which achieve much higher attack effects) to the random attack baseline to claim that our framework can uncover far more hard instances than the inherent instability caused by the CO problems .
Reinforcement learning attacker (ROCO-RL) .
We use Eq.
3 as the objective and resort to reinforcement learning (RL) to find ˜ Q in a data-driven manner.
Specifically, we modify the graph structure and compute c ( f θ ( ˜ Q ) , ˜ Q ) alternatively, getting rewards that will be fed into the PPO (Schulman et al., 2017) framework and train the agent.
Given an instance ( f θ , Q ) along with a modification budget K , we model the attack via sequential edge modification as a Markov Decision Process (MDP):
State.
The current problem instance Q k (i.e. the problem instance ˜ Q obtained after k actions) is treated as the state.
The original problem instance Q
=
Q is the starting state.
Action.
The attacker is allowed to modify edges in the graph.
So a single action at time step k is a k ∈ A k ⊆ E k .
Here our action space
A k is usually a subset of all the edges E k since we can narrow the action space (i.e. abandon some useless edge candidates) according to the previous solution f θ ( Q k ) to speed up our algorithm.
Furthermore, we decompose the action space ( O ( | V | 2 ) → O ( | V | ) ) by transforming the edge selection into two sequential node selections: first selecting the start node, then the end node.
Reward.
The objective of the new CO problem Q k +1 is c ( f θ ( Q k +1 ) , Q k +1 ) .
To maximize the new cost value, we define the reward as the increase of the objective:
[Equation]
Terminal.
Once the agent modifies K edges or edge candidates become empty, the process stops.
The input and constraints of a CO problem can often be encoded in a graph G =
( V, E ) , and the PPO agent (i.e. the actor and the critic) should behave according to the graph features.
We resort to the
Graph Neural Networks (GNNs) for graph embedding:
[Equation]
where G k is the corresponding graph of problem Q k , the matrix n (with the size of node number × embedding dim) is the node embedding.
An attention pooling is used to extract a graph level embedding g .
The GNN model can differ by the CO problems (details in Appendix D).
After graph feature extraction, we design the actor and critic nets:
Critic.
It predicts the value of each state Q k .
Since it aims at reward maximization, a max-pooling layer is adopted over all node features which are concatenated (denoted by [ ·||· ] ) with the graph embedding g , fed into a network (e.g. ResNet block (He et al., 2016)) for value prediction:
[Equation]
Actor.
The edge selection is implemented by selecting the start and end node sequentially.
The action scores are computed using two independent ResNet blocks, and a Softmax layer is added to regularize the scores into probability [0 , 1] :
[Equation]
where n [ a 1 ] denotes node a 1 ’s embedding.
We add the feature vector of the selected start node for the end node selection.
For training, actions are sampled by their probabilities.
For testing, beam search is used to find the optimal solution: actions with top- B probabilities are chosen for each graph in the last time step.
Only those actions with top- B rewards will be reserved for the next search step (see Alg. 1).
Other attacker implementations.
We also implement three traditional attack algorithms for comparison: random search, optimum-guided search and simulated annealing.
i)
Random Search (ROCO-RA):
In each iteration, an edge is randomly chosen to be modified in the graph and it will be repeated for K iterations.
Different from the random attack baseline without any search process, we will run N attack trials and choose the best solution.
Despite its simplicity, it can reflect the robustness of solvers with the cost of time complexity O ( NK ) .
ii) Optimum-Guided Search (ROCO-OG): This method focuses on finding the optimum solution during each iteration.
We use beam search to maintain the best B current states and randomly sample M different actions from the candidates to generate next states.
The number of iterations is set to be no more than K . The time complexity is O ( BMK ) .
[Table]
iii)
Simulated Annealing (ROCO-SA): Simulated annealing (Van Laarhoven & Aarts, 1987) comes from the idea of annealing and cooling used in physics for particle crystallization.
In our scenario, a higher temperature indicates a higher probability of accepting a worse solution, allowing to jump out of the local optimum.
As the action number increases, the temperature decreased, and we tend to reject bad solutions.
The detailed process is shown in Appendix B and we will repeat it for N times.
SA is a fine-tuned algorithm and we can use grid search to find the best parameter to fit the training set.
Its time complexity is O ( NMK ) .
Tbl. 7 in Appendix I concludes the attackers’ property and complexity.
Since the three traditional algorithms are inherently stochastic, we run them multiple times.
The trained
RL attacker behaves deterministicly and we choose the best trained model.
S TUDIED CO T ASKS AND R ESULTS
We conduct experiments on four representative CO tasks: Directed Acyclic Graph Scheduling, Asymmetric Traveling Salesman Problem, Maximum Coverage and Maximum Coverage with Separate
Coverage Constraint.
Considering the page limitation, the details of the last task is postponed to
Appendix C. We also test the robustness of representative solvers including heuristics (Helsgaun, 2017; Grandl et al., 2014), branch-and-bound solvers (Forrest et al., 2022; The SCIP Optimization
Suite 8.0, 2021; Gurobi Optimization, 2020) and learning-based methods (Kwon et al., 2021).
The detailed graph embedding methods for the four tasks is shown in Appendix D. In Appendix
I, we provide training and evaluation hyperparameters of different solvers for fair time comparison and reproducibility.
Experiments are run on our heterogeneous cluster with RTX 2080Ti and RTX 3090.
Metrics.
The quantitative robustness has been defined in Def. 2, and here we propose a more intuitive metric of robustness in terms of ratios, which is also against the attacker’s objective.
Typically, we just adopt the relative change rate between the cost value after attacked and the cost of the incumbent optimal solution, i.e. the best solution we can find, so that the ratio is proportional to the attacker’s objective.
In the result tables, we report the raw cost value of different solvers under different attacks.
Besides, we also report the robustness ratio for ROCO-RL, which is the best attacker in most cases.
3. T ASK I: D IRECTED A CYCLIC
Many task scheduling systems formulate the job stages and their dependencies as a Directed Acyclic
Graph (DAG)
(Saha et al., 2015; Chambers et al., 2010), as shown in Fig.
The data center has limited computing resources to allocate the jobs with different resource requirements.
These jobs can run in parallel if all their parent jobs have finished and the required resources are available.
Our goal is to minimize the makespan (i.e. finish all jobs as fast as possible.)
Solvers.
We choose three popular heuristic solvers as our attack targets.
First, the Shortest Job
First algorithm chooses the jobs greedily with minimum completion time.
Second, the Critical Path algorithm finds the bottlenecks and finishes the jobs in the critical path sequence.
Third, the Tetris
[Table]
[Figure]
et al., [Figure] job dependencies, and removing edges will relax the constraints.
After removing existing edges in a DAG, it is obvious that the new solution will be equal to or better than the original one since there are fewer restrictions.
Therefore, in DAG scheduling tasks, the attack model selectively removes existing edges.
Dataset.
We use TPC-H dataset ( http://tpc. org/tpch/default5.asp ), composed of business-oriented queries and concurrent data modification.
Many DAGs referring to computation jobs, have tens or hundreds of stages with different duration and numbers of parallel tasks.
We gather the DAGs randomly and generate three different sizes of datasets, TPC-H-50, TPC-H-100,
TPC-H-150, each with 50 training and 10 testing samples.
The DAG nodes have two properties: execution time and resource requirement.
Results and analysis.
Tbl.
2 reports the results of our four attack methods.
In the designed attack methods, RL outperforms other learning-free methods in most cases, illustrating the correctness of our feature extraction techniques and training framework.
It is worth noting that even the simplest random search (ROCO-RA) can cause a significant performance degradation to the CO solvers, showing their vulnerability and the effectiveness of the attack framework.
Besides, all four attack methods perform better than the random attack baseline, which usually leads to a shorter makespan than the clean instances (unsuccessful attack), proving that our attack framework can find more hard instances than those because of the inherent instability of CO problems.
Fig.
2 (left) shows the attack ratio for different solvers on TPC-H-50.
In general, the attack ratio increases then flattens out w.r.t.
attack budgets, corresponding with the fact that we can find more hard instances under a larger searching space (but harder to find new successfully attacked instances as we remove more constraints).
Fig.
2 (right) demonstrates the effect of attack budgets on different sizes of datasets using the solver Tetris.
The figure illustrates that attack ratios on smaller datasets tend to flatten first (larger datasets allow for more attack actions), which can guide us to select the right number of attack actions for different datasets.
We also conduct experiments to verify the generalizability of
ROCO-RL in Appendix L, which shows that our attacker can be trained efficiently on small problem instances, then be directly used on larger problems.
3. T ASK II: A SYMMETRIC T RAVELING S ALESMAN
The classic traveling salesman problem (TSP) is to find the shortest cycle to travel across all the cities.
Here we tackle the extended version asymmetric TSP (ATSP) for generality.
Solvers.
The robustness of four algorithms are evaluated: First, Nearest Neighbour greedily adds the nearest city to the tour.
Second, Furthest Insertion finds the city with the furthest distance to [Table] existing cities in the tour and inserts it.
Third, Lin-Kernighan Heuristic (LKH3) (Helsgaun, 2017) is the traditional SOTA TSP solver.
Finally, Matrix Encoding Networks (MatNet) (Kwon et al., 2021) claims as a SOTA learning-based solver for ATSP and flexible flow shop (FFSP).
Attack model.
The attack is to choose an edge and half its value, by which we get a no-worse theoretical optimum since the length of any path will maintain the same or decrease.
To reduce the action space, we will not select the edges in the current path predicted by the solver at the last time step.
An attack example is shown in Fig.
5 in Appendix E.
Dataset.
The dataset comes from (Kwon et al., 2021), consisting of ‘tmat’ class ATSP instances which have the triangle inequality and are widely studied by the operation research community (Cirasella et al., 2001).
We solve the ATSP of three sizes, 20 , 50 and 100 cities.
The distance matrix is fully connected and asymmetric, and each dataset consists of 50 training samples and 20 testing samples.
Results and analysis.
Tbl.
3 reports the attack results of four target solvers.
where the RL based attack outperforms other methods in most cases.
Note that LKH3 is the most robust among all compared solvers.
We attribute the phenomenon to two aspcets of LKH3: Different initial states-
LKH3 will take 4 runs with diffenent initial states in our setting; Step by step improvement-
Specifically, LKH uses k-opt (Lin, 1965) to iteratively reorder and optimize the result tour sequence.
The introduction of randomization and optimization should contribute to the strong robustness of LKH3.
Inspired by this idea, we conduct experiments to improve the robustness of Nearest Neighbor and Furtherest
Insertion in Tbl.
9 and Tbl.
MatNet generates different random problem instances for training in each iteration.
The comparison between “MatNet” and “MatNet (fixed)” proves that the huge amount of i.i.d.
training instances promotes both the performance and the robustness of the model.
As Yehuda et al.
(2020) points out, learning-based solvers for CO cannot be trained sufficiently due to the theoretical limitation of data generation.
Thus, the training paradigm with large unlabeled data can be a good choice for learning-based CO solvers.
Besides, feeding hard samples discovered by
ROCO to “MatNet (fixed)” to conduct adversarial training may also promote robustness (see Tbl. 11).
3. T ASK III: M AXIMUM C OVERAGE
The maximum coverage (MC) problem is widely studied in approximation algorithms.
Specifically, as input we will be given several sets S =
{ S 1 , S 2 , ..., S m } and a number k .
The sets consist of weighted elements e ∈ E and may have overlap with each other.
Our goal is to select at most k of these sets such that the maximum weight of elements are covered.
The above problem can be formulated as follows, where w ( · ) :
E → R denotes the weight of a certain element.
[Equation]
Solvers.
It has been shown that the greedy algorithm choosing a set that contains the largest weight of uncovered elements achieves an approximation ratio of (1 − 1 e ) (Hochbaum, 1996).
Besides, the greedy algorithm has also been proved to be the best-possible polynomial-time approximation algorithm for maximum coverage unless P=
NP (Feige, 1998).
So we choose it to serve as our first attack target.
Besides, the maximum coverage problem is quite suitable to be formulated as an integer linear program (ILP).
Using Google ORTools API ( https://developers.google.com/ [Table] optimization ) , we transform Eq.
9 into the ILP form and attack three other general-purpose solvers: Gurobi (the SOTA commercial solver) (Gurobi Optimization, 2020), SCIP (The SCIP
Optimization Suite 8.0, 2021), and CBC (an open-sourced MILP solver written in C++)
(Forrest et al., 2022), to find their vulnerabilities.
These solvers are set the same appropriate time limit.
Calibrated time.
In line with Nair et al.
(2020), we use calibrated time to measure running time of task solving to reduce the interference of backend running scripts and the instability of the server.
The main idea is to solve a small calibration MIP continuously and use the average running time to measure the speed of the machine when solving different instances.
Details are given in Appendix G.
Attack model.
The MC problem can achieve a no-worse optimum if we add new elements into certain sets, as we can cover more elements while not exceeding the threshold k .
MC can be treated as a bipartite graph, where edges only exist between a set node and an element node inside the set.
So our attack model is to add edges between set nodes and elements nodes, leading to a theoretically better optimum but can mislead the solvers.
To reduce the action space, we only add edges for the unchosen sets, since adding edges for selected sets will not affect the solver’s solution.
Dataset.
For the MC problem, the distribution follows ORLIB ( http://people.brunel.
ac.uk/~mastjjb/jeb/info.html ).
The dataset consists of three set-element pairs 100-200, 150-300, and 200-400, each with 50 training and 20 testing samples.
Results and analysis.
Tbl.
4 records the attack results of four target solvers.
The RL based attack outperforms other methods across all cases in this scenario, showing the promising power of our feature extraction method and policy gradient model.
Besides, the attackers have even found some instances for CBC where it cannot figure out feasible solutions under the given time limit, strongly demonstrating the vulnerability of the solvers.
This effect also appears in the more challenging task Maximum Coverage with Separate Coverage Constraint MCSCC for Gurobi in Appendix C.
Typically, our attack model to MC problems is equivalent to modifying the objective coefficients of the original ILP model of MC problem, which might incur numerical issues for ILP solver such as unstable matrix factorization.
Since we can access the source code of CBC solver and SCIP solver respectively, we find that the SCIP solver is equipped with more modules improving numerical stability of solving process, such as scaling, presolving and primal heuristics, etc .
Besides, SCIP also benefits from implementing more classes of branching policy and cutting planes that aim to improve the dual bound, than CBC.
The experimental results are in line with the expectation for the tested
ILP solvers, where CBC solver performs worse than SCIP solver in terms of both solution quality and robustness.
Compared to the open-sourced solvers, the commercial solver Gurobi performs undoubtedly better in the perspective of both solution quality and robustness.
One step further, we conduct experiments on tuning the hyperparameters of Gurobi to improve its robustness in Tbl.
C ONCLUSION
We propose Robust Combinatorial Optimization (ROCO), the first general framework to evaluate the robustness of combinatorial solvers on graphs without requiring optimal solutions or the differentiable property.
Alleviating these two requirements makes it generalized and flexible, helping us conduct an extensive evaluation of the robustness of 14 unique combinations of different solvers and problems, showing the effectiveness and potential of our methods.
Moreover, we also provide insights to the robustness of solvers and conduct various experiments to improve their robustness, which we believe can encourage future works to develop more robust CO solvers.
A R ELATED W ORKS
Combinatorial optimization solvers.
Traditional CO algorithms include but not limited to greedy algorithms, heuristic algorithms like simulated annealing (SA) (Van Laarhoven & Aarts, 1987),
Lin–Kernighan–Helsgaun (LKH3) (Helsgaun, 2017), and branch-and-bound solvers like CBC (Forrest et al., 2022), SCIP (The SCIP Optimization Suite 8.0, 2021), and Gurobi (Gurobi Optimization, 2020).
Besides, learning-based methods have also been proposed to tackle these problems.
A mainstream approach using deep learning is to predict the solution end-to-end, such as the supervised model
Pointer Networks (Vinyals et al., 2015), reinforcement learning models S2V-DQN (Khalil et al., 2017) and MatNet (Kwon et al., 2021).
Though these methods did perform well on different types of
CO problems, they are not that robust and universal, as discussed in Bengio et al.
(2021), the solvers may get stuck around poor solutions in many cases.
The sensitivity of CO algorithms is theoretically characterized in Varma & Yoshida (2021), however the metric in Varma & Yoshida (2021) requires optimal solutions which are usually unavailable in practice concerning the NP-hard challenge in CO.
Adversarial attack for neural networks.
Since the seminal study Szegedy et al.
(2014) shows that small input perturbations can change model predictions, many adversarial attack methods have been devised to construct such attacks.
In general, adversarial attacks can be roughly divided into two categories: white-box attacks with access to the model gradients, e.g. Goodfellow et al.
Madry et al.
; Carlini & Wagner (2017), and black-box attacks, with only access to the model predictions, e.g. Ilyas et al.
; Narodytska & Kasiviswanathan (2016).
Besides image and text adversarial attacks (Jia & Liang, 2017), given the importance of graph-related applications and the successful applications of graph neural networks (GNN) (Scarselli et al., 2008), more attention is recently paid to the robustness of GNNs (Dai et al., 2018).
B T RADITIONAL A TTACK A LGORITHM
As an example of traditional attack algorithm, we list the pseudo code of SA in Algorithm 2.
Algorithm
Input: Input problem Q ; solver f θ ; max number of actions K ; action sample number M ;
Temperature decay ∆ T ; coefficient β ; bias eps .
Q 0 ← Q ; Q ∗ ← Q 0 ; T ← 1 ; # initial temperature for k ← 1 ..K do flag =
False; # if action is available for i ← 1 ..
M do
Random sample an edge ( x, y ) in edge candidates of Q k − 1 ; ˜ Q ← add/delete the edge ( x, y ) in Q k − 1 ; # new state by tentative action
P = exp( β ( c ( f θ , ˜ Q ) − c ( f θ ,Q k − 1 )+ eps T ) ; # action acceptance probability if Random(0 , 1) ≤ P then flag =
True; Q k ← ˜ Q ; Q ∗ ← Q k break ; if flag =
False then break ;
T = T · ∆ T ;
Output: Problem instance G ∗ .
C E XPERIMENTS -T ASK IV: M AXIMUM C OVERAGE WITH S EPARATE C OVERAGE C ONSTRAINTS
We also study a more complicated version of the former MC problem: the maximum coverage problem with separate coverage constraint (MCSCC), which is NP-hard as proved in Appendix F.2.
Specifically, the elements E can be classified into the black ones B and the white ones W (i.e.
E = W ∪ B , W ∩ B = ∅ ).
And our goal is to select a series of sets S ′ ⊆ S to maximize the coverage of black element weights while covering no more than k white elements.
The problem
[Figure]
u r ob i
S o l u t i on
T i [Figure]
[Figure]
Experiments can be represented by a bipartite graph, where edges only exist between a set node and an element node covered by the set.
The problem can be formulated as:
[Equation]
where w ( · ) denotes the weight of a certain element, C ( · ) denotes the set of elements covered by a set, and C + ( · ) and C − ( · ) denotes the set of elements in C ( · ) with black and white labels, respectively.
Solvers.
As an enhanced version of MC, the MCSCC problem is very challenging and here we propose three different solvers as the target for attacking.
First, the trivial Local algorithm iterates over the sets sequentially, adding any sets that will not exceed the threshold k .
Second, we adopt a more intelligent Greedy Average algorithm that always chooses the most cost-effective (the ratio of the increase of black element weights to the increase of a number of covered white elements) set at each step until the threshold k is exceeded.
Third, we formulate the problem into the standard ILP form (details in Appendix F.3) and solve it by Gurobi (Gurobi Optimization, 2020).
Attack model.
Our attack model chooses to add non-existing black edges that connect sets to black elements, which can lead to a theoretically better optimum since we can possibly cover more black elements while not exceeding the threshold k .
Further, in order to reduce the action space, we only select the unchosen sets, otherwise it will be useless since adding edges for selected sets will not affect a solver’s output solution.
A successful attack example is shown in Fig.
6 in Appendix E.
Dataset.
There is a lack of large-scale real-world datasets for MCSCC.
Thus, we keep the data distribution similar to the MC problem and randomly generate the dataset (Kwon et al., 2021) for training.
Specifically, the dataset consists of three configs of set-black element-white element: 30150-3K, 60-300-6K and 100-500-10K, each with 50 training samples and 10 testing samples.
More details about the dataset are presented in Appendix H.
Results and analysis.
Tbl.
5 shows the attack results on our simulated dataset.
Both traditional and
RL approaches achieve significant attack effects, while RL outperforms the others in most cases (especially for Gurobi).
We can see from Tbl.
5 that the degradation of Gurobi after attack is much more obvious than that in MC, possibly because the computation progress of MCSCC can include more vulnerable steps than MC.
Fig.
3 shows the average time cost (no time limit) for Gurobi in solving MCSCC problems.
The figure illustrates that solving time on the attacked instances is longer than those on the clean ones by a large margin, proving the promising power of our attack framework.
[Table]
D G RAPH E MBEDDING FOR SPECIFIC T ASKS
We discuss specific graph embedding metrics for different kinds of CO problems for reproducibility.
D. TASK I: DIRECTED ACYCLIC GRAPH SCHEDULING
Since the task is a directed acyclic graph, we use GCN to encode the state in the original graph and its reverse graph with inversely directed edges separately.
Then we concatenate the two node embeddings and use an attention pooling layer to extract the graph-level embedding for Eq. 6: n = [ GCN 1 ( G ) || GCN 2 ( reverse ( G ))] , g = AttPool ( n ) .
D. T ASK II: A SYMMETRIC T RAVELING S ALESMAN P ROBLEM
Since the graph is fully connected, we use GCN to encode the state in the graph.
Then we use an attention pooling layer to extract the graph-level embedding.
Eq.
6 becomes:
[Equation]
= [ GCN
[Equation]
[Equation]
D. T ASK III: M AXIMUM C OVERAGE
For the RL attack method, different from DAG and ATSP, MC has a unique bipartite graph structure.
Therefore, we resort to SAGEConv, which can handle bipartite data, for graph feature extraction.
As input, we classify the nodes into two classes (subsets I s and elements I e ) and associate them with two dimension one-hot tensors.
Besides, we add one more dimension for element nodes, which records their weights.
Eq.
6 becomes:
[Equation]
D. T ASK IV: M AXIMUM C OVERAGE WITH S EPARATE C OVERAGE C ONSTRAINTS
The graph embedding mechanism for MCSCC is exactly the same as MC since they can both be represented by a bipartite graph except that we add one more one-hot dimension to distinguish between black and white elements.
E S UCCESSFUL A TTACK E XAMPLES E. T ASK I: D IRECTED A CYCLIC G RAPH S CHEDULING
A successful attack example is shown in Fig.
E. T ASK II: A SYMMETRIC T RAVELING S ALESMAN P ROBLEM
A successful attack example is shown in Fig.
E. T ASK III: M AXIMUM C OVERAGE AND T ASK IV: M AXIMUM C OVERAGE WITH S EPARATE C OVERAGE C ONSTRAINTS
Successful attack examples are shown in Fig.
F F ORMULAS AND P ROOFS F. N O - WORSE O PTIMAL S OLUTION G UARANTEE
We will prove that the attack constrains can guarantee no-worse optimal solution.
Proposition 1.
Given the clean CO problem Q and the new instance ˜ Q obtained by the attacker.
If the attacker removes some of the constraints or makes constraints easier to reach, which is defined as the first strategy loosening the constraints , the optimal solution will be no-worse.
If the attacker decreases the value of the objective function obtained from some feasible solutions, which is defined as the second strategy lowering the cost , the optimal solution will be no-worse.
Proof.
We define x ∗ and ˜ x ∗ as the optimal solution, X and ˜ X as the feasible solution set in Q and ˜ Q .
According to the formulation in Eq.1, we have
[Equation]
The attacker modifies the constraints
h i to ˜ h
i in three ways: 1) Stay the same.
For each same solution x , ˜ h
i ( x , ˜ Q ) =
h i ( x , Q ) .
2) Remove the constraint.
The constraint h i ( x , Q ) is no longer in ˜ Q . 3)
Acquire lower value.
For each same solution x , ˜ h
i ( x , ˜ Q ) ≤ h i ( x , Q ) .
Hence, the less than zero inequality is easier to achieve.
Because if we have h i ( x , Q ) ≤ 0 , there must be ˜ h
i ( x , ˜ Q ) ≤ 0 .
Three parts of the constraints ensure each feasible solution x ∈ X satisfies the constraints in Q still satisfies the constraints in ˜ Q .
Therefore, we have X ⊆ ˜ X .
Since the cost function does not change and x ∗ ∈ ˜ X , the optimal solution in new instance ˜ Q can be at least the same.
Meanwhile,
The new feasible solution set ˜ X − X may provide a better solution.
In conclusion, the optimal solution will be no-worse.
The attacker decreases the cost function in the new instance ˜ Q where c ( x , ˜ Q ) ≤ c ( x , Q ) for each instance x .
The constraints stay the same so that X = ˜
X .
For x ∗ , there is c ( x ∗ , ˜ Q ) ≤ c ( x ∗ , Q ) and x ∗ ∈ ˜ X which means the optimal solution in ˜ Q will be no-worse.
F. MCSCC NP- HARD P ROVEMENT
We prove that the decision problem of MCSCC is NP-complete, thus the optimization problem of
MCSCC in our paper is NP-hard.
For the sake of our proof, here we redefine MCSCC and give definition of a traditional NP-Complete problem: Set Covering .
The decision problem of MCSCC .
Given a series of sets R = { R 1 , R 2 , ..., R n } along with an element set consisted of white and black elements E = W ∪B .
The sets cover certain elements (white
C − ( R i ) or black C + ( R i ) ) which have their weights w ( e ) .
Does there exist a collection of these sets
R ′ ⊆
R to cover ≥ M black elements weight while influence no more than k white elements?
Set Covering .
Given a set U of elements, a collection S 1 , S 2 , . . .
, S n of subsets of U , and integer k , does there exist a collection of ≤ k of these sets whose union equals to U ?
First, we need to show that MCSCC is NP.
Given a collection of selected sets R ′
=
{ R 1 , R 2 , . . .
, R m } , we can simply traverse the set, recording the covered black and white ele- ments.
Then, we can assert whether the covered black element number is no more than k and fraudulent monetary value is no less than m .
Since the certification process can be done in O ( n 2 ) , we can tell that MCSCC is NP.
Then, for
NP-hardness, we can get it reduced from Set Covering .
Suppose we have a Set Covering instance, we construct an equivalent MCSCC problem as follows:
• Create | U | black elements with weight 1, corresponding to elements in Set Covering .
• Create n sets, set C
+ ( R i ) =
S i .
• Connect each set to a different white element of weight 1.
• Set the white element threshold
k w equal to the set number threshold k s .
• Set black element weight target M = | U | .
Suppose we find a collection of sets which meet the conditions of MCSCC , then we select subsets in Set Covering S
i iff we select R i .
The total number of subsets is no more than k w ( k s ) since the influenced white element number is equal to |R ′ | .
The subsets also cover U since the covered black element weight (each with amount 1) is no less than | U | .
Similarly, we can prove that we can find a suitable subset collection for MCSCC if we have found a collection of subsets that meet the conditions of Set Covering .
So we can induce that Set Covering ≤ p MCSCC .
Thus, we have proved that the decision problem of MCSCC is NP-Complete.
F. MCSCC ILP F ORMULATION
As discussed in the main text, B denotes the set of black elements, W represents the set of white elements, S refers to the collection of sets and E denotes the element set.
Using notations above, we can translate Eq.
10 into standard ILP form as follows: max (cid:88) i ∈B
Y [ i ] × W [ i ] , s.t. (cid:88)
i ∈W Y [ i ] ≤ k for i = 1 . . .
|E| , ( Y [ i ] − 0 . 5)(0 .
5 − |R| (cid:88) j
=
X [ j ] × I ( i in E [ j ])) ≤(16) where X [ j ] ∈ { 0 , 1 } denotes set S j is chosen (1) or not (0), Y [ i ] ∈ { 0 , 1 } shows whether element i has been covered by the chosen sets.
Besides, W [ i ] ∈ R records the weight of the elements while
E [ j ] ⊆
E is the corresponding elements of set S j .
The third equation ensures the element binary variable Y [ i ] to be 1 iff it has been covered by a certain set (
if ∃ X [ j ] = 1 and element i ∈ E
[ j ] , then the formula in the second bracket is negative, ensuring Y
[ i ] to be 1; else if element i is not covered by any chosen sets, then the second formula is positive and Y
[ i ] must be 0).
G C ALIBRATED T IME
Considering the instability of the server and the interference of other running programs, calibrated time is designed to evaluate the server speed within a small period of time.
Specifically, we define the speed as the reciprocal of the Wall clock time to solve a given calibration MIP.
The calibration MIP is generated in the stable condition and solved K times to record the basic speed speed base .
When evaluating the other instances, we will first evaluate the calibration MIP K times to calculate the current speed speed now .
K is the number of samples, which is set as 20 in our experiments.
To fairly measure the solvers’ performance under different computational resources, take MC problems as an example, a given calibration MIP is also a specified MC problem to design time limits for different server speeds.
Specifically, the new time limit for the solver will be set as:
[Equation]
H MCSCC D ATASET
Besides the distribution rules of MC, we generate the MCSCC dataset mainly by two rules: 1) The black element weights are uniformly distributed in the range [0 , 1] .
The number of black elements is 5% of white elements.
The covered black element weights and wight element numbers of sets follow a Gaussian distribution with a large standard deviation, to enlarge the differences between the sets.
[Figure] [Figure]
[Figure] [Figure]
6+
[Figure]
Fig.
7 shows the black element weights covered by different sets in a 100-500-10K problem instance.
As we can see, few well-designed sets can cover most weights while the others can be regarded as complementary to these sets.
This design enlarges the difference between the different quality of solutions, therefore promoting the potential to be attacked.
I E XPERIMENT S ETUPS
Experiment environments.
DAG and ATSP experiments are run on a GeForce RTX 2080Ti while
MC and MCSCC experiments are run on a GeForce RTX 3090 (20GB).
We implement our models with Python 3.7, PyTorch 1.9.0 and PyTorch Geometric 1.7.2.
RL settings.
Tbl.
6 records the hyperparameters for RL during the training process.
Trust region clip factor is a parameter in PPO agent to avoid model collapse.
We also adopt some common policygradient training tricks like reward normalization and entropy regularization during training processes.
Attacker hyperparameters.
For fair comparison of different attackers and the consideration of RL inference time, the hyperparameters are set to ensure similar evaluation time across different attack methods.
According to the time complexity we calculate in Tbl. 7, here we specify the following parameters: number of iterations N , beam search size B and number of different actions M in each iteration.
• DAG : ROCO-RA N = 30 ; ROCO-OG B = 3 , M = 9 ; ROCO-SA N = 5 , M = 6 ; ROCO-RL B = 3 .
• ATSP : ROCO-RA N = 130 ; ROCO-OG B = 5 , M = 25 ; ROCO-SA N = 13 , M = 10 ; ROCO-RL B = 5 .
• MC : ROCO-RA N = 220 ; ROCO-OG B = 6 , M = 36 ; ROCO-SA N = 22 , M = 10 ; ROCO-RL B = 6 .
[Table] [Table]
[Table]
• MCSCC : ROCO-RA N = 250 ; ROCO-OG B = 6 , M = 36 ; ROCO-SA N = 25 , M = 10 ; ROCO-RL B = 6 .
J A NALYSIS OF A TTACK B UDGET
Unnoticeable Property.
The perturbation to the problem instance is conducted by modifications to the graph structure.
To get non-trivial hard instances around clean examples, the attack is limited within a given attack budget K , which means at most K edges can be modified in a graph.
Tbl.compares our framework to classical works on adversarial attacks for images and graphs.
Similar to the ϵ -Perturb budget (Goodfellow et al., 2015) in image classification task and the number of edges modification (Dai et al., 2018) in graph classification task, we use the number of edges modification budget to satisfy the unnoticeable property.
Tbl.
8 shows the ratio of budget K to the average number of edges in these tasks.
The low ratio shows that our modification to the original graph is minor.
Apart from this, we also show the relation between the attack effect and the attack budget in the next paragraph.
[Figure]
[Figure]
Sensitivity to the Budget K .
To evaluate sensitivity of the proposed method to the budget K across different tasks, we define attack percentage as the ratio of the performance loss in current edge budget to the the performance loss in the budget upper bound.
For example, in ATSP task, the budget upper bound is 10 and we can will get 0% attack percentage using 0 edge modification and 100% using
[Table]
[Table] [Table]
10 edge modification.
Fig.
8-11 show the relation between attack percentage to attack budget in different tasks.
In Fig.
8, the three solvers all reach the high attack percentage using few budget.
It may be more appropriate to reduce the budget because of marginal utility.
Corresponding to this, it is possible for LKH3 to improve the attack performance as the budget rises shown in Fig.
Therefore, the sensitivity to budget K can help set the appropriate attack budget in practice.
[Figure]
[Figure]
K R OBUSTNESS I MPROVEMENT
Apart from evaluating the robustness of the solvers
[Figure]
here we provide some preliminary yet positive results on improving solvers’ robustness, while a full solution is indeed beyond the scope of this paper as we focus more on practical robustness evaluation
[Figure]
[Table] [Table]
Adversarial Training.
We perform adversarial training (AT) for MatNet (fixed training data) on dataset ATSP-20.
Tbl.
11 presents our main results on AT.
Specifically, MatNet-1000 denotes we use 1000 fixed ATSP instances to train the MatNet, while MatNet-1000-AT represents that we use 1000 hard instances generated by the ROCO-RL attacker to adversarially train the MatNet.
The table shows that after AT, MatNet achieves obviously better results on the hard instances, at the cost of a bit performance degradation on the clean instances.
However, AT cannot compensate for the insufficiency of training data, as we can see that the performance of MatNet-1000-AT is still worse than MatNet-10000 on the hard examples.
The reason why we do not conduct AT on the original
MatNet is that it trains for 2000 epochs, each with 10000 randomly generated ATSP instances, which is too large a number for us to generate adversarial samples.
Notably, the results in the table are relatively close to each other which may introduce bias in the experiments.
This implies that we may further explore other CO problems to test the effectiveness of AT, which we leave for the future work.
LKH3-guided robustness improvement.
As we find from Tbl.
3 that LKH3 is the most robust solver for the ATSP problem, we try to adapt its randomization and optimization strategies to the
Nearest Neighbour and the Furthest Insertion algorithm to promote their robustness.
First, we develop Nearest Neighbour (Step by step improvement).
Instead of the well-designed optimization method k-opt (Lin, 1965) of LKH3, here we just greedily swap a pair of nodes on the solution obained by Nearest Neighbour for k times.
For each time we will enumerate all the node pairs and calculate the corresponding reduced tour length, then find the best pair to swap.
We do experiments for different number of swaps k on the dataset ATSP20.
The robustness ratios (the lower the better) in Tbl.
9 are calculated w.r.t.
the incumbent optimal solution.
The incumbent optimal tour length is 1.4595 obtained by LKH3.
We can find the clean tour length decreases, and the robustness increases with a larger value of k .
Another interesting phenomenon is that the solver with k
=and k = 8 have similar clean tour lengths, while the latter is obviously more robust than the former.
These results demonstrate that the optimization process does contribute greatly to the robustness of the solver.
Second, we develop Furthest Insertion (Different start state) .
The Furthest Insertion algorithm will be influenced by the initial city and generate different solutions accordingly.
Therefore, we can set s different cities as the initial city and choose the best solution as our result.
We do experiments for different number of start states s on the dataset ATSP20.
From Tbl.
10, we can find that the clean tour length decreases, and the robustness increases with a larger value of s , which proves that randomization does contribute to the robustness of the solver.
Hyperparameter tuning.
We use hard examples to tune the hyperparameters of Gurobi on the MC problem (problem size 100-200).
We refer to Gurobi’s Documentation in Gurobi Optimization (2020) and choose two important hyperparameters: MIPFocus and Heuristics, which are easily set by the python interface of Gurobi.
Specifically, MIPFocus (integer, default 0, set between 0 and 3) allows us to modify our high-level solution strategy.
By default, the Gurobi MIP solver strikes a balance between finding new feasible solutions and proving that the current solution is optimal.
A higher
MIPFocus will focus on proving optimality more.
Heuristics (double, default 0.05, set between 0 and 1) determines the amount of time spent in MIP heuristics.
Larger values produce more and better feasible solutions, at a cost of slower progress in the best bound.
We generate 100 hard examples from the testing dataset and tune the two hyperparameters using grid search for MIPFocus ∈ [0, 1, 2] and Heuristics ∈ [0, 0.05, 0.10].
As we can see from Tbl.
12, the robustness ratio (Clean Weight - ROCO-RL) /
Clean Weight declines almost as the hard example weight (averaged over the cost value on 100 hard examples) increases.
This shows that tuning the Gurobi hyperparameters is indeed useful to promote its robustness.
Moreover, we can see that the hard example weight is the largest when MIPFocus=1 and Heuristics=0.0, which shows that Gurobi needs to pay more attention to narrowing the optimal bound to better deal with the hard examples.
L G ENERALIZABILITY
One potential drawback of ROCO is its scalability.
Since the training of the proposed RL strategy requires calling of the solver at each timestep, the training process indeed takes a relative long time (taking at most one day in our largest dataset setting, acceptable compared to common RL pipelines).
However, the inference process is still efficient since we only need to take one forward pass with beam search.
To resolve the concern that ROCO cannot scale to graphs in real-world scenarios, we provide additional results to show ROCO-RL’s generalizability from small to large graphs.
[Table]
As we can see from Tbl. 13, the RL agents trained on small graphs can be used to modify problems with larger graph sizes, achieving great performance degradation for different solvers.
These results strongly support that ROCO can be trained on small graphs and then be used to measure robustness on large graphs.
M F URTHER D ISCUSSIONS
Limitations.
First of all, as our framework is designed for combinatorial optimization problems on graphs, it needs non-trivial reformulation to fit into CO problems cannot encoded by graphs.
Secondly, the principle of designing our attack action is based on the graph modification to achieve no-worse optimum, which may not be applicable to all CO problems.
Potential Negative Impacts.
As we propose the first general framework to evaluate the robustness of combinatorial solvers on graphs, this may be used by unscrupulous people to attack different solvers, increasing the burden on engineers to cope with the malicious attacks.
Outlook.
Our paradigm opens up new space for further research, including the aspects: 1) Utilizing the hard cases generated by our framework to design defense models, such as adversarial training and hyperparameter tuning.
2) Developing a more general attack framework for CO problems that cannot be represented by graphs. 3) Besides our studied tasks, the attacking idea of loosening constraints and lowering costs is also generalizable on a wide range of CO problems such as Maximum Cut and
Minimum Vertex Cover, which need further evaluation on their robustness.