Mind Your Solver! On Adversarial Attack and Defense for Combinatorial Optimization

1 INTRODUCTION . The combinatorial optimization ( CO ) problems are widely studied due to their importance in practice ( e.g . job scheduling , routing , matching , etc ) . In the last century , a variety of heuristic methods ( Van Laarhoven & Aarts , 1987 ; Whitley , 1994 ) are proposed to tackle these standing and often NP-hard problems . Driven by the recent development of deep learning and reinforcement learning , many learning-based methods ( Khalil et al. , 2017 ; Mao et al. , 2019 ; Kwon et al. , 2021 ) are also developed in this area , which show promising potential often for their cost-efficiency . Despite the success of solvers in various combinatorial optimization tasks , few attention has been paid to the vulnerability and robustness of combinatorial solvers , regardless of whether they are learning based or not . A line of relevant works aims at handling combinatorial optimization under uncertainty ( Buchheim & Kurtz , 2018 ) . However , to our best knowledge , ensuring the robustness of combinatorial solvers with slightly modified problem instances remains relatively unexplored . It is worth noting that many CO problems can be essentially formulated as a graph problem ( Khalil et al. , 2017 ; Bengio et al. , 2020 ) , hence it is attractive and natural to modify the problem instance by modifying the graph structure , to generate more test cases for solvers . In fact , vulnerability can often be an inherent challenge for CO solvers since the problem is often strong nonlinear and NP-hard . From this perspective , we consider attack and defense CO solvers in the following aspects . From the attack side , developing attack models can be useful for thoroughly evaluating a solver ’ s robustness . The solvers may be more fragile than the general impression : for traditional learning-free solvers , in some cases , their heuristics and hyperparameters may not be universal and stable enough such that a small change on problem condition or graph structure may deteriorate the performance notably . This also holds for recent machine learning based solvers as the model may be overfit and the objective landscape can be complex due to the inherent difficulty of discrete CO problems . As a result , it is imperative to develop defense mechanisms and techniques to improve the robustness of CO solvers , either for learning-based models or traditional ones , especially if the approach can be in black-box mode without knowing the details of the solvers . In particular , it is even desirable to develop out-of-box defense mechanism . Our hope is that this may be realized when the problem instance change1 involves only graph structure variation – which is often the case . 1Readers may argue that there are little deliberate attacks to CO solvers , while one can regard such attacks as the problem instance variation which can often happen in real-world e.g . when the network takes a small daily change in Directed Acyclic Graph ( DAG ) and Fraud Coverage problems as will be studied in our experiments . To this end , we present Robust Combnaotorial Optimization ( ROCO ) , a framework for testing and improving the robustness of a given combinatorial optimization solver . Table 1 compares our framework to classical works in images and graphs . Our attacker limits the number of attacked edges in the graph and guarantees that the optimal solution must not become worse . Our defender ensures that the new solution is also feasible for the pre-defended problem . The overview of ROCO framework is summarized in Fig . 1 . In summary , this paper makes the following contributions : 1 ) Given the fact that combinatorial problems can often be represented by underlying graphs , we propose to perform adversarial attacks toward CO solvers to deteriorate their solution quality . To our best knowledge , this is the pioneering work that formally studies adversarial attacks on combinatorial solvers , though their vulnerability has been occasionally recognized by the community . 2 ) We propose ROCO , an adversarial framework that consists of both attack and defense models on top of CO solvers . We design our attack models with both learning-based and traditional simulated annealing methods by slightly modifying the graph structures ( e.g . add , delete or modify edges ) . To increase the robustness of the combinatorial solvers , we further propose defense mechanism against attacks . Our attack and defense models are applicable to solvers regardless of learning-based or not . 3 ) We implement and apply our adversarial attack and defense models to three common combinatorial optimization tasks : Directed Acyclic Graph Scheduling , Asymmetric Traveling Salesman Problem and Fraud Coverage . The experimental results on black-box attack/defense show the effectiveness and generality of our approach . The source code will be made public available . 2 RELATED WORK . Combinatorial optimization . As a widely studied problem , there exist many traditional algorithms for CO , including but not limited to greedy algorithms , heuristic algorithms like simulated annealing ( SA ) ( Van Laarhoven & Aarts , 1987 ) or Lin–Kernighan–Helsgaun ( LKH3 ) ( Helsgaun , 2017 ) , as well as commercial solvers like Gurobi ( Gurobi Optimization , 2020 ) . Besides , driven by the recent development of deep learning and reinforce learning , many learning-based methods have also been proposed to tackle these problems . A mainstream approach using deep learning is to predict the solution end-to-end , such as the supervised model Pointer Networks ( Vinyals et al. , 2015 ) , reinforcement learning models S2V-DQN ( Khalil et al. , 2017 ) and MatNet ( Kwon et al. , 2021 ) . Though these methods did perform well on different types of COPs , they are not that robust and universal , as discussed in ( Bengio et al. , 2020 ) , the solvers may get stuck around poor solutions in many cases . Different from works ( Moon et al. , 2019 ; Zang et al. , 2020 ) which apply CO for attack against neural networks , we take an initiative on the adversarial attack and defense on CO. Adversarial attack and defense . Since the seminal study ( Szegedy et al. , 2014 ) showed that small input perturbations can change model predictions , many adversarial attack methods have been devised to construct such attacks . In general , adversarial attacks can be roughly divided into two categories : white-box attacks with access to the model gradients , e.g . ( Goodfellow et al. , 2015 ; Madry et al. , 2018 ; Carlini & Wagner , 2017 ) , and black-box attacks , with only access to the model predictions , e.g . ( Ilyas et al. , 2018 ; Narodytska & Kasiviswanathan , 2016 ) . Besides image and text adversarial attacks ( Jia & Liang , 2017 ) , given the importance of graph-related applications and the successful applications of graph neural networks ( GNN ) ( Scarselli et al. , 2008 ) , more attentions are recently paid to the robustness of GNNs . In the mean time , many defense strategies like adversarial training ( Ganin et al. , 2016 ; Tramèr et al. , 2020 ) have also been proposed to counter this series of attack methods . Since CO problems can usually be encoded by a graph structure and inspired by ( Dai et al. , 2018 ) , which develops an RL based attack policy towards GNNs , we propose a novel and flexible attack and defense framework for CO solvers using both heuristic and RL methods . Note that the recent adversarial graph matching ( GM ) network show how to fulfill attack or defense via perturbing or regularizing geometry property on the GM solver . ( Zhang et al. , 2020 ) degrades the quality of GM by perturbing nodes to more dense regions while ( Ren et al. , 2021 ) improves robustness by separating nodes to be distributed more broadly . However , the techniques are deliberately tailored to the specific problem and can hardly generalize to the general CO problems . Meanwhile they work in a white box mode while we aim to develop more flexible black box models . 3 COMBINATORIAL OPTIMIZATION WITH ATTACK AND DEFENSE . 3.1 PROBLEM FORMULATION . In general , a traditional CO problem Q defined on graph G = ( V , E ) can be formulated as : Q : min x f ( x|G ) s.t . hi ( x , G ) ≤ 0 , i = 1 , . . . , I ( 1 ) where x denotes the decision variable , f ( x|G ) represents the target function w.r.t . the specific CO problem and hi ( x , G ) denotes the set of constraints ( usually encoded in graphs ) . However , due to the NP-hard nature ( which is often the case in CO ) , it can be infeasible to find the optimal solution within polynomial time . Therefore , we denote a different solver S ( which gives the feasible solution f ( S ( Q ) |G ) ) to approach the global optimum f∗ ( Q ) . It is worth noting that the optimum f∗ ( Q ) of Eq . 1 will become no worse if we loosen part of the constraints hi since the previous decision variable x is still feasible under the new setting . Intuitively , we may expect the solver to give a better ( at least the same ) solution on the new problem Q′ . However , we will show in this paper that many solvers are vulnerable to such perturbations and their solutions can become worse under our attacks , despite the loose bound f∗ ( Q′ ) ≤ f∗ ( Q ) . Given a solver S and an original problem Q represented by a graph G , the adversarial attacker g is asked to modify the graph G into G′ to attack the solver S , such that : max G′ f ( S ( Q′ ) |G′ ) − f ( S ( Q ) |G ) s.t . G′ = g ( S , G ) , hence Q→ Q′ , f∗ ( Q′ ) ≤ f∗ ( Q ) , T ( G , G′ ) = 1 ( 2 ) Here T ( · , · ) → { 0 , 1 } is an equivalency indicator ( Dai et al. , 2018 ) that tells whether two graphs G and G′ satisfy a specified constraint . In short , the above equation tells that the attacker is aiming at making small modifications to the original graph , loosening the constraints while making the solver solution as bad as possible . In this paper , concretely our attacker g is allowed to modify edges ( e.g . adding or removing edges ) from G to construct the new graph . Accordingly , we define the equivalency indicator as : T ( G , G′ ) = I ( | ( E − E′ ) ∪ ( E′ − E ) | ≤ K ) ( 3 ) which ensures that the attacker can modify no more than K edges of the original graph . On the other hand , it is imperative to develop defense mechanism for against the above attacks . Notice that the attack methods we mentioned before have some degree of symmetry ( adding/deleting edges ) , we can simply do reverse operations for defense . For example , if we can relax the constraints by removing edges while worsening the solver ’ s solution , then we can add some edges ( constraints ) and get a better solution ( that is , the symmetry ) . Besides , the new solution under stronger constraints is surely feasible for the original graph ( then we can use it in the original graph to get f ( S ( Q′ ) |G ) ) . Hence , the new problem can be formulated as : min G′ f ( S ( Q′ ) |G ) − f ( S ( Q ) |G ) s.t . G′ = d ( S , G ) , hence Q→ Q′ , Hj ( G′ , G ) ≤ 0 , for j = 1 . . . J , T ( G , G′ ) = 1 ( 4 ) here the constraints Hj ( G′ , G ) ≤ 0 ensure that the feasible space of G′ is a subset of G. Concrete Examples . Fig . 2 shows the attack and defense of the Shortest Job First algorithm on DAG ( TSP and FC examples are in Appendix A ) . We remove an edge but get a worse finish time ( objective – the smaller the better ) . Then we add an edge for defense , which leads to a better solution . In this paper , we focus on black-box attack and defense , which means we have no idea on the solver . This setting is practical especially considering there are plenty of commercial solvers e.g . Gorubi and CPLEX etc . We leave white box attack and defense for future work .

Summary: This paper presents a framework of adversarial attack and defense for solvers to combinatorial optimization problems. In particular, the attacker can modify the problem instance by removing graph edges, while the defense side seeks to add edges to mitigate the attacking effect. The paper presents attack methods based on reinforcement learning and heuristic methods, and proposes a defense method based on reinforcement learning. Experimental studies are provided on three combinatorial problems.