USCO-Solver: Solving Undetermined Stochastic Combinatorial Optimization Problems

1 Introduction . Combinatorial optimization problems are not only of great theoretical interest but also central to enormous applications . Traditional research assumes that the problem settings ( i.e. , objective function ) are completely known [ 1 ] , but the reality is that the underlying system is often very complicated and only partial knowledge ( from historical data ) is provided . In a recommendation system , the item-user similarities could be unknown [ 2 ] , which makes it impossible to compute the optimal recommendation scheme . In the study of wireless communications , the backbone network depends on stochastic node connections [ 3 ] , incurring extra difficulties in designing management strategies . In the most general sense , we can formalize such scenarios by assuming that the objective function is governed by a configuration space associated with an unknown distribution , where our goal is to maximize the expected objective . For example , the configuration space may specify possible item-user similarities or candidate network structures . We call such problems as undetermined stochastic combinatorial optimization ( USCO ) problems . Since the distribution of the configuration space is unknown , one can adopt the learning-andoptimization strategy [ 4 ] , where the unknown distribution is first learned from data so that the subsequent optimization problem can be solved using existing methods . While such a strategy is natural and proved successful in several applications [ 5 ] , it may require a large amount of data to learn an excessive number of parameters – for example , the weight between each pair of nodes in a network or the similarity between each pair of user and item in a recommendation system . In the worst case , we may not even have access to such kind of data ( due to , for example , privacy issues 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . [ 6 ] ) . Furthermore , a more technical concern is that the learning process is often independent of the optimization process , causing the situation that optimizing the learned objective function does not produce a good approximation to the true objective function , which is theoretically possible [ 7 ] . These concerns motivate us to think of the settings that can eschew model learning and can address USCO problems aiming right at the approximation guarantees . To this end , we consider the regression between the input space and solution space , and wish to directly compute the solution for future inputs without learning the hidden objective function . USCO-Solver . In this paper , we present USCO-Solver , a general-purpose solver to USCO problems . Our framework is designed through two main techniques : randomized function approximation and approximate structured prediction . The key idea is to construct a hypothesis space composed of affine combinations of combinatorial kernels generated using random configurations , from which a large-margin machine is trained using input-solution pairs via approximate inference . The main advantage of USCO-Solver is that it can handle an abstract USCO problem as long as an oracle to solve its deterministic counterpart is given . Another significance of USCO-Solver lies in our use of combinatorial kernels , which suggests a novel and principled way for incorporating an approximation algorithm into a learning process . In doing so , we are able to handle combinatorial objects easily without worrying much about their indifferentiability or the inherited symmetries , which is often not possible for mainstream learning methods [ 8 ] . Theoretical analysis . For USCO-Solver , we present a learning-error analysis under the PACBayesian framework , where we introduce the multiplicative margin dedicated to bounding the approximation guarantees . In particular , we prove that the approximation ratio of the predictions is essentially bounded by O ( α2 ) , whenever the considered problem admits an α-approximation in the deterministic sense . Such a result is possible due to the fact the hypothesis space of USCO-Solver can approximate the configuration space arbitrarily well . To our knowledge , this is the first result of such kind . Empirical studies . We conduct experiments with USCO problems concerning three classic combinatorial objects : path , coverage , and matching . On various real-world datasets , we consistently observe that USCO-Solver not only works the way it is supposed to , but also outperforms other competitors by an evident margin in most cases . In many case studies , near-optimal solutions can be computed without consulting the true configuration distribution . Supplementary material . The proofs , together with more results and discussions on the experiments , can be found in the supplementary material . In addition , we release the experimental materials , including source code , datasets , and pretrain models , as well as their instructions . The experiment materials can be found online1 . 2 Preliminaries . 2.1 Undetermined stochastic combinatorial optimization problems . We are concerned with an abstract combinatorial optimization problem associated with three finite combinatorial spaces : the input space X , the output space Y , and a configuration space C ; in addition , we are given a bounded non-negative function f ( x , y , c ) : X ×Y ×C → R+ denoting the objective value to be maximized . Let Φ be the set of all distributions over C. We consider stochastic combinatorial optimization problem in the sense that we seek to maximize f in terms of a distribution φtrue ∈ Φ rather than a fixed configuration in C. This is desired because the system that defines the objective values is often essentially probabilistic , which may stem from random networks or functions with random parameters . Therefore , the objective function is given by F ( x , y , φtrue ) = ∫ c∈C φtrue ( c ) · f ( x , y , c ) dc . ( 1 ) Given x and φtrue , we are interested in the problem max y∈Y F ( x , y , φtrue ) . ( 2 ) We may wish to compute either the optimal solution H ( x , φtrue ) : = arg maxy∈Y F ( x , y , φtrue ) or its α-approximation Hα ( x , φtrue ) for some α ∈ ( 0 , 1 ) . Such problems are fundamental and also 1https : //github.com/cdslabamotong/USCO-Solver ubiquitous , ranging from the stochastic version of classic combinatorial problems [ 9 ] to applications subject to environmental uncertainties , such as commercial recommendation [ 10 ] , viral marketing [ 11 ] , and behavioral decision making in autonomous systems [ 12 ] . Taking the stochastic longest path problem [ 13 ] as an example , the length of each edge could follow a certain distribution , and therefore , each configuration c ∈ C corresponds to a weighted graph ; in such a case , x = ( u , v ) denotes a pair of source and destination , y denotes a path from u to v , and f ( x , y , c ) amounts to the length of y in c. While the above formulation is natural , we are always limited by our imperfect understanding of the underlying system , which could be formalized by assuming that the distribution φtrue over the configuration is not known to us . Since the true distribution φtrue is unknown , the objective function F ( x , y , φtrue ) can not be directly optimized . Essentially , a learning process is required . In such a sense , we call these problems undetermined stochastic combinatorial optimization ( USCO ) problems . Oracle . In general , USCO problems are made difficult by a ) the learning challenges in dealing with the unknown distribution φtrue and b ) the approximation hardness in solving Equation 2 . We focus on the learning challenges and thus assume the approximation hardness ( if any ) can be resolved by oracles . Notice that the objective function ( Equation 1 ) lies in the positive affine closure of its discretizations , which is ∪∞k=1 { k∑ i=1 wi · f ( x , y , ci ) : wi ∈ R+ , ci ∈ C } . ( 3 ) For some fixed α ∈ ( 0 , 1 ] , we assume the access to a polynomial oracle for computing an αapproximation to maximizing any function in the above class , which implies that Hα ( x , φ ) is obtainable for each x ∈ X and φ . We can think of α as the best ratio that one can obtain by a polynomial algorithm , with α = 1 denoting the scenario when the optimal solution can be efficiently computed . For a vector w = ( w1 , ... , wk ) ∈ Rk and a subset C = { c1 , ... , ck } ⊆ C of configurations , we denote the induced kernel function as KC ( x , y ) : = ( f ( x , y , c1 ) , ... , f ( x , y , ck ) ) and the associated affine combination as F̂w , C ( x , y ) : = k∑ i=1 wi · f ( x , y , ci ) = wTKC ( x , y ) . Finally , we denote the α-approximation to maxy∈Y F̂w , C ( x , y ) as hαw , C ( x ) . 2.2 Learning settings . We consider samples of input-solution pairs : Sm : = { ( xi , y α i ) } m i=1 ⊆ 2X ×Y where yαi : =Hα ( xi , φtrue ) is an α-approximation associated with the input xi . Such samples intuitively offer the historical experience in which we successfully obtained good solutions to some inputs . From a learning perspective , we have formulated a regression task between two combinatorial spaces X and Y . Some formal treatments are given as follows . Let us assume that the true distribution φtrue is unknown but fixed , and there is a distribution DX over X . Our goal is to design a learning framework AS : X → Y that can leverage a set S of samples to compute a prediction AS ( x ) ∈ Y for each future x ∈ X . For a prediction ŷ ∈ Y associated with an input x , let l ( x , ŷ ) ∈ [ 0 , 1 ] be a general loss function . Since we aim to examine if ŷ is a good approximation to Equation 2 , the loss l is determined jointly by ŷ and x. Consequently , we measure the performance of A by L ( AS , DX , l ) : =Ex∼DX [ l ( x , AS ( x ) ) ] 3 A learning framework . In this section , we first present the learning framework and then analyze its generalization bounds . Finally , we discuss training algorithms . 3.1 USCO-Solver . Following the standard idea of structured prediction [ 14 , 15 , 16 ] , we wish for a score function F̂ ( x , y ) such that , for each x ∈ X , the y ∈ Y that can maximize F̂ ( x , y ) is a good solution to maximizing F ( x , y , φtrue ) . Suppose that we are provided with a set Sm of m ∈ Z samples . USCO-Solver consists of the following steps : • Step 1 . Select a distribution φem ∈ Φ over C and decide a hyperparameter K ∈ Z . • Step 2 . Sample K configurations CK = { c1 , ... , cK } ⊆ C independently following φem . • Step 3 . Compute a seed vector w̃ = ( w̃1 , ... , w̃K ) ∈ RK using the training data . • Step 4 . Sample a vector w ∈ RK from Q ( w|β · w̃ , I ) , which is an isotropic Gaussian with identity convariance and a mean of β · w̃ , with β being β : = 4 minp |w̃p| · α2 √ 2 ln 2mK ‖w̃‖2 . ( 4 ) • Score function . With CK and w = ( w1 , ... , wK ) , we adopt the score function F̂w , CK ( x , y ) : = K∑ i=1 wi · f ( x , y , ci ) . • Inference . For each input x ∈ X , the prediction is given by hαw , CK ( x ) , which can be computed by the oracle . Given that the framework is randomized , the error of interest is L ( USCO-SolverSm , DX , l ) : =Ew∼Q , CK∼φem , x∼DX [ l ( x , hαw , CK ( x ) ) ] . ( 5 ) So far , we have not seen how to determine the distribution φem , the parameter K , and the seed vector w̃ . We will first analyze how they may affect the generalization bound ( Sec . 3.2 ) , and then discuss how to make selections towards minimizing the generalization bound ( Sec . 3.3 ) .

Summary: This paper proposes a framework for learning to predict approximate solutions to stochastic combinatorial optimization problems. The class of problems is referred to as "undetermined" stochastic combinatorial optimization. "Undetermined" here refers to the following assumption on the available training data: each data point is a pair (x, y), where x is the input to the stochastic optimization problem (e.g., source-target pairs in an s-t shortest path problem on a fixed graph), and y is an approximate solution; notice how information about the probability distribution over which the true stochastic objective function is defined (e.g., edge weights) is not available in the training set (neither via a set of sample scenarios nor a generator), hence "undetermined".  USCO-Solver does the following in the training phase: it selects an arbitrary distribution over the unknown stochastic parameters of the problems (e.g., edge weights in a graph), samples K configurations, then searches for a weight vector for the K samples such that the true solution (y) is also near-optimal in another optimization problem whose objective function is a linear combination of the K configurations' objectives. This training problem is framed as a structured prediction task which is amenable to structured SVM-style optimization algorithms. Key to the training is the use of an oracle for the deterministic counterpart of the stochastic problem.  Experiments on randomly generated datasets for stochastic shortest path, set cover, and bipartite matching show that USCO-Solver performs significantly better than "direct" ML methods that are trained to simply fit the training set without regard to the combinatorics or stochastic nature of the true optimization problem.