Learning Scenario Representation for Solving Two-stage Stochastic Integer Programs

1 INTRODUCTION . Stochastic integer programs ( SIPs ) are a class of combinatorial optimization problems ( COPs ) with uncertain parameters . They focus on finding a best decision for a practical task to optimize both the deterministic cost and the expectation of stochastic costs on a set of potential scenarios . Since the uncertain elements commonly exist in reality , SIPs are widely applied and studied in many fields such as transportation , inventory management , energy optimization , and so on ( Jammeli et al. , 2019 ; Dillon et al. , 2017 ; Bruno et al. , 2016 ; Wallace & Ziemba , 2005 ) . However , optimally solving SIP is intractable especially when many scenarios are involved , due to the fact that : 1 ) most integer programs themselves are already NP-hard ; 2 ) considerable ( even continuous ) scenarios significantly increase the computational complexity ( Klein , 2021 ) . In reality , solving moderate-sized problems may require prohibitively long time . Thus efficient and high-quality approximate solutions for SIPs are often practically pursued . The advances in deep ( reinforcement ) learning has aroused extensive studies on solving optimization problems with neural networks . The related works have shown promising results in tackling some classic COPs , e.g. , vehicle routing , job shop scheduling , combinatorial auction , etc . ( Kool et al. , 2019 ; Zhang et al. , 2020 ; Lee et al. , 2020 ; Kwon et al. , 2020 ; Gasse et al. , 2019 ; Chen & Tian , 2019 ; Ichter et al. , 2018 ; Hottung et al. , 2020 ; Wu et al. , 2021 ) . However , most of the existing deep learning based methods only focus on deterministic problems , which could hinder their practical applications in the uncertain situation . On the other hand , a few early attempts introduce deep learning to solve SIPs ( Larsen et al. , 2018 ; Abbasi et al. , 2020 ; Joe & Lau , 2020 ; Bengio et al. , 2020 ; Nair et al. , 2018 ) . However , they are generally designed for only specific problems and also lack the ability of generalizing across different problem sizes . In this paper , we focus on graph-based SIPs , which is a general class of problems with broad applications . We propose to learn the deep latent representation of scenarios in SIPs , which can be ∗Equal contribution . †Corresponding author ( zhiguangcao @ outlook.com ) . applied to various downstream tasks in solving SIPs , e.g. , scenario reduction and objective prediction . Based on the basic two-stage form of SIPs , we leverage conditional variational autoencoder ( CVAE ) to learn latent and continuous representations of stochastic scenarios while considering the deterministic part of a SIP instance , which we term the context . Specifically , we employ a graph convolutional network ( GCN ) to learn embeddings of each deterministic integer program , which models the first-stage problem with a single scenario . The learned embeddings are mapped into continuous representations , and then decoded back into corresponding scenarios given the embedding of the context . This design ensures that the learned representations of scenarios are correctly linked to the corresponding instances . The resulting latent space well represents scenarios in a class of SIP instances , and we demonstrate its power with two tasks commonly used in solving SIPs . First , we leverage it for scenario reduction by finding representative scenarios via an off-the-shelf clustering algorithm , which decreases the problem complexity and meanwhile attains high-quality approximate solutions . Second , we further extend the encoder with an additional sub-network to predict objective values for each scenario . Due to the generalization in the latent continuous space , we readily deploy the prediction task in semisupervised manner with a small number of target objective values . In doing so , the latent space is augmented with the predicted objective value and hence yields more representative scenarios , which further shrinks the approximation gap to the optimal solutions of the original SIPs . In summary , this paper aims to learn scenario representation for solving graph-based two-stage SIPs . The main contributions are as follows : • We firstly present a deep generative framework to learn representations for scenarios in graphbased two-stage SIPs . It considers dependencies of the scenarios on the deterministic context of each instance , and hence well differentiates scenarios in the latent space . • We apply the learned representations to scenario reduction and objective prediction for SIPs . The representative scenarios derived from clustering of the representations can be used to efficiently compute high-quality solutions for SIPs . Moreover , we further shrink the approximation error by predicting objective values of scenarios via semi-supervised learning . • We evaluate the proposed method on two SIP problems , i.e. , the network design problem and facility location problem . Experimental results verify the superiority of the learned scenario representation for solving SIPs . Our method significantly outperforms existing deep learning based baselines . Notably , it also generalizes well to problems with larger sizes or more scenarios . 2 RELATED WORK . Learning based methods for SIPs . Recent works of deep learning for COPs mostly address deterministic problems , with scarce attention to uncertainty that commonly exist in practice . Though a few methods apply deep learning to improve approximate solutions for two-stage SIPs , they are mostly designed for some specific problems . Nair et al . ( 2018 ) train a bit-flipping policy with reinforcement learning to improve solutions iteratively , but it is limited to binary decision variables and assumes no constraints in the first-stage problem . Larsen et al . ( 2018 ) predict properties of optimal solutions to integer programs ( i.e . the first-stage problem with respective single scenario ) in a supervised manner . This method is only applicable when operational solutions are not needed . In contrast , Abbasi et al . ( 2020 ) predict values of decision variables for blood transshipment problem with fixed-sized instances . To reduce the complexity of SIPs , Bengio et al . ( 2020 ) aim to yield one scenario for the first-stage problem , so that the derived solution achieves a small objective value when evaluated with the original set of scenarios . This method assumes complete recourse , with any generated scenario being feasible for the second-stage problem . All the above methods attempt to reduce the expected cost of stochastic scenarios but their applications are limited by distinct assumptions or fixed problem sizes . In contrast , our method can be used for more general two-stage SIPs , and generalizes well to varying problem sizes or scenario quantities . Scenario reduction for SIPs . In the SIP formulation , a set of scenarios is used to model stochastic events on top of the first-stage problem . It introduces extra complexity which may exponentially increase with its cardinality , owing to the combinatorial search space ( Dyer & Stougie , 2006 ) . Since the direct optimization with massive scenarios is intractable , the scenario reduction is often conducted to find a subset of scenarios that well replaces the original set with a small approximation error . Existing reduction methods mainly branches into distribution-oriented and problem-oriented paradigms . The former ones pursue a group of scenarios that closely estimate the probability distribution under the scenario space ( Henrion & Römisch , 2017 ) . The distance between the estimated and actual probability are often measured by Wasserstein distance ( Rujeerapaiboon et al. , 2018 ) , polyhedral discrepancy ( Henrion et al. , 2008 ) or difference of moments ( Høyland et al. , 2003 ) . These methods neglect the influence of the context on the objective function , e.g. , similar scenario distributions may cause different expected costs in disparate SIP instances . Thus , there is a recent trend towards the study of problem-oriented methods ( Henrion & Römisch , 2018 ; Fairbrother et al. , 2019 ; Keutchayan et al. , 2020 ; 2021 ) , which attempts to involve problem-specific properties in scenario reduction . For example , Keutchayan et al . ( 2021 ) propose to find K scenarios with their objective values separately approximating the expected objective values in the K scenario subsets . Most the above methods are mainly for theoretical proof on stochastic programs ( without the restriction of integers ) . In contrast , we firstly propose a deep learning method for SIPs to learn scenario representation , which can be used for both scenario reduction and objective prediction . 3 PRELIMINARIES . 3.1 TWO-STAGE STOCHASTIC INTEGER PROGRAMS . As an optimization problem , SIP is usually characterized by both the uncertain parameters that potentially follow certain probability distributions and the discrete solution space due to the integer restrictions . It is commonly described by a two-stage formulation as below : min x µ⊤x+ Eω [ Q ( x , ω ) ] ( 1 ) s.t . Ax ≤ b , x ∈ Rn1−p1 × Zp1 , ( 2 ) where Q ( x , ω ) : = miny { q⊤ω y|Wωy ≤ hω − Tωx ; y ∈ Rn2−p2 × Zp2 } . In particular , Eq . ( 1 ) and ( 2 ) prescribe the first-stage problem with x ∈ Rn1 being the decision variable , where n1 ≥ p1 ; Q ( x , ω ) prescribes the second-stage problem , with y ∈ Rn2 being the decision variable , where n2 ≥ p2 . We refer to the group of static parameters µ ∈ Rn1 , A ∈ Rm1×n1 and b ∈ Rm1 in the first stage as the context , and assume that the group of uncertain parameters qω ∈ Rn2 , Wω ∈ Rm2×n2 , Tω ∈ Rm2×n1 and hω ∈ Rm2 follows a distribution P . Here we focus on SIPs defined on graphs , which are a family of problems with practical applications in many domains such as networks , transportation and scheduling ( Rahmaniani et al. , 2018 ; An & Lo , 2016 ) . The primary method to solve the above SIP is sample average approximation ( Kleywegt et al. , 2002 ) , which converts it into a mixed integer program ( MIP ) by Monte Carlo simulation and in turn optimizes the following objective : O ( x ) : = min x µ⊤x+ 1 N ∑N i=1 Q ( x , ωi ) , ( 3 ) where { ωi } Ni=1 is a set of scenarios from P , i.e. , an independently and identically distributed ( i.i.d . ) random sample of N realizations of uncertain parameters . Typically , a large scenario set is often required to make the distribution of scenarios P̃ well fit P , which may cause intractable MIP for a given solver . In this paper , we aim to find a small number of informative representatives in { ωi } Ni=1 by learning representations of scenarios , which is supposed to considerably boost the computation efficiency with tolerable approximation errors . 3.2 CONDITIONAL VARIATIONAL AUTOENCODER . As an unsupervised generative model , CVAE is developed on top of VAE , and it further controls the data generation process conditioned on additional random variables ( Sohn et al. , 2015 ) . These conditional variables could be either class labels or certain properties of the data with specific distributions , which are engaged in the input to both the encoder and decoder . Optimized with the stochastic gradient variational bayes ( SGVB ) framework ( Kingma & Welling , 2014 ) , the evidence lower bound objective ( ELBO ) of CVAE on the marginal likelihood for the input data is defined as : log pθ ( X , c ) ≥ ELBO ( X , c ) = Eqϕ ( z|X , c ) [ log pθ ( X|z , c ) ] − KL [ qϕ ( z|X , c ) ∥p ( z|c ) ] , ( 4 ) where X , c , z denote input , latent and conditional variables , respectively ; both the encoder qϕ and decoder pθ are typically parameterized by deep neural networks . Rather than simple distributions of conditional variables in most existing works , in this paper we exploit CVAE to learn scenario representation in SIPs , with conditional variables derived from continuous parameters in the context . For more details of CVAE , we refer readers to ( Kingma et al. , 2014 ; Feng et al. , 2021 ) .

Summary: The paper aims to use deep-learning towards solving generic stochastic integer programs. The proposed approach involves using a conditional VAE to model the distribution over scenarios, and leveraging the scenario's latent space embeddings to enable better representative scenario selection: the embeddings are used as inputs to a standard clustering algorithm that returns representatives.  The paper demonstrates that using VAE embeddings improves performance when compared to using the same clustering algorithm on the raw scenario representations. It further demonstrates that using embeddings concatenated with a predicted objective generally improves performance over using the embeddings alone. Finally, the paper shows improvement over two recent approaches.