Title: You Shall not Pass: the Zero-Gradient Problem in Predict and Optimize for Convex Optimization
Venue: NeurIPS 2023
Accepted: no
Keywords: Predict and optimize

differential optimization

convex optimization
Abstract: Predict and optimize is an increasingly popular decision-making paradigm that employs machine learning to predict unknown parameters of optimization problems. Instead of minimizing the prediction error of the parameters, it trains predictive models using task performance as a loss function. In the convex optimization domain, predict and optimize has seen significant progress due to recently developed methods for differentiating optimization problem solutions over the problem parameters. This paper identifies a yet unnoticed drawback of this approach -- the zero-gradient problem -- and introduces a method to solve it. The suggested method is based on the mathematical properties of differential optimization and is verified using two real-world benchmarks.

Abstract

Predict and optimize is an increasingly popular decision-making paradigm that employs machine learning to predict unknown parameters of optimization problems. Instead of minimizing the prediction error of the parameters, it trains predictive models using task performance as a loss function. In the convex optimization domain, predict and optimize has seen significant progress due to recently developed methods for differentiating optimization problem solutions over the problem parameters. This paper identifies a yet unnoticed drawback of this approach - the zero-gradient problem - and introduces a method to solve it. The suggested method is based on the mathematical properties of differential optimization and is verified using two real-world benchmarks.

1 Introduction

Mathematical programming is one of the fundamental tools of applied mathematics. It is utilized in various domains, such as finance [Cornuejols and Tütüncü, 2006], power systems [Bansal, 2005], robotics [Raja and Pugazhenthi, 2012], and many others. The main practical limitation of mathematical programming is that it requires a fully-defined model describing the problem which is not always available in reality. A promising approach to overcome this limitation is to employ machine learning (ML) to predict missing parts of the model [Ning and You, 2019].

Predict and optimize (P&O) [Elmachtoub and Grigas, 2017] is a decision-making paradigm that combines ML with mathematical programming. It considers optimization problems where some parameters are unknown and should be predicted prior to solving the problem. The P&O approach builds upon the observation that naively training an ML algorithm to match the distribution of unknown parameters is inefficient [Elmachtoub and Grigas, 2017], as this approach does not take the actual task performance into account. Instead, P&O aims at using task performance as the objective function for ML models directly.

The standard approach to training models in machine learning is to use gradient-based algorithms, such as stochastic gradient descent Kiefer and Wolfowitz [1952]. In predict and optimize, computing the gradient of the task performance involves differentiating the solution of the optimization problem with respect to the parameters, which is a non-trivial task. In their seminal work, Agrawal et al. [2019a] have shown that a large class of convex optimization problems indeed can be differentiated.

In this paper, we identify a fundamental drawback of differential optimization - the zero-gradient 30 problem . Specifically, we show that the Jacobian of convex problems often has a large null space, 31 and hence the task performance, as a function of the ML model parameters, is flat in a significant part 32 of its domain. Therefore, it can not be optimized using gradient-based methods. Consequently, we 33 introduce a way to compute an approximate gradient that is zero only in the optimal solution and is 34

guaranteed to not decrease performance. Finally, we validate the superiority of this method using two 35 real-world problems: the portfolio optimization problem and the optimal power flow problem. 36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

57

58

59

60

61

62

63

64

65

66

67

68

69

70

71

72

73

74

2 Predict and optimize

In this section, we provide an overview of the existing research in the predict and optimize domain. Then, we define the P&O problem and introduce the solution approach that we are going to investigate later in this manuscript.

2.1 Related work

To the best of our knowledge, the predict and optimize framework was first introduced by Elmachtoub and Grigas [2017]. They consider optimization problems with linear objectives and derive a convex approximation of the task performance function. Then, they optimize the prediction model by using sub-gradients of this approximation. Later, this method was extended onto combinatorial problems by Mandi et al. [2020]. Several other approximations were introduced in other studies focusing on combinatorial problems. Vlastelica et al. [2019] derive a differentiable piecewise-linear approximation for the task performance; Berthet et al. [2020] employ stochastic perturbations to approximate derivative of combinatorial problems.

Unlike in the combinatorial case, continuous convex optimization problems do allow exact differentiation of the loss function. The sequence of works [Amos and Kolter, 2017], [Agrawal et al., 2019b], [Agrawal et al., 2019a] developed a differential optimization technique to compute the derivative of convex optimization problems. In their latest work [Agrawal et al., 2019a], the authors delivered a general method that allows differentiating disciplined convex programs [Grant et al., 2006]. This result gave rise to new applications of P&O to convex optimization: Uysal et al. [2021] applied convex differential optimization to the risk budgeting portfolio optimization problem; Wang et al. [2020] utilized it to learn surrogate models for predict and optimize; Donti et al. [2017] applied the method to three different real-world benchmarks. Moreover, several studies applied differential optimization to predict and optimize for other problem classes. In Wilder et al. [2019], it was used in linear optimization via constructing a quadratic approximation of the problem. Later, Mandi and Guns [2020] improved upon this result by using logarithmic approximations. Ferber et al. [2020] combined a similar idea with the cutting plane approach and used differential optimization in combinatorial problems.

Outside of predict and optimize, differential optimization also has found several applications. Chen et al. [2021] used it to train reinforcement learning agents in the action space with convex constraints, and Agrawal et al. [2019c], employed it for tuning model predictive control algorithms.

While the benefits of the differential optimization approach to predict and optimize are numerous, it is still not fully understood. It was reported in several studies Vlastelica et al. [2019], Wilder et al. [2019], that the gradient of a linear problem is zero everywhere, except for the finite set of points where it is undefined. Since any linear problem is convex, this observation suggests that the gradients of convex problems should be also thoroughly investigated.

2.2 Problem formulation

In this section, we introduce the P&O problem. We refer readers to Elmachtoub and Grigas [2017] for further details. In predict and optimize, we solve optimization problems of the form

where x ∈ R n is the decision variable, w ∈ R u is a vector of unknown parameters, f : R n × R u → R 75 is the objective function, and C is the feasibility region. The defining feature of this problem is that 76 the parameters w are unknown at the moment when the decision must be made. Therefore, the true 77 optimization problem is under-defined and cannot be solved directly. 78

One way to deal with the unknown parameters w is to use a prediction ˆ w instead. Then, the decision 79 can be computed by solving the following problem, which we refer to as the internal problem: 80

A commonly made assumption is that instead of w, we observe a feature vector o that contains some 81 information about w. Also, we have a dataset D = { ( o k , w k ) } , e.g., of historical data, which we 82 can use to learn the relation between w and o. This setup enables using ML models to compute the 83 prediction. We denote the prediction model by ϕ θ , and thus we have ˆ w = ϕ θ ( o ) . 84

The problem described above is not specific to predict and optimize. What separates the P&O 85 paradigm from earlier works is the approach to training the model ϕ θ . In the past, machine learning 86 models would be trained to predict w as accurately as possible, e.g., in Mukhopadhyay and Vorobey87 chik [2017]. However, the parameter prediction error is merely an artificial objective and our true 88 goal is to derive a decision x that maximizes the task performance f ( x, w ) . The main goal of the 89 P&O approach is to utilize this objective for training the model ϕ θ . The task performance achieved 90 by ϕ θ on the dataset D can be quantified by the following loss function: 91

Most machine learning algorithms for training models are based on computing the gradient of 92 the loss function (Kiefer and Wolfowitz [1952]). To train ϕ θ with a gradient-based algorithm, we 93 need to differentiate L over θ, and hence we need to compute the gradient ∇ θ f ( x ∗ ( ˆ w ) , w ) , where 94 ˆ w = ϕ θ ( o ) . Applying the chain rule, it can be decomposed into three terms: 95

The second term, ∇ ˆ w x ∗ ( ˆ w ) , is the Jacobian of the solution of the optimization problem over the prediction ˆ w. An exact method to compute this Jacobian was introduced in Agrawal et al. [2019a], but it has never been thoroughly analyzed. In the next section, we show that ∇ ˆ w x ∗ ( ˆ w ) has a large null space, thereby causing the total gradient in Eq. 2 to be zero even outside of the optimum.

3 Differentiable optimization

In this section, we study the derivative of convex optimization programs over the parameters of the objective function. We show that the gradient in Eq. 2 is often zero outside of the optimum, and hence it causes gradient-following methods to get stuck in suboptimal solutions. In the second part of this section, we introduce a method to solve this problem.

Without loss of generality, we consider a single instance of the problem, i.e., one sample ( o, w ) ∈ D . Everywhere in this section, we denote the prediction by ˆ w = ϕ θ ( o ) . Then, the decision is computed as a solution of the internal optimization problem defined as follows:

We use ˆ x to denote the value of x ∗ ( ˆ w ) for a given prediction ˆ w. As we are interested in convex optimization problems, we make the following assumptions:

Assumption 1. The objective function f ( x, w ) is concave and twice continuously differentiable in x for any w.

Assumption 2. The feasibility region C is convex, i.e., {C = { x | g i ( x ) ≤ 0 , i = 1 , . . . , l } , where g i ( x ) are convex differentiable functions. Moreover, for any x ∈ C , the gradients {∇ x g i ( x ) | g i ( x ) = 0 } of the active constraints are linearly independent. 1

Additionally, we make an assumption about how f depends on w , which holds for many real-world problems, including linear and quadratic optimization problems.

Assumption 3. The objective function f ( x, w ) is twice continuously differentiable in w.

Throughout this paper, we use derivatives of different objects. For clarity, we first provide an overview of them: the gradient of the true objective function over the decision, ∇ x f (ˆ x, w ); the Jacobian of the decision over the prediction, ∇ ˆ w x ∗ ( ˆ w ); the Jacobian of the prediction over the ML model parameters, ∇ θ ˆ w ; and the gradient of the predicted objective in the internal problem, ∇ x f ( x, ˆ w ) . In the next section, we establish some crucial properties of the Jacobian ∇ ˆ w x ∗ ( ˆ w ) .

96

97

98

99

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

3.1 The zero-gradient theorem

We begin by investigating the relation between the values of the function x ∗ ( ˆ w ) and the gradient of the internal objective, ∇ x f ( x, ˆ w ) . Let n i := ∇ x g i (ˆ x ) , i = 1 , . . . , l be the normal vectors of the constraints at ˆ x, Then, the KKT conditions Kuhn and Tucker [1951] at ˆ x state that there exist real values α 1 , . . . , α l such that the following holds:

Under Assumptions 1 and 2, the KKT multipliers α i are uniquely defined by ˆ w and ˆ x. Thus, as ˆ x is defined by ˆ w, we sometimes write α i ( ˆ w ) to emphasize that it is, in fact, a function of ˆ w. To provide a geometrical perspective on the KKT conditions, we introduce the following definition:

Definition 3.1. Let x ∈ C and let I ( x ) = { i | g i ( x ) = 0 } be the set of indices of the constraints active at x. Let n i = ∇ x g i ( x ) , ∀ i ∈ I ( x ) , be the normal vectors of these constraints. The gradient cone , G ( x ) := { ∑ i ∈ I α i n i | α i ≥ 0 } , is the positive linear span of normal vectors n i .

Combining the KKT conditions with Definition 3.1, we immediately arrive at the following property:

Property 3.2. Let x ∈ C and let ∇ x f ( x, ˆ w ) be the internal gradient at x. Then, x is a solution to the problem in Eq. 3 if and only if ∀ i ∈ I ( x ) , ∃ α i ≥ 0 , such that ∇ x f ( x, ˆ w ) = ∑ i ∈ I ( x ) α i n i ∈ G ( x ) , where I ( x ) is the set of indices of active constraints, I ( x ) = { i | g i ( x ) = 0 } .

While trivial, this property provides a geometrical interpretation of the problem. Effectively, a point x is a solution to the problem in Eq. 3 if and only if the internal gradient at this point lies inside its gradient cone. Figure 1 illustrates this property.

Before studying the Jacobian ∇ ˆ w x ∗ ( ˆ w ) , we first need to address the question of when this Jacobian exists. Sufficient conditions for existence are given in Fiacco [1976]. Under Assumptions 1-3, these conditions can be reformulated as follows:

Lemma 3.3 (Theorem 2.1 in Fiacco [1976]) . Let Assumptions 1-3 hold and let

be the representation of the internal gradient with the normals of the active constraints. Then, suppose that the strict complementary slackness condition holds, i.e., α i ( ˆ w ) > 0 , ∀ i ∈ I (ˆ x ) . Then, the Jacobian ∇ ˆ w x ∗ ( ˆ w ) exists at ˆ w. Moreover, α i ( · ) is continuous around ˆ w for any i ∈ I (ˆ x ) .

Proof of this lemma is given in Fiacco [1976]. This result establishes that strict complementary slackness is sufficient for the Jacobian ∇ ˆ w x ∗ ( ˆ w ) to exist. In most cases, the points that violate strict complementary slackness form a zero-measure set and hence can be neglected in practice.

Now, we have all the necessary tools to describe the structure of the Jacobian ∇ ˆ w x ∗ ( ˆ w ) . Suppose that the strict complementary slackness condition holds at ˆ x and hence the Jacobian exists. Assume that we perturb ˆ w and obtain ˆ w ′ . Let ˆ x ′ = x ∗ ( ˆ w ′ ) denote the solution corresponding to ˆ w ′ . What can be said about ˆ x ′ ? Strict complementary slackness implies that the constraints active at ˆ x will remain active at ˆ x ′ if the difference ∥ ˆ w ′ -ˆ w ∥ 2 2 is small enough. Therefore, the decision ˆ x ′ can only move within the tangent space of C at ˆ x , i.e., orthogonally to all n i , i ∈ I (ˆ x. ) Hence, when more constraints are active, ˆ x ′ can move in less directions. Formally, we obtain the following lemma:

158

159

160

161

162

163

164

165

166

167

168

169

170

171

172

173

174

175

176

177

178

179

180

181

182

183

184

185

186

187

188

189

190

191

192

193

194

195

196

197

198

199

200

201

202

203

204

205

206

Lemma 3.4. Suppose that the strict complementary slackness conditions hold at ˆ x and let ∇ x f (ˆ x, ˆ w ) = ∑ i ∈ I (ˆ x ) α i n i , α i > 0 , ∀ i ∈ I (ˆ x ) be the internal gradient. Let N (ˆ x ) = span ( { n i | i ∈ I (ˆ x ) } ) be the linear span of the gradient cone. Then N (ˆ x ) is contained in the left null space of ∇ ˆ w x ∗ ( ˆ w ) , i.e., v ∇ ˆ w x ∗ ( ˆ w ) = 0 , ∀ v ∈ N (ˆ x )

The formal proof of this result can be found in the appendix. Lemma 3.4 is very important, as it specifies in what directions x ∗ ( ˆ w ) can move as a consequence of changing ˆ w. Now, the first term in the chain rule in Eq. 2, ∇ x f (ˆ x, w ) , specifies in what directions x ∗ ( ˆ w ) should move in order for the true objective to increase. Naturally, if these directions are contained in the null space of ∇ ˆ w x ∗ ( ˆ w ) , then the total gradient in Eq. 2 is zero. This observation constitutes the main theorem of this paper the zero-gradient theorem.

Theorem 3.5 (Zero-gradient theorem) . Let ˆ w be a prediction, and let ˆ x be the solution of the internal optimization problem defined in Eq. 3. Suppose that the strict complementary slackness conditions hold at ˆ x and let N (ˆ x ) = span ( { n i | i ∈ I (ˆ x ) } ) be the linear span of the gradient cone at ˆ x. Then, ∇ x f (ˆ x, w ) ∈ N (ˆ x ) = ⇒ ∇ θ f (ˆ x, w ) = 0 .

The proof of this theorem is obtained by simply applying Lemma 3.4 to the chain rule in Eq. 2. The theorem claims that the gradient of the P&O loss in Eq. 1 can be zero in the points outside of the optimal solution. Hence, any gradient-following method 'shall not pass' these points. In particular, the zero-gradient phenomenon happens in such points ˆ x where the true gradient ∇ x f (ˆ x, w ) is contained in the space N (ˆ x ) spanned by the gradient cone G (ˆ x ) . As the dimensionality of this space grows with the number of active constraints, the zero-gradient issue is particularly important for problems with a large number of constraints. In the worst case, N (ˆ x ) can be as big as the whole decision space R n , thereby making the total gradient ∇ θ f (ˆ x, w ) from Eq. 2 zero for any value of the true gradient ∇ x f (ˆ x, w ) . In the following sections, we introduce a method that resolves the zero-gradient problem and provides theoretical guarantees for its performance.

3.2 Quadratic programming approximation

The fundamental assumption of the predict and optimize framework is that training ϕ θ using the task performance loss is better than fitting it to the true values of w. Hence, the models trained with predict and optimize might output ˆ w that is significantly different from the true w and yet produces good decisions. Taking this argument one step further, we claim that the objective function f ( x, ˆ w ) in the internal optimization problem in Eq. 3 does not need to be the same as the true objective f ( x, w ) . In particular, we suggest computing decisions using a simple quadratic program (QP):

2

The reasons for this choice are manyfold. First, the internal objective f QP ( x, ˆ w ) = -∥ x -ˆ w ∥ 2 , is strictly concave and hence x ∗ QP ( ˆ w ) is always uniquely-defined. Moreover, the range of x QP ( ˆ w ) is C , i.e., ∀ x ∈ C , ∃ ˆ w such that x = x ∗ QP ( ˆ w ) . Hence, it can represent any optimal solution. However, the most important property of QP is that its Jacobian is very simple, which we explain below.

The problem in Eq. 4 has a simple geometrical interpretation: the point x = ˆ w is the unconstrained maximum of f QP ( x, ˆ w ) and x ∗ QP ( ˆ w ) is its Euclidean projection on the feasibility set C , see Figure 2. To compute the Jacobian ∇ ˆ w x ∗ QP , we need to understand how perturbations of ˆ w affect x ∗ QP . Employing the geometrical intuition above, we obtain the following lemma:

Lemma 3.6. Let ˆ w be a prediction and ˆ x be the optimal solution of the QP problem defined in Eq. 4. Let the strict complementary slackness condition hold and let { n i | i ∈ I (ˆ x ) } be the normals of the active constraints. Let { e j | j = 1 , . . . , n -| I (ˆ x | ) } be an orthogonal complement of vectors { n i | i ∈ I (ˆ x ) } to a basis of R n . Then, the representation of the Jacobian ∇ ˆ w x QP ( ˆ w ) in the basis { n i } ∪ { e j } is a diagonal matrix. Its first | I (ˆ x ) | diagonal entries are zero, and the others are one.

Proof of this lemma can be found in the appendix. Lemma 3.6 implies that the Jacobian ∇ ˆ w x QP ( ˆ w ) has a simple form and can be easily computed by hand. While providing computational benefits, this approach does not address the zero-gradient problem. In the next section, we introduce a method to compute an approximate of the Jacobian ∇ ˆ w x QP ( ˆ w ) that has a strictly one-dimensional null space. Combined with the QP approximation, it is guaranteed to at least not decrease the task performance.

207

208

209

210

211

212

213

214

215

216

217

218

219

220

221

222

223

224

225

226

227

228

229

230

231

232

233

234

235

236

237

238

239

240

241

242

243

244

245

A

3.3 Local smoothing

We identified a fundamental issue of differential optimization - the zero-gradient problem. We showed that the null space of the Jacobian ∇ ˆ w x ( ˆ w ) depends on the number of constraints active at ˆ x. Generally, this number can be as large as the number of optimized variables n, and the gradientdescent algorithms can get stuck in certain points on the boundary of the feasibility region.

In this section, we propose a simple way to modify the feasibility region - we smooth C locally around the point for which we compute the Jacobian, thereby ensuring that its null space becomes one dimensional. First, we define a method for the general setup, without imposing any assumptions on the optimization problem. Then, we demonstrate that combined with the QP approximation from Section 3.2, this smoothing approach has theoretical guarantees.

We begin with the general case - the problem in Eq. 3. Let ∇ x f (ˆ x, ˆ w ) = ∑ i ∈ I (ˆ x ) α i n i be the internal gradient at ˆ x for some α i ≥ 0 , ∀ i ∈ I (ˆ x ) . Then, we introduce the following definition:

Definition 3.7. Let r > 0 be a positive real number. Let c = ˆ x -r ∇ x f (ˆ x, ˆ w ) ∥∇ x f (ˆ x, ˆ w ) ∥ 2 . The local r -smoothed feasibility region , C r (ˆ x, ˆ w ) := { y | y ∈ R n , ∥ y -c ∥ 2 ≤ r } , is a ball of radius r around c. The local r -smoothed problem P r (ˆ x, ˆ w ) with parameters ˆ x, ˆ w is defined as x ∗ r ( ˆ w ) := arg max x ∈C r (ˆ x, ˆ w ) f ( x, ˆ w ) .

Figure 2 shows an example of the local r -smoothed problem. Now, let ˆ x r = x ∗ r ( ˆ w ) denote the solution of P r (ˆ x, ˆ w ) . By construction, the internal gradient at ˆ x r lies in the one-dimensional gradient cone, and hence, by Property 3.2, ˆ x r = ˆ x. The main purpose of smoothing is to approximate the gradient in Eq. 2 by substituting ∇ ˆ w x ∗ ( ˆ w ) with ∇ ˆ w x ∗ r ( ˆ w ) . We highlight that the decisions are still computed using the non-smoothed problem x ∗ ( ˆ w ) and x ∗ r (ˆ x, ˆ w ) is used exclusively to perform the gradient update step. In other words, we use the following expression to compute the gradient:

It is worth mentioning that the strict complementary slackness in the original problem is a stronger condition than the strict complementary slackness on P r (ˆ x, ˆ w ) . Therefore, the Jacobian of the r -smoothed problem can exist even for predictions ˆ w where the true Jacobian does not.

Generally, the efficiency of r -smoothing depends on the form of the internal problem in Eq. 3. Below, we show that combining r -smoothing with the QP approximation has guarantees on its performance. First, we notice that Lemma 3.6 prescribes the Jacobian of the r -smoothed QP problem:

Property 3.8. Let ˆ x = x ∗ QP ( ˆ w ) be a decision derived via QP. Suppose that the complementary slackness conditions hold for P r (ˆ x, ˆ w ) and let e 1 = ∇ x f QP (ˆ x, ˆ w ) be the internal gradient. Let { e 2 , . . . , e n } be a complement of e 1 to an orthogonal basis of R n . Then, the Jacobian ∇ ˆ w x ∗ r ( ˆ w ) of the local r -smoothed problem expressed in the basis { e 1 , e 2 , . . . , e n } is a diagonal matrix. Its first entry is zero, others are ones.

As C r (ˆ x, ˆ w ) is defined by a single constraint, the null space of ∇ ˆ w x ∗ r (ˆ x, ˆ w ) is always one-dimensional. Hence, the zero-gradient problem can only occur when the internal gradient ∇ x f QP (ˆ x, ˆ w ) and the true gradient ∇ x f (ˆ x, w ) are exactly collinear. Hence, we expect r -smoothing to significantly improve upon the zero-gradient problem. Next, we show that the r -smoothed Jacobian is actually a good approximation. In the following theorem, we demonstrate that the local r -smoothing of the QP approach indeed yields a 'good' direction for the gradient steps.

246

247

248

249

250

251

252

253

254

255

256

257

258

259

260

261

262

263

264

265

266

267

268

269

270

271

272

273

274

275

276

277

278

Theorem 3.9. Let ˆ x = x ∗ QP ( ˆ w ) be the decision obtained via QP and let ∇ ˆ w x ∗ r ( ˆ w ) be the Jacobian of the r -smoothed QP problem. Let ∆ˆ w = ∇ x f (ˆ x, w ) ∇ ˆ w x ∗ r ( ˆ w ) be the prediction perturbation obtained by using this Jacobian and let ˆ w ′ ( t ) = ˆ w + t ∆ˆ w be the updated prediction. Then, for t → 0 + , using ˆ w ′ ( t ) results in a non-decrease in the task performance. In other words, f ( x ∗ QP ( ˆ w ′ ( t )) , w ) ≥ f ( x ∗ QP ( ˆ w ) , w ) .

Interestingly, this result does not depend on r. However, this is to be expected - no matter the radius of C r , the Jacobian of P r (ˆ x, ˆ w ) is still the same by Lemma 3.6. Theorem 3.10 shows that using r -smoothing together with the QP approximation results in analytically computable Jacobian that has a strictly one-dimensional null space. Therefore, we are much less likely to encounter the zero-gradient problem when using this approximation. However, the resulting one-dimensional null space contains the only direction that can move the prediction ˆ w , and hence the decision ˆ x, inside C . This might become crucial, for example, when the optimal solution with respect to the true objective lies in the interior of C . To resolve this problem, we use the projection distance regularization method first suggested in Chen et al. [2021]. Specifically, we add a penalty term

where α ∈ R + is a hyperparameter. Minimizing this term, we push ˆ w along the null-space of the Jacobian towards the feasibility region and eventually move ˆ x inside C .

3.4 The training process

In this section, we summarize the results of Sections 3.1-3.3 and describe the final algorithm we use to solve the P&O problems. For each problem instance ( o, w ) , we first compute the prediction, ˆ w = ϕ θ ( o ) , and the decision using the QP approximation method, ˆ x = x ∗ QP ( ˆ w ) . Then, we obtain the achieved objective value, f (ˆ x, w ) . During training, we update the model parameters θ by performing the steps described in Algorithm 1.

4 Experiments

The main result of Section 3 is the zero-gradient theorem, which describes when the gradient ∇ θ f ( x ∗ ( ˆ w ) , w ) is zero. To deal with it, we introduced the QP approach for computing the decisions, r -smoothing for approximating the Jacobian ∇ ˆ w x ∗ ( ˆ w ) , and projection distance regularization to deal with the remaining null space dimension. Our solution deals with the zero gradient problem by combining these methods. In this section, we use two real-world P&O problems to evaluate the efficiency of our method.

4.1 Portfolio optimization

Following Wang et al. [2020], we apply the predict and optimize framework to the Markowitz mean-variance stock market optimization problem Markowitz and Todd [2000]. In this problem, we act as an investor who seeks to maximize the immediate return but minimize the risk penalty. The

(a)

(b)

(c)

decision variable, x ∈ R n , is a positive vector representing our investment in different securities. The 279 budget constraint forces the investments to add up to one, i.e., ∑ i x i = 1 . The objective is defined as 280 f ( x, p, Q ) = p ⊤ x -λx ⊤ Qx, where p ∈ R n is the immediate return of the securities, λ ≥ 0 is the 281 risk-aversion weight, and Q ∈ R n × n is the positive definite matrix of covariance between securities. 282 The portfolio optimization problem is then defined as follows: 283

This is a quadratic optimization problem with unknown parameters ( p, Q ) , as neither the immediate 284 return nor the true covariance matrix is known at the decision-making moment. Following Wang 285 et al. [2020], we use historical data from QUANDL WIKI prices QUANDL [2020] for 505 largest 286 companies on the American market for the period 2014-2017. The dataset is processed and for every 287 day we obtain a feature vector summarizing the recent price dynamic. For further details on the 288 processing we refer readers to the code 2 and to Wang et al. [2020]. For each run, we randomly split 289 data into train, validation, and test sets by using 70%, 15%, and 15% of the whole dataset respectively. 290 To evaluate the performance of different algorithms, we use regret, defined as 291

In the experiments, we used λ from the set { 0 , 0 . 1 , 0 . 25 , 0 . 5 , 1 , 2 } . For the larger value of λ, the true 292 objective f ( x, p, Q ) is 'more' quadratic, and hence its maximum is more likely to lie in the interior 293 of C . For smaller λ 's, on the other hand, the true objective becomes almost linear and hence it usually 294 attains its maximum on the boundary of C . 295

First of all, we define the QP approximation of the portfolio optimization problem: 296

297

298

299

300

301

302

303

304

305

306

307

As the problem is quadratic, the only difference introduced by the QP approximation comes from using the identity matrix I instead of Q. The results in Table 1 indicate that it performs at least as well as learning to predict both p and Q, and hence in all other experiments we used the QP approximation to compute the decisions.

To investigate the zero-gradient effect, we compared four ways to train the predictor: with/without r -smoothing and with/without the penalty term from Eq. 6. The model used for

308

309

310

311

312

313

314

315

316

317

318

319

320

321

322

323

324

325

326

327

328

329

330

331

332

333

334

335

336

337

338

339

340

341

342

343

344

345

346

347

348

349

350

351

352

353

354

the predictor is a 2-layer neural network, further details on the training process are described in the appendix. The results in Figure 3 indicate that r -smoothing significantly improves the performance when the true objective is more linear. This result matches the theory from Section 3, as linear true objective pushes the decision ˆ x towards the boundary of C , and hence it is more likely to enter points with a large gradient cone. For the more quadratic objectives, the true maximum is often in the interior of C , and hence r -smoothing alone is not sufficient to reach it. In this case, the regularization term from Eq 6 becomes crucial, as it is the only method that can push ˆ x inside C .

4.2 Optimal power flow in a DC grid

To further understand the zero-gradient phenomenon, we consider the optimal power flow problem (OPF) for DC grids [Li et al., 2018]. Due to power losses, the constraints in this problem are non-linear, thus making it computationally hard. In our experiments, we used a linearized version of the problem that represents a DC grid without power losses. The decision variable is the vector of nodal voltages v ∈ R n , and the unknown parameter w represents either the value gained by serving power to a customer or the price paid for utilizing a generator. The reference voltage v 0 ∈ R , the admittance matrix Y, and the constraint bounds represent the physical properties of the grid.

We refer the reader to Li et al. [2018] for further details of the problem. Importantly, the feasibility region is defined by multiple linear constraints, and therefore we expect it to have numerous vertices with large gradient cones. The objective function f ( v, w ) quantifies the social welfare [Veviurko et al., 2022] generated by all the users of the power grid. Importantly, f ( v, w ) is linear, and hence its maximum lies on the boundary of the feasibility region.

We compared the same four methods as before on this problem using randomly generated grids with four generators and twelve loads. Same as before, we use QP approximation, x ∗ QP ( ˆ w ) = arg max x ∈C ( -∥ ˆ w -x ∥ 2 2 ) , to compute the decisions. The results in Figure 4 confirm our hypothesis even though we differentiate through a quadratic problem, the linearity of the true objective causes the zero-gradient effect as the decision is pushed towards the boundary of the feasibility region. Then, due to a large number of constraints, it is likely to enter a vertex with a large gradient cone and get stuck there. In this case, r -smoothing greatly outperforms the standard differential optimization method from Agrawal et al. [2019a], while the projection distance regularization does not help a lot.

5 Conclusion

In this work, we discover and explain the zero-gradient problem in P&O for convex optimization. In particular, we show that the null space of the Jacobian of a convex optimization problem can get arbitrarily large, especially in the case with numerous constraints. This phenomenon prevents gradient-following algorithms from learning optimal solutions in convex P&O problems.

To resolve this issue, we introduce a method to compute an approximation of the Jacobian. It is done by smoothing the feasibility region around the current solution and thereby reducing the dimensionality of the null space to one. We prove that the combination of smoothing with the QP approximation results in the gradient update steps that at least do not decrease the task performance, but often allow to escape the zero-gradient cones. To enable movement along the remaining onedimensional null space, we add a projection distance regularization term. The suggested method leads to significantly better results for the convex P&O problems that suffer from the zero-gradient problem the most - those with many constraints and with the true optimum lying on the boundary of the feasibility set.

355

356

357

358

359

360

361

362

363

364

365

366

367

368

369

370

371

372

373

374

375

376

377

378

379

380

381

382

383

384

385

386

387

388

389

390

391

392

393

394

395

396

397

398

399

400

References

Akshay Agrawal, Brandon Amos, Shane Barratt, Stephen Boyd, Steven Diamond, and J Zico Kolter. Differentiable convex optimization layers. Adv. Neural Inf. Process. Syst. , 32, 2019a.

Akshay Agrawal, Shane Barratt, Stephen Boyd, Enzo Busseti, and Walaa M Moursi. Differentiating through a cone program. April 2019b.

Akshay Agrawal, Shane Barratt, Stephen Boyd, and Bartolomeo Stellato. Learning convex optimization control policies. pages 1-26, 2019c.

Brandon Amos and J Zico Kolter. OptNet: Differentiable optimization as a layer in neural networks. 34th International Conference on Machine Learning, ICML 2017 , 1:179-191, 2017.

RC Bansal. Optimization methods for electric power systems: An overview. International Journal of Emerging Electric Power Systems , 2(1), 2005.

Quentin Berthet, Mathieu Blondel, Olivier Teboul, Marco Cuturi, Jean Philippe Vert, and Francis Bach. Learning with differentiable perturbed optimizers. Adv. Neural Inf. Process. Syst. , 2020Decem:1-24, 2020.

Bingqing Chen, Priya L Donti, Kyri Baker, J Zico Kolter, and Mario Bergés. Enforcing policy feasibility constraints through differentiable projection for energy optimization. e-Energy 2021 Proceedings of the 2021 12th ACM International Conference on Future Energy Systems , pages 199-210, 2021.

Gerard Cornuejols and Reha Tütüncü. Optimization methods in finance , volume 5. Cambridge University Press, 2006.

Priya L Donti, Brandon Amos, and J Zico Kolter. Task-based end-to-end model learning in stochastic optimization. Adv. Neural Inf. Process. Syst. , 2017-Decem:5485-5495, 2017.

Adam N Elmachtoub and Paul Grigas. Smart" predict, then optimize". arXiv preprint arXiv:1710.08005 , 2017.

Aaron Ferber, Bryan Wilder, Bistra Dilkina, and Milind Tambe. MIPaaL: Mixed integer program as a layer. AAAI 2020 - 34th AAAI Conference on Artificial Intelligence , pages 1504-1511, 2020.

Anthony V Fiacco. Sensitivity analysis for nonlinear programming using penalty methods. Mathematical programming , 10(1):287-311, 1976.

Michael Grant, Stephen Boyd, and Yinyu Ye. Disciplined Convex Programming , pages 155-210. Springer US, Boston, MA, 2006. ISBN 978-0-387-30528-8.

Jack Kiefer and Jacob Wolfowitz. Stochastic estimation of the maximum of a regression function. The Annals of Mathematical Statistics , pages 462-466, 1952.

Harold W Kuhn and Albert W Tucker. Nonlinear programming. In Berkeley Symposium on Mathematical Statistics and Probability , 2:481-492, 1951.

Jia Li, Feng Liu, Zhaojian Wang, Steven H. Low, and Shengwei Mei. Optimal Power Flow in Stand-Alone DC Microgrids. IEEE Transactions on Power Systems , 33(5):5496-5506, 2018. ISSN 08858950. doi: 10.1109/TPWRS.2018.2801280.

J Mandi, E Demirovic, P J Stuckey, and T Guns. Smart predict-and-optimize for hard combinatorial optimization problems. Proc. Conf. AAAI Artif. Intell. , 2020.

Jayanta Mandi and Tias Guns. Interior point solving for LP-based prediction+optimisation. October 2020.

Harry M Markowitz and G Peter Todd. Mean-variance analysis in portfolio choice and capital markets , volume 66. John Wiley & Sons, 2000.

Ayan Mukhopadhyay and Yevgeniy Vorobeychik. Prioritized allocation of emergency responders based on a continuous-time incident prediction model. In International Conference on Autonomous Agents and MultiAgent Systems , 2017.

401

402

403

404

405

406

407

408

409

410

411

412

413

414

415

416

417

Chao Ning and Fengqi You. Optimization under uncertainty in the era of big data and deep learning: When machine learning meets mathematical programming. Computers & Chemical Engineering , 125:434-448, 2019.

QUANDL. Quandl wiki prices, 2020., 2020.

Purushothaman Raja and Sivagurunathan Pugazhenthi. Optimal path planning of mobile robots: A review. International journal of physical sciences , 7(9):1314-1320, 2012.

Ayse Sinem Uysal, Xiaoyue Li, and John M Mulvey. End-to-End risk budgeting portfolio optimization with neural networks. July 2021.

Grigorii Veviurko, Wendelin Böhmer, Laurens Mackay, and Mathijs de Weerdt. Surrogate dc microgrid models for optimization of charging electric vehicles under partial observability. Energies , 15 (4):1389, 2022.

Marin Vlastelica, Anselm Paulus, Vít Musil, Georg Martius, and Michal Rolínek. Differentiation of blackbox combinatorial solvers. pages 1-19, 2019.

Kai Wang, Bryan Wilder, Andrew Perrault, and Milind Tambe. Automatically learning compact quality-aware surrogates for optimization problems. June 2020.

Bryan Wilder, Bistra Dilkina, and Milind Tambe. Melding the Data-Decisions pipeline: DecisionFocused learning for combinatorial optimization. AAAI , 33(01):1658-1665, July 2019.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper first characterizes the 'zero-gradient' issue---a challenge associated with learning a model in the 'predict-then-optimize' paradigm---in terms of the number of active KKT constraints of the optimization problem. It then proposes a surrogate optimization problem for which the zero-gradient does not arise and evaluates these surrogates on 2 domains.\n\nSTRENGTHS:\n* The paper addresses an important problem, i.e., zero-gradients in predict-then-optimize.\n* The paper proposes a novel surrogate.\n\nWEAKNESSES:\n1. **The 'zero-gradient theorem' is not novel:** The paper claims to 'discover and explain the zero-gradient problem in P&O for convex optimization'. However, a number of predict-then-optimize papers acknowledge the zero-gradient issue and propose their own surrogates, e.g., Elmachtoub and Grigas [2017] and Wilder et al. [2019]. In fact, for well-defined LPs (with unique solutions), it is known that the task performance/SPO loss is piecewise constant. In contrast, it is known that if the function and constraints are strongly convex, e.g., the portfolio optimization problem in the experiments with $\\lambda > 0$, there are no zero-gradients. As a result, it's not clear why this is the domain that the paper chooses to run experiments on...\n1. **The experiments have no baselines:** While the paper does provide a surrogate that does not run into a zero-gradient issue, the bar for publication is typically higher, i.e., that this specific surrogate outperforms others from the literature. As the related work section notes, there are other ways to get around the zero-gradient issue, like creating surrogate problems by adding quadratic/exponential regularization terms, however there are no comparisons to any methods not presented in the paper, not even simple baselines like 2-stage, random and optimal.\n1. **The paper is poorly written:** There is almost no information in the about the contributions in the abstract and introduction and, as noted above, the paper does not adequately engage with past work.\n\n### Update (20 Aug 2023)\n\nAfter the discussion with the authors, reading the other reviews and thinking about this paper more, I find myself still recommending rejection. Here are the reasons:\n1. The *submitted* version of the paper lacks any discussion of or comparison to related work, and also overclaims (e.g., \"the first to discover the zero-gradient theorem\"). While the authors have refined their position significantly in the rebuttals (and much improved their contribution as a result), I believe that (a) there still remain important unanswered questions, and (b) these changes lead to a paper with significantly different claims. I discuss both of these in terms of specific contributions below.\n1. **Zero-gradient Theorem:** In the responses, the authors acknowledge that the zero-gradient issue has been known for optimization problems with linear objectives. The modified claim, as I understand it, is that the characterization of zero-gradients in this paper is significantly different/improved from the existing understanding. However, (a) the proof is not particularly novel (imo) because it formalizes existing knowledge in the language of KKT conditions, and moreover (b) it is not clear whether this characterization is significantly more powerful than the existing understanding (specifically, do gradients lie in the null space of the normals of the active constraints when the Jacobian matrix isn't zero, and what does this mean intuitively?). I believe that the strength of this contribution is dependent on the answers to these two (imo) unanswered questions.\n1. **r-Smoothing Surrogate:** While avoiding zero-gradient issues is definitely a desirable property for a surrogate, it is by no means a *sufficient* for good performance - a number of papers in the literature do not run into zero-gradient issues. As a result, r-smoothing needs to be better motivated and compared to past work. While the authors have done an admirable job of running experiments in the author response period, there are some papers that do very similar things that the authors have not compared to, e.g., Sahoo et. al. (2022). I think there is work that remains to be done in situating their surrogate in the context of the large body of recent work on PtO.\n\nQUESTIONS:\n1. How does your proposed approach do in comparison to baselines from the literature?\n\nLIMITATIONS:\nThere are no limitations discussed; in contrast, I believe this paper over-claims its contributions.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-04"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper identifies the zero-gradient problem in Predict and Optimize (P&O) for convex optimization and proposes a method to address it. The method is based on using a Quadratic Programming (QP) approximation for computing decisions, smoothing the feasibility region around the current solution to reduce the dimensionality of the null space to one, and adding a projection distance regularization term. The proposed method demonstrates significant improvements for convex P&O problems with many constraints and with the true optimum lying on the boundary of the feasibility set.\n\nSTRENGTHS:\nOriginality: The paper identifies a previously unnoticed problem in convex optimization and proposes a novel method to solve it.\n\nQuality: The proposed method is technically sound, and the experiments demonstrate its effectiveness in addressing the zero-gradient problem.\n\nClarity: The paper is well-written and clearly explains the concepts and methodology.\n\nSignificance: The proposed method has the potential to improve optimization in convex P&O problems, which are common in various domains.\n\nWEAKNESSES:\nInsufficient experiments: The paper might lack a comprehensive set of experiments or fail to compare the proposed method with alternative approaches. This could make it difficult for readers to evaluate the true effectiveness and novelty of the proposed method.\n\nQUESTIONS:\nIn Section 3.2, the authors present a Quadratic Programming (QP) approximation for computing decisions, which substitutes the original objective function f(x, w) with an alternative objective function $f_{QP}(x, w)$. Although it may appear counterintuitive that using a separate objective function does not influence the final solution, the key motivation behind the QP approximation is its ability to simplify Jacobian computations and tackle the zero-gradient problem. However, in the experimental section, the authors only demonstrate the effectiveness of the QP approximation in one case, varying the parameter $\\lambda$ and presenting the results in Table 1. Consequently, I am curious about how this approximation performs in other optimization problems. Additionally, the authors only test their overall method in two cases, which may not sufficiently demonstrate the method's robustness and generalizability. I would appreciate further insights into the performance of the proposed approach across a wider range of problems and scenarios.\n\nIs the zero-gradient problem universal in all predict and optimize for convex problems?  For example, if the surrogate function is not KKT-based and instead uses an extra large penalty term to penalize the constraints, would the same phenomenon exist[1], or is it specific to KKT-based techniques or the standard technique mentioned in the experiments? It would be helpful if the authors could clarify the scope of their contributions and avoid overclaiming.\n\nIn Line 260, the authors introduce the parameter $\\alpha$, but it seems that this parameter is not discussed in the experiments section. Can the authors provide more information on how $\\alpha$ is chosen or tuned in the experiments, and how its choice affects the performance of the proposed method?\n\nDoes the term \"standard\" in the figure refer to the work of [2]? It appears that the authors do not explicitly mention this term in the main text. Can the authors clarify the connection between the \"standard\" and the cited work, and if possible, provide a clearer definition or explanation of the term within the paper?\n\n\nReferences:\n\n[1] A Surrogate Objective Framework for Prediction+ Programming with Soft Constraints. Advances in Neural Information Processing Systems, 2021.\n\n[2] Differentiable convex optimization layers. Advances in Neural Information Processing Systems, 2019\n\nLIMITATIONS:\nSee Weaknesses", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-05"}

comments: SUMMARY:
This paper studies predict and optimize problem which utilizes machine learning to predict unknown parameters of optimization problems. The paper identifies the zero-gradient problem and proposes a method to solve this issue. Additionally, the paper conducts an experimental study to verify the proposed method.

STRENGTHS:
1. This paper is technically sound. The claims regarding the zero-gradient problem in the paper are well-supported by theoretical analysis. The assumptions are clearly presented, and proof ideas are discussed after each theorem. The efficiency of the proposed solution to the zero-gradient problem is verified by the experimental results.
2. The paper is well-organized. It begins by introducing the problem formulation of predict and optimize and discusses the typical methods used to solve the problem. Then, the paper introduces the zero-gradient problem along with the theoretical analysis. Finally, the proposed solution and experimental results are presented.

WEAKNESSES:
The paper can be improved by also experimentally demonstrating the yet noticed zero-gradient problem claimed in his paper.  Demonstrating the the consistency between the theoretical findings and experimental observations can enhance the significance of this paper.

QUESTIONS:
1. Line 90-91. The parameter w is unknown. In this case, how can one minimize the loss function define in Eq (1)?
2. Is there any way to theoretically analyze and show how the proposed OP and local smoothing methods solve the zero-gradient problem?

LIMITATIONS:
NA
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-09

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nPredict+Optimize (P+O) is an emerging paradigm that lies in the intersection of classical optimization  and machine learning. Specifically, it considers the setting where a parameterized optimization problem:\n\n$$x^{\\star}(w) = \\operatorname*{argmin}_{x} f(x,w) \\text{ subject to } x \\in \\mathcal{C}$$\n\nmust be solved yet the parameters $w$ are unknown. Given observational data $o$ that is correlated with $w$, a natural approach is to train a machine learning model $\\hat{w} = \\phi_{\\theta}(0)$ so that $\\hat{w} \\approx w$. Then at test time, we solve\n\n$$ x^{\\star}(\\hat{w}) = \\operatorname*{argmin}_{x} f(x,\\hat{w}) \\text{ subject to } x \\in \\mathcal{C} $$\n\nThe secret sauce to P+O is, instead of training to minimize prediction error $\\|\\hat{w} - w\\|^2$, to use a loss function aligned with the actual goal {\\em i.e.} that encourages $ x^{\\star}(\\hat{w}) \\approx  x^{\\star}(w)$. There are several ways to do this, but one is to simply reuse the objective function and train $\\phi_{\\theta}$ so as to minimize\n\n$$ \\mathbb{E}_{(o,w)}\\left[f(x^{\\star}(\\hat{w}, w)\\right]  $$\n\nwhere $\\hat{w} = \\phi_{\\theta}(o)$. Although a formula for the derivative of this loss is well-known, the core claim of this paper is that this derivative is less informative than previously thought. In fact, it is frequently zero. This idea is formalized through a theorem. The authors then propose a way to overcome this aplty named \"zero-gradient\" problem. Finally, the paper is rounded out by numerical experiments on two datasets.\n\nSTRENGTHS:\n- The main strength of this paper is Lemma 3.4 and Theorem 3.5, which crystalize the core claim of this paper. This result is surprising, but I checked the proof to the best of my ability and I believe it is correct. This result is an important reality check for the field of Predict-and-Optimize.\n- I enjoyed reading the proofs. The result of [1] were new to me. I liked the way the strict complementary slackness is used in the proof of Lemma 3.4\n- Adding to the above, the authors do a good job of making their core results accessible through intuitive explanations and diagrams.\n\n[1] Anthony V Fiacco. _Sensitivity analysis for nonlinear programming using penalty methods_ (1976).\n\nWEAKNESSES:\n- Reusing the parameterized objective function $f(\\cdot,w)$ as the loss function for training is not the only way to do P+O. One could also use the SPO+ loss [1], a perturbation based approach [2], or the least squares loss $\\|x^{\\star}(w) - x^{\\star}(\\hat{w})\\|^2$ [3]. This should be mentioned as the zero-gradient theorem need not apply in these settings.\n- I am perplexed at the stated motivation behind the quadratic programming approximation. While it is true that $f_{QP}(x,\\hat{w}) = \\|x - \\hat{w}\\|^2$ is strongly concave, and so on, it need not bear any relation to the actual problem we wish to solve, namely $f(x,w)$. So, this seems to run counter the spirit of P+O. The only case that makes sense to me is when $f(x,w) = w^{\\top}x$. Expanding out we get:\n\n$$ f_{QP}(x,\\hat{w}) = \\|x - \\hat{w}\\|^2 = -2\\hat{w}^{\\top}x + \\|x\\|^2 + \\|\\hat{w}\\|^2 $$\n\nSo ignoring the irrelevant $\\|\\hat{w}\\|^2$ term,  it appears the authors are simply proposing to add a quadratic regularizer, which has been explored thoroughly in the literature (see [4] and elsewhere).  Could the authors comment on this?\n- I find the motivation behind $r$-smoothing a little opaque too. It seems as though the solution to the $r$-smoothed problem $P_{r}(\\hat{x}\\hat{w})$ might not be feasible (i.e. might not lie in $\\mathcal{C}$). Is this correct?\n- Using the Jacobian $\\nabla_{\\hat{w}} x_r^{\\star}(\\hat{w})$ in place of $\\nabla_{\\hat{w}}x^{\\star}(\\hat{w})$ is, as you show, essentially the same as just replacing $\\nabla_{\\hat{w}}x^{\\star}(\\hat{w})$ with the identity (independent of what $r$ is). This procedure is already well-studied, see [3, 5--7]. These papers should be cited and discussed. \n\n*Minor Stuff:*\n- In Figure 2, the smoothed feasibility region $\\mathcal{C}_r(\\hat{x}\\hat{w})$ is the disk (i.e. the interior of the circle) right? If yes, this should be made clear in the caption and figure. Right now it looks as though the feasible region is just the boundary.\n - In Definition 3.7, as the scale of $r$ doesn't really matter, I'd recommend not normalizing and simply writing $c = \\hat{x} - r\\nabla_xf(\\hat{x},\\hat{w})$.\n - The experiments in Section 4 feel like ablation studies (i.e. just removing one element at a time from your proposed approach). I would like to see some benchmarking results, e.g. comparing the performance of your proposed algorithm to existing P+O approaches. You may find the benchmarking software PyEOPO useful for this [8]\n\n[1] Adam N Elmachtoub and Paul Grigas. _Smart predict, then optimize_ (2017)\n\n[2] Quentin Berthet et al. _Learning with differentiable perturbed optimizers_ (2020)\n\n[3] Daniel McKenzie et al _Faster predict-and-optimize with three-operator splitting_ (2023)\n\n[4] Bryan Wilder et al _Melding the Data-Decisions pipeline: Decision Focused learning for combinatorial optimization_ (2019)\n\n[5] Samy Wu Fung et al _JFB: Jacobian-Free Backpropagation for Implicit Networks_ (2022)\n\n[6] Zhengyang Geng et al _Is attention better than matrix decomposition?_ (2022)\n\n[7] SS Sahoo et al _Backpropagation through combinatorial algorithms: Identity with projection works_ (2022)\n\n[8] Tang and Khalil _PyEPO: A PyTorch-based end-to-end predict-then-optimize library for linear and integer programming_ (2023).\n\nQUESTIONS:\nSee \"weaknesse\" above.\n\nLIMITATIONS:\nN/A.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-11"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper focus on the topic of \"predict and optimize\" and identify the zero-gradient problem. This issue is characterized by a situation where the gradient related to the best decision concerning parameters in machine learning models might be zero. This can occur even when assuming convexity, smoothness, and strict complementary slackness. The authors introduce a QP approximation and an r-smooth technique to address this problem, effectively reducing the likelihood of encountering the zero-gradient issue. The numerical results show the merits of their proposed approach.\n\nSTRENGTHS:\n1. Section 3.1 uses straightforward theoretical outcomes to shed light on a significant practical problem. This section is clear and insightful.\n2. Section 3.3 stands out for its clarity and insight as well, particularly Theorem 3.9. Initially, I was skeptical that the gradient with local smoothness would accurately approximate the original gradient, but Theorem 3.9 effectively argued this point. Although the locally smoothed gradient may not ensure the fastest improvement in function value as the original gradient does, it can be guaranteed to at least be a non-decreasing direction.\n3. Figures (b) and (c) look interesting, showing that the new methods outperform standard approaches during the final training phase. This aligns with the theories in Section 3.1, where, towards the end of training, the conditions described in Theorem 3.5 become more likely. This leads to training difficulties with the standard method, whereas the proposed techniques can overcome them. Further validation on this matter is needed, as indicated in the third point under \"Weaknesses.\"\n\nWEAKNESSES:\n1. Sec 3.4, Algorithm 1. The notation seems unclear. What's the dimension of $f_x$ and $\\hat{f_x}$? Why $f_x$ can be directly multiplied with $\\nabla_{\\hat{w}}x^*_r(\\hat{w})$? Does it mean inner product of two vectors? What's the meaning of $f_x - f^0$ given $f^0$ is a scalar while $f_x$ is a gradient (vector)?\n\n2. Sec 4.1, equation (8). What's the meaning of $w$? Does $f(x,w)$ mean the original function defined in (7) or the QP approximation defined in (9)? This is quite critical: if (8) measures the regret based on QP approximation rather than the original function, the experiment results would be meaningless. While if (8) measures the original function, the numerical results will be good.\n\n3. Is it possible to provide the norms of the gradients you observed in the experiments? If the gradients calculated with your proposed approaches have larger norms than the traditional calculation way, it would be a more direct and strong support of your approach.\n\nQUESTIONS:\nsee \"Weaknesses\"\n\nLIMITATIONS:\nsee \"Weaknesses\"", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-31"}