Title: Gradient-Free Methods for Nonconvex Nonsmooth Stochastic Compositional Optimization
Venue: NeurIPS 2024
Accepted: yes
Keywords: compositional optimization; nonsmooth analysis
Abstract: The stochastic compositional optimization (SCO) is popular in many real-world applications, including risk management, reinforcement learning, and meta-learning. However, most of the previous methods for SCO require the smoothness assumption on both the outer and inner functions, which limits their applications to a wider range of problems. In this paper, we study the SCO problem in that both the outer and inner functions are Lipschitz continuous but possibly nonconvex and nonsmooth. In particular, we propose gradient-free stochastic methods for finding the $(\delta, \epsilon)$-Goldstein stationary points of such problems with non-asymptotic convergence rates. Our results also lead to an improved convergence rate for the convex nonsmooth SCO problem. Furthermore, we conduct numerical experiments to demonstrate the effectiveness of the proposed methods.

Gradient-Free Methods for Nonconvex Nonsmooth Stochastic Compositional Optimization

Abstract

Stochastic compositional optimization (SCO) problems are popular in many realworld applications, including risk management, reinforcement learning, and metalearning. However, most of the previous methods for SCO require the smoothness assumption on both the outer and inner functions, which limits their applications to a wider range of problems. In this paper, we study the SCO problem in that both the outer and inner functions are Lipschitz continuous but possibly nonconvex and nonsmooth. In particular, we propose gradient-free stochastic methods for finding the ( δ, ϵ ) -Goldstein stationary points of such problems with non-asymptotic convergence rates. Our results also lead to an improved convergence rate for the convex nonsmooth SCO problem. Furthermore, we conduct numerical experiments to demonstrate the effectiveness of the proposed methods.

1 Introduction

In this paper, we consider the following stochastic compositional optimization (SCO) problem:

where the outer and inner functions f : R m → R and g : R d → R m has the form of

and the stochastic components F ( y ; ξ ) and G ( x ; ζ ) are Lipschitz continuous but possibly nonconvex and nonsmooth. Random variables ξ and ζ are independent. Such formulation is popular in many realworld applications, including risk management [1], statistical learning [2], reinforcement learning [3], and model agnostic meta-learning [4].

Most of the existing work [2, 5, 6, 7, 8] for nonconvex SCO problem is based on the assumption that both functions f ( · ) and g ( · ) are smooth. Unfortunately, many modern machine learning

models including deep neural networks do not satisfy the smoothness condition. Ruszczynski [9] proposed a single time-scale stochastic subgradient method for solving the Problem (1). However, the author only provided asymptotic convergence analysis for the approach. In recent work, Liu and Davanloo Tajbakhsh [10], Hu et al. [11] presented the non-asymptotic convergence for the nonconvex nonsmooth SCO problem, while their analysis requires additional assumptions such as the weak-convexity and the relative smoothness condition.

The non-smoothness in the Problem (1) implies the classical gradient-based approaches and the convergence measure in terms of the gradient norm cannot be applied. The Clarke subdifferential [12] for the Lipshitz continuous functions is a natural extension of gradients for the smooth functions. However, hard instances have shown that no deterministic or randomized algorithms can find an ϵ -stationary point with respect to the Clarke subdifferential of a Lipschitz function in finite time [13, 14]. To address this issue, Zhang et al. [13] proposed a refined notion of the ( δ, ϵ ) -Goldstein stationary point in terms of the Goldstein δ -subdifferential, which considers the convex hull of the Clarke subdifferential at points in the δ -neighbourhood [15].

In this paper, we propose a zeroth-order stochastic method called gradient-free compositional optimization method (GFCOM) for solving Problem (1) in finite time. In particular, we show that the GFCOM can find a ( δ, ϵ ) -Goldstein stationary point of the objective function Φ( · ) = f ( g ( · )) within the stochastic zeroth-order oracle complexity of O ( d 3 . 5 δ -3 ϵ -6 ) . Furthermore, we improve the GFCOM by using the variance reduction technique [16, 17, 18, 19] to establish a more efficient first-order oracle estimator, leading to the algorithm GFCOM + which achieves a tighter upper complexity bound of O ( d 3 . 5 δ -3 ϵ -5 ) . In addition, we study convex nonsmooth SCO problems. In this regime, prior methods [2, 20, 21] suffer two major limitations: (i) Their convergence analysis is based on the smoothness condition of the outer function. (ii) Their convergence result is measured by the sub-optimality of the function value gap, while the non-asymptotic convergence rate for finding the stationary point has not been studied. We overcome these issues by involving a warm-start strategy into GFCOM + , which is guarantee to find a ( δ, ϵ ) -Goldstein stationary point within O ( d 3 δ -2 . 4 ϵ -4 + d 3 . 5 δ -2 ϵ -5 ) stochastic zeroth-order oracle complexity. We summarize the complexity of proposed methods in Table 1.

2 Related Work

In this section, we review prior work for stochastic compositional optimization and classical nonconvex nonsmooth optimization.

2.1 Stochastic Compositional Optimization

In a pioneer work, Wang et al. [2] studied the non-asymptotic convergence of nonconvex smooth stochastic compositional optimization by proposing the stochastic compositional gradient descent (SCGD), which contains two sequences of stepsizes for different time scales to update the variable and track the inner function value, respectively. The authors also heuristically extended their methods to zeroth-order optimization. Wang et al. [5] proposed an accelerated variant of SCGD using an extrapolation-smoothing scheme, and Ghadimi et al. [6] proposed a single time-scale approach to accelerate the convergence further. Additionally, Hu et al. [7], Lin et al. [20], Yuan et al. [22] incorporated the variance reduction technique into the first-order iteration, achieving a tight stochastic first-order complexity under the mean-squared smoothness assumption.

Compared with the smooth counterpart, the study of nonsmooth compositional optimization is relatively scarce. Ruszczynski [9] proposed a single time-scale stochastic subgradient method for nonconvex nonsmooth SCO problems, while the theoretical analysis only provided the asymptotic convergence rate. Liu and Davanloo Tajbakhsh [10] introduced the stochastic composition Bregman gradient method and provided a non-asymptotic convergence analysis under the relative smoothness condition. Vladarean et al. [23] proposed a Frank-Wolfe algorithm for constrained nonconvex nonsmooth SCO problems. Their analysis assumes that the outer function is convex but possibly non-differentiable, and the inner function is smooth. Very recently, Hu et al. [11] studied stochastic methods for the finite-sum coupled compositional optimization problem. Their convergence rate is established by assuming both the outer and inner functions are weakly convex, and the outer function is non-decreasing. In addition, Kalogerias and Powell [24] studied the zeroth-order stochastic optimization for a specific compositional optimization problem in risk-aware learning.

2.2 Non-Asymptotic convergence Analysis of Nonconvex Nonsmooth Optimization

In this subsection, we present a literature review for classical nonconvex nonsmooth optimization. The study of this field has a long history [12, 25], but the non-asymptotic convergence analysis of nonsmooth optimization has only emerged in recent years. Zhang et al. [13] provided the nonasymptotic complexity analysis of the interpolated normalized gradient descent method to achieve a ( δ, ϵ ) -Goldstein stationary point of a Lipschitz function with a nonstandard subgradient oracle. Davis et al. [26], Tian et al. [27] improved this method by introducing random perturbations in each iteration to remove the assumptions. Recently, Cutkosky et al. [28] proposed the optimal algorithm via the reduction from nonconvex nonsmooth optimization to online learning.

The development of non-asymptotic convergence analysis of zeroth-order methods for nonsmooth optimization was initiated by Nesterov and Spokoiny [29]. Later, Lin et al. [30] proposed gradientfree methods for this problem by establishing a relationship between the Goldstein δ -subdifferential and randomized smoothing. Chen et al. [31], Liu et al. [32] improved their results by leveraging the variance-reduction technique. Kornowski and Shamir [33] obtained a sharper bound by applying the reduction technique introduced by Cutkosky et al. [28] to the gradient-free setting. Liu et al. [34], Grimmer and Jia [35] further extends the methodology to the constrained setting. However, these methods do not apply to the nonconvex nonsmooth SCO Problem (1).

3 Preliminaries

In this section, we first present the notations and assumptions used in this paper, then introduce the convergence criteria for nonsmooth optimization and the randomized smoothing technique.

3.1 Notations and Assumptions

We use ∥·∥ to denote the Euclidean norm of a vector. We define B δ ( x ) ≜ { y ∈ R d : ∥ y -x ∥ ≤ δ } as the Euclidean ball centered at x ∈ R d with a radius δ > 0 . We let conv( A ) be the convex hull of the set A . For two given sets A and B , we define A × B as their Cartesian product. In addition, we denote f ◦ g as the function composition such that ( f ◦ g )( x ) ≜ f ( g ( x )) .

Throughout this paper, we assume the objective function ( 1 ) satisfies the following assumptions.

Assumption 3.1. We assume the stochastic component F ( · , ξ ) is L f ( ξ ) -Lipschitz for any given ξ , and the stochastic component G ( · , ζ ) is L g ( ζ ) -Lipschitz for any given ζ . That is, it holds

for any x , y ∈ R d and ˆ x , ˆ y ∈ R m . We also assume the Lipschitz parameters L f ( ξ ) and L g ( ζ ) have bounded second-order moments such that E ξ [ L f ( ξ ) 2 ] ≤ G 2 f and E ζ [ L g ( ζ ) 2 ] ≤ G 2 g for some constants G f , G g > 0 .

Remark 3.2 . We can verify that Assumption 3.1 implies the function f ( · ) is G f -Lipschitz, and the function g ( · ) is G g -Lipschitz by Jensen's inequality.

Assumption 3.3. We assume that there exists a constant σ 0 as the upper bound on the variance of the functions G ( · , ζ ) , such that for any x ∈ R d , we have E ζ [ ∥ G ( x , ζ ) -g ( x ) ∥ 2 ] ≤ σ 2 0 .

Assumption 3.4. We assume that the composite function Φ( · ) ≜ ( f ◦ g )( · ) is lower bounded such that Φ ∗ ≜ inf x ∈ R d Φ( x ) > -∞ .

3.2 Convergence Criteria for Nonsmooth Functions

We introduce the definitions of the Clarke subdifferential and approximate Clarke stationary points.

Definition 3.5 (Clarke [12]) . The Clarke subdifferential of a Lipschitz function f at a point x is defined as ∂f ( x ) ≜ conv { g : g = lim x k → x ∇ f ( x k ) } . Furthermore, we call a point x an ϵ -Clarke stationary point of f if it holds that min {∥ g ∥ : g ∈ ∂f ( x ) } ≤ ϵ .

Zhang et al. [13], Kornowski and Shamir [14] showed that no deterministic or randomized algorithm could find an ϵ -Clarke stationary point in finite time. Consequently, Zhang et al. [13] considered a refined notion of approximate stationary point in terms of the Goldstein δ -subdifferential.

Definition 3.6 (Zhang et al. [13]) . Given a Lipschitz function f : R d → R and δ > 0 , the Goldstein δ -subdifferential of f at point x ∈ R d is defined as ∂ δ f ( x ) := conv( ∪ y ∈ B δ ( x ) ∂f ( y )) , which is the convex hull of the Clarke subdifferential at the points in the δ -neighbourhood of x . Additionally, a point x ∈ R d is called a ( δ, ϵ ) -Goldstein stationary point of f ( · ) if it holds that min {∥ g ∥ : g ∈ ∂ δ f ( x ) } ≤ ϵ .

Recent work [13, 26, 28] has shown that it is possible to find a ( δ, ϵ ) -Goldstein stationary point of a nonsmooth problem without a composition structure in finite time. However, these theories are not applicable to nonsmooth SCO. In particular, we can infer from Rademacher's theorem and Assumption 3.1 that the composite function Φ( · ) is differentiable almost everywhere. Let Q ⊆ R d be the set on which Φ is differentiable, then R d \ Q is of measure zero. Recent work assumes they have access to the unbiased stochastic gradient estimator of the objective function for any x ∈ Q . In our setting, the unbiased gradient estimator of the composite function Φ( x ) is J G ( x ; ζ ) ∇ F ( g ( x ); ξ ) , where J G is the Jacobian matrix of the function G ( · ; ζ ) . However, such an estimator is hard to obtain because the function value g ( x ) is an expectation.

3.3 Randomized Smoothing

The randomized smoothing is a popular technique for nonsmooth analysis [36] and gradient-free optimization [29]. Concretely, given a Lipschitz function f and a uniform distribution P on a unit ball, we define its smoothed surrogate as f δ ( x ) = E u ∼P [ f ( x + δ u )] , which has the following properties.

Lemma 3.7 (Lin et al. [30, Proposition 2.3]) . Let f δ ( x ) = E u ∼P [ f ( x + δ u )] where P is a uniform distribution on a unit ball in ℓ 2 -norm. Suppose the function f is L -Lipschitz, then we have (a) | f δ ( x ) -f ( x ) | ≤ δL ; (b) f δ ( · ) is differentiable everywhere and L -Lipschitz with cL √ dδ -1 -Lipschitz gradient, where c is some positive constant; (c) ∇ f δ ( x ) ∈ ∂ δ f ( x ) for any x ∈ R d .

Moreover, we consider the following unbiased gradient estimator of the smoothed surrogate function f δ ( · ) , which can be obtained from two function query oracle calls on points uniformly sampled from a unit sphere [37].

Lemma 3.8 (Lin et al. [30, Lemma D.1]) . Let f ( x ) = E [ F ( x ; ξ )] be a L -Lipschitz function. We denote

where u is uniformly sampled from a distribution on a unit sphere in R d space. Then, we have E [ ι ( x ; u , ξ )] = ∇ f δ ( x ) and E [ ∥ ι ( x ; u , ξ ) ∥ 2 ] ≤ 16 √ 2 πdL 2 .

4 Algorithms for Nonconvex Nonsmooth SCO

In this section, we propose zeroth-order stochastic algorithms for solving the nonconvex nonsmooth SCO problem. We also provide non-asymptotic convergence analysis for the proposed methods.

4.1 The Algorithms

In this subsection, we propose the gradient-free compositional optimization method (GFCOM) and its accelerated variant GFCOM + . We first introduce the main intuition of the GFCOM. Consider the following hypothetical zeroth-order gradient estimator

where b f > 0 is the mini-batch size of the gradient estimator. By Lemma 3.8, the vector ¯ v t is an unbiased estimator of ∇ Φ δ ( x t ) . Unfortunately, it is intractable to obtain the function values g ( x t ± δ w t,j ) because g ( · ) is an expectation of stochastic component functions G ( · ; ζ ) . To remedy this issue, we introduce auxiliary variables y t,j and z t,j to approximate the inner function values g ( x t + δ w t,j ) and g ( x t -δ w t,j ) , respectively. In particular, the vectors y t,j and z t,j are mini-batch function estimators defined as follows

where b g > 0 is the mini-batch size of the function estimator. Accordingly, we use these two variables to replace the function calls of g ( · ) in the gradient estimator v t of Eq. (2). The complete procedure of the GFCOM is presented in Algorithm 1.

For the GFCOM + , we leverage the variance reduction technique to approximate ∇ Φ δ ( x t ) . In particular, we consider the following hypothetical recursive gradient estimator

where ¯ q t and ¯ q t -1 are mini-batch gradient estimator defined as follows.

where b ′ f > 0 is the mini-batch size and k ∈ { t -1 , t } . Compared with the mini-batch gradient estimator (2), the recursive gradient estimator (4) has been shown to achieve a sharper complexity bound in nonconvex optimization literature [17, 18, 31]. However, the gradient estimator is computationally intractable due to the unknown function g ( · ) . Similar to the development of Algorithm 1, we define y t , z t to estimate the inner function values g ( x t ± δ w t,j ) . We also introduce variables y t -1 , z t -1 to approximate the inner function values g ( x t -1 ± δ w t,j ) at the previous iteration. Then we define stochastic gradient estimators q t and q t -1 in terms of y t , y t -1 , z t and z t -1 .

for k ∈ { t -1 , t } . We replace the minibatch gradient estimator ¯ q t and ¯ q t -1 in the recursive gradient estimator ¯ v t of Eq. (4) with the refined gradient estimators q t and q t -1 . The complete procedure of GFCOM + is presented in Algorithm 2.

4.2 Convergence Analysis

In this subsection, we consider the complexity analysis of the proposed algorithms introduced in Section 4.1. We assume that Φ( x 0 ) -Φ ∗ ≤ R , where R > 0 is some constant.

The following theorem shows the convergence rate of solving the Problem (1) with the GFCOM method presented in Algorithm 1.

Theorem 4.1. Under Assumption 3.1, 3.3 and 3.4, running the GFCOM algorithm (Algorithm 1) with η ≤ δ/ ( cG f G g √ d ) where c > 0 is some constant, then the output x R satisfies

Using Theorem 4.1 with the parameter setting

we obtain the following oracle complexity result for Algorithm 1.

Corollary 4.2. Under Assumption 3.1, 3.3 and 3.4, the GFCOM algorithm (Algorithm 1) requires at most O ( d 3 . 5 G 5 f G 3 g σ 2 0 Rδ -3 ϵ -6 + d 3 . 5 G 6 f G 4 g σ 2 0 δ -2 ϵ -6 ) stochastic zeroth-order function query calls to obtain a ( δ, ϵ ) -Goldstein stationary point of Φ .

After giving the complexity bound of GFCOM in Corollary 4.2, we now consider the convergence analysis of GFCOM + . We will show that it enjoys a sharper complexity bound due to the utilization of the recursive gradient estimator. The following theorem shows the convergence rate of solving Problem (1) with the GFCOM + (Algorithm 2).

Theorem 4.3. Under Assumption 3.1, 3.3 and 3.4, running the GFCOM + algorithm (Algorithm 2) with η = δ/ (2 cG f G g √ d ) , b ′ f =Θ( dG f G g ϵ -1 ) and m =Θ( G f G g ϵ -1 ) , then the output x R satisfies

Algorithm 3: WS-GFCOM( x 0 , η 0 , T 0 , b g, 0 , η, T, b f , b g , b ′ f , b ′ g , m )

1 Let x 1 = GFCOM( x 0 , η 0 , T 0 , 1 , b g, 0 ) .

2 Option I (WS-GFCOM 2 ) : x 2 = GFCOM( x 1 , η, T, b f , b g ) .

3 Option II (WS-GFCOM + ) : x 2 = GFCOM + ( x 1 , η, T, b f , b ′ f , b g , b ′ g , m ) .

4 Return : x 2 .

Using Theorem 4.3 with the parameter setting

we obtain the following oracle complexity result for Algorithm 2.

Corollary 4.4. Under Assumption 3.1, 3.3 and 3.4, the GFCOM + algorithm (Algorithm 2) requires at most O ( d 3 . 5 G 4 f G 2 g σ 2 0 Rδ -3 ϵ -5 + d 3 . 5 G 5 f G 3 g σ 2 0 δ -2 ϵ -5 ) stochastic zeroth-order function query calls to obtain a ( δ, ϵ ) -Goldstein stationary point of Φ .

For both Theorem 4.1 and 4.3, we take c = 1 according to Lemma 8 of Duchi et al. [36].

4.3 Discussion

In Algorithm 2, we only apply the variance reduction technique to the outer function f ( · ) to accelerate our algorithm. In contrast, existing methods [2, 7, 22] for nonconvex smooth SCO also apply the technique on the inner function estimator to obtain an improved complexity bound. Here we briefly discuss the cause that leads to such a difference. For smooth optimization, the main intuition of the variance reduction technique is to establish a connection between the bound of the meansquare error term E [ ∥ v t -∇ Φ( x t )) ∥ 2 ] and the expected distance of iterates at successive iterations E [ ∥ x t -x t -1 ∥ 2 ] , which diminishes asymptotically. In our algorithm, we exploit the randomized smoothing with perturbed iterates x t ± δ w t,j to approximate the gradient of the smoothed surrogate function Φ δ ( x t ) . If we apply the variance reduction to the inner function estimator, the meansquare error E [ ∥ v t -∇ Φ δ ( x t )) ∥ 2 ] is bounded by the expected distance of the perturbed iterates at successive iterations E [ ∥ x t -x t -1 ± δ ( w t,j -w t -1 ,j ) ∥ 2 ] , which does not vanish asymptotically.

5 Extensions to Convex Nonsmooth SCO

In this section, we extend the result in Section 4.1 to study the convex nonsmooth SCO problem. Firstly, we introduce an additional assumption as follows.

Assumption 5.1. We suppose the function f ( x ) is convex and non-decreasing, G ( x ; ζ ) is convex for any given ζ , and the solution set X ∗ = arg min x ∈ R d Φ( x ) is non-empty.

From this assumption and Section 3.2.4 by Boyd and Vandenberghe [38], we can deduce that Φ( x ) is a convex function. We will show that stochastic algorithms obtain an improved convergence rate for the nonsmooth SCO problem with Assumption 5.1. To obtain a ( δ, ϵ ) -Goldstein stationary point of the problem, we propose a two-phase gradient-free stochastic method called warm-started GFCOM (WS-GFCOM) in Algorithm 3. The intuition is that we use the GFCOM method to get a sufficiently small sub-optimality in the first phase, and then we apply the proposed methods in Section 4.1 to find the stationary point in the second phase. We remark that using the GFCOM method for the first phase is justified by the observation that it can achieve optimal convergence rate in terms of the sub-optimality of function value gap given the access to the exact function value g ( x ) [39].

5.1 Convergence Analysis

In this subsection, we consider the complexity analysis of the proposed WS-GFCOM method. Let ˆ R ≜ min x ∈X ∗ ∥ x -x 0 ∥ . We characterize the convergence rate of the WS-GFCOM method for the convex nonsmooth SCO problem at the first phase with the following theorem.

Theorem 5.2. Under Assumption 3.1, 3.3 and 5.1, running the WS-GFCOM algorithm (Algorithm 3) with parameters η 0 = Θ ( ˆ R/ ( G f G g √ dT 0 ) ) , T 0 = Θ ( dG 2 f G 2 g ˆ R 2 ρ -2 ) , b g, 0 = Θ ( G 2 f σ 2 0 ρ -2 ) and δ = Θ ( ρG -1 f G -1 g ) , then the output x 1 satisfies E [Φ( x 1 ) -Φ ∗ ] ≤ ρ . In addition, the total zerothorder stochastic oracle complexity is at most O ( dG 4 f G 2 g σ 2 0 ˆ R 2 ρ -4 ) .

Theorem 5.2 implies that the initial suboptimality at the beginning of the second phase is bounded by ρ . Consequently, the total complexity of WS-GFCOM 2 (Algorithm 3 with Option I) is bounded by O ( dG 4 f G 2 g σ 2 0 ˆ R 2 ρ -4 + d 3 . 5 G 5 f G 3 g σ 2 0 ρδ -3 ϵ -6 + d 3 . 5 G 6 f G 4 g σ 2 0 δ -2 ϵ -6 ) . An appropriate choice of ρ leads to the following oracle complexity of Algorithm 3 with Option I.

Corollary 5.3. Under Assumption 3.1, 3.3 and 5.1, the WS-GFCOM 2 algorithm (Algorithm 3 with Option I) requires at most O ( d 3 G 4 . 8 f G 2 . 8 g σ 2 0 ˆ R 0 . 4 δ -2 . 4 ϵ -4 . 8 + d 3 . 5 G 6 f G 4 g σ 2 0 δ -2 ϵ -6 ) stochastic zeroth-order function query calls to obtain a ( δ, ϵ ) -Goldstein stationary point of Φ .

With a similar deduction, we can show that using the GFCOM + for the second phase can obtain an improved complexity bound. The following theorem shows the oracle complexity of Algorithm 3 with Option II.

Corollary 5.4. Under Assumption 3.1, 3.3 and 5.1, the WS-GFCOM + algorithm (Algorithm 3 with Option II) requires at most O ( d 3 G 4 f G 2 g σ 2 0 ˆ R 0 . 4 δ -2 . 4 ϵ -4 + d 3 . 5 G 5 f G 3 g σ 2 0 δ -2 ϵ -5 ) stochastic zeroth-order function query calls to obtain a ( δ, ϵ ) -Goldstein stationary point of Φ .

6 Experiments

Wecompare the proposed methods GFCOM and GFCOM + with a Kiefer-Wolfowitz style zeroth-order baseline method [2, 40]. In particular, the baseline gradient estimator is defined as

where y t,j and z t,j are function estimators defined in Eq. (3). ξ t,j and ξ ′ t,j are independent random variables. We test all the methods on the nonconvex penalized risk-averse portfolio management problem and a reinforcement learning (RL) problem. We set δ = 0 . 1 for the GFCOM and GFCOM + methods.

6.1 Nonconvex Penalized Portfolio Management

We consider the portfolio management problem with cappedℓ 1 regularizer [41]. Let x denote the investment quantity corresponding to N assets and r t ∈ R N denote the returns of N assets at timestamp t . We can formulate the portfolio management problem as the following nonsmooth compositional optimization problem

where β ( x ) = λ ∑ N i =1 min {| x i | , α } and λ, α > 0 are tunable hyperparameters. Specifically, the inner function G ( x ; ξ ) and outer function F ( w ; ζ ) can be formulated as

and

Both random variables ξ and ζ are uniformly sampled from { 1 , . . . , T } . We choose λ = 10 -5 and α = 2 in our experiments. The goal of the Problem (6) is to maximize the return while controlling the variance of the portfolio.

We compare all the methods on 6 different portfolio datasets formed on Size and Operating Profitability 2 . For all algorithms, we tune the stepsize among { 1 × 10 -5 , 3 × 10 -5 , . . . , 1 × 10 -3 , 3 × 10 -3 } .

0

1

2

3

Function Calls

1e8

0

100

200

300

400

500

Loss

GFCOM

GFCOM

0

1

2

3

Function Calls

1e8

0

100

200

300

400

500

Loss

GFCOM

GFCOM

0

1

2

3

Function Calls

1e8

0

100

200

300

400

Loss

GFCOM

GFCOM

(a) Asia Pacific ex Japan OP

(b) Europe OP

(c) Global ex US OP

0

1

2

3

Function Calls

1e8

0

100

200

300

Loss

GFCOM

GFCOM

0.0

0.5

1.0

1.5

2.0

Function Calls

1e8

0

200

400

600

800

Loss

GFCOM

GFCOM

0.0

0.5

1.0

1.5

2.0

Function Calls

1e8

0

200

400

600

Loss

GFCOM

GFCOM

(e) Global OP

(f) Japan OP

(g) North America OP

0.0

0.5

1.0

Function Calls

1e7

0.250

0.275

0.300

0.325

0.350

0.375

0.400

Loss

GFCOM

GFCOM

0.00

0.25

0.50

0.75

1.00

Function Calls

1e7

0.250

0.275

0.300

0.325

0.350

0.375

0.400

Loss

GFCOM

GFCOM

0.0

0.5

1.0

Function Calls

1e7

0.250

0.275

0.300

0.325

0.350

0.375

0.400

Loss

GFCOM

GFCOM

(a)

n

= 400

(b)

n

= 600

(c)

n

= 800

We choose the mini-batch size b f = b g = 1000 . In addition, we set b ′ f = 100 , b ′ g = 1000 and m = b f /b ′ f = 10 for the GFCOM + algorithm. Figure 1 shows that the GFCOM + algorithm converges much faster than the GFCOM and the baseline method across all datasets.

6.2 Application to Reinforcement Learning

We demonstrate an experiment on RL and verify the effectiveness of the proposed methods on value function evaluation. Let V π ( s ) be the value function of a state s under a policy π for all state s ∈ S where |S| = n . Let r s ′ ,s be the reward transition from s ′ to s , and γ > 0 is a discounting factor. Furthermore, we assume that the value of each state can be parameterized as a linear map of some feature map ψ s ∈ R d of the state s such that V π ( s ) = ⟨ ψ s , w ⟩ . Then we formulate the RL problem as a Bellman residual minimization problem

where P ss ′ is the probability transition matrix and h ( x ) = 1 -exp( -| x | /σ ) is a nonconvex nonsmooth loss which is more robust to adversarial outliers than the squared loss [42, 43]. Specifically, the inner function G ( x ; ξ ) and outer function F ( w ; ζ ) can be formulated as

and

In the above formulation, each ξ i is uniformly sampled from { P i 1 , . . . , P in } and ζ is uniformly sampled from { 1 , . . . , T } . We follow a similar experiment setup by Yuan et al. [22]. Specifically, we generate a Markov decision process with different numbers of states n ∈ { 400 , 600 , 800 } and 10 actions at each state. The transition probability matrix is generated from the uniform distribution from [0 , 1] . In addition, the rewards are sampled uniformly from [0 , 1] . In terms of hyperparameter setting, we choose b f = b g = 100 for all algorithms. In addition, we set b ′ f = 10 , b ′ g = 100 and m = b f /b ′ f = 10 for the GFCOM + algorithm. For other hyperparameters, we use the same setting for the portfolio management problem. The experimental results in Figure 2 show that the GFCOM + significantly outperforms other methods.

7 Conclusion

In this work, we propose novel zeroth-order algorithms for nonconvex nonsmooth stochastic compositional optimization. We present the non-asymptotic convergence rate of the proposed algorithms for obtaining a ( δ, ϵ ) -Goldstein point of the problem. Furthermore, we extend our methods with a warmstart phase to solve the convex nonsmooth SCO problem with improved convergence guarantees. We conduct numerical experiments on portfolio management and reinforcement learning problems to demonstrate the effectiveness of the proposed algorithms.

In future work, it is interesting to study the lower bound of the zeroth-order algorithms on nonconvex nonsmooth SCO. It is also interesting to investigate whether the complexity bound of zeroth-order algorithms can be further improved.

References

Darinka Dentcheva, Spiridon Penev, and Andrzej Ruszczy´ nski. Statistical estimation of composite risk functionals and risk optimization problems. Annals of the Institute of Statistical Mathematics , 69:737-760, 2017.

Mengdi Wang, Ethan X Fang, and Han Liu. Stochastic compositional gradient descent: algorithms for minimizing compositions of expected-value functions. Mathematical Programming , 161:419-449, 2017.

Richard S Sutton and Andrew G Barto. Reinforcement learning: An introduction . MIT press, 2018.

Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In International conference on machine learning , pages 1126-1135. PMLR, 2017.

Mengdi Wang, Ji Liu, and Ethan X Fang. Accelerating stochastic composition optimization. Journal of Machine Learning Research , 18(105):1-23, 2017.

Saeed Ghadimi, Andrzej Ruszczynski, and Mengdi Wang. A single timescale stochastic approximation method for nested stochastic optimization. SIAM Journal on Optimization , 30 (1):960-979, 2020.

Wenqing Hu, Chris Junchi Li, Xiangru Lian, Ji Liu, and Huizhuo Yuan. Efficient smooth non-convex stochastic compositional optimization via stochastic recursive gradient descent. Advances in Neural Information Processing Systems , 32, 2019.

Tianyi Chen, Yuejiao Sun, and Wotao Yin. Solving stochastic compositional optimization is nearly as easy as solving stochastic optimization. IEEE Transactions on Signal Processing , 69: 4937-4948, 2021.

Andrzej Ruszczynski. A stochastic subgradient method for nonsmooth nonconvex multilevel composition optimization. SIAM Journal on Control and Optimization , 59(3):2301-2320, 2021.

Yin Liu and Sam Davanloo Tajbakhsh. Stochastic composition optimization of functions without lipschitz continuous gradient. Journal of Optimization Theory and Applications , 198 (1):239-289, 2023.

Quanqi Hu, Dixian Zhu, and Tianbao Yang. Non-smooth weakly-convex finite-sum coupled compositional optimization. Advances in Neural Information Processing Systems , 36, 2024.

Frank H. Clarke. Optimization and nonsmooth analysis . SIAM, 1990.

Jingzhao Zhang, Hongzhou Lin, Stefanie Jegelka, Ali Jadbabaie, and Suvrit Sra. Complexity of finding stationary points of nonsmooth nonconvex functions. In Proc. ICML , pages 1117311182, 2020.

Guy Kornowski and Ohad Shamir. Oracle complexity in nonsmooth nonconvex optimization. Journal of Machine Learning Research , 23(314):1-44, 2022.

AA Goldstein. Optimization of lipschitz continuous functions. Mathematical Programming , 13: 14-22, 1977.

Lam M Nguyen, Jie Liu, Katya Scheinberg, and Martin Takáˇ c. Sarah: A novel method for machine learning problems using stochastic recursive gradient. In International conference on machine learning , pages 2613-2621. PMLR, 2017.

Cong Fang, Chris Junchi Li, Zhouchen Lin, and Tong Zhang. Spider: Near-optimal nonconvex optimization via stochastic path-integrated differential estimator. Advances in neural information processing systems , 31, 2018.

Zhe Wang, Kaiyi Ji, Yi Zhou, Yingbin Liang, and Vahid Tarokh. Spiderboost and momentum: Faster variance reduction algorithms. Advances in Neural Information Processing Systems , 32, 2019.

Zhize Li, Hongyan Bao, Xiangliang Zhang, and Peter Richtárik. Page: A simple and optimal probabilistic gradient estimator for nonconvex optimization. In International conference on machine learning , pages 6286-6295. PMLR, 2021.

Tianyi Lin, Chengyou Fan, Mengdi Wang, and Michael I Jordan. Improved sample complexity for stochastic compositional variance reduced gradient. In 2020 American Control Conference (ACC) , pages 126-131. IEEE, 2020.

Yibo Xu and Yangyang Xu. Katyusha acceleration for convex finite-sum compositional optimization. INFORMS Journal on Optimization , 3(4):418-443, 2021.

Huizhuo Yuan, Xiangru Lian, and Ji Liu. Stochastic recursive variance reduction for efficient smooth non-convex compositional optimization. arXiv preprint arXiv:1912.13515 , 2019.

Maria-Luiza Vladarean, Nikita Doikov, Martin Jaggi, and Nicolas Flammarion. Linearization algorithms for fully composite optimization. In The Thirty Sixth Annual Conference on Learning Theory , pages 3669-3695. PMLR, 2023.

Dionysios S Kalogerias and Warren B Powell. Zeroth-order stochastic compositional algorithms for risk-aware learning. SIAM Journal on Optimization , 32(2):386-416, 2022.

Marko M Makela and Pekka Neittaanmaki. Nonsmooth optimization: analysis and algorithms with applications to optimal control . World Scientific, 1992.

Damek Davis, Dmitriy Drusvyatskiy, Yin Tat Lee, Swati Padmanabhan, and Guanghao Ye. A gradient sampling method with complexity guarantees for lipschitz functions in high and low dimensions. Advances in neural information processing systems , 35:6692-6703, 2022.

Lai Tian, Kaiwen Zhou, and Anthony Man-Cho So. On the finite-time complexity and practical computation of approximate stationarity concepts of lipschitz functions. In International Conference on Machine Learning , pages 21360-21379. PMLR, 2022.

Ashok Cutkosky, Harsh Mehta, and Francesco Orabona. Optimal stochastic non-smooth nonconvex optimization through online-to-non-convex conversion. In International Conference on Machine Learning , pages 6643-6670. PMLR, 2023.

Yurii Nesterov and Vladimir Spokoiny. Random gradient-free minimization of convex functions. Foundations of Computational Mathematics , 17(2):527-566, 2017.

Tianyi Lin, Zeyu Zheng, and Michael Jordan. Gradient-free methods for deterministic and stochastic nonsmooth nonconvex optimization. Advances in Neural Information Processing Systems , 35:26160-26175, 2022.

Lesi Chen, Jing Xu, and Luo Luo. Faster gradient-free algorithms for nonsmooth nonconvex stochastic optimization. In International Conference on Machine Learning , pages 5219-5233. PMLR, 2023.

Chengchang Liu, Chaowen Guan, Jianhao He, and John Lui. Quantum algorithms for nonsmooth non-convex optimization. arXiv preprint arXiv:2410.16189 , 2024.

Guy Kornowski and Ohad Shamir. An algorithm with optimal dimension-dependence for zero-order nonsmooth nonconvex stochastic optimization. arXiv preprint arXiv:2307.04504 , 2023.

Zhuanghua Liu, Cheng Chen, Luo Luo, and Bryan Kian Hsiang Low. Zeroth-order methods for constrained nonconvex nonsmooth stochastic optimization. In Forty-first International Conference on Machine Learning , 2024.

Benjamin Grimmer and Zhichao Jia. Goldstein stationarity in lipschitz constrained optimization. arXiv preprint arXiv:2310.03690 , page 9, 2023.

John C Duchi, Peter L Bartlett, and Martin J Wainwright. Randomized smoothing for stochastic optimization. SIAM Journal on Optimization , 22(2):674-701, 2012.

John C Duchi, Michael I Jordan, Martin J Wainwright, and Andre Wibisono. Optimal rates for zero-order convex optimization: The power of two function evaluations. IEEE Transactions on Information Theory , 61(5):2788-2806, 2015.

Stephen P Boyd and Lieven Vandenberghe. Convex optimization . Cambridge university press, 2004.

Ohad Shamir. An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. Journal of Machine Learning Research , 18(52):1-11, 2017.

Jack Kiefer and Jacob Wolfowitz. Stochastic estimation of the maximum of a regression function. The Annals of Mathematical Statistics , pages 462-466, 1952.

Tong Zhang. Analysis of multi-stage convex relaxation for sparse regularization. Journal of Machine Learning Research , 11(3), 2010.

Chao Qu, Yan Li, and Huan Xu. Non-convex conditional gradient sliding. In international conference on machine learning , pages 4208-4217. PMLR, 2018.

Zebang Shen, Cong Fang, Peilin Zhao, Junzhou Huang, and Hui Qian. Complexities in projection-free stochastic non-convex minimization. In The 22nd International Conference on Artificial Intelligence and Statistics , pages 2868-2876. PMLR, 2019.

The appendix is organized as follows. Section A introduces supporting lemmas that are essential for the analysis of the proposed gradient-free SCO methods. Section B proves the convergence rate of the GFCOM method introduced in Section 4. Section C proves the convergence rate of the GFCOM + method which enjoys a better function oracle call complexity. Section D provides the convergence analysis of the WS-GFCOM 2 and WS-GFCOM + proposed in Section 5.

A Supporting Lemmas

Throughout the work, we define the variable g t ( x ) = 1 b g ∑ i ∈ [ b g ] G ( x ; ζ t,i ) for the GFCOM algorithm, and we denote

for the GFCOM + method. Now we introduce an important lemma which is useful for the analysis of both GFCOM and GFCOM + algorithms.

Lemma A.1. Under Assumption 3.1 and 3.3, for both Algorithms 1 and 2 it holds that

Proof. From the smoothness of Φ δ = ( f ◦ g ) δ , we have

Taking expectations on both sides of the inequality, we get the desired result.

B Convergence Analysis of Algorithm 1

Before giving the analysis of the convergence rate of Algorithm 1, we first present the bound of the mean-square error term E [ ∥ v t -∇ Φ δ ( x t ) ∥ 2 ] .

Lemma B.1. Under Assumption 3.1 and 3.3, for Algorithm 1 it holds that

Proof. Recall that v t = 1 b f ∑ j ∈ [ b f ] d 2 δ ( F ( y t,j ; ξ t,j ) -F ( z t,j ; ξ t,j )) , we have

The first inequality is due to ∥ a + b ∥ 2 ≤ 2 ∥ a ∥ 2 +2 ∥ b ∥ 2 for any a , b ∈ R d . The second inequality is due to Lemma 3.8. The last inequality follows from the G f -Lipchitzness of f ( · ) and Assumption 3.3.

We present the formal proof of Theorem 4.1 and Corollary 4.2 below.

B.1 Proof of Theorem 4.1

Proof. If we take η = δ cG f G g √ d and rearrange the formula in Lemma A.1, we have

Sum the above inequality from t = 0 to T -1 and divide both sides by ηT 2 , we have

In addition, by Lemma 3.7, we have

Combining the last two inequalities, we get the desired result.

B.2 Proof of Corollary 4.2

Proof. The total zeroth-order oracle calls can be bounded by

C Convergence Analysis of Algorithm 2

In this section, we consider the formal proof of the convergence rate of the GFCOM + method. First, we introduce the following lemma to bound the mean-square error between the recursive gradient estimator and the gradient of the surrogate function.

Lemma C.1. Let n t = ⌊ t/m ⌋ m . Under Assumption 3.1 and 3.3, for Algorithm 2 it holds that

Proof. By ∥ a + b ∥ 2 ≤ 2 ∥ a ∥ 2 +2 ∥ b ∥ 2 , we can infer that

To bound the first term of R.H.S., we have

The second equality follows from Lemma 3.8. The last inequality is due to E [ ∥ x -E [ x ] ∥ 2 ] ≤ E [ ∥ x ∥ 2 ] . Observe that

The first inequality is due to ∥ a 1 + · · · + a n ∥ 2 ≤ n ∥ a 1 ∥ 2 + · · · + n ∥ a n ∥ 2 . The second inequality is due to the Lipschitzness of both inner and outer functions with Assumption 3.3. Consequently, one has

The last inequality follows from Lemma 3.8. In addition, for t mod m = 0 we can bound

̸

The last inequality follows from the Lipschitzness of f and Assumption 3.3. Similarly, for t mod m = 0 we can bound

Putting everything together, we get the desired bound.

We present the formal proof of Theorem 4.3 and Corollary 4.4 below.

C.1 Proof of Theorem 4.3

Proof. Rearrange the terms in Lemma A.1, sum t from 0 to T -1 , and divide both sides by ηT 2 ,

The last inequality follows from Lemma C.1. If we choose hyperparameters as follows

Then we can deduce that 1 -cηG f G g √ d δ -10 md 2 G 2 f G 2 g η 2 δ 2 b ′ f ≤ 0 . By Lemma 3.7, we have

Therefore, we obtain the following result

C.2 Proof of Corollary 4.4

Proof. The total zeroth-order oracle calls can be bounded by

D Extensions to Convex Nonsmooth Functions

In this section, we present the formal proof of theorems presented in Section 5.

D.1 Proof of Theorem 5.2

Proof. Since G ( · ; ζ ) is convex function, g t ( · ) is also convex. Using the result of Section 3.2.4 of [38] and Assumption 5.1, we can deduce that f ◦ g t is a convex function. Let x ∗ = arg min x ( f ◦ g ) δ ( x ) , then we have

The first inequality follows from the convexity of f ◦ g t . The last inequality is due to Lemma 3.8. Observe that for ∀ x ∈ R d ,

Consequently, one has

where c 1 is some constant. The second equality is due to the linearity of expectations. The last inequality follows from Assumption 3.3 and Lipschitzness of f . Consequently, we have

To obtain E [( f ◦ g )( x 1 ) -( f ◦ g )( x ∗ )] ≤ ρ , we choose

D.2 Proof of Corollary 5.3

Proof. The total stochastic function oracle calls can be bounded by

D.3 Proof of Corollary 5.4

Proof. The total stochastic function oracle calls can be bounded by

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification:

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification:

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification:

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification:

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [No]

Justification: We are clearing the code with internal compliance and will release it upon approval.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification:

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification:

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification:

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification:

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [NA]

Justification: There are no potential positive or negative societal impacts of this work.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification:

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification:

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification:

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification:

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: The paper does not involve crowdsourcing or research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose a zero-th order algorithm for Stochastic Compositional Optimization (SCO) problems. Such problems are given in the form of a composition of two functions, each of them being random depending, and the corresponding objective should be minimizer in expectation. The authors consider the nonsmooth nonconvex setting and propose an algorithm which to tackle such problems. The algorithm is called GFCOM, and a variance reduction variant, GFCOM+ is proposed. The authors propose a complexity analysis for the proposed method, with explicit estimates for reaching Goldstein approximate stationary points. They extend their analysis to the convex setting and conclude with numerical experiments and a comparison with the method of Kiefer-Wolfowitz.\n\nI did not read all the details of the proof of the variance reduced algorithm and the convex analysis.\n\nSTRENGTHS:\nThe paper is well written, the assumptions are clearly presented. The analysis flows well and looks technically solid.\n\nWEAKNESSES:\nThe numerical section does not seem to use a specific compositional structure, so it constitutes a weak illustration of the relevance of the method. The paper fail to provide a convincing motivating example related to machine learning.\n\nThe value of $c$ should be given to actually implement the algorithm.\n\nQUESTIONS:\nLemma 3.7 and 3.8, what are the precise references to the paper of Lin et al, which part of the paper do the authors refer to?\n\nAre the random variables $\\xi$ and $\\zeta$ independent? If yes it should probably be stated.\n\nThis sentence is misleading: \"hard instances have shown that finding an $\\epsilon$-stationary point with respect to the Clarke subdifferential of a Lipschitz function in finite time is computationally intractable\". Being $\\epsilon$ stationary is not the right notion for nonsmooth optimization: even in the convex setting, the subdifferential does not tend to 0. The hardness result of Kornowski and Shamir is actually different from what is suggested by the sentence and relates to the impossibility of getting close to the critical set.\n\nTable 1: what is WS-GFCOM?\n\nThis sentence is misleading: \"Consequently, they considered a refined notion of approximate stationary point in terms of the Goldstein $\\delta$-subdifferential\". Kornowski and Shamir do not use this notion.\n\nSection 6: what is the underlying compositional structure? What are the random variables?\n\nEquality (2) is extremely misleading as a different expression is used in the algorithms and proofs.\n\nLIMITATIONS:\nSection 4.2, what is the value of $c$. Without this the algorithm cannot be implemented.\n\nThe convex section is probably of more limited interest than the rest.\n\nThe paper should probably contain a discussion, comparing the obtained complexity estimate with the more usual zero-th order stochastic optimization literature. What is the added cost of the stochastic compositional structure?", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-06-20"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper investigates stochastic compositional optimization (SCO) problems, which are popular in many real-world applications. The authors focus on nonconvex and nonsmooth SCO, and propose gradient-free stochastic methods for finding the $(\\delta,\\epsilon)$-Goldstein stationary points of such problems with non-asymptotic convergence rates. Furthermore, they also use variance reduction technique to accelerate their algorithms with improved results.\n\nSTRENGTHS:\n* The study of the SCO problem is highly significant, and the research on gradient-free algorithms is also of considerable value. \n* This paper is well-written and easy to understand.\n\nWEAKNESSES:\n* The technical novelty of this work is limited, as the proposed algorithms are mostly based on SGFM [1]. \n* The authors do not prove that $\\mathbf{v}\\_t$ is an unbiased gradient estimator of $\\nabla \\Phi_\\delta (\\mathbf{x}_t)$ (See Question 1 for details). \n* The authors utilize variance reduction technique to accelerate their algorithms. However, they do not review some important related work on variance reduction in this paper, e.g., SVRG, STORM and so on. \n\n[1] Lin et al. Gradient-free methods for deterministic and stochastic nonsmooth nonconvex optimization. NeurIPS, 2022.\n\nQUESTIONS:\n* My primary concern is whether $\\mathbf{v}\\_t$ in Algorithm 1 is an unbiased estimator of $\\nabla \\Phi\\_\\delta (\\mathbf{x}\\_t)$. Although Lin et al. (2022) have proven that $\\mathbf{v}\\_t$  in Eq. (2) is an unbiased estimator of $\\nabla f\\_\\delta (\\mathbf{x})$, such gradient estimation cannot be directly applied to the SCO problem by introducing auxiliary variables $\\mathbf{y}\\_t$ and $\\mathbf{z}\\_t$ in Eq. (3). Specifically, although the estimation of each layer function and its gradients is unbiased, i.e., $\\mathbb{E}\\_{\\xi\\_1} [F(\\mathbf{x};\\xi\\_1)]=f(\\mathbf{x})$, $\\mathbb{E}\\_{\\xi\\_1} [\\nabla F(\\mathbf{x};\\xi\\_1)]=\\nabla f(\\mathbf{x})$, and $\\mathbb{E}\\_{\\xi\\_2} [G(\\mathbf{x};\\xi\\_2)]=g(\\mathbf{x})$, the main challenge in SCO lies in obtaining an unbiased estimation of gradient $\\nabla f(g(\\mathbf{x}))$. This is because the expectation over $\\xi\\_2$ cannot be moved inside of $\\nabla f$ such that $\\mathbb{E}_{\\xi_1,\\xi_2} [\\nabla F(G(\\mathbf{x};\\xi_2);\\xi_1)]\\neq \\nabla f(g(\\mathbf{x}))$. \n* The proposed algorithms in this paper appear to be a direct application of existing algorithms [1] to the SCO problem. Additionally, the accelerated algorithms employ variance reduction technique, which have already been widely used for other  settings in SCO problems [2,3]. Could you elaborate more details on the technical challenge and novelty of your work? \n\n[1] Lin et al. Gradient-free methods for deterministic and stochastic nonsmooth nonconvex optimization. NeurIPS, 2022. \n\n[2] Yuan et al. Efficient smooth non-convex stochastic compositional optimization via stochastic recursive gradient descent. NeurIPS, 2019. \n\n[3] Zhang and Xiao. A stochastic composite gradient method with incremental variance reduction. NeurIPS, 2019.\n\nLIMITATIONS:\nSee above", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-07-10"}

comments: SUMMARY:
This work proposes two zeroth-order methods (including one variance-reduced method) for solving non-convex non-smooth stochastic compositional optimization (SCO). These two methods are further extended to solving convex non-smooth SCO. Theoretical analysis are provided to show the convergence guarantee of all proposed methods.

STRENGTHS:
The main contribution of this work is that it proposes the first zeroth-order methods for non-smooth SCO under non-convex and convex setting and presents convergence analysis. The paper is well-written and easy to follow.

WEAKNESSES:
My main concern is in the novelty of the proposed methods and their convergence analysis. Base on my understanding, the proposed four methods are extensions of the existing work [31]. [31] proposed four zeroth-order methods (GFM, GFM+, WS-GFM, WS-GFM+) for solving non-smooth optimization without compositional structure under both non-convex and convex setting. Based on GFM in [31], the GFCOM in this work simply replaces the stochastic function values in the gradient estimators into stochastic function values of the compositional function, and use large batches to ensure the accuracy of the inner function value estimation. The convergence analysis is thus similar to GFM as well. 


Reference.

[31] Lesi Chen, Jing Xu, and Luo Luo. Faster gradient-free algorithms for nonsmooth nonconvex stochastic optimization. In International Conference on Machine Learning, pages 5219–5233. PMLR, 2023.

QUESTIONS:
Please see the weakness section.

LIMITATIONS:
No significant limitations in this work.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-13

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studied the zero-order method for computing an approximately stationary point for a Lipschitz function with a composition structure. The main difficulty lies in the function value evaluation. The composition structure involves multiple expectations, requiring multiple rounds of sampling to obtain a satisfactory estimation. The authors also discuss the situation when the objective function is convex. They show that the complexity can be improved under the convexity assumption.\n\nSTRENGTHS:\nOverall, I think this is an okay paper, in the sense that it makes a meaningful contribution to an important setting that was previously unexplored. The technique is intuitive, and the proof is very neat and clean, which is good and should be easy to follow.\n\nWEAKNESSES:\nMy impression is that the technique is okay but not that surprising. There are two expectations, so two sampling sequences are needed to evaluate the function value. I would say the technical contribution is a little bit insufficient, but I do not strongly oppose it solely on this point.\n\nThe following are minor points: \n* I would recommend specifying the distribution P in the main text, rather than in the statement of Lemma 3.7. It seems the definition of f_\\delta appears in L134, with a very general distribution P following. The definition of this P is not specified even in Algorithms 1 and 2. \n* L118: \\mathbb{B}(x, \\delta) should be \\mathbb{B}_\\delta(x), according to L93. \n* Theorem 4.1, Corollary 4.2, Theorem 4.3, etc.: The upper bound on gradient norm should be in the sense of expectation, right? It seems highly non-trivial to exactly select one R \\in [T] such that the gradient norm is minimal in the telescoping sum, due to the difficulty in evaluating the gradient of the smoothed function. \n* For Section 5, I recommend considering studying tighter stationarity notions for convex functions, e.g., those whose definition requires weak convexity. The notion of a Goldstein approximate stationary point is rather loose. Thus, whenever possible, considering tighter solution notions would be better.\n\nQUESTIONS:\nSee above.\n\nLIMITATIONS:\nSee above.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-15"}