Title: How Does Black-Box Impact the Learning Guarantee of Stochastic Compositional Optimization?
Venue: NeurIPS 2024
Accepted: yes
Keywords: Stochastic compositional optimization

theoretical guarantee; black-box optimization; stability analysis
Abstract: Stochastic compositional optimization (SCO) problem constitutes a class of optimization problems characterized by the objective function with a compositional form, including the tasks with known derivatives, such as AUC maximization, and the derivative-free tasks exemplified by black-box vertical federated learning (VFL). From the learning theory perspective, the learning guarantees of SCO algorithms with known derivatives have been studied in the literature. However, the potential impacts of the derivative-free setting on the learning guarantees of SCO remains unclear and merits further investigation. This paper aims to reveal the impacts by developing a theoretical analysis for two derivative-free algorithms, black-box SCGD and SCSC. Specifically, we first provide the sharper generalization upper bounds of convex SCGD and SCSC based on a new stability analysis framework more effective than prior work under some milder conditions, which is further developed to the non-convex case using the almost co-coercivity property of smooth function. Then, we derive the learning guarantees of three black-box variants of non-convex SCGD and SCSC with additional optimization analysis. Comparing these results, we theoretically uncover the impacts that a better gradient estimation brings a tighter learning guarantee and a larger proportion of unknown gradients may lead to a stronger dependence on the gradient estimation quality. Finally, our analysis is applied to two SCO algorithms, FOO-based vertical VFL and VFL-CZOFO, to build the first learning guarantees for VFL that align with the findings of SCGD and SCSC.

How Does Black-Box Impact the Learning Guarantee of Stochastic Compositional Optimization?

Abstract

Stochastic compositional optimization (SCO) problem constitutes a class of optimization problems characterized by the objective function with a compositional form, including the tasks with known derivatives, such as AUC maximization, and the derivative-free tasks exemplified by black-box vertical federated learning (VFL). From the learning theory perspective, the learning guarantees of SCO algorithms with known derivatives have been studied in the literature. However, the potential impacts of the derivative-free setting on the learning guarantees of SCO remains unclear and merits further investigation. This paper aims to reveal the impacts by developing a theoretical analysis for two derivative-free algorithms, black-box SCGD and SCSC. Specifically, we first provide the sharper generalization upper bounds of convex SCGD and SCSC based on a new stability analysis framework more effective than prior work under some milder conditions, which is further developed to the non-convex case using the almost co-coercivity property of smooth function. Then, we derive the learning guarantees of three black-box variants of non-convex SCGD and SCSC with additional optimization analysis. Comparing these results, we theoretically uncover the impacts that a better gradient estimation brings a tighter learning guarantee and a larger proportion of unknown gradients may lead to a stronger dependence on the gradient estimation quality. Finally, our analysis is applied to two SCO algorithms, FOO-based vertical VFL and VFL-CZOFO, to build the first learning guarantees for VFL that align with the findings of SCGD and SCSC.

1 Introduction

In recent years, stochastic compositional optimization (SCO), a class of optimization methods that incorporate the compositional form f ( g ( w )) , has garnered significant attention in the research

community [1, 2, 3, 4, 5, 6, 7]. It represents a special case of stochastic bilevel optimization [8]

where E z [ · ] represents the expectation with respect to (w.r.t.) the sample z , parameters w ∈ W ∈ R p and v ∈ R d . If the inner function h z ( w,v ) = ∥ v -g z ( w ) ∥ 2 and the outer function f is only a function of v , i.e., f ¯ z ( w,v ) = f ¯ z ( v ) , the stochastic bilevel optimization reduces to the SCO:

where F ( w ) is the compositional population risk, f : R d → R , g : R p → R d , f ( v ) = E ¯ z [ f ¯ z ( v )] and g ( w ) = E z [ g z ( w )] .

Many applications adhere to the form of SCO (2) such as risk averse optimization [9], group distributionally robust optimization [10], AUC maximization [11, 12, 13, 14], model-agnostic metalearning [15], and first-order-optimization-based vertical federated learning (FOO-based VFL) [16]. Apart from the above applications with available derivatives, there exist derivative-free scenarios as well, such as reinforcement learning [17] and zeroth-order-optimization-based (ZOO-based) VFL [18, 19]. [2] discussed the extension of stochastic compositional gradient descent (SCGD) to the derivative-free setting, called black-box SCGD, where the zeroth-order information of f or g is available through sampling.

From the perspective of statistical learning theory [20], the theoretical guarantees pertaining to generalization and optimization performance for SCO algorithms are worth studying to validate their empirical behaviors. The former assesses the disparity between the empirical performance and the population performance for the trained model. The latter measures the empirical performance gap between the trained model and the empirical optimal model. To our knowledge, there is only one study attempting to provide a generalization guarantee in this area. [21] has pioneered the generalization understanding of two notable SCO algorithms, i.e., SCGD and SCSC [5] via algorithmic stability tool. They have achieved satisfactory excess risk bounds by selecting some specific values of T to balance stability results and optimization errors. However, the intricacy of their analysis framework brings some unnecessary terms in their results leading to these so large T values that it is practically challenging to complete these iterations within a reasonable timeframe. Furthermore, the study has yet to consider a more practical scenario beyond convex and strongly convex cases, specifically, the non-convex case. For the optimization guarantee, plenty of work devotes to studying the convergence behaviors of some first-order SCO algorithms [2, 3, 4, 5] and their extensions [22, 23, 24, 25]. For example, [2] proved that SCGD can converge almost surely to an existing optimal solution with the rate O ( T -1 / 4 ) for non-smooth convex problems and the rate O ( T -2 / 7 ) for smooth convex problems, where T is the total number of iterations. [5] presented that stochastically corrected stochastic compositional gradient method (SCSC) can achieve the same convergence rate O ( T -1 / 2 ) as SGD for non-compositional problems. However, there exists a research gap in the optimization analysis for the derivative-free SCO algorithm as well as in its generalization analysis.

Considering these problems, this paper leverages algorithmic stability to obtain some similar and even superior results of SCGD and SCSC under the milder parameter selection and the non-convex condition. More importantly, to apply a broader class of stochastic optimization problems, this paper pioneers the theoretical analysis of the black-box SCO algorithms, which uncovers the impacts of black-box on the learning guarantees of SCO algorithms. Our main contributions are listed as follows.

Generalization guarantees under some milder settings. Firstly, we provide the sharper generalization upper bounds of convex SCGD and SCSC based on a new stability analysis framework more effective than prior work [21] with a more practical selection of T . Subsequently, we develop the above convex analysis to the non-convex case by introducing the almost co-coercivity property of smooth function, which yields satisfactory generalization guarantees of SCGD and SCSC under the non-convex condition.

Learning guarantees for black-box SCO algorithms. To apply a broader class of stochastic optimization problems, we further consider three black-box variants (outer, inner, and full black-box) of SCGD and SCSC to obtain the generalization and optimization upper bounds similar to the ones of SCGD and SCSC. Comparing the first-order and zeroth-order results, several key insights into the impacts of black-box on the learning guarantees of SCO algorithms are shown: 1) a closer estimation distance brings a better result; 2) more

estimation directions lead to a better result; 3) a larger proportion of unknown gradients results in a stronger dependence on the gradient estimation quality.

Applications on VFL. Finally, we explore the applications of our analysis framework to two specific SCO algorithms, i.e., FOO-based VFL and VFL-CZOFO, where we build the pioneering stability-based generalization and optimization guarantees for first-order and zeroth-order VFL algorithms that align with the findings of SCGD and SCSC.

2 Preliminaries

This section describes the learning paradigm of SCO and introduces two popular SCO algorithms (SCGD and SCSC) and their black-box variants in detail at first. Then, some necessary definitions and assumptions are provided for our theoretical analysis. The explanations for all symbols are shown in Table 3 located in Appendix A .

Considering a stochastic compositional optimization algorithm (2), the distributions of sample z and ¯ z are unknown. The training dataset S = S z ∪ S ¯ z = { z 1 , ..., z n } ∪ { ¯ z 1 , ..., ¯ z m } is available to obtain the final output model parameter A ( S ) via minimizing the following compositional empirical risk

where g S ( w ) = 1 n ∑ n i =1 g z i ( w ) , f S ( v ) = 1 m ∑ m j =1 f ¯ z j ( v ) , z 1 , ..., z n , ¯ z 1 , ..., ¯ z m are independent. Besides, we denote by w ( S ) , w ∗ the empirical optimal model parameter on S and the global optimal model parameter, defined as w ( S ) = arg min w ∈W F S ( w ) and w ∗ = arg min w ∈W F ( w ) . Then, the generalization error, optimization error and excess risk of A ( S ) are given by | F ( A ( S ) -F S ( A ( S )) | , F S ( A ( S )) -F S ( w ( S )) and F ( A ( S )) -F ( w ∗ ) , respectively. Since E [ F ( A ( S )) -F ( w ∗ )] ≥ 0 and E [ F S ( w ( S )) -F ( w ∗ )] ≤ 0 , the excess risk of A ( S ) can be decomposed as the summation of generalization error and optimization error as follows

where E [ · ] denotes the expectation w.r.t. all randomness.

In this work, we primarily investigate the learning guarantees of two prevalent SCO algorithms, i.e., SCGD [2] and SCSC [5], along with their black-box variants. Algorithm 1 presents the detailed parameter update procedures of these algorithms. The difference between SCGD and SCSC lies in the update of the outer model parameter v t . For SCGD, v t +1 is the linear combination of v t and g z i t ( w t ) . However, this update may lead to a suboptimal convergence rate when the learning rate β of the outer model update is smaller than the learning rate η t utilized for the inner model update. To alleviate this problem, SCSC updates v t +1 with the combination of the "corrected" v t and g z i t ( w t ) , where v t is corrected by g z i t ( w t ) -g z i t ( w t -1 ) so that v t +1 approximates g z i t ( w t ) [5]. Besides, [2] discussed the extension of SCGD to the derivative-free setting where only the zeroth-order information of g or f is available through sampling, which potentially applies to a broader class of stochastic optimization problems. Here, we show the first-order gradient estimation of f , which is similar to that of g . The unknown first-order gradient of f is estimated by Equation (4) and then approximated by Taylor expansion (5) in our analysis

where { u t,l } b l =1 is the set of independent and identically distributed (i.i.d.) random direction vectors (obeying the d -dimensional uniform distribution), and µ is the distance between two model parameters ( v t +1 + µu t,l and v t +1 ) used to estimate the gradient in the l -th direction.

Drawing inspiration from the classical non-compositional stability analysis work [26] and the pioneering work [21] investigating the generalization of SCO, we introduce the definition of uniform model stability as follows.

Algorithm 1 (Black-box) SCGD / SCSC

Definition 1. The randomized algorithm A for SCO problem is uniformly model ( ϵ z , ϵ ¯ z ) -stable if

where ∥ · ∥ is the Euclidean distance ∥ · ∥ 2 and S = { z 1 , ..., z n , ¯ z 1 , ..., ¯ z m } , S i,z = { z 1 , ..., z i -1 , z ′ i , z i +1 , ..., z n , ¯ z 1 , ..., ¯ z m } , S j, ¯ z = { z 1 , ..., z n , ¯ z 1 , ..., ¯ z j -1 , ¯ z ′ j , ¯ z j +1 , ..., ¯ z m } for any i ∈ [ n ] , j ∈ [ m ] .

According to the foundational concept of algorithmic stability, Definition 1 considers the two datasets obtained from the perturbation of a single sample in { z i } n i +1 and { ¯ z j } j m =1 respectively, where the altered sample z ′ i is i.i.d. to z i , and so does ¯ z ′ j . Prior to filling the relationship gap between the uniformly model stability and the generalization error E [ | F ( A ( S )) -F S ( A ( S )) | ] , it is essential to make some fundamental assumptions, i.e., Lipschitz continuity (bounded first-order gradient) of g, f and bounded variance of g .

Assumption 1. For any parameters w,w ′ ∈ W , v, v ′ ∈ R d and some L g , L f > 0 , functions g z ( w ) and f ¯ z ( v ) are Lipschitz continuous, i.e., ∥∇ g z ( w ) ∥ ≤ L g and ∥∇ f ¯ z ( v ) ∥ ≤ L f , which also mean that ∥ g z ( w ) -g z ( w ′ ) ∥ ≤ L g ∥ w -w ′ ∥ and | f ¯ z ( v ) -f ¯ z ( v ′ ) | ≤ L f ∥ v -v ′ ∥ .

In numerous compositional [2, 5, 6, 7] and non-compositional studies [26, 27], Assumption 1 serves as a general theoretical bridge analyzing the generalization and optimization performance.

Assumption 2. For any w ∈ W and some V g > 0 , the variance of function g z ( w ) is upper bounded by V g , i.e., E z [ ∥ g z ( w ) -g ( w ) ∥ 2 ] ≤ V g .

The bounded variance is also a classical condition for statistical learning theory [1, 2, 3, 5, 6, 7, 28, 29, 30] which limits the ranges of the variance value of the given functions g . Utilizing these two fundamental assumptions, Theorem 1 builds a rigorous relationship between stability and generalization error, thereby enabling stability to measure the generalization performance in the subsequent analysis. Note that, Theorem 1 was previously proved by [21] (Theorem 2.3), so we omit its detailed proof here for brevity.

Theorem 1. [21] Let Assumptions 1, 2 hold. Assume the randomized algorithms A for SCO problem is uniformly model ( ϵ z , ϵ ¯ z ) -stable, then,

Remark 1. As mentioned in [21], Theorem 1 is the compositional counterpart of Theorem 2.2 in [26]. In other words, the above result is equivalent to E [ | F ( A ( S )) -F S ( A ( S )) | ] ≤ L f ϵ ¯ z when g z ( w ) = w , i.e., F ( w ) = E ¯ z [ f ¯ z ( w )] and F S ( w ) = 1 m ∑ m j =1 f ¯ z j ( w ) . If the order of ϵ ¯ z is faster than O ( ϵ z + n -1 2 ) , the generalization upper bound of SCO algorithm will be primarily constrained by the term 4 L g L f ϵ z + L f √ n -1 V g attributed to the compositional structure. Otherwise, there is little difference between Theorem 1 and Theorem 2.2 [26].

The subsequent assumptions and definition are required by the stability analysis in Section 3.

Assumption 3. For any parameters w,w ′ ∈ W , v, v ′ ∈ R d and some α g , α f , α > 0 , functions g z ( w ) , f ¯ z ( v ) and f ¯ z ( g z ( w )) are smooth, i.e., ∥∇ 2 g z ( w ) ∥ ≤ α g , ∥∇ 2 f ¯ z ( v ) ∥ ≤ α f and ∥∇ 2 f ¯ z ( g z ( w )) ∥ ≤ α , which also mean that

and

Definition 2. For any parameter v, v ′ ∈ R d , a function f : R d → R is convex if f ( v ) ≥ f ( v ′ ) + ⟨∇ f ( v ′ ) , v -v ′ ⟩ .

Assumption 4. For any w ∈ W , v ∈ R d , direction vector u , step size µ > 0 and some M g , M f , M ′ g , M ′ f > 0 , the following inequalities hold

and

Remark 2. Assumption 3 is the most important condition for our analysis since there are several key properties (Lemma 4) of smoothness required to measure the algorithmic stability. In addition, our stability analysis framework relies crucially on another key lemma (called co-coercive lemma, Lemma 1) derived from the smoothness and convexity of the function f ( w ) . Therefore, we provide the definition of convexity in Definition 2. Except for the convex case (Section 3.1), this work mainly considers some non-convex cases (Sections 3.1, 3.2). Although the co-coercive lemma is not available without the convexity condition, a surrogate lemma (called almost co-coercive lemma, Lemma 3) takes a similar role within our analysis framework. Finally, Assumption 4 gives the upper bounds of the difference between two adjacent function values for g, f, ∇ g, ∇ f , which represents a less stringent assumption compared to the general bounded condition [26]. Specifically, Assumption 4 is different from the assumption | f | ≤ M . Assumption 4 requires the distance between two adjacent function outputs to be bounded, i.e., || g ( w + µu ) -g ( w ) || ≤ M g , | f ( v + µu ) -f ( v ) | ≤ M f , which is milder than | f | ≤ M . Besides, it also requires the distance between two adjacent gradient outputs to be bounded, i.e., ||∇ g ( w + µu ) - ∇ g ( w ) || ≤ M ′ g , ||∇ f ( v + µu ) - ∇ f ( v ) || ≤ M ′ f , which is milder than bounded gradient condition ||∇ f || ≤ L [26].

3 Main Results

This section presents the learning guarantees of two SCO algorithms (SCGD and SCSC) under several cases. The comparisons among our results and previous work are summarized in Tables 1, 2, and their proofs are provided in Appendices C, D .

3.1 Learning Guarantees for General SCO

Firstly, we consider the generalization analysis for the general convex SCO algorithm.

Theorem 2. Let Assumptions 1, 3 hold and the function f ( g ( w )) is convex. Assume that the randomized algorithms A (Algorithm 1) for SCO problem brings the model sequences { w t } T t =1 and { w i,z t } T ({ w j, ¯ z t } T ) on S and S i,z ( S j, ¯ z ) with the step size sequence { η t } T t =1 .

(a) For SCGD with η t ≤ 2 β αt , the final output A ( S ) = w T is uniformly model ( ϵ z , ϵ ¯ z ) -stable with

(b) For SCSC with η t ≤ 2 αt , the final output A ( S ) = w T is uniformly model ( ϵ z , ϵ ¯ z ) -stable with

Remark 3. Based on a new stability analysis framework more effective than prior work [21], Theorem 2 states the stability upper bounds O (( n -1 + m -1 ) β log T ) for SCGD and O (( n -1 + m -1 ) log T ) for SCSC under the convex condition, which derives a generalization bound O ( ( n -1 + m -1 ) log T + n -1 2 ) by combining with Theorem 1. Previously, [21] provided the stability results O ( ηT ( n -1 + m -1 ) + η ( T 1 2 + β -c 2 T -c 2 +1 + β 1 2 T ) + η 2 β -1 T ) for SCGD and O ( ηT ( n -1 + m -1 ) + η ( T 1 2 + β -c 2 T -c 2 +1 + β 1 2 T ) + η 2 β -1 2 T ) for SCSC in the same setting, where c > 0 is an arbitrary constant. They selected η = O ( T -6 7 ) , β = O ( T -4 7 ) , c > 2 , T = O ( max { n 7 2 , m 7 2 }) for SCGD and η = O ( T -4 5 ) , β = O ( T -4 5 ) , c > 4 , T = O ( max { n 5 2 , m 5 2 }) for SCSC to yield the bounds O ( max { n -1 , m -1 } · max { n 1 2 , m 1 2 }) which is slight larger than O ( n -1 2 + m -1 2 ) . Compared with [21], Theorem 2 enjoys not only tighter bounds but also some more practical parameter selections of η, β, T . There are some experiments [2, 5, 34, 35] to validate this statement. (1) For T : [21] provided some generalization bounds for convex SCGD and SCSC with some impractical T such as T = O (max( n 7 / 2 , m 7 / 2 )) in Theorem 4. While our convex result (Theorem 2) can achieve similar rates even taking T = O (max( n, m )) which better matches some empirical observations (Fig. 1, 2 in [5] and Fig. 2 in [2]). (2) For η t : Theorem 4 in [21] took η t = T -6 / 7 which is too small when T is large. While Theorem 2 takes η t = O ( t -1 ) closer to some empirical selections ( η t = O ( t -3 / 4 ) in [2, 5] and η t = O ( t -1 ) in [34, 35]). (3) For β t : Theorem 4 in [21] took β t = T -4 / 7 which is also too small since [2, 5, 35] empirically select β t = t -1 / 2 or β t = t -1 . In contrast, Theorem 2 have no special restriction on β t .

Moreover, the bounds of Theorem 2 are similar to some popular stability bounds in the noncompositional literature. For example, the most classical work [26] achieved the uniform stability bound O ( n -1 log T ) for convex SGD with η t ≤ O ( t -1 ) . [31] showed the uniform stability bound O ( n -1 T 1 2 + n -1 2 √ log(1 /δ ) ) for convex pairwise SGD with η t = O ( T -1 2 ) . The proof of Theorem 2 is provided in Appendix C .

To further weaken our assumptions, the generalization analysis of the convex SCO algorithm is developed into the non-convex setting by introducing the almost co-coercivity property of smooth function.

Theorem 3. Let Assumptions 1, 3 hold. Assume that the randomized algorithms A (Algorithm 1) for SCO problem brings the model sequences { w t } T t =1 and { w i,z t } T t =1 ({ w j, ¯ z t } T t =1 ) on S and S i,z ( S j, ¯ z ) with the step size sequence { η t } T t =1 . For SCGD with η t ≤ 1 2 ρt , ρ = α g L f + βL 2 g α f and SCSC with η t ≤ 1 2 ρt , ρ = α g L f + L 2 g α f , the final output A ( S ) = w T is uniformly model ( ϵ z , ϵ ¯ z ) -stable with

Remark 4. Under the non-convex setting, Theorem 3 elucidates a satisfactory stability bound O ( ( n -1 + m -1 ) T 1 2 log T ) but T 1 2 -times larger than Theorem 2. When T = O (max { n, m } ) and ignoring logarithmic terms, this bound is equivalent to O ( max { n -1 , m -1 } · max { n 1 2 , m 1 2 } ) . Therefore, under the further weakening condition, i.e., non-convexity, Theorem 3 achieves the stability results similar to the ones of the convex SCGD and SCSC in [21] and some non-compositional, non-convex results [26, 30, 32, 33]. The proof of Theorem 3 is provided in Appendix C .

3.2 Learning Guarantees for Black-box SCO

The aforementioned results lay the groundwork for elucidating the impacts of black-box on the learning guarantees for the non-convex SCGD and SCSC, including additional optimization analysis. Three black-box cases shown in Algorithm 1 are considered in this part. Prior to the analysis, we provide the following assumption required by the optimization analysis.

Assumption 5. For any w ∈ W and parameter γ > 0 , the empirical risk F S ( w ) satisfies E [ ∥∇ F S ( w ) ∥ 2 ] ≥ 2 γ E [ F S ( w ) -F S ( w ( S ))] .

In the absence of the convexity assumption, the gradient of empirical risk ∥∇ F S ( w ) ∥ = 0 does not guarantee a global optimal parameter. Assumption 5 postulates that all empirical local optimal parameters are, in fact, empirical global optimal parameters, which prepares for the characterization of E [ F S ( A ( S )) -F S ( w ( S ))] .

Theorem 4. Let Assumptions 1, 2, 3, 4, 5 hold. Assume that the randomized algorithms A (Algorithm 1) for SCO problem brings the model sequences { w t } T t =1 and { w i,z t } T t =1 ({ w j, ¯ z t } T t =1 ) on S and S i,z ( S j, ¯ z ) with the step size sequence { η t } T t =1 . For the outer black-box SCGD with

η t = 1 pγt , p ≥ max { √ 2 α γ , 2 ( α g M f + βL 2 g M ′ f ) µγ } and the outer black-box SCSC with η t = 1 pγt , p ≥ max { √ 2 α γ , 2 ( α g M f + L 2 g M ′ f ) µγ } , the final output A ( S ) = w T has the learning guarantee

where

d

2

=

d

-

(2

p

+1)

β

+

p

+

1

2

β

for SCGD and

The proof of Theorem 4 is provided in Appendix D .

Remark 5. Theorem 4 considers the outer balck-box SCGD and SCSC algorithms where only the gradient of the outer function f is unknown. It establishes the excess risk bound O ( ( n -1 + m -1 ) T 1 2 log T + n -1 2 + µ 2 + b -1 d 2 ) . [21] derived the excess risk bound O ( max { n -1 , m -1 } · max { n 1 2 , m 1 2 }) for the convex SCGD and SCSC with the parameter selections in Remark 3. Theorem 4 can derive this bound with the milder condition T = O (max { n, m } ) and η t = O ( t -1 ) .

For the generalization bound, Theorem 4 is consistent with Theorem 3. As for the optimization bound, [2] proved the convergence rate E [ ∥∇ F ( w T ) ∥ 2 ] ≤ O ( T -1 4 ) of SCGD for non-convex problems.

[5] proved the convergence rate T -1 ∑ T -1 t =0 E [ ∥∇ F ( w t ) ∥ 2 ] ≤ O ( T -1 2 ) of SCSC for non-convex problems. Compared with [2, 5], Theorem 4 obtains the black-box-related bound O ( µ 2 + b -1 d 2 ) with some smaller learning rates required by our analytical framework. This bound is composed of two dependencies on the estimation distance µ and the number b of estimation directions in Equation (4) , which shows the following two key insights into the impacts of black-box on the learning guarantees of SCO algorithms. Firstly, a small µ indicates a small distance between the two function values f ¯ z ( v + µu ) and f ¯ z ( v ) selected to make gradient estimation ˜ ∇ f ¯ z ( v ) in the direction of the unit vector u . Therefore, a smaller µ , i.e., a closer estimation distance, brings a better gradient estimation, resulting in a better excess risk bound. Secondly, a large b indicates that plenty of unit vectors u l , l = 1 , ..., b with different directions are selected to make gradient estimation ˜ ∇ f ¯ z ( v ) . Then, a larger b , i.e., more estimation directions, leads to a better gradient estimation, resulting in a better excess risk bound. In summary, the bound of Theorem 4 verifies the fact that a better gradient estimation brings a tighter learning guarantee for the outer black-box SCGD and SCSC.

Except for the black-box-related term, Theorem 4 chooses the learning rates affected by µ , which is also different from Theorem 4. Although µ can be very small such as 10 -4 [19], its negative impact on η t can be eliminated by M f , M ′ f since the two inequalities M f /µ ≤ L f and M ′ f /µ ≤ α f hold.

The inner black-box and the full black-box SCO algorithms are studied in the following two corollaries, respectively.

Corollary 1. Let the assumptions of Theorem 4 hold. For the inner black-box SCGD with η t = 1 pγt , p ≥ max { √ 2 α γ , 2 ( βL g α f M g + M ′ g L f ) µγ } and the inner black-box SCSC with η t = 1 pγt , p ≥ max { √ 2 α γ , 2 ( L g α f M g + M ′ g L f ) µγ } , the final output A ( S ) = w T has the learning guarantee

where d 2 = d -(2 p +1) β + ( p + 1 2 ) 2 β 2 for SCGD and d 2 = d -2 p -1 + ( p + 1 2 ) 2 for SCSC. Corollary 2. Let the assumptions of Theorem 4 hold. For the full black-box SCGD with η t = 1 pγt , p ≥ max { √ 2 α γ , 2 ( βL g M ′ f M g + M f M ′ g ) µ 2 γ } and the full black-box SCSC with η t = 1 pγt , p ≥ max { √ 2 α γ , 2 ( L g M ′ f M g + M f M ′ g ) µ 2 γ } , the final output A ( S ) = w T has the learning guarantee

(

)

2

2

d

=

d

-

2

p

-

1 +

(

p

+

1

2

)

2

for SCSC.

2

The proofs of Corollaries 1, 2 are provided in Appendix D .

Remark 6. Corollary 1 provides the excess risk bound with the same order as Theorem 4 for the inner black-box SCGD and SCSC. The excess risk bound for the full black-box SCGD and SCSC in Corollary 2 presents the different dependencies on µ and b , i.e., µ 4 + b -2 d 2 2 + b -1 d 2 . These dependencies also comply with the two key insights uncovered by Theorem 4 and Corollary 1. Besides, when µ 4 b > d 2 or µ 2 b < d 2 holds, the term is dominated by µ 4 or b -2 d 2 2 which denotes a stronger dependence on the gradient estimation quality. Hence, Corollary 2 shows that a larger proportion of unknown gradients may lead to a stronger dependence on the gradient estimation quality.

4 Applications

Considering the existing derivative-free cases in VFL, the analysis framework of SCGD and SCSC is herein applied to two VFL algorithms, FOO-based VFL [16] and VFL-CZOFO [19]. As outlined in Algorithm 2 of Appendix A , FOO-based VFL algorithm comprises two components, the K local clients with the model parameters w k , k ∈ [ K ] and the central server with the global model v . Concerning the data privacy, different clients do not communicate with each other directly, but exchange information indirectly through a server. For this reason, the objective function of the k -th client adopts the same compositional structure f ( g ( w k )) as SCGD and SCSC. To further ensure data privacy without additional protection techniques, VFL-CZOFO is proposed by introducing the idea of ZOO. Different from the general ZOO-based VFL [18], VFL-CZOFO employs a zeroth-order gradient on the output layer of every client, with other parts utilizing the first-order gradient, which preserves the privacy protection of ZOO while significantly enhancing convergence.

Before stating our remaining results, it should be noted that there are a few differences between the setting of FOO-based VFL (VFL-CZOFO) and the one of SCGD (SCSC). First of all, we set S = { z 1 , ..., z n } and S i,z = { z 1 , ..., z i -1 , z ′ i , z i +1 , ..., z n } according to the learning paradigm of VFL. Therefore, Theorem 1 is simplified as Corollary 3. Secondly, the update of the outer model (global model) for FOO-based VFL (VFL-CZOFO) is not based on the simple moving average in SCGD (SCSC). Thirdly, Assumptions 1, 3, 4, 5 hold for every client in all K clients. Without loss of generality, we only study the learning guarantees of FOO-based VFL and VFL-CZOFO for the k -th client.

Corollary 3. Let Assumption 1 hold. Assume the randomized VFL algorithms A is uniformly model ϵ z -stable, then, E [ | F ( A ( S )) -F S ( A ( S )) | ] ≤ L g L f ϵ z .

Corollary 3 gives the relationship between uniform model stability and generalization error under the setting of VFL. The proof of Corollary 3 is omitted since it can be proved by Equation (15) in the proof of Theorem 2.3 [21] without the decomposition in Equation (14). The last two results study the theoretical performance of FOO-based VFL and VFL-CZOFO.

Corollary 4. Let Assumptions 1, 3, 4, 5 hold. For the k -th client ( k ∈ [ K ] ), assume that the randomized FOO-based VFL algorithm (Algorithm 2) brings the model sequences { w k t } T t =1 and { w i,z,k t } T t =1 on S and S i,z with the step size sequence { η t } T t =1 , η t ≤ 1 2 ρt , ρ = α g L f + L 2 g α f . Then, the final output A ( S ) = w k of the k -th client has the generalization guarantee

T

Corollary 5. Let Assumptions 1, 3, 4, 5 hold. For the k -th client ( k ∈ [ K ]) , assume that the randomized VFL-CZOFO algorithm (Algorithm 2) brings the model sequences { w k t } T t =1 and { w i,z,k t } T t =1 on S and S i,z with the step size sequence { η t } T t =1 , η t = 1 pγt , p ≥ max { √ 2 α γ , 2 ( α g M f + L 2 g M ′ f ) µγ } .

Then, the final output A ( S ) = w k T of the k -th client has the generalization guarantee

where d 2 = d -2 p -1 + ( p + 1 2 ) 2 .

The proofs of Corollaries 4, 5 are provided in Appendix E .

Remark 7. Corollaries 4, 5 both provide the first stability-based generalization bound O ( n -1 T 1 2 log T ) . As for the optimization bound, Corollary 5 gives O ( µ 2 + b -1 d 2 ) , which originates from the outer black-box setting.

Apart from VFL, the generalization guarantee of zeroth-order horizontal federated learning (HFL) was studied with the algorithmic stability tool. [36] established the systematic theoretical assessments of synchronous federated zeroth-order optimization (FedZO) by developing the on-average model stability analysis. Its generalization bounds and optimization bounds all depend on the two gradient estimation-based parameters µ and b . From our perspective, the complicated compositional structure may be a key factor that makes the generalization bound in Corollary 5 unaffected by the impact of the estimated gradient quality. The reason is that the analysis framework of Theorem 2 in [36] requires a decomposition (Equation (6)) which is hardly achieved due to the compositional structure in our analysis.

5 Conclusions

In this paper, we provide a novel, more effective theoretical analysis of two SCO algorithms, SCGD and SCSC, and their black-box variants utilizing the uniform model stability tool. The analysis framework is applied to the two VFL algorithms, FOO-based VFL and VFL-CZOFO. Our results not only offer satisfactory learning guarantees but also theoretically validate the impacts of black-box that a better gradient estimation brings a tighter learning guarantee and a larger proportion of unknown gradients leads to a stronger dependence on the gradient estimation quality. We hope our study can facilitate future theoretical analyses of SCO problems and inspire new practical algorithms.

Acknowledgments

This work was supported in part by the National Natural Science Foundation of China (Nos. 12071166 and 62376104) and the Fundamental Research Funds for the Central Universities of China (No. 2662023LXPY005).

References

Saeed Ghadimi, Guanghui Lan, and Hongchao Zhang. Mini-batch stochastic approximation methods for nonconvex stochastic composite optimization. Mathematical Programming , 155(12):267-305, 2016.

Mengdi Wang, Ethan X. Fang, and Han Liu. Stochastic compositional gradient descent: algorithms for minimizing compositions of expected-value functions. Mathematical Programming , 161(1-2):419-449, 2017.

Mengdi Wang, Ji Liu, and Ethan X. Fang. Accelerating stochastic composition optimization. Journal of Machine Learning Research , 18:105:1-105:23, 2017.

Zhe Zhang and Guanghui Lan. Optimal algorithms for convex nested stochastic composite optimization, 2020.

Tianyi Chen, Yuejiao Sun, and Wotao Yin. Solving stochastic compositional optimization is nearly as easy as solving stochastic optimization. IEEE Transactions on Signal Processing , 69:4937-4948, 2021.

Wei Jiang, Bokun Wang, Yibo Wang, Lijun Zhang, and Tianbao Yang. Optimal algorithms for stochastic multi-level compositional optimization. In International Conference on Machine Learning (ICML) , volume 162, pages 10195-10216, 2022.

Bokun Wang and Tianbao Yang. Finite-sum coupled compositional stochastic optimization: Theory and applications. In International Conference on Machine Learning (ICML) , volume 162, pages 23292-23317, 2022.

Tianyi Chen, Yuejiao Sun, and Wotao Yin. Closing the gap: Tighter analysis of alternating stochastic gradient methods for bilevel problems. In Advances in Neural Information Processing Systems (NeurIPS) , pages 25294-25307, 2021.

Andrzej Ruszczynski and Alexander Shapiro. Optimization of risk measures. Mathematics of Operations Research , 31:433-452, 2006.

Qi Qi, Zhishuai Guo, Yi Xu, Rong Jin, and Tianbao Yang. An online method for a class of distributionally robust optimization with non-convex objectives. In Advances in Neural Information Processing Systems (NeurIPS) , pages 10067-10080, 2021.

Purushottam Kar, Bharath K. Sriperumbudur, Prateek Jain, and Harish Karnick. On the generalization ability of online learning algorithms for pairwise loss functions. In International Conference on Machine Learning (ICML) , volume 28, pages 441-449, 2013.

Yiming Ying, Longyin Wen, and Siwei Lyu. Stochastic online AUC maximization. In Advances in Neural Information Processing Systems (NIPS) , pages 451-459, 2016.

Yunwen Lei and Yiming Ying. Stochastic proximal AUC maximization. Journal of Machine Learning Research , 22:61:1-61:45, 2021.

Tianbao Yang and Yiming Ying. AUC maximization in the era of big data and AI: A survey. ACM Computing Surveys , 55(8):172:1-172:37, 2023.

Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In International Conference on Machine Learning (ICML) , volume 70, pages 1126-1135, 2017.

Tianyi Chen, Xiao Jin, Yuejiao Sun, and Wotao Yin. VAFL: a method of vertical asynchronous federated learning, 2020.

Christoph Dann, Gerhard Neumann, and Jan Peters. Policy evaluation with temporal differences: A survey and comparison. Journal of Machine Learning Research , 15(1):809-883, 2014.

Qingsong Zhang, Bin Gu, Zhiyuan Dang, Cheng Deng, and Heng Huang. Desirable companion for vertical federated learning: New zeroth-order gradient based algorithm. In ACM International Conference on Information and Knowledge Management (CIKM) , pages 2598-2607, 2021.

Ganyu Wang, Bin Gu, Qingsong Zhang, Xiang Li, Boyu Wang, and Charles X. Ling. A unified solution for privacy and communication efficiency in vertical federated learning. In Advances in Neural Information Processing Systems (NeurIPS) , 2023.

V. N. Vapnik. Statistical learning theory. Encyclopedia of the Sciences of Learning , 41(4):31853185, 1998.

Ming Yang, Xiyuan Wei, Tianbao Yang, and Yiming Ying. Stability and generalization of stochastic compositional gradient descent algorithms, 2023.

Huizhuo Yuan, Xiangru Lian, Chris Junchi Li, Ji Liu, and Wenqing Hu. Efficient smooth non-convex stochastic compositional optimization via stochastic recursive gradient descent. In Advances in Neural Information Processing Systems (NeurIPS) , pages 6926-6935, 2019.

Adithya M. Devraj and Jianshu Chen. Stochastic variance reduced primal dual algorithms for empirical composition optimization. In Advances in Neural Information Processing Systems (NeurIPS) , pages 9878-9888, 2019.

Saeed Ghadimi, Andrzej Ruszczynski, and Mengdi Wang. A single timescale stochastic approximation method for nested stochastic optimization. SIAM Journal on Optimization , 30(1):960-979, 2020.

Andrzej Ruszczynski. A stochastic subgradient method for nonsmooth nonconvex multilevel composition optimization. SIAM Journal on Control and Optimization , 59(3):2301-2320, 2021.

Moritz Hardt, Ben Recht, and Yoram Singer. Train faster, generalize better: Stability of stochastic gradient descent. In International Conference on Machine Learning (ICML) , volume 48, pages 1225-1234, 2016.

Olivier Bousquet and André Elisseeff. Stability and generalization. Journal of Machine Learning Research , 2:499-526, 2002.

Arkadi Nemirovski, Anatoli B. Juditsky, Guanghui Lan, and Alexander Shapiro. Robust stochastic approximation approach to stochastic programming. SIAM Journal on Optimization , 19(4):1574-1609, 2009.

Léon Bottou. Large-scale machine learning with stochastic gradient descent. In International Conference on Computational Statistics (COMPSTAT) , pages 177-186, 2010.

Yi Zhou, Yingbin Liang, and Huishuai Zhang. Understanding generalization error of SGD in nonconvex optimization. Machine Learning , 111(1):345-375, 2022.

Yunwen Lei, Antoine Ledent, and Marius Kloft. Sharper generalization bounds for pairwise learning. In Advances in Neural Information Processing Systems (NeurIPS) , pages 2123621246, 2020.

Yunwen Lei, Mingrui Liu, and Yiming Ying. Generalization guarantee of SGD for pairwise learning. In Advances in Neural Information Processing Systems (NeurIPS) , pages 2121621228, 2021.

Yunwen Lei. Stability and generalization of stochastic optimization with nonconvex and nonsmooth problems. In Conference on Learning Theory (COLT) , pages 191-227, 2023.

Zhouyuan Huo, Bin Gu, Ji Liu, and Heng Huang. Accelerated method for stochastic composition optimization with nonsmooth regularization. In Thirty-Second Conference on Artificial Intelligence (AAAI) , pages 3287-3294, 2018.

Junyu Zhang and Lin Xiao. A stochastic composite gradient method with incremental variance reduction. In Advances in Neural Information Processing Systems (NeurIPS) , pages 9075-9085, 2019.

Jun Chen, Hong Chen, Bin Gu, and Hao Deng. Fine-grained theoretical analysis of federated zeroth-order optimization. In Advances in Neural Information Processing Systems (NeurIPS) , 2023.

Dominic Richards and Ilja Kuzborskij. Stability & generalisation of gradient descent for shallow neural networks without the neural tangent kernel. In Advances in Neural Information Processing Systems (NeurIPS) , pages 8609-8621, 2021.

Yunwen Lei, Rong Jin, and Yiming Ying. Stability and generalization analysis of gradient methods for shallow neural networks. In Advances in Neural Information Processing Systems (NeurIPS) , 2022.

John C. Duchi, Michael I. Jordan, Martin J. Wainwright, and Andre Wibisono. Optimal rates for zero-order convex optimization: The power of two function evaluations. IEEE Transactions on Information Theory , 61(5):2788-2806, 2015.

A Notations

The main notations of this paper are summarized in Table 3.

The pseudo code of FOO-based VFL and VFL-CZOFO is present in Algorithm 2.

B Lemmas

Lemma 1. Assume the function f is convex and α -smooth. Then, for any w,w ′ , we have

Lemma 2. Let e be the base of the natural logarithm. The following inequalities hold:

Algorithm 2 FOO-based VFL / VFL-CZOFO

Require: v 1 , w k 1 : initial global model and K local models; η 0 , η 1 : initial learning rates for all t = 1 , ..., T -1 do for all k ∈ [ K ] in parallel do

FOO-based VFL: Receive ∇ f ( g z ( w k

end for

Server receives g z i t ( w k t ) from K clients

FOO-based VFL: Obtain and send ∇ f ( g z i t ( w k t )) to the k -th client VFL-CZOFO: Compute ˜ ∇ f ( g z i t ( w k t )) and send it to the k -th client Obtain ∇ f ( v t ) and update v t +1 = v t -η 0 ∇ f ( v t )

end for

Ensure: K final client models w 1 T , ..., w K T

Lemma 3. [37, 38] Consider the gradient-based optimization method w t +1 = w t -η t ∇ ˆ f ( w t ) . For two iteration sequences { w t } t ∈ [ T ] and { w ′ t } t ∈ [ T ] , if the function ˆ f ( w t ) is ρ -smooth, η t ≤ 1 / (2 ρ ) , and the minimum eigenvalue λ min ( ∇ 2 ˆ f ( w t ) ) ≥ -ϵ , then

Lemma 4. If the function f is α -smooth, then, for any w,w ′ , we have

Lemma 5. [39] Assume a random vector X ∈ R d is d -dimensional uniform distribution. For any k ∈ N , there holds E [ ∥ X ∥ k ] = d/ ( d + k ) .

Lemma 6. [39] Let u l ∈ R d , l ∈ { 1 , 2 , ..., b } be i.i.d. random vectors satisfying d -dimensional uniform distribution. For every random vector v ∈ R d independent of all u l and β ∈ (0 , 1) , the following inequality holds

and

C Proofs for General SCO

Proof of Theorem 2 :

(a) SCGD: As Definition 1, we define S = { z 1 , ..., z n , ¯ z 1 , ..., ¯ z m } , S i,z = { z 1 , ..., z i -1 , z ′ i , z i +1 , ..., z n , ¯ z 1 , ..., ¯ z m } and S j, ¯ z = { z 1 , ..., z n , ¯ z 1 , ..., ¯ z j -1 , ¯ z ′ j , ¯ z j +1 , ..., ¯ z m } for any i ∈ [ n ] , j ∈ [ m ] . The two terms E A [∥ ∥ ∥ w T -w i,z T ∥ ∥ ∥ ] and E A [∥ ∥ ∥ w T -w j, ¯ z T ∥ ∥ ∥ ] will be estimated as follows.

̸

1) E A [∥ ∥ ∥ w T -w i,z T ∥ ∥ ∥ ] : There are two cases that need to be considered. Firstly, when i t = i , there holds

Due to two properties of the function f ( g ( z )) , i.e., convexity and smoothness, Lemma 1 implies that

where β ∇ f ¯ z j t ( v t +1 ) = ∇ f ¯ z j t ( g z i t ( w t )) based on the update of SCGD. Then,

where the second inequality is caused by η t ≤ 2 β αt ≤ 2 β α . That is ∥ ∥ ∥ w t +1 -w i,z t +1 ∥ ∥ ∥ ≤ ∥ ∥ ∥ w t -w i,z t ∥ ∥ ∥ . Secondly, when i t = i , there holds

Combining the above two cases, we can get that

̸

Taking expectation over i t ,

̸

Then, taking expectation over A and taking summation from t = 1 to T -1 to get that

where the last inequality is from Lemma 2 (b).

̸

Due to two properties of the function f ( g ( z )) , i.e., convexity and smoothness, Lemma 1 implies that

where β ∇ f ¯ z j t ( v t +1 ) = ∇ f ¯ z j t ( g z i t ( w t )) based on the update of SCGD. Then,

f

η

t

where the second inequality is caused by η t ≤ 2 β αt ≤ 2 β α . That is ∥ ∥ ∥ w t +1 -w j, ¯ z t +1 ∥ ∥ ∥ ≤ ∥ ∥ ∥ w t -w j, ¯ z t ∥ ∥ ∥ . Secondly, when j t = j , there holds

Combining the above two cases, we can get that

̸

Taking expectation over j t ,

̸

Then, taking expectation over A and taking summation from t = 1 to T -1 to get that

(b) SCSC: Similar to the stability proof of SCGD except for the equation ∇ f ¯ z j t ( v t +1 ) = ∇ f ¯ z j t ( g z i t ( w t )) based on the update of SCSC, we have that, for η t = η 1 t ≤ 2 αt ≤ 2 α ,

̸

and

Proof of Theorem 3 :

Without the convexity of the function f ( g ( z )) , Lemma 1 can not hold. An almost co-coercivity of gradient operator (Lemma 3) is introduced to build the relationship between the inner product term 〈 w t -w i,z t , ∇ g z i t ( w t ) ∇ f ¯ z j t ( v t +1 ) -∇ g z i t ( w i,z t ) ∇ f ¯ z j t ( v i,z t +1 ) 〉 and the two squared terms ∥ ∥ ∥ w t -w i,z t ∥ ∥ ∥ 2 , ∥ ∥ ∥ ∇ g z i t ( w t ) ∇ f ¯ z j t ( v t +1 ) -∇ g z i t ( w i,z t ) ∇ f ¯ z j t ( v i,z t +1 ) ∥ ∥ ∥ 2 . With Assumption 3, the terms ∇ g ( w t ) and ∇ f ( v t +1 ) are both differentiable. Thus, ∇ g ( w t ) ∇ f ( v t +1 ) is also differentiable, which means that it is continuous on its domain. As we all know, a continuous function has primitive functions. Then, it is reasonable to assume that there exists a primitive function ˆ f ( w t ) at least whose derivative function ∇ ˆ f ( w t ) = ∇ g ( w t ) ∇ f ( v t +1 ) . For example, 1 β f ( v t +1 ) is one of the primitive functions ˆ f ( w t ) . Then

Taking derivative of ∇ ˆ f z i t , ¯ z j t ( w t ) over w t , we get that

Thus,

Let ρ = α g L f + βL 2 g α f , then ˆ f z i t , ¯ z j t ( w t ) is ρ -smooth. Since ∥ ∥ ∥ ∇ 2 ˆ f z i t , ¯ z j t ( w t ) ∥ ∥ ∥ equals to the largest singular value of ∇ 2 ˆ f z i t , ¯ z j t ( w t ) , we can know that λ min ( ∇ 2 ˆ f z i t , ¯ z j t ( w t ) ) ≥ -∥ ∥ ∥ ∇ 2 ˆ f z i t , ¯ z j t ( w t ) ∥ ∥ ∥ ≥ -ρ . According to Lemma 3, we can get that

Now, plugging the above inequality back into Equation (9) yields

where the second inequality is due to η t ≤ 1 2 ρt ≤ 3 2 ρ . The above inequality implies

Secondly, when i t = i , there holds

Combining the above two cases, we can get that

Taking expectation over i t

̸

Then, taking expectation over A and taking summation from t = 1 to T -1 to get that

where the fourth inequality is from e x ≥ 1 + x .

̸

We assume that there exists a primitive function ˆ f ( w t ) at least whose derivative function ∇ ˆ f ( w t ) = ∇ g ( w t ) ∇ f ( v t +1 ) . Then

̸

Taking derivative of ∇ ˆ f z i t , ¯ z j t ( w t ) over w t , we get that

Thus,

Let ρ = α g L f + βL 2 g α f , then ˆ f z i t , ¯ z j t ( w t ) is ρ -smooth. And we can know that λ min ( ∇ 2 ˆ f z i t , ¯ z j t ( w t ) ) ≥ -∥ ∥ ∥ ∇ 2 ˆ f z i t , ¯ z j t ( w t ) ∥ ∥ ∥ ≥ -ρ . According to Lemma 3, we can get that

Now, plugging the above inequality back into Equation (11) yields

where the second inequality is due to η t ≤ 1 2 ρt ≤ 3 2 ρ . The above inequality implies

Secondly, when j t = j , there holds

Combining the above two cases, we can get that

Taking expectation over j t ,

̸

̸

Then, taking expectation over A and taking summation from t = 1 to T -1 to get that

where the fourth inequality is from e x ≥ 1 + x .

SCSC: Similar to the stability proof of SCGD except for ∇ 2 ˆ f z i t , ¯ z j t ( w t ) = ∇ 2 g z i t ( w t ) ∇ f ¯ z j t ( v t +1 )+ ∇ g z i t ( w t ) ∇ 2 f ¯ z j t ( v t +1 ) ∇ g ⊤ z i t ( w t ) based on the update of SCSC, we have that, for η t ≤ 1 2 ρt ≤ 3 2 ρ , ρ = α g L f + L 2 g α f ,

□

and

D Proofs for Black-box SCO

Proof of Theorem 4 :

̸

With Assumption 3, the terms ∇ g ( w t ) and f ( v t +1 ) are both differentiable. Thus, ∇ g ( w t ) 1 b ∑ b l =1 u t,l µ ( f ( v t +1 + µu t,l ) -f ( v t +1 )) is also differentiable. It is reasonable to assume that there exists a primitive function ˆ f ( w t ) at least whose derivative function ∇ ˆ f ( w t ) = ∇ g ( w t ) 1 b ∑ b l =1 u t,l µ ( f ( v t +1 + µu t,l ) -f ( v t +1 )) . Then

Taking derivative of ∇ ˆ f z i t , ¯ z j t ( w t ) over w t , we get that

Thus,

Now, plugging the above inequality back into Equation (15) yields

where the second inequality is due to η t ≤ 1 2 ρt ≤ 3 2 ρ . The above inequality implies

Secondly, when i t = i , there holds

Combining the above two cases, we can get that

Taking expectation over i t ,

̸

Then, taking expectation over A and taking summation from t = 1 to T -1 to get that

̸

where the fourth inequality is from e x ≥ 1 + x .

̸

With Assumption 3, the terms ∇ g ( w t ) and f ( v t +1 ) are both differentiable. Thus, ∇ g ( w t ) 1 b ∑ b l =1 u t,l µ ( f ( v t +1 + µu t,l ) -f ( v t +1 )) is also differentiable. It is reasonable to assume that there exists a primitive function ˆ f ( w t ) at least whose derivative function ∇ ˆ f ( w t ) = ∇ g ( w t ) 1 b ∑ b l =1 u t,l µ ( f ( v t +1 + µu t,l ) -f ( v t +1 )) . Then

Taking derivative of ∇ ˆ f z i t , ¯ z j t ( w t ) over w t , we get that

Thus,

Now, plugging the above inequality back into Equation (17) yields

where the second inequality is due to η t ≤ 1 2 ρt ≤ 3 2 ρ . The above inequality implies

Secondly, when j t = j , there holds

Combining the above two cases, we can get that

Taking expectation over i t ,

̸

̸

Then, taking expectation over A and taking summation from t = 1 to T -1 to get that

where the fourth inequality is from e x ≥ 1 + x .

Next, we will study the optimization bound. According to Equation (6) in Lemma 4, ∇ f ( w t ) = β ∇ g ( w t ) ∇ f ( v t +1 ) , second-order Taylor expansion, Lemmas 5 and 6, we provide that, for any p ≥ √ 2 ,

where the second inequality is due to η t = 1 pγt ≤ p 2 αt ≤ p 2 α when p ≥ √ 2 α γ , and the last inequality is from Equation 8 in Lemma 4. Then,

We multiply both sides of the above inequality by t to get that

Then

That is

Combining Theorem 1, Equations (16), (18) and (19), we can get that

SCSC: Similar to the stability proof of SCGD except for

and

based on the update of SCSC, we have that, for η t ≤ 1 2 ρt ≤ 3 2 ρ , ρ = 1 µ ( α g M f + L 2 g M ′ f ) ,

Similar to the optimization proof of SCGD, we also have that

where d 1 = d -1 , d 2 = d -(2 p +1) + ( p + 1 2 ) 2 . Combining Theorem 1, Equations (20), (21) and (22), we can get that

Proof of Corollary 1 :

̸

With Assumption 3, the terms ∇ f ( v t +1 ) and g ( w t ) are both differentiable. Thus, 1 b ∑ b l =1 u t,l µ ( g ( w t + µu t,l ) -g ( w t )) ∇ f ( v t +1 ) is also differentiable. It is reasonable to assume

that there exists a primitive function ˆ f ( w t ) at least whose derivative function ∇ ˆ f ( w t ) = 1 b ∑ b l =1 u t,l µ ( g ( w t + µu t,l ) -g ( w t )) ∇ f ( v t +1 ) . Then

Taking derivative of ∇ ˆ f z i t , ¯ z j t ( w t ) over w t , we get that

Thus,

Let ρ = 1 µ ( βL g α f M g + M ′ g L f ) , then ˆ f z i t , ¯ z j t ( w t ) is ρ -smooth. And we can know that λ min ( ∇ 2 ˆ f z i t , ¯ z j t ( w t ) ) ≥ -∥ ∥ ∥ ∇ 2 ˆ f z i t , ¯ z j t ( w t ) ∥ ∥ ∥ ≥ -ρ . According to Lemma 3, we can get that 〈 w t -w i,z t , ∇ ˆ f z i t , ¯ z j t ( w t ) -∇ ˆ f z i t ¯ z j t ( w i,z t ) 〉

Now, plugging the above inequality back into Equation (23) yields

where the second inequality is due to η t ≤ 1 2 ρt ≤ 3 2 ρ . The above inequality implies

Secondly, when i t = i , there holds

Combining the above two cases, we can get that

Taking expectation over i t ,

̸

Then, taking expectation over A and taking summation from t = 1 to T -1 to get that

where the fourth inequality is from e x ≥ 1 + x .

̸

̸

With Assumption 3, the terms ∇ f ( v t +1 ) and g ( w t ) are both differentiable. Thus, 1 b ∑ b l =1 u t,l µ ( g ( w t + µu t,l ) -g ( w t )) ∇ f ( v t +1 ) is also differentiable. It is reasonable to assume that there exists a primitive function ˆ f ( w t ) at least whose derivative function ∇ ˆ f ( w t ) = 1 b ∑ b l =1 u t,l µ ( g ( w t + µu t,l ) -g ( w t )) ∇ f ( v t +1 ) . Then

Taking derivative of ∇ ˆ f z i t , ¯ z j t ( w t ) over w t , we get that

Thus,

Let ρ = 1 µ ( βL g α f M g + M ′ g L f ) , then ˆ f z i t , ¯ z j t ( w t ) is ρ -smooth. And we can know that λ min ( ∇ 2 ˆ f z i t , ¯ z j t ( w t ) ) ≥ -∥ ∥ ∥ ∇ 2 ˆ f z i t , ¯ z j t ( w t ) ∥ ∥ ∥ ≥ -ρ . According to Lemma 3, we can get that 〈 w t -w j, ¯ z t , ∇ ˆ f z i t , ¯ z j t ( w t ) -∇ ˆ f z i t ¯ z j t ( w j, ¯ z t ) 〉

Now, plugging the above inequality back into Equation (25) yields

where the second inequality is due to η t ≤ 1 2 ρt ≤ 3 2 ρ . The above inequality implies

Secondly, when j t = j , there holds

Combining the above two cases, we can get that

Taking expectation over i t ,

̸

Then, taking expectation over A and taking summation from t = 1 to T -1 to get that

̸

where the fourth inequality is from e x ≥ 1 + x .

Next, we will study the optimization bound. According to Equation (6) in Lemma 4, ∇ f ( w t ) = β ∇ g ( w t ) ∇ f ( v t +1 ) , second-order Taylor expansion, Lemmas 5 and 6, we provide that, for any p ≥ √ 2 ,

where the second inequality is due to η t = 1 pγt ≤ p 2 αt ≤ p 2 α when p ≥ √ 2 α γ , and the last inequality is from Equation 8 in Lemma 4. Then,

We multiply both sides of the above inequality by t to get that

Then

That is

Combining Theorem 1, Equations (24), (26) and (27), we can get that

SCSC: Similar to the stability proof of SCGD except for

based on the update of SCSC, we have that, for η t ≤ 1 2 ρt ≤ 3 2 ρ , ρ = 1 µ ( L g α f M g + M ′ g L f ) ,

Similar to the optimization proof of SCGD, we also have that

where d 1 = d -1 , d 2 = d -(2 p +1) + ( p + 1 2 ) 2 . Combining Theorem 1, Equations (28), (29) and (30), we can get that

Proof of Corollary 2 :

̸

and

With Assumption 3, the terms f ( v t +1 ) and g ( w t ) are both differentiable. Thus, 1 b 2 ∑ b l =1 u t,l µ ( f ( v t +1 + µu t,l ) -f ( v t +1 )) ∑ b l =1 u t,l µ ( g ( w t + µu t,l ) -g ( w t )) is also differentiable. It is reasonable to assume that there exists a primitive function ˆ f ( w t ) at least whose derivative function is 1 b 2 ∑ b l =1 u t,l µ ( f ( v t +1 + µu t,l ) -f ( v t +1 )) ∑ b l =1 u t,l µ ( g ( w t + µu t,l ) -g ( w t )) . Then

Taking derivative of ∇ ˆ f z i t , ¯ z j t ( w t ) over w t , we get that

Thus,

Now, plugging the above inequality back into Equation (31) yields

where the second inequality is due to η t ≤ 1 2 ρt ≤ 3 2 ρ . The above inequality implies

Secondly, when i t = i , there holds

Combining the above two cases, we can get that

Taking expectation over i t ,

̸

Then, taking expectation over A and taking summation from t = 1 to T -1 to get that

̸

where the fourth inequality is from e x ≥ 1 + x .

̸

With Assumption 3, the terms ∇ f ( v t +1 ) and g ( w t ) are both differentiable. Thus, 1 b 2 ∑ b l =1 u t,l µ ( f ( v t +1 + µu t,l ) -f ( v t +1 )) ∑ b l =1 u t,l µ ( g ( w t + µu t,l ) -g ( w t )) is also differentiable. It is reasonable to assume that there exists a primitive function ˆ f ( w t ) at least whose derivative function is 1 b 2 ∑ b l =1 u t,l µ ( f ( v t +1 + µu t,l ) -f ( v t +1 )) ∑ b l =1 u t,l µ ( g ( w t + µu t,l ) -g ( w t )) . Then

Taking derivative of ∇ ˆ f z i t , ¯ z j t ( w t ) over w t , we get that

Thus,

Now, plugging the above inequality back into Equation (33) yields

where the second inequality is due to η t ≤ 1 2 ρt ≤ 3 2 ρ . The above inequality implies

Secondly, when j t = j , there holds

Combining the above two cases, we can get that

Taking expectation over i t ,

̸

Then, taking expectation over A and taking summation from t = 1 to T -1 to get that

where the fourth inequality is from e x ≥ 1 + x .

As for the optimization analysis of the full black-box SCGD, we can combine the proofs of Theorem 4 and Corollary 1 to get that

where d 2 = d -2 √ ( p + 1 2 ) β + ( p + 1 2 ) β . Combining Theorem 1, Equations (32), (34) and (35), we can get that

SCSC: Similar to the proof of SCGD except for

̸

based on the update of SCSC, we have that, for η t ≤ 1 2 ρt ≤ 3 2 ρ , ρ = 1 µ 2 ( L g M ′ f M g + M f M ′ g ) ,

As for the optimization analysis of the full black-box SCGD, we can combine the proofs of Theorem 4 and Corollary 1 to get that

where d 2 = d -2 √ ( p + 1 2 ) + ( p + 1 2 ) . Combining Theorem 1, Equations (36), (37) and (38), we can get that

E Proofs of Applications

Before stating our remain proofs, it should be noted that there are a few differences between the setting of FOO-based VFL (VFL-CZOFO) and the one of SCGD (SCSC). First of all, we set S = { z 1 , ..., z n } and S i,z = { z 1 , ..., z i -1 , z ′ i , z i +1 , ..., z n } according to the learning paradigm of VFL. Secondly, the update of the outer model (global model) for FOO-based VFL (VFL-CZOFO) is not based on the simple weighted summation of SCGD (SCSC). Luckily, these differences will not make a difference in our proofs.

Proof of Corollary 4 :

FOO-based VFL: Considering the independence of all clients, we just prove the corresponding result of the k -th client for some k ∈ [ K ] . Firstly, when i t = i , there holds

̸

where w i,z,k t is simplified as w i,k t . With Assumption 3, the terms ∇ g ( w k t ) and ∇ f ( g ( w k t )) are both differentiable. Thus, ∇ g ( w k t ) ∇ f ( g ( w k t )) is also differentiable, which means that it is continuous on its domain. As we all know, a continuous function has primitive functions. Then, it is reasonable to assume that there exists a primitive function ˆ f ( w k t ) at least whose derivative function ∇ ˆ f ( w k t ) = ∇ g ( w k t ) ∇ f ( g ( w k t )) . For example, considering the independence between v t and g ( w k t ) , f ( g ( w k t )) is one of the primitive functions ˆ f ( w k t ) . Then

Taking derivative of ∇ ˆ f ( w k t ) over w k t , we get that

and

Thus,

Let ρ = α g L f + L 2 g α f , then ˆ f ( w k t ) is ρ -smooth. And we can know that λ min ( ∇ 2 ˆ f ( w k t ) ) ≥ -∥ ∥ ∥ ∇ 2 ˆ f ( w k t ) ∥ ∥ ∥ ≥ -ρ . According to Lemma 3, we can get that

Now, plugging the above inequality back into Equation (39) yields

where the second inequality is due to η t ≤ 1 2 ρt ≤ 3 2 ρ . The above inequality implies

Secondly, when i t = i , there holds

Combining the above two cases, we can get that

Taking expectation over i t ,

̸

Then, taking expectation over A and taking summation from t = 1 to T -1 to get that

̸

where the fourth inequality is from e x ≥ 1 + x . Combining Theorem 3 and Equation (40), we can get that

□

Proof of Corollary 5 : Similar to the proofs of Theorem 4 and Corollary 4, we can get the result of Corollary 5. □

F Key Challenges and Technical Tools

In this section, the key challenges and technical tools of extending the theoretical analysis for SCO problems from white-box cases to black-box cases are listed as follows.

(1) Generalization: Considering three different types of black-box SCO methods, we apply our new non-convex analysis (Theorem 3) to these cases (Theorem 4, Corollary 1 and 2) in Section 3.2. Due to the difference related to function form, there are some differences related to the upper bounds of first-order and second-order gradients of ˜ ∇ f between Theorem 3 and the generalization part of Theorem 4. The differences among Theorem 3 and Corollary 1, Corollary 2 are the same as Theorem 4.

(2) Optimization: For optimization, the estimated gradient does introduce several extra terms regarding the accuracy of the gradient estimation, i.e., ˜ ∇ f -( p + 1 / 2) β ∇ f and ˜ ∇ f -β ∇ f . These terms are derived from some special strategies (such as a special decomposition ˜ ∇ f = ˜ ∇ f +( p +1 / 2) β ∇ f -( p +1 / 2) β ∇ f ). We propose an extended lemma (Lemma 6) from [39] and combine this lemma with these strategies to limit the expansion of E [ F S ( w t +1 ) -F S ( w ( S ))] during the iterations. Otherwise, these extra terms will lead to the divergence of our result.

Finally, we want to emphasize our advantages compared with previous work related to the generalization guarantee of SCO [21].

(1) Better results: For convex optimization, Theorem 2 leverages the co-coercivity property of convex and smooth function to provide the stability bound O (( n -1 + m -1 ) β log T ) under milder parameter selection than [21]. And our proof is more concise since it avoids the intermediate step which measures the distance between v and g ( w ) in the analysis of [21].

(2) Non-convex guarantee: We leverage a special lemma, almost co-coercivity lemma, to develop our proof framework to non-convex case to obtain the first stability bound O (( n -1 + m -1 ) T 1 2 log T ) under milder parameter selection than [39].

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: The paper's contributions and scope can be found at the end of the abstract and introduction.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [No]

Justification: Despite the limitation isn't discussed in a separate "Limitations" section, some remarks of results have demonstrated the limitation. For example, Remark 4 shows that Theorem 3 is looser than Theorem 2. Remark 5 shows that Theorem 4 is derived from a more stringent condition, i.e., a smaller learning rate.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: The full set of assumption and a complete (and correct) proof are provided in the Section 2 and Appendices C, D, E .

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [NA]

Justification: The paper's contributions are from the theoretical analysis perspective.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [NA]

Justification: The paper's contributions are from the theoretical analysis perspective. There isn't any data or code.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [NA]

Justification: The paper's contributions are from the theoretical analysis perspective.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [NA]

Justification: The paper's contributions are from the theoretical analysis perspective.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [NA]

Justification: The paper's contributions are from the theoretical analysis perspective.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: The research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [NA]

Justification: The paper's contributions are from the theoretical analysis perspective. It theoretically explains the impact of black-box on the learning guarantees of SCO algorithms, which may benefit the algorithm designing of SCO algorithm.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: The paper's contributions are from the theoretical analysis perspective without any data or model being released.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [NA]

Justification: The paper's contributions are from the theoretical analysis perspective. The algorithms analyzed in the paper have been cited properly.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: The paper's contributions are from the theoretical analysis perspective without any new assets being released.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: The paper's contributions are from the theoretical analysis perspective.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: The paper's contributions are from the theoretical analysis perspective.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this paper, the authors systematically analyzed the generalization error and optimization error of the stochastic compositional optimization problems (for black-box cases).\n\nSTRENGTHS:\n1. The paper is well-written and well-organized. \n\n\n2. The paper provides the generalization analysis and optimization analysis for stochastic compositional optimization problems in a systematic way.   The convergence rates for three different black-box cases are provided.\n\nWEAKNESSES:\n1. The technical challenge of extending the theoretical analysis (stability analysis and optimization analysis) for stochastic compositional optimization problems from white-box cases to black-box cases is not clearly discussed.  What are the key technical tools employed to address the key challenge?  It seems that the black-box gradient approximation in Eq.(5) satisfies the bounded variance assumption. So it is easy to plug this into the existing white-box analysis framework to achieve a generalization error and optimization convergence rate. What are the non-trivial techniques involved in improving the theoretical results compared with the previous work? \n\n\n2.  Assumption 5 for the non-convex analysis is weird.   Assumption 5 implies that any stationary point, i.e., $\\mathbb{E}[|| \\nabla F_S(w) ||^2]=0$, to be the global optimal point $w(S)$.    Is this assumption too strong for non-convex analysis? \n\n\nMinor:\n\nTypos:   Eq.(5) is a second-order approximation of Eq.(4),  \"=\" may be incorrect.\n\nQUESTIONS:\n1. What is the key technical challenge of extending the theoretical analysis (stability analysis and optimization analysis) for stochastic compositional optimization problems from white-box cases to black-box cases?\n\n2.   What are the key technical tools employed to address the key challenge? \n\n3.  Is this assumption 5  too strong for non-convex analysis?\n\nLIMITATIONS:\nN.A.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-12"}

comments: SUMMARY:
This work presents the generalization upper bound for two stochastic compositional optimization methods, SCGD and SCSC, under convex and non-convex setting. For convex setting, the presented generalization bound is tighter compared to the existing work and it matches the generalization bound of SGD for optimization without compositional structure. For non-convex setting, this work establishes the first generalization bound in SCO literature. Furthermore, this work studies the zeroth-order extension of SCGD/SCSC and presents their excess risk bound in non-convex setting. This is also new in the literature.

STRENGTHS:
1. The main contribution of this work is the theoretical analysis as I summarized in the summary section. 
2. It is interesting to see how estimation distance $\mu$ and the number of estimation directions $b$ affect the excess risk bound.
3. This work is well-written and easy to follow.

WEAKNESSES:
I do not recognize any significant weakness in this work. But I do have some questions.

1. In Table 1, I noticed that the assumption V (bounded variance) and M (bounded function) do appear in SCGD/SCSC (Thm. 2), but not in SGD ([26] Thm. 3.8). Based on my understanding, the assumptions on both of these two analysis are the Lipschitz and smoothness of the unbiased stochastic function values. I do not see any major difference between them. Can the authors clarify what V (bounded variance) and M (bounded function) stand for?

2. In Table 2, the optimization bounds for the black-box methods seem to omit the terms involving $T,n,m$. This confuses me. Is there any reason that the $T,n,m$ terms are ignored here?

QUESTIONS:
Please see the weakness section.

LIMITATIONS:
No significant limitations in this work.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-13

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies stability-based generalization bound for SCGD and SCSC, as well as the convergence rated of their black-box variants. The authors provide sharper generalization bounds for these algorithms, and the first convergence bounds for the black-box variants of these algorithms under PL-conditions.\n\nSTRENGTHS:\n### Strengths:\n\n1. **Novelty of the Problem**: The generalization bounds of stochastic optimization algorithms have been previously studied (e.g., [21]), and basic mathematical tools (algorithm stability) for such studies have been proposed. This paper is the first to study block-box SCSC. Although from the proofs of Theorems 3 and 4, the analyses for SCGD and SCSC are essentially the same.\n\n2. **Significance of the Results**: \n   - **Generalization Bound**: The authors provide a generalization bound of \\(O((1/n + 1/m)\\log T + 1/\\sqrt{n})\\) for SCGD and SCSC, which is significantly sharper than the previous \\(O(1/n + 1/m)\\log T + 1/\\sqrt{n})\\) bound. This is a noteworthy contribution.\n   - **Algorithm Convergence**: The paper presents convergence bounds for algorithms using estimated (black-box) gradients. While the dependence on \\(T\\), \\(n\\), and \\(m\\) is similar to [21], there are additional terms related to the estimation distance mu and sample size b. Moreover, the authors proved this results under a milder condtion (convexity vs PL-condtion. )\n\nThere is no experiments in this paper. However, the main goal of this paper is to study the properties of existing algorithms, so experiments are not necessary. But in the remarks the authors say that their theorem provide a more \"practical choise\" for paratmeres, so experiments might still be helpful to demonstrate this. \n\n### Weaknesses/Questions:\n\n1. **Comparing Convergence Rates**: In Table 2, it is difficult to compare the convergence rate of the proposed algorithm with other methods. The theoretical results of other methods are provided with respect to \\(T\\), but for the proposed methods, the dependence is given with respect to \\(\\mu\\) and \\(d\\). This inconsistency makes the table unclear. For instance, Theorem 3.7 of [21] also provides a way to configure \\(T\\). It should be made more clear that mu and b are extra terms.\n   - **Parameter \\(\\mu\\)**: The parameter \\(\\mu\\) is mentioned only once in Line 112, and it is unclear how good the \\(O(\\mu^2)\\) bound is. It seems that \\(\\mu\\) is used in the estimation of the gradient and is a constant. Therefore, Theorem 4 suggests that even if \\(n\\), \\(m\\) and \\(b\\) approach infinity, the difference between function values will converge to a constant instead of zero.\n\n2. **Redundancy in Proofs**: In the appendix, the proofs of Theorems 3 and 4 appear nearly identical (at least for parts 1) and 2)), with the main difference being the use of the estimated gradient. Additionally, the proof of Corollary 1 repeats similar arguments. I encourage the authors to reduce redundancy, shorten the proofs, and highlight the main differences to enhance readability.\n\n3. **Challenges of Estimated Gradient**: The current form of the proofs makes it difficult to tell the main challenges of introducing the estimated gradient in Theorem 4 compared to Theorem 3. From Lines 546-547, it seems the estimated gradient only introduces several extra terms regarding the accuracy of the gradient estimation, which appears to be standard. Clarifying the main challenges and differences would improve the paper.\n\nWEAKNESSES:\nPlease see above.\n\nQUESTIONS:\nPlease see above.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-15"}