Projection-free Constrained Stochastic Nonconvex Optimization with State-dependent Markov Data
Anonymous Author(s)
AffiliationAddress email
Abstract
We study a projection-free conditional gradient-type algorithm for constrainednonconvex stochastic optimization problems with Markovian data.
In particular, wefocus on the case when the transition kernel of the Markov chain is state-dependent.
Such stochastic optimization problems arise in various machine learning problemsincluding strategic classification and reinforcement learning.
For this problem, weestablish that the number of calls to the stochastic first-order oracle and the linearminimization oracle to obtain an appropriately defined ϵ -stationary point, are ofthe order O (1 /ϵ 2 . 5 ) and O (1 /ϵ 5 . 5 ) respectively.
We also empirically demonstratethe performance of our algorithm on the problem of strategic classification withneural networks
Introduction
We consider the following stochastic optimization problem
[Equation]
where (i) the expectation is taken over the stationary distribution, π θ , of the random vector x , (ii) F(and hence f ) is a potentially non-convex function in θ , and (iii) Θ is a compact and convex constraintset.
Stochastic approximation algorithms for solving problem (1), given an independent and identicallydistributed ( iid ) data stream { x k } k drawn from π , are well-studied.
Such iid assumptions arecommonly made in various machine learning and statistical problems including empirical riskminimization [SSBD14], sparse recovery [BJMO12] and compressed sensing [FR13, Lan20].
Werefer to [MB11, ABRW12, RSS12, GL13, SZ13, LZ16, ACD + 19] for a partial list of non-asymptoticupper and lower bounds on the oracle complexity of widely-used stochastic approximation algorithmslike the Stochastic Gradient Descent (SGD) and the Stochastic Conditional Gradient Algorithm.
Our focus in this work is on the case when the data sequence { x k } k is drawn from a Markovchain with a state-dependent transition kernel P θ .
Such a setting arises in several machine learningapplications including but not limited to strategic classification [HMPW16, CDP15, MDPZH20,
LW22] and reinforcement learning [Bar92, GSK13, ZJM21, KMMW19, QW20].
Despite theirprevalence in practice, a deeper understanding of the non-asymptotic oracle complexity of stochasticapproximation for Markovian data is only now starting to emerge.
We establish non-asymptoticoracle complexity results for the stochastic conditional gradient algorithm for non-convex constrainedstochastic optimization with Markovian data.
To establish our results, from a methodologicalpoint-of-view, we leverage the moving-average stochastic gradient estimation technique recentlyused in [ZSM + 20, GRW20, XBG22] in the context of constrained optimization with iid data.
Thistechnique avoids having to a use a mini-batch of samples in each iteration, which turns out to be crucialin the non- iid setup we consider.
From a theoretical point-of-view, we assume the so-called driftconditions, a classical assumption in Markov Chain literature [AMP05].
This ensures the existence of a solution to the Poisson equation associated with the underlying Markov chain [DMPS18] whichenables one to decompose the noise present in the stochastic gradient into three components: amartingale difference sequence, a time-decaying sequence, and a telescopic sum type sequence.
Thekey idea of our paper is to use this decomposition to construct an auxiliary sequence of iterates with atime-decaying noise-variance and show that these sequence of iterates are close to the iterates of theoriginal sequence produced by our algorithm.
This novel technique in then used in combination witha merit-function based analysis to establish the oracle complexity results.
1.1 Motivating Example
Problems of the form in (1) arise in various important applications, e.g., strategic classification, andreinforcement learning as mentioned above.
Below we illustrate the motivation of this work throughthe example of strategic classification with adapted best response [LW22].
In strategic classification,there is a learner whose task is to classify a given dataset which is collected from a set of agents .
Given the knowledge of the classifier, the agents can distort some of their personal features, in order toget classified in a predetermined target class.
This scenario arises in various applications, e.g., spamemail filtering, and credit score classification.
Optimizing the classifier to classify such strategicallymodified data where the agents modify the data iteratively can be formulated as problem (1).
Formally, let the classifier be h ( x, θ ) where x ∈ R d is the feature and θ is the parameter to beoptimized.
h ( x ; · ) :
Θ → R is potentially nonconvex.
Let the loss function be logistic loss which fora sample ( x, y ) , where y ∈ {− 1 , 1 } denotes the corresponding class, is given by,
[Equation]
We use x S , and x − S to denote the subset of feature x which are respectively strategically modifiable,and non-modifiable by the agents.
Then the modified feature (the best response) x ′ S reported by theagent is the solution to the following optimization problem:
[Equation]
where c ( x, x ′ ) is the cost of modifying x S to x ′ S .
Let the agents iteratively learn x ′ S similar to[LW22].
Note that unlike [LW22], where the authors deploy a logistic regression classifier and theclosed form solution of the best response is readily known to the agents, it may not be the case ingeneral.
In that case the agents have to possibly learn the best response x ′ S using some iterativeoptimization algorithm.
For example, if the agents use Gradient Ascent then, at every iteration k , aset I k of n 1 ≤ M randomly chosen agents out of M agents modify their features as:
[Equation]
where α is the stepsize.
With a little abuse of notation, we use ∇ h ( x k − 1 S,i ; θ ) in (4) to denote the fact that the gradient is with respect to x k − 1 S,i while x − S,i remains unchanged.
This introduces the state-dependent Markov chain dynamics in the training data.
The objective function, analogous to f ( θ ) in (1), is
[Equation]
where π θ is the stationary joint distribution of ( x, y ) , and Θ is a convex and compact set, e.g., sparsityinducing constraint ∥ θ ∥ 1 ≤ R from some R > 0 .
The loss evaluated at a single data point ( x, y ) , L ( θ ; x, y ) , is analogous to F ( θ ; x ) in (1).
[DX20], and [LW22] study this problem theoretically andempirically respectively in an unconstrained strongly convex setting.
Our results takes a step towardsanalyzing this problem in constrained nonconvex setting.
We empirically show the performance ofthe stochastic conditional gradient algorithm on a strategic classification problem in Section 4.
1.2 Preliminaries and Main Contributions
Before we present our main contributions, we introduce our convergence criterion.
In constrainedoptimization literature, most commonly used convergence criteria are: (i) Gradient Mapping (GM),and (ii) Frank-Wolfe Gap (FW-gap).
The Gradient Mapping at a point ¯ θ ∈ Θ is defined as
[Equation]
where Π Θ ( x ) denotes the orthogonal projection of the vector x onto the set Θ , i.e.,
[Equation]
We will use Π Θ (¯ θ, ∇ f (¯ θ ) , β ) to denote Π Θ (cid:0) ¯ θ − ∇ f (¯ θ ) /β (cid:1) when there is no confusion.
Note thatwhen
Θ ≡ R d we have G Θ (¯ θ, ∇ f (¯ θ ) , β ) = ∇ f
(¯ θ ) .
In other words, for constrained optimizationgradient mapping plays an analogous role of the gradient for unconstrained optimization.
The gradientmapping is a frequently used measure in the literature as a convergence criterion for nonconvexconstrained optimization [Nes18].
We should emphasize here that although the gradient mappingcannot be computed in the stochastic setting, one can still use it as a convergence measure.[BG22] shows that the above notion of convergence criterion is closely related to the so-called
Frank-Wolfe Gap .
The FW-gap is defined as
[Equation]
The following proposition from [BG22] establishes the relation between the gradient mappingcriterion and the Frank-Wolfe gap:
Proposition 1.1 [BG22] Let g Θ ( · ) be the Frank-Wolfe gap defined in (6) and G Θ ( · ) be the gradientmapping defined in (5) .
Then, we have
[Equation]
Moreover, under standard regularity assumption in smooth optimization (specifically, Assumption 2.1,and 2.2), we have
[Equation]
In this work we use a suboptimality measure, closely related to both GM and the FW-gap.
At point¯ θ ∈ Θ , we define the suboptimality measure V (¯ θ, z ) :
R d × R d →
[Equation]
as [GRW20]
[Equation]
where z , formally defined in Algorithm 1, is the moving-average estimate of ∇ f (¯ θ ) .
We show therelation among V ( θ, z ) , GM G Θ ( θ, z, β ) , and FW-gap g ( θ k , ∇ f ( θ k )) in the following proposition.
Proposition 1.2 Let { z k } be the sequence generated in Algorithm 1.
Then, for k = 1 , 2 , · · · , N ,
[Equation]
The proof is provided in Appendix A. Observe that by establishing ϵ -stationarity in terms of 93 E [ V ( θ k , z k )] one can have a tighter bound on FW-gap than gradient mapping ϵ -stationarity asdescribed in Definition 1.
Indeed E [ V ( θ k , z k )] ≤ ϵ implies E [ g Θ ( θ k , ∇ f ( θ k ))]
≤ ϵ unlike gradientmapping criterion where, according to (7), E (cid:104) ∥G ( θ k , z k ) ∥ 2 2 (cid:105) ≤ ϵ implies E [ g Θ ( θ k , ∇ f ( θ k ))] ≤ √ ϵ .
The main objective of this work is to find an ϵ -stationary solution to (1), where an ϵ -stationary solutionis defined as follows:
Definition 1 A point ¯ θ is said to be an ϵ -stationary solution to (1) , if E (cid:2) V (¯ θ, z ) (cid:3) ≤ ϵ , where theexpectation is taken over all the randomness involved in the problem.
For stochastic Frank-Wolfe-type algorithms, the oracle complexity is measured in terms of number ofcalls to the Stochastic First-order Oracle (SFO) and the Linear Minimization Oracle (LMO) used tothe solve the sub-problems of the algorithm which involves minimizing a linear function over theconvex constraint set.
Formally, we have the following definition.
Definition
For a given point θ ∈ Θ , SFO returns the stochastic gradient ∇ F ( θ, x ) .
Given a vectorz , LMO returns a vector v := argmin y ∈ Θ ⟨ z, y ⟩ .
Hence, in this work, the oracle complexity is measured in terms of the number of calls to SFO and
LMO required by the proposed algorithm to obtain an ϵ -stationary solution as in Definition 1.
Withthe above preliminaries, we now list our main contributions :
[Table] iid
[Table]
Criterion
[Table] [Table]
[Table]
O (cid:0)
[Table] (cid:1)
O (cid:0) ϵ − [Table]
(cid:1) ✗ ✗ ✗ ✗ (ASA+ICG)
[XBG22]
GM
[Table] (cid:0)
[Table]
− 2 (cid:1)
[Table]
(cid:1) ✗ ✗ ✗ ✗ (ASA [Table] ICG) [
[Table] paper] [Table] [Table]
• In Theorem 3.1, we show that the number of calls to the SFO and LMO required by thestochastic conditional gradient-type method in Algorithm 1, with state-dependent Markoviandata, is of order O ( ϵ − 2 . 5 ) and O ( ϵ − 5 . 5 ) respectively.
To the best of our knowledge, theseare the first oracle complexity results for projection-free one-sample stochastic optimizationalgorithm for constrained nonconvex optimization in the Markovian setting.•
In Theorem 3.2, for the sake of completion, we also show that the number of calls to the
SFO and LMO required for the case of state-independent Markovian data is of the order˜ O
( ϵ − 2 ) and ˜ O ( ϵ − 3 ) respectively.
In particular, this turns out to be of the same order as thatof iid data ignoring the logarithmic factors.
A summary of the our contributions is provided in Table 1.
We also empirically evaluate our algorithmon a strategic classification problem with 2-layer neural network classifier and show that the proposedmethod obtains encouraging results.
We provide an experiment on single-index model regressionwith sparsity-inducing nuclear-norm ball constraint in Appendix C.
1.3 Related Work
Stochastic Optimization with Dependent Data.
Understanding stochastic approximation algorithmslike SGD with dependent data in the asymptotic setting has been well-explored in the optimizationliterature.
We refer to [KY03, Bor09, BMP12] for a text-book introduction to such classical results.
A few recent results include [AMP05, TD17].
In the unconstrained non-asymptotic setting, [DAJJ12]studies convex optimization with ergodic data sequence.
[DL22] uses multi-level gradient estimatorand analyze AdaGrad for nonconvex optimization with Markovian Data.
Block coordinate descentwith homogeneous Markov chain has been analyzed in [SSXY20] for nonconvex unconstrainedoptimization.
[DX20] studies stochastic optimization with decision-dependent data distribution forstrongly convex functions in the context of strategic classification.
Sample-average approximation algorithms for constrained convex optimization with ϕ -mixing datawas considered in [WPT + 21].
[SSY18], and [AL22] analyze projected SGD for constrained nonconvex optimization with time-homogeneous Markov chain.
None of these works consider statedependent data distribution except [DX20].
But unlike [DX20], we consider constrained nonconvexoptimization.
There also exists work in the reinforcement learning literature on understanding stochastic optimization with Markovian data; see, for example [XXLZ20, BRS18, DNPR20].
However, suchworks are invariably focused on specific objective functions arising in the reinforcement learningsetup, while our focus is on obtaining results for a general class of functions.
Conditional Gradient-Type Method.
There has been significant recent advancements in theconditional gradient algorithm literature although it was developed long back [FW56, LP66];see [Mig94, Jag13, LJJ15, LJJ15, HJN15, GKS21, BS17], for a non-exhaustive list of recent works.[HK12, HL16] provided expected oracle complexity results for stochastic conditional gradient algorithm in the stochastic convex setup.
Better rates were provided by a sliding procedure in [LZ16].
Inthe non-convex setting, [RSPS16, YSC19, HL16] considered variance reduced stochastic conditionalgradient algorithms, and provided expected oracle complexities.
[QLX18] analyzed the slidingalgorithm in the non-convex setting and provided results for the gradient mapping criterion.
All ofthe above works use increasing orders of mini-batch based gradient-estimate.
To avoid mini-batches, a moving-average gradient estimator based on only one-sample in each iteration for a stochastic conditional gradient-type algorithm was proposed in [MHK20] and [ZSM + 20]for the convex and non-convex setting.
However, several restrictive assumptions have been made in[MHK20] and [ZSM + 20].
Specifically, [ZSM + 20] requires that the stochastic gradient G 1 ( x, ξ 1 ) hasuniformly bounded function value, gradient-norm, and Hessian spectral-norm, and the distribution ofthe random vector ξ 1 has an absolutely continuous density p such that the norm of the gradient of log p and spectral norm of the Hessian of log p has finite fourth and second-moments respectively.
Incontrasts, we do not require such stringent assumptions.
Assumptions
We now introduce the precise assumptions we make in this work.
Let F k be the filtration generatedby { θ 0 , · · · , θ k , z 0 , · · · , z k , x 1 , · · · , x k } .
For any mapping g : R d → R d define the following normwith respect to a function V : R d → [1 , ∞ ) :
[Equation]
, and let L V
= { g : R d → R d , sup x ∈ X ∥ g ∥ V < ∞} .
Assumption 2.1 (Constraint set)
The set Θ ⊂ R d is convex and closed with max x,y ∈ Θ
∥ x − y ∥ 2 ≤ D Θ ,form some D Θ > 0 .
Assumption 2.2 Let f be a continuously differentiable function.
Assumption 2.3 Let ξ k +1 ( θ k , x k +1 ) :
= ∇ F ( θ k , x k +1 ) − ∇ f ( θ k ) .
Then,
E
[Equation]
Assumption 2.4 Let { x k } k be a Markov chain with transition kernel P θ .
For any θ ∈ Θ , P θ isirreducible and aperiodic.
Additionally, there exists a function V : R d → [1 , ∞ ) and a constantα ≥ 2 such that for any compact set Θ ′ ⊂ Θ :
(a) There exist a set C ⊂ R d , an integer I , constants 0 < λ < 1 , b , κ , δ > 0 , and a probability measure ν such that,
[Equation]
where B R d is the Borel σ -algebra over R d
(b) There exists a constant c > 0 , such that, for all x ∈ R d ,
[Equation]
(c) There exists a constant c > 0 , such that, for all ( θ, θ ′ ) ∈ Θ ′ × Θ ′ ,
[Equation]
Some comments regarding the assumptions are in order.
Assumption 2.1, and Assumption 2.2 arecommon for constrained optimization [GRW20, XBG22, AL22, ZSM + 20].
Assumption 2.1, and
Assumption 2.2 together imply the Lipschitz continuity of f ( · ) , i.e., there is a constant L > 0 suchthat for any θ 1 , θ 2 ∈ Θ , we have | f ( θ 1 ) − f
( θ 2 ) | ≤ L ∥ θ 1 − θ 2 ∥ 2 .
Assumption 2.3 is common instochastic optimization literature.
Assumption 2.4(a) is a frequently used assumption in Markovchain literature.
It implies that for every θ ∈ Θ , there exists a stationary distribution π θ ( x ) , and thechain is V α -uniformly ergodic [AMP05].
Assumption 2.4(c) provides smoothness guarantee on the function f ( · ) .
More formally, we have the following proposition.
Proposition 2.1 (Lipschitz continuous gradient [AMP05]) Let Assumption 2.4 be true.
Then f ( · )has Lipschitz continuous gradient, i.e., there is a constant L G > 0 such that for any θ 1 , θ 2 ∈ Θ :
[Equation]
Finally, the most important implication of Assumption 2.4 is that it ensures the existence and regularityof a solution u ( θ, x ) to Poisson equation of the transition kernel P θ given by u ( θ, x ) − P θ u ( θ, x ) =
∇ F ( θ, x ) − ∇ f ( θ ) .
Solution of Poisson equation has been crucial in analyzing additive functionalsof Markov chain (see [AMP05] for details).
In this work, the Poisson equation solution facilitates adecomposition of the noise as presented in Lemma 3.1 which is a key component of our analysis.
Main Result
In this section we present our main result on the oracle complexity to establish a bound on 191 E [ V ( θ k , z k )] .
In order to do so we use Algorithm 1, and 2 similar to [XBG22].
If an exact Algorithm
Input: z 0 , θ 0 ∈ R d , η k = ( N + k ) − a , 1 / 2 < a < 1 , β . for k
= 1 , 2 , · · · , N do y k =
  min y ∈ Θ (cid:110) ⟨ z k , y − θ k
[Equation]
y [Equation]
θ
[Equation] 22 (cid:111) (Projection)
ICG
[Equation]
) (No Projection)
[Equation]
z
[Equation]
) end for
Output: θ k
Input: z, θ, β, t, ω .
Set w {+
+} = θ for i = 1 , 2 , · · · , t − 1 do Find v i such that
[Equation]
argmin
[Equation]
w
[Equation]
µ i v i where µ i = {+
+}
i + end for
Output: w t 192 minimizer of the following subproblem, which is the projection of θ k − z k /β on to Θ , is available,then Algorithm 1 is same as ASA algorithm introduced in [GRW20].
[Equation]
When a projection operator is unavailable or computationally costly, we use Algorithm 2 instead tosolve (17). At iteration k , Algorithm 2 finds an approximate solution to (17) based on the conditionalgradient algorithm.
Algorithm 2 needs access to LMO which is often much cheaper and simplerto compute than projection operator.
We should emphasize that our results are not limited to ICGmethod but are valid for any method which can solve (17) within an error of the order of { η k } .
Theorem 3.1 Let Assumption 2.1-2.4 be true.
Then, for Algorithm 1,(a) when a projection operator is available, choosing [Equation] for k = 1 , 2 , · · · , N we have [Equation]
(b) when Algorithm 2 is used to solve (17) ,choosing
[Equation]
for k = 1 , 2 , · · · , N we have
[Equation]
where the expectations are taken with respect to all the randomness of the algorithm, and anindependent integer random variable R ∈ { 1 , 2 , · · · , N } with probability mass function,
[Equation]
Remark 1 Note that total number of LMO calls are (cid:80)
Nk =
1 t k = (cid:80) Nk =1 t k = (cid:80) Nk
=1 ( N + k ) 2 a =
O ( N 11 / 5 ) .
In other words, to achieve ∥G Θ ( θ R , ∇ f ( θ R ) , β ) ∥ 22 ≤ E
[ V ( θ R , z R )] ≤ ϵ , SFO and LMOcomplexities are respectively ϵ − 2 .
5 , and ϵ − 5 .
Note that the SFO complexity will be ϵ − 2 .
5 as long asone has an approximation of the projection operator with approximation error O ( η k ) . {+Remark 
+}
In Theorem 3.1, one obtains sublinear rate max( N a − 1 , N 2 − 4 a ) with η k = ( N + k ) − afor 1 / 2 < a < 1 .
Choosing a = 3 / 5 provides the fastest rate of convergence.
Before sketching the outline of the proof, we present the following lemma which provides a decomposition of the noise ξ k ( θ k − 1 , x k ) – one of the key result used in the proof of the main theorem.
The lemma and its proof are almost same as Lemma A.5 in [Lia10] with the only difference thatunlike [Lia10], where the iterates are of SGD, we need to prove it for the iterates of Algorithm 1.
Weprovide the proof in Appendix A.
Lemma 3.1 Let Assumption 2.1-2.4 be true.
Then the following decomposition takes place:
[Equation]
where, { e k } is martingale difference sequence, E [ ∥ ν k ∥ 2 ] ≤ η k , and ζ k =
(˜ ζ k − ˜ ζ k +1 ) /η k , where
E (cid:104) ∥
˜ ζ k ∥ 2 (cid:105) ≤ η k .
Outline of the proof: A key step in the analysis of Algorithm 1 involves controlling the expectationof interaction with noise of the form ⟨∇ f ( θ k ) − ∇ f ( θ k − 1 ) , ξ k +1 ( θ k , x k +1 ) ⟩ .
For iid or martingaledifference data it is easy to control because E [ ⟨∇ f ( θ k ) − ∇ f
( θ k − 1 ) , ξ k +1 ( θ k , x k +1 ) ⟩|F k ] = 0 .
But this is no longer true for Markov chain data.
To resolve the issue, first notice that under ourassumptions, the noise sequence ξ k can be decomposed into the sum of a martingale differencesequence { e k } and some residual terms { ν k } , and { ζ k } as shown in Lemma 3.1.
Then the key stepis to introduce a different sequence of hypothetical iterates (˜ θ k , ˜ y k , ˜ z k ) for which the noise is smallenough so that we can bound E (cid:104) V (˜ θ k , ˜
z k ) (cid:105) , and then show that these hypothetical iterates and theoriginal sequence generated by Algorithm 1 are close enough so that E [ V ( θ k , z k )] is of the sameorder as E (cid:104) V
(˜ θ k , ˜ z k ) (cid:105) .
This step is the main novelty of the proof.
Specifically, consider the following sequence:
[Equation] [Equation]
[Equation] [Equation]
This also means,
[Equation]
[Equation]
where, ˜ ϵ
k = e k + ν k + ˜ ζ k .
Note that by Lemma 3.1, E [ e k ] = 0 , and E (cid:104)(cid:13)(cid:13) (cid:13)
ν k + ˜ ζ k (cid:13)(cid:13) (cid:13) 2 (cid:105) ≤ η k .
First weshow that by choosing η k = ( N + k ) − a , 1 / 2 < a < 1 , and t k = 1 /η 2 k one has E (cid:104) ∥
˜ θ k − θ k ∥ 22 (cid:105) =
O (cid:0) N 2 − 4 a (cid:1) , and E [ V ( θ k , z k )] ≤ 2 E (cid:104) V
(˜ θ k , ˜ z k ) (cid:105)
+ O (cid:0) N 2 − 4 a (cid:1) .
Then we establish the bound on
V (˜ θ k , ˜ z k ) .
Combining the above two facts proves Theorem 3.1.
We defer the detailed proof to
Appendix A.1.
3.1 State-independent Markov Chain
While our main goal in this work is to analyze Algorithm 1 for constrained nonconvex optimizationwith state-dependent Markov chain data, we provide the following result on the complexity of
Algorithm 1 for Markov chain data with state-independent transition kernel for the sake of completion.
Here we use P to denote the transition kernel (as opposed to P θ for state-dependent kernel).
Notethat under Assumption 2.4(a), for each θ , the chain is V -uniformly ergodic, and hence, exponentiallymixing [MT12] in the following sense:
Definition
A Markov chain is said to be exponentially mixing, if there exists C, r > 0 such that,for any initial state x , [Equation]
[Equation]
where P n ( x, · ) is the distribution of X n with initial state X
0 = x .
Now we present our result on the complexity of Algorithm 1 to find an ϵ -stationary solution to (1) forexponentially-mixing Markov chain data with state-independent transition kernel.
Theorem 3.2 Let Assumption 2.1-2.3 be true.
Let Assumption 2.4(a)-(b) be true with P θ replaced by
P .
Then, for Algorithm 1,(a) when the projection operator is available, choosing [Equation] [Equation] for k = 1 , 2 , · · · , N we have
[Equation]
(b) when Algorithm 2 is used, choosing
[Equation]
for k = 1 , 2 , · · · , N we have
[Equation]
where the expectation is taken with respect to all the randomness of the algorithm, and an independentinteger random variable R ∈ { 1 , 2 , · · · , N } whose probability mass function is given by,
[Equation]
We defer the proof to the Appendix.
Remark [-
-]3 To find an ϵ -stationary point, the total number of calls to SFO and LMO are ˜ O (cid:0) ϵ − 2 (cid:1) ,and ˜ O (cid:0) ϵ − 3 (cid:1) , where ˜ O ( · ) denotes the order ignoring logarithmic factors.
Remark [-
-]4 The authors of [AL22] obtain the same rate as in Theorem 3.2 for constrained (butprojection-based) nonconvex optimization with state-independent exponentially mixing data.
In thestate-dependent case, since the transition kernel of the Markov chain is controlled by θ k , and thetransition kernel is assumed to be only Lipschitz smooth in θ (15) , the chain does not necessarilyexponentially mix.
In the state-independent case, since the chain mixes exponentially we obtain thesame rate as well.
While their results are for projection-based algorithms, we analyze a projectionfree LMO-based algorithm since LMO is often computationally cheaper than projection.
Experimental Evaluation
In this section we illustrate our algorithm on the strategic classification problem as described in
Section 1.1 with the GiveMeSomeCredit 1 dataset.
The main task is a credit score classification
10^ −        20 k l og V ( θ k , z k )
ProjectionICG 10^ 0.
20 k
T e s t
A cc u r a cy
ProjectionICG problem where the bank (learner) has to decide whether a loan should be granted to a client.
Giventhe knowledge of the classifier the clients (agents) can distort some of their personal traits in order toget approved for a loan.
Here we use a 2-layer neural network with width m as the classifier, given by
[Equation] where υ ( · ) is the activation function, W i ∈ R d , W
= [ W 1 , W 2 , · · · , W d ] ⊤ ∈ R m × d , A =(
A 1 , A 2 , · · · , A m ) ∈ R m , B =
( B 1 , B 2 , · · · , B m ) ∈ R m .
We will use θ to collectively denote( W , A , B ) .
We impose the constraint of sparsity on the classifier given by ∥ θ ∥ 1 ≤ R for some R > 0 .
As loss function we consider logistic loss as shown in (2).
We consider a quadratic cost given byc ( x, x ′ ) =
∥ x S − x ′ S ∥ 22 / (2 λ ) where λ is the sensitivity of the underlying distribution on θ .
Weassume that the agents iteratively learn x ′ S similar to [LW22].
Note that unlike [LW22], the closedform of best response is not known here.
So we assume that the agents use Gradient Ascent (GA) tolearn the best response.
For ∥ θ ∥ 1 ≤ R constraint, the LMO in Algorithm 2 at iteration k is given by− R sign ( q i ) , where i = argmax j =1 , ··· ,d | q j | , q = z
+ β ( w k − θ ) , and q j is the j -th coordinate of q .
We select a subset of randomly chosen M = 2000 samples (agents) such that the dataset is balanced.
Each agent has 10 features.
Note that since Algorithm 1 computes the gradient on one sample atevery iterate, the computation time is independent of the total number of agents.
We assume thatthe agents can modify Revolving Utilization, Number of Open Credit Lines, and Number of Real Estate Loans or Lines.
In this experiment we set n 1 = 200 .
Similar to [LW22], we set α = 0 .
5 λ ,and λ = 0 .
For the classifier, the activation function is chosen as sigmoidal , and m = 400 .
Weset N = 20000 , and R = 4000 .
All the parameters of Algorithm 1 are chosen as described in (19).
Figure 4 shows that Algorithm 1 finds an ϵ -stationary point of the strategic classification problem.
We show that Algorithm 1 performs comparably with Averaged Stochastic Approximation with theprojection operator.
Each curve in Figure 4 is an average of 50 repetitions.
Discussion
In this work we provide oracle complexity results for the stochastic conditional gradient algorithmto find an ϵ -stationary point of a constrained nonconvex optimization problem with state-dependent
Markovian data.
In Theorem 3.1, we show that the number of calls to the SFO and LMO requiredby the stochastic conditional gradient-type method in Algorithm 1, with state-dependent Markoviandata, is O ( ϵ − 2 . 5 ) and O ( ϵ − 5 . 5 ) respectively.
To the best of our knowledge, these are the first oraclecomplexity results in this setting.
In Theorem 3.2, we show that SFO and LMO complexity in thecase of state-independent Markovian data is ˜ O ( ϵ − 2 ) and ˜ O
( ϵ − 3 ) respectively, which matches thecorresponding results in the iid setting.
There are various avenues for further extensions.
Establishing lower bounds on the oracle complexityof projection-free algorithms in the Markovian data setting is extremely interesting.
It is alsointriguing to establish upper and lower bounds on the oracle complexity for more general types ofdependent data sequences arising in applications, including ϕ and α mixing sequences.
Yet anotherexciting direction is that of designing algorithms adaptive to the dependency in the data that achievepotentially better oracle complexity bounds.
Checklist
The checklist follows the references.
Please read the checklist guidelines carefully for information onhow to answer these questions.
For each question, change the default [TODO] to [Yes] , [No] , or[N/A] .
You are strongly encouraged to include a justification to your answer , either by referencingthe appropriate section of your paper or providing a brief inline description.
For example:
• Did you include the license to the code and datasets?
[Yes] See Section ?? .
• Did you include the license to the code and datasets?
[No] The code and the data are
proprietary.
• Did you include the license to the code and datasets?
[N/A]
Please do not modify the questions and only use the provided macros for your answers.
Note that the
Checklist section does not count towards the page limit.
In your paper, please delete this instructionsblock and only keep the Checklist section heading above along with the questions/answers below.
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’s
contributions and scope?
[Yes]
(b) Did you describe the limitations of your work?
[Yes]
(c) Did you discuss any potential negative societal impacts of your work?
[N/A]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to
them?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[Yes]
(b) Did you include complete proofs of all theoretical results?
[Yes]
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi-
mental results (either in the supplemental material or as a URL)?
[Yes]
(b) Did you specify all the training details (e.g., data splits, hyperparameters,
how they
were chosen)?
[Yes]
(c) Did you report error bars (e.g., with respect to the random seed after running experi-
ments multiple times)?
[Yes] We report the average of 50 trails
(d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
[No] We did not calculate the exact
timings.
However, our experiments are fairly small-scale ones run on a personal laptop
computer, and our main contributions are theoretical.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes]
(b) Did you mention the license of the assets?
[Yes]
(c) Did you include any new assets either in the supplemental material or as a URL?
[N/A]
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable
[Table]
information or offensive content?
[N/A]
[Table]
If you used crowdsourcing or conducted research with human subjects...
[Table]
(a) Did you include the full text of instructions given to participants and screenshots, if
[Table]
applicable?
[N/A]
[Table]
(b) Did you describe any potential participant risks, with links to Institutional Review
[Table]
Board (IRB) approvals, if applicable?
[N/A]
[Table]
(c) Did you include the estimated hourly wage paid to participants and the total amountspent on participant compensation?
[N/A]
[Table]