SlowGuess/NeurIPS_2023_OCR huggingface dataset:
yzd82@psu.edu
Ilja Kuzborskij
iljak@google.com
We consider the problem of learning a model from multiple heterogeneous sources with the goal of performing well on a new target distribution. The goal of learner is to mix these data sources in a target-distribution aware way and simultaneously minimize the empirical risk on the mixed source. The literature has made some tangible advancements in establishing theory of learning on mixture domain. However, there are still two unsolved problems. Firstly, how to estimate the optimal mixture of sources, given a target domain; Secondly, when there are numerous target domains, how to solve empirical risk minimization (ERM) for each target using possibly unique mixture of data sources in a computationally efficient manner. In this paper we address both problems efficiently and with guarantees. We cast the first problem, mixture weight estimation, as a convex-nonconcave compositional minimax problem, and propose an efficient stochastic algorithm with provable stationarity guarantees. Next, for the second problem, we identify that for certain regimes, solving ERM for each target domain individually can be avoided, and instead parameters for a target optimal model can be viewed as a non-linear function on a space of the mixture coefficients. Building upon this, we show that in the offline setting, a GD-trained overparameterized neural network can provably learn such function to predict the model of target domain instead of solving a designated ERM problem. Finally, we also consider an online setting and propose a label efficient online algorithm, which predicts parameters for new targets given an arbitrary sequence of mixing coefficients, while enjoying regret guarantees.
With a rapidly increasing amount of decentralized data, multiple source domain adaptation has been an important learning scheme in modern machine learning, e.g., in learning with data collected from multiple sources (e.g. crowdsourcing) or learning in distributed systems where the data can be highly heterogeneous such as federated learning. In this learning scenario, given an input space  $\mathcal{X}$  and output space  $\mathcal{Y}$ , we assume access to  $N$  sources of data, each with its own underlying distributions  $\mathcal{D}_j$ ,  $j \in [N]$  over  $\mathcal{X} \times \mathcal{Y}$ . Then, given i.i.d. training samples  $\widehat{\mathcal{D}}_1, \ldots, \widehat{\mathcal{D}}_N$ , and a hypothesis space  $\mathcal{H}$ , our goal is to learn a model on the combination of these sources, for instance through the Empirical Risk Minimization (ERM) procedure  $\widehat{h}_{\alpha} = \arg \min_{h \in \mathcal{H}} \sum_{j=1}^{N} \alpha(j) \mathcal{L}_{\widehat{\mathcal{D}}_j}(h)$ , where  $\mathcal{L}_{\widehat{\mathcal{D}}_j}(h)$  is the empirical loss of a model  $h \in \mathcal{H}$  over data samples in  $\widehat{\mathcal{D}}_j$ , and  $\alpha \in \Delta^N$  is some mixing parameter, such that predictor  $\widehat{h}_{\alpha}$  entails a good generalization performance on a target domain characterized by a distribution  $\mathcal{T}$ , i.e., yielding a small true risk  $\mathcal{L}_{\mathcal{T}}(\widehat{h}_{\alpha}) = \int \ell(\widehat{h}_{\alpha}(\mathbf{x}), y) \mathrm{d}\mathcal{T}(\mathbf{x}, y)$ . It is natural to measure the quality of  $\widehat{h}_{\alpha}$  in terms of the excess risk — namely, the difference between the risk of optimal model for target domain  $h_{\mathcal{T}}^* = \arg \min_{h \in \mathcal{H}} \mathcal{L}_{\mathcal{T}}(h)$ , and that achieved by  $\widehat{h}_{\alpha}$ . Clearly, the performance of  $\widehat{h}_{\alpha}$  will be influenced by several factors, such as the choice of mixing coefficients
$\alpha$  to aggregate the empirical losses, capacity of  $\mathcal{H}$ , and discrepancy between target and source data distributions. So, in order to design a good procedure for learning  $\widehat{h}_{\alpha}$  we need to understand aforementioned trade-offs. Over the years the literature on the multiple source learning has dedicated a considerable attention to this problem [25, 5, 35, 17, 12]. To this end, we consider the following bound on the excess risk of  $\widehat{h}_{\alpha}$ :
Theorem 1 (Multi-source learning bound [17]). Given  $N$  source data distributions  $\mathcal{D}_1, \ldots, \mathcal{D}_N$  and a target data distribution  $\mathcal{T}$ , let  $\widehat{h}_{\alpha} = \arg \min_{h \in \mathcal{H}} \sum_{j=1}^{N} \alpha(j) \mathcal{L}_{\widehat{\mathcal{D}}_j}(h)$  be the ERM solution with fixed mixture weights  $\alpha \in \Delta^N$ . Then for any  $\nu \geq 0$ , with probability at least  $1 - 4e^{-\nu}$  it holds that
$$
\mathcal {L} _ {\mathcal {T}} (\widehat {h} _ {\boldsymbol {\alpha}}) \leq \mathcal {L} _ {\mathcal {T}} (h _ {\mathcal {T}} ^ {*}) + \mathcal {C} (\mathcal {H}, \boldsymbol {\alpha}) + \sup _ {h \in \mathcal {H}} \sum_ {j = 1} ^ {N} \alpha (j) | \mathcal {L} _ {\widehat {\mathcal {T}}} (h) - \mathcal {L} _ {\widehat {\mathcal {D}} _ {j}} (h) | + C \sqrt {\frac {\nu}{2} \sum_ {j = 1} ^ {N} \frac {\alpha^ {2} (j)}{m _ {j}}}
$$
where  $C$  is some constant, the complexity term is  $\mathcal{C}(\mathcal{H},\alpha)\coloneqq \sum_{j = 1}^{N}\alpha (j)\Re_{j}(\mathcal{H})$  with  $\Re_j(\mathcal{H})$  being the Rademacher complexity of  $\mathcal{H}$  w.r.t. data source  $j$ , and  $m_j$  is the number of samples from source  $j$ .
The above bound indicates that, the generalization ability of a model learnt by ERM on an  $\alpha$ -combined sources, depends on the  $\alpha$ -weighted sum of target-source discrepancies, and the number of samples drawn from each source. To entail a good generalization on target domain, it naturally motivates us to minimize right hand side of the bound over  $\alpha \in \Delta^{N}$  to get a good mixture parameter. In this paper we can cast this idea as solving the following minimax optimization problem:
$$
\min  _ {\boldsymbol {\alpha} \in \Delta^ {N}} \max  _ {h \in \mathcal {H}} \sum_ {j = 1} ^ {N} \alpha (j) \left| \mathcal {L} _ {\widehat {\mathcal {T}}} (h) - \mathcal {L} _ {\widehat {\mathcal {D}} _ {j}} (h) \right| + C \sqrt {\sum_ {j = 1} ^ {N} \frac {\alpha^ {2} (j)}{m _ {j}}}, \tag {1}
$$
where we drop the complexity term as it becomes identical for all sources by fixing the hypothesis space  $\mathcal{H}$  and bounding it with a computable distribution-independent quantity such as VC dimension [31], or it can be controlled by choice of  $\mathcal{H}$  or through data-dependent regularization. [17] gave a simple algorithm to minimize the bound of theorem 1 for binary classifiers and 0-1 loss, however their algorithm does not extend to a more general setting. [24] also looked at minimization of a similar bound with the goal to find weights for an optimal mixture, but they did not give a practical algorithm, nor a provable convergence guarantee. However, none of these works aimed to solve (1) because of its complex structure, and so an efficient algorithm for solving (1) so far has not been proposed. In particular, the first difficulty with (1) is that it is a convex-nonconcave objective, which means all minimax algorithms that require inner concavity [23, 22, 27, 28] or PL-condition [30] will fail to converge to a stationary point. However, recently the literature on optimization of this type of objectives has recently made a tangible progress: The first provable convex-nonconcave algorithm was proposed by [34], where they consider alternating gradient descent ascent algorithm. Their algorithm is deterministic, but in practice, we favor a stochastic gradient method. The second difficulty in solving (1) is its compositional structure, which means that simply replacing gradient with stochastic gradient in [34] will not retain convergence guarantees. To tackle these two difficulties, we propose a stochastic corrected gradient descent ascent algorithm, with provable convergence guarantee for solving (1). Our method can be viewed as a variant of the Stochastic Gradient Descent-Ascent (SGDA) algorithm, and moreover here we give a positive answer to the question posed by [34], on whether an algorithm performing simultaneous updates can optimize convex-nonconcave problem?, which could be interesting by its own right.
The discussion above concerns learning with one target domain, but in practice, a more common scenario is that we have multiple target domains to adapt to. For example, in federated learning [26], millions of users might wish to learn a good model from multiple sources, which can have good performance on their own data distribution. Hence, we propose to study Multi-source Multi-target Domain Adaptation scenario (M²DA). Here we assume that we have  $M$  target domains, each of them characterized by its own distribution  $\mathcal{T}_i$ ,  $i \in [M]$  over  $\mathcal{X} \times \mathcal{Y}$ . Adapting to  $M$  different target domains requires different mixture weights  $\alpha_1, \ldots, \alpha_M$ , either obtained by solving (1), or supplied by the user. Equipped with mixing parameters, next we have to solve  $M$  weighted ERM problems to tailor solutions for each target domain  $\mathcal{T}_i$ , that is
$$
\widehat {h} _ {i} = \arg \min  _ {h \in \mathcal {H}} \sum_ {j = 1} ^ {N} \alpha_ {i} (j) \mathcal {L} _ {\widehat {\mathcal {D}} _ {j}} (h), \qquad i \in [ M ].
$$
Notice that these  $M$  ERM objectives share the same component functions, so we name this problem as a co-component empirical risk minimization. A straightforward and naive approach is to solve all  $M$  weighted ERMs individually which becomes computationally inefficient when dealing with a large number of data sources. Nevertheless, given the benign structure of these  $M$  ERM problems, we may inquire whether there is a computationally efficient method for discovering all solutions without the necessity of solving each one individually.
We give an affirmative answer to this question by replacing the learning of the target model by predicting the target model and propose two efficient strategies to learn such predictors (for instance, a neural network). Our algorithm designs are based on the following observation: if we assume that each empirical risk  $\mathcal{L}_{\widehat{\mathcal{D}}_j}(h)$  is strongly convex and smooth in parameters of a hypothesis  $h$  then the optimal parameters are given by a Lipschitz function of mixture weights  $\alpha$ . More formally, denoting  $\mathbf{w}^{*}(\alpha)$  as optimal parameters of a hypothesis  $h$  for  $\alpha$ -weighted ERM,  $\mathbf{w}^{*}(\alpha)$  is Lipschitz in  $\alpha$ . This means that we can learn function  $\mathbf{w}^{*}(\cdot)$  with, say, a neural network and gradient descent, with provably small generalization error. Moreover, analysis of generalization error allows us to understand when such target model prediction is more efficient than direct learning. In particular, we look at such a phase transition of efficiency, and conclude that when the number of targets  $M$  is much larger than number of sources  $N$ , i.e.,  $M \geq \Omega((1/\epsilon)^{N/2})$ , learning to predict solution is more efficient than optimizing to solve all  $M$  ERMs. When  $M$  is relatively smaller, optimizing to solve ERMs is more efficient than learning to predict.
Finally, as a second learning scenario we consider an online learning setting, where mixture weights  $\alpha_{1},\ldots ,\alpha_{M}$  are arriving sequentially, and may not even originate from the same distribution. We cast this problem as an online non-parametric regression problem with inexact labels and propose an label-efficient online algorithm to predict models.
Our contributions The main contributions of this paper is summarized as follows:
- We study the multi-source domain adaptation problem where there are multiple source domains and we wish to learn a new model given the mixture of source domains, that can perform well on a given target domain. We build upon existing learning-theoretic results on multi-source domain adaptation and design a new algorithm for weighing of source domains, that casts this problem as a convex-nonconcave minimax optimization problem. In section 2 we give the first stochastic optimization algorithm for this problem which provably converges to a stationary point. The proposed algorithm is the first provably convergent algorithm for a stochastic compositional convex-nonconcave minimax problem.
- We further consider the above adaptation problem with multiple target domains, the Multi-source Multi-target Domain Adaptation  $(\mathbf{M}^2\mathrm{DA})$ . We observe that these multiple adaptation might share a common structure, which allows us to avoid solving adaptation problem for each target domain individually, and instead we can replace it by direct prediction of parameters for a new problem. We consider offline and online settings for prediction of target parameters, and propose computationally efficient algorithms for both. For the offline setting, in section 3.1 we propose to use a two-layer neural network to learn optimal parameters  $\mathbf{w}^{*}(\alpha)$  using bilevel gradient descent. We show that our algorithm can achieve  $O(n^{-\frac{2}{2 + N}})$  excess risk. We also identify the regime where our learning based approach is more efficient compared to directly solving each target problem individually.
- Finally, in section 3.2 we focus on scenario where target problems arrive sequentially (and could be dependent) and extend our study of direct target parameter prediction to the online setting. We propose a label-efficient algorithm which enjoys  $O\left(n^{-\frac{1}{1 + N}}\right)$  average regret.
Notation We introduce some basic definitions and notation that will be used throughout the paper. Let  $\mathbb{B}_q^d (r)$  be the ball in  $q$ -metric centered at the origin and of radius  $r > 0$ , and let  $\mathbb{S}^{d - 1}(r) = \{\mathbf{x}\in \mathbb{R}^d:\| \mathbf{x}\| _2 = r\} \subset \mathbb{R}^d$  be the  $\ell_2$ -norm unit sphere centered at the origin, and finally let  $\mathbb{S}^{d - 1} = \mathbb{S}^{d - 1}(1)$ . In addition, the probability simplex is defined as  $\Delta^{N} = \{\alpha \in [0,1]^{N}:\| \alpha \|_{1} = 1\}$ . Concatenation of vectors is denoted by parentheses, that is  $(\mathbf{w}_1,\ldots ,\mathbf{w}_m) = [\mathbf{w}_1^\top ,\ldots ,\mathbf{w}_m^\top ]^\top$ . A vector norm  $\| \cdot \|$  is understood as Euclidean norm, while  $\| \mathbf{x}\|_{\infty} = \max_i|x_i|$ . For a matrix  $\mathbf{M}$ ,  $\| \mathbf{M}\|_{\mathrm{op}}$  denotes its spectral norm while  $\| \mathbf{M}\| _F$  is its Frobenius norm. For some  $f:\mathbb{S}^{d - 1}\to \mathbb{R}$  the empirical semi-norm is defined as  $\| f\| _n^2 = \frac{1}{n} (f(\mathbf{x}_1)^2 +\dots +f(\mathbf{x}_n)^2)$  and is always taken
w.r.t. the training sample  $S$ . In addition, for  $g:\mathbb{S}^{d - 1}\to \mathbb{R}$ , we define an empirical inner product  $\langle f,g\rangle_{n} = \frac{1}{n} (f(\mathbf{x}_{1})g(\mathbf{x}_{1}) + \dots +f(\mathbf{x}_{n})g(\mathbf{x}_{n}))$ . At the same time,  $\| f\| _2 = \| f\|_{L^2 (P_X)}^2$ .
2 Mixture Weights Estimation via Convex-nonconcave Minimax Optimization
In this section we focus on a single target domain and present an Algorithm 1 designed to solve a minimax problem (1) to estimate the mixture weights. We assume that hypothesis  $h$  is parameterized by a vector space  $\{\mathbf{w} \in \mathcal{W} \subseteq \mathbb{R}^d\}$ , and use  $f_j(\mathbf{w}) = \mathcal{L}_{\widehat{\mathcal{D}}_j}(h)$  to denote the empirical risk over data source  $j$ . Similarly, we define  $f_{\widehat{\mathcal{T}}}(\mathbf{w}) = \mathcal{L}_{\widehat{\mathcal{T}}}(\boldsymbol{h})$ . We do the following standard relaxations. First, for the sake of simplicity in computation, we relax the square root on the quadratic term w.r.t.  $\alpha$ . Second, since the absolute value function is non-smooth, we shall use the smooth approximation function  $g$  to replace it, e.g.,  $g(x) = \sqrt{x^2 + c}$  where  $c$  is some small number (here  $g(\cdot)$  is smooth approximation of  $|\cdot|$ ). These relaxations lead to solving the following compositional convex-nonconcave minimax optimization problem:
$$
\min  _ {\alpha \in \Delta^ {N}} \max  _ {\mathbf {w} \in \mathcal {W}} F (\boldsymbol {\alpha}, \mathbf {w}) := \sum_ {j = 1} ^ {N} \alpha (j) g \left(f _ {\widehat {\mathcal {T}}} (\mathbf {w}) - f _ {j} (\mathbf {w})\right) + C \boldsymbol {\alpha} ^ {\top} \mathbf {M} \boldsymbol {\alpha}, \tag {2}
$$
where  $\mathbf{M} = \mathrm{diag}\{\frac{1}{m_1},\dots ,\frac{1}{m_N}\}$ . We are interested in developing a stochastic optimization algorithm to solve (2). It is a strongly-convex-nonconcave minimax problem, and it is one of most difficult type of minimax problem due to the absence of inner concavity. To the best of our knowledge, only Xu et
Algorithm 1: Mixture Weight Estimation
Input: Target domain  $\mathcal{T}$  Source domains  $\mathcal{D}_1,\dots,\mathcal{D}_N$  Initialization variable  $\mathbf{w}^0 = \mathbf{w}^{-1}$ $z_1^0,\ldots ,z_N^0$  Positive hyper-parameters  $(B,\beta ,\eta ,\gamma)$  (see theorem 2).   
for  $t = 0,\dots,T - 1$  do Sample a minibatch  $\xi_T^t$  of size  $B$  from target domain  $\mathcal{T}$  and  $\xi_1^t,\dots,\xi_N^t$  from source domains  $\mathcal{D}_1,\dots,\mathcal{D}_N$ $z_{j}^{t + 1} = (1 - \beta^{t})\left(z_{j}^{t} + f_{\widehat{T}}(\mathbf{w}^{t};\xi_{T}^{t}) - f_{j}(\mathbf{w}^{t};\xi_{j}^{t}) - (f_{\widehat{T}}(\mathbf{w}^{t - 1};\xi_{T}^{t}) - f_{j}(\mathbf{w}^{t - 1};\xi_{j}^{t}))\right.$ $+\beta^{t}(f_{\widehat{T}}(\mathbf{w}^{t};\xi_{T}^{t}) - f_{j}(\mathbf{w}^{t};\xi_{j}^{t}))$  Compute gradient for w:  $\mathbf{g}_{\mathbf{w}}^{t} = \sum_{j = 1}^{N}\alpha_{j}^{t}\nabla g(z_{j}^{t + 1})(\nabla f_{\widehat{T}}(\mathbf{w}^{t};\xi_{T}^{t}) - \nabla f_{j}(\mathbf{w}^{t};\xi_{j}^{t}))$ $\mathbf{w}^{t + 1} = \mathcal{P}_{\mathcal{W}}(\mathbf{w}^t +\gamma \mathbf{g}_{\mathbf{w}}^t)$  Make vector  $\mathbf{v}\in \mathbb{R}^N$  whose jth coordinate is  $g(z_j^t)$  Compute gradient for  $\alpha$  ..  $\mathbf{g}_{\alpha}^{t} = \mathbf{v} + 2C\mathbf{M}\boldsymbol{\alpha}^{t}$ $\pmb{\alpha}^{t + 1} = \mathcal{P}_{\Delta^N}\left(\pmb {\alpha}^t -\eta \pmb {g}_{\pmb{\alpha}}^t\right)$
al. [34] proposed a deterministic algorithm to solve it, but it is still unknown whether a stochastic algorithm can solve it with provable guarantee. We give an affirmative answer to this question, by proposing an algorithm built on celebrated stochastic gradient descent ascent [23]. In addition to nonconcavity nature of 2, another difficulty that arises from the compositional structure of objective is that we cannot simply compute stochastic gradients, namely (with  $\mathbb{E}[\cdot ]\equiv \mathbb{E}[\cdot \mid \mathrm{data}]$ ):
$\mathbb{E}[g'(f_{\widehat{\tau}}(\mathbf{w}; \xi_{\mathcal{T}}) - f_j(\mathbf{w}; \xi_j)) (\nabla f_{\widehat{\tau}}(\mathbf{w}; \xi_{\mathcal{T}}) - \nabla f_j(\mathbf{w}; \xi_j))] \neq g'(f_{\widehat{\tau}}(\mathbf{w}) - f_j(\mathbf{w})) (\nabla f_{\widehat{\tau}}(\mathbf{w}) - \nabla f_j(\mathbf{w})),$  where  $\xi_1, \xi_2, \ldots$  are independent random elements in sample space  $\Xi = \mathcal{X} \times \mathcal{Y}$  that capture stochasticity of the algorithm. To alleviate this issue, we borrow 'the stochastic corrected gradient' idea from [7], and maintain an auxiliary variable by introducing
$$
\begin{array}{l} z _ {j} ^ {t + 1} = \left(1 - \beta^ {t}\right) \left(z _ {j} ^ {t} + f _ {\widehat {\mathcal {T}}} \left(\mathbf {w} ^ {t}; \xi_ {\mathcal {T}} ^ {t}\right) - f _ {j} \left(\mathbf {w} ^ {t}; \xi_ {j} ^ {t}\right) - \left(f _ {\widehat {\mathcal {T}}} \left(\mathbf {w} ^ {t - 1}; \xi_ {\mathcal {T}} ^ {t}\right) - f _ {j} \left(\mathbf {w} ^ {t - 1}; \xi_ {j} ^ {t}\right)\right)\right) \\ + \beta^ {t} \left(f _ {\hat {\mathcal {T}}} \left(\mathbf {w} ^ {t}; \xi_ {\mathcal {T}} ^ {t}\right) - f _ {j} \left(\mathbf {w} ^ {t}; \xi_ {j} ^ {t}\right)\right) \\ \end{array}
$$
In words, for each source  $j$ , we maintain a variable  $z_{j}^{t}$  serving as a correction term, to ensure that  $z_{j}^{t}$  is getting closer to inner function component  $f_{\widehat{\mathcal{T}}}(\mathbf{w}^t) - f_j(\mathbf{w}^t)$ . Relying on the auxiliary variable, we then have gradient estimates
$$
\mathbf {g} _ {\mathbf {w}} ^ {t} = \sum_ {j = 1} ^ {N} \alpha^ {t} (j) \nabla g (z _ {j} ^ {t + 1}) (\nabla f _ {\widehat {\mathcal {T}}} (\mathbf {w} ^ {t}; \xi_ {\mathcal {T}} ^ {t}) - \nabla f _ {j} (\mathbf {w} ^ {t}; \xi_ {j} ^ {t})), \quad \mathbf {g} _ {\alpha} ^ {t} = [ g (z _ {1} ^ {t}), \dots , g (z _ {N} ^ {t}) ] + 2 C \mathbf {M} \boldsymbol {\alpha} ^ {t}.
$$
Then, denoting the projection operator onto convex set  $\mathcal{C}$  by  $\mathcal{P}_{\mathcal{C}}(\cdot)$ , the update rule becomes
$$
\mathbf {w} ^ {t + 1} = \mathcal {P} _ {\mathcal {W}} \left(\mathbf {w} ^ {t} + \gamma \mathbf {g} _ {w} ^ {t}\right), \qquad \boldsymbol {\alpha} ^ {t + 1} = \mathcal {P} _ {\Delta^ {N}} \left(\boldsymbol {\alpha} ^ {t} - \eta \mathbf {g} _ {\alpha} ^ {t}\right).
$$
2.1 Convergence Analysis
In this section we are going to present the convergence guarantee for Algorithm 1. We make the following standard assumption on objective in (2).
Assumption 1. We make the following assumptions on  $g$  and  $f$ :
1.  $g(z)$  is  $G_{g}$  Lipschitz and  $L_{g}$  smooth.  $f_{j}(\mathbf{w};\xi)$  is  $G_{f}$  Lipschitz and  $L_{f}$  smooth,  $\forall \mathbf{w} \in \mathcal{W}, j \in [N], \xi \in \Xi$ .
2.  $\mathbb{E}\left\| \nabla f_j(\mathbf{w};\xi) - \nabla f_j(\mathbf{w})\right\|^2 \leq \sigma^2, \forall \mathbf{w} \in \mathcal{W}$ .
3.  $\max_{\mathbf{\alpha} \in \Delta^{N}, \mathbf{w} \in \mathcal{W}} F(\mathbf{\alpha}, \mathbf{w}) \leq F_{\max}$ ,  $\max_{\mathbf{w} \in \mathcal{W}} g(f_{\widehat{\mathcal{T}}}(\mathbf{w}) - f_{j}(\mathbf{w})) \leq B_{g}, \forall j \in [N]$ .
Points 1 and 2 of Assumption 1 are standard in the literature on compositional optimization [7]. Point 3 guarantees boundedness of objective value, which can be ensured since we are working in the bounded parameter domain. Assumption 1 also implies the following property of  $F$ .
Proposition 1. Under Assumption 1,  $F(\alpha, \mathbf{w})$  is  $L \coloneqq \max \left\{4G_f^2 L_g + 2G_gL_f, \frac{2C}{m_{\min}}\right\}$  smooth, and  $\mu = \frac{2C}{m_{\max}}$  strongly convex in  $\alpha$ .
Next, we consider the following convergence measure:
Definition 1 (Convergence Measure [34]). Given two parameters,  $\alpha$  and  $\mathbf{w}$ , we define the following quantity as a stationary gap
$$
\nabla G (\boldsymbol {\alpha}, \mathbf {w}) = \left( \begin{array}{c} \frac {1}{\eta} \left(\boldsymbol {\alpha} - \mathcal {P} _ {\Delta^ {N}} \left(\boldsymbol {\alpha} - \eta \nabla_ {\boldsymbol {\alpha}} F (\boldsymbol {\alpha}, \mathbf {w})\right)\right) \\ \frac {1}{\gamma} \left(\mathbf {w} - \mathcal {P} _ {\mathcal {W}} \left(\mathbf {w} + \gamma \nabla_ {\mathbf {w}} F (\boldsymbol {\alpha}, \mathbf {w})\right)\right) \end{array} \right)  .
$$
Given the nonconcave nature of (2), we are only able to show the convergence to a stationary point. Definition 1 measures the stationarity given parameter pair  $(\alpha, \mathbf{w})$  by examining how much the parameter will change if we run one step projected gradient descent-ascent on them. Alternatively, one could consider the widely employed primal function [23] as a convergence measure,  $\| \nabla \Phi (\pmb{\alpha})\|$  with  $\Phi (\pmb{\alpha}) = \max_{\mathbf{w}\in \mathcal{W}}F(\pmb{\alpha},\mathbf{w})$ , but it is ill-suited to express stationarity since  $F(\pmb{\alpha},\cdot)$  is non-concave.
One of our main results, proved in appendix A, establishes the convergence rate of Algorithm 1:
Theorem 2. Consider Assumption 1 and let  $L$  and  $\mu$  be defined in Proposition 1. Then, letting  $B = \Theta\left(\max\left\{\frac{G_g^2N\sigma^2}{\epsilon^2},\frac{\kappa L\sigma^2}{\epsilon^2}\right\}\right)$ ,  $\beta = 0.1$ ,  $\eta = \Theta\left(\frac{\mu}{L^2}\right)$ ,  $\gamma = \Theta\left(\frac{\mu^3}{NG_g^2G_f^2L^2}\right)$ , the Algorithm 1 guarantees that
$$
\frac {1}{T} \sum_ {t = 1} ^ {T} \mathbb {E} \left\| \nabla G (\pmb {\alpha} ^ {t}, \mathbf {w} ^ {t}) \right\| ^ {2} \leq \epsilon^ {2}
$$
with the gradient complexity bounded by: