Riemannian Federated Learning via Averaging Gradient Streams
Authors: Zhenwei Huang
Wen Huang
Pratik Jawanpuria
Bamdev Mishra
Venue: ICLR 2026 Poster (2026)
Keywords: Riemannian federated learning
Averaging gradient streams
Partial participation
Heterogeneity data
Riemannian distributed optimization
Primary area: optimization
TL;DR: This paper presents and analyzes a federated learning algorithm that can handle generic manifold constraints, partial participation, and data heterogeneity.

Abstract: Federated learning (FL) as a distributed learning paradigm has a significant advantage in addressing large-scale machine learning tasks. 
In the Euclidean setting, FL algorithms have been extensively studied with both theoretical and empirical success. However, there exist few works that investigate federated learning algorithms in the Riemannian setting. In particular, critical challenges such as partial participation and data heterogeneity among agents are not explored in the Riemannian federated setting. This paper presents and analyzes a Riemannian FL algorithm, called RFedAGS, based on a new efficient server aggregation---averaging gradient streams, which can simultaneously handle partial participation and data heterogeneity. We theoretically show that the proposed RFedAGS has global convergence and sublinear convergence rate under decaying step sizes cases; and converges sublinearly/linearly to a neighborhood of a stationary point/solution under fixed step sizes cases.  These analyses are based on a vital and non-trivial assumption induced by partial participation, which is shown to hold with high probability. Extensive experiments conducted on synthetic and real-world data demonstrate the good performance of RFedAGS.

Published as a conference paper at ICLR 2026
RIEMANNIAN FEDERATED LEARNING VIA AVERAGING GRADIENT STREAMS
Zhenwei Huang', Wen Huang'; Pratik Jawanpuria", Bamdev Mishra²
1 Xiamen University, zwhhuang@stu xmu edw.eu.ccoo. en, wen. huangêxmi edu. cn
2 Indian Institute of Technology Bombay, pratik. jawanpuria@iiiieo in 3 Microsoft India, bandevm@mi wttp:/mmmmmcmoooo...mmmommmm
ABSTRACT
Federated learning (FL) as distributed learning paradigm has significant advan- tage addressing large-scale machine learning tasks. the Fuclidean setting, FI. algorithms have been extensively studied with both theoretical and empirical suc- cess. However, there exist few works that investigate federated learning algorithms in the Riemannian setting. In particular, critical challenges such as partial partici- pation and data heterogeneity among agents are not explored in the Riemannian federated setting. This paper presents and analyzes Riemannian FL algorithm, called RFedAGS, based on a new efficient server aggregation averaging gradient streams, which can simultaneously handle partial participation and data heterogene- ity. We theoretically show that the proposed RFedAGS has global convergence and sublinear convergence rate under decaying step sizes cases; and converges sublinearly/lineate to neighborhood of stationary point/solution under fixed step sizes cases. These analyses are based on vital and non-trivial assumption induced by partial participation, which is shown to hold with high probability. Extensive experiments conducted on synthetic and real-world data demonstrate the good performance of RFedAGS.
INTRODUCTION
Modern learning tasks handle massive amounts of data, which are geographically distributed across heterogeneous devices. Conventional centralized algorithms, e.g., stochastic gradient descent (SGD), need collect the data into single device for training, which consumes significant storage and com- puting resource. Additionally, from the perspective of privacy security, transmitting raw training data may leak data privacy. A promising distributed learning paradigm paradigm-federated federated learning (FL) (FL)-allows allows
a center server to coordinate with multiple agents (e.g., mobile phones and tablets) to train a desired model parameter without raw data sharing, which is ideal solution the issues aforementioned.
In recent years, with the development of Riemannian optimization, many machine learning problems have data structures that can be inscribed by low-dimensional smooth manifolds, and thus they can be modeled on manifolds. There are such examples including but not limited to principal component analysis Ye & Zhang 2021), Fréchet mean computation (Han et al. 2021 2023a), hyperbolic structured prediction (Xiong et al.) (2022). low-rank matrix/tensor learning (Jawanpuria & Mishra 2018: Nimishakavi et al.) 2018 Mishra et al. 2019). natural language processing (Jawanpuria et al.
2019a 2020), domain adaptation (Han et al. 2022: 2024a Jawanpuria et al. 2025). and neural network training (Magai 2023 Han et 2023b). This motives us to develop efficientt Riemannian FL algorithm.
This paper focuses on the following Riemannian federated optimization problem
N
1 N
arg min F(x)
Σ M(x), with fi(x) [,
(1.1)
=EM
where M is d-dimensional Riemannian manifold, N is the number of agents, F M R the global objective, and fi M + IR and D; are local objectives and the data distribution held by agent
"Corresoonding author: wen. huanguunnne..aa...aoo xmu edu.

---

Published as a conference paper at ICLR 2026
Vi E [N] = 21, NJ. Throughout this paper, focus the expected minimization but the resulting conclusions are also true for the finite sum minimization in which the local objective is defined by fi(x)= = N. Σ (1; ze.s) with Dj {zz,1, 22,2: ..... N, J the local dataset held by agent i. We may not necessarily assume that D,, Vi [N], are the independently identical distribution (I..DD.) i.e. the data distributions across different agents are independent but not identical (non-I.I.D).
A well-known Euclidean FL. algorithm is Federated Averaging (FedAvg) (McMahan et al., (017). which is adapted from the local stochastic gradient descent (local SGD) method. Specifically, at the beginning, FedAvg takes an initial guess 2 as input and then sends it to all agents. Subsequently, the following steps are performed alternately:
agent j updates its the local parameter via performing K-step SGD with 2 being the initial guess and generates the trained local parameter xi,x (this is called "local update" or "inner iteration"), and then the local parameter =²,xx is uploaded to the server,
(ii) the server at random samples a subset of size S from all agents, denoted by S,, and then averages the received local parameters to generate the next global parameter 11+1, i.e.,
1 Σ x
J++1
3
(1.2)
JES.
which is called "server aggregation", and then sends 2141 to all agents.
The two steps above constitute round of communication (or outer iteration).
Related works. Early works primarily analyzed the convergence of FedAvg and its variants in
limited settings, typically relying
or both of the following assumptions: (i) full participation
(i.c.. N) and (ii) I.I.D. data distributions: sec, c.g., (Zhou Congl 2018 Stich 2019; Yu
ct al. 2019 Haddadpour et al. 2019: Wang & Joshi 2021 Gu ct al.
2023) and references therein.
Subsequently, numerous works have studied the convergence of FL algorithms under (iii) partial
participation and (iv) non-LLDD data assumption; see e.g., (Li et al.) 2020b a Rizk et al. 2022) and references therein. In these works, partial participation is implemented by random sampling- server randomly selects subset of agents perform local updates in each outer iteration.
Due to heterogeneity in the computational capabilities and the environment conditions across agents, their availability and response speeds are hardly predictable. This unpredictability makes random sampling canpling-based approaches unsuitable for such scenarios. Recent works have instead adopted an arbitrary participation model, where agents may respond to the server in stochastic and uncontrolled
manner (Gu et al.) [021] Wang & Ji. 2022, Ribero et al. 2023) Xiang et al., 2023 Yan et al.] 2023
Wang & Ji] Ji][[222] 2024] Xiang al.] 2025| Ying et | 2025). These works can be roughly divided into
three categories: (i) time-varying statistic, i.e. agent i participates in the I-th outer iteration with
probability pi varying over time (Wang& & Ji 2022 Ribero et al. 2023 Xiang et al. 2023 Wang & Ji 2024; Xiang et al., (205); ; (ii) time-invariant statistic. i.e., the participation probability for agent is not varying over time (meaning pi = p, for all t 2 1) (Wang & Ji. 2024 Ying et al. 2025): and
(iii) periodic participation, i.e., each agent must participate in at least one communication round
within a fixed iteration interval (Gu et al. 2021 Yan et al. 2023).
The FL algorithms mentioned earlier operate solely in Euclidean space and thus cannot directly handle such problems whose parameters are located in manifolds due to the inherent curvature effects of manifolds. Only a limited number of studies have explored the design and analysis of FL algorithms on Riemannian manifolds. (Li Ma (203)) proposed a Riemannian counterpart of (1.) and thus developed a Riemannian FL algorithm. Their algorithm involves in exponential mapping. its inverse, and parallel transport. Nevertheless, for some manifolds, e.g., the Stiefel manifold, the inverse of the exponential mapping and parallel transport have no closed forms, and only iterative methods can be used to compute them, which brings an extra computation burden. (Huang et al.) 2024) adopted a framework similar to that of (Li & Ma 2023) but integrate differential privacy to strengthen privacy guarantees. Under the non-i.I.D, setting, most convergence results in (Li & Ma 2023 Huang et al. 2024) are established for the case K = and full participation, i.e., all agents just perform one step local update (notably, for K > 1, the convergence analyses of both algorithms further assume that only one agent participates in communication). The algorithm proposed in (Zhang
al. 2024) supports general settings where K > 1 and S > 1, but its convergence analysis relies on
2

---

Published as a conference paper at ICLR 2026
the full participation assumption. Additionally, the algorithm therein involves orthogonal projector onto the manifold and requires that this projector is a singleton. Thus, its applicability is restricted to problems compact Riemannian submanifolds embedded in Euclidean spaces. Subsequently, Wang et al. (Wang et al. 2025) proposed a zeroth-order gradient estimator and integrate it into ReddProj resulting zeroth-order Rimannian FI. algorithm called Feedro. The algorithms in (Xiao et al. 2024 2025) incorporated the Barzilai-Borwein method into the framework of (Li Ma 2023). Despite the efforts of some, all of the Riemannian FL algorithms above have no theoretical guarantee under both partial participation and data heterogeneity setting. See Table [ for comprehensive comparisons of existing Riemannian FL algorithms and the proposed RFedAGS. Table 2 summarizes the computational (communication) complexity required for these methods to complete one outer iteration. The table includes the local iteration complexity per agent (LICpA), server computational complexity (SCC), communication complexity (CC), and total computational complexity (TCC), where TCC- LICpA + SCC.
Table 1: Summary of existing algorithms and the proposed RFedAGS.
Algorithms
Manifold
Partal Participation
Non-LLD.
Retraction
Vector transport
RFedSVRG
General
Conditioned Conditioned
Exponential mapping
Parallel transport
RPriFed Hahing el
2024
General
x
Exponcntial mapping
Parallel transport NiA
REedPro Ziing
Compact submanifold
NIA
20-RFddPmn Pny Wang ef
21125
Compact suhmanifold
NIA
RFedSVRG. - Ailo DF 2424
General
X²
Conditioned
Exponencial mapping
Parallel transport
RFed5VRG-BB Xiao
General
X²
Conditioned
Exponential mapping
Parallel transport
RFedAGS (this paper)
General
General retraction
Bounded
Although these methods are suiablee for peneral manifolds, due to the usago exponentia mapping and lit inverse, they may work in some manifold where the closed. form expressions, for example, the Suelel manifokd. global and is used by agents to perform local SVRG step. Hence, these
exponential muppings
"hese algarithes each
gradient algorihhms ure not suitblle for puntial purtkipatioon
conpute
We highlight that these mothods overooe the non-1 L.D. data challenge only when = and N. iie... agonts perform one-step local updltte. For K cases, the LLD. and assumptioos are indispensable. Hence, these alganthms are suitable for the non L.D.D data setting conditioned on - and N..
Table 2: The computational complexity of RFedAvg (Li & Ma 2023). RFedSVRG LL & Ma 2023
Reddrroj (Zhang et al. 2024), and RFedAGS over a compact Riemannian submanifold embedded in Rdxp. Here N is the number of agents, K is the number of local iterations, B is the batch size, S is the number of local samples, and r, ir, v, P. and g respectively denote the flops in retraction evaluation, an inverse evaluation of the retraction, a vector transport evaluation, a projection evaluation onto the manifold, and a gradient evaluation of single sample loss fi 21,5).
LICPA
SCC
CC
TCC
RFedAvg
rK + RBK dpk
(Ir + do; +r
2dpN
(Ir + dp) r(K + 1) RBK dpR
RFedSVRG
*k + vk RBK RS 30pR
(Ir + 2dp) N +
4dpN
(Ir 4 24p0 r(K + 1) vK +R(BK 8) 3d
RFedPojj
PiK + 2) gBK du(4K +3)
P dp(N 4-2)
26pN
p(K + 3) gBK 4p(4KC + +
RFedAGS
*KC v(K 1) gRK + 2dpR
dpW
2dpN
v(K 41 v(K 1) gBK dy(2K + N)
Here we assume that all agents participute communication.
Challenges. In this paper, we focus on investigating an FL algorithm on general Riemannian manifolds, which works under an arbitrary participation and data heterogeneity setting. In that case, the challenges of designing and analyzing such an algorithm arise mainly from (i) the curvature effects of manifolds, (ii) multiple-step local updates at each agent, (iii) stochastic error of arbitrary participation, and (iv) data heterogeneity across agents. The biggest challenge brought about by (i) and (iii) is how the server generates new global parameters based on the local update information from multiple agents, which directly affects the design of the algorithm. However, (ii) and (iv) bring local errors into the global parameter and even make algorithms diverge, which is called agent drift effects. These issues often combine and make the convergence analysis more complicated.
Contributions. The main contributions of this paper are summarized as follows.
1. The server aggregation (SA) proposed in (Li & Ma 2023) is inspired by the Euclidean weighted average (.2). Although this SA is feasible in practice, it has significant challenges in terms of theory analysis and computation efficiency. This paper present new SA which can avoid the issues mentioned above. The idea behind the presented SA is that it does not handle local parameters but rather averages local gradient information, which retains linearity to some extent.
2. We investigate the availability of the proposed RFedAGS under arbitrary participation and non- I.I.D. data, where the arbitrary participation setting is based on the time-invariant statistic model
3

---

Published as a conference paper at ICLR 2026
without requiring prior knowledge of the participation probabilities. This model encompasses many practical scenarios, including random sampling.
We establish the convergence guarantees of the proposed RFedAGS under the arbitrary participa- tion and non-I.I.D data setting with the standard assumptions in FL and Riemannian optimization except Assumption 3.8 which is important and nontrivial. We also discuss the reasonability of this assumption when using the frequencies to estimate the true probabilities.
4. Extensive numerical experiments with synthetic/real-world data are conducted to demonstrate the efficacy of the proposed RFedAGS.
Notations. Throughout this paper, we use IR, IK,, and R to denote the real numbers, the space real vectors of dimension n, and the space real matrices of size m n. respectively. We use M to denote the Riemannian manifold and the equipped Riemannian metric is denoted by (s, :), whose induced norm on the tangent space T.M is denoted by ||~ (omitting Exp. R, T, and gradf denote exponential mapping, retraction, vector transport, and the gradient of :.M + R, respectively. Also, (). || ||e. and f denote the Euclidean inner product, the norm induced by the Euclidean inner product, and the Euclidean gradient of f.
subscript sometimes).
2 RFEDAGS: RIEMANNIAN FEDERATED AVERAGING GRADIENT STREAMS
A basic background in Riemannian geometry and optimization is assumed, and the details can be found in Appendix
The proposed RFedAGS (stated in Algorithm T are explained as follows.
Algorithm Riemannian Federated Learning via Averaging Gradient Streams: RFedAGS
Input: Initial global model æ1 E M, number of aggregations T, numbers of local iterations K, local step size sequence {a, global step size w, batch size sequence (B,Y==i
Output: {xx)]=+
1: for 1. =
2: The server broadcasts x, to all agents, i.e., :1,0 -- It. EN:
3: for Agent j E / in parallel do
4: Set </ 0z,i
5:
for k = 0,1..... K - 1-10 do
6:
Agent j finds indices of the mini-batch sample By by sampling B, times;
7:
Set Th, + B, Ebeo, gradf, ( Set Set 1 +- RRy, (-arn),))
8:
9:
Set 61,411 t G2, + T (amrle) with Vh satisfying R2g, (7h.)) = xei
end for
11:
Upload the gradient stream G. to the server with an unknown but fixed probability PA;
12: end for
13: The server computes the approximate probability o. Vj E S;;
The server updates the new global model r++1 by AAGSAPP) with &f replacing Pý;
15:
end for
A new Riemannian SA. Due to the curvature effects of manifolds, the addition of two points in manifold is not valid, and thus the SA via the weighted average of local parameters [1.2 does not work in the Riemannian setting. Li & Ma (2023) proposed a SA, called tangent mean, defined
Je+1
Exp..
Exp eeSt
(xi,, (x),k)
(TM)
which is an approximate to the weighted average of points on a manifold. On the one hand, (TM involves the inverse of exponential mapping, which has closed-form expression in some manifolds. e.g., the Stiefel manifold. This limits its scope of availability. Additionally, due to the curvature effects of manifolds, exponential mapping and its inverse almost are nonlinear. Hence, when agents perform multiple-step local updates, (TM) involves multiple consecutive exponential mappings, resulting in that the increment of parameters, Expz. (I++1), is difficult to be bounded in analysis, which makes convergence analysis fairly challenging. In view of the discussions above, this paper
4

---

Published as a conference paper at ICLR 2026
resorts to another aggregation which can not only implement SA efficiently but also analyze algorithm convergence conveniently.
Back to the Euclidean setting, the increment of parameters of FedAvg can be expanded as
K-1
Σ K= BEB
1441
B
FES
Observing the expression shows that the increment of parameters is given by the average of mini- batch gradients of active agents. We can adopt the similar idea in the Riemannian setting but require making some adaptations, since directly combining the mini-batch gradients located in different tangent spaces is not well defined. With the aid of vector transport, the combination can be defined. Specifically, we define the the Riemannian "increment of parameters"
G Ru, (I++1) cxe
Σ bEB
T (grad Ki,
Be
JESI
k=
Specific to agent j. it just need to upload Gi,K at called gradient stream, the server. The resulting new SA is given via averaging gradient streams
E
B. EbER, Tap (grad (* St S.,,,))).
Ee+1 R2, (Ct) = Pare
(AGS-RS)
S.
JES
is worth noting that when the manifold reduces to a Euclidean space, (AGS-RS) is equivalent to the Euclidean SA (1.2). In opinion, this aggregation more essential generalization from the Euclidean setting to the Riemannian setting.
From the perspective of geometry, tangent mean TM "projects" the final inner iterates 2/4K back to the tangent space Zt, then averages them and finally retracts the average into the manifold. While in aggregation (AGS-RS), the intermediary negative mini-batch-gradienss -E, Ebes,, gradfi (,,,,)) are transported to the tangent space at æ; in some way, then aver- ages them and finally retracts the average into the manifold. The (TM) actually is an approximation of the weighted averages of inner iterates x When the degree of heterogeneity across clients are large, the inner æ²| is closer to the minimizer of local function fj. and their average may be far away from the minimizer of the global function; while, the proposed (AGS-RS) leverages the gradient information drawn from clients to generate global direction and thus helps to alleviate this bias; see Figure I(c) In particular, letting the proposed aggregation (AGS-RS) use the exponential map and parallel transport, the two aggregations coincide when (i) M T or (ii) K 1. See Figure | for
geometric interpretation and an experimental comparison of TM) and AGS-RS).
10
L RFad405
en RFodhig
puvia
100
Ple)
12, 00.
Ierosioos
(a)
TM
(b) AGS-RE
(c) (AGS) v.s. (TM)
Figure 1: (a)-(b) diagrams of TM) and (AGS-RC where 2, two agent participate in commu- nication, and (x(x) denotes the local stochastic gradient of agent i at x. (c) (AGS) v.s. (TM) on
mingc(eER*0: F(x) = ( E (TT 2F T 72 k2)
Arbitrary partial participation. Now we are ready extend (AGS-RS) to the arbitrary partial participation setting under consideration, which is formally modeled in Assumption 2.1
It should be highlighted that vector transport used in AGS-RS) (subsequent need inverse of retraction; see Line 9 Algorithm
AGS-AP usually does not
5

---

Published as a conference paper at ICLR 2026
Assumption 2.1. Assume that each agent i independently participates in any round of communication with probability ps > O.
Under Assumption 2.1 when the participation probabilities are not exactly equal to each other, using (AGS-RS) simply may introduce stochastic participation errors. In that case, the next theorem points out that the algorithm equipped with (AGS-RS) may work incorrectly since it may solve another problem different from the original problem.
Theorem 2.1 (Proved in Appendix
let St + 0 denotes the set of agents L who respond to the server at the t-th round of communication. Then, E Ejes, T.. 1rg
E.I Under Assumption 2.1
Gi=PPgrrad (((), with pú = pi
=p Jo [1 - + pyT)dT.
Therefore, if ps # Py for some i,j E [N], then çt # Pj. and thus there exists no X > such
that E Pigradf((x) = xgradf(x). That is, the algorithm may not solve the original problem mingeM F(x) since each of its search directions leads the iterate iy to the minimizer of another problem minzeM F(x) = Ex= (zf((z).
Back again to Assumption 2.1
at the t-th round of communication, note that
Σ LiEs
1 P.N
(x))
E
P.N Ls, (1grad/:(x)
1
E [1s
P.N
P.N (pigradf((z)) gradF(x))
(2.1)
where Is. (1) = i St otherwise (t) = O. Hence, if the participation probabilities, p. 's, are known, one of the feasible aggregation patterns can be chosen as
F4+1 <- Ro:
P.N si.x) with x > the global step size,
(AGS-AP)
FES
which ensures that the algorithm correctly solves the original problem minge. F(x).
On the other hand, in practical applications, the server is actually unaware of the true probabilities. In this case, what the server can do is to estimate the true probabilities as possible in some ways, that is. the server computes of in the (-th round of communication and uses it to serve as the true probability p:. Summarizing above, this paper proposes a Riemannian FL algorithm, called RFedAGS, which can address the partial participation setting, as stated in Algorithm] |
3 CONVERGENCE ANALYSIS
In this section, we establish the convergence properties of RFedAGS (Algorithm I on the partial participation and the non-I.L.D. data settings. All of the proofs can be found in Appendix D
3.1 ASSUMPTIONS
We first present a set of assumptions follows that necessary for the convergence analysis. All assumptions except Assumption 3.8 have been used in e.g.. (Bonnabel) 2013 Tripuraneni et al. 2018 Sato et al. 2019 Han & Gao, 2021 and their reasonability is discussed in Appendix
Assumption 3.1. The retraction R is such that its restriction to Tg for all x E -,, Rus is of class C²,, and the associated vector transport T is continuous and bounded in the sense that there exists a constant Y >> O such that for any & E M, Su, Tz. E TTM,, it holds that [|T,, (Sa)ll S Mllsall- Assumption 3.2. For sequence of the outer iterates {aeleza and sequence of the inner iterates {(()) (1 [-1] h121 generated by Algorithm 1 there exists a W W-totally retractive set W C M such that {2a)121 W and (1 inside W. Additionally, there exists compact and connected set X C CM such that W c X.
W. The minimizers of Problem (.1) are
Assumption 3.3. The cost function F is continuously differentiable in W, the local cost functions fi..... fN are continuously differentiable in W, and their components fz(,E) for & D, with j E [N] are continuously differentiablee in W.
Assumption 3.4. The local objective functions f,- j E [N), are LipLishchizz continuously differen- tiable in W with the retraction R and the vector transport T (see Definition B.I), implying that F is also -ipschtt continuously differentiable.

---

Published as a conference paper at ICLR 2026
Assumption 3.5. F is Lg retrrccat smooth over W with respect to R. (see Definition B.2
Assumption 3.6. For any parameter E M, the Riemannian stochastic gradient grad f (x; {2 is an unbiased estimator of the gradient grad i.e., Rgilgradfj { = gradff Vj E [M].
Assumption 3.7. For any fixed parameter x E M, there exists a positive constant Øg such that for -
all j E [ it holds that Elll 1 B EsEB: f, 5,) (z))l
o with |B|| B.
The method estimating the probabilities is discussed in Section 3.3 Now we just make an assumption requiring that the approximate probability of in each round of communication is not far away from the true probability p:. formally stated in Assumption 3.8
Assumption 3.8. There exist constants qmins Imax E (0, 1] and G independent of t 2 1 and i
[N], such that the approximate probabilities qt 's satisfy
1 S /Gau, that quin S of. S +maxx
qi P.
and that of independent of S,, Vt 2 1, [NV, where at is the local step size in the t-th round of communication.
Note that the constant G controls the accuracy of the approximate probabilities and when the true probabilities are available to the server, G can take exactly zero. In Section 3.3 we discuss the reasonability of Assumption 3.8
Remark 3.1. In (Wang & Ji 2024), the authors imposed the following bound on the approximate
probabilities: E (1) P4
S 81- This bound essentially requires that 4t 1 less than some constant, which is consistent with Assumption 3.8 fixed step size cases. Note that this assumption considered in (Wang & Ji. (2024) only for fixed step size cases, but Assumption 3.8 considers another situation where the bound varies over time t when decaying step sizes are used.
3.2 CONVERGENCE PROPERTIES
In this section, we establish the convergence properties of the proposed RFedAGS. Theorem 3.1. Let Assumptions 3.7 hold. Suppose Algorithm is run with fixed global step size 0 and decaying local step size sequence {ar} satisfying Conditions
Σ
au = 00,
< 00.
(3.1)
Then, lim inf, Miti->-EEElllee E[llgradF(r = O.
In what follows, we further characterize the nonasymptotic convergence. Theorem 3.2. Under the same conditions as Theorem except that the local step size sequence {at} is determined by (B+ttr with constants on, 8 > U and p E (1/2, satisfying way K K I
the weighted average norm of the squared gradients satisfy, with Ar
Et= ag
Σ a.ElllraadF(x))1
Ol (ntFTt) O ((+Z))-)))
p
p (1/2,1).
Remark 3.2. In particular, if the full agent participate in any round of communication and agents use the full local gradient in local update, i.e,, G = O and a O, can relax the step sizes to Ot [B+1)7 where = 1/3 + a with a E (0, 2/3). In this case, for large T, the upper bound can be improved to Ar ET= Ellgadd ((rrdFF(x,))) 1121 < O ((347)378--)) (see Appendix
ag
1
Theorem 3.3. Under Assumptions 3.7 3.8 suppose that F' satisfies RPL condition, i.e., there exists O, such that for all E W, it holds that F(x) F(x*) S I (x)))
constant
If we run Algorithmm [ with the batch size Be [Brow Bupl and the step sizes satisfying at ytt for some 7 > and B w=k such that de L, S 1, then the iterates (2efe2) satisfy
8
2Lgu
[[F((+)) -()) 7+t
and EllgradF((::)1( +
(3.2)
(7 + 1)0(a1) ((m)) Q(K, B,, =) (2K 1)(K (f
where max RKB Brom 0nn m
F(m) F(x"). and
Bu K
p2
7

---

Published as a conference paper at ICLR 2026
1" with P, J, and H. being three constants depended on the problem, manifold and the 28.
retraction and 81,82, 03, da being constants depended on q Ps.Vi E [M] That is, Algorithm 7 converges sublineurly to the minimizer in expectation.
Theorems 3.1 3.3
3.3 provide the global convergence of Algorithm Under mild assumptions, the first theorem states that Algorithm D has global convergence in expectation for general objectives while the other theorems further provide the convergence rate of Algorithm However, all of these theorems require the usage of the decaying step sizes. When decaying step sizes are used, a large number of iteration are required for Algorithm| to converge. A compromise is to use a fixed step size of moderate size, the advantage of which is that the convergence rate is sublinear (even linear) while the disadvantage of which is that may not converge to the minimizers but to an e-stationary point/solution (see Definition B.4); see Theorems and and 3.5
Theorem 3,4. Suppose that Assumptions 3.1 3.8 hold. We run Algorithm 7 with fixed global step size w, a fixed batch size B, and a fixed number of local updates K.
1. If the fixed step sizes a and w satisfy oew NeoKL, Lg S 1, then
20(m))
EllggradF(::)11 WaKT + 20Q(K, a.).
(3.3)
If the true probabilities are known, meaning G and one takes local and global step sizes and such that V 828K with T satisfying T 2 max T6:0), 0(x))(K- 1
KL,B(Z)
then
B50 KTB T
2P
Σ EllgradF(aa)11 & 4TV 1)0(1)))
Remark 3.3. If the probabilities pa are known, iee. of = Pi. and Punin = min; {p:} is not too small
and not fairly far away from Pmax mmxx {pi}, such that the constants 07 8b 82, 03 are of
N
-
(1.) D
87 N EØ= (1-pal S
N
( In particular, the probabilities are the same across agents, e.g., Ps with S S N. then 83 S. and 03 N-S
Nraii then, Item 2 gives the upper bound as O p/min NKTB + O
VPmin NT
NN S 3. It follows that Item 2 gives the upper bounds as O
JSKTA + O YST
The
bound of 0 VST matches with the existing result for FedAvg given Karimireddy et al.
2020
Theorem 1) and improves by VK over that given in (ang al. 2021 Corollary 2).
Theorem 3.5. Under Assumptions 3.7 suppose that F satisfies RPL condition with a constant > O. If we run Algorithm 1 with batch size B. E [Blow Bupl and step sizes at and w satisfying owK min((1114 Lgg 1/u), then the resulting iterates {x+)1=1 satisfy
[[F(ET)] - F(x') s (1 -ooo-uoKaaa" (reK)))(((())) + + Blow, a. T-o Q(K, Blow, a, w). (3.4)
From Theorem 3.5 if one lets T + xc, then the expected optimality gaps [E[F(eT)] F(x*)) are bounded from above by Q Blow a. w). which implies that any accumulation point of the sequence of iterates {æt} generated by Algorithm is a c-solution if taking a I Q(KK Smaller a means smaller upper bound as well as slower convergence speed.
Remark 3.4. Similar to the Euclidean setting, the RPL property is weaker than the strong retraction- convexity. In fact, if the objective f ::M- - IR is p-strongly retraction-conves, then il also satisfies the RPL property with parameter μ (proved in Appendix E.2 Therefore, Theorems 3.3 and 3.5 also hold under strong retraction-convexiny.
3.3 ESTIMATING THE PARTICIPATION PROBABILITIES
At the t-th round of communication. let S, denote the set of participating agents. Then, under Assumption 2.1, Is, (i) follows the Bermoulli distribution, ie.. Is, ~ Bernoullipp).. At each round of communication, for each agent whether it participates in communication be regarded
8

---

Published as a conference paper at ICLR 2026
as Bernoulli trial. Therefore, by Bernoulii's Large Number Theorem, the frequency of agent i participating in communication goes closely to the true probability p. as the growth of t, the number of communications. Formally, let q: = Σ:= T.s, (t), and compute the approximate probability by df -////// (t//(11)) ensuring the independency of of and S.. Then we have lim P( -p:l I < for any small > O. This justifies the use of frequencies to estimate probabilities. The next theorem shows that Assumption 3.8 holds with high probability when the step size takes the form of [O))
Oz = (t= with a E (1/2.1 1]
Theorem 3.6 (Proved in Appendix D.7.
Under Assumption 2.1 for each agent i, we have
1 pi
fe2
4(1
p.)
2e oari
4(1 -P)
P
G
2 min
2e
min
(3.5)
fp.
G3ptit-a
where of = Er= Is. (i)/2, f = and 9 and a are nonnegative constants.
In practice taking a = 0 leads to fixed step size cases or a E (1/2, 1] to decaying step size cases. Therefore, it follows from Theorem 3.6 that Assumption 3.8 holds with probability not less than
1 - min fuf 4(1-P-)) min ip 1(1-p.) enough and properly chosen G make the probability high.
Gapatt- with a proper constant g 2 0. Large
4 EXPERIMENTS
Here we conduct numerical experiments principal component analysis (PCA) over the Stiefel manifold. hyperbolic structured prediction (HSP) over the hyperbolic manifold, and the Fréchet mean computation (FMC) over the SPD manifold such that we can compare RFedAGS with existing RFL algorithms. including RFedAvg (Li & Ma 2023). RFedSVRG (Li & Ma 2023), RFedProj (Zhang al. 2024) (used in PCA), and ZO-RFedProj (Wang et al.) 2025) (used in PCA). Additionally, we still conduct two experiments on principal eigenvector computation and low-rank matrix completion shown in Appendices A.1 AA22 The first one tests the comprehensive performance of RFedAGS, while the second compares RFedAGS with some existing centralized algorithms showing the comparable availability of RFedAGS with those. The experiment settings in this section can be found in Appendix A.3
PCA. The PCA problem has the form of min F(X) == + EM f((X), with f(() =
- E tr(X X where St(rr, d) is the Stiefel manifold, Zzz is the covariance matrix of local datum Ziz- E Rdxp. We generate D; = {f in two ways: (i) synthetic data by sampling from the Gaussian distribution (O, $ such that D. non-LLD; (ii) real-world data from CIFARIO dataset. We observe from Figure that our proposed RFedAGS outperforms the existing three RFL. algorithms under the arbitrary participation setting in terms of accuracy of solutions and consumed time. This justifies the efficacy of the proposed RFedAGS.
should be noted that the tools used in our RFedAGS are fairly general (as stated in Assumption B.1 however RFedAvg and RFedSVRG require more strict tools (the inverse of exponential and parallel transport), and RFedProj and ZO-RFedProj require the orthogonal projector onto the manifold. These requirements limit the application scope of RFedAvg, RFedSVRG, RFedProj and ZO-RFedProp For instance, RFedProj and ZO- ZO-RFedProj can not be used in the HSP and FMC problems below.
HSP.
Given a set of training pairs D [D.) (1 Mi. J.M, I where waj E R" is the feature and Yi.j E Hd is the hyperbolic embedding of the class of Wi.j- Then for a test sam- ple we the task of HPS is to predict its hyperbolic embeddings by solving the following problem arg mig F(z) : 1 Σ =11 fz(x). with f((x) " ot.; N'
(t.)) where the hyper- bolic manifold dd is characterized via the Lorentz hyperbolic model, [a (w)..., aN (w))" e R S a parameter matrix. We use the WordNet dataset datasee test RFedAGS. RFeeAAg, and RRG From the reported Figure we can observe that the proposed RFedAGS outperforms both RFedAGS and RFedSVRG in terms of distance to the true point. Figure 3(c) directtyy demonstrates this advantage of RFedAGS.
"Our code is available at mttttpss://qthhal http://hww..ooooeeeeeeeeeaaaaaaa
httpp:///s/ww.cs.tttt www.::o...toooooocccoo wttt:/ccccccc/ccccaaaaimmmm kriz/cifar.hmml
*See https//wondneetrmieetoo..ct

---

Published as a conference paper at ICLR 2026
10
CPU line (a1
(a) Synthetic data
(b) Synthetic data
(c) CIFARIO dataset
(d) CIFARIO dataset
Figure 2: PCA: RFddAGS consistently performs better than the competing methods across both
synthetic and real datasets.
0.45
(4.8 51 (2.8.33))
(4. 04.00 (3, 3.322
RFadAG8 liTliag AFNdSARS
Duaicee the toee. dww.
0.25
0.2
CPU
1kxc
Dersoorn
(a)
(b)
(c)
Figure 3: HSP with WordNet dataset. Here "primate" is the test sample (true point).
FMC. Given a set of SPD matrices, D {(X 1 the FMC of these SPD matri- ces is the solution to the problem arg min minxess, F(X) == N ( with fi(X) $ ES dist? (X, Xzz), where dist(:;)) is the Riemannian distance. We use the PATHMNIST dataset to test the algorithms. From Figure 4 we still observe that RFedAGS outperforms RFedAvg and RFedSVRG.
(4.N.8) (9.50 400)
66N, 5) (9.00 400)
RFNANGS
10"
Optinaltty D0
10-
Optinality
10
10-4
10
100
200
300
50
100
100
Bornsions
CIU time
(a)
(b)
Figure 4: FMC with PATHMNIST dataset: RFedAGS consistently performs better than RFedAvg and RFedSVRG.
5 CONCLUSIONS
In this work, we propose a Riemannian FL algorithm, called RFedAGS, that addresses critical chal- lenges caused by curvature effects of manifolds, the partial participation, and the heterogeneity data, Unlike the commonly studied random sampling setting, RFedAGS accommodates a more practical and challenging scenario where agents' participation statistics may be unknown. Theoretically, we prove that the proposed RFedAGS, under decaying step sizes, achieves global convergence and provide sublinear convergence rate. When using fixed step size, it attains sublinear-or even convergence near a neighborhood of a stationary point/solution. Numerical experiments we conducted have confirmed the efficacy of RFedAGS and in particular, it outperforms existing RFL algorithms methods on PCA, HSP, and FMC with synthetic and real-world data.
Current analyses on partial participation rely on time-invariant statistical assumptions. An important direction for future research is to analyze more realistic and complex scenarios, such as settings with time-varying participation probabilities.
SSee https://medmmist.com/
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
WH was partially supported by the National Natural Science Foundation China (No. 12371311), the Natural Science Foundation of Fujian Province (No. 2023J06004), and the Fundamental Research Funds for the Central Universities (No. 20720240151).
REFERENCES
P.-.. Absil, R. Mahony, and Rodolphe Sepulchre. Optimization Algorithms on Matrix Manifolds. Princeton University Press, Princeton, 2008
Mikhail Belkin and Partha Niyogi. Laplacian eigenmaps for dimensionality reduction and data representation. Neural Computation, (5(6))1373-1399) 2003.
Silvere Bonnabel. Stochastic gradient descent on Riemannian manifolds. IEEE Transactions on Automatic Control, 58(9):2217-2229, 2013.
W. M. Boothby. An introduction to differentiable manifolds and Riemannian geomelry. Academic Press, 1975.
Léon Bottou, Frank E. Curtis, and Jorge Nocedal. Optimization methods for large-scale machine learning. SIAM Review, (0(2):223-311, 2018
N. Boumal, B. Mishra, P.-A. Absil, and Sepulchre. Manopt, Matlab toolbox for optimization on manifolds. Journal of Machine Learning Research, 15(42):1455 (544))91455-159, 2014. URL https
/ www wwwopt. org
Nicolas Boumal. An introduction to optimization on smooth manifolds. Cambridge University Press, 2023.
Nicolas Boumal, Pierre-Antoine Absil, and Coralia Cartis. Global rates of convergence for nonconvex optimization on manifolds. IMA Journal of Numerical Analysis, 39(1):1-33, 2019.
Li Deng. The MNIST database of handwritten digit images for machine learning research. IEEE Signal Processing Magazine, 141-)14 2012.
OP Ferreira and PR Oliveira. Proximal point algorithm on Riemannian manifolds. Optimization, 51 (2):257-270, 2002.
Xinran Gu, Kaixuan Huang, Jingzhao Zhang, and Longbo Huang. Fast federated learning in the presence of arbitrary device unavailability. In Advances in Neural Information Processing Systems, 2021.
Xinran Gu, Kaifeng Lyu, Longbo Huang, and Sanjeev Arora. Why (and when) does local SGD generalize better than SGD? In International Conference Learning Representations, 2023.
Farzin Haddadpour, Mohammad Mahdi Kamani, Mehrdad Mahdavi, and Viveck Cadambe. Local SGD with periodic averaging: Tighter analysis and adaptive synchronization. In Advances in Neural Information Processing Systems, 2019.
A. Han, B. Mishra, P. Jawanpuria, and J. Gao. Riemannian block SPD coupling manifold and its application to optimal transport. Machine Learning Journal, 113(4):5955-162) 2022.
A. Han, B. Mishra, P. Jawanpuria, and J. Gao. Riemannian accelerated gradient methods via extrapolation. In International Conference on Artificial Intelligence and Statistics (AISTATS), 2023a.
A. Han, Mishra, P. Jawanuuri,, and A. Takeda. A framework for bilevel optimization on riemannian manifolds. In Conference on Neural Information Processing Systems (NeurIPS), 2024a.
Andi Han and Junbin Gao. Improved variance reduction methods for Riemannian non-convex optimization. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(11):7610- 7623, 2021.
11

---

Published as a conference paper at ICLR 2026
Andi Han, Bamdev Mishra, Pratik Jawanpuria, and Junbin Gao. On Riemannian optimization positive definite matrices with the Bures-Waasd Wasserstein geometry. In Advances in Neural Information Processing Systems, 2021.
Andi Han, Bamdev Mishra, Pratik Jawanpuria, Pawan Kumar, and Junbin Gao. Riemannian Hamil- tonian methods for min-max optimization on manifolds. SIAM Journal Optimization, 33(3): 1797-1827, 2023b.
Andi Han, Bamdev Mishra, Pratik Jawanpuria, and Junbin Gao. Differentially private Riemannian optimization. Machine Learning. 133(3):113311161 2024b.
Wen Huang and Ke Wei. Riemannian proximal gradient methods. Mathematical Programming, 194 (1):371-413, 2022.
Wen Huang, K. A. Gallivan, and P.-A. Absil. A Broyden class of quasi-Newton methods for Riemannian optimization. SIAM Journal Optimization, 25(3):1660-168: 2015.
Wen Huang, P.-A. Absil, and K. A. Gallivan. Riemannian BFGS method without differentiated retraction for nonconvex optimization problems. SIAM Journal on Optimization, 28(1):470-4499 2018.
Zhenwei Huang, Wen Huang, Pratik Jawanpuria, and Bamdev Mishra. Federated learning on Riemannian manifolds with differential privacy. arXivv, (oniv.or//bs///00010009111]] 1), 2024.
Jawanpuria, A. Balgovind, A. Kunchukuttan, and B. Mishra. Learning multilingual word em- beddings in a latent metric space: geometric approach. Transactions of the Association for Computational Linguistics, 103):077-120, 2019a.
P. Jawanpuria, M. Meghwanshi, and B. Mishra. Low-rank approximations of hyperbolic embeddings. In IEEE Conference on Decision and Control, 2019b.
Jawanpuria, M. Meghwanshi, and B. Mishra. Geometry-anwe aware domain adaptation for unsuper- vised alignment of word embeddings. In Annual Meeting of the Association for Computational Linguistics, 2020.
P. Jawanpuria, D. Shi. B. Mishra, and G. Gao. A Riemannian approach to ground metric learning for optimal transport. In International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2025.
Pratik Jawanpuria and Bamdev Mishra. unified framework for structured low-rank matrix learning. In International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pp. 2254-2263, 2018.
Sai Praneeth Karimireddy, Satyen Kale, Mehryar Mohri, Sashank Reddi, Sebastian Stich, and Ananda Theertha Suresh. Scaffold: Stochastic controlled averaging for federated learning. In International Conference on Machine Learning, pp. 5132-5143, 2020.
H. Kasai, P. Jawanpuria, and B. Mishra. Riemannian adaptive stochastic gradient algorithms matrix manifolds. In International Conference on Machine Learning, 2019.
Jaxiang Li and Shiqian Ma. Federated lcarning Riemannian manifolds. Applied Set- Valued Analysis and Optimization, 5(2):213-23,, 08 2023.
Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, and Virginia Smith, Federated optimization heterogeneous networks. In Proceedings of Machine Learning and Systems, volume 2, pp. 450, 2020a.
Xiang Li, Kaixuan Huang, Wenhao Yang, Shusen Wang, and Zhihua Zhang. On the convergence of FedAvg on non-iid data. In International Conference on Learning Representations, 2020b.
German Magai. Deep neural networks architectures from the perspective of manifold learning. In IEEE 6th International Conference on Pattern Recognition and Artificial Intelligence, pp. 1021-1031, 2023.
12

---

Published as a conference paper at ICLR 2026
H. Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. Communication-efiin learning of deep networks from decentralized data, In Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, 2017.
George A. Miller. WordNet: A lexical database for English. Commun. ACM, 38:39 41, 1995.
Bamdev Mishra, Hiroyuki Kasai, Pratik Jawanpuria, and Atul Saroop. A Riemannian gossip approach to subspace learning on Grassmann manifold. Machine Learning, 108: 1783 1803, 2019.
Maximilian Nickel and Douwe Kiela. Poincaré embeddings for learning hierarchical representations. Advances in Neural Information Processing Systems, 2017.
Maximillian Nickel and Douwe Kiela. Learning continuous hierarchies in the Lorentz model of hyperbolic geometry. In International Conference on Machine Learning, pp. 3779-3788, 2018.
M. Nimishakavi, P. Jawanpuria, and B. Mishra. A dual framework for low-rank tensor completion. In Advances Neural Information Processing Systems, 2018.
Mónica Ribero, Haris Vikalo, and Gustavo de Veciana. Federated learning under intermittent client availability and time-varying communication constraints, IEEE Journal of Selected Topics in Signal Processing, 17(1):98-111, 2023.
Elsa Rizk, Stefan Vlaski, and Ali H. Sayed. Federated Learning Under Importance Sampling. IEEE Transactions Signal Processing, 1.5331-5399,, 2022.
Hiroyuki Sato, Hiroyuki Kasai, and Bamdev Mishra. Riemannian stochastic variance reduced gradient algorithm with retraction and vector transport. SIAM Journal on Optimization, 29(2):144411772, 2019.
Sebastian U. Stich. Local SGD converges fast and communicates little. In International Conference on Learning Representations, 2019.
Nilesh Tripuraneni, Nicolas Flammarion, Francis Bach, and Michael I Jordan. Averaging stochastic gradient descent Riemannian manifolds. In Conference On Learning Theory, pp. 650-687, 2018.
Oncel Tuzel, Fatih Porikli, and Peter Meer. Region covariance: A fast descriptor for detection and classification. In European conference on computer vision, pp. 589-600, 2006.
Hongye Wang, Zhaoye Pan, Chang He, Jiaxiang Li, and Bo Jiang. Federated learning on riemannian manifolds: A gradient-free projection-basse approach. arXiv preprint urXiv:2507.22855 2025.
Jianyu Wang and Gauri Joshi. Cooperative SGD: A unified framework for the design and analysis of
local-update SGD algorithms. Journal of Machine Learning Research, 22(213):1-50, 2021.
Shiqiang Wang and Mingyue Ji. A unified analysis of federated learning with arbitrary client participation. In Advances in Neural Information Processing Systems, volume 35, pp. 19124- 19137, 2022.
Shiqiang Wang and Mingyue Ji. A lightweight method for tackling unknown participation statistics in federated averaging. In International Conference on Learning Representations, 2024.
Ming Xiang, Stratis loannidis, Edmund Yeh, Carlee Joe-War Wong, and Lili Su. Towards bias correction of FedAvg over nonuniform and time-varying communications. IEEE Conference on Decision and Control, pp. 6719-6724, 2023.
Ming Xiang, Stratis Ioannidis, Edmund Yeh, Carlee Joe- Wong, and Lili Su. Empowering federated learning with implicit gossiping: Mitigating connection unreliability amidst unknown and arbitrary dynamics. IEEE Transactions Signal Processing, 73:766-788 2025.
He Xiao, Tao Yan, and Kai Wang. Riemannian SVRG using order approximation for federated learning. Symmetry, 16(9): 1001, 2024.
method second-
He Xiao, Tao Yan, and Shimin Zhao. Riemannian SVRG with Barzilai-Borwein scheme for federated learning. Journal of Industrial and Management Optimization, 112::1546-1577 2025.
13

---

Published as a conference paper at ICLR 2026
Bo Xiong, Michael Cochez, Mojtaba Nayyeri, and Steffen Staab. Hyperbolic embedding inference for structured multi-labe prediction. Advances in Neural Information Processing Systems, 2022.
volume 35, pp.
Yikai Yan, Chaoyue Niu, Yucheng Ding, Zhenzhe Zheng, Shaojie Tang, Qinya Li, Fan Wu, Chengfei Lyu, Yanghe Feng, and Guihai Chen. Federated optimization under intermittent client availability, INFORMS Journal on Computing. 36(1):1185-22)) 202, 2023.
Haibo Yang. Minghong Fang. and Jia Liu. Achieving linear speedup with partial worker participation in non-iid federated learning. In International Conference on Learning Representations, 2021.
Jiancheng Yang. Rui Shi, Donglai Wei, Zequan Liu, Lin Zhao, Bilian Ke, Hanspeter Pfister, and Bingbing Ni. Medmnist v2-a large-scale lightweight benchmark for 2d and 3d biomedical image classification. Scientific Data, 10(1):41, 2023.
Haishan Ye and Tong Zhang. DcEPCA: Decentralized exact PCA with linear convergence rate. Journal of Machine Learning Research, 22(238):1-27, 2021.
Bicheng Ying. Zhe Li, and Haibo Yang. Exact and linear convergence for federated learning under arbitrary client participation attainable. arXiv preprint arXiv:2503.201 2025.
Hao Yu, Sen Yang, and Shenghuo Zhu. Parallel restarted SGD with faster convergence and less communication: Demystifying why model averaging works for deep learning. In Proceedings of the AAAI Conference on Artificial Intelligence, pp. 5693-5700, 2019.
Hongyi Zhang and Suvrit Sra. First-order methods for geodesically convex optimization. In Conference on learning theory, pp. 1617-1638. PMLR, 2016.
Jiaojiao Zhang, Jiang Hu, Anthony Man-Cho So, and Mikael Johansson. Nonconvex federated learning compact smooth submanifolds with heterogeneous data. In Advances in Neural Information Processing Systems, volume 37, pp. 109817-109844. Curran Associates, Inc., 2024.
Fan Zhou and Guojing Cong. On the convergence properties a K-step averaging stochastic gradient descent algorithm for nonconvex optimization. In International Joint Conference Artificial Intelligence, 2018.
Xiaojing Zhu and Hiroyuki Sato. Cayley-transffrrm-basss gradient and conjugate gradient algorithms on Grassmann manifolds. Advances Computational Mathematics, 47:1-28 2021.
14

---

Published as a conference paper at ICLR 2026
Appendix
Experiment settings and additional experiment results
16
A.I
Comprehensive tests
16
A.
Comparison of two aggregation patterns
17
A.1.2
Comparisons of different participation schemes
18
A.1.3
Influence of the level of data heterogeneity performance
19
A.1.4
Effect of local multiple-step update)
19
A.1.5
Comparisons with existing Riemannian FL algorithms
20
A.2
Comparisons with some centralized algorithms
20
A.2.1
Synthetic case
21
A.2.2
A world application
22
A.3
The details of experiment settings in Section
23
A.3.1
PCA.
23
A.3.2
HSP
24
A.3.3
FMC.
25
Preliminaries on Riemannian optimization
25
Additional Discussions
27
Discussions for Assumptions
27
Discussions for Implementations
28
Proofs of Theorems in Section
28
D.1
Supporting lemmas
28
D.2
Proof of Theorem
34
D.3
Proof of Theorem 3.2
35
Proof of Theorem
35
D.5
Proof of Theorem 3.4
36
D.6
Proof of Theorem 3.5
37
D.7
Proof of Theorem 3.6
37
Supplementary Proofs
38
Proof of Theorem
38
Proof of the claim in Remark
38
15

---

Published as a conference paper at ICLR 2026
A EXPERIMENT SETTINGS AND ADDITIONAL EXPERIMENT RESULTS
In this section, supplement the numerical experiments conducted to demonstrate the performance of RFedAGS (Algorithm D on non-I.I.D, data setting. We focus on the finite sum minimization of 1.1..
The decaying local step size determined by the following formula
if
0t
with Ct
C--1+1 if mod(t,)) d) 0,
2 1,
Ct-1
otherwise,
where og and are two positive constants, and is positive constant integer, which results in the step size decaying once after each d iterations. Optimality gap defined as F(x;)) F(xx) with r E arg min Fx) a commonly-usee measure evaluate the performance of algorithms. In all experiments, the global step size is set 1. The CPU time consists of the server computation time and the local computation time of active agents, without the communication time between the server and agents. Unless otherwise specified, frequencies are used in Algorithm [ to estimate the true probabilities. All of algorithms involved our experiments are implemented built on Manopt (Boumal er al. 2014). All of the experiments are conducted under Windows 11 and MATLAB R2024b running on a laptop (Intel(R) Core(TM) i7-1 165G7 CPU @ @2.80GHz, 16.0G RAM).
A.1 COMPREHENSIVE TESTS
Consider the principal eigenvector computation (PEC) problem over the sphere manifold, formulated as follows
N
1
min F(x) ZE
=
fl()), with f((x)
x 21
2
(A.1)
5
j=1
where Sa-1 = {o E R" I- x samples held by agent Problem A..I is in the form of finite sum minimization of (1.I)
2i is the local is the sphere manifold, D; (z.,,...,
The sphere manifold S is viewed as Riemannian embedded submanifold of IR", that is, the Riemannian metric is induced by the Euclidean metrie: (E. 7) E" n for all E. T- The exponential mapping is chosen as the Retraction and the parallel transport along the geodesic correspondingly is selected the isometric vector transport. The MNIST dataset (Deng) 2012 consists of 60000 hand- written gray images of size 28 x 28 each of which is associated with a label taking values from column vector by column. In addition, to test the effectiveness of the proposed RFedAGS under the heterogeneity data setting, according to the FL setting, the MNIST dataset shuffled into different levels of heterogeneity following the way in (McMahan al. 2017). Figure demonstrates histograms of the MNIST dataset with three different levels of heterogeneity.
our experiments, each image is concatenated into 784-dimensional
1000
(a) Non-L.DD (heavy)
(b) (slight)
(c) L.D.
Figure 5: Sample distributions across different agents on MNIST dataset. --axis is the ID of each agents and y-axis is the number of local samples.
"See http:://aameeeenn.cc//ccccc////a
16

---

Published as a conference paper at ICLR 2026
A.1.1 COMPARISON OF TWO AGGREGATION PATTERNS
First we demonstrate the importance of the aggregation pattern (AGS-AP). As shown in (2.1) the aggregation of RFedAGS in Line [1] of Algorithm actually is unbiased in the sense of 1 [ies, P.N (x)) = gr)dF(x). Nevertheless, if the participation probabilities are not
considered and the usual aggregation, #++1 < Ra, (-=Ejes, [s. (t.K) ). is used, then the output of the algorithm equipped with this aggregation will tend towards a minimizer of another objective function different from the original objective when there exist i,j E [N] such that p: # Ps. which exactly what Theorem 2. points out.
10
10
True Approsimating Blased
True Approximating ling Biased
Optimulity gop
Optinallty: gaap
10
10
10
10
10
10
100
300
400
500
200
400
600
800
1000
Iterations
Iterations
(a) Fixed step size
(b) Decaying step size
Figure 6: PEC with non-L.LDD (slight) MNIST dataset: comparisons of the two aggregations pat- terns (AGS-RS) and (AGS-AP).
Figure reports the experiment results, where the two curves "True" and "Approximating" adopt the aggregation pattern (AGS-AP), the curve "Approximating" uses the frequency estimate the true probability, and the curve "Biased" uses the usual aggregation (AGS-RS). Besides, the participation probabilities p3 's are uniformly and randomly generated (i.e.. Ps.
[NV, follows the uniform distribution U0, 1)), the fixed step size is set as = 8.0 x 10 the parameters for decaying steps sizes are set as (ao, B, d) (3.5 x 10-1, 0.1, 20), batch size is B 00.55, and the number of local updates is set as K = 5. It is observed from Figure 6 that RFedAGS equipped with the aggregation pattern (AGS-AP) gives a better solution to Problem (A.I) than that generated by RFedAGS equipped with the usual aggregation pattern (SG The reason lies on that the usual aggregation pattern (AGS-RS) leads the iterates to the minimizer of f := -> Pi with p; p; Io [1 - P; + pjt)dt, as stated by Theorem 2.1 Meanwhile, due to p; # pj for some i E [N]. it follows that there exists no X > such that F X F. Hence, the minimizers of F may be not consistent with those of F.
10
10
True Approximating
True Approximating Biased
Optimality gap
10
Biased
Optimallity sap
2
10-
10-4
10
10.
0
100
200
300
400
500
200
400
600
800
1000
Tterations
Iterations
(a) Fixed step size
(b) Decaying step size
Figure 7: PEC with non-L.LD (slight) MNIST dataset: RFedAGS with the two aggregations solve the re-weighted problem arg min F(x).
17

---

Published as a conference paper at ICLR 2026
Furthermore, Figure 7 shows the curves of optimality gap v.s. iterations for the re- re-weighted ob-
jective F' valued at the iterates given in Figure Combining Figures 6 and 7 we conclude that RFedAGS equipped with the aggregation pattern (AGS-RS) does solve the re-weighted problem arg minge. F(x) rather than the original problem.
A.1.2 COMPARISONS OF DIFFERENT PARTICIPATION SCHEMMES
Here we consider the special case where each agents participates in any round of communication with the same participation probability, i.e., Pt = p. with i [N]. In this case, the random sampling scheme is denoted by Scheme I, while our arbitrary participation scheme denoted by Scheme II, where we frequencies estimate the true probabilities. For Scheme I, the sampling rate (the ratio of the number of sampled agents to the number of total agents) is as 0.3 (0.5, or 0.7). For Scheme II, the participation probability agent i respectively set as p; 0.3 (0.5, or 0.7) for all i [M] such that the number of participating agents in Scheme II is equivalent to that of Scheme I in
expectation, which means Ei=1 P: p.N. The fixed step size is a 8 x 10-5 the parameters for decaying step sizes are set as (ao, B, d) (3.5 x 10 0.1, 20). batch size is B 0.58, and Figure 8 the performance of
the number of local updates is set as K = 5. As demonstrated two participation schemes are extremely the same. This indicates that Scheme I can be viewed as special case of our participation scheme and that using frequencies to estimate the true probabilities sufficient ensure convergence.
18
Stoae E-3
Siaaes 145
Siree E15
Soese 15:1
Shee 234.3
Stoce
106
CPC
(a) Fixed step size
(b) Fixed step size
(c) Decaying step size
(d) Decaying step size
Figure 8: PEC with non-L.LD. (slight) MNIST dataset: comparisons of the two participation schemes.
Next, we simulate the scenario of straggling agent participation. Suppose that the first three agents are stragglers and make their local computation time become 10 times as much that under normal conditions. Specifically, for Scheme I, one of the three stragglers are chosen, then its local computational time becomes 10 times as much as that under normal conditions; for Scheme II, setting the stragglers' participation probabilities as 0.05 ensures that they rarely participate in local updates, and when one of the stragglers responds to the server, its local computational also becomes 10 times as much as that under normal conditions. The participation probabilities of the other agents are properly set such that Σ Ps ~ P.V. The fixed step size is set a 8 x 10 10-5, the parameters for decaying step sizes are set as (ao, 3, d) = (2.8 10 0.1, 20), batch size is B = 0.55, and the number of local updates is set as K The experiment results are shown in Figure 9
Caae Sewee 18 Fl
Kaee TEG Samnn Big 6C
Mona Sabee Bi
Shev 11 1m
10
CP 30
021 taar lt
cask 06
CTC 120
(a) Fixed step size
(b) Fixed step size
(c) Decaying step size
(d) Decaying step size
Figure 9: PEC with non-L.DD (slight) MNIST dataset: the situation where the FL system has three stragglers. Here in the legends. Scheme-ll-Truee (or Scheme-II-Fre means that the Scheme II is equipped with the true probabilities (or frequencies serving as the true probabilities).
18

---

Published as a conference paper at ICLR 2026
By the definition of Scheme I, each agent is sampled with probability p (e.g., 0.3 and 0.5 in our experiments), which much greater than 0.05 in Scheme for the three stragglers. Hence, the number of stragglers participating local updates of Scheme I is greater than the one of Scheme II, leading to the CPU time of Scheme are greater than the one of Scheme II. The results in Figure is consistent with analysis. Meanwhile note that the performance of using the true probabilities and frequencies is extremely the same, which indicates again the validity of using frequencies serving as the true probabilities.
It should be noted that in practical situation, if some agents do not respond to the server in a certain round of communication, then scheme may not work in this case, because one of these agents may be sampled by the server, but it will not respond to the server. This will cause the algorithm to stagnate. Nevertheless, Scheme II does not encounter this issue since the server does not choose the agents which do not respond.
A.1.3 INFLUENCE OF THE LEVEL OF DATA HETEROGENEITY ON PERFORMANCE
Next we test the impact of the heterogeneity level of the MNIST dataset on the performance of RFedAGS. Here the participation probabilities p, 's are uniformly and randomly generated, that is, ~ U(O, 1) for i E [M]. The fixed step size is set a 8 x 10 the parameters for decaying step sizes are
(co, B, d) (2.8 x 10-4, 0.1, 20). batch size is B 0.5.S, and the number of local updates is set as K = 5. The experiment results are reported in Figure I0 where we observe that the quality of the solution generated by Algorithm gets worse as the growth of the levels of heterogeneity of the training data across agents. Additionally, Theorems 3.1 and 3.2 point out that if decaying step sizes satisfying equation 3.1 are used, Algorithm I has global convergence. Hence, it is expected that the higher-quality solutions may be found when using decaying step sizes and running more rounds of communication compared with the case using a fixed step size. This is consistent with the experiment results shown in Figure Figure 10
1
10a
LLD.
A LLD.
Non-L.LDD-sligh
N.n-L.L...slihhh
Optimality gap
109
Son-L..D.-baavy
Optimality gap
10
Non-I.L.D-baayy
10
10
10
10
10
100
400
500
500
1000
1500
2000
2500
Iterstions
Iterations
(a) Fixed step size
(b) Decaying step size
Figure 10: PEC with different non-I.I.D. datasets: impact of heterogeneity level.
1.4 EFFECT OF LOCAL MULTIPLE-STEP UPDATE
In addition, test the impact of different number of local updates K on the performance of A
[ The participation probabilitics p.'s are uniformly and randomly generated, that is, p: U(0 1) for i [N]. The fixed step size is set 8 10-3 and batch size is B 0.5S. The
experiment results are shown in Figure
When using a fixed step size, Item of Theorem 3.4 states that the convergence upper bound consists 2)(21) of two terms: a decaying term wa KT as K (or 7) increases, and increasing (or constant) term QoQ(K, B, w) with respect to K (or T). The initial guess is usually generated at random such that (zx) relatively large, and thus the first term dominates at the initial stage. As a result, at the initial stage, the convergence speed is accelerated when using larger K. Subsequently, due to the growth of T. the second term begins to dominate and thus when using larger K the error of solution generated by Algorithm Io to the minimizer get larger. This analysis is verified by Figure [I] Additionally, we note that in fixed step size cases, Algorithm I numerically demonstrates linear convergence seen Figures 11
19

---

Published as a conference paper at ICLR 2026
10
105
105
Optmmlnyy pmp
10
Optinalit top
Oatinally emp
106
1
10*
10
10
10-1
200
400
600
800
1005
200
400
800
800
1000
0
200
400
600
800
1000
Berratonn
Brraaionn
Eteratiioa
(a) LLD.
(b) Non-L.LD. (slight)
(c) Non-L.LD. (heavy)
Figure 11: PEC with non-I.I.D. (slight) MNIST dataset: impact of number local updates.
A.1.5 COMPARISONS WITH EXISTING RIEMANNIAN FL ALGORITHMS
Here we test the performance of RFedAGS, RFedAvg, RFedSVRG, and RFedProj as (i) data distributions diverge or (ii) participation becomes sparse.
(i) for the first purpose, we use the MNIST dataset partitioned as three different levels of heterogeneity;
Figure(3jfor the sample distributions. Figures 12(a) (2(b) show the results, where the participation probabilities p; 's are uniformly and randomly generated (i.e., pi ~ (0 1) for i E [ND, the fixed step size is set a 8 10 and batch size B 0.5S. We can observe from Figure 12 that as expected, for all of algorithms, as data distributions diverge, the performance becomes poor. Besides, the same level of data heterogeneity, RFedAGS consistently outperforms compared to the other algorithms.
(ii) for the second purpose, we use the non- non.L.L.D. (slight) MNIST dataset (see Figure [(b). The experimental results are reported in Figures [2(c) 12(d) where 0.5, 0.4, 0.3 in the legends denote the
expected participation ratios, i.e., N Σm= Epp = 0.5, 0.4, 0.3. Specifically, for participation ratio 0.5, we set the participation probabilities as p; ~ U(0. 1); next, for participation ratio 0.4 (or, 0.3), let p 0.8 pi (or, 1/1 0.6 x p.). It follows from Figures (2(c) ( that participation becomes sparse, the performance of all algorithms becomes poor. On the other hand, at the same participation ratio, our RFedAGS consistently performs compared other algorithms.
A.2 COMPARISONS WITH SOME CENTRALIZED ALGORITHMS
Low-rank matrix completion (LRMC) aims to recover the missing entries of unknown matrix from small account of accesible entries with low-rank constraint for the matrix. Mishra et al. (Mishra ctt al. 2019) formulate LRMC in the form of finite sum, which can be extended to the FL setting with finite sum minimization follows:
N
s
1
min F(U): ueGr(r.m)
Σ f(0). with f((1)
Σ
0.5 0.5 (UW (ju) Pa., (Y)))
N
S 3-1
(A.2)
+ + JJU Pa., TWWT
where
Gr(r, m) is the Grassmann manifold, i.e. the set of all the r-dimension subspaces of Rm U St(r,, m) is the matrix characterization of E m), WiJu E Rn xr with En 2n=1 Mij = n is the least-squares solution to arg winl x**""* (ol Pay MB + AAJ Pa, (UW into Y [Yi....... Y i........ Y N.I..... elements of Y": the (1, keeeemenn of Y is nonzero if and only if its index belongs to S2 and is also partitioned similar to the way of Y: = (QI.1.... Q1,s...., IN. (...s) and opera- tor Pn., is the orthogonal sampling operator defined by [Po., (Y) the (1, k)-element of Y kI) E Sr and (Ps., (Y)luk = otherwise. It is worthy mentioned that Problem (A.2 is defined on Gr(r. m) but the computation can be implemented with matrices U in St(r, m). The werrsampling ratio (OS) is the ratio of number entries of Ω and the freedom degree of Y i..., OS [0/(((m- r)r)
S
i3 Il Y E mRx is the known matrix and is partitioned Y with Yi, E Rmxns, Ω is the indices set of
20

---

Published as a conference paper at ICLR 2026
10
10"
Optinality gop
10
Optimality gap.
10
BEEddSVI heary
10
10
200
400
600
6
Iteratione
CPU time
(+)
(a)
(b)
00"
10
RFaAAGS444
T00
Optinnaltty gop
RFaAAGS-4
RRalAgp-1
RRalAg-0.
BRalAsp-0a
10
10
10"
Berations
CPU time
(c)
(d)
Figure 12: PEC with the MNIST dataset. (a)-(b): performance as data distributions diverge. (c)-(d); performance as participation becomes sparse.
The Grasssmann manifold (r(r,m) is equipped with the quotient structure Gr(r, m) St(r, m))(r)) [[U] U E St(r,m), with O(r) the orthogonal group of the order r. The Riemannian metric on Gr(r, m) induced by the inner product, i.e., (NR, Eu trace Eu. where Eu, the horizontal lift of Eu. The retraction via Cayley transform (CT) (Zhu & Sato 2021 is given by
Cay u
(Eu)
U Eut,
U 122
tem, ) (1. + 1
(I++
St. Suh
and the inverse of Cay (Zhu & Sato 2021 is computed by
(R
Cay
(M)
2(V -U"V
U4
Correspondingly, the isometric vector transport associated with Cay (Zhu & Sato 2021 is given by
mJ (Gu)) = Eir -ant--uuuuu (U+)mm))
(, (1++
1 Tii M1i,T mu,,
Nee Eu,
with = Cay (πt). We point out that Algorithm I does not require the usage of the inverse of retraction. Here. what we use the inverse of retraction is just to assist in the implementation of the vector transport. Moreover, if one uses the vector transport by projection, then the inverse of retraction does not need.
A.2.1 SYNTHETIC CASE
AB"", mm |S| entries randomly removed with uniform probability. Each of the rest entries is perturbed by noise obeying and standard deviation 10-6, In the experiment, the rank is set as r = 5, the OS is set as OS = 6, and (m, n. = (100, 2000). The other parameters are set as [N], B 0.58, and 2 x 10-3
Sample at random two matrices A E mxx and B E guxr, Let Y
the Gaussian distribution with mean
0, (N, S) (20, 100), Pt U(0.
21

---

Published as a conference paper at ICLR 2026
Let U be the solution given by Algorithm 1 Then Wü I1ū...... ........ W IS W NIŪ W Wsu and thus the approximation to Y is given by Y UW Relative enror (lower is better)
between Y and Y computed by
Y Y |
rel vel_er(()
used to measure the performance of Algorithm!! From Figure 13 the number of inner iterations significantly affects the convergence. It is worth mentioning that the results demonstrate Algorithm has a linear convergence rate.
we also observe a similar result:
10
10
Optinality gap
10
Relative Errors
K=
K=20
10:5
10:10
10:5
10
100
60
Tterations
Teerations
(a)
(b)
Figure 13: LRMC with synthetic data: performance of RFedAGS with different K.
A.2.2 A REAL-WORLD APPLICATION
We use MovieLens IM dataset which consists of 1000209 ratings with 6040 users rating 3952 movies. In LRMC setting, Y E R=xn with 3952, = 6040, and [S]] 1000209, whose nonzero elements are the ratings. We randomly sample 80% ratings for each column of Y as the training samples, denoted by Y and the testing dataset, denoted by Y is consisted of the remainder. In terms the FL setting. Y equally divided into = 40 agents by column at order, Le., Y |Y!,.... ..... Ytr), and each agent has S 151 columns, ie., Y!! [Y!r i....... Y
where Y E Th The other parameters are set as A 10-2 ps U(0, 1), E [N]. B 0.55, and a = 6 x 10-4
In order to evaluate the performance of those methods, the root mean square error (RMSE) is used and is computed by
RMSE(Ý)
Σ (ij)enne
Y Y 2
with Y. Y and f being the approximation Ytc, the testing matrix, and the indices set of known entries of Y respectively. We observe in Figure 14 and Table 3 that the proposed RFedAGS comparable these centralized methods in solving LRMC in terms of RMSE when choosing an appropriate K.
"See https://wmmoooaoo.///aaaa/ccnona
22

---

Published as a conference paper at ICLR 2026
Km2
EASSE ez tavinx dacaant
9
0.9
0.0
ROUSE teeting
Kell BBC LISPUS
on teeting
Kmlj
0.85
3
LRSPCe
188PC3
668-
0.8
Ieeattonn
Beratione
Ierracioos
(a)
(b) 5
(c) P
7
Figure 14: LRMC with MovieLens 1M dataset: comparisons of RFedAGS (with different K) with
RSD, RCG, and LRBFGS.
Table 3: The best RMSE scores (lower is better) on testing set for different subspace dimension r and different number of local update K. Here the scalar a.bedg denotes a.bed 10
RFedAGS
RSD
RCG
LRBFGS
K == 2
K = 4
K = 8
K = 12
K 16
r 3
8.260
8.101-1
8.023
7.968
7.948
7.925
7.925
7.925
r = 5
8.095
7.902-1
7.757-1
7.679
7.654
7.616-1
7.614
7.614
r 7
7.966
7.743_1
7.577-1
7.507-1
7.468_1
7.392_1
7.384
7.382_1
A.3 THE DETAILS OF EXPERIMENT SETTINGS IN SECTION
In this section, we detail the experiment settings Section
A.3.1 PCA.
We restate the PCA problem as follows for convenience:
N
8 Σ tr(X" (Ziz j=1
min F(X)= = N Σ f((), with f((X) = S XESt(r,d)
(A.3)
where St(r, d) = (X E RRdx X = }, is the Stiefel manifold, D, = 1 7: C Rdxp the local dataset held by agent i, Vi E [N].
For the Stiefel manifold St(r, d), we view it as Riemannian manifold embedded in R Thus the Riemannian metric chosen as (U, V)x = (U, V)p for all E St(r, d) and U, V E St,t d). The retraction is the qr-retraction (Absil et al. 2008) and the vector transport is given via the projection, i.c., U = PRT (U). In theory, RFedAvg and RFedSVRG (Li & Ma 2023 require the exponential mapping, its inverse, and parallel transport. But on the Stiefel manrfold, the last two operators have closed-form expressions. Thus use retraction, its inverse, and vector transport to replace them.
For the synthetic data, set and generate the local datesets by setting (Za Z.s = Z, drawn from the Gaussian distribution
Setup details corresponding to Figures 2(a) 2(b)
Z; (0, i ). In experiment, all parameters are set as (r, d) = (5, 100), (N, 8) (40, 100).
a = x 10 B = 0.5S, K = 5, and p; ~ U(0, 1).
Setup details corresponding to Figures 2(c) 2(d) For CIFAR10 dataset, whose training dataset contains 50000 RGB images with size 32 x 32 each channel, it is also shuffled following the way McMahan al. (2017) such that the local datasets are non-I.L.D, (see Figure 15 beloww. In experiment, we flatten each image into a vector in IR 3072 and thus each local data point Zij is inside 3072 The other parameters are set as (r, d) = (072),,, (N, S) = (50, 1000), a = x 1 B 0.5.S K 5, and Pi U(0,1)
23

---

Published as a conference paper at ICLR 2026
Display Label Distribution (Non-I.ID,, slight)
800
600
400
200
(a) Local dataset distributions
Figure 15: Local dataset distributions of the CIFAR10 dataset
Scalability of RFedAGS on PCA. Here we conduct additional experiments to empirically explore the scalability of RFedAGS. The results are reported in Figure [T6 where the local update step is set K = 5, batch size is B 0.5S. In the first column, we fix the local dataset size and the manifold dimension and enlarge the number of agents. In the second column, we enlarge local dataset size and fix the other two factors. In the last column, we enlarge the manifold dimension and fix the other two factors. solve these problems of such scale, showing the scalability of RFedAGS. We would like to point out that as shown in Table 2 number of agents, local dataset size, and manifold dimension have a linear relationship with the total computation complexity, so their increase will not cause the total computation complexity to increase sharply.
summary, it can be observed from Figure [6 that the RFedAGS can all
N
20003
No.00 No.0
5-853
10"
1411-1200
No110
1035
Opiinality zup
N-200
14th- H42--400,, 101
Optimaltt
Uplinallty
10*
10-
10a
500
1000
1500
2000
2500
600
1000
1500
2000
2500
500
1000
1500
2000
2500
Itersttons
Iereatiune
(a) Increasing number of agents
(b) Increasing local dataset size
(c) Increasing manifold dimension
- 1000
N 200
(MT,8) B) (96, 1000)
S--00
Optiuality aup
Optimality anp
10
5--1000
Hay 12000
10
Openalit
to
10
10
10
(d) Increasing number of agents
(c) Increasing local dataset size (f) Increasing manifold dimension
Figure 16: PCA with synthetic data: scalability of RFedAGS.
A.3.2 HSP.
Given set of training pairs D = [D:),, f((wi,. 1 | where wij R is the feature and Yi,j Hd the hyperbolic embedding of the class of way. Then for test sample we the task of
24

---

Published as a conference paper at ICLR 2026
HPS is to predict its hyperbolic embeddings by solving the following problem
1 Σ f(()), with f((z)
1 S and (w)) (x, v1.5)
arg min F(x) #EA4
where the hyperbolic manifold 7d is characterized via the Lorentz hyperbolic model Hd { Rd+1; (x,44) _ -1} with (x, v) x² 20131 a (w)² = (ar,1 (w),... (w,)( E R is a pre-given constant vector related to w, and dist(:..) : M x M + E is the Riemannian distance. A commonly used option of a, (w is computed by a a,(w) = (K, + 71) Kiw. where y is the regularization parameter, and K2 E RSxs and Ki E R are given by (Ki)ah k(w,, Wi,)) and (K = k(ws.j. w) for a raial basis function (RBF) kernel k(w, w') = exp(-1l) w 1//1// (2v) with constant 0.
Setup details corresponding to Figure 3 The WordNet dataset Millerr 1995) is used to conduct the experiment inferring hyperbolic embeddings (Nickel & Kiela 2018 Jawanpuria et al. 2019b). Following (Nickel & Kiela 2017), the pretrained hyperbolic embeddings
Ht of the mammals subtree with the transitive closure containing n = 1180 nodes (words) and 6540 edges (hierarchies) used
[ The features are stemmed from Laplacian eigenmap (Belkin & Niyogi 2003 to dimension r == 3 of the adjacency matrix formed by the edges. In other words, we obtained [[w;, y/) ) C R3 x H.2 This setting is in line with the work in (Han et al. 2024b). In the experiments, the word "primate" is selected as the test sample, and the remainder is used to train. Therefore, the hyperbolic embedding of the word "primate" is known and is viewed as the true embedding, i.e., Ttrue- For other parameters, they are set as SN = (9, 131), a = 6 x 10 2,-2, B = 0.5S, K = 5. Pi C(0, 1), and (7,2) = -10-9,0.3)
A.3.3 FMC.
Given a set of training SPD matrices D = [D.) (( where (Xi,), S
SN+= = (X E NNN, xx = X, > 0}, the FMC of these SPD matrices is the solution to the following problem
N
9 dist (x,
arg min minmmiFFa F(X) = N
Σ f((),, with f(() = 5
1
Σ
XESY where dist(X, = [logm(X (X ( X1-1/2 x x-1/2) with logm(-) the principal matrix logarithm is the Riemannian distance.
j=1
Setup details corresponding to Figures|4 The PATHMNIST dataset (Yang et al. 2023) consists of 89996 RGB images and we transform each image into a 9 x 9 SPD matrix by the covariance descriptor (Tuzel et al. 2006). In the experiment, randomly selects 20000 images to construct the training dataset. The parameters are set as (N,S) = (50, 400), a = 0.01, B = 0.5S, K = 5, and p: U(00 1).
B PRELIMINARIES ON RIEMANNIAN OPTIMIZATION
In this section, we briefly review the basic ingredients for Riemannian optimization, which are drawn from the standard literature, e.g., (Boothby 1975 Absil et al. 2008). Let M be a d-dimensional Riemannian manifold equipped with a Riemannian metric (--)) (7h Sx) + (Ne Sa), E R for any M, Mx &e T.M (when it is clear in the context, we omit the subscript and write (n.C) for short). For all z M, the tangent space T.M is d-dimensional linear space. The norm induced by the Riemannian metric in the tangent space T..M is |l = v 77) for all E T,M. An open ball centered
T with radius in T.M is denoted by B(n, = {5 E T.M T ||C nll < r). The union of all tangent spaces is tangent bundle, denoted by T.M. A vector field is a mapping which maps from M to T.M, formally defined by 1) M TM + 7/2 T. Given differentiable function f M + R, the Riemannian gradient of f. denoted by grad is vector
It is referred to website htwp:///ww...caaeooooooooooaooae
25

---

Published as a conference paper at ICLR 2026
field such that for any M, (2) is the unique vector satisfying Df(=) = (grad (z), for any 77 E T,M, where [n) is the directional derivative of f at x along 17.
A critical concept in Riemannian optimization is retraction, which defines smooth mapping, denoted by R, from the tangent bundle to the manifold, i.c., R. TM + M,, satisfying
1. R(O,) for all M, where 0g is the origin of M.M 2. [R( 7y for all 17 E T,M, which implies that DR(0,) idr,M with idr, MM being the identity in T.M.
When restricted to M, we denote R by Rg. i.e., R IT.M. Note that the domain of R does not need to be the whole tangent bundle. In practice, it is usually the case. In this paper, we always assume that R is well-defined whenever needed. A special retraction is the exponential mapping, dented by Exp, satisfying Exp,, 7(1) where is the geodesic such that 7(0) y0(0) 7m- Geodesic is the generalization of straight line in the Euclidean setting to the Riemannian setting, and naturally the exponential mapping is the generalization of addition to the Riemannian setting. Additionally, retraction is first-order approximation to the exponential mapping. A r-totally retractive set W is subset of M such that for any y W, it holds that W S Ry (B(0,, r)) and R, is a diffeomorphism on B(, r). Hence, R (v) is well-defined, whenever
and
E W.
For our RFedAGS, another essential concept is vector transport, denoted by T, which is usually associated with retraction R. Given retraction R, vector transport associated with R maps from TM @ TM, the Whitney sum, to T.M, i.e., T TM @ TM + T.M, and satisfies that for any (2r, na) E domain(R) and all Sa E M,M, the followings hold that
1. (Sx) E TR(..
2. To, =
3. Th is linear, i.e., for all a1 R and Eas So E T2M, it holds that Trg (arEa + a25x)
a Tm, (Sx) + 02 TnTT, (Cz).
We say T isometric if for any (z, nz) E domain(R), & E T,M, it satisfies (Tn, (Ex). Tm. (C.))) tor transport is the parallel transport, which is isometrie; refer to (Absil al. 2008 Boumal 2023)
Cx) which implies that [[T,, (Cx) || = |[ An important vec-
for the rigorous definition.
In the Euclidean setting, the convergence analyses of FedAvg established under the assumption that F is smooohhh where a continuously differentiable function ʃ : RR" - IR is said L-smooth if
[V+()) -V(()) s T.1x vll Vx, y IR"
in which case we have
f(0) < f(x) (Vf()),
-x) x) (V 1y - x||2.
Both properties above are critical in the analyses of FedAvg. Similar assumptions are made in the Riemannian setting for the analysis of the proposed RFedAGS; see Definitions [. (Huang et al. 2018) and I (Huang & Wei, 2022). The first one called L-Lipschitz continuously differentiable (Definitions B.1) and the second one is called L-retraction-smook (Definitions B.2)
Definition B.1 (L-Lipschitz continuous differentiabilityy. Let T be a vector transport associated with retraction R. A function f MM- + R is said L-Lipschitz continuous differentiable with respect to T on l C M if there exists a constant I. > 0 such that
Iggaad f (gradd((())]) & Lllnll
for all x E U and E MT satisfying y = (T).
Definition B.2 respect to retraction IR in N C M for any E N and any Nr S T.M satisfying R2(N) N.
function f M + R is called L-retraction-smoott with
holds that
L Inll?
(R), < f(x) (gradf(x),
for all 1) E NN.
26

---

Published as a conference paper at ICLR 2026
A function which is L-Lipschitz continuously differentiable is not necessarily E-retraction-smooodd however it is the case the Euclidean setting. It should be highlighted that there exist some cases where L-Lipschitz continuous differentiability implies also L-retraction smoothness (Huang et al. 2018 Boumal et al. 2019 [oouaa 2023).
Next review convexity and strong convexity in the Riemannian setting (Huang & Wei 2022
Definition B.3 (Strongly retraction-convex retraction-convex). A function f M IR is called u--ttronly retraction-conver with respect to a retraction R in N S M if any I E N and any Nr S T. satisfying Ra (N,)) S N, there exist constant p > 0 and a tangent vector ( E T.M
such that fx = fo o Rg satisfies
Sa(1) 2 (x(S) + (6,1 () + 11 E11 E112 Vn.5 N..
In particular, if p= = 0, we call| retraction-conver with respect to R N.
Note that Ç = gradf if f is differentiable; otherwise, Ç iss any Riemannian subgradient of at E. In literature, convexity has been studied based on geodesic; see, e.g., (Ferreira & Oliveira 2002 Zhang & Sral 2016), in which case a function ʃ .M +
called geodesic convex, if for any y, E M,, there exists a tangent vector Co T such that f(y) 2 f(x + (Gz, Exp (yy
can be verified if taking & = O and exponential mapping as the retraction in Definition B.3 then reduces to geodesic convexity.
retraction-convee
We end this section with an introduction to the concept of c-stationary points/solutions.
Definition B.4. We say that FT E M,, the output from Algorithm 7 is an e-stationary point of Problem 1.1) i holds that [[llgradF(())lll S e, or is an e-solution if it holds that [F(() F(x*) S 6, where x² E arg min F(x) F(z).
C ADDITIONAL DISCUSSIONS
C.I DISCUSSIONS FOR ASSUMPTIONS
Assumptions3.. 3.1 3.7 are standard for Riemannian stochastic gradient-based methods. Assumptions 3.1 imposes requirements for the retraction under consideration to be C² and the vector transport under consideration to be continuous and bounded from above. These requirements are fairly standard in Riemannian optimization. Note that the boundedness for vector transport can be achieved by requiring isometricness, in which case we have [T,, (Sx)) || = Szlll implying = 1. In fact, a lots of papers do have such requirements, e.g., (Sato et al.) 2019 Li & Ma 2023). Additionally, if the Riemannian manifold M is a submanifold embedded in a Euclidean space and equipped with the inner product as its Riemannian metric, then an option for vector transport is based on the orthogonal operation onto the tangent space, i.c., Tra (Sx) = (Sx) with (u( arg min,eT,,. || wlli.. in which case by the nonexpansivity of the orthogonal projection we have TT, (G.) S [| [|. also implying Y 1.
In the deterministic optimization, the compactness the sublevel set of the objective function is required ensure that the iterates generated by the algorithms which are monotonically decreasing are still located in that compact set. However, in the stochastic setting, it is difficult to ensure that the iterates generated by the algorithms all fall within the sublevel since the algorithms are not necessarily monotonically decreasing, and thus, it is not sufficient to require the sublevel set to be compact under stochastic optimization. In this case, Assumption 3.2 becomes a commonly
used choice in Riemannian stochastic optimization; see, e.g., (Bonnabel) 2013 Zhang & Sra 2016)
Tripuraneni et al. 2018: Sato al., 2019 Kasai et al.) [019 |Han & Gao 2021]
Li& & Ma (023). For
some manifolds that compact themself. e.g., the Stiefel manifold and the Grassmann manifold, the compactness assumption naturally holds. Moreover, in all experiments we conducted, it is observed that the generated iterates Te, with f 1, fall into the sublevel set {i E M: fxx) & f(z1))
Assumptions 3.6 and 3.7 impose requirements on the first- and second-order moments for the local stochastic gradient estimator, which are necessary for Riemannian/Euclidean stochastic gradient- based methods. the analyses for Euclidean federated learning algorithms, majority of works make extra assumptions for addressing the heterogeneity data. These assumptions essentially require that
27

---

Published as a conference paper at ICLR 2026
the divergence between local and global gradients is bounded, i.e., there exists constant > 0 such that for all 2,
V (z)) V
In our analyses for the proposed RFedAGS, we do not explicitly make the similar assumption, since Assumption 3.2 implies the counterpart requirement. Indeed, under Assumption 3.2 there exists
constant P > 0, such that (i)))] < P and llgrad S P for all E [ and W.
Hence, it holds that
llgrad gr)df(x)]1 < f 2llgaadd()) S 4P
Assumption 3.8 imposes the requirement that the approximate probabilities are how close to the true probabilities. As discussed in Section 3.3 when using frequencies the approximation, this assumption holds with high probability. Numerically, the reported results show that the performance using frrquencies is comparable to the
using true probabilities. We note that in the fixed step 2024) also makes an equivalent assumption. The difference lies in that the assumption in (Wang & Ji
size case, existing work (Wang & Ji
2024) only considers fixed step size cases, but Assumption 3.8
more finely encompasses the cases of decaying Step sizes.
In summary, except Assumption [38 that aims address the arbitrary partial participation, there exists no assumption beyond those made for Riemannian (stochastic) optimization and federated learning. In theory, the proposed RFedAGS is the first algorithm that can simultaneously address the challenges caused by the partial participation and the heterogeneity data settings. The partial participation under consideration allows arbitrary participation which more practical than the commonly-counter participation scheme based on random sampling. Even without the Riemannian manifold constraint, i... M RR,, the proposed RFedAGS can reduce one proposed (Wang & Ji] 2024). This paper establishes the convergence propoerties of RFedAGS under both the decaying (see Theorems 3.1 3.2 and 3.3) and fixed (see Theorems 3.4 and 3.5) step size cases. Under decaying step size case, vital and non-trivial observation (see Assumption 3.8). However, (Wang & Jil (022) only considered the assumption of the fixed step case, and thus only established convergence under the fixed step size case, which does not ensure global convergence rather only converges to a c-stationary point.
global convergence is guaranteed. These analyses depend
C.2 DISCUSSIONS FOR IMPLEMENTATIONS
In Algortihm[] there exists scenario (called NA) where in certain round of communication no agent participates in communication. We emphasize that this scenario happens with fairly low probability. For example, considering a FL system where agents participate in communication with probability
P: 0.1, = 1,2,... ... 20, and 5 agents participate with probability p; = 0.5, i 21, 22, 25. Then the scenario NA happens only with probability not greater than 0.38%. For the purpose of robustness, when the scenario NA happens, one option is set 2441 +- x, to restart the next round of local updates.
D PROOFS OF THEOREMS IN SECTION 3
D. SUPPORTING LEMMAS
If the objective F in Problem is Ly-reraaction smooth (Assumption 3.5 under Assumption 3.1
it follows that
[F( F(x)) < (. R R7 Lg E, I|
(D.1)
Without considering the arbitrary participation, recallin (M and (AGS-RS) we have
Exp 1ap=l(aa (xI-1) =
Expa. 1 (s/,x) and jES.
Σ
(D.2)
K-1
R (11-1) =
1
Σ
T
Σ S(lle
(D.3)
B,
jES,
k=0
BEBY
28

---

Published as a conference paper at ICLR 2026
When K > and S > 1. from the increment of parameters of (TM) it follows that analyzing the upper bounds of the two terms in the right-hand side of (D.I) is fairly challenging, since the nonlinearity of exponential and its inverse leads to difficulty to expand (D.2) into the desired one involved gradient information. However, the form of (D.3) is very similar to the Fuclidean version and thus significantly address the issue.
Lemmas D.I together with D.3 have provided an upper bound for the first term in the right-hand side of D.I.
Lemma D.1. Under Assumptions and a batchsize B,, we have that
3.1 3.5
at the t-th outer iteration of Algorithm I with a stepsize an
K-1
(grad Rz (1x+1) )))l (
1309
max
Σ
Ee
E.ll
(12
Da
K-1
wax IgradF (x:)11
E
Pj
T (gradf
(D.4)
2
2
de N
k=U
where 67 max
+ N
and 82
N
N
jj
Proof of Lemma D.1 On the one hand, we have
EeKgdddF (xx), R (m1+1)) (()(()))))
= Egladd (zt), RT, (1)+1) + (graaFF)((as)) war
= --wa,K | [r (x:))] )|1 + [grad (R,), (xx-1) + wa, (gradf(())))
(D.5)
where for the second term of the equality the right-hand side, we have
Ey (gradF (xt), R,, (x(+1) + oa,KgradFF( (I:))]
E. ( gradF((x)) (xx), -
K-1
(4r St,o)) + (raddF())(..)) |
W0
T
B
JES,
k=0
BEB(,
( (rradF(x))
K
T (z), k..,))) + gradF(x:)) )
E.
Σ way
Σ
Σ
&N
k=
jES:
BEB
K-1
Σ
E,
gradd(()))
-w0t
(grad5)(r)..))
P3 P3N N (rad((((())
1
N.N
jES.
K
[ ( -
Σ
E,
Vwat
(T (grad) - - grad ((aa)
N
P3
JES.
K-1
Σ
E, (
wwr.grddF
N
Is, ())
/aar
(TT
TH (gradf - gradl
N
k=
#=1
K-1
N
Σ
Es
woigradF(xt)
Pi wa
T mg.
(grad/ (x( grad
N
j=1
K
N
KarK (gradF(xx))12
wa
E,
T (grad
grad)
2
k=0
j=1
N
ge
Pj
K-1
30
Ex
Pj
(grad f3 (of
(D.6)
2
q
N
k=
29

---

Published as a conference paper at ICLR 2026
where the third equality follows (2.1 the sixth equality follows B[Is, (j)] Py- and the last equality is due to (u, 2)
= (1|u]] + ||ll u =112). Morcover, we note that
1 gradf (1x)) |
ΣE
E
Pj
THe (gradf, (x(,))
Dj
K-1
Σ
E.
NV
Pj
T
(xt, gradf + (- T
1
gzadff
N
Pj
K-1
N
$ 2
Σ
E.
Σ
Pj
TTiz
(gradf(a),,)) gradff
N
A=
+2
Σ
E₂
Σ
Dj
-
1
)
Pj
1=1
2L
N
P1
K
E. I (1t
N (
1/
N
N
R-1
S L3 max
E. R (a (a9,,))19 + 222,
(D.7)
where the first inequality follows ||u + u11 < 2/ + 2/|011 2 the second inequality is due to the T Ly-tetraction smoothness of grad for 1....... N. Assumption 3.2 (which implies that there
exists P > 0 such that figradf((1) (Ix) I P), 3.4 and 3.8 and the third inequality follows that
2
of max
N
21
and 03 = E N Combining D.5), (D6 and (D.7) yields the
N
desired result.
In order to further bound EE, (x.) (xx+ for K 1, from Lemma D.1 necessary to estimate the bounds for E, [[[[ (x²,.) )1 as theoretically discussed in Lemma D.3 which states that for agent j. the "distance" between the k-th local update -.,n and the the f-th outer iterate de are controlled by the sum of squared step sizes. Intuitively, the "distance" increases as the number of local iterations grows, which is shown in Lemma D.4 Meanwhile, it also reflects the drift between an agent's 's local update parameter oi,x and the global parameter T₁. A general result is provided in Lemma D.2
Lemma D.2. Under Assumptions 3.3 let F M R be a smooth function. consider the following update formulation
JtA Rea, (-atAGr((Tt.))
where (F(,)) is an estimator of gradF((x..). xt = It,0, and os,r is the step size, then it follows that
k-1
[R (x,,))
2k
Σ 2 (J2 + (I,,IP
where J and H are two positive constants related with the manifold and retraction.
The proof of Lemma needs the following inverse function theorem on manifolds.
Theorem D.1 (Inverse function theorem). Given smooth mapping P P + M' defined between two manifolds, DP(x) invertible at some point æ E M, then there exist neighborhoods Us C M of x and Vr(a) M' of P(x) such that Plu. U2 - VP(x) is diffeomorphism. Meanwhile, if p-1 is the inverse of P in Uz. then we have (DP(x)) DP-1(P((a))
Now are ready to prove Lemma D.2
30

---

Published as a conference paper at ICLR 2026
Proof of Lemma D.2 For two points 2, y E W, consider the map Pz.y RRy" o R T + T TyM: The R. (Re()) which is defined between vector spaces. According the chain rule for the differential of a map and the first-order property of the retraction, i.e., (R,( ITM. we have
(R2 DP2.8 2,3 (0₂) D(R D (0,))
DR, (02)
(R, (ka))) ITMM
(A
where the third equality is due to the inverse function Theorem D.1 Noting that the map (-) is defined in Tw = g. y W. 17 E (,-(W)), which is inside a compact set, according to Assumption 3.2 thus, smoothness of the retraction implies that the Jacobin and Hessian of (.) with respect to the third variable is uniformly bounded in norm on the compact set. We, thus, use C2, C2 denote bounds on the operator norms of the Jacobin and Hessian (-) with respect to the third variable in the compact set. Noting that
Prlja-122 (Me) )- = R (R (Re, (Mei, = Ryd (x7,)); and 1 (0) = -Rz, R,,,(R,, /
1 (0)) = R (o)
with na
-4, 100 using Taylor expansion for Pa., yields
(x( = Pai (-01A-100
-P (0) + DP
2 k- (0 + ai ai,--1c
=R - ( at
(gr(tt, a at
where || 1C' (9 ( Hence, repeatedly, we have
R R (12
Σ Oi (A
) -Gr (x(,))
Σ 01.6
(D.8)
where we used RT, (xr) Oz.. Combining D.8 [A (for all 1,2,.... desired result.
To -
(Gr( -- C2l1g
and 1,2,.... 1), and E uill 2 S Di. ||uill yields the
When M reduces into a Euclidean space, e.g., M = Rd, the constants in Lemma D.2 will be-
come C2 = and C3 = 0. In this case, the results correspondingly becomes || - t ll <
+ F ( ).? In Lemma D.2 one uses 1
B. EneR, gradf, (a) (1) Si,,,) to replace
Gr(#) (,),)) then the desired result is obtained in Lemma D.3
Lemma D.3. Under Assumptions 3.1 3.3 at the k-th inner iteration of the 1-th outer iteration of Algorithm Z for each agent E St and k = .., K
we have
[R (1),)) < 2k p2 + 22²22
(D.9)
where is positive constant such that for all æ W. 1.2,..., N and & ~ D,, holds that IggradF(z) < P, |lgrad (x)) < P and [lgrad (z; {)ll < P by Assumption 3.2
Proof of Lemma D.3
From Algorithm
letting (r(1(x)
1 B. Ebert, grad (2) el C...)))
then, have
of 1 Σ gradf
5
1
Σ I
&l s P.
B.
B
BEB
BEB
Hence, combining the inequality above and Lemma
D.2 gives rise the desired result D.9
31

---

Published as a conference paper at ICLR 2026
Under the same conditions as Lemma D.1
plugging (D9 into (.4) yields
K-1
(a)) (a+1))
w0
Σ k=0
E
Σ
Pj
T
N
K
2 Igrad
0 (2K - 1)KKK -11 + 22)22H)
+ K
(D.10)
The next bound the second term E, R (2)+1) 10++1)11 )|
Lemma D.A. Under Assumptions 3.1 3.13.8 3.8 the iterates {xx ( generated by Algorithm
1 with fixed
stepsize an and fixed batchsize B3 within parallel inner iterations satisfies
K-1 Σ
E R. (F
+ waRKK
E.
Σ
Pj
T (gradf, (2)
B.
N
+ a p2 k2 83
(D.11)
where 83 = No
and 83 N² Σك=
(1(pp))) (1)
1=
Proof of Lemma D.4
Let Ty denote the t-th aggregation by the server. Then,
E, RT, 10
E.
Σ
1
gradf, shAe))
|]]
Tf
N
B
jES,
R=
BEB t.k
K-1
=
Σ
Ts. (1)
Ts
1
Σ
grad fj (5t (lA.o))
N
J=1
K=
bEB
K-1
Σ Us, (
Σ
1
Σ
grad fy kt (rad))
N
Mi.k
B
k=
BEB
+ grad fy (x
K 1s. (i)
- )
a
Σ
TAi.
Σ
grad (1), Sl. grad(,
B
k=
bEBs
K-1
+ 2
Σ
Us. (
Tril.
gr
C N
j=1
k=0
N
(7)
K-1 1
grad (1 S.M grads ((r.)) |
= aa2
Σ
-grdd
B
k=0
bEB
N
K
+ t020
Σ
(us, (i) Pj + p,)
Tri (radds(zt.))
d NN
k=0
2 af T2 of K
N
K
Tet (grad (3( 27
Pj -
~ aa 0 E
Σ
(1s) - -p;)
Σ
NBB,
2
of N
/=1
k=0
Pj
Σ
T n
(grad
N
32

---

Published as a conference paper at ICLR 2026
3 T
N
Pj
a
N
pf(1 ps)
Be
K-1
TR. (grad (x
NºB,
N
(gt )2
==1
==1
K-1
+
Pj
(gradf
N
k=
3² 2
o
B
Pj Tat. (crad
a N
k=
where the fourth equality follows that
1
Is. (i) TT (or,
Σ
gradf,
Te N
Be
of N
BEB A.
and that E ||||
|
= B1||
Blu]11? |B[u]] the first incquality follows that
N K- Us. (i)
Σ
T
Σ
grad (27, (,, gradff
of N
B
BEB
and that [8[
Σ R
u.l
E
Bl 1 with u, being independent and having zero mean, that
[T (S) S T (Assumption 3.1 and Assumption 3.6 the sixth equality follows that
E
(Tss(t))
-p;)
ΣT,
T (gradf,(z(.
N k=0
and that B[(Ls, (j) p]) N
=Dy(L pj), and the last inequality follows that 83
max
1
Ex=1 (2)) f
P1
82 max421 { N Di=1 (x))) j.
N p=(1-p,)) }.
and Em=1u uil?
No
n Ex= ||/.|12.
Now can formally state the descent lemma in the Riemannian FL setting.
Lemma D.5. Under Assumptions 3.7 3.8 we run Algorithm 7 with batch size B, and step sizes
w > and {ar] satisfying
K T. w0..
(D.12)
Then, we have
work K
[.[F(r++1) -F(ss))))))(()) < |gradF(x:)] kakK
(D.13)
2
where Q(K, ((K,B,,a, 04;
B,, w)
(2K 1)(K 1)1388pp2 + afp² )./ GP
T L2 +
L, T"=
2B.
Proof of Lemma D.5
By the -eterccciio smoothness of F, it follows for t 2 that
Lg
F(141) & (xx)) + ( (I(+1))
+
R ( (11+
2
where the existence of R (2141) is guaranteed by Assumption sides over the randomness over the t-th outer iteration yields
3.2
Taking expectation on both
F.(F( < F(x) + Ea((gradF (2e+1)) Lg E (241))
(D.14)
Inequality equation D.14 together with Lemmas D.I
D.3 and D.4
give rise to
E, [F(F44411 F(x.) < w0,K K
KI 2663
(g:)d + a
2
2B,
33

---

Published as a conference paper at ICLR 2026
(2K
[)K(K -
13 + H
K-1
5a
K w
Σ
R
Σ
T (fradss(f
Pj
(D.15)
2
N
k=
Under Condition D.12). the third term on the right-hand side of (D.15) can be discarded and then obtain
w0 K
grad F(x))l + og KLgo
[.(F(()+1)] - F(a.) v
2
2Bc
02 (2K 1)K(K -)) + 2 Ha
+
sa,K
IgradF(x;) (x;) 122 + wu,KQ(K B,, w)
2
where Q(K, B1, w))
(2K 1)(K 1) + a2P²H')a//6
GP
T2 + 1,,30
2B.
Note that Q(K, B₁, Ct, ②) in D.13 consists of four enror terms: the first one resulted from the agent drift effect and non-I.I.D, setting, the second one brought by the probability approximating, the third one caused by partial participation, and the fourth one caused by the local stochastic gradient.
D.2 PROOF OF THEOREM 3
Now are ready to prove Theorem 3.1
Theorem 3.2
The second condition in (3.1) ensure { - 0, and thus, without loss of generality, may assume that Tyg K wa,, < for all E N4. Then, follows from D.5 D.5 that
(x)(()) + ajQ(K, Be, at w) at (x:lgradF(x))) 2(F((1+1)
Kw
Summing the inequality above overt 1,2,.... T and taking total expectation yields
2E(((r)) - F(TT+1))
Σ algglaad
Σ o Q)Q( B,, 04 w)
Kw
2(F(ro) F(+)))
B.
K
Dividing the both side by A7 ΣT T squared gradients as follows - 04 results in the bound for the weighted average norm of the
T Σ (lggaaaaaF(
2(F(rr) F(x*))
T +
Σ
QK(, B,,
(D.16)
A1
KEAr
Ay 1=
which, under Conditions
implies that
lim Σ or Bllggaaa T-000 Ar
The desired result follows the fact above.
34

---

Published as a conference paper at ICLR 2026
D.3 PROOF OF THEOREM 3.2
Theorem 3.2 By the definition of Or,, there exists a positive constant M > 0 such that
Et= 2 Ei= a Ei= aj Ei of S M for all T 2 1. Then,
QiQ(K, Blow; Og, 3) 0 (2K - 1)(K -) 1) P
L L
+ GPP88M G g
M.
(D.17)
BBow
On the other hand,
Ar
0g
T+1
0
dt
+) 1m p= (1++ - (b+
(B
+ 1)p
(3 + 1)
Pe (1/2,1),
t=1
which gives
1
([d[[[T+44))-m((tt)) co((T+++p)) P P (b+1)1>) p E (1/2, 1).
p = 1,
(D.18)
Ar
Plugging D.17) and (D.18) into D.16 ensures the desired result.
In particular, if full agents participate in any round communication and agents use local full gradient in local updates, implying G 0, o O, and of 0, then have
T
1 Q,Q, at x) (2K 1)( 1)L308 2
Σ ot +
2
H
Σ
Hence, we can relax the condition for a; as Σe= at xa and Σe= of < oc. If one takes
ag (B+1)
with constants ag, 3,p 1/3 + and (0,2/3) properly small, it follows that
M(a) (3 + 2/3/3-a'
AT
where (a) is constant depended on The smaller the larger M (a).
D.4
PROOF OF THEOREM
Theorem
3.3
By Lemma D.5 and the RPL condition, we have
[. (F()))
S Plo Blow; Ccty w)
Rearranging this inequality yields
- [[F((++1)] F(=") I (1 - (=F Ko, Blow 01, w), (D.19)
where we take the total expectation on both sides. Subsequently, we prove the desired result by induction. For t = 1, it follows from the definition of v. Now assume that (3.2 holds for t > 1. Then, from (D.19 12. it follows that
BuoK
KKB2
E B[F(x41)) F(x*) S
Q(K, Blow (1, w)
Buui K
KB
V
QKK, (K Blow 01, w)
| K -
KK3 v
D
Q(K, B Blow. a1, w)
f2
12
(2
(D.20)
t 1'
35

---

Published as a conference paper at ICLR 2026
where t = 7 + t, the last inequality due to definition of v and 2 (t - 1)(t+ + 1).
[unnk p
xK3² v
Q(K, Blow) S 0 by the
On the other hand, for any two points x, W, it follows from the Lg -smoothness of F that
F(y) - F(x) + (gradF(x). R (v)) + Lg |R_ (v)1
2
Plugging R.r 1 L, grad F (x)) into the inequality above yields
F(x") S Fly)= S F(x) 2Lg
(ggaad
which gives ž [gradF(()] S F(æ) F F(x"). Replacing æ with as and plugging the replaced
inequality into Inequality (D220) yields
24g
B[lgraddF(x)))1 7 +t
which completes the proof.
D.5 PROOF OF THEOREM 3.4
Here rewrite Theorem 3.5 the following more complete statement.
Theorem D.2. Suppose that Assumptions 3.7 2.8hlld. We run Algorithm 7 with fixed global step size w, a fixed batch size B, and a fixed number of local updates K.
1. If the fixed step sizes a and wo satisfy L S then
20(x1)
T Σ
Elllerad(a:)111
waKT + 2aQ(K, B,,,
(D.21)
If local full gradient descent step is performed local updates, i.e., ag and one takes local fixed step size a > O such that P- +T kL, KKT with T satisfying (KL)0(21) 2P (Ga) K 188
((z))
((=))K-1))) 1) 13 +2 p²
2 max
then
T2PLL, kGo
(( 4P 20(z))
G5
Lg T382
wKT
T
3. If the true probabilities are known, meaning G 0. and one takes local and global step sizes a and w such that aw (6 @(a18
42P with satisfying 2
KL, B max (6 2P4 S
then
030 KTB
2P T
BlllgradF < 4T Lg0(z1)
Proof.
Item
Using a, a and B, B in Lemma D.5 we have
Fl 2E[F(x,) F F(1+11)] + 2aQ(K, B, w)
E
Sak
Summing the inequality above over t. 1,2...., gives rise to
T
2E (an) F(IT+1)) + 2aQ(K, B,
E [[ld)aaF(x)))111
T
waKT
2(F(xo) - F(x*)) + 2aQ B.a,
DaKT
36

---

Published as a conference paper at ICLR 2026
where the last inequality follows F(x") F(1r+1). In particular, suppose that let a and w satisfy
Item
(2K - 1)(K - 1) (J² + a Pd a GP KK
(D.22)
6
Define
h(a) oakT
28(z))
+ 4aGP 82 4T p2 KLy Solving a² - arg milla h(a) results in
(
Gd3
Lg TgT
V 2T + L and h(aa) 4P 20(z1)
wKT
T
Taking
w wKIZ((m))
+ - ( + p2 112
T max
(G2 + K k
T2 + T2))
can ensure that a²= LL I 1 and that (D22 holds. Hence, the left-hand side of (D21) is not greater than h(a"). The proof for Item 3 similar to that for Item 2
if the probabilities pi are known, i.e., qf Pi and Pmnin = min; [pi} is not too small and not fairly far away from Pmax = maxx (pi). Item 2tem gives the
Remark D.1. Continuing with Remark 3.3
upper bound as O
1 VT In particular, pf N with S N, the upper bound
becomes O
VST
D.6 PROOF OF THEOREM 3.5
Theorem 3.5
Using a fixed stepsize a; = a S 1/(wwk) satisfying Condition (D122 and batchsize
BI E Blow B it follows from (D.19 that
B[ - -(x") S
(1- w Blow, a, w),
which implies that
B[F(xT)] F S (1- + wa KQ(K, Blow, a, w)
$ (1 - +
+ KQ(K, Blow l, w)
T-1
s (BI
Σ
(1 μw Ka) wa KQ(K, Blow a2 w)
(1-1 Ka)
= (1 (1-
w KQ KQ(K, Blow; a, w)
N kwaKa Ka a Q((, Blow, a,
< pa
which completes the proof.
D.7 PROOF OF THEOREM 3.6
Theorem 3.6 Let f t --1 1. Restricting of E [p,/2, 3p./2p 2| yields PflØ -pal S p./2) 1
min (2c /2 4(1 -2
(fpp) by the Hoeffding's and Chebyshev's inequalities. Then
1
9: P
2
lgi
Pil
ot Pi q.P:
p?
holds with probability not less than 1 1 -min(2e [p:/2, 3p,///.
12. 12,4(11 (fp:). Noting that under of
2 of pil < -t2 (i.e., lai pal o/2 implies ((g)) - p, Gt-0/2
and that p.
min Opptie- it 4(1 -p-pp))
papt
P lgi pil S 20/2 a/2 2 m-minn
'2
where we use the Hoeffding's 's and Chebyshev's inequalities again. Let == -1 Gt at=a/ B = {q E [p./2, / and C := {lai p.l < { a/2). The desired result follows
BnC and P() > 1 P(B) P(()).
37

---

Published as a conference paper at ICLR 2026
E SUPPLEMENTARY PROOFS
E.1 PROOF OF THEOREM 2.1
Lemma E.1. Let 3ι, 22,42, 22,..,, I2 ~ Bernoull(pp)). Then,
EN be independent Bernoulli random variables with pt i.e.
N
E
II
pi + p:()dl.
1+ > mm
J
1=1
Proof. Let = Σx Tt. Considering that for any a > 0, it follows 1 -0e e-lld.. Picking 1 S yields
e e
Ei=
1+S
Taking expectation for both sides of the equality above, we have
E Joe e e-
E
-1=1
dt
11+
where the second equality is due to that e eSt St a discrete random variable. Since æ; is independent
and 5 = Σi= Tú, follows TEee St1 = II Bee-= Noting that [[ pie + (1 - pt),
obtain E/e St II (p.e t + 0, and u + 0 as Co. Hence,
p.)). Finally, let e- Then du - 1 as
| e Ele
St dl
II
pt + pru)du;
which completes the proof.
Now we are ready to prove Theorem 2.1
Proof of Theorem 2.7 At the t-th outer iteration, S, denotes the indices set of agents who send their
{ 0
E S₁ Then
gradient streams to the server. Let 5; =
i $ S..
((()
Σ grad gradf((x)R
(E.1)
St
N Ei=1
J.S.
Noting that E ΣN ai
E
E
P.E
Since I, Bernoullip,)) is
independent, by Lemma
E.1
we have E
So II
I (1 Ps + ptt)dt. Plugging these
intermediate results into E. leads to the desired result.
F.2 PROOF OF THE CLAIM IN REMARK 3.4
In general, is difficult verify directly whether the objective function satisfies the PL (in the Euclidean setting) or RPL (in the Riemannian setting) property. There are some stronger but useful sufficient conditions that imply or RPL condition. Specifically, in the Euclidean setting, strongly convex function satisfies the PL. condition (Bottou et al.) 2018)). Similarly, in the Riemannian setting, the geodesic strong convexity of real-valued functions implies the RPL property (Boumal 2023). However, geodesic strong convexity usually requires the use of exponential mapping and its inverse, whose the closed-form expression is not available in some manifolds, e.g., the Stiefel manifold. the next theorem, retraction-come cutraction-covvxxity. in the Riemannian setting than geodesic strong convexity and claim that strongly retraction-convex function also satisfies RPL condition.
more general notion of the strong convexity of real-valued functions strong
38

---

Published as a conference paper at ICLR 2026
Theorem E.1. Suppose that function q .M - R is twice continuously differentiable and p- strongly retraction-con convex with respect to the retraction R on W M, which is totally retractive neighborhood of x², minimizer of q on W. Then,
((x) (xx)) = |graag(())1
2p
that q satisfies the RPL condition W.
Proof. From the proof of Huang al. 20055 Lemma 3.2), the u-strongly retraction-convexity of implies that
( ((x) 2 (gradq(x). n) |17l1
(E.2)
for any E W, E W., M, and = R₂(r) E W. Define Zee (1) = ((x) + (gradg(x)) n) + M/l with 17 T, which p-strongly convex with respect to 77 (in classical), implying that the unique minimizer of is given by 1/' gradd(()). Thus, M. (x Oz (x("7)
((x) [ggrad(()))11 follows from E.2 that
q(x") ) 2 ((z) + (gradg(x), n) + " |nl 2 qx(11) = ((x) ggradg(>)]
2
2p
which completes the proof.
39


Reviews:
[{"review_id": "BMszM2Agv4", "reviewer": "Reviewer_zBLo", "summary": "This paper proposes a new federated optimization algorithm for learning over Riemannian manifolds. The proposed RFedAGS allows efficient and theoretically sound updates even under partial participation and non-IID data. The authors provide convergence guarantees and validate RFedAGS on synthetic and real datasets, showing consistent improvements over baselines.", "questions": "See questions in the weaknesses part.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* The proposed aggregation mechanism is novel and easy to follow.\n* The paper provides comprehensive convergence analysis.\n* The main experiments effectively demonstrate the proposed method’s effectiveness.", "weaknesses": "* While I understand the reasonableness of $G$, I am wondering what the value of $G$ would be when the true probabilities are not available to the server in the experiments.\n* How are the data partitioned across clients? How many total clients are included in the experiments, and what is the client participation ratio?\n* The ablation study is somewhat limited, and the sensitivity of several important parameters is missing—for example, different participation ratios, varying numbers of local steps, and comparisons between using approximate probabilities and true probabilities.\n* The assumption of Lipschitz continuity for each $f_i$ seems a bit strong, although it may be necessary for the Riemannian SGD convergence analysis. I am also curious whether this Lipschitz continuity can be empirically verified in the experiments.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "U3keTa78D9", "reviewer": "Reviewer_mU3X", "summary": "This paper proposes RFedAGS, a new Riemannian federated learning algorithm that replaces parameter averaging with gradient-stream aggregation to preserve linearity in tangent spaces and improve computational efficiency. RFedAGS proposes an efficient retraction-based aggregation which eliminates the need for the computationally burdensome inverse exponential map or parallel transports required by prior Riemannian FL methods, and theoretically establishes global (and sublinear/linear neighborhood) convergence under various step size regimes. Extensive experimental results are provided across synthetic and real-world tasks, demonstrating RFedAGS’ advantages over competing Riemannian FL baselines under general participation and data heterogeneity.", "questions": "1. Can the authors provide empirical or theoretical discussion regarding the scalability of the method as the number of agents, local dataset size, or manifold dimension increases? \n2. Could the method be efficiently applied to other manifolds? Are there limitations?\n3. Could the method be compared with recent or advanced Riemannian federated learning algorithms (e.g., Wang et al., 2025 [1]) ?\n4. The paper claims computational efficiency due to the removal of exponential/logarithmic maps, yet provides no quantitative analysis.\nCould the authors offer detailed communication and computation cost metrics per round, beyond total CPU time, to support this claim?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "This paper proposes and analyzes RFedAGS, a Riemannian federated learning algorithm that introduces a new and efficient server aggregation scheme based on averaging gradient streams.\n\nThe method is designed to effectively handle both partial client participation and data heterogeneity. Theoretical analysis establishes that RFedAGS achieves global convergence and a sublinear convergence rate under decaying step sizes, and further converges sublinearly or linearly to a neighborhood of a stationary point or solution when using fixed step sizes. Extensive experiments on both synthetic and real-world datasets demonstrate the strong empirical performance and stability of the proposed approach.", "weaknesses": "1. Limited novelty. The key idea—aggregating gradient flows in tangent space—is conceptually straightforward once the FedAvg update is projected to a manifold setting. \n2. The paper lacks a argument for why RFedAGS offers a distinct or superior geometric interpretation.\n3. Limited Scope of Baselines: Although several strong Riemannian FL baselines are included (RFedAvg, RFedSVRG, RFedProj) for targeted tasks, more recent algorithms are not considered, e.g., Wang et al., 2025 [1].\n4. Some results tied too closely to specific manifolds. Experiments and implementation notes (Appendix A.3) are mostly focused specific manifolds. Broader applicability to more exotic or high-dimensional manifolds remains an open question\n5. Although the authors claim computational efficiency (no need for exponential/logarithmic maps), no quantitative results support this.\n\n[1] Wang H, Pan Z, He C, et al. Federated Learning on Riemannian Manifolds: A Gradient-Free Projection-Based Approach[J]. arXiv preprint arXiv:2507.22855, 2025.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "fbEFfPmaVx", "reviewer": "Reviewer_mABS", "summary": "They propose RFedAGS, a Riemannian Federated Learning algorithm that aggregates gradient streams instead of model parameters to handle manifold curvature and arbitrary partial participation. The method introduces a new aggregation correction AGS-AP, supported by convergence proofs under generalized retraction and bounded vector transport assumptions. Experimental results on several manifold-based tasks  show consistent improvement over prior Riemannian FL baselines.", "questions": "1. Mislabels algorithm in line 438.\n\n2. In A.1.4 it seems like the effect of heterogeneity is almost unseen as the convergence improves consistently when K increases. Is it possible to show results for K > 10? Since the algorithm is not designed to mitigate heterogeneity, there should be a certain level of performance degradation observed with extremely large K.", "limitations": "", "rating": 4, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The algorithm avoids the computational burden of exponential map inverses and parallel transport used in earlier Riemannian FL methods.\n\n2. It is the first Riemannian FL method proven to work under arbitrary partial participation.", "weaknesses": "1. Theoretical clarity and novelty: While the proposed framework claims to generalize existing Riemannian FL methods by relaxing the requirements on retraction and vector transport, the theoretical advancement remains unclear. Specifically, the main difficulty in proving convergence under assumptions like 3.1, 3.2, and 3.5 is not explicitly articulated. The authors should clarify why convergence analysis becomes more challenging under generalized retraction and bounded vector transport, and in what way their proof techniques go beyond those established in prior works. In other words, the paper should highlight which parts of the analysis cannot be handled by the existing Riemannian FL theoretical tools and why this generalization is nontrivial.\n\n2. Significance of AGS-AP extension: The transition from AGS-RS to AGS-AP appears to be a relatively straightforward correction that compensates for non-uniform participation probabilities by reweighting expectations. While this adjustment enables handling arbitrary participation, it is not evident that it introduces fundamentally new theoretical challenges. The proposed fix seems more like an incremental adaptation rather than a substantial methodological contribution. The authors should therefore elaborate on why the treatment of partial participation in the Riemannian context poses unique analytical obstacles that cannot be addressed by simply adapting existing Euclidean analyses with weighted expectations.\n\nIn summary, the current formulation does not convincingly demonstrate that the proposed theoretical extensions constitute a significant leap beyond existing works. The authors are encouraged to explicitly contrast their convergence analysis with prior proofs, detailing where prior methods would fail or become inapplicable under their new setting.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "GjQkJNoZ3A", "reviewer": "Reviewer_iW4m", "summary": "The authors propose RFedAGS, a federated learning algorithm optimized for optimization on Riemannian manifolds under realistic scenarios of partial client participation and non-IID data. The method introduces a server aggregation strategy called Averaging Gradient Streams (AGS), which averages transported gradient information instead of local model parameters. This approach addresses the non-linearity of manifolds and avoids costly exponential map computations. The authors theoretically demonstrate global and sublinear convergence under decaying step sizes, as well as sublinear/linear convergence to a neighborhood under fixed step sizes. These results are supported by an assumption that holds with high probability as training progresses. Extensive experiments on both synthetic and real-world manifold-structured datasets showcase RFedAGS’s ability to achieve superior accuracy and faster convergence compared to existing Riemannian FL baselines.", "questions": "1. The convergence proofs rely heavily on Assumption 3.8, which bounds the deviation between estimated and true participation probabilities. However, it remains unclear how $q_{i,t}$ is actually computed during training.\n- Are these probabilities updated as empirical participation frequencies over rounds, or are they fixed a priori?\n- How sensitive is RFedAGS to inaccurate or time-varying participation estimates (e.g., if some clients drop out permanently)?\n\n2. While the theory emphasizes arbitrary participation and heterogeneous data, the experiments do not explicitly test these conditions.\n- Could the authors provide additional experiments that vary (a) the proportion of participating clients per round and (b) the degree of data heterogeneity across clients?\n- How does RFedAGS perform compared to baselines as participation becomes sparse or data distributions diverge?\n\n3. The proposed AGS framework involves transporting and averaging gradients in the manifold’s tangent space, which may introduce additional computational overhead compared to standard Riemannian FedAvg.\n- How does this affect runtime and communication efficiency when the number of clients or model dimensionality scales up?\n- Are there specific manifolds (e.g., Stiefel or SPD) where vector transport becomes a bottleneck?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- The paper introduces an aggregation mechanism, Averaging Gradient Streams (AGS), which avoids direct parameter averaging on curved manifolds by accumulating and transporting gradient updates in a common tangent space. This design elegantly resolves the geometric inconsistency inherent in prior Riemannian FL methods and extends federated optimization to non-Euclidean domains under partial client participation—a setting unexplored before.\n\n- The work demonstrates theoretical depth and rigor, providing global and sublinear convergence guarantees under decaying step sizes and linear convergence under Riemannian PL/strong convexity conditions. The proofs are comprehensive, grounded in standard FL and manifold optimization assumptions, and even include a non-trivial probabilistic justification (Assumption 3.8) for estimating client participation probabilities.\n\n- Despite the technical complexity, the paper is logically structured, moving smoothly from motivation to algorithm design, theory, and experiments. The authors provide sufficient mathematical background, clear notation, and consistent use of terminology. While additional intuition could help non-specialists, the exposition is coherent and reproducible, with algorithmic and experimental details clearly documented.\n\n- The proposed RFedAGS algorithm broadens the applicability of federated learning to manifold-constrained models (e.g., SPD matrices, hyperbolic embeddings, Stiefel subspaces), an area of high emerging interest.", "weaknesses": "- While the paper is theoretically rigorous, its dense and mathematically demanding nature may hinder its accessibility to a broader audience at ICLR. Consequently, many critical ideas, such as the geometric intuition behind averaging gradient streams and how vector transport preserves consistency, are primarily presented in formal notation.\n\n- Although the paper claims to handle arbitrary participation and non-IID data, the experiments do not explicitly vary these factors to show robustness. It remains unclear how the algorithm performs under different participation rates or degrees of client heterogeneity.\n\n- The convergence proof relies on Assumption 3.8, which establishes a bound on the estimation error between the true and empirical participation probabilities. While this assumption is theoretically justified, the paper lacks a clear description of how these probabilities are computed in practice and how sensitive the algorithm is to their inaccuracies.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)