The Gaussian-Head OFL Family: One-Shot Federated Learning from Client Global Statistics
Authors: Fabio Turazza
Marco Picone
Marco Mamei
Venue: ICLR 2026 Poster (2026)
Keywords: One-Shot Federated Learning
data-free aggregation
Gaussian Discriminant Heads
Knowledge Distillation
Primary area: other topics in machine learning (i.e., none of the above)
TL;DR: We present GH-OFL, a one-shot federated learning family where clients send only per-class counts and moments; the server builds Gaussian heads achieving data-free SOTA accuracy under strong non-IID.

Abstract: Classical Federated Learning relies on a multi-round iterative process of model exchange and aggregation between server and clients, with high communication costs and privacy risks from repeated model transmissions. In contrast, one-shot federated learning (OFL) alleviates these limitations by reducing communication to a single round, thereby lowering overhead and enhancing practical deployability. Nevertheless, most existing one-shot approaches remain either impractical or constrained, for example, they often depend on the availability of a public dataset, assume homogeneous client models, or require uploading additional data or model information. To overcome these issues, we introduce the Gaussian-Head OFL (GH-OFL) family, a suite of one-shot federated methods that assume class-conditional Gaussianity of pretrained embeddings. Clients transmit only sufficient statistics (per-class counts and first/second-order moments) and the server builds heads via three components: (i) Closed-form Gaussian heads (NB/LDA/QDA) computed directly from the received statistics; (ii) FisherMix, a linear head trained on synthetic samples drawn in an estimated Fisher subspace; and (iii) Proto-Hyper, a lightweight low-rank residual head that refines Gaussian logits via knowledge distillation on those synthetic samples. In our experiments, GH-OFL methods deliver state-of-the-art robustness and accuracy under strong non-IID skew while remaining strictly data-free.

Published as a conference paper at ICLR 2026
THE GAUSSIAN-HEAD OFL FAMILY: ONE-SHOT FEDERATED LEARNING FROM CLIENT GLOBAL STATISTICS
Fabio Turazza',, Marco Piconel:2, Marco Mamei
1 *Dpartment of Sciences and Methods for Engineering
2 Artificial Intelligence Research and Innovation Center University of Modena and Reggio Emilia, Reggio Emilia, Italy name. surname @unimore.
ABSTRACT
Classical Federated Learning relies on multi-round iterative process of model exchange and aggregation between server and clients, with high communication costs and privacy risks from repeated model transmissions. contrast, one-shot federated learning (OFL) alleviates these limitations by reducing communication to single round, thereby lowering overhead and enhancing practical deploya- bility. Nevertheless, most existing one-shot approaches remain either impracti- cal or constrained, for example, they often depend on the availability of a public dataset, assume homogeneous client models require uploading additional data or model information. To overcome these issues, we introduce the Gaussian- -eadd OFL (GH-OFL) family, a suite of one-shot federated methods that assume class- conditional Gaussianity of pretrained embeddings, Clients transmit only sufficient statistics (per-class counts and firs//eccon--rrde moments) and the server builds heads via three components: (i) Closed-ford form Gaussian heads (NB/LDA/QDA) computed directly from the received statistics; (ii) FisherMix, linear head trained synthetic samples drawn in an estimated Fisher subspace; and (iii) Proto-Hyper, a lightweight low-rank residual head that refines Gaussian logits via knowledge distillation those synthetic samples. In our experiments, GH-OFL methods deliver state-of-the-art robustness and accuracy under strong non- IID skew while remaining strictly data-free.
INTRODUCTION
Federated Learning (FL.) enables distributed training without centralizing raw data: clients (e.g.. phones, IoT, edge) keep their data local and share updates with a server. This paradigm, motivated by privacy and bandwidth-sensitiie constraints (mobile, healthcare, finance), has spurred many variants but most methods remain iterative and multi-round.
The seminal Federated Averaging (FedAvg) (McMahan et al. 2017) performs several local epochs per client, then aggregates weights the server. A key limitation is the heavy reliance on many rounds for convergence. Empirically, on federated MNIST FedAvg needs 18 rounds to reach 99 9 i.i.d. vs. 206 in non-i.i.d. (Zhao et al., 2018 Cao et al. (022): on CIFAR-10 about 154 rounds for 75% and 442 for 80 %, while on CIFAR-100 nearly 700 rounds to move from 40% % to 50% (Zhang et al. 2022b); under severe non-i.i.d. partitions, >170 rounds may be required on CIFAR- 10 for just 55% % accuracy (Perazzone et al. 2022). These results expose the cost of multi-round communication. its latency sensitivity and degradation under heterogeneity. To mitigate this, one- shot federated learning (OFL) targets high-qualin global model in single exchange, reducing communication and synchronization while preserving privacy.
Extending the "Capture of global feature statistics for One-Shot FL" firstly introduced by Guan al. 2025). we present GH-OFL, a one-shot, server-centric Gaussian -head scheme: clients upload
"Corresponding author.

---

Published as a conference paper at ICLR 2026
only per-class sufficient statisties such as counts and first/second moments being covered by an optional random- projeccionn sketch; the server (i) fits closed-form Gaussian heads (NB/LDA/QDA with shrinkage) and (ii) trains two lightweight heads will call FisherMix and Proto- Proto-Hyper, using a data-free generator that draws "Fisher-ghost" samples in Fisher subspace with a QDA/LDA teacher blend. This shifts learning to the server, handles non-i.i.d. via class-balanced synthesis and needs no public data or client-side inference.
Contributions.
Closed-form Gaussian heads (GH-OFL-CF). From client moments alone, we compute Naïve Bayes (diagonal), LDA and QDA in one shot. Fisher-guided pipeline with tar- geted shrinkage (pooled/class-wis) diagonal//ow-ranl variants) and compressed random- projection sketch improves conditioning and cuts bandwidth, remaining strietly data-free.
Trainable synthetic heads (GH-OFL-TR)) We propose FisherMix and Proto-Hyper (low- rank residual) trained solely on synthetic Fisher-space samples. A blended LDA/QDA teacher in the Fisher space corrects closed-form bias with minutes of compute and no public data.
Robustness and accuracy. Across CIFAR-10 10, CIFAR-100 (Krizevskyy 2009). CIFAR- 100-C (Hendrycks & Dietterich] 2019) and SVHN (Netzer et al. 2011), with diverse back- bones, GH-OFL achieves strong single-round accuracy, remains fully data-free and shows state-of-the-arr robustness under non-i.idd partitions and corruptions, without client-side inference or auxiliary datasets.
RELATED WORK
One-shot federated learning (FL) seeks a global model in single exchange, cutting rounds, band- width and exposure. A first strand builds on knowledge distillation (KD): clients train locally and transfer to a server model via public/proxy set. FedMD enables collaborative distillation with- out raw data (Li & Wang 2019), while FedDF distills from client ensembles using unlabeled data (Lin et al. 2020). Analyses show one-shot KD can remain effective under severe non-IID (Zeng cet al. (0224) and explicit ensembling further stabilizes performance (Allouah et al. 2024). In paral- lel, single-round parameter optimization and meta-learning avoid multiple exchanges: early work aggregates parameters with regularization form usable global model (Guha et al.. 2019); meta-
learning provides favorable initializations for quick adaptation (Fallah et al.) 2020); aggregation- only schemes such MA-Echo remove server-side training (Su al. (023)) and applied studies like FedISCA validate feasibility in sensitive domains (Kang et al.) (023). ). A second strand is data-free generative or probabilistic. Generative methods synthesize surrogates
the server can train with- out accessing client data, e.g., FedGAN (Rasouli et al.] (020).. Fully data-free one-shot approaches avoid proxy sets entirely such as DENSE (Zhang et al., 2022a) and Co-Boosting (Dai et al. 2024) combine synthesis and ensembling, while probabilistic formulations integrate client evidence in Bayesian way, as in layer-wise posterior aggregation FedLPA (Liu et al., 2024). Our method sits at this intersection: we keep communication to per-class moments, remain data-free and instantiate Gaussian heads directly from statistics, complementing them with lightweight trainable heads on synthetic features.
Positioning of our work GH-OFL belongs to data-free probabilistic one-shot FL:: unlike KD (public/proxy data) or parameter/mettt-eaming (full model uploads), it uses only per-class moments to build Gaussian heads and lightweight Fisher-space heads from synthetic features.
3 PRELIMINARIES
We consider a one-shot federated setting where each client uses a frozen, pretrained encoder (c.g., a ResNet for images a Transformer-based model for text) to transform its local data into embedding vectors z E Rdd, with determined by the backbone architecture. Clients never share raw data or gradients. Instead, they send only linear statistics that the server global statistics, the server (i) instantiates closed-form Gaussian discriminant heads (NB/LDA/QDA) exclusively on synthetic samples drawn
across clients. From these
and (ii) trains lightweight heads (FisherMix/PPoooo-HHe)
2

---

Published as a conference paper at ICLR 2026
in discriminative subspace. This section explains the objects exchange, why they suffice and how they are used.
3.1 FEDERATED SETTING AND SUFFICIENT STATISTICS
What clients send.
Let C=(1,
the local dataset at C} be the class set and D(")= y:)}
client
A(4)
Σ Fj:
N(W) f yi c
B(a)
Σ J,
(1)
5(
D = Σ
(ot x.))
Aggregating across clients gives
A. Σ (w)
No = NNO B= EBO,,
B(u) S. So= ES= De Ddo
a
Usage by head: LDA/Fisher use (A, B); QDA uses N,,; NBdiug uses (A, D) (or (A, N, diag(S)) if S is sent).
Ciient 1
D Σ 40a
== 2 R, fe
BB
Ry ~ (O.
44
A
N!
D Σ 404
BB
Ry (0)
8/
Ciient
BB
R, Rig (0
R f2
M!
1 md
D Σ
th
484
Σ 44
(all A. Beged 8.eat D. N.E..
drained Backbone
Figure 1: Client-side flow with secure aggregation. Each device encodes images via frozen Ima- geNet backbone, projects embeddings with public RP (==x R) and updates additively-aggreaable stats in z: No. Ag. global B and SJD, as required by the chosen heads. Secure aggregation reveals only ()), which suffice estimate means/covariances and run Gaussian and Fisher-space heads without sharing raw data or gradients.
Why these sufficient. The aggregated moments provide all parameters required by heads: class means and priors come from (Ae, N.); ;); the pooled covariance (centered second moment) comes from B; when S. is available it yields unbiased class covariances; when only diagonal model is needed, D. gives per-dimension variances. Hence {A, N, B, B/SDD) are sufficient to instantiate Gaussian Heads and the Fisher subspace (see App. for more details).
A No
N.
N Σ No. Epool N- (B- N-c(B- Σ NeHette)
Pe
Tec
(2)
Σ, N

---

Published as a conference paper at ICLR 2026
Class covariances (QDA) and diagonal variances (NBdiag):
De Ne
Eg N: (Se No NoMoo))
Vare
(M))
(3)
If Sr is available then D. = (Sag
Compressed candom-projecttinn sketch. maps E R to ==rft R with public matrix Re Baxk (k d, shared seed) and accumulates moments directly in 2. By linearity,
To reduce bandwidth and strengthen privacy, each client
A No
A Ar, === BR, S R RTSRS S,R, D -iag((.))
(=00),
re
(4)
If full St is not sent, clients can still transmit the per-class elementwise sums of squares De. from which diagonal variances follow:
DE Ne
Var
(1
All quantities remain additively aggregable across clients and preserve the one-shot contract.
3.2 GAUSSIAN HEADS AND THE FISHER SUBSPACE
Closed-form discriminants (what they assume and what they use). We instantiate three Gaus- sian heads directly from the aggregated moments, each making a different covariance assumption:
NBding (class-diagonal covariances). Each class its own per-dimension variance; no cross-dimension correlations are modeled. Statistics used: (A, N, D) or (A, N, dSag(S)). This head extremely light and robust when features are close to axis-aligned.
LDA (shared covariance). All classes share a single covariance estimated from the centered second moment; scores linear in x. Statistics used: (A, B). In practice set We Xmaalfe and use the usual linear rule with log-priors.
QDA (class-specific full covariances). Each class has own covariance, enabling class- dependent shapes and correlations. Statistics used: (A,N,)) This is the most expressive closed-form option when is available.
Numerical stability. We apply a standard shrinkage Σ = (1 - a)Σ + a tr(2) I (for a [0, 1) to Epost (LDA/Fisher) and to Σ. (QDA) when needed.
Why a Fisher subspace helps. Discriminative structure often concentrates in a low-dimensional subspace where between-class variation dominates within-class variation. We form the be- tween/within scatters Sg and =pooo and solve the generalized problem
Sgu XSwr.
Taking the top k eigenvectors as columns of V, we work with 2, = VT and the projected moments ud V Mes Eqpoa VT V E (and Σς if available). This reduces dimension, improves signal- to-noise and speeds up both synthesis and training while leaving all heads unchanged.
3.3 DATA-FREE SYNTHESIS AND TRAINABLE HEADS (FISHERMIX. PROTO-HYPER)
Class-conditional synthesis in zy. Using only aggregated moments in the Fisher space, the server samples per class
2f Nag, -²E))
where Eif is the shrunk class covariance if S, is available. otherwise the (shrunk) pooled covariance E and 7 is global dispersion scaling factor. This strictly data-free: real samples or client- side inference.

---

Published as a conference paper at ICLR 2026
FisherMix (linear head). On synthetic pairs (2,, FisherMix trains linear classifier in the Fisher subspace using cross-entropy loss:
FPM = CE(softmax + b), y).
Proto-Hyper (low-rank residual over a Gaussian base). We learn a small low-rank residual h(zy) = 12 that adds to Gaussian base head (NB ANNaai///LDD//OOO computed from moments:
Istudent (zx) Ghase (zy) + k((2))
and optimize KD+CE blend with Gaussian teacher in 2 (e.g.. LDA or QDA) at temperature T:
CpH a 7 KL soft maxxxaa soft sox maxxxxuneeee (1 xx) CE (ondeennn (snden((yy) y)
The residual corrects systematic bias of the base Gaussian rule with a tiny parameter footprint, preserving the one-shot, data-free contract.
Note. If only diagonal variances are available (from D), sampling uses diag(2!)) and an LDA teacher.
3.4 NON-IID MODELING, REGULARIZATION COMMUNICATION AND PRIVACY
Non-IID via Dirichlet splits. We model label skew by partitioning a fixed dataset D across clients with Dir(a) over class proportions (smaller = stronger non-IID). Clients send only the additively- aggregable statistics in Sec. 3.1
Partition invariance of global moments (and of their RP sketches). For any partition P = (Lu of D, secure aggregation produces the same global moments as the samplewise sums; with fixed public R, the projected moments coincide by linearity:
A. Σ
A(
Σ It
N = Σ N(O) N [íi: y1 c
Ry:-C
Σ B(u) Σ
S. Σ 5(u) Σ TT De=
De= Σ Oz)),
D(
1
A = AcR, = BR,
= R ScR,
D = diag(S"))
(5)
Hence He, Te, Saoal Eg (and their projected/diagonal variants) are independent of P and of a.
Fisher subspace and closed-form heads. Since Sw=Epoal and SB are partition-invarian the Fisher eigenspace (up to within-eigenspace rotations) is invariant as well; closed-form heads com- puted either in x or in 2, (NBaing. LDA, QDA; with fixed shrinkage) are therefore partition-invarian.
Trainable heads: invariance in expectation. FisherMix and Proto-Hyper are trained on synthetic (z drawn from distribution Q whose parameters deterministic functions of the partition- invariant moments. The population objective
min
therefore identical for any Minor accuracy fluctuations arise only from Monte-Carlo sampling, optimizer non-determinism and finite precision, not from the partition.
3.5 GH-OFL HEADS: DISCUSSION AND ADVANTAGES
NBdiag. class-conditional Gaussian with diagonal covariance per class, computed in the pro- jected space (z) from aggregated moments only: means from A/N and per-dimension variances from SUMSO/N (with shrinkage toward the pooled variance). The intuition is that pretrained em- beddings often near axis-aligned after RP/Fisher, ticity without the cost/instability of full covariances. This improves calibration and class-specifit decision boundaries while staying extremely light (no raw data, O(Ck) storage).
diagonal model captures heteroscedas-
5

---

Published as a conference paper at ICLR 2026
Apprepating clest makks
Agprgacioo YI' do No, Bookes rr
v (Ag Del SecAas (ril) 2mn
VT (d 18) Secdeef 2o lan
YT's (Au, No S.) T Becdaaf 2 plny Just
v (..NN D. Do Soskeet Aars Σ rro (ell in
7000 (ull in
Σ pou (all in
plu (all
ruu (all in
Betimate pe, digg)
Estimate Per Epet (la.s)
Estimate ou Ee. Wy. (In a]
Berimate momoute
Betimate momente
(it
Fiaber space
cumpute Fiaker
Systhreton class conditional data in trais Inseee head
Synzhmtte data build Clamatiin buse legils (e.g, LDDA/BB)
Compute NBay sccees
Compute scome
Compute QIA acoree
Use FisberMir logits in prodict ang max
Train
Prodict: ang mar s.(s)
Prodict: magaax sels)
Prodict: ongmax x//)
racidral lagite; peetlict
Closed- foosed-form methods
Trainable methods
Figure 2: Server-side pipelines for our GH-OFL family. Secure aggregation first collects class- wise sufficient statistics in the projected space 2. Closed-form heads compute scores directly, while FisherMix and Proto-Hyper estimate Fisher subspace, synthesize features and fit lightweight heads without any raw data. Shaded panels summarize per-method steps and outputs.
LDA (shared covariance). A linear discriminant head using the pooled covariance Epool (shrunk) and class means, computed in 2f. The intuition that single well-conditioned covariance captures most geometry of strong encoders; Fisher space, discriminative energy concentrates in few directions, making LDA both stable and accurate. Advantages: closed-form training, tiny footprint, fast inference; serves as a reliable teacher and as a robust baseline under wide non-IlD regimes.
QDA (class covariances). quadratic discriminant head with (shrunk) E. per class when class- wise second moments are available (either directly or after projection). The intuition is to model class-dependent spreads and correlations, which can matter for fine-grained classes or domain shifts. Fisher projection mitigates overfitting by compressing to the most discriminative direc- tions; shrinkage stabilizes inverses. When available, QDA is our most expressive closed-form option and strong teacher.
4. FisherMix (linear head, trained synthetic data). A lightweight linear classifier trained server-side only on synthetic, class-conditional samples drawn in the Fisher subspace from the Gaussian heads. The intuition to refine decision boundaries in the most discriminative direc- tions identified by Fisher projection, improving separation when closed-form Gaussian heads are slightly biased. Benefits: no real data needed, tiny model, strong performance when prototypes are good but margins are tight; especially effective when clusters are roughly convex in 2 yet not perfectly separable by the closed-form heads.
Proto-Hyper (low-rank residual on top of a Gaussian base), A compact residual head h(2y) = V2U that adds corrections Gaussian base (NB/LDA) and learns via temperature- distilled soft targets from a blended Gaussian teacher (e.g.. LDA/QDA). The intuition is bias correction: keep the closed-form geometry for stability and learn only small low-rank "delta" to fix systematic mismatches (non-Gaussian tails, mild correlations, calibration). Advantages: very few parameters, fast convergence synthetic data, strong robustness to non-lllD and en- coder/backbone changes while preserving the data-free guarantee.
4 EXPERIMENTS
To demonstrate the versatility and robustness of our proposed GH-OFL methods, this section ports empirical results comparing them against SOTA OFL architectures on widely used benchmark datasets. To further validate performance and generality, also evaluate all GH-OFL variants across multiple well-known backbones. Additional experimental details such as datasets, splits, training protocols and hyperparameters are provided throughout this section.

---

Published as a conference paper at ICLR 2026
Table Accuracy (%) GH-OFL methods compared to OFL variants and standard multi-round FL baselines under different Dirichlet client splits. Baseline OFL values are from Guan et al. (2025) under the same declared setup and we also compare against conventional FL methods including FedAvg, FedProx (Li et al., 2020) and SCAFFOLD (Karimireddy al. 2020). For clarity, bold entries denote the top-2 accuracies among OFL methods only, while underlined entries indicate
the overall best accuracy per column across all methods (OFL and FL baselines). More details on
additional experiments on natural language (NLP) datasets and the reproducibility the baseline
methods can be found in the Appendix A and
Method
CIFAR-10
CIFAR-100
SVHN
α 0.05
00100 0.50 a=0.55 a=0.10
a=0.50 a=0.55 a=0.10 a=0.50
FL baselines (multi-round, R E {1, 10, 50))
FodAvg round)
27.38
24.53
31.17
2.15
2.03
1.50
19.26
11.94
19.25
FodArg (I0 rounds)
64.13
73.08
82.62
29.19
29.92
33.41
54.94
74.16
80.73
FodAvg (50 rounds)
77.42
84.60
91.52
62.46
63.58
68.55
78.79
87.71
92.13
FedProx (1 rourd)
27.49
32.24
35.65
2.24
2.00
1.47
12.99
11.97
19.23
FedProx (10 rounds)
64.18
81.82
89.02
29.51
29.79
33.71
57.84
74.15
81.01
FedProx (50 rounds)
77.54
85.68
91.74
62.76
63.68
68.61
76.30
87.88
92.17
SCAFFOLD (1 round)
20.36
27.71
31.81
1.93
1.63
1.50
12.87
12.05
18.79
SCAFFOLD rounds)
30.96
SCAFFOLD (30 rounds)
73.78
88.39
91.66
61.63
64.09
68.60
75.99
87.45
92.00
OFI baselines
DENSE
31.26
56.21
62.42
14.31
17.21
26.49
37.49
51.53
77.44
Co- TooBo
44.37
60.41
67.43
20.30
24.63
34.43
41.90
57.13
84.65
FedPFT
56.08
56.43
56.80
36.79
37.16
37.95
42.55
43.03
43.84
FedCGS
63.95
63.95
63.95
39.95
39.95
39.95
57.77
57.77
$7.77
GH-OFL (ours)
GH-NNBing
78.84
78.84
78.84
55.51
55.51
55.51
39.24
39.24
39.24
GH-LDA
86.05
86.05
86.05
63.92
63.92
63.92
62.16
62.16
62.16
GH-QDA
84.40
84,40
84.40
66.52
66.52
66.52
55.30
55.30
55.30
FisherMix
84.74
84.74
84.74
66.99
66.99
66.99
57.79
57.79
57.79
Proto-Hyper
85.74
85.74
85.74
64.05
64.05
64.05
61.97
61.97
61.97
4.1 DATASETS, NON-IID SPLITS AND BACKBONES
We evaluate our methods on four image classification benchmarks that are standard in FL tasks: CIFAR-10 (10 classes, 60k images, 32x 32), CIFAR-100 (100 classes, 60k images, 32) SVHN (10 digit classes, 733 train/ ~26k test) and CIFAR-1O0-C for robustness. CIFAR-10 compar- atively easy (coarse categories), CIFAR- 100 is harder (fine-grained classes with higher inter-class similarity), SVHN sits in between (clean digits but real-world nuisances), while CIFAR- 1-AAR---0-C111 is the most challenging due distribution shift: following common practice, average accuracy over 19 corruption types divided in 4 families (noise, blur, weather, digital) at the highest severity (5), which typically causes a 20-40 point degradation even for strong encoders. Client heterogeneity is modeled via Dirichlet splits: for U clients and C classes, per-client class proportions are drawn as Pr. Dir(alc) and samples of class are assigned to client with probability Pu.e; smaller yields stronger non-IlI) (fewer classes per client and more imbalance), larger approaches IID. We use a E (0.01, 0.1, 0.5) to cover strong and moderate skew while keeping global class totals fixed for fair comparisons. Unless noted, all clients share the same fixed backbone avoid architec- tural confounds: ResNet-18 pretrained on ImageNet- using its penultimate embedding (d=512) to compute client-side aggregated statistics and to instantiate server-side Gaussian heads; in abla- tions we also consider 50, MobileNerV2, EfficientNer-Bi and VGG-16 (all ImageNet-Ie 1K pretrained) and when probing domain shift also included ResNet- 18 pretrained Places365.

---

Published as a conference paper at ICLR 2026
When QDA is not an option. Storing per-class second moments S costs (CC2²) memory; for high-d backbones (e.g., ResNet-50, d-2048) this often impractical, so QDA is unavailable. FisherMix//Proooopper remain viable be- cause they rely on a pooled covariance (or class-wise when available) within a compact Fisher subspace. As shown in Table 2 Fisher heads consistently improve over LDA on CIFAR- 100-C and approach QDA without the per-class dxd d storage; when feasible, QDA remains the upper bound.
CIFAR-100-
Method
Shared Stats
Acc.
FedCGS
A, B,
24.4%
GH-NBaiu
A, D
25,4%
GHILDDA
A. B.N
37.6%
FisherMix
A, B, N,
40.1%
ProtoHyper
A, D,D
39.8%
GH-QDAtul
N,N,S
64.3%
Table 2: Accuracy (%) on CIFAR-100-C (severity 5, averaged over 19 corruptions) and compar- ison of the required client-to-server statistics. The results illustrate the trade-off between model expressiveness and memory: diagonal and pooled heads are lightweight but less robust under shift, whereas full QDA achieves the highest accuracy at (Cd))) cost. Fisher-space trainable heads offer balanced intermediate alternative.
4.2 BEHAVIOR UNDER CORRUPTION, CAPACITY AND PRETRAINING SHIFT
Building on the previous Table 1 we repeat the same protocol in three complementary settings to
probe robustness and generality: (i) we evaluate the same architectures on CIFAR-IOO-C at severity across all corruption types, stressing robustness to noise, blur, weather and digital artifacts (Table (ii) we vary the frozen encoder ReesNer-18/50, MobileNetV2, EfficientNet-BB VGG-16) to assess how the heads scale with representational capacity and architecture (Table Ba); (iii) we keep the original architecture (ResNet-18) but switch pretraining to Places365, a scene-centric dataset with 365 categories and 1M images, that features reflect different domain than ImageNet (Table (b). Unless otherwise noted, clients transmit the same aggregated statistics as in the previous table and the server instantiates the same Gaussian and Fisher-space heads; no raw images are ever shared.
Backbone ablation and pretraining shift. Across backbones clean CIFAR- 10 under the GH- OFL pipeline (Table performance scales with representational capacity and alignment to Ima- geNet pretraining, following the ordering VGGI6 < MobileNerV2 x ResNer18 < EfficientNer--B < ResNez50. Under distribution shift (CIFAR-100 and CIFAR-100-C),, the same ranking persists while performance gaps widen, as stronger encoders induce improved class separability and better- conditioned covariance estimates in feature space. As shown in Table switching to scene-centric pretraining (Places365) changes the relative behavior of the heads. On CIFAR-I0 and CIFAR- 100, the trainable Fisher-space heads tend to benefit from this mismatch and become more competitive. However, under strong corruption CCFAR--10-CC) modeling class-specific covariances remains im- portant and full QDA provides the most stable performance. These trends can be understood geo- metrically. Stronger backbones (e.g., ResNet50 or EfficientNer-BOO yield embeddings with larger inter-class margins and tighter intra-class concentration, improving the conditioning of the estimated means and covariances from aggregated moments. Since all GH-OFL heads are built directly from aggregated moments, better feature geometry directly translates into more stable discriminants and more informative Fisher subspaces. Conversely, weaker encoders produce noisier class clusters, which degrades both closed-form and synthetic heads. Under distribution shift (CIFAR-100-C), the performance gap widens because corruption perturbs class geometry in a class-dependent man- ner. In this regime, shared-covariance assumptions (LDA) may become restrictive, while modeling class-specific covariances (QDA) allows the model to better adapt when distortions affect each class differently. Pretraining shift further exposes this effect. With ImageNet (object-centric) pretraining, downstream CIFAR features remain relatively aligned with Gaussian and shared-covariane assump- tions in Fisher space. In contrast, scene-centric pretraining (Places365) induces less class-aligned embeddings, increasing the bias of closed-form Gaussian heads. Here, trainable Fisher-space heads (FisherMix and Proto-Hyper) gain competitiveness by learning margin-based or low-rank residual corrections that compensate for mild deviations from Gaussianity. Overall, these results indicate that GH-OFL performance is primarily governed by feature geometry: when representations are well aligned, linear Gaussian heads operate near the accuracy-overhead Pareto frontier; when fea- ture alignment degrades due to shift or corrpptio,, richer covariance modeling or lightweight dis- criminative refinements restore robustness without extra communication cost.
8

---

Published as a conference paper at ICLR 2026
(a) CNN backbones pre-trained on ImageNetIK
Backbone
FedCGS
Gli-
GH-LDD
GH-QDAmi
FisherMix
ProtoHyyper
efficientnet.00
84.15
84.89
90.01
88.43
90.02
89,99
mobilcnet_v2
78.11
79.15
86.61
84.67
86.70
86.62
resnet18
76.58
77.58
86.17
84.76
86.04
85,95
resnet50)
81.06
81.96
91.26
86.68
91.27
91.23
vgg16 16
66.08
67.83
81.39
77.65
81.71
81.32
(b) ResNet-18 pre-trained on Places365
Dataset
FedCGS
Gi-N
GH-LDA
GH-QDAmi
FisherMix
ProtoHyper
CIFAR-I0
77.20
78.85
86,26
85.12
86.56
86.31
CIFAR-100
53.52
55.51
64,12
64.88
66.42
65.27
SVHN
34.84
39.24
62.88
57.16
62.26
62.35
CIFAR-I00-C
13.53
15.11
25.99
46.54
29.08
38.64
Table 3: Accuracy (%) of GH-OFL heads under different encoder initializations. (a) Performance across multiple AmageNettlK-pettraine CNN backbones, highlighting how stronger or better- aligned representations consistently benefit Gaussian and Fisher-space heads. (b) Results with ResNet- 18 pretrained on Places365, illustrating the impact of domain mismatch: trainable Fisher heads become comparatively competitive under pretraining shift, while full QDA remains the most expressive closed-form upper bound when reliable class covariances are available.
CIFAR-10
CIFAR-100-C
Top-1 Accuracy (96)
$4.0
Accuracy
32.5
37,60
Pretraining
Prstruining
ImageNet 3K pretrain
ImageNict pretrain
Places365 pretrain
Places365 pretrain
Mothoss
195
NB disg
FisherMix
ODA
NB diag
FisherMix
ODA
LDA
B-typer
LOA
P-Hyper
49.dag
LD4
Rpherbi P-Hsper
ab4
kB-dag
Los
Rpheniix P-dyper
ab4
10.65
Server-side overhead (relative units)
Server-side overhead (relative units]
(a) CIFAR-10
(b) CIFAR-100-C
Figure 3: Accuracy-overhead trade-off of GH-OFL heads on CIFAR-I0 (left) and CIFAR-100-C (right) under different pretraining (ImageNet-IK vs. Places365). On CIFAR-10. linear Gaussian heads (notably LDA) lie near the Pareto frontier, with FisherMix and Proto-Hype achieving similar accuracy at modest cost. Under strong shift (CIFAR-100-C) richer covariance modeling becomes beneficial: full QDA is most robust, while Fisher-space heads offer an intermediate trade-off.
4.3 DISCUSSION
Scalability and communication. GH-OFL communicates only additively aggregable moments in compressed space ==IR of dimension & Per client, the payload scales as O(Ck: + k22) (with A e BC N e RC B² pk optionally SUMSQ* or S:), independent of local sample size, practical for large fleets. Consistent with the partition-invarian
of our moments, we replicated the CIFAR-10 setting of Table TJby distributing the training set across 50 and 100 clients (same Dirichlet a): the top-1 accuracy remained unchanged (within negligible noise), confirming that performance is insensitive to the number of clients for fixed global data and RP. Server-side, Closed-form heads are lightweight (O(Ck2) for LDA: OCCk²)) for QDA via class-wise inversions. The Fisher subspace solves generalized eigenproblem in l and is amortized across heads/synthesis. FisherMix and Proto-Hyper train on synthetic features only; runtime is dominated by sampling and small dense ops in k. LDA is strong default when only (A, B, N) are available; NB exploits heteroscedasticity

---

Published as a conference paper at ICLR 2026
from D, N) with minimal footprint; QDA is best when reliable per-class second moments S are feasible (Fig. 3).). FisherMix/Proto- Hyper typically exceed LDA under shift or mild non-Gaussir structure (Fig.) 3-a) (see Appendix C for further details on this section).
Privacy and security. Clients never share raw data or gradients, statistics are revealed only af- ter secure aggregation and a public random projection further scrambles coordinates. Compared to model/gradient exchange as in FedAvg, vulnerable to gradient/model inversion, membership and property inference, our method generally exposes smaller attack surface, since only aggregated first/second moments leave the clients. That said, moments can still leak information in small-N regimes or with very fine-grained statistics (e.g.. per-class So). In addition, GH-OFL removes the temporal exposure typical of multi-round federated learning, where gradients or model updates are exchanged repeatedly and may allow observer to track how parameters evolve time and po- tentially exploit these dynamics; by contrast, our protocol reveals only a single aggregated snapshot of low-order statisties, without iterative feedhack loops opportunities for adaptive probing across communication rounds. The information that is shared consists exclusively of coarse summaries of frozen encoder features rather than full models or gradient signals and since many distinct datasets can produce identical aggregated moments, especially after random projection, any attempt at recon- struction remains inherently ambiguous. Although this does not eliminate all risks, substantially reduces the level of detail that can be inferred compared to multi-round parameter exchange. The protocol naturally integrates with secure aggregation because all transmitted quantities are purely additive statistics (class counts and summed moments), which means the server only needs the global sums and never the individual client contributions to compute the final heads and global (server- side) differential privacy can be applied top of these aggregated statistics by injecting calibrated noise once at the server level. In our setting, this combination represents favorable trade-off be- tween computational overhead, formal privacy guarantees and predictive performance, as noise is added only after aggregation and therefore benefits from averaging across clients.
Overall, for the same encoder and model, GH-OFL offers stronger default privacy posture than multi-round and one-shot classical model exchange, while remaining strictly data-free (see Ap- pendix B for an extended discussion).
Limitations and domain shift. Our pipeline assumes a pretrained encoder. With object-centric pretraining (e.g., ImageNet), linear Gaussian heads in Fisher already perform strongly; with scene- centric or otherwise mismatched pretraining (e.g., Places365), the feature geometry departs from the shared-covariance assumption and trainable heads gain relevance. Empirically, Tabll 3/ shows accu- racy scales with backbone strengthhalignmeen, while (Table 3p (ResNet-18 pretrained on Places365) highlights that FisherMix/Proto- -ayee Hyper match surpass LDA on CIFAR- SVHN, where features remain nearly linearly separable in the Fisher space. In short, GH-OFL ben- efits from good pretraining but under domain shift the synthetic, Fisher-space heads the primary mechanism to recover performance without public data.
yet not
CONCLUSION
We presented GH-OFL, a data-free, one-shot federated approach. Clients send only per-class counts and first/second moments after public random projection; the server then (i) builds closed-form Gaussian heads and (ii) trains lightweight Fisher-space heads on synthetic features. QDA is best in hard domains when class covariances are available; trainable methods are generally more robusts with respect to the GH methods in most scenarios. The estimators are insensitive to the Dirichlet a, remaining robust under strong non-IIlD conditions and require very little communication offering a simple, privacy-friendly solution that works across heterogeneous backbones; outside of these findings, GH-OFL gives a clear direction for scalable, private edge FL. Because the synthesis and heads are modality-agnostic, the approach extends beyond classification to structured prediction and multimodal settings. These properties align well with privacy-sensitie deployments (e.g.. healthcare. finance, edge/loT, inspection, retail video. remote sensing), where pretrained encoders are common and public random projections further decouple shared statistics from raw content.
and bandwidth-limiin
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
The project has received funding within the Chips Joint Undertaking in collaboration with the Horizon Europe Framework Programme under project Ecomobility, Grant agreement number 101112306, CUP: E83D22000150006.
REPRODUCIBILITY STATEMENT
Section and describe the GH-OFL framework and assumptions, while Appendix [ details the multi-round FL baselines, experimental settings and convergence analyses and includes links the GitHub repositories for both the classical FL baselines and the GH-OFL implementation. Minor numerical differences may arise from code updates or non -on-determiiistic components but they do affect the reported trends or conclusions.
REFERENCES
Youssef Allouah, Akash Dhasade, Rachid Guerraoui, Nirupam Gupta, Anne-Marie Kermarrec, Rafael Pinot, Rafael Pires, and Rishi Sharma. Revisiting ensembling in one-shot federated learn- ing. In Advances in Neural Information Processing Systems (NeurIPS), 2024.
Mei Cao, Yujie Zhang, Zezhong Ma, and Mengying Zhao. C2s: Clare-aware client selection for effective aggregation in federated learning. Tligh-Confidene Computing, 2(3): 100068, 2022.
Rong Dai, Yonggang Zhang, Ang Li, Tongliang Liu, Xun Yang, and Bo Han. Enhancing shot federated learning through data and ensemble co-boosting. In International Conference on Learning Representations (ICI.R), 2024.
Alireza Fallah, Aryan Mokhtari, and Asuman Ozdaglar. Personalized federated learning with theo- retical guarantees: A agnel-ag meta-learning approach. In Advances Neural Information Processing Systems (NeurIPS), 2020.
Zenghao Guan, Yucan Zhou, and Xiaoyan Gu. Capture global feature statistics for one-shot feder- ated learning. In AAAI Conference on Artificial Intelligence (AAAI), 2025.
Neel Guha, Ameet Talwalkar, and Virginia Smith. One-shot federated learning. arXiv preprint 111. 2019.
Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common ruptions and perturbations. In International Conference on Learning Representations (ICLR), 2019.
M. Kang, P. Chikontwe, S. Kim, K. H. Jin, E. Adeli, K. M. Pohl, and S. H. Park. One-shot federated learning on medical data using knowledge distillation with image synthesis and client model adaptation. In International Conference Medical Image Computing and Computer-Assisted Intervention (MICCAI), 2023.
Sai Pranceth Karimireddy, Satyen Kale, Mehryar Mohri, Sashank Reddi, Sebastian Stich, and Ananda Theertha Suresh. SCAFFOLD: Stochastic controlled averaging for federated learning. In Hal Daumé III and Aarti Singh (eds.), Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings Machine Learning Research, pp. 5132-5143. PMLR, 13-18 Jul 2020.
Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical re- port, University of Toronto, 2009. URL https::/www/ http:s//www..sssoooottteeedd/// learning features 20.9-TR. TR.
Daliang Li and Junpu Wang. Fedmd: Heterogenous federated learning via model distillation. arXiv preprint arXiv: :Xiv:1900.0358/ 2019.
11

---

Published as a conference paper at ICLR 2026
Qinbin Li. Bingsheng He, and Dawn Song. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 10708-10717, Los Alamitos, CA, USA, June 2021. IEEE Computer Society, doi: 10. 10.11099/CVPR4443..2201... 01057.
Federated Learning In 2021
Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, and Virginia Smith. Federated optimization in heterogeneous networks. In Machine Learning and Systems (MLSys), 2020.
Tao Lin, Lingjng Kong, Sebastian U Stich, and Martin Jaggi. Ensemble distillation for robust model fusion in federated learning. In Advances in Neural Information Processing Systems (NeurIPS), 2020.
Xiang Liu, Liangxi Liu, Feiyang Ye, Yunheng Shen, Xia Li, Linshan Jiang, and Jialin Li. Fedlpa: one-shot federated learning with layer-wise posterior aggregation. In Advances in Neural Infor- mation Processing Systems (NeurIPS), 2024.
Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. Coomuunication--fiicca Learning of Deep Networks from Decentralized Data. In International Conference on Artificial Intelligence and Statistics (AISTATS), 2017.
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y. Ng. Reading digits in natural images with unsupervised feature learning. In NIPS Workshop on Deep Learning and Unsupervised Feature Learning, 2011.
Jake Perazzone, Shiqiang Wang, Mingyue Ji, and Kevin Chan. Communication-efficee device scheduling for federated learning using stochastic optimization. In Conference Computer Communications (INFOCOM), 2022.
Mohammad Rasouli, Tao Sun, and Ram Rajagopal. Fedgan: Federated generative adversarial net- works for distributed data. arXiv preprint arXiv:2006.07228. 2020.
Shangchao Su, Bin Li, and Xiangyang Xue. One-shot federated learning without server-side train- ing. Neural Networks, 164(C):203-215, 2023.
Hui Zeng, Minrui Xu, Tongqing Zhou, Xinyi Wu, Jiawen Kang, Zhiping Cai, and Dusit Niyato. One-shot-but wew--buu-no--eerrdee federated learning, In ACM Multimedia, 2024.
Jie Zhang, Chen Chen, Bo Li, Lingjuan Lyu, Shuang Wu, Shouhong Ding, Chunhua Shen, and Chao Wu. Dense: Data-free one-shot federated learning. In Advances in Neural Information Processing Systems (NeurIPS), 2022a.
Lin Zhang. Li Shen, Liang Ding, Dacheng Tao, and Ling- Yu Duan. Fine-tuning global model via data-free knowledge distillation for non-iid federated learning. In JEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022b.
Yue Zhao, Meng Li, Liangzhen Lai, Naveen Suda, Damon Civin, and Vikas Chandra. Federated learning with non-iid data. arXiv preprint arXiv:1806.00588 2018.
APPENDIX
A ADDITIONAL ANALYSIS AND EXTENSIONS FOR GH-OFL
This section provides additional empirical and theoretical analysis that complements the main text. We focus on two main aspects: (i) scalability of GH-OFL beyond computer vision; (ii) the trade-off between full QDA and the proposed singonal-plus-loe lows-llowrank (DLR) covariance sketches.
A.1 NON-VISION EXPERIMENTS: NLP BENCHMARKS
To complement and strengthen the results reported in Table we include experiments on NLP datasets in addition to vision tasks. NLP benchmarks are inherently more challenging due their linguistic variability and complex input structure, making them an effective testbed to validate the robustness of our approach beyond standard image-llasificationn settings.
12

---

Published as a conference paper at ICLR 2026
Table NLP one-shot FL results with frozen DistilBERT RP (dRp 256). Test accuracy (%) for OFL baselines (client ensemble, Dense-style student, Co-Boosting ensemble) and GH-OFL heads (best Gaussian NB NB-diA,/LDD//ODA, FisherMix, Proto-Hyper) under Dirichlet client splits with a {0.5, 0.01}.
Dataset
OFL baselines
GH-OFL (ours)
Dataset
Ensemble
Dense
Co-Boost.
Best Gauss
FisherMix Proto-Hyper
0.50
85.64
85.79
84.42
88.96(LDA)
89.14
88.87
AG NEWS
0.01
25.00
25.00
25.00
88,96 (LDA)
89.14
88.87
0.50
98.70
98.74
98.72
98.33(LDA)
97.21
98.33
DBPEDIA-14
0.01
51.49
51.65
51.37
98.33(IDA)
97.21
98.33
0.50
83.14
82.68
83.26
82.68 (QDA)
70.53
82.80
SST-2
0.01
51.26
51.26
51.38
82.68 (DAA)
70.53
82.80
0.50
30.62
30.03
23.28
77.6 (D.60
73.64
80.26
BANKING77
0.01
5.32
4.97
3.54
(7.60
73.64
80.26
0.50
43.30
43.24
36.37
84.80(LDA)
82.89
85.69
CLINC150
0.01
3.66
3.70
2,43
(LDA)
82.89
85.69
A.1.1 DATASETS AND SETUP
To demonstrate that GH-OFL is not limited image encoders, evaluate it five standard NLP classification tasks:
AG NEWS: 4-way news topic classification.
DBPEDIA-14 14-way ontology classification.
SST-2: binary sentiment classification.
BANKING77: 77 intent classes.
CLINC150: 151 intent classes.
In all cases we use the same frozen encoder:
DistiBBERT model di -bbettbaae-uuccaa and pre-trained on large-scale English corpora (Wikipedia and BookCorpus) with a masked- language-modellng objective. From this encoder we extract the CLS embedding as our feature vector. To reduce both bandwidth and computational cost and to match the vision setting, we apply the same public random projection R RR with dgp 256. Clients only transmit class-wise sufficient statistics (counts, first and second moments) in this RP space.
distilled from bert-base-uncasee
At the server, we instantiate:
NB-diag. LDA and full QDA Gaussian heads in RP space.
FisherMix, the synthetic-data linear head trained on Fisher subspace samples.
Proto-Hyper, the low-rank residual head distilled from a Gaussian teacher.
All heads are fully data-free with respect to raw client examples: they only use the aggregated moments received through GH-OFL
A.1.2 QUANTITATIVE RESULTS
Table 4 reports test accuracy (%) for the NLP benchmarks. For each dataset we highlight the best Gaussian head (NB-diag/LDA//DAA and the two synthetic-data heads.
On AGLNEWS and SST-2, the synthetic heads (especially Proto-Hyper) match slightly outper- form the best Gaussian head, consistent with our vision results: when Gaussian assumptions are mildly violated, low-rank residual correction close the remaining bias. On DBPEDIA-14 and
13

---

Published as a conference paper at ICLR 2026
Table Full QDA vs diagonal--lus--ow-rall QDA (DLR-QDA) in RP space (dRp 256). "Mem" is relative memory; "Time" is relative inference time per sample (NB-diag =)).
Head
CIFAR-100 100 Ace.
AG NEWS Acc.
DBPEDIA-14 Acc.
Mem (rel.)
Time (rel.)
NB-diag
49.68
83.86
92.33
1.0
1.0
LDA
59.46
88,93
98.32
1.0
1.1
QuA
48.20
68.20
92.33
16.0
4.0
DLR-QDA (r=4)
49.99
67.08
92.32
2.0
1.4
DLR-QDA (r=8)
47.77
56.53
92.35
3.0
1.6
OLR-QDA (r=16)
42.01
38.32
92.19
the other tested datasets (BANKING77, CLINCI50), the relative ordering between NB-diag, LDA, full QDA, FisherMix and Proto-Hyper depends primarily the class count and label imbalance.
A.2 DIAGONAL-
QDA FUI.L QDA
In GH-OFL, closed-form Gaussian heads are built directly from aggregated class-wise moments, making QDA the most expressive option when per-class second moments are available (see empir- ical results in Table 2). However, full QDA requires storing and inverting covariance matrix per class, which scales quadratically in memory and cubically compute (this quickly becoming im- practical in nigh-dimensional encoder or RP spaces). To retain most of QDA's modeling power at a fraction of its cost, we also analyze a diagonal-plus-oow-ral class-specific variances and only small number of principal correlation directions. This subsection motivates the approximation and empirically compares OLR-QDA to full QDA across datasets and ranks.
(DLR) approximation that captures
A.2.1 MOTIVATION AND COMPLEXITY
In high-dimensional encoder spaces, full QDA with per-class covariance matrices Eg Rdxd in- curs both O(Cd²) memory and (dd) computational cost for decompositions, which becomes pro- hibitive as d grows. We therefore propose
(DLR) approximation:
E Dr + UUU
where D. is diagonal and U. E Rdxrr with < d (e.g., E 84,8, 16)). 16))) Using the Woodbury identity, we can evaluate QDA log-likelhhoods with (dr cost instead of (dds), while the number of free parameters per class is reduced from O(d) to O(d dr).
In GH-OFL, we can estimate De and U. directly from the aggregated moments in RP space, so DLR-QDA fits seamlessly into the same communication budget NB-diag and LDA.
A.2.2 EMPIRICAL COMPARISON
In Table 5 we compare full QDA and DLR-QDA (for three ranks) against NB-diag and LDA in RP space for CIFAR-100 and NEWSEWS. We report accuracy and relative memory/time footprints (normalized so that NB-diag has memory 1.0 and time 1.0).
Across both datasets we observe the following patterns:
On medium-scale RP dimensions (dRp = 256), DLR-QDA with r E [4, 8] tracks full- QDA accuracy within x 1-2 percentage points while using substantially less memory and compute.
Full QDA clearly more expensive and, in our setting, can underperform well- regularized LDA, whereas DLR-QU IDA with small behaves similarly to full QDA while being numerically simpler and more lightweight.
Relative to NB-diag and LDA, DLR- DLR-QDA offers flexible trade-off: with it reduces to NB-diag and increasing r gradually moves closer to full QDA in both expressiveness and cost.
14

---

Published as a conference paper at ICLR 2026
These results justify the use of in GH-OFL, especially when the encoder dimension or RP dimension is large and memory/compute budgets are constrained.
sketches a practical alternative full QDA
B EXTENDED PRIVACY CONSIDERATIONS AND COMPARISON WITH TRADITIONAL FEDERATED LEARNING
In GH-OFL, each client uploads only class counts and firt-/sseonn-oorde moments of frozen- encoder features, optionally after public random projection (RP) to < d dimensions. Because the summaries are not granular, many distinct datasets map to the same moments; RP then com- presses this information even more. By contrast, FedAvg (our proxy for traditional FL) transmits high-dimensional model parameters/gradiett across T > rounds. Below we formalize the privacy gap and provide fair comparison.
What a client actually sends. Client u holds Du { y1) )1 with Iy Rd and y {I...., Cj. Optionally the client computes z xf with public R Rdxk, k < d. then forms per-class and global sums
A
Σ
N = [ M1 =c}]
Fi=
((9)_ Σ xT
s(") Σ 2,2,
DC = Σ (xs x.). s:y:=c
Aggregating across clients yields Ac. No, S,, Dr. from which the server computes Ne: Te, E Eg (cf. main text, Sec. 3.1-3.2). NB-diag/LDN/ODA consume these directly; train- able heads (FisherMix, Proto-Hyper) are trained server-side synthetic Fisher-space samples and require no extra client uploads.
Why these messages leak less than model sharing. Non-uniqueness. Fixing (N, A, B/S/D) does not identify individual examples; infinitely many datasets match the same low-order moments (classical moment comeent-matchng ambiguity). Compression via RP. With z=rft, we have = ===R. BR, S = S,R; by the data processing inequality, any leakage functional that is mono- tone under post-procsssing cannot increase and the observable second-ordes structure drops from d(d-1) to k(*+1) One shot vs. multi round. FedAvg exposes parameters/rradients for T rounds; observations accumulate and typically reveal more than single fixed-size upload.
2
Compact formal statement. For the same frozen encoder:
Let L(M, Du.) be any leakage measure obeying data processing.
((NNAADAODA) S L(M
LMM MMMFahheMis/Pnna-Hyyyy = LMM
where "base" the underlying closed-fore head whose moments are used (typically LDA for Fish- erMix; NB-diag/LDA//QU for Proto-Hyper). The inequality is strict under mild conditions (e.g., per-class batch and/or le < d).
Practical caveats. Moment sharing still leak under tiny N: very fine-grained Se; recom- mend secure aggregation and tinssunnt on N₂ (drop/merge underpopulated classes) as in the main pipeline.
Why Secure Aggregation (vs. other privacy mechanisms). Our protocol combines coarse statis- ties with secure aggregation (SA), so that the server only observes client sums, never individual mes- sages. SA is strong fit here because the server-side estimators (NB/LDA/QDA and Fisher-space construction) depend solely on additive quantities (Ne; Ac, B, D.).
Compared to alternatives: (i) Local differential privacy (LDP, client-side noise) guarantees privacy per client even against a malicious server but in our setting would require adding noise to every coordinate of Ag, B, Sc, De. Since second-order terms scale O the total LDP noise grows quickly with the RP dimension and can significantly distort the moments, leading to a noticeable drop in accuracy even for moderate privacy budgets. (ii) Central DP (server-side noise) instead adds
15

---

Published as a conference paper at ICLR 2026
calibrated noise only once, after SA, the aggregated statistics to the final classifier parameters. This preserves accuracy much better as the signal is averaged over many clients before noise is added. The trade-off is that must trust the SA protocol and the DP accountant, while LDP does not rely on these assumptions. (iii) Homomorphic encryption / generic MPC protects individual updates but is heavier computationally and communication-wii this overhead is often prohibitive at mobile scale. (iv) Trusted execution environments reduce cryptographic overhead but introduce hardware trust assumptions and potential side-channel risks.
than SA for simple summations;
Relation to differential privacy. GH-OFL is orthogonal to DP and can be combined with both LDP and central DP. In practice, we view central DP on top of SA the most attractive compromise: clients first send exact moments through SA, the server aggregates them and then adds Gaussian (or Laplace) noise the global statistics or the final heads obtain formal (s, 8' d-D guarantees at population level. This design exploits the fact that our estimators depend only on sums, so the DP mechanism is simple and the noise can be tuned using standard sensitivity bounds on the sufficient statistics. LDP remains possible as clients could locally perturb their moments before SA but, due to the high dimensionality of second-order terms, the resulting utility loss is typically much larger for the same (e, 6). For deployments where the server is semi-trusted (or protected by SA/TEE) and where model quality is critical, central DP is therefore more realistic; LDP is better suited to highly adversarial server models, at the cost of accuracy.
Practical SA details. We use dropout-resilient masking (pairwise or group masks) so that, even with client churn, only the aggregate unseals. We also enforce minimum contribution threshold on N₂ (drop/merge underpopulated classes) to mitigate rare-class leakage and we can optionally quantize/round messages before SA limit precision without retraining costs. Overall, SA matches our additive objectives, adds minimal overhead and avoids the accuracy loss typical of local DP.
C SCALABILITY W.R.T. NUMBER OF CLIENTS: THEORY AND EMPIRICAL ABLATIONS
Scalability at a Glance. A key property of GH-OFL is its partition invariance. For a fixed global dataset D, redistributing its samples across any number of clients (e.g., CIFAR-I0 split across 5, 50 or 100 devices) does not alter the aggregated sufficient statistics (assuming a shared RP, if used) and therefore leaves the learned Gaussian parameters and the Fisher subspace unchanged (cf. Sec. 3. 3.143.4). This contrasts with traditional multi- hould- FL, where increasing the number of clients often induces client drift, noisier updates and accuracy degradation unless communication increases.
Additivity and theoretical invariance. Let = U D, be fixed. Summing all client uploads yields:
Σ (
Mr
dr,
N
1[(x,, v) E D: D::1 cbb
(6)
K
Σ xw
Σ s(N)
Σ
Ml = c) wx
(R.V)ED
(x.
The identities in Eq. Glsso hold for De. With shared random projection R. the same relationships hold in the projected dimension via linearity (A,=AR, B.-R" BR, S S. Closed- form head parameters (means, priors, pooled/class covariances. Fisher subspace) depend only on these global sums; hence they are independent of the number of clients K and of the particular par- tition. Trainable heads (FisherMix, Proto-Hyper) depend on synthetic samples whose distributions are deterministic functions of the same global moments, so they are invariant in expectation.
Experimental ablations on K and non-lIID level (Table To empirically validate the above invariance, we vary both the number of clients K and the Dirichlet non-lID parameter a. For CIFAR-10 and NEWS we keep the global dataset fixed and evaluate:
E (10, 20, 30, 40, 50, 100), For each (K, a) generate Dirichlet partition and run GH-OFL with the same encoder, RP dimension and heads as in the main experiments. We report the accuracy of the best GH-OFL head (NB-diag, LDA, QDA, DLR-QDA FisherMix Proto-Hyper).
a E {0.5, 0.1, (..00....0005).
16

---

Published as a conference paper at ICLR 2026
Table 6: Theoretical scalability as a function of the number of clients K (per-client upload, total communication and accuracy behavior).
Method
Per-client upload
Total (K clients)
Accuracy K
NB-diag (RP k)
C(1 2k-
K C(1 2k) K C(1 k|C(1 + k) A(h-1)
Invariant
LDA (RP k)
C(1 + k) + R(h+1)
Invariant
QDA (RP k)
c(1 + A(++D)
K KC le ++ k(k+1)
Invariant
FisherMix (on LDA)
same as LDA base
same as as base
Invariant (in expectation)
Proto-Hyper (on base)
Invariant (in expectation)
FedAvg
p per round
T'p
Can degrade (needs tuning)
Table 7: Accuracy (%) of the best GH-OFL head under varying number of clients K and Dirichlet concentration a or class-per-client setting (#C). CIFAR- 10, same encoder and RP dimension as in main experiments. Accuracy remains stable across all configurations.
Setting
K
10
K
20
K
30
K
40
K
50
K 100
a 0.5 (mild non-IID)
87.4
87.6
87.5
87.3
87.2
87.3
0.1 (non-IID)
87.1
87.2
87.0
86.9
86.8
86.9
0.05 (strong non-Ill)
86.1
86.3
86.2
86.1
86.0
86.1
#C (two classes per client)
86.9
87.0
86.8
86.7
86.6
86.7
#C (one class per client)
86.1
86.0
85.9
85.8
85.8
85.9
Discussion.
Across all configurations, accuracy remains effectively stable as increases:
For fixed global data and a, increasing K from 10 to 50 produces only marginal changes (< 1pp) in the best GH-OFL accuracy.
The relative ranking among heads stable: LDA and Proto-Hyper typically dominate.
When is small (strongly non-IID) and K is very large, some classes become underrep- resented on certain clients, slightly degrading all GH-OFL heads in similar way, due to noisier moment estimates.
Takeaway. As shown in Table and both the theoretical analysis and the empirical ablations agree: GH-OFI. scales gracefully to large client populations. Per-client communication remains constant, the total uplink grows linearly global sample size and class coverage, not by the number of clients (small deviations in accuracy in this setting arise from external factors such as stochastic minibatch sampling, random Dirichlet partitions, model initialization seeds and the inherent noise of training dynamics). By avoiding multi-round on-device optimization, GH-OFL avoids the degradation trends typical of traditional FL K increases.
and, crucially, accuracy is governed by the effective
C.1 COMPUTATIONAL PERFORMANCE ANALYSIS
The computational analysis reported in Table g top highlights the significant efficiency advantages of GH-OFL. compared to classical FL methods. Conventional approaches such as Centralized FT, FedAvg and Co-Boosting/Deensef operate in multi-round or data-dependent regime, tipically requiring access to full model updates and consequently incur substantial GPU time (30-90 minutes), high memory consumption (6-9 GB) and expensive communication overhead. In contrast, all GH- OFL variants operate in one-shot setting, relying exclusively compact client-side statistics or low-dimensional random projections and therefore never require access to complete updates or multi-round synchronization.
This design leads to GPU time reductions of more than one order of magnitude: all GH-OFL variants execute in approximately 1-2.5 minutes both CIFAR-100 and AGNEWS, while also
17

---

Published as a conference paper at ICLR 2026
Method
GPU Time (min)
Train data used
GPU Mem (GB)
Applicability
C100
AGN
C100
AGN
Centralized FT
~-42
300
full row dalinet
7
~600
requires server raw data
FedAvg (mulii-round)
66
updates over i2 rounds
55
heavy comm, beterogeneity
issues
Co-Boosting DenseFL
~31-~/3
26
raw activations/logits across rounds
~ ~94-~
7.00
multi-round expensive up- loads
GH-OFL variants (ours)
CH--OFL-NB (diug)
~11
~0.88
synthetic stats only (A.N)
~0.85
0066
lightest variant; closed form rank cow; closed full cov.; heaviest head
~00.9
synthetic (A.D,N)
0.70
GH-OOLL-QDA-tiI
~-777
11.6
synthetic (A.B.D.N)
~3.88
2.0
GII DLR QDA (r=8)
~1.6
1=1.2
synthetic (A.B.D,N)
0099
low rank sketch QDA
1444
synthetic feats small gen-
110
0.88
lightweight head training
crator
GH--OHL-Proo-HHppr
synthetic feats only
09
hypernetwork: very cheap
Method/ Config
Stats
Upload
Total MB
Head
cost
C100 Ace.
AGN Acc.
(uss/ample)
C100
AGN
C100
AGN
FedAvg (1 round)
Full model 0
43859
261543
428.3
2554.1
4-6
20-25%
70-75%
FedAvg (R)
full model Ø
43859 x R
261543 R
428.3 R
2554.1 x
4-6 x
59-63%
85-88%
Co-Bousting
model
43859
428.3
17.49%
88
DenseLL
Full model 0
43859
261543
428.3
2554.1
5,5-7.0
46-50%
88-89%
GH-OFL (RP-512)
A.B.D.N
8.92
58.4%
GH-OFL (RP-256)
A.D.N
200
1.96
0.08
0.05-25.0
56.1%
88866
GH-OFL (RP-128)
A.N
50
0.49
0.02
0.05-25.0
53.2%
87.8%
computational overhead comparison enriched with empirical GPU time, memory and head costs for classical FL and GH-OFL variants. Bottom: compact ablation one-shot commu- nication vs. accuracy (CIFAR-100 + AGNEWS) and empirical head cost, fully contained within single-column layout.
Table 8: Top:
reducing GPU memory to below GB for most heads. Furthermore, GH-OFL. training involves only lightweight head optimization on synthetic features, making it substantially easier deploy in resource-constrained settings and robust to system heterogeneity. We reported approximate GPU time and memory values that reflect the average operational range observed across multiple runs under our experimental setup. Although absolute measurements naturally depend on the underlying hardware and system load, the reported values capture the expected regime of each method and are consistent enough to support the comparative conclusions.
The table bottom reports targeted ablation study isolating the communication/cccurazyy trade-off induced by different RP dimensions (512, 256, 128). By progressively reducing the RP dimension, the size of the client upload shrinks from hundreds of kilobytes to only a few kilobytes, while aceu- racy degrades smoothly and predictably. This ablation demonstrates that GH-OFL offers tunable operating regime: users may select a configuration that balances communication cost, head complex- ity and accuracy according to system constraints. Remarkably, even in the most compressed setting (RP-128), GH-OFL preserves competitive accuracy on both CIFAR-10 and AGNEWS while re- ducing communication by up three orders of magnitude compared to FedAvg and other classical FL. baselines.
Together, the results confirm that GH-OFL consistently achieves high accuracy with drastically lower computational, memory and communication requirements, positioning it as a practical and scalable alternative to multi-round FL for real-world heterogeneous deployments.
D EMPIRICAL ANALYSIS OF GAUSSIAN ASSUMPTIONS AND FISHER SUBSPACES
This section provides additional empirical evidence supporting the two main modeling choices in GH-OFL: (i) the use class-conditional Gaussian heads estimated from sufficient statistics and (ii) the restriction of the head to a now-dinensional Fisher subspace. We report diagnostics across multiple vision and NLP benchmarks and quantify how much discriminative performance is affected by these approximations.
18

---

Published as a conference paper at ICLR 2026
D.1 UNIVARIATE DIAGNOSTICS OF GAUSSIANITY
Our head models feature embeddings Niue, 2c); estimated from class-wise first and second moments communicated by the clients. This does require the true distribution to be exactly Gaussian but it is important understand how far real embeddings deviate from this idealization.
R& via class-conditional Gaussians,
~
Given aggregated sufficient statistics, we approximate, for each feature dimension and class the univariate skewness 71 (n-) and excess kurtosis 72
(e.5) of the embedding coordinates. We then
summarize their absolute values 17/-5)), ln2 (e.i) 3 across all classes and dimensions, reporting mean, median and 90th percentile. Values close to zero would indicate near-Gaussian marginals, whereas large values indicate heavier tails strong asymmetry.
Vision benchmarks (ResNet-18). For all vision datasets we extract embeddings from ResNet- 18 backbone (ImageNet-pretrained) and compute diagnostics from the federated sufficient statistics:
CIFAR-10 (10 classes):
mean sskew| = 1.69, median 1.64, 90th 2.34;
mean excess kurtosis = 4.22, median 3.41, 90th 7.61.
CIFAR-100 (100 classes):
mean |skew] 1.91, median 1.83, 90th 2.61; excess kurtosis = 5.64, median 4.57, 90th 10.52.
mean
SVHN (10 classes):
mean sskew] = 2.35, median 2.06, 90th 4.09; mean excess kurtosis 11.05, median 5.97, 90th 25.27.
These values are clearly non-zero, confirming that exact Gaussianity does not hold; however, they are far from pathological (no extreme skewness or infinite-variance tails). Empirically, class- conditionals live in moderately non-Gaussian regime where Gaussian discriminants remain rea- sonable and compact approximation.
NLP benchmarks (DistilBERT). To test whether this picture extends beyond vision (see also Appendix A we repeat the same diagnostics on three standard benchmarks, using embeddings from a frozen DistilBERT encoder:
AGNews (4-way news classification),
DBPedia-14 (14-way ontology classification),
SST-2 (binary sentiment classification).
The resulting skewness and kurtosis values are much closer to zero than the vision case: for exam- ple, AGNews we obtain mean [skew] 0.17 and mean [kurtE| 0.24 (90th percentiles 0.36 and 0.55 respectively); DBPedia-14 and SST-2 show slightly larger, but still moderate, departures from Gaussianity. Table summarizes mean and 90th-percentile absolute skewness and excess kurtosis across all datasets.
Overall, these diagnostics suggest that class-conditional embeddings across both vision and lan- guage tasks exhibit moderate, but not extreme, departures from Gaussianity. This supports the use of Gaussian discriminant heads compact summary the information that can be communicated via low-order moments in one-shot, data-free federated learning.
D.2 INFORMATION PRESERVATION IN THE FISHER SUBSPACE
The GH-OFL head operates a low-dimensional Fisher subspace spanned by the top general- ized cigenvectors of the between-class and within-class scatter matrices. In the classical Gaussian- Shared-covariande model z = ~ N(H,, E), the Fisher subspace of dimension at most C' is sufficient for Bayes-optimal classification: all discriminative information lies in that subspace.
19

---

Published as a conference paper at ICLR 2026
Table 9: Univariate Gaussianity diagnostics for class-conditional embeddings: mean and 90th per-
centile of absolute skewness and excess kurtosis across features. Vision datasets use a ResNet-18 backbone; NLP datasets use DistilBERT.
Dataset
mean Iskew
90th |skew]
mean |kurtE]
90th kurtE
CIFAR-I0
1.69
2.34
4.22
7.61
CIFAR-100
1.91
2.61
5.64
10.52
SVHN
2.35
4.09
11.05
25.27
AGNews
0.17
0.36
0.24
0.55
DBPedia- 14
0.32
0.64
0.60
1.21
SST-2
0.27
0.50
0.57
0.86
In practice, embeddings are only approximately Gaussian and covariances are estimated from finite samples. To quantify how much discriminative information actually lost by the Fisher projection, compare:
Full-space LDA, trained on the aggregated class means and pooled covariance in the original feature space (d 312 for ResNet-18; 768 for DistilBERT).
2. Fisher-subspace LDA, trained the same statistics projected onto the top-k Fisher directions for varying k.
We report test accuracy AccFistost captured by the top-k eigenvalues.
Fisherk: as a function of k, together with the fraction of "Fisher energy"
Vision benchmarks. On the three vision datasets considered, observe that relatively small number Fisher directions suffices to essentially match full-space LDA:
CIFAR-10 (10 classes, d = 512):
full LDA:
86.26 26%;
Fisher-k: 69.55% (@k=4, energy 0.71) + 82.86% (@k=8, energy 0.97) + 86.26%
(@k=16, energy Rs 1.00).
CIFAR-100 (100 classes, 512):
full LDA: 64.12%;
Fisher-k: 19.02% (@k=4, energy 0.26) + 53.25% (@k=32, energy 0.77) - 64.15% (@k=128, energy 1.00).
SVHN (10 classes, d 512):
full LDA: 62.88%;
Fisher-k: 51.98% (@ 44, energy 0.72) - 62.12% (@k-8, energy 0.97) 62.86% (@k=16, energy s 1.00).
In all cases, once k: is large enough to capture Ry 95-99---000 of the Fisher energy (typically A: [8, 32] for 10-class tasks and 128 100 classes), Fisher-subspace LDA matches or slightly exceeds full-space LDA. This indicates that the directions discarded by the projection carry little additional discriminative information.
Extended benchmarks and robustness (Table 10). We perform the same experiment on three NLP datasets (AGNews, DBPedia-14, SST-2, using DistilBERT embeddings) and on a robustness benchmark, CIFAR-100-
which evaluates the CIFAR- 100 classifier under 19 types of common reuse the class statistics estimated on
corruptions five levels of severity. For CIFAR-100- clean CIFAR- 100 and only change the test distribution.
On AGNews obtain full-space LDA accuracy 90.290..0000 90.28%, while LDA in the Fisher subspace reaches 90.29 90.29% already at k=128 with Fisher energy x 1.00. DBPedia-1 14 even more concentrated: full- space LDA achieves 98.87% and Fisher-subspace LDA reaches 98.88% at k= (energy 1.00). SST-2 shows full-space accuracy 83.60%, which is exactly matched at k=16 with Fisher energy 1.00.
20

---

Published as a conference paper at ICLR 2026
Table 10: Information preservation the Fisher subspace. For each dataset, we compare full-space LDA with LDA restricted to top-k Fisher subspace. We report full-space accuracy, best Fisher- subspace accuracy, Fisher energy at k*, the number of components k* and the resulting compression ratio d/k".
Dataset
LDAnoll
LDA Fisher-k
Fisher energy at ke'
k*
Compression (d/k")
CIFAR-10
86.26%
86.26%
0.99
16
32.0x
C10AR-100
64.12%
64.15%
Rs 0.99
128
4.0x
SVHN
62.88%
62.88%
R 0.99
16
32.0x
AGNews
90.28%
90.29%
R 0.99
128
6.0x
DBPedia-14
98.87%
98.88%
0.99
32
24.0x
SST-2
83.60%
83.60%
2 0.99
16
48.0x
CIFAR-100-4 (avg sev.)
39.37%
39.35%
R3 0.99
128
4.0x
For CIFAR-100-C, averaging across the five severities, full-space LDA attains 39.37% accuracy. Restricting LDA to Fisher subspace of dimension k=128 (capturing essentially all Fisher energy) yields a mean accuracy of 39.35% across severities, i.c., practically identical full-space LDA despite the strong corruptions.
LDA Across all considered benchmarks, we consistently observe that Fisher & Acc once the Fisher energy is close to 1.0. This provides direct, quantitative argument that the Fisher projection used by GH-OFL discards very little discriminative information, while enabling strong reduction the dimensionality of the head.
LDA
Why LDA is used probe. In these diagnostics deliberately focus LDA the probe classifier, for three reasons:
LDA is the canonical classifier under the Caussian-shared-ovarii model: it is Bayes- optimal when the assumptions hold, hence the most natural tool for testing the impact of Gaussian and Fisher approximations.
LDA can be reconstructed in closed form directly from the same aggregated moments that GH-OFL receives (class means and pooled covariance), without extra training, hyperpa- rameters optimization noise; this keeps the diagnostic fully aligned with our data-free communication model.
LDA provides a conservative baseline: if even this simple linear model does not lose ac- curacy when restricted the Fisher subspace, flexible heads operating the same subspace (e.g., -tA-style variants, ProtoHyper residual heads) can only match or improve upon this behavior.
D.3 DISCUSSION
The analyses above support the two key modeling choices in GH-OFL:
1. Gaussian heads from low-order moments. Across diverse vision and NLP benchmarks, class- conditional embeddings exhibit moderate but extreme deviations from Gaussianity. In this regime, Gaussian discriminants offer an effective trade-off between expressivity and the strict communication constraints of one-shot, data-free federated learning.
Fisher-subspace restriction. Empirically, projecting embeddings onto a Fisher subspace that captures & 95-100% of the Fisher energy leads to LDA accuracy that is essentially indistin- guishable from full-dimensional LDA on all considered datasets. This suggests that the discarded directions carry little additional discriminative power, while the dimensionality reduction signifi- cantly simplifies the head.
At the same time, these results do not exclude scenarios where the Gaussian approximation may be insufficient (c.g., highly multimodal or heavy-tailed per-class distributions). In such cases, ex- tensions like Lixture-of-Gaussian heads, robust covariance estimation non-linear heads defined
21

---

Published as a conference paper at ICLR 2026
in the Fisher subspace natural directions for future work and remain compatible with the same moment-basee communication protocol used by GH-OFL.
E DEEPENING AND DISCUSSION ON MULTI-ROUND BASELINES AND REPRODUCIBILITY
We detail here the hyperparameters used for the multi-round federated baselines (FedAvg and Fed- Prox) and, following, a detailed discussion about the results obtained empirically. All choices designed to ensure full reprodueibility while maintaining methodological fairness with the GH-OFL framework through shared initialization, identical preprocessing and identical client partitions.
E.1 HYPERPARAMETERS FOR MULTI-ROUND FL BASELINES
Backbone and training protocol. All federated baselines use ResNet-18 pretrained on Ima- geNet-IK, matching the feature extractor used throughout the main paper. Differently from GH- OFL, which operates on frozen embeddings, FL traditional methods perform full-model fine-tuning, as this corresponds to their standard formulation in order to create a strong and fair baseline. Freez- ing the encoder would artificially weaken these methods, whereas starting from the same pretrained weights ensures comparable initial representation space across all approaches.
Non-IID client partitions. Data are split across clients using a Dirichlet distribution with (0.05, 0.10, 0.50), covering severe, moderate and mild heterogeneity. The resulting partitions are fixed and reused across all methods, ensuring identical local data difficulty and label skew.
Local optimization and objective. Each client trains for one local epoch using SGD with momen- tum (Ir 0.001, momentum 0.9, batch size 256). FedAvg minimizes the standard cross-entropy loss while FedProx adds the proximal regularizer
11 [|w w. Lprox LEE ""
/4 0.01,
which stabilizes local updates under strong heterogeneity.
Server aggregation and communication budget. Global updates follow the canonical sample-size weighted averaging:
Σ &
((+1) n& W
Wi++1
Σn TAA
where n the number of local samples at client k. We evaluate E {1, 10, 100} communication rounds, enabling comparison between one-shot methods (R 1), lightly interactive FL (R 10) and communication- intensive regimes (R = 100).
Reproducibility. All experiments use fixed seeds, identical client splits, the same pretrained initial- ization and the same preprocessing pipeline, ensuring that the reported results are fully deterministic and directly comparable across baselines.
E.2 CLASSICAL MULTI-ROUNT FL vs. GH-OFL: ADDITIONAL ANALYSIS
We report experimental curves on CIFAR-1O 10, CIFAR- and SVHN under Dirichlet client splits with a E {0.50, 0.10, 0.05) and analyze their behavior in relation to communication cost, sensitivity heterogeneity, stability and convergence speed.
E.3 BEILAVIOR OF CLASSICAL FL METHODS
Figure 4shows that classical FL algorithms are strongly affected by client heterogeneity. On CIFAR- 10, FedAvg exhibits delayed convergence for small values, whereas FedProx reduces oscillations by penalizing large client drift. On CIFAR-100, the effect is further amplified due to the dataset complexity: both methods converge slowly and remain far from the central optimum, especially under a = 0.05. On SVHN, the easier visual structure leads to faster convergence but performance remains sensitive non-Ill partitions.
22

---

Published as a conference paper at ICLR 2026
100
100
50-
90-
1op Accurcy (6)
80-
70-
60-
50-
40
6--3
30
20
15
20
25
30
35
45
50
10
20
25
40
Communication unu
Communication Ro und
SCAFFOLD on DIARR11 with Dirihlet diint spilts
Fedligg on CITAR-20O with Drichlet client sp
90
Top Acoursiy (861
80-
80-
60-
60
50-
30-
30
10-
Communication
FudProx 01) CIRAR-100
Birihheet client
SCAFFOLD CIFAR-100 Déichlet client
100
100
90
Toe Acouracy (Ng)
70
70
60
50
40-
40
a-6.2
20
2
g-0.35
10
5
15
20
25
30
35
40
45
50
10
20
25
30
40
Communication Round
Communicatin Bound
FedAvg on SVHN with Dirchlet client spilts
FedProx (p- 01) on SVHN with Dirichiet client splits
100
90
Top Accursty 1961
00
n0-
60-
50
30
3-3
30
10
20
Communication
Communication
SCAFROLD SVHN with Circcet
90
1op-1 Accurary
70
80
40
oregen
231
20
10
72
1.5
20
30
25
40
30
Communication Round
Figure 4: Convergence curves of classical multi-round federated learning baselines (FedAvg. Fed- Prox and SCAFFOLD) on CIFAR-10, CIFAR-100 and SVHN under Dirichlet client partitions (a {0.50, 0.10, 0.05)). All methods use identical hyperparameters: 10 clients, 50 communi- cation rounds, one local epoch per round, SGD with momentum 0.9, learning rate 10-3 batch size 256 and full-model transmission. FedProx reduces client drift relative FedAvg, especially under stronger heterogeneity. SCAFFOLD applies control variates to correct client drift; however, when restricted to single local epoch per round, its variance-reduction mechanism operates in limited regime (Li et al. 2021) and does not consistently outperform FedAvg or FedProx.
23

---

Published as a conference paper at ICLR 2026
Across all datasets, classical FL requires tens of communication rounds to reach competitive per- formance and each round involves transmitting the full set of model weights between entities. This significantly increases communication cos while stability also degrades under strong heterogeneity. These patterns are consistent with known limitations of multi-round FL. and match the degradation trends discussed in Table the main paper.
E.4 COMPARISON WITH GH-OFL
In contrast to traditional FL. methods, the GH-OFI. family operates in strictly one-shot commu- nication regime. Each client sends only aggregated per-class statistics and the server constructs either closed-form Gaussian heads (NB-diag. LDA, QDA) trainable Fisher-space heads (c.g., FisherMix, Proto-Hyper). As described Section 3 of the main manuscript, these estimators are partition-invariant they depend only global first and second-order moments and not the spe- cific Dirichlet configuration.
As result:
GH-OFL is robust to non-IID heterogeneity. Since the aggregated statistics unbiased estimators the global distribution, performance remains unchanged even for extreme a values. This sharply contrasts with the degradation observed in multi-round FL.
Communication is reduced by multiple orders of magnitude. Traditional baselines must transmit full model parameters repeatedly across 50+ rounds. GH-OFL requires a single transmission of lightweight statistics whose size is independent of model depth and dataset scale.
Accuracy matches or surpasses multi-round FL. As shown in Table of the manuscript, GH-LDA and the Fisher heads consistently outperform traditional OFL and FL methods in one-shot settings and sometimes even after many communication rounds with the conver- gence reached.
E.5 SUMMARY
These results highlight a fundamental distinction between classical and one-shot FL. Multi-round methods suffer from slow and unstable convergence under heterogeneous settings while GH-OFL leverages principled Gaussian and Fisher-space formulation to achieve high accuracy in sin- gle communication round. The curves presented here complement the main paper by showing the empirical limitations of three traditional FL baselines such as FedAvg, FedProx and SCAFFOLD thereby reinforcing the motivation for one-shot, statistics-driven federated learning.
Code Availability. The complete multi-round FL baseline implementations (FedAvg, Fed- Prox and SCAFFOLD) used in this study are publicly available at: httts: httts:://githbb.. com/ FabioTur dev/ Sttps//AAFFOOLDbaaeee
The official implementation of the GH-OFL family, including Gaussian and Fisher-space heads, is FL.
available at: htttp:://// wwwpahccooaaaooooooooooooo
24


Reviews:
[{"review_id": "ViNiqzaJ2i", "reviewer": "Reviewer_QtMw", "summary": "This paper presents GH-OFL, a novel family of one-shot federated learning methods that achieves state-of-the-art performance while being strictly data-free and communication-efficient. The work is technically sound, methodologically innovative, and addresses practical constraints effectively.", "questions": "Please refer to weaknesses for details.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The proposed method is strictly data-free, requiring no public datasets or client-side inference, which is a major advantage for privacy.\n2. The use of sufficient statistics and random projection sketches makes the approach highly communication-efficient.\n3. The paper is generally well-written, with a clear narrative and a logical flow from problem to solution.", "weaknesses": "1. It relies on the core assumption that the feature embeddings for each class follow a Gaussian distribution, which may not hold perfectly in practice.\n2. The Fisher subspace projection, while beneficial for dimensionality reduction, may discard some discriminative information present in the discarded dimensions.\n3. The paper does not sufficiently explore the limitations of the Gaussianity assumption for certain types of data.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "77JDZ3mMuq", "reviewer": "Reviewer_G9mr", "summary": "The paper proposes GH-OFL, a one-shot FL framework where clients send only per-class sufficient statistics (counts and first/second moments of frozen-encoder features, optionally after a public random projection). The server constructs (i) closed-form Gaussian heads (NB-diag/LDA/QDA), (ii) FisherMix, a cosine-margin linear head trained on synthetic Fisher-space samples, and (iii) Proto-Hyper, a low-rank residual head distilled from a Gaussian teacher.", "questions": "- Can you provide more experiments on non CV tasks?\n- Can you discuss the potential privacy-preserving methods (i.e., DP) integrating with proposed approaches?\n- For high-dim encoders, how do diagonal-plus-low-rank sketches compare to full QDA in accuracy vs. memory/compute?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "+ The moment set is additively aggregable and sufficient to instantiate NB-diag/LDA/QDA; random-projection sketches reduce bandwidth and preserve additivity. The derivations for means/covariances and pooled covariance are explicit.\n\n+ The paper motivates a Fisher subspace generalized eigen problem and uses data-free Gaussian sampling there to train FisherMix and Proto-Hyper, offering practical gains when closed-form heads are biased.\n\n+ Results compare GH-OFL heads to OFL baselines across datasets and backbones show consistent advantages where expected.", "weaknesses": "- The datasets used are all computer-vision datasets, which limits the scalability of the proposed approach.\n\n- Privacy discussion is not deep and the privacy analysis is qualitative.\n\n- The experiments in this paper are not extensive, i.e., the ablation study of FL (e.g., number of clients) is not well discussed.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "NWCgYZkb3W", "reviewer": "Reviewer_tfGS", "summary": "The paper proposes Gaussian-Head One-shot Federated Learning (GH-OFL), a data-free framework that transmits only statistical summaries to achieve state-of-the-art accuracy and robustness under non-IID settings with a single communication round. The core framework of data-free is about synthetic labeled pairs for one-shot FL.", "questions": "- How about a experiments that the normal FL gradually transits into a one-time by reducing the number of communication rounds?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- Comprehensive alternatives and corresponding clear discriptions are provided.\n- Ablation study on modules shows the importance and effects of each components.", "weaknesses": "- Comparison of the computational overheads between data synthesis and training with public datasets is anlayses with details and empirical evidence.\n- Ablation study would better demonstrate the trade-off between communication and performance applying one-shot FL.  \n- Baselines of general FL should be provided to general audience to support the advantage of one-shot FL. In Dirichlet of 0.5 cases, FedAvg reach a accuracy at least 86.02 on ResNet-18, which is not mentioned in the main table.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)