You Only Communicate Once: One-shot Federated Low-Rank Adaptation of MLLM
Authors: Binqian Xu
Haiyang Mei
Zechen Bai
Jinjin Gong
Rui Yan
Guo-Sen Xie
Yazhou Yao
Basura Fernando
Xiangbo Shu
Venue: NeurIPS 2025 poster ()
Keywords: MLLMs
One-shot FL
LoRA
Adaptive Ensemble
True One-shot Communication
Primary area: social_and_economic_aspects_of_machine_learning

Abstract: Multimodal Large Language Models (MLLMs) with Federated Learning (FL) can quickly adapt to privacy-sensitive tasks, but face significant challenges such as high communication costs and increased attack risks, due to their reliance on multi-round communication. To address this, One-shot FL (OFL) has emerged, aiming to complete adaptation in a single client-server communication. However, existing adaptive ensemble OFL methods still need more than one round of communication, because correcting heterogeneity-induced local bias relies on aggregated global supervision, meaning they still do not achieve true one-shot communication. In this work, we make the first attempt to achieve true one-shot communication for MLLMs under OFL, by investigating whether implicit (i.e., initial rather than aggregated) global supervision alone can effectively correct local training bias. Our key finding from the empirical study is that imposing directional supervision on local training substantially mitigates client conflicts and local bias. Building on this insight, we propose YOCO, in which directional supervision with sign-regularized LoRA B enforces global consistency, while sparsely regularized LoRA A preserves client-specific adaptability. Experiments demonstrate that YOCO cuts communication to $\sim$0.03\% of multi-round FL while surpassing those methods in several multimodal scenarios and consistently outperforming all one-shot competitors.

You Only Communicate Once: One-shot Federated Low-Rank Adaptation of MLLM
Binqian Xu', Haiyang Mei', Zechen Bai', Jinjin Gong', Rui Yan', Guo-Sen Xie Yazhou Yao Basura Fernando and Xiangho Shu
Nannjnng University of Science and Technology Show Lab, National University of Singapore IInstitute of High- Perfo-Performance Computing, S*SARR https://8e thub. wtt.cooo///o// 2ttmm/1bgg//eeMLL
Abstract
Multimodal Large Language Models (MLLMs) with Federated Learning (FL) can quickly adapt to privacy-sensitre tasks, but face significant challenges such as high communication costs and increased attack risks, due to their reliance on multi- round communication. To address this, One-shot FL (OFL) has emerged, aiming to complete adaptation in a single client-server communication. However, existing adaptive ensemble OFL methods still need more than one round of communication, because correcting heterogencity-induuee local bias relies on aggregated global supervision, meaning they still do not achieve true one-shot communication. In this work, we make the first attempt to achieve true one-shot communication for MLLMs under OFL, by investigating whether implicit (i.e., initial rather than aggregated) global supervision alone can effectively correct local training bias. Our key finding from the empirical study is that imposing directional supervision on local training substantially mitigates client conflicts and local bias. Building on this insight, we propose YOCO, in which directional supervision with sign-regularized LoRA B enforces global consistency, while sparsely regularized LoRA A preserves client-specific adaptability. Experiments demonstrate that YOCO cuts communi- cation to ~0.03% of multi-round FL while surpassing those methods in several multimodal scenarios and consistently outperforming all one-shot competitors.
Introduction
Multimodal Large Language Models (MLLMs) have rapidly advanced in recent years 29,, driven by their powerful capabilities and broad applicability, achieving impressive performance across diverse multimodal tasks through fine-tuning [57
62 However, privacy-sensitive scenarios such
as healthcare, centralized data collection for fine-tuning is often impractical 43
prompting growing interest in adapting MI.LMs via Federated Learning (FI.) [54 53 63). Although Parameter-
37 52
Efficient Fine- Fine-Tuning (PEFT) methods [19 23 help reduce communication costs in these approaches, federated fine-tuning still involves transmitting large amounts of parameters across multiple rounds, resulting in high system overhead and increased security risks 44 511 as shown in Figure I (a).
One-shot FL (OFL), involving single round of communication, is an advantageous FL paradigm, with the communication cost equivalent to that of a single model parameter [32 42 Specifically, without relying on additional data or generative models, adaptive ensemble OFL methods 42
achieve good performance by effectively correcting local training biases using supervision derived from aggregated global weights. "Local training bias" refers to overfitting local data during client-side training, leading to spurious correlations caused by data heterogeneity and degrading generalization.
Corresponding author
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

-T
Multi-round communication
Pseudo one-shot communication
True one-shot communication
(T> 1)
Client
(Local)
Server
(Global)
(a) Traditional
(b) FuseFL
(c) FENS
(d) YOCO (Ours)
Figure I: (a) Traditional FL involves multi-round communication, resulting in high system overhead and increased security risks. (b) OFL: FuseFL 42 is a pseudo one-shot communication method that runs beyond single round, guided by global supervision from bottom-up aggregated blocks. (c) OFL: FENS [3] is a pseudo one-shot communication method that operates for more than one round, guided by global supervision from multi-round lightweight aggregators. (d) YOCO true one-shot communication method, guided by implicit global supervision. Without additional data or generative models, FuseFL and FENS are the two most representative adaptive ensemble OFL methods.
To mitigate this, "global supervision", represented by aggregated weights containing shared knowl- edge, is used to guide local training. However, the generation and propagation of global supervision contradict the single-round (one-shot) communication requirement, shown in Figure (b) and (c).
In contrast, as shown in Figure (d), during one-shot federated low-rank adaptation of MLLMs, all client models share the same pre-trained weights. Unlike the aggregated "global supervision" mentioned above, pre-trained weights, defined as "implicit global supervision" contain more universal and generalized knowledge. With true one-shot communication to reduce overhead and attack risks, using implicit global supervision to guide local LoRA training results in the core problem shifting to,
Can implicit global supervision in true one-shot comm. correct local bias?
To answer this question, we explore bias correction for local LoRA training, supervised by the magnitude and direction of implicit global supervision. Here, each LoRA [23] weight consists of an A matrix (LoRA A) and a B matrix (LoRA B). For weights, the magnitude refers to the absolute value, while the direction refers to the sign of each parameter. Based on our empirical study, we draw the following two important conclusions: Direction constraints cause less interference with optimization than magnitude constraints. Constraining only the direction of LoRA B, combined with prior initialization, further reduces conflicts and helps correct local LoRA training bias (Section B
Based on the above conclusions, we propose YOCO (Figure 2 true one-shot communication method. In YOCO, the initial direction across clients follows the prior direction (Figure 2 (a)), using matrix decomposition of pre-trained weights as initial weights of LoRA (for both LoRA and B). During local training (Figure[](hh) (b)), progressive regularization guides the updates of LoRA B toward the direction implicit supervision, by measuring the disparity in the signs of parameters, thus mitigating aggregation conflicts from model discrepancies. While this consistency constraint benefits aggregation, it may hinder adaptation to client-specific knowledge. In response this, we introduce sparse regularization to LoRA A, encouraging the learning of critical client-specific knowledge, followed by woise-free aggregation weighted by principal components (Figure ( (c)). Compared to outperforming
multi-round methods, YOCO uses only ~0.03% of the communication cost, them in both aligned and missing modal scenarios on the CrisisMMD dataset. In the true one-shot communication setting, YOCO consistently outperforms all baselines, proving its effectiveness.
Our main contributions can be summarized as follows,
True one-shot communication. We are the first to explore true one-shot communication for MI.LMs with OFI., addressing the key challenge of whether implicit global supervision can effectively correct local training bias in this true one-shot communication setting.
Balance consistency and adaptability. We propose YOCO, which uses implicit global supervision to correct local training bias. Derived from this supervision, prior initialization and sign- based regularization on LoRA B ensure global consistency, while sparsity regularization and principal component- weighted aggregation on LoRA A enhance local adaptability.
Communication-aware Comparison. Compared to multi-round communication methods, YOCO requires only about 0.03% of the communication cost and even outperforms these methods in some

---

(a) Initialization
(b) Local Training
[c) Aggregation
Elect Sign
2
B
PCA
Initialize
A
0
Epoch
p
Sparsify
Matrix Decomposition
Vanilla Aggregation
Figure 2: Illustration of YOCO. (a) Initialization: LoRA weights are initialized from the matrix decomposition of pre-trained weights, providing a shared global prior direction and the sign (i.e., +/-) of each parameter for LoRA B. (b) Local Training: In addition to the original task loss, LoRA is incrementally regularized by the prior sign to maintain global consistency across clients, while LoRA A is sparsely regularized with the & norm to strengthen client-specific knowledge and minimize irrelevant noise. (c) Aggregation: LoRA uses vanilla averaging due to lower cross cross-cliem client aggregation to enhance adaptability.
conflict, while LoRA A adopts principal component-weighhe
multimodal scenarios. When compared multiple true one-shot communication baselines, YOCO delivers the best performance across all multimodal scenarios.
2 Background
One-shot Federated Learning (OFL) involves single communication round, with cost equivalent to the size of one model's parameter [32] 42). Compared to traditional FL, OFL offers notable advantages privacy and efficiency [16]. Existing OFL methods are generally categorized into four
types: static ensemble, adaptive ensemble, knowledge distillation, and generative models. Knowledge distillation and generative models 58
10 21 are closely linked. Specifically, knowledge distillation includes both data [66] 40) and model distillation [33 64 [5], with data sourced from both public and generative model-generated synthetic data. Unlike the methods above, static and adaptive ensemble methods offer greater flexibility in combining with others. Early approaches focused static ensemble [14 24 45), analyzing local clients' model statistics to derive global parameters. Recently, leverage aggregated global supervision to correct local bias, and improve generalization. In contrast to pseudo one-shot communication methods such as FuseFL [42 and FENS (3], YOCO first enables true one-shot communication for federated low-rank adaptation of MLLMs via implicit global supervision. More detailed related works are provided in the appendix.
adaptive ensemble methods 42 K
Empirical Motivation
Magnitude Direction. To investigate
the key question "Can implicit global supervi-
3.0
unconstraints
sion in true one-shot communication correct lo- cal training bias?", we first decouple implicit global supervision into two distinct constraint components-magnitudee and direction and ap- ply them separately to guide local LoRA train- ing. As illustrated in Figure 3 we randomly se- lect client and present the training loss curves of LoRA under three different settings: (i) the original training without any constraints; (ii) training with only a magnitude constraint, where the L2 norm is used to measure the distance be- tween LoRA weights and the implicit global supervision; and (iii) training with only a di- rection constraint, where the per-element sign differences are minimized to align the direction of LoRA weights with that of the implicit global supervision. In the design of the constraint, the implicit global supervision projected onto the dimensions of LoRA through matrix decomposition. The experimental results demonstrate that applying the direction constraint yiclds a faster loss descent
magnitude direction
2.0
0
1.5
0.5
0.0
0
200
400
600
800
1000
1200
Figure 3: Loss curves under three LoRA training setups: unconstraints, magnitude constrain, and di- rection constrain. Direction constrain shows lower loss than magnitude constrain and gradually ap- proaches that of the unconstrained setup.
3

---

Unconstrained Baseline
Direction Constraints for
Direction Constraints for LoRA B
SVD Direction
LoRA A and B
Constraints for LoRA B
Lo
2
(8)
(b)
(c)
(d)
Figure 4: Conffict matrices across clients: (a) Unconstrained bascline. (b) Direction constraints for
LoRA A and B, (c) Direction constraints for LoRA B, and (d) SVD + Direction constraints for LoRA
B. The x- and y-axes denote client indices; lighter colors indicate lower conflict and bias. In summary, SVD initialization, combined with direction constraints only on LoRA B, most effectively reduces conflicts and corrects local training biases under the same constraint level.
and eventually converges to a level close the unconstrained case, outperforming the setting with only a magnitude constraint. This indicates that the direction constraint can effectively convey the implicit global supervision without significantly disturbing the original LoRA training process.
Insights on Direction Constraints for LoRA Fine-Tuning, LoRA updates are restricted to low- rank subspace, limiting their ability to capture the full parameter space of full-parameter fine-tuning. Aligning these updates with the pre-trained weight direction allows LoRA to follow the adaptation path of full fine-tuning while remaining lightweight. Crucially, it is the direction, not the magnitude, that provides effective implicit global supervision to guide local updates. Prior studies support this: LoRA-GA 46 uses SVD-based initialization preserve pre-trained directions and align initial gradients; LoRA- Pro 48] enforces this alignment throughout training; and FR-LoRA (56) applies a similar approach in federated settings, yielding consistent gains. Collectively, these results show that maintaining alignment with pre-trained directions is key to narrowing the gap between LoRA and full full-parameter fine-tuning, highlighting the central role of directional constraints in LoRA fine-tuning.
LoRA A or LoRA B. To assess the effectiveness of direction constraints in correcting the training bias of LoRA (including both LoRA A and LoRA B) across different clients, we introduce the conflict matrix as a visualization tool. The conflict matrix consists of pairwise conflict rates which measure the proportion of parameters with opposing signs, reflecting weight disagreements between clients (53. When kę client signs are consistent on parameter i, the average pairwise conflict rate is p M E, k( k,), and the overall conflict rate is C' MN En min(k,, N k2). Here. M- = card A W) represents the number of parameters in A W and N is the total number of clients. C is positively correlated with p, i.e., C x p. and serves as an upper bound approximation of p.
2N -
C
MNZ [(min(ki, N kc))
Assume each parameter's 's magnitude follows a distribution with mean μ and variance o². The expected value and variance of the global LoRA weights A AW follow,
E|AWgi = (1 - 20)p; Var( g) = N 1o2 + 4C(1-C))A
(2)
As E [0, 0.5) decreases, |1 - 2C increases, amplifying the expected magnitude of the global weights and clarifying their direction. Meanwhile, 4C(1 C)2 decreases, reducing variance and stabilizing the update, effectively correcting local training bias across clients. For example, in a tug-of-war, C' is the proportion of force opposing the target. As C' decreases, more people align their

---

force with the target, stabilizing the process and correcting individual force biases. The detailed derivation is in the appendix. Lemma 3.1 concludes the proof, as follows, Lemma 3.1 (Less Conflict, Les Bias). A decrease in p. along with reduction in C, an increase in the expected magnitude AEAAW g)1, and a decrease in the variance Var(AWg) g), helps stabilize the global update direction, thereby effectively mitigating local training bias.
For LoRA A and B. we initially attempt impose direction constraints on both simultaneously. However, as shown in Figure 4 (b), constraining both LoRA A and B introduces excessive restrictions, which in turn increases the direction conflicts of LoRA A across clients, as it limits A's projection space and forces clients into narrow subspace, making A more sensitive to local data variations. Therefore, we choose to constrain only one of LoRA A or LoRA B. Since the conflict between client LoRA B is greater in the unconstrained baseline (Figure 4 (a)), we decide to constrain only LoRA B. As shown in Figure E (c), the conflict among client LoRA A remains almost unchanged, while the conftict among LoRA B is significantly reduced. Finally, inspired by 44 44), we initialize LoRA A and B using the Singular Value Decomposition (SVD) of the implicit global supervision to provide prior direction for all clients. Under the guidance of direction constraints, conflicts are further mitigated (Figure (d)). Notably, excessive bias correction may be counterproductive, potentially impairing LoRA's s ability to learn client-specific knowledge. Thus, balancing bias correction with local exploration is essential.
4
YOCO
Based on the above motivation, we propose YOCO for true one-shot communication in federated low-rank adaptation of MI.LMs. Specifically, YOCO consists of initialization, local training with sign-based consistency and noise-free sparse regularization, followed by aggregation. The direction supervision based on implicit global supervision enables the SVD initialization of LoRA and the sign-based consistency regularization of LoRA B. To balance bias correction with local exploration, noise-free sparse regularization of LoRA is introduced, followed by principal component-weighed aggregation. The illustration of YOCO is shown in Figure
Notation and Definitions. In the classical FL framework, such as FedAvg 34], the system typically consists of N clients, represented by the set N = {m] Each client possesses a local private dataset D, and conducts local training on a model parameterized by 0, optimized using a task-specific loss function L, [54) After completing local training in the hd round (t E [1, Tj, T, > 1), the system
randomly selects a subset of clients No C N to upload their models (0), E Mr} to the server for
aggregation, resulting in global model, i.e., 0 = =iEN, p:O%, where pi Eyex, [D3]' Unlike
[D.
FI., OFI. 132 limits client-server communication to single round (7 1), reducing privacy risks and communication costs. In one-shot federated low-rank adaptation for MLLMs. most parameters are frozen and correspond to the pre-trained weights W, while only the small sets of LoRA parameters are trained and uploaded for aggregation, i.c., the updated weights AW- = BA. Here, B and A represent the weights of LoRA B and LoRA A, respectively.
Initialization. Following
49 we use SVD of the pre-trainet weights to initialize LoRA A and B,
U.E. V SVD(W)
U, U [.1 Σ, (.r) V
V [1m]
(3)
A = A diag( Σ, V.. B B U,diag V E,)
where r denotes the rank of LoRA, and diag(-) represents the operation of constructing a diagonal matrix. A and B" are the initialized weights of LoRA A and B, respectively. It is worth noting that while using SVD to decompose pre- pre-traine weights for LoRA initialization is common [65] 49], in this true one-shot task, the focus of this prior initialization is to provide different clients with more informative and consistent starting point and direction for local LoRA training.
Sign-be begn-basedoe Consistency Regularization. In the initialization phase, B" is obtained by extracting the sign of B i.e., B = Bign(B"). Inspired by (55), during aggregation, the proportion of elements with reversed signs in the parameters reflects the conflict between models. Smaller conflicts facilitate less bias (Lemma 3.1). Therefore, to correct the local training bias for improved one-shot aggregation, we present direction-alignee aaliommenignnm regularization method that measures the distance between the signs
5

---

of B and B", called Sign-baser based Consistency Regularization, Since (B) is non-differentiable, we use the smooth approximation tanh(-) instead. The regularization Riign computed as follows,
|| tanhh> -B
Reign
(4)
|Ba 12
where "y controls the smoothness of the soft sign function, and & small positive number (le 6) used to prevent division by zero. Notably, only the direction of B is aligned with W here. Simultancously constraining the directions of both A and would impose excessive restrictions, thereby increasing the direction conflicts of A across different clients, shown in Figure 4
LoRA A
LoRA
LORA A
LoRA B
Dar11
09
09
Missing Modal Scenario
Cross Modal Scenario
Figure 5: PCA-based Visualization. After local training with loss L. we use PCA to visualize LoRA and B across clients. LoRA more dispersed, while LoRA B is more clustered, suggesting that LoRA B captures shared knowledge, whereas LoRA A reflects client-specific information.
Noise-free Sparse Regularization. Aligning the global direction of B reduces conflicts between clients and aids aggregation. However, the specific information in clients' private datasets often differs from the general features learned during pretraining. To balance global consistency and local adaptability, we enhance client-specific knowledge through LoRA A, using the simple and effective é norm for noise-free sparse regularization Repaase À x ||A|| Where À controls the degree of sparsity enhancement. The total loss of the local training is L L. + Reigg + Rypasse After local learn shared knowledge, while
training with loss the regularization terms encourage LoRA LoRA A captures client-specific features. To illustrate this, we visualize the distribution of LoRA A and B across different clients using Principal Component Analysis (PCA), shown in Figure 5 The visualization reveals that the more dispersed LoRA A across clients reflects stronger client-specific information, while the more clustered LoRA suggests a tendency toward shared knowledge.
Principal Component- Aggregation. As shown in Figure 5 LoRA B, which captures shared information, is more concentrated and suited for vanilla aggregation, while LoRA A, capturing client-specific information, is more dispersed and should maintain its diversity during aggregation. Inspired by the PCA visualization, present a principal component-weightted aggregation method for LoRA A across clients. The process is as follows,
Z = (CA(AA), Z E NN
[a,, a2 aNj E IR Nxd a= vec(A) Rd
D
(5)
A, = Z, PTP E Exd Z =
Z; My E g
1=1 |D1
1=
where A' is obtained by flattening and stacking the LoRA A matrices from all clients, and PT denotes the transpose of the principal-componen matrix. Z. represents the top-k PCA coordinates of client i's LoRA A, and P is the PCA loading matrix (written as P in the paper) used to map the weighted average Zg back to the original parameter space. Ag denotes global weights from principal component-weighh aggregation. B, obtained via vanilla aggregation. The aggregation phase occurs on the server, where A, and B, merge with the pre-trained weights for evaluation.
Experiments
5.1 Experimental Setup
Datasets. evaluate YOCO's effectiveness true one-shot communication for MILMM adaptation, five public multimodal datasets: Hateful-Memes (25). CrisisMMD 2. VQA-RAD [27).

---

Table 1: Comparison of YOCO with nolli-round methods and one-shot baselines across aligned (a) and missing (3) modal scenarios on Hateful-Memes and CrsisMMD. YOCO with 1x1x1 incurs only 0.03% of communication cost of multi-roum methods (25(rounds) x ((lellyyerss(((RRRAAA or 40 xxx), 4), yet outperforms them in all CrisisMMID scenarios, and surpasses all one-shot baselines.
Comm.
Hateful-Memes
Comm.
Mri
Method
cost
n=5.0 a=1.0 o=0.5 9=30%
3=40%
8=50%
cost
==500==1.0 n=0.5
9930%
8=40%
(350%
FedYogi
2
75.74
72.50 74.32
76.58
76.75
75.12
40x28
60.95
60.82 61.46
60.07
56.76
53.78
FedAdam
25x28x4
74.02
73.24 72.97
76.82
77.28
75.58
40x28
61.80
59.12
60.56
59.54
57.64
54.82
FedAvgM
2
74.08
72.94 64.22
60.62
$7.90
57.34
40x28 x4
58.28
56.87
54.22
32.64
32.97
31.72
FedAdagrad
25x28x4
74.53
73.34 71.80
71.97
70.64
70.59
40x28 x
60.77
60.43
60.21
56.96
57.48
55.55
Local
0x1x1
68.84
66.75 66.69
66.66
66.24
66.31
0x1x1
54.62
47.05 37.27
36.96
36.36
36.34
FedAdam
1x1x1
68.85
67.31 68.15
67.45
67.32
67.33
1x1x1
51.27
46.15
44.65
42.13
42.81
39.57
FedAvgM
1x1x1
69.06
69.64 68.27
69.20
68.58
67.57
1x1x1
61.52
59.72
55.17
50.64
42,90
47.05
FedAdagrad
1x1x1
69.24
68.16 68.17
67.64
67.35
67.35
1x1x1
33,90
33.82 33.42
32.50
32.23
30.31
FodAvg
1x1x1
64.72
20.44 68.19
69.21
68.58
66.97
1x1x1
61.57
59.01 46.06
50.57
41.75
46.96
Mcan
1x1x1
69.16
68.15 67.28
67.49
66.69
66.09
1x1x1
62.47
60,50
60.05
59.70
58.37
59.55
Ensemble
1x1x1
68.42
67.89 68.99
68.22
68.21
68.03
1x1x1
62.13
59.88
53.71
56.43
56.74
54,46
Combine
1x1x1
69.06
69.98 68.79
69.31
68.79
67.58
1x1x1
62.38
61.00
60.14
58.88
59.23
57.66
YOCOna
1x1x1
68,90
70.18 69.60
68.81
69.03
69.03
1x1x1
63.23
63.05
60.98
58.77
59.90
57.63
YOCOmaι
1x1x1
69.61
70.42 70.42
68.89
69.21
69.33
1x1x1
63,58
63.17 61.12
59.90
59 94
57.77
YOCOнαаBA
1x1x1
69.92
69.20 70.64
69.05
69.76
69.45
1x1x1
63.51
62.27
61.73
60.12
60.06
$7.57
1x1x1
70.14
70.76
70.65
69.48
69.39
69.96
1x1x1
63.69
63.66
61.28
59.95
59.71
58.10
SLAKE 30 and MedAlpaca [8] (the last three are medical datasets). Following [54] we conduct experiments under aligned, missing, cross, and hybrid modal scenarios with non-llD partitioning from a Dirichlet distribution (a (5.0, 1.0, 0.5)), considering multimodal heterogeneity (8 (80%, 70%, 60%{). Evaluation metrics are AUC
500%,444% 50%), 1-T= 5-5, 7-3), and for HatefullMemes, F1 for CrisisMMD, and GPT- D -assessed accuracy for the medical datasets.
Baselines. To ensure a fair comparison in the true one-shot communication setting, we construct a series of "1 x1x15 x 1" baselines, representing minimal communication cost with a single round, layer, and LoRA. "FedYogi". "FedAdam", and "FedAdagrad" (38) use dynamic optimizers, while "Fe- dAvgM" 22 applies momentum. Unlike "Mean", "FedAvg" 54 incorporates sample-size weighting. "Ensemble" combines LoRA votes from all clients. "Combine" denotes the different aggregations for LoRA A and B. Local in "0x1 11"" represents the average result from independent client training. We experiment with four YOCO variants: YOCO (initialization only). YOCO mii& (initialization with sign-based consistency regularization), YOCOmi&BA (noise-free sparse regularization built on YOCOmius)) and YOCO mn&BA (adding principal component-
-eeerhheettwwggheed weighted aggregation to YOCOmiuBB))
Configurations. Similar to [54), we adopt MiniCPM-V-2 6-int4 [62 as the default version of MLLMs. Only the last layer's q_proj is fine-tuned using LoRA (23], with a rank of 8, a lora_alp lora_alpha of 8. and a dropout rate of 0.05. Under the default setting, the total number of clients is 10, with each client trained for 10 epochs. We report the best performance of different baselines at both the 5th and 1hth epochs. For the Hateful-Memes dataset, the initial learning rate set to 2c-5, while for the other datasets, it is set to 2c-4. A cosine learning rate scheduler is used, with a warmup ratio of 1 1%. The per-device training batch size is set to I, and the gradient accumulation steps are set to 4. As shown in Table we also conduct experiments with different MILM versions and varying total numbers of clients. Details of y and X are in the appendix. All experiments are performed on NVIDIA A40.
5.2 Experimental Results
Main results. As shown in Tables D to 3 YOCO is comprchensively evaluated on four public datasets. We can see that: 1) YOCO outperforms all one-shot basclines and consumes only ~0.03% of the communication cost of multi-round methods, even surpassing them in some Mr and medical scenarios; 2) YOCOmiи outperforms YOCO in most cases, validating the effect of regularization, while the superior performance of YOCOmiиBA and min&BA highlights the importance of enhancing local adaptability, 3) YOCO with 1x1x1 also lowers computation cost.
Loss curves for YOCO variants. As shown in Figure 6 we present the training loss curves of multiple YOCO variants on three randomly selected clients. Compared to YOCOmiis YOCOmin which incorporates sign-based consistency regularization for LoRA B, converges to similar loss
7

---

Table Comparison of YOCO with multi- methods and one-shot baselines in cross (I-T) and hybrid (p) modal scenarios on Sattful-Memes and CrsisMMM. YOCO with 1x1x] 1 uses only ~0.03%
of communication cost of multi-round methods (arrounnd)x2(llayyeee x 44LoRA))) or 40x28x4), yer performs comparably in some CrisisMMD scenarios and exceeds all one-shot baselines.
Comm.
Hateful- Memes
Comm.
Mr)s
Method
cost
1:3-T:7
1:55T:5
1:7-T:3
y=80%
p=10%
60%%
cost
1:3
1:5 T:5 1:7-T:3 y=80% p=70%
f=60%%
FedYogi
25x28x4
68.71
73.93
75.94
68.41
68.55
71.52
40x28x4
49.30
55.53
59.20
59.73
63.04
58.96
FedAdam
2
65.71
74.44
74.95
72.17
70.12
72.36
40x28x4
53.17
56.62
59.22
60.02
61.01
59.93
FedAvgM
2
58.20
61.57
57.40
61.55
60.01
58.93
40x28x4
32.61
38.62
33.46
46,26
42.77
39.44
FedAdagrad
x
71.26
72.48
71.57
68.77
72.96
69.26
20
48.02
49.97
51.13
57.58
58.91
53.84
Local
xx1x1
66.04
66.93
67.23
65.61
66.42
66.44
0x1x1
32.76
33.88
35.51
37.41
36.04
37.78
FedAdam
1x1x1
66.75
67.24
67.85
66.82
67.85
67.65
1x1x1
32.41
33.16
36.12
41.94
38.36
37.12
FedAvgM
1x1x1
66.68
68.89
69.28
67.63
67.16
67.35
1x1x1
45.21
50.38
51.67
54.92
46.26
47.54
FedAdagrad
1x1x1
66.85
66.75 66.66
67.74
67.16
1x1x1
28.27
29.94
29.49
32.72
32.05
31.08
FodAvg
1x1x1
66.48
68.49
68.69
67.43
67.96
67.35
1x1x1
45.01
51.20
51.51
55.13
45.53
47.38
Mcan
1x1x1
66.22
68.03
68.02
67.17
67.63
68.44
1x1x1
44.07
47.89
52.15
56.61
54.31
55.48
Ensemble
1x1x1
67.94
68.25
68.26
67.93
68.45
68.34
1x1x1
51.48
52.84
55.69
56.93
56.31
57.60
Combinc
1x1x1
66.48
69.83
69.10
67.13
67.56
68.16
1x1x1
42.55
47.19
52.24
56.24
54.11
55.03
YOOOm
1x1x1
67.76
68.72 69.73
68.14
67.86
68.85
1x1x1
52.22
53.50
55.60
58.13
55.75
58.58
YOCOnass
1x1x1
68.04
69.44
69.63
68 23
68.07
68 94
1x1x1
$1.59
53.78
56.12
57.96
57.98
58.61
YOCOmBAB
1x1x1
68.
69.83
68.74
69.13
68.36
68.84
1x1x1
52.40
56.55
55.01
58.77
$9.06
58.26
YOCO miaaB
1x1x1
68.85
69.85
70.16
67.95
68.58
69.09
1x1x1
43.33
47.51
50.33
58.43
54.99
54.76
2.5
2.0
3.0
YOCO
YOGO,,
YOCO
4.0
2.5
3.0
3.5
YOCO
YOCO
YOCO
1.00
1.75
1.50
1.25
yOCO
YOCOne
1.5
2.0
0.13
1.0
1.5
1.0
0.50
0.5
0:25
0.0
0.0
0.00
500
1000
1500 2000 Step
2500
3000
0
1000
2000
3000
4000
5000
0
200
400
600
800
1000
Step
Step
Figure 6: Training loss curves of YOCO YOCO and YOCOmiйвA (YOCO from three randomly selected clients. Sign- Sign-based regularization on LoRA B results in a final loss similar to that without regularization, while sparse regularization on LoRA A has little impact on the training loss.
Table 3: Comparison of YOCO with multi-round methods and one-shot baselines in cross (I-T) and hybrid (p) modal scenarios on the medical datasets. YOCO with the 1 1x1x setting incurs only
~0.02% of the communication cost of nound methods (idrroondss22(layyeosa
4(LRAs)), yet
outperforms them in most scenarios, and consistently surpasses all one-shot baselines in "Overall".
Comm
VQA-RAD
SLAKE
VQA-RAD
SLAKE
Method
Scen.
Soen.
cost
Open Closed Overall Open Closed Overall
Open Closed Overall Open Closed Overall
Local
2x28xx
42,74
69.97
59.16
58.71
65.83
61.63
43.86
70.04
59.64
58.19 65.19
60.98
Yegio
50x28x4
45.25
68.02
58.98
53.70
66.15
58.67
44.42 71.14
60.53
52.55
63.04
56.74
FedYogi
1x1x1
43.58
58.09
62.32
61.36
61.94
42.46 71.69
60.09
60.41
61.72
60.94
FedAvgM
1x1x1
42.46
68.01
57.87
61.92
60.41
61.32
42.46 66.18
56.76
59.54
63.04
60.94
YOCO M
1x1x1
46,37
69.12
60.09
63.51
60.41
62.27
44.13 73.16
61.64
60.89
66.27
63.04
Local
0x28x4
41.90
71.69
59.87
59.22
62.32
60.46
41.06 71.87
59.64
52.23
67.34
58.26
Yegio
50x28x4
41.90
73.53
60.98
52.78
60.41
55.83
44.13 70.96
60.31
53.74
64.00
57.83
FedYogi
1x1x1
45.25
69.85
60.09
61.05
64.47
62.42
40.22 71.69
59.20
59.14
64.11
61.13
FedAvgM
1x1x1
44.13
70.96
60.31
59.86
62.68
60.98
42.46 72.43
60.53
57.39
62.56
$9.46
YOCOmaAR
1x1x1
71.69
62.53
61.69
66.63
63.66
46.93 69.85
60.75
58.82
64.71
61.17
Local
0x28x4
45.25
69.85
60.09
53.02
65.67
58.07
46.93 70.96
61.42
56,36
63.52
59.22
Yegi
50x28x4
43.02 72.06
60.54
54.46
64.60
58.50
46.64 73,90
63.08
52.43
64.00
57.05
FedYogi
1x1x1
46.37
68.38
59.65
60.33
62.68
61.27
43.02 70.59
59.65
58.35
65.19
61.08
FodAvgM
1x1x1
45.81
68.38
59.42
56.60
64.59
59.79
46.37
73.16
62.53
57.00
63.64
59.65
YOCO
1x1x1
45.25
72.79
61.86
60.25
63.64
61.60
45.81
73.90
62.75
60.02
65.67
62.27
level. Introducing noise-free sparse regularization on LoRA A in YOCOmaRRA (or YOCO top of YOCO i has minimal impact on the loss convergence trend.
initkBA) on
8

---

Table 4: Comparison of different lightweight MLLMs under different numbers of total clients (N 10, N = 20, N = 30). YOCO with the 1x1xx setting incurs only ~0.03% of the communication
cost of multi-round methods (25(rounds) (
yet outperforms them in N of V-2_6 6 MILMM and N = 20 of V-2_5 MILM,, consistently surpassing all one-shot baselines.
Comm.
Total Clients
Total Clients
Method
MLLMS
ML..Ms
cost
N= 10
N= 20
N 30
N= 10
N 20
N 30
Local
0x28x4
66.80
64.46
63.50
67.02
63.21
63.26
FodAdam
25x28x4
75.58
69.70
66.91
MiniCPM-L Llama V-2 mnt4
72.44
68.82
71.16
Local
0x1x1
MinICPM-VV22 6..4
66.31
65.05
65.47
67.12
62.71
65.20
FedAdam
1x1x1
67.33
68.05
67.29
68.62
67.80
69.60
FodAvgM
1x1x1
67.57
67.69
65.97
68.51
67.42
65.15
FodAdagrad
1x1x1
67.35
67.66
67.88
68.25
68.53
69.10
FodAvg
1x1x1
66.97
67.49
66.17
68.51
67. 57.42
65.15
Mean
1x1x1
66.09
68.06
68.14
68.26
67. 67.20
68.64
Ensemblee
1x1x1
65.90
66.57
Combine
1x1x1
67.58
66.99
66.58
68.51
67 /2
65.15
YOCOm
1x1x1
69.96
69.10
68.49
69.19
68.85
70.11
Different MLLMs and client numbers. We evaluate the impact of two MLLM versions with = 10, 20, and clients, as shown in Table Key findings: 1) YOCO consistently achieves top one-shot performance, with improvements of up to 6.14% over Local baseline and up to 4.96% over other baselines. 2) It is more robust than multi-round methods, maintaining comparable or superior performance, particularly at N 20 and 30, even under low communication conditions.
Different LoRA ranks and com- ponents. To investigate the ef-
71
71
fect of rank on performance, we evaluate configurations with ranks and 16, as shown in Figure 7 Performance differ- ences among ranks 4, 8, and are marginal. Considering the trade- off between performance and pa- rameter efficiency. we adopt rank as the default. Furthermore, in addition to the default use of the q LoRA component, we explore the v, kt and o components as well. Results show that the commonly used 9 and components yield comparable performance, while and lead slight degradation.
70
70.59
70,65
70.85
70
70.65
70.75
69
69:44
69
69:22
69,73
68
67
r=2
r=4
r=8
1=16
9
0
Figure Effect of LoRA ranks r=2, 4, 8, and components q, k, v, q LoRA (r=8) balances performance and efficiency.
Cosine similarity matrices. As shown in the cosine similarity matrix of Figure 8 we com- pare the LoRAs trained by each
LoRA
LoRA
A50
client in YOCO with those from other clients, as well as with the aggregated global LoRAs in mia and VOCOmniBBA The cosine similarity matrix for LoRA A cxhibits lower similar- ity compared to LoRA B, partic- ularly with YOCO which employs Principal Component- weighted Aggregation enhance the integration of specific knowl- edge and strengthen adaptability.
Client
675
Client/ / Global
Client / Global
Figure 8: Cosine similarity matrix for LoRA A and B. The 10 x 10 matrix shows client-client similarities, while 10 x matrix (orange box) shows client- global similarities (G1: YOСO G2: VOCOmaaBB) LoRA A has lower cosine similarity, with VOCOmiuBB preserving key specific knowledge.
Discussion and Conclusion
Differences from prior works. Under one-shot federated low-rank adaptation, 33 fine-tunes the model with an additional server-side dataset to extract prototypes for client training. In contrast,

---

YOCO corrects local bias using implicit global supervision without extra data. To correct local bias, FuseFL
42 aggregates blocks bottom-up, and FENS [ uses lightweight aggregators, but both violate one-shot communication constraint. YOCO is the first to correct bias via implicit supervision true one-shot setting, making a fair performance comparison with FuseFl. and FENS infeasible.
One-shot communication. This work tackles the challenge of using implicit global supervision correct local bias and improve performance under one-shot communication. In multi-round scenarios, explicit global supervision is typically gained through model aggregation. Whether implicit supervision provides additional benefits over explicit supervision remains an open question, requiring further empirical investigation. We look forward to future research on this topic.
Limitations. YOCO is better suited for LoRA in initialization and regularization, while local bias correction in other PEFT methods remains unresolved. We use the last-layer LoRA for fine-tuning, but this is suboptimal. Future work could explore automated parameter selection for better efficiency.
In conclusion, we propose YOCO, a true one-shot communication method, after analyzing the rationality of implicit global supervision to correct local biases. YOCO introduces sign-based and noise-free regularization strategies, along with a PCA-based aggregation, to enhance consistency and adaptability, building on SVD initialization, which ensures clients share unified global starting point. Experiments demonstrate the effectiveness of YOCO with minimal communication cost.
Acknowledgments and Disclosure of Funding
The work is supported by the National Natural Science Foundation of China (Grant No. 62222207, 62427808), the National Research Foundation, Singapore, under its NRF Fellowship (Award# NRF- NRFF14-2022-000)) and the China Scholarship Council program.
References
[1] Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint 2r37v:2303.08774, 2023.
[2] Firoj Alam, Ferda Ofli, and Muhammad Imran. Crisismmd: Multimodal twitter datasets from natural disasters. In Proceedings of the international AAAI conference web and social media, volume 12, 2018.
[3] Youssef Allouah, Akash Balasaheb Dhasade, Rachid Guerraoui, Nirupam Gupta, Anne-Marie Kermarrec, Rafael Pinot, Rafael Pereira Pires, and Rishi Sharma. Revisiting ensembling in one-shot federated learning. Advances in Neural Information Processing Systems, 2024.
[4] Flora Amato, Lingyu Qiu, Mohammad Tanveer, Salvatore Cuomo, Fabio Giampaolo, and Francesco Piccialli. Towards one-shot federated learning: Advances, challenges, and future directions. arXiv preprint arXiv:2505.0242: 2025.
[5] Sara Babakniya, Ahmed Roushdy Elkordy, Yahya H Ezzeldin, Qingfeng Liu, Kee-Bong Song, Mostafa El-Khamy, and Salman Avestimehr. Slora: Federated parameter efficient fine-tuning of language models. arXiv preprint arXiv:2308 06522, 2023.
[6] Jiamu Bai, Daoyuan Chen, Bingchen Qian, Liuyi Yao, and Yaliang Li. Federated fine-tuning of large language models under heterogencous tasks and client resources. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024.
[7] Jieming Bian, Yuanzhe Peng, Lei Wang, Yin Huang, and Jie Xu. A survey on parameter-efficient fine-tuning for foundation models in federated learning, arXiv preprint arXiv:2504.2 1099, 2025.
[8] Jieming Bian, Lei Wang, Letian Zhang, and Jie Xu. Lora-fair: Federated lora fine-tuning with aggregation and initialization refinement. arXiv preprint arXiv:2411. /4961, 2024.
[9] Jieming Bian, Lei Wang, Letian Zhang, and Jie Xu. Fedalt: Federated fine-tuning through adaptive local training with seatooftthe-worte lora. arXiv preprint arXiv:2503.11 11880, 2025.
10

---

[10] Haokun Chen, Hang Li, Yao Zhang, Jinhe Bi, Gengyuan Zhang, Yueqi Zhang, Philip Torr, Jindong Gu, Denis Krompass, and Volker Tresp. Fedbip: Heterogeneous one-shot federated learning with personalized latent diffusion models. arXiv preprint arXi::2410.048100 2024.
[11] Shuangyi Chen, Yuanxin Guo, Yue Ju, Harik Dalal, and Ashish Khisti. Robust federated finetuning of Ilms via alternating optimization of lora, arXiv preprint arXiv:2502.01755, 2025.
[12] Yae Jee Cho, Luyang Liu, Zheng Xu, Aldi Fahrezi, and Gauri Joshi. Heterogeneous lora for federated fine-tuning of on-device foundation models. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 12903-12913, 2024.
[13] Rong Dai, Yonggang Zhang, Ang Li, Tongliang Liu, Xun Yang, and Bo Han. Enhancing one- shot federated learning through data and ensemble co-boosting. In The Twelfth International Conference Learning Representations, 2024.
[14] Don Kurian Dennis, Tian Li, and Virginia Smith. Heterogeneity for the win: One-shot federated clustering. In International conference machine learning, pages 2611-2620, 2021.
[15] Xuan Gong, Abhishek Sharma, Srikrishna Karanam, Ziyan Wu, Terrence Chen, David Doer- mann, and Arun Innanje. Preserving privacy in federated learning with ensemble cross-domain knowledge distillation. In Proceedings of the AAAI Conference on Artificial Intelligence. volume 36, pages 11 11891-11899, 2022.
[16] Neel Guha, Ameet Talwalkar, and Virginia Smith. One-shot federated learning. arXiv preprint arXiv:192..11175 2019.
[17] Pengxin Guo, Shuang Zeng, Yanran Wang, Huijie Fan, Feifei Wang, and Liangqiong Qu. Selec- tive aggregation for low-rank adaptation in federated learning. arXiv preprint arXiv:2410.01463 2024.
[18] Tianyu Han, Lisa C Adams, Jens-Michalis Papaioannou, Paul Grundmann, Tom Oberhauser, Alexander Löser, Daniel Truhn, and Keno K Bressem. Medalpaca an open-source collection of medical conversational ai models and training data. arXiv preprint arXiv:2304.082477 2023.
[19] Zeyu Han, Chao Gao, Jinyang Liu, Jeff Zhang, and Sai Qian Zhang. Parameter-efffcient fine-tuning for large models: A comprehensive survey. arXiv preprint arXiv:2403. 2024.
[20] Jie Hao, Yuman Wu, Ali Payani, Myungjin Lee, and Mingrui Liu. Personalized federated fine-tuning for heterogeneous data: An automatic rank learning approach via two-level lora. arXiv preprini arXiv:2503.03920 2025.
[21] Clare Elizabeth Heinbaugh, Emilio Luz-Ricca, and Huajie Shao. Data-free one-shot federated learning under very high statistical heterogeneity. In The Eleventh International Conference on Learning Representations, 2023.
[22] Tzu-Ming Harry Hsu, Hang Qi, and Matthew Brown. Measuring the effects of non-identical data distribution for federated visual classification. arXiv preprint 1XXx:1990.00635. 2019.
[23] Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. ICLR, 1(2):3, 2022.
[24] Divyansh Jhunjhunwala, Shiqiang Wang, and Gauri Joshi. Towards a theoretical and practical understanding of one-shot federated learning with fisher information. In Federated Learning and Analytics in Practice: Algorithms, Systems, Applications, and Opportunities, 2023.
[25] Douwe Kiela, Hamed Firooz, Aravind Mohan, Vedanuj Goswami, Amanpreet Singh, Pratik Ringshia, and Davide Testuggine. The hateful memes challenge: Detecting hate speech in multimodal memes. Advances in neural information processing systems, 33:2611-2622 2020.
[26] Jabin Koo, Minwoo Jang, and Jungseul Ok, Towards robust and efficient federated low-rank adaptation with heterogeneous clients. arXiv preprint arXiv:2410.22815, 2024.
11

---

[27] Jason J Lau, Soumya Gayen, Asma Ben Abacha, and Dina Fusnerr A dataset of clinically generated visual questions and answers about radiology images. Scientific data, 5(1):1-10, 2018.
[28] Royson Lee, Minyoung Kim, Fady Rezk, Rui Li, Stylianos Venieris, and Timothy Hospedales. Fedp2eft: Federated learning to personalize parameter efficient fine-tuning for multilingual 1lms. arXiv preprint arXiv:2502.0m 043877 2025.
(29] Zijing Liang. Yanjie Xu, Yifan Hong. Penghui Shang, Qi Wang, Qiang Fu, and Ke Liu. A survey of multimodel large language models. In Proceedings of the 3rd International Conference on Computer; Artificial Intelligence and Control Engineering, pages 405-409, 2024.
[30] Bo Liu, Li-Ming Zhan, Li Xu, Lin Ma, Yan Yang, and Xiao-Ming Wu. Slake: semantically- labeled knowledge-cnhaneg international symposium on biomedical imaging (ISBI), pages 1650-1654, 2021.
dataset for medical visual question answering. In 2021 IEEE 18th
[31] Jun Liu, Yunming Liao, Hongli Xu, Yang Xu, Jianchun Liu, and Chen Qian. Adaptive parameter- efficient federated fine-tuning on heterogeneous devices. arXiv preprint arXiv:2412.2002 2024.
[32] Xiang Liu, Zhenheng Tang, Xia Li, Yijun Song, Sijie Ji. Zemin Liu, Bo Han, Linshan Jiang, and Jialin Li. One-shot federated learning methods: A practical guide. arXiv preprint 2rXiv:2502.09104. 2025.
[33] Guodong Long, Tao Shen, Jing Jiang, Michael Blumenstein, et al. Dual-personalizing adapter for federated foundation models. Advances in Neural Information Processing Systems, 37:39409- 39433, 2024.
[34] Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. Communication iattumiiaatio--efccietttt learning of deep networks from decentralized data. In Artificial intelligence and statistics, pages 1273-1282, 2017.
[35] Shannon How Shi Qi, Jagmohan Chauhan, Geoff V Merrett, and Jonathon Hare. Fedtmos: Efficient one-shot federated learning with tsetlin machine. In The Thirteenth International Conference Learning Representations, 2025.
[36] Huangsiyuan Qin and Ying Li. Fedinc: One-shot federated tuning for collaborative incident recognition. In International Conference on Artificial Neural Networks, pages 174-185, 2024.
[37] Hongyu Qu, Rui Yan, Xiangbo Shu, Hailiang Gao, Peng Huang, and Guo-Sen Xie. Mvp- shot: Multi-velocity progressive-aligmeen framework for few-shot action recognition. JEEE Transactions on Multimedia, 2025.
[38] Sashank Reddi, Zachary Charles, Manzil Zaheer, Zachary Garrett, Keith Rush, Jakub Konečný, Sanjiv Kumar, and H Brendan McMahan. Adaptive federated optimization. arXiv preprint arXiv:2003.00295. 2020.
[39] Raghav Singhal, Kaustubh Ponkshe, and Praneeth Vepakomma. Exact aggregation for federated and efficient fine-tuning of foundation models. arXiv preprint arX0v:24/0.09432,, 2024.
[40] Rui Song, Dai Liu, Dave Zhenyu Chen, Andreas Festag, Carsten Trinitis, Martin Schulz, and Alois Knoll. Federated learning via decentralized dataset distillation in resource-constraine edge environments. In 2023 International Joint Conference on Neural Networks (IJCNN), pages 1-10, 2023.
[41] Youbang Sun, Zitao Li, Yaliang Li, and Bolin Ding. Improving lora in privacy-preserving federated learning. In The Twelfth International Conference on Learning Representations, 2024.
[42] Zhenheng Tang, Yonggang Zhang, Peijie Dong, Yiu-ming Cheung, Amelie Zhou, Bo Han, and Xiaowen Chu. Fusefl: One-shot federated learning through the lens of causality with progressive model fusion. Advances in Neural Information Processing Systems, 11-23333-28422 2024.
[43] Arun James Thirunavukarasu, Darren Shu Jeng Ting, Kabilan Elangovan, Laura Gutierrez, Ting Fang Tan, and Daniel Shu Wei Ting. Large language models in medicine. Nature medicine, 1930- 2023.
12

---

[44] Naibo Wang, Yuchen Deng, Shichen Fan, Jianwei Yin, and See-Kiong Ng. Multi-modal one- shot federated ensemble learning for medical data with vision large language model. arXiv preprini arXiv:2501.03292, 2025.
[45] Naibo Wang. Wenjie Feng, Moming Duan, Fusheng Liu, See- Bee-Kiong Ng. et al. Data-free diversity-based ensemble selection for one-shot federated learning. Transactions on Machine Learning Research, 2023.
[46] Shaowen Wang, Linxi Yu, and Jian Li. Lora-ga: Low-rank adaptation with gradient approxima- tion. In Advances in Neural Information Processing Systems, volume 37, pages 54905-54931, 2024.
[47] Shaowen Wang, Linxi Yu, and Jian Li. Lora-ga: Low-rank adaptation with gradient approxima- tion. Advances in Neural Information Processing Systems, 37:54905-55431, 2024.
[48] Zhengbo Wang, Jian Liang, Ran He, Zilei Wang. and Tieniu Tan. Lora-pro: Are low-rank adapters properly optimized? In The Thirteenth International Conference on Learning Repre- sentations, 2025.
[49] Zhengbo Wang, Jian Liang. Ran He, Zilei Wang, and Tieniu Tan. LoRA-pro: Are low- rank adapters properly optimized? The Thirteenth International Conference on Learning Representations, 2025.
[50] Ziyao Wang, Zheyu Shen, Yexiao He, Guoheng Sun, Hongyi Wang, Lingjuan Lyu, and Ang Li. Flora: Federated fine-tuning large language models with heterogeneous low-rank adaptations. In The eighth- Annual Conference on Neural Information Processing Systems, 2024.
[51] Ziyao Wang, Bowei Tian, Yexiao He, Zheyu Shen, Luyang Liu, and Ang Li. One commu- nication round is all it needs for federated fine-tuning foundation models. arXiv preprint arXiv:4412.04650, 2024.
[52] Shijie Wu, Ozan Irsoy, Steven Lu, Vadim Dabravolski, Mark Dredze, Sebastian Gehrmann, Prabhanjan Kambadur, David Rosenberg, and Gideon Mann. Bloomberggpt: A large language model for finance. arXiv preprint arXiv:2303.17566 2023.
[53] Baochen Xiong, Xiaoshan Yang, Yaguang Song, Yaowei Wang, and Changsheng Xu. Pi- lot: Building the federated multimodal instruction tuning framework. arXiv preprint 25X1:25501.1385,, 2025.
[54] Binqian Xu, Xiangbo Shu, Haiyang Mei, Guosen Xie, Basura Fernando, Mike Zheng Shou, and Jinhui Tang. Fedmllm: Federated fine-tuning milm multimodal heterogeneity data. arXiv preprint vxXiv22411.14711 2024.
[55] Prateek Yadav, Derek Tam, Leshem Choshen, Colin A Raffel, and Mohit Bansal. Ties-merging Resolving interference when merging models. Advances in Neural Information Processing Systems, 36:7093-7115, 2023.
[56] Yunlu Yan, Chun-Mei Feng, Wangmeng Zuo, Rick Siow Mong Goh, Yong Liu, and Lei Zhu. Federated residual low-rank adaptation of large language models. In The Thirteenth International Conference on Learning Representations, 2025.
[57] An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2. 5 technical report. urXiv preprint arXiv:2412.55113 15115, 2024.
[58] Mingzhao Yang, Shangehao Su, Bin Li, and Xiangyang Xue. Exploring one-shot semi- supervised federated learning with pre-trained diffusion models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 16325-16333, 2024.
[59] Mingzhao Yang, Shangchao Su, Bin Li, and Xiangyang Xue. Feddeo: Description-enhance one- shot federated learning with diffusion models. Proceedings of the 32nd ACM International Conference Multimedia, pages 6666-6675, 2024.
13

---

[60] Yiyuan Yang, Guodong Long, Tianyi Zhou, Qinghua Lu, Shanshan Ye, and Jing Jiang. Federated adapter on foundation models: An -ustr--disttibuttio approach. arXiv preprint arXiv:2505.010755 2025.
[61] Zhengyuan Yang, Linjie Li, Kevin Lin, Jianfeng Wang, Chung-Ching Lin, Zicheng Liu, and Lijuan Wang. The dawn of Imms: Preliminary explorations with gpt-4v (ision). arXiv preprint arXiv: 9()):1, 2023.
[62] Yuan Yao, Tianyu Yu, Ao Zhang, Chongyi Wang, Junbo Cui, Hongji Zhu, Tianchi Cai, Haoyu Li Weilin Zhao, Zhihui He, et al. Minicpm-v: A gpt-4v level milm on your phone. arXiv preprint arXiv:2408. 01 01000, 2024.
[63] Jianyi Zhang, Hao Frank Yang, Ang Li, Xin Guo, Pu Wang, Haiming Wang, Yiran Chen, and Hai Li. MIlm-fl: Multimodal large language model assisted federated learning on heterogeneous and long- -ailed data. arXiv preprint arXiv:2409 06067, 2024.
[64] Jie Zhang, Chen Chen, Bo Li, Lingjuan Lyu, Shuang Wu, Shouhong Ding, Chunhua Shen, and Chao Wu. Dense: Data-free one-shot federated learning, Advances in Neural Information Processing Systems, 35:21414-21428,, 2022.
[65] Qingru Zhang, Minshuo Chen, Alexander Bukharin, Pengcheng Hc, Yu Cheng, Weizhu Chen, and Tuo Zhao. Adaptive budget allocation for parameter--fffiiien fine-tuning. In International Conference on Learning Representations, 2023.
[66] Yanlin Zhou, George Pu, Xiyao Ma, Xiaolin Li, and Dapeng Wu. Distilled one-shot federated learning. arXiv preprint (rXXi:2009.07999 2020.
14

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The main claims made in the abstract and introduction accurately reflect the paper's contributions and scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We have discussed the limitations of our work in Section 6
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
15

---

Justification: We construct conflict matrix and demonstrate in Lemma 3.1 that fewer conflicts result in lower bias. The full proof of Lemma 3.1 is provided in the appendix of the supplemental material.
Guidelines:
The answer NA means that the paper does not include theoretical results. All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proof's provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We provide the experimental settings, datasets, baselines, and configurations in Section [5] with additional hyperparameters and the algorithm detailed in the appendix of the supplementary material. The code also included the supplementary material.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data often one good way to accomplish this, but reproducibilityy can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
specific model and empirical evaluation, may
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way access this model for reproducing the results or way to reproduce the model (e.g., with an open- open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which authors are welcome to describe the particular way they provide for reproducibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
16

---

5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, described in supplemental material?
Answer: [Yes]
Justification: We have provided the code in the supplementary material. The datasets used are public and listed in Section and can be downloaded online.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.co
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" an acceptable answer. Papers cannot be rejected simply for including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
Codeeuomi ssinnpolicy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Answer: [Yes]
Justification: Yes, have included all details in Section 5 and the appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer:
[No]
Justification:
We do report the error bars in our experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
17

---

The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma enror bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are range (e.g. negative enror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We have included the details on compute resources in Section
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics http:///wwurrr...ccaaaaaaaaacaaa
Answer: [Yes]
Justification: This work does not incorporate any ethic concerns of NeurIPS. The datasets and models are commonly used in the community, and the method does not incorporate potential concerns.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We have discussed the broad impacts our work in appendix.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
18

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification:
The datasets and baselines, used libraries are properly credited.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website). the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in package should be provided. For popular datasets, paperswithcode com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
19

---

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer:
[Yes]
Justification:
The datasets and baselines, used libraries are well documented and cited.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Bthies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, not include any information that would break anonymity (if applicable), such the institution conducting the review.
20

---

16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, originality of the research, declaration is not required.
Answer: [Yes]
Justification: Yes. This paper proposes one-shot federated low-rank adaptation method based on existing multimodal large language models (MLLMs), where the use of MILMs is an integral part of the methodology.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM policy hattop///aauuss.ccc///ofeerrrers for what should or should not be described.
21

---

Organization of the Appendix
A More Related Works
B Detailed Proof of Lemma 3.1
C
Theoretical Guarantees for Convergence and Optimality
D Scalability Evaluation Beyond the Benchmark Setting
E Details of Experiment Configuration
E.1 Hyper Parameters
Aggregation Strategies
F Details of Algorithm
Experiments the Single-Modality Setting
More Visualizations of the Conflict Matrix
Broader Impact
A More Related Works
One-shot federated learning. In recent years, One-shot Federated Learning (OFL) 132 requiring only a single round of communication between the server and clicnts-haa has attracted increasing research interest due to its significant advantages in communication efficiency and data privacy over traditional Federated Learning (FL). Current approaches to OFL can be broadly classified into
four categories: knowledge distillation 40
15
64
[3] 21], generative models [64] 13 21
59 [58..
static ensemble methods [45] 35| [5] (64), and adaptive ensemble methods [3, 42 3 Knowledge
distillation can be divided into data distillation and model distillation. In data distillation
40 66).
clients generate distilled data from private data and upload it to the server for model training. In model
distillation [15 64
13 21 the knowledge from ensembled client models is distilled into the server model using public or synthetic data. Due to the distribution gap between public and private data,
some generative model methods (64 13 21 59 58) for synthetic data have emerged. Generative
models are mainly classified into generative adversarial networks (GANs) 64
13 variational
autoencoders (VAEs) (1]. and stable diffusion models [59 58).
Without additional data or support from generative models, static 45 35
and adaptive 42
ensemble methods offer direct solution to data and model heterogeneity, while their high flexibility allows seamless integration with other approaches. Early static ensemble methods [16] 45) primarily constructed the global model by applying strategies such as averaging, selection, mization to the models uploaded by clients. Unlike static ensemble methods that rely on extracting fixed characteristics from client models aid global model construction, recent adaptive ensemble approaches aggregate global supervision developed from the perspectives of causal graphs [42] and secondary generalizers B3. This aggregated global supervision is used to correct local training bias, thereby achieving strong global model performance. However, aggregated global supervision violates the one-shot communication constraint of OFL, increasing the risk of attacks. To address this, our YOCO employs implicit global supervision- using an initial prior instead of aggregation- to correct local bias while preserving true one-shot communication.
output maxi-
Federated learning with LoRA. Most existing federated learning with LoRA approaches adopt multi-round training and primarily focus addressing three issues: server-side aggregation bias [1 26 IT
56 28], and computational 31 50). To mitigate the issue of server-side aggregation bias, one class of methods [41, 26 I1 adopts an alternating training strategy for the A and B matrices in LoRA, while another explicitly computes the residual between the ideal global LoRA and the aggregated global LoRA 39 g. To address data heterogeneity, some methods [17 20 33 9 use two LoRA modules or assign matrices and capture shared and specific knowledge, respectively. Moreover, FRLoRA [56] addresses data heterogeneity and expands the LoRA parameter space by adding the global model to the residual LoRA and reinitializing parameters each local training. To tackle computational heterogeneity, FLORA [50] and FIe (6) multiply matrices A and B before aggregation, while HETLOOR 12 directly pads lower-rank LoRA modules with zeros. Compared to multi-round FL with LoRA, research on OFL with LoRA under single communication round is
[50 39 8] data heterogeneity [17 20 33 60)
heterogeneity [12
5
22

---

limited. FedIne [36] fine-tunes LoRA using server-side public data and generates prototypes to assist local training. Unlike FedIne, which relies on additional public data and prototype transmission, our method achieves true one-shot communication in OFL with LoRA by transmitting only the LoRA weights, without any extra data.
B Detailed Proof of Lemma 3.1
For given parameter let k; denote the number of client models with positive sign, and N - denote the number with negative sign. N the total number of clients. Then, the total number of client pairs exhibiting sign disagreement this parameter is & kj). The total number of client N(N- and thus the pairwise conflict rate P: for parameter is:
pairs (2)
2
2k N R;)) Pi
(6)
N 1)
The average pairwise conflict rate p is defined as the mean of the pairwise conflict rates across all parameters
2
Σ P:
Σ ka,NN k;)
M
MN(N
where MM card AW) represents the number of parameters in AW. The overall conflict rate C' is defined as:
C = MN
min(k,, N k:)
(8)
By the identity min(k, - k;) = N -112
kNN k2) can be expanded as:
IN - k;) = min(ki, N - ki) - [min(k,, N k:)] Substitution into the expression of gives:
(9)
M
2
p MN(N
N
min(k; N N- k.) >
kz)l
(10)
In conjunction with the definition of C:
min(k, N k.) -
(11)
Therefore, the relationship between the average value أ of the pairwise conflict rate p and the overall conflict rate C' is given by:
2N
C
Σ minn(s, N k:)]
(12)
N-1
MN2
is positively correlated with C', but the exact mathematical form depends on the distribution of parameter conflicts.
ID setting. For the sake simplifying the proof, the global LoRA weights W are computed by averaging the weights from all clients:
AW
Σ AW,
(13)
N
1=1
Let each parameter A W.. have sign 8ni = sign( AW mi) and magnitude a,i = | W asl. where and follows distribution with mean y and variance o². Given parameter i with overall conflict rate C; min(k.,N-k)) if the majority of signs are positive (probability C,), then 8; +1; otherwise, if the majority are negative (probability C.), then -1. The expectation of parameter i expressed as:
N
E[s,a,]
C.) # C. (-x) (1- -22))
(14)
23

---

The expected value of the global weight A We is the average of the expected values of all parameters:
g
M
E A gl
E s;a, _e -C
(15)
M
Upon substitution of C'
MN > i= min (kis - ki) = M7
1
Dim1 Ci. we obtain:
M
1
E B 1 1
C₁
-1
(16)
M
i=1
The variance of parameter is:
Var(s;a,) = -B((())))((
[[(saa,) E[a = 0² +1²; (E[s;a,))² = (1:200))²
(17)
Var(s;a;) = 0² + 11² (1 - 4C, + 40 = 0² -
Since the global weight is the average of the client weights and the parameters are independent, the variance is given by:
M Var(s,a;)
1
Var W
(18)
N
M
Substituting the variance of a single parameter:
Var A W
02 + (1.(1-)))p) _
(19)
NM
By substituting the overall conflict rate C= # E C, and assuming uniform distribution of the conflict rates (i.e., C zs C), we obtain the approximation:
1 10 (0(1-C)p²)
Var A
(20)
the overall conflict rates are not uniformly distributed, higher-order correction terms need to be introduced:
1 1 + 4C(1 C) 4pp Var(C:)] N
Var A W
(21)
ΣS where Var(C.) + 2i=1(6 (C, 2) For simplicity, we typically assume Var(C;) implying uniform distribution of conflict rates. Other patterns warrant future exploration.
Non-IID setting. The above derivation of E A W g] and Var( A W g)) assumes that client data is IID. To account for the non-IID setting, we provide a more general derivation below. The main differences lie in modeling parameter heterogeneity across clients-ine and variances and in allowing the conflict rate C₁ to vary across parameter dimensions.
each client may have different means
Similarly, the global LoRA weights are obtained by averaging the weights from all clients, i.e., AW N Σ AW.. n. For client n, AW, ni has sign Sna sign( A Wn.) and magnitude and AW with ani following distribution of mean Nn,i and variance of The overall conflict rate of parameter i is C, Min(k;.N-kk where k; is the number of clients with positive N
sign. For parameter i, the global weight is:
N
AW 9.1
Snaans
(22)
N
n=1
Since the sign sni has conflict rate of C, and the mean magnitude for each client is Pa,i the expected value is:
Ana ] = {
Pn,
with probability - C, (majority sign = +) with probability (majority sign -)
(23)
Pn,

---

Therefore, the global expectation is:
N and
N
9.1e
Σ (1
2n,
(24)
N
N
==1
Let the global mean be defined as μıi
N
Mn,,, then:
E
A g.al (1 20:)
(25)
Compute the mean over all M parameters:
E AAw, g)
FAW anl Σ (1)20))
(26)
M
Defining the overall conflict rate as C = E C₁ and the global average magnitude as M Eis1 Mi, the global expectation can then decomposed as:
M Σ Cips
EAW
1 - 2.
(27)
M
Assuming that C, and μ4; are independent, i.e., the conflict rate is unrelated to the parameter magnitude, other cases are left for future work. Then:
E E W r 20μ (1 -
(28)
The variance of parameter can be decomposed into:
N Σ
Var AW 9.4) Var
Sndans
(29)
N
n=1
Under the assumption that clients are independent, i.e., Cov(snaans 8ma0mi) = for n * m, which is common simplification in the literature, especially for theoretical analysis, the variance of parameter i can be decomposed as:
N
Var W a
Var(snc(nss
(30)
However, in non-llD settings, this independence assumption may not strictly hold, and cross-client co- variances could potentially affect the global variance. We leave the investigation of these correlations for future work. The variance of the parameters for each client is:
Var(sns@nz)) = [[mman))) - (E(sniani))
[[(( 31 = Elanil = , + ph.s (B[ssnanni)) = (1- =1--cc)²k uhn Var(sn: -o, + 1 (1 40, + (4,)) = on, + 4C, Ci)
(31)
By averaging the variances of all clients, we obtain:
1 Do + 4C, 2
Var A W g.s)
(32)
N2
Further decomposed into two parts: the average variance within clients of = N Σn oh and the variance mean differences between clients de + E. (Hn,i 41)². By expanding 2 = (p++ + (pn,s - p:))?, we obtain:
N N
2 u1 +
(33)
n=1
Thus, the variance of parameter is:
1 1o +C, +d4]
1
Var AW g.)
(34)
N
25

---

Take the average over all M parameters:
Var A AWS.
M Σ
Var 9.))
1 NM
2
(35)
the overall conflict rates C, ~ C, and the means and variances are similar across clients μ4, R3 μ.. of x o², dl d,, then the global variance can be approximated as:
Var (ar
1o dl 4C(1 C)
(36)
Assuming sign conflicts only affect the global expected direction, while client magnitude differences (df) exist independently. Thus, df is retained direct measure of data heterogeneity. Specifically, d² captures magnitude distribution differences, and C' reflects sign conflict levels. Since they are mathematically uncorrelated and physically independent, they can be approximated as separate additive terms the model. If the overall conffict rate C, is unevenly distributed, higher-order terms need to be introduced:
Wwg) g)) ~ 1002 de 40 C)p 4p22 Var(C.)]
Var AWW
(37)
where Var(C,) M EE (C; - C The above derivation results are summarized as follows:
IID setting:
Non-IID setting:
E AW & = (1 2C) Var (A & = 10 4C(1-C)pp
R & Rs -22)j Var( = N0² + d2 + 4C - C)
(38)
Regardless of whether under the IID or Non-IID setting, the conclusion of Lemma 3.1 holds: reducing the average conflict rate p. decreasing the conflict degree C, increasing the expected magnitude of the global update A W gll and reducing its variance Var( AW g) all contribute to stabilizing the global update direction, thereby effectively mitigating local training bias.
C Theoretical Guarantees for Convergence and Optimality
Let W denote all trainable LoRA parameters, while the pretrained backbone remains frozen. We analyze single-round (T 1) federated optimization under both IID and non-IID data, explicitly modelling sign conflicts among clients. Two new hypotheses are added to the classical smooth-convex framework to capture inter-client directional disagreement.
Assumptions
Let the global objective be L(W). We assume:
(A1) L-smoothness:
(VC(W) VC(W")II LI W'l
(A2) X-strong convexity:
(VC(W)- VL(W"), W ww) 2 M|lW w7
(A3) Bounded per-sample gradient variance:
Var(z,g)~D, (V((Sw()),)) < 5o
(A4) Bounded sign-conffict variance:
M
1
Σ [C CG- 1=1
M
(A5) Gradient-sign alignment:
(VV,,, VL(W)) >(1 -1122)
where C= M EM C₁. C < 0.5, and 7 (0, 1] quantifies directional coherence.
26

---

Convergence under IID Data
From Lemma 3.1, the global update across N clients satisfies:
B| p.
Var(AW,) 10 + 4C(11
(1.1)
where f denotes the average local update direction and C captures the extent of directional disagree- ment.
To account for bias arising from such conflicts, we prove the following:
Lemma C.1 (Gradient Bias Bound). Under (A1)-(A)) the bias of the aggregated gradient satisfies:
[[V - VC(W) s 2Ç 0
N
This result reflects that dispersion in sign conflicts (C) introduces systematic bias, while the variance term scales inversely with client number N.
Theorem C.1 (IID Convergence). Under (AI)-A5), with 17 = 142-1K
B(C(W)) [(W*)] (1 -1)-m (K(W)) [(W")] 0 off
2XN
where
c 0² +40(1 4)p1-C)pp² + 4 k= 4C(1 C)
N
The term (1- -+ captures the contraction due to strong convexity and smoothness, degraded
by conffict-induced variance K. The residual error sign conflict (C), and dispersion (().
aa 2XN
includes noise from client sampling (o²),
Extension to Non-IID Settings
Under data heterogeneity, we define d² as the inter-client drift (see Appendix B):
M
d
(Mn.i 1:)
(1.2)
M
measuring deviation in client update directions. The global update variance now becomes:
Var(AW,) se N 10" de 40(1
(1.3)
Theorem C.2 (Non-IID Convergence). Under data heterogeneity and (A1)-(A5), with =
T+A
Paid
B(C(W1) - ((
S (1-1))711 - 2C33) - -(W*)] + 2XN
where
40(1 C) d
Knidd
ai = 0² + 40(1-0)pp2-11 4
N
The exponential term exp
An(1 2 1-+
encodes alignment efficiency y and conflict rate C, attenu- aggregates variances from sampling, conflict, dispersion,
Kais
ated by drift d², The residual error and heterogeneity.
o 2XN
27

---

Asymptotic Optimality (One-shot Limit)
The residual error N + 3o (large client count) simplifies to:
de + 4
Jim B[C(Wi) - [(W")] -0000
nisd
(1.4)
2AN
2AN
This confirms that conflict (C") and dispersion (C) remain additive error sources in the one-shot setting, alongside data heterogeneity (d?).
Key Implications for T' = 1
1. Conflict Sensitivity: Optimality gap scales with C(L C) and <² peaking C' 0.5. Alignment strength critically amplifies the exponential decay in non-IID settings.
2. Client Scaling:
Residual error decays as (1/NN, emphasizing the benefit of large client pools to mitigate variance.
3. Algorithmic Guidance: Conflict-aware aggregation (e.g., reweighting clients with low C₁) directly reduces de and and Drift reduction techniques (e.g., regularization) remain essential for non-IID robustness.
These results provide theoretical grounding for federated LoRA fine-tuning in communication- constrained scenarios, highlighting the irreducible impact of client disagreement in single-round optimization.
D
Scalability Evaluation Beyond the Benchmark Setting
Based on the adopted benchmark, we primarily evaluated YOCO with 10-30 clients. The method, however, is not restricted to this range: constrained only by GPU resources, further conducted experiments with clients and analyzed the scalability the confict-mattis computation. As shown in Table YOCO consistently maintains its effectiveness larger scales, outperforming both basic one-shot baselines and multi-round FL approaches.
Table 5: Performance comparison with N 50 clients under two MLLM variants: V2 6 (MiniCPM- V-2_ 22_6in and V2_5 (MiniCPM-Llamal 1-2 (imt4). Comm. cost denotes communication overhead.
Method
Comm. cost
50 (V2_6)
50 (V2_5)
FedAdam
2xx28x4
66.67
68.64
Local
0x
65.71
62.93
FedAdam
1x1x1
67.69
69.09
FedAvgM
1x1x1
66.03
67.69
FedAdagrad
1x1x1
67.07
69.18
FedAvg
1x1x1
66.23
67.69
Mean
1x1x1
67.58
68.31
Combine
1x1x1
65,84
68.09
YOCO minBB
1x1x1
68.15
69.83
IID Scenario: Statistical-Scale Derivation N
Let the probability that parameter i takes positive sign be p;- Then
k; Binom(Nnn))) p:).
OSpi S1
Expected Global Conflict Rate
B MN
1
Σ
min(k; k:) 2
Σ
I 2/p: - +ll O(VN))/N
28

---

If =2 0.5 (signs nearly balanced), B[C] + - ((N-122)
If p; biased to one side, then E[C] < 0.5 and decreases further with N.
Conclusion 1. In non-extreme mixed-sign settings, C does not grow with N; indeed, because most parameters converge to the same sign, its upper bound decreases monotonically as increases.
Global Weight Noise
The original derivation gives
Var ,), N -0a 40(1
Leading term o²/N = variance 1/N more clients make each global update more stable.
Secondary term 4C C)2² shrinks C decreases; YOCO's directional constraint is designed to reduce C.
Conclusion 2. With hundreds or thousands clients, the O(1/N) noise is already far smaller than directional bias, so YOCO still delivers significant gains.
Non-IID Extension: Cluster-Level Beta-Binomial Model
Partition the clients into G clusters, with proportions (gg Eg Og 1). positive-sign probability of parameter is p
Within cluster the
G
1 Binom N, ps (0))
k
119)
g=1
Conflict Rate
B|C| MN
ΣΣ E min (k/ .g,N
.a, - ]b())]] O(G))= - + (
When G & N (typical in practice), the dominant term remains O(1/N).
Variance
0² + 4C(1-
Var Var(AW,)
which differs from the IID result only by the O(G/N) adjustment.
Conclusion 3. As long as each cluster contains (2(N) clients (equivalently, G < N), the O(1/N) scaling of both C' and Var(AW, is retained, and YOCO's directional-statee sttbbiilitt effect remains valid.
E Details of Experiment Configuration
E.1 Hyper Parameters
As shown in Table 6 we provide the specific hyperparameter settings of y and À for sign- based consistency regularization and noise-free sparse regularization, respectively.
E.2
A ggregattion Strategies
As shown in Tables 7 and g the aggregation strategies of different YOCO versions are presented.
Details of Algorithm
Details of our algorithm is shown in Algorithm
29

---

Table 6: Hyperparameter settings for y and
Hateful-Memae
a=5.0
a=1.0
a=0.5
8=30%
3=40%
3=50%
(1,
(1,25)
(1,200)
(1,50)
(1, 1,00
(1,
A
le-2
le-1
1e-4
1e-2
1e-2
1e-3
Hateful-Memes
1:3-T:7
1:5-T:5
1:-T:3
p=80%
p=70%
p=60%
(1,300)
(1, 100)
(1,25)
(00)
(1,25)
100)
1e-3
1e-2
1e-2
1e-1
1e-3
1e-4
CrisisMMD
a=5.0
a=1.0
a=0.5
B=30%
3=40%
3=50%
(0,
(1,5)
(1, (00)
(1,50)
(1,
(1, (1)
le-4
1e-3
1e-2
1e-4
1e-3
1e-4
CrisisMMD
I:3-T:7
1:5-::5
1:7T:3
p=80%
p=70%
p=60%
(1,5)
(1,25)
(1,25)
(1,50)
(1,50)
(1, 25)
A
le-4
1e-3
1e-2
le-4
le-4
1e-4
Medical Datasets
1:3-T:7
1:5-T:5
1:7-T:3
p=80%
p=70%
p=60%
(1,
(1,25)
(1,25)
(1,
(1,25)
(1, 25)
le-4
1e-4
1e-4
le
1e-4
1e-4
Table 7: Aggregation strategies in YOCO on the Hateful-Memes dataset. "Combine" denotes
"FedAvgM" aggregation for LoRA and "Mean" aggregation for LoRA B.
Hateful-Mene
a=5.0
a=1.0
a=0.5
3=30%
B=40%
6=50%
YOCO
Combine
FedAvg
Combine
Combine
Combine
FedAvg
YOCO IniL
FedAvg
FedAvg
Combine
FedAvg
Combine
FedAvg
YOCO IniRRA
FedAvg
Combine
Combine
FedAvg
FedAvg
FedAvg
YOCO INnRBA (for B)
FedAvg
FedAvgM
FedAvgM
FedAvg
FedAvg
FedAvg
Hateful-Memes
1:3-T:7
I:5-T:5
1:7-T:3
p=80%
p=70%
p=60%
YOCO
Ensemble
FedAvgM
FedAvgM
Ensemble
FedAdagrad
Ensemble
YOCO IniB
Ensemble
FedAvgM
FedAvg
Ensemble
FedAAagrad
Mean
YOCO JmiAABB
Ensemble
FedAvgM
FedAvgM
Ensemble
Ensemble
Mean
YOCO InKRBA (for B)
FedAvgM
FedAvgM
FedAvgM
FedAdagrad
Mean
FedAvgM
G Experiments on the Single-Modality Setting
To further evaluate the generalizability of YOCO in a single-modality setting, we conducted additional experiments, as presented in Table The results show that YOCO consistently outperforms all one- shot baselines in the single-modality setting and even surpasses multi-round FL on the text modality of CrisisMMD. These findings demonstrate that YOCO remains highly effective in single-modality scenarios, underscoring its strong generalization capability.
H More Visualizations of the Conflict Matrix
To illustrate the changes in conflicts on LoRA A and B from YOCO,, to YOCO initB.. visualize the conflict matrices under two data heterogeneity settings. As shown Figures and introducing sign-based consistency regularization for LoRA B effectively reduces its conflicts, while the conflicts on LoRA A remain nearly unchanged.
I Broader Impact
Federated learning (FL. is distributed training framework designed to protect data privacy, making particularly well-suited for privacy-sensitive domains such as finance and healthcare. In recent years, many multimodal large models (MLLMs) have become increasingly mature, exhibiting strong
30

---

Table 8: Aggregation strategies YOCO on the CrisisMMD and Medical datasets. "Combine"
denotes "Mean" aggregation for LoRA A and "FedAvgM" aggregation for LoRA B.
CrisisMMD
a=5.0
a=1.0
a=0.5
3=30%
8=40%
3=50%
YOCO init
Combine
Mean
Combine Combine
Mean
Combine
Combine
I
Combine
Mean
Mean
Mean
Mean
YOCO IniL&BA
Mean
Mean
Mean
Combine
Combine
Combine
YOCO (for B)
Mean
Mean
Mean
Mean
Mean
Mean
CrisisMMD
1:3-T:7
I:5-T:5
1:7-T:3
p=80%
p=70%
p=60%
YOCO
Ensemble
Ensemble
Ensemble
Combine
Ensemble
Ensemble
YOCOmine
Ensemble
Ensemble
Ensemble
Combine
Ensemble
Ensemble
YOCO INiuBBA IA
Ensemble
Ensemble
Ensemble
Mean
Ensemble
Ensemble
YOCO IniuBA (for B)
Mean
Mean
Mean
FedAdagrad
Mean
Mean
Medical Datasets
1:3-T:7
I:5-T:5
1:7-T:3
p=80%
p=70%
p=60%
YOCO Init& (for
FedAvgM
FedAvgM
Fed
FedYogi
Fed
FedYogi
Algorithm YOCO
Input: Local dataset n,, hyperparameters y, X, client set N,, task loss C,
2: Output: Global weights A W
3: Server executes Eq. (3) to obtain A and B
D Initialization
4: for each client N do
Local Training with Regularization
5: Initialize local LoRA: A + A B B
6: AW, n A,
7:
Rsign (tanhbBB.... BB.)
Sign-based Consistency Regularization
[ llate
8:
Reparse
A
> Noise-free Spare Regularization
9: C(AW,, n: Dx) L. R Rign R Rpuase
10: Update parameters: AW, + AW, VL
11: end for
12: Server Aggregation:
b Principal Component-weighted Aggregation
Aggregate B, with vanilla strategy, Ag with Eq. (5)
14: Return global weights WW g for evaluation
Table 9: Performance comparison under 0.5 in the single-modality setting. denotes communication cost.
"Comm. cost"
Hateful-Memes
CrisisMMD
Method
Comm. cost
Comm. cost
Image
Text
Image
Text
FedAdam
25x28x4
72.70
68.26
40x28
59.59
37.95
Local
0x1x1
67.47
65.41
0x 0x1xI
41.76
33.05
FedAdam
1x1x1
67.67
66.24
1xlxI
39.24
30.04
FedAvgM
1x1x1
68.99
66.43
1x1x1
44.98
32.95
FedAdagrad
1x1x1
67.68
66.36
1x1x1
30.74
26.47
FedAvg
1x1x1
68.79
66.42
1x1xI
44.25
33.35
Mean
1x1x1
67.36
66.49
1xlxI
53.35
35.86
Ensemble
1x1x1
69.07
66.39
1x1xI
53.21
36.54
Combine
1xlxl
69.19
66.33
1xlxI
53.45
37.08
YOCO init
1xlxl
69.22
66.31
1xlxI
53.30
37.66
VOCOmin
1x1xl
69.53
66.41
1x1x1
53.99
37.78
YOCO BA
IxIxI
69.88
66.93
IxIxI
54.37
38.10
YOCO. BA
1x1x1
70.10
67.11
1x1x1
54.51
38.65
31

---

YOCO,p
YOCO
6:00
2:23 BAN
LORA A
CM
800
045
030
6:00
0.4
347
8:0
0:00
30
K.
0.00
549
5:44
LoRA 8
0.0
s49
0.0
3.5
0000
02053
0300
Figure Conflict matrices of YOCO init and YOCO I on Hateful- -Meess with a = 5.0.
YOCO
YOCO,aaR
4.300
0.31 300
11000
SEL E:01
0075
ACL 0:13
6:00
3.00
044
6:30
Figure 10: Conflict matrices of YOCO mnt and YOCO mit& on Hateful-Memes with 1.0.
general prior knowledge and the ability to handle wide range of multimodal tasks in unified manner. These models can also achieve impressive performance on specific tasks through rapid fine-tuning. Integrating MLLMs with FL offers significant potential to accelerate the development of intelligent systems in privacy-critical settings.
However, conventional MLLMs+FL frameworks typically rely on multiple rounds of communication, making them susceptible to performance degradation in resource-constrained client environments. This limitation poses challenges for deployment on edge devices and in low-resource scenarios. To address this, we propose YOCO, the first framework to enable efficient federated low-rank adaptation of MILMMs under a truly one-shot communication setting. YOCO introduces an implicit global supervision mechanism to mitigate local training bias without relying on additional data or generative models. It substantially reduces communication overhead while maintaining oor even enhancing-modde performance.
The introduction of YOCO broadens the applicability of MLLMs+FL, particularly in settings in- volving resource-limited edge devices, remote healthcare facilities, and small-scale organizations. This advancement holds great promise for promoting the real-world adoption of MLLMs in privacy- sensitive domains such as finance and healtheare.
32


Reviews:
[{"review_id": "y3mPVQtwT0", "reviewer": "Reviewer_P4k5", "summary": "This paper addresses the challenge of federated fine-tuning of multimodal large language models (MLLMs) in privacy-sensitive scenarios, focusing on reducing communication costs in federated learning (FL). Toward the goal, the authors propose YOCO, a framework that achieves true one-shot communication by leveraging implicit global supervision (from pre-trained weights) to guide local LoRA adaptation. Extensive experiments on multiple multimodal benchmarks validate YOCO’s superiority over both one-shot and multi-round baselines, while maintaining minimal communication cost.", "questions": "See **Weaknesses**.", "limitations": "See **Weaknesses**.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "**Strengths:**\n1. The paper demonstrates consistent gains across diverse modalities, datasets, and model scales, showing strong generalization and scalability of YOCO with minimal communication overhead. \n2. The paper is well-organized and easy to follow.\n\n**Weaknesses:**\n1. In Section 4, I cannot see the proposed method is strongly related to multimodal settings. I wonder whether this approach is also suitable for the single-modality setting. \n2. This work relies on an important assumption, i.e., pre-trained weights provide sufficient global knowledge to guide local updates. However, in federated learning, clients' private data may not be consistent to the pre-trained data, leading to an unreasonable local updates.", "was_revised": true, "final_justification": "The authors addressed my concerns. I think this is an interesting research topic for federated learning community, although the method relies on the assumption that \"pre-trained weights provide sufficient global knowledge to guide local updates\".", "rebuttal": "We thank **Reviewer P4k5** for the valuable comments and questions. We have provided detailed responses to address these concerns and look forward to further discussion and feedback.\n\n> **Q1:** In Section 4, I cannot see the proposed method is strongly related to multimodal settings. I wonder whether this approach is also suitable for the single-modality setting.\n\n**A1:** Thank you for pointing this out. We propose the YOCO method in a multimodal setting primarily for two reasons:\n\n1. The chosen benchmark emphasizes multimodal heterogeneity, which intensifies data distribution differences across clients and exacerbates the *local training bias* issue in OFL (see lines 32–33 of the paper and FuseFL \\[24]).\n2. Compared to LLMs, MLLMs offer broader applicability in real-world scenarios.\n\nTo further evaluate the generalizability of YOCO in a unimodal setting, we additionally conducted the following experiments:\n\n| α = 0.5              | Comm. cost | Hateful-Memes |       | Comm. cost | CrisisMMD |       |\n|----------------------|------------|---------------|-------|------------|-----------|-------|\n|                      |            | Image         | Text  |            | Image     | Text  |\n| FedAdam              | 25×28×4    | 72.70         | 68.26 | 40×28×4    | 59.59     | 37.95 |\n| Local                | 0×1×1      | 67.47         | 65.41 | 0×1×1      | 41.76     | 33.05 |\n| FedAdam              | 1×1×1      | 67.67         | 66.24 | 1×1×1      | 39.24     | 30.04 |\n| FedAvgM              | 1×1×1      | 68.99         | 66.43 | 1×1×1      | 44.98     | 32.95 |\n| FedAdagrad           | 1×1×1      | 67.68         | 66.36 | 1×1×1      | 30.74     | 26.47 |\n| FedAvg               | 1×1×1      | 68.79         | 66.42 | 1×1×1      | 44.25     | 33.35 |\n| Mean                 | 1×1×1      | 67.36         | 66.49 | 1×1×1      | 53.35     | 35.86 |\n| Ensemble             | 1×1×1      | 69.07         | 66.39 | 1×1×1      | 53.21     | 36.54 |\n| Combine              | 1×1×1      | 69.19         | 66.33 | 1×1×1      | 53.45     | 37.08 |\n| YOCO$_{init}$        | 1×1×1      | 69.22         | 66.31 | 1×1×1      | 53.30     | 37.66 |\n| YOCO$_{initB}$     | 1×1×1      | 69.53         | 66.41 | 1×1×1      | 53.99     | 37.78 |\n| YOCO$_{initBA}$    | 1×1×1      | 69.88         | 66.93 | 1×1×1      | 54.37     | 38.10 |\n| YOCO$_{initB\\bar{A}}$ | 1×1×1   | 70.10         | 67.11 | 1×1×1      | 54.51     | 38.65 |\n\nAs shown in the table above, YOCO consistently outperforms all one‑shot baselines under the unimodal setting, and even surpasses multi‑round FL in the text modality of CrisisMMD. This demonstrates that YOCO remains effective in unimodal scenarios, highlighting its strong generalization capability.\n\n> **Q2:** This work relies on an important assumption, i.e., pre-trained weights provide sufficient global knowledge to guide local updates. However, in federated learning, clients' private data may not be consistent to the pre-trained data, leading to unreasonable local updates.\n\n**A2:** Thank you for raising this concern. To clarify:\n\n1. **Rationale from the original paper**\n\n   The original paper concludes that the **direction of the pre-trained weights** (rather than the pre‑trained weights themselves, which include both magnitude and direction) can serve as a global supervisory signal to guide local updates in LoRA (see lines 47–49 and Section 3). This is further supported by our evidence in **Figure 4** and **Tables 1–4**.\n\n2. **Support from prior work**\n\n   This perspective is reinforced by several recent works. As LoRA updates are restricted to a low‑rank subspace, alignment with the **pre‑trained direction**—so as to approximate full fine‑tuning—is essential for effective adaptation. Specifically:\n   * **\\[a]** Theoretically shows that initializing LoRA’s A and B matrices with the SVD of pre-trained weights—scaled to preserve direction—helps align the initial gradient with that of full fine-tuning.\n   * **\\[b]** Extends this approach by aligning gradients at every step.\n   * **\\[c]** Applies a similar SVD-based initialization in a federated learning context.\n\n3. **Our conclusion**\n\n   Taken together, these works demonstrate that aligning LoRA updates with the **direction of pre‑trained weights** helps narrow the gap to full fine‑tuning and enhances performance. Building on this principle, our YOCO constrains the updates of LoRA B in OFL using the direction of the pre‑trained weights, thereby effectively mitigating local training bias.\n\n**References**\n* \\[a] Wang, Shaowen, Linxi Yu, and Jian Li. *LoRA-GA: Low-Rank Adaptation with Gradient Approximation.* NeurIPS 2024.\n* \\[b] Wang, Zhengbo, et al. *LoRA-Pro: Are Low-Rank Adapters Properly Optimized?* ICLR 2025.\n* \\[c] Yan, Yunlu, et al. *Federated Residual Low-Rank Adaptation of Large Language Models.* ICLR 2025."}, {"review_id": "scQat3mGzf", "reviewer": "Reviewer_6tEs", "summary": "This paper introduces YOCO (You Only Communicate Once), a method that achieves one-shot federated learning for multimodal large language models using only ~0.03% of traditional communication costs. The key insight is that directional constraints on LoRA weights cause less optimization interference than magnitude constraints, enabling effective local bias correction through implicit global supervision from pre-trained weights rather than aggregated supervision. YOCO employs SVD-based initialization for unified starting points, sign-based consistency regularization on LoRA B for global consistency, sparse regularization on LoRA A for client-specific adaptability, and differentiated aggregation strategies. Experiments across five multimodal datasets show YOCO consistently outperforms all one-shot baselines and even surpasses multi-round methods in certain scenarios while dramatically reducing communication overhead.", "questions": "1. Could you provide stronger theoretical guarantees beyond Lemma 3.1?\n2. Experiments are limited to 10-30 clients. How does YOCO perform with hundreds or thousands of clients, which is more realistic for federated learning deployments? Have you conducted any analysis on the scalability of your conflict matrix computation?\n3. Given that Wang et al. demonstrated large foundation models inherently perform well in one-shot federated learning due to smoothness, smaller updates, and fewer epochs, how do you disentangle your method's contributions from these natural advantages?", "limitations": "The authors acknowledge some limitations (LoRA scope, suboptimal parameter selection) but miss several critical issues:\n\n1. Lemma 3.1 lacks convergence guarantees and optimality bounds\n2. Evaluation limited to 10-30 clients; unclear performance with hundreds/thousands of clients\n3. Fails to disentangle method contributions from inherent large model advantages in one-shot FL", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 1, "originality": 1, "strengths_and_weaknesses": "Strengths:\nAchieves only ~0.03% of multi-round communication costs while maintaining competitive performance.\n\nWeaknesses:\n1. Lemma 3.1 provides intuition but lacks rigorous theoretical guarantees about convergence or optimality.\n2. Evaluation limited to relatively small client numbers.\n3. Uses only last-layer LoRA fine-tuning, which the authors acknowledge is suboptimal.\n4. The paper fails to disentangle whether its performance gains stem from the proposed technical innovations (SVD initialization, sign-based regularization, etc.) or simply from the inherent advantages of large foundation models in one-shot federated learning, as demonstrated by the paper (One Communication Round is All It Needs for Federated Fine-Tuning\nFoundation Models)", "was_revised": true, "final_justification": "The response addressed most of my concerns. Especially compared with reference [d], this paper has made sufficient contributions.", "rebuttal": "We sincerely appreciate **Reviewer 6tEs** for the insightful comments and questions. We have provided comprehensive responses to clarify these points and look forward to continued discussion and feedback.\n\n> **Q1:** *Lemma 3.1 provides intuition but lacks rigorous theoretical guarantees about convergence or optimality.*\n\n**A1:** Thank you for this suggestion. Based on Lemma 3.1, we give theoretical guarantees on convergence and optimality. Owing to space limits, only key results are shown; full proofs will appear in the final version.\n\n#### **Background**\n\n* **Problem**: One‑round federated optimization $T = 1$ with LoRA parameters $W$ (backbone frozen). Data may be IID or non‑IID.\n* **Challenge**: Sign conflicts in client gradients, measured by conflict rate $C$ and dispersion $\\zeta$.\n* **Goal**: Establish convergence of global loss $\\mathcal{L}(W)$.\n\n#### **Core Assumptions**\n\n1. **Smoothness & Convexity**\n   $\\mathcal{L}(W)$ is $L$-smooth and $\\lambda$-strongly convex with unique minimizer $W^*$.\n2. **Gradient Noise**\n   Per‑client gradient variance $\\le \\sigma_L^2$.\n3. **Sign Conflict**\n\n   * **Dispersion $\\zeta$**: $\\tfrac{1}{M}\\sum_i (C_i - C)^2 \\le \\zeta^2$.\n   * **Alignment $\\gamma$**: Local gradients align with the global one ($\\gamma \\in (0,1]$; larger $\\gamma$ = better).\n\n#### **Convergence Results**\n\n#### (i) IID Data (Theorem 3.3)\n\n*Bound*:\n\n$$\n\\mathbb{E}\\bigl[\\mathcal{L}(W_1) - \\mathcal{L}(W^\\*)\\bigr]\n\\le\n\\left(1 - \\frac{\\lambda}{L + \\lambda^{-1}K}\\right)\\Delta_0\n+\n\\frac{\\sigma_{\\text{eff}}^{2}}{2\\lambda N}\n$$\n\n*Key terms*:\n\n* $\\Delta_0 = \\mathcal{L}(W_0) - \\mathcal{L}(W^*)$ (initial loss gap)\n* $K = \\tfrac{4C(1-C)\\mu^2}{N}$ (conflict‑induced variance)\n* $\\sigma_{\\text{eff}}^{2} = \\sigma^2 + 4C(1-C)\\mu^2 + 4\\zeta^2\\mu^2$ (sampling $\\sigma^2$ + conflict + dispersion)\n\n*Insight*: Conflicts ($C>0$) and dispersion ($\\zeta>0$) inflate both $K$ and $\\sigma_{\\text{eff}}^{2}$, slowing convergence. The residual error still shrinks as $\\mathcal{O}(1/N)$.\n\n#### (ii) Non‑IID Data (Theorem 3.4)\n\n*Bound*:\n\n$$\n\\mathbb{E}\\bigl[\\mathcal{L}(W_1) - \\mathcal{L}(W^*)\\bigr] \\le\n\\bigl(1 - \\eta\\lambda\\gamma(1-2C)^2\\bigr)\\Delta_0\n+\n\\tfrac{\\sigma_{\\text{niid}}^{2}}{2\\lambda N}\n$$\n\n*Key terms*:\n\n* $\\eta = \\tfrac{1}{L + \\lambda^{-1}K_{\\text{niid}}}$ (learning rate)\n* $K_{\\text{niid}} = \\tfrac{4C(1-C)\\mu^2 + d^2}{N}$ (conflict + heterogeneity)\n* $\\sigma_{\\text{niid}}^{2} = \\sigma^2 + d^2 + 4C(1-C)\\mu^2 + 4\\zeta^2\\mu^2$\n* $d^2 = \\tfrac{1}{M}\\sum_i \\tfrac{1}{N}\\sum_n (\\mu_{n,i}-\\mu_i)^2$ (client drift)\n\n*Insight*: Contraction depends on alignment $\\gamma$ and conflict $C$. Heterogeneity $d^2$ and dispersion $\\zeta$ dominate the residual error.\n\n#### (iii) Asymptotic Optimality ($N \\to \\infty$)\n\n*Residual error*:\n\n$$\n\\lim_{N\\to\\infty}\\mathbb{E}\\bigl[\\mathcal{L}(W_1) - \\mathcal{L}(W^*)\\bigr]\n\\approx\n\\frac{d^2 + 4\\zeta^2\\mu^2}{2\\lambda N}\n$$\n\n*Interpretation*: Sampling noise ($\\sigma^2$) and conflict ($C$) vanish, but heterogeneity ($d^2$) and dispersion ($\\zeta$) remain the bottlenecks.\n\n> **Q2:** *How does YOCO scale to hundreds or thousands of clients? Any analysis on conflict‑matrix scalability?*\n\n**A2:** Thank you for pointing this out. Based on the adopted benchmark, we evaluated with 10–30 clients. Our YOCO method is not limited to this range: due to GPU constraints, we also tested with 50 clients and analyzed the scalability of conflict matrix computation. \n\n| Method | Comm. cost | N = 50 (V2\\_6) | N = 50 (V2\\_5) |\n| ------ | ---------- | -------------- | -------------- |\n| FedAdam| 25 × 28 × 4 |66.67 |68.64 |\n| Local                | 0 × 1 × 1  |   65.71 |  62.93  |\n| FedAdam              | 1 × 1 × 1  |   67.69 |  69.09  |\n| FedAvgM              | 1 × 1 × 1  |   66.03 |  67.69  |\n| FedAdagrad           | 1 × 1 × 1  |   67.07 |  69.18  |\n| FedAvg               | 1 × 1 × 1  |   66.23 |  67.69  |\n| Mean                 | 1 × 1 × 1  |   67.58 |  68.31  |\n| Combine              | 1 × 1 × 1  |   65.84 |  68.09  |\n| YOCO$ _{initB\\bar{A}}$ | 1 × 1 × 1  | 68.15 |  69.83  |\n\n*Note: V2\\_6 = MiniCPM‑V‑2\\_6int4; V2\\_5 = MiniCPM‑Llama3‑V‑2\\_5int4.*\n\nAs shown in the above table, YOCO continues to demonstrate its effectiveness at larger scales compared with basic one‑shot baselines and multi‑round FL.\n\n#### **Scalability Analysis**\n\n| Setting            | Key Result                                                                 | Implication                                                            |\n| ------------------ | -------------------------------------------------------------------------- | ---------------------------------------------------------------------- |\n| **IID, $N\\to\\infty$** | *Conflict* $\\mathbb{E}[C]\\approx\\tfrac12-\\Theta(N^{-1/2})$ *(or lower if $p_i$ biased)* | $C$ **does not grow** with $N$; it actually falls as signs align.               |\n|                    | *Noise* $\\operatorname{Var}(\\Delta W_g)=\\dfrac{\\sigma^2+4C(1-C)\\mu^2}{N}$ | Variance shrinks like $1/N$; YOCO further lowers the $4C(1-C)\\mu^2$ term. |\n| **Non‑IID, $G\\ll N$** | *Conflict* $O(1/N)+O(G^2/N^2)$; *Noise* same $1/N$ term + $O(G/N)$    | Cluster heterogeneity adds tiny $G/N$ effects; $1/N$ scaling — and YOCO’s gains — remain. |\n\n**Notation:** $G$-number of client clusters (non‑IID groups). $p_i$-probability that parameter $i$ is positive. \n\n**Bottom line:** Whether IID or moderately non‑IID, increasing clients **always reduces both conflict and variance at $O(1/N)$**. YOCO’s constraint further cuts the $C$‑linked variance, so its benefits persist—even with hundreds or thousands of clients.\n\nWe will include these additional experiments and scalability analyses in the final version.\n\n> **Q3:** Do YOCO’s gains come from its innovations or from the inherent advantages of large foundation models in one‑shot FL, as shown in [d] One Communication Round is All It Needs for Federated Fine‑Tuning Foundation Models?\n\n**A3:** Thank you for your thoughtful question.\n\nReference [d] shows that in one-shot FL, a single communication round can match multi-round FL for large foundation models, but this relies on several assumptions. From [d] and its 2025 ICLR reviews, these include:  \n\n1. Pre-training and fine-tuning datasets are closely aligned.  \n2. Only FedAvg is used for federated optimization.  \n3. Random splits yield relatively homogeneous data from one or two sources.  \n\nTo this end, we first present the results of [d] in our evaluation setting, confirming that one-shot performance falls short of multi-round training. We then analyze the advantages of YOCO’s innovations based on the aforementioned assumptions.\n\n**Evaluation of [d] in Our Setting**\n\nBecause [d] uses a text-based LLM unsuited to our multimodal evaluation, we instead adopted a multimodal LLM. The results below show that one-shot performance falls short of multi-round training (More in Tables 1–2).\n\n|Multi-round FL|74.02|73.24|72.97|76.82|77.28|75.58|\n|:-:|:-:|:-:|:-:|:-:|:-:|:-:|\n|OFL [d]|68.85|67.31|68.15|67.45|67.32|67.33|\n\n**YOCO**\n\n#### 1. Distribution Mismatch\n\n* **Setting:** The pre-training and fine-tuning distributions differ (e.g., MLLM better suited for QA tasks, while downstream tasks involve binary classification on *Hateful Memes* or multi-class classification on *CrisisMMD*).\n* **Observation:** As shown in Tables 1–2 of the paper, the MLLM+OFL baseline underperforms multi-round FL, with a performance gap up to **6.5 points**.\n* **YOCO Result:** Under the I:5–T:5 setting on *CrisisMMD*, YOCO achieves performance comparable to multi-round FL, outperforming the OFL ensemble baseline by **3.71 points** and the best non-ensemble baseline by **5.35 points** — at only \\~**0.03%** of the communication cost.\n\n#### 2. Beyond FedAvg\n\n* **Setting:** When using adaptive optimizers such as **FedAdam** and **FedAdagrad** instead of FedAvg.\n* **Observation:** On *CrisisMMD*, one-shot FL performance drops notably, likely because these dynamic optimizers are less robust to the increased local training bias inherent in one-shot settings.\n* **YOCO Result:** YOCO, specifically designed to mitigate local training bias, maintains strong performance despite these conditions.\n\n#### 3. Complex Data Heterogeneity\n\n* **Setting:** A total of **12 heterogeneous scenarios**, including 9 multimodal heterogeneous cases.\n* **Observation:** In cross-model cases with stronger distribution shifts (e.g., I:–T:), OFL one-shot baselines degrade sharply compared to multi-round FL.\n* **YOCO Result:** YOCO continues to deliver substantial improvements, with particularly strong results on *CrisisMMD*.\n\n> **Q4:** Uses only last-layer LoRA fine-tuning, which the authors acknowledge is suboptimal.\n\n**A4:** Thank you for raising this point. It is important to distinguish two separate optimization directions here. **First**, the choice of parameter configurations—such as the number of LoRA fine‑tuned layers and their automatic selection strategy. **Second**, and the central focus of our work, is how to effectively reduce local training bias under truly one‑shot communication. While both are challenging and critical for MLLMs with OFL, our current study concentrates on mitigating local training bias, and we leave parameter selection strategies for future exploration.\n\nOur experiments further show that **fine‑tuning a single LoRA layer already yields strong performance**, in some cases surpassing baselines that train all layers over multiple rounds. We also tested different layer positions; the overall performance differences were small. Within the same layer, choosing among *q\\_proj*, *k\\_proj*, *v\\_proj*, or *o\\_proj* led to only minor variations, as illustrated in the right panel of Figure 7."}, {"review_id": "4UkcZndt06", "reviewer": "Reviewer_HPxi", "summary": "This paper proposes YOCO, a novel one-shot communication framework for federated learning with MLLMs. The authors discover that imposing directional supervision on local training substantially mitigates client conflicts and reduces local bias. YOCO leverages this insight by applying directional supervision through sign-regularized LoRA B to enforce global consistency, while using sparsely regularized LoRA A to preserve client-specific adaptability.", "questions": "- In the \"Empirical Motivation\" section, the authors claim that the training loss under directional constraints indicates effective implicit global supervision without disrupting the LoRA training process. However, Figure 3 only demonstrates lower training loss. Could the authors clarify the logic why lower training loss necessarily indicates better supervision? It appears that unconstrained training might potentially achieve better results under perspective of training loss.", "limitations": "Yes.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 3, "originality": 4, "strengths_and_weaknesses": "Strengths:\n\n- The paper is well-written with clear logical flow and fluent presentation.\n- The empirical motivation provides valuable insights into the federated learning challenges.\n- The SVD-based initialization is an elegant technical contribution.\n\nWeaknesses:\n\n- The principal component-weighted aggregation shows inconsistent improvements across different datasets (e.g., Hateful-Memes and CrisisMMD), raising questions about its generalizability.", "was_revised": false, "final_justification": "", "rebuttal": "We would like to thank **Reviewer HPxi** for the thoughtful feedback and valuable questions. We have addressed these concerns with detailed responses and welcome further discussion and suggestions.\n\n> **Q1:** The principal component-weighted aggregation shows inconsistent improvements across different datasets (e.g., Hateful-Memes and CrisisMMD), raising questions about its generalizability.\n\n**A1:** Thank you for pointing this out. The inconsistent improvements of principal component-weighted aggregation across Hateful-Memes and CrisisMMD mainly stem from task difficulty and the choice of aggregation strategy.\n\nAs shown in Table 3 of the supplementary material, YOCO adjusts its aggregation method to dataset characteristics. CrisisMMD, unlike Hateful-Memes, includes more categories with pronounced modality gaps (e.g., I:-T:), making the task more challenging. In such cases, the ensemble approach (line 212 of the main text, where *\"Ensemble\"* aggregates LoRA votes from all clients) significantly outperforms methods that merge all client updates into a single model. Consequently, when principal component‑weighted aggregation compresses all client weights into a single set, its performance falls short of the ensemble strategy. This explains why, in CrisisMMD, the principal component‑weighted aggregation sometimes demonstrates less pronounced advantages than in Hateful‑Memes.\n\n\n> **Q2:** In the \"Empirical Motivation\" section, the authors claim that the training loss under directional constraints indicates effective implicit global supervision without disrupting the LoRA training process. However, Figure 3 only demonstrates lower training loss. Could the authors clarify the logic why lower training loss necessarily indicates better supervision? It appears that unconstrained training might potentially achieve better results under perspective of training loss.\n\n\n**A2:** We sincerely apologize for any confusion caused by the previous ambiguous phrasing. We now provide a clear explanation of the **purpose and insights** of Figure 3, followed by a discussion from the perspective of LoRA fine‑tuning to highlight **the role of directional guidance** in achieving effective adaptation.\n\n#### 1. Purpose of Figure 3\n\nFigure 3 presents the **training‑loss curves** under either a **directional** or **magnitude** constraint.\nOur objective is to identify which constraint does not disrupt the original training process and can thus serve as **a preliminary selection criterion**.\nThe subsequent subsection on **conflict matrix** analysis then explains **why** the direction constraint provides an effective implicit global supervision.\n\n#### 2. Insights from Figure 3\n\n* **Training stability.**\n   Directional constraints converge rapidly and soon reach the same loss level as the unconstrained baseline, unlike magnitude constraints.\n\n* **Interference with optimization.**\n   Magnitude constraints keep the loss consistently higher, suggesting stronger distortion of the optimization trajectory, whereas directional constraints cause only a mild detour with minimal gradient perturbation.\n\n* **Quality of supervision.**\n   Directional guidance is gradual and readily integrated, enabling the model to align with pre‑trained directions while retaining local learning flexibility. By contrast, magnitude constraints act as a rigid clamp that often obstructs convergence.\n\nTogether, these observations indicate that **directional supervision guides without disruption**, whereas magnitude supervision tends to constrain too aggressively.\n\n#### 3. Role of Directional Guidance in LoRA Fine‑Tuning\n\nAs LoRA updates are restricted to a low‑rank subspace, alignment with the **pre‑trained direction**—so as to approximate full fine‑tuning—is essential for effective adaptation. Prior studies support this view:\n\n- **\\[a] LoRA‑GA**: Uses SVD‑based initialization to preserve pre‑trained directions, aligning gradients with full fine‑tuning.\n- **\\[b] LoRA‑Pro**: Promotes gradient‑direction alignment at each step, further narrowing the gap.\n- **\\[c] FRLoRA**: Applies similar SVD initialization in federated settings, achieving consistent improvements.\n\nHence, leveraging pre‑trained weight directions as a lightweight yet effective form of **implicit global supervision** naturally extends these approaches.\n\n**Conclusion:** Our claim is **not** that lower training loss alone guarantees better performance. Instead, we show that **directional constraints** (i) ensure stable training, (ii) minimally disturb LoRA’s local updates, and (iii) embody a proven mechanism of global guidance, as validated by \\[a–c].\n\n\n**References**\n* \\[a] Wang, Shaowen, Linxi Yu, and Jian Li. *LoRA-GA: Low-Rank Adaptation with Gradient Approximation.* NeurIPS 2024.\n* \\[b] Wang, Zhengbo, et al. *LoRA-Pro: Are Low-Rank Adapters Properly Optimized?* ICLR 2025.\n* \\[c] Yan, Yunlu, et al. *Federated Residual Low-Rank Adaptation of Large Language Models.* ICLR 2025.\n\n\n> **Q3:** 1.Please use $\\ell_1$-norm instead of \"L1 norm\" for consistency with standard mathematical notation.\n\n**A3:** Thank you for your careful review. We will correct this in the final version."}, {"review_id": "nMB6gSenvs", "reviewer": "Reviewer_Cz1u", "summary": "This work introduces YOCO, a true one-shot communication method, after analyzing the rationale behind implicit global supervision for correcting local biases. Specifically, YOCO employs directional supervision with sign-regularized LoRA B to enforce global consistency, while sparsely regularized LoRA A preserves client-specific adaptability. Moreover, it adopts PCA-based aggregation to enhance both consistency and adaptability.", "questions": "- I have some doubts about using pre-trained models as “implicit global supervision” to correct local training biases. Since the dataset used for the pre-trained model may differ from the data used in this FL setting, is it really reasonable to use this model as “implicit global supervision” to correct local training biases?\n- There is some confusion in Equation (5). For example, what does Z_i represent? How is P obtained?\n- Why is LoRA applied only to the last layer's q_proj?", "limitations": "yes", "rating": 2, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n- This work studies a novel question: using LoRA to fine-tune MLLMs under the OFL setting.\n- The design of using pre-trained models as “implicit global supervision” to correct local training biases is novel and interesting.\n\nWeaknesses:\n- The paper seems to present conflicting conclusions. Figure 4 indicates that the conflict among LoRA A is smaller than that of LoRA B. Does this mean that A learns shared knowledge (resulting in smaller direct conflicts), while B learns client-specific features (leading to greater conflicts)? Why is it that after PCA in Figure 5, the learned A appears more similar, while B appears less so, leading to the conclusion that \"LoRA B learns shared knowledge, while LoRA A captures client-specific features\"? \n- Some phenomena in this work are not well explained. For example, in lines 132-133, \"Constraining both LoRA A and B introduces excessive restrictions, which in turn increases the direction conflicts of LoRA A across clients.\" Why does this happen?\n- In lines 184-185, \"The regularization terms encourage LoRA B to learn shared knowledge, while LoRA A captures client-specific features.\" How is this achieved? Why does the sparse regularization help A learn specific knowledge?\n- In Figure 4, what are the results when only using SVD without constraints? Since the effect in Figure 4(d) may be attributed to SVD, it would be better to include results that use only SVD.\n- The statement in line 8, \"meaning they never attain true one-shot communication,\" seems a bit too absolute. According to the authors' discussion in Section 2, there are indeed some works that achieve \"true one-shot communication.\"", "was_revised": true, "final_justification": "Most of my concerns are not resolved, such as the shared knowledge part, the rationale of the method design of this work, the explanation of the main experimental results, etc.  \\\nTaking the most important issues, i.e., the method design, as an example: The authors state, \"In the visualization of the conflict matrix, where directional constraints were applied simultaneously to both LoRA A and B, we observed that the conflicts in LoRA A increased instead. Based on this observation, we chose to apply global directional constraints only to LoRA B, which originally exhibited larger conflicts, thereby effectively reducing conflict\". However, the most important phenomenon, \"directional constraints were applied simultaneously to both LoRA A and B, we observed that the conflicts in LoRA A increased instead\", is not well explained. Moreover, why apply global directional constraints only to LoRA B and not to A is also not well explained. These experiments (Q10) were added during the rebuttal period, not the method design period, which raises doubts about the rationale of this method design. \\\nWhen I continue to ask these questions, the authors seem only to be muddying the issue (by posting 'Main Concern Resolved; New question raised by the reviewer'; however, the main concern remains, and no new question arises) and attacking the reviewer (\"We believe such a self-contradictory statement appears irresponsible\"), rather than focusing on addressing the reviewer’s concerns. Thus, I stand for the rejection of this work.", "rebuttal": "We sincerely appreciate **Reviewer Cz1u** for the insightful comments and constructive questions. We have provided comprehensive responses to clarify these points and look forward to further dialogue and feedback.\n\n>**Q1:** The paper seems to present conflicting conclusions. **(Q1-1)** Figure 4 indicates that the conflict among LoRA A is smaller than that of LoRA B. Does this mean that A learns shared knowledge, while B learns client-specific features? **(Q1-2)** Why is it that after PCA in Figure 5, the learned A appears more similar, while B appears less so, leading to the conclusion that \"LoRA B learns shared knowledge, while LoRA A captures client-specific features\"?\n\nThank you for raising these questions. We address Q1-1 and Q1-2 separately in A1-1 and A1-2, providing point-by-point responses.\n\n**A1-1:** We begin by clarifying the definition of **conflict rate**, emphasizing that its value **cannot directly indicate the type of knowledge**. Under unconstrained conditions, the conflict rate is often shaped by multiple factors acting together. We then explain **why LoRA B tends to capture shared knowledge, whereas LoRA A is more inclined to reflect client‑specific knowledge**.\n\n#### **1. Conflict rate $\\not\\Rightarrow$ Knowledge type**\n\nLet $k_i$ denote the number of client signs consistent on parameter $i$, with $M = \\operatorname{card}(\\mathbf{\\Delta W})$ representing the number of parameters and $N$ the total number of clients. The overall conflict rate $C$ and the average pairwise conflict rate $\\bar{\\rho}$ are defined as follows (see lines 112-127):\n$$\nC=\\frac{1}{MN}\\sum_i\\min(k_i,N-k_i)\n$$\n\n$$\n\\bar{\\rho}=\\frac{2N}{N-1}\\left[C-\\frac{1}{MN^2}\\sum_i[\\min(k_i,N-k_i)]^2\\right]\n$$\n\nBoth metrics rely only on the multiset ${k_i}_{i=1}^{M}$, failing to capture parameter semantics and being easily affected by initialization, noise, or sign imbalance. Hence, the conflict rate alone cannot determine the type of knowledge learned.\n\n#### **2. Knowledge type of LoRA B and A**\n\nDuring training, we impose a common **directional constraint** on every client to align the LoRA $B$ matrix with the pre‑trained weight direction. Without this constraint, LoRA updates are dictated entirely by local gradients: the *consistent* component across clients forms shared knowledge, while *differences* constitute specific knowledge.\n\nWith the constraint, each client’s LoRA $B$ converges toward the same global direction, explicitly injecting a common component and thus favouring the capture of shared patterns.  \nMeanwhile, client individuality is maintained—indeed emphasized—through an $\\ell_1$‑norm sparsity regularizer applied to the LoRA $A$ matrix. \n\n**In short:** LoRA B learns more shared knowledge due to explicit global directional guidance, whereas LoRA A, lacking such guidance and relying on local data, becomes increasingly biased toward client‑specific knowledge with more training epochs.\n\n**A1-2:** In the PCA visualisation of **Figure 5**, the features of LoRA $A$ from different clients are **more scattered**, whereas those of LoRA $B$ are **more clustered**. This supports our claim that **LoRA $A$ leans towards client‑specific knowledge, while LoRA $B$ leans towards shared knowledge**.\n\nIt is worth noting that in Figure 5, the regions separated by the dashed line correspond to two different data partitioning configurations. The PCA visualization assesses feature similarity based on the degree of dispersion and aggregation: the more aggregated the features, the higher their similarity, and thus the more likely they reflect shared knowledge. Therefore, it is reasonable to conclude that LoRA B tends to capture shared knowledge.\n\n>**Q2:** Some phenomena in this work are not well explained. For example, in lines 132-133, \"Constraining both LoRA A and B introduces excessive restrictions, which in turn increases the direction conflicts of LoRA A across clients.\" Why does this happen?\n\n**A2:** Thank you for the insightful comment. In the OFL setting, imposing identical directional constraints on both LoRA matrices ($A$ and $B$) overly restricts the adaptation space, thereby weakening the model’s ability to handle non‑IID data. Under such restrictions, clients are forced to update within the same narrow subspace. This limitation is particularly severe for the $A$ matrix, which is responsible for core feature projection and is highly sensitive to local data distributions. Consequently, gradient updates on $A$ are more likely to diverge across clients, further amplifying directional inconsistency. We will incorporate this clarification in the final version.\n\n>**Q3:** In lines 184-185, \"The regularization terms encourage LoRA B to learn shared knowledge, while LoRA A captures client-specific features.\" How is this achieved? Why does the sparse regularization help A learn specific knowledge?\n\n**A3:** Thank you for your insightful question. In this study, we impose a unified directional constraint on the LoRA $B$ matrix, aligning it with the direction of the pre-trained weights. This encourages all clients to share a common set of output basis vectors, thereby naturally capturing cross-client shared patterns. \n\nMeanwhile, we apply $\\ell_1$‑norm sparsity regularization to the LoRA $A$ matrix, which drives each client to retain only a small number of non-zero coefficients that are most representative of its local data. Due to the heterogeneity in data distributions, the activated sets differ across clients, enabling the model to effectively capture personalized features. \n\nThe effectiveness of this design is supported by the feature distribution visualization (Figure 5) and the cosine similarity analysis (Figure 8).\n\n>**Q4:** In Figure 4, what are the results when only using SVD without constraints? Since the effect in Figure 4(d) may be attributed to SVD, it would be better to include results that use only SVD.\n\n**A4:** Thank you for the suggestion. Since figures cannot be included in the rebuttal, we provide a quantitative explanation instead. In the original paper, **YOCO$_{\\text{init}}$** (lines 161 and 214) in Tables 1 and 2 corresponds to the **SVD‑only result**, while Figure 4(d) shows YOCO$_{\\text{initB}}$, which additionally incorporates the directional constraint on LoRA $B$. \n\nAcross most scenarios, YOCO$ _{initB} $ consistently outperforms YOCO$ _{init} $, with the largest gain of **2.23 points** observed at $p=80$% on the CrisisMMD dataset. This confirms that combining SVD with a directional constraint on LoRA $B$ yields better performance than using SVD alone. We will also provide a visualization of the conflict matrix for the SVD‑only case in the final version to further substantiate this conclusion.\n\n>**Q5:** The statement in line 8, \"meaning they never attain true one-shot communication,\" seems a bit too absolute. According to the authors' discussion in Section 2, there are indeed some works that achieve \"true one-shot communication.\"\n\n**A5:** Thank you for pointing this out. The full sentence on line 8 of the original manuscript is:\n\n> *“However, existing adaptive ensemble OFL methods still need more than one round of communication, because correcting heterogeneity‑induced local bias relies on aggregated global supervision, meaning they never attain true one‑shot communication.”*\n\nThe conclusion *“they never attain true one‑shot communication”* rests on two premises:\n\n1. “existing adaptive ensemble OFL methods,” and\n2. “correcting heterogeneity‑induced local bias relies on aggregated global supervision.”\n\nWe sincerely appreciate the reviewer’s attention to the precision of our wording and fully agree with this rigorous academic attitude. In the final version, we will revise the statement to *“meaning they still do not achieve true one‑shot communication”* to convey the intended meaning more accurately.\n\n>**Q6:** I have some doubts about using pre-trained models as “implicit global supervision” to correct local training biases. Since the dataset used for the pre-trained model may differ from the data used in this FL setting, is it really reasonable to use this model as “implicit global supervision” to correct local training biases?\n\n**A6:** Thank you for raising this concern. Due to space limitations, please refer to our response **A2** to **Reviewer P4k5**.\n\n>**Q7:** There is some confusion in Equation (5). For example, what does $Z_i$ represent? How is $P$ obtained?\n\n**A7:** We apologize for the confusion. Here, $Z_i$ denotes the coordinate vector of the $i$‑th client in the top‑$k$ principal components, representing the position of client $i$’s LoRA $A$ in the principal component space.\n\nDuring PCA, we obtain both the principal component coordinates $Z$ and a loading matrix $P \\in \\mathbb{R}^{d\\times k}$. In the paper, this is written as $P^{\\top} \\in \\mathbb{R}^{k\\times d}$—the transpose of the loading matrix—used to reconstruct the original parameter space from the global coordinates $Z_g$.\n\nHere, $Z_g$ is the weighted average of all $Z_i$ vectors, with weights determined by each client’s local data size $|D_i|$. We will add this clarification in the final version to improve clarity and completeness.\n\n>**Q8:** Why is LoRA applied only to the last layer's q_proj?\n\n**A8:** Thank you for raising this point. The choice of LoRA layers—including their number and positions—falls under **parameter selection** for LoRA fine‑tuning, whereas this paper focuses on mitigating **local training bias** under one‑shot communication. While distinct, both are critical for MLLMs with OFL; we leave layer selection strategies to future work.\n\nOur experiments show that fine‑tuning even **one layer** achieves strong performance, sometimes surpassing multi‑round baselines. Evaluations across different single‑layer positions revealed only **minor differences**, with slight variations among $q_{proj}$, $k_{proj}$, $v_{proj}$, and $o_{proj}$, as shown in Figure 7, with $q_{proj}$ and $v_{proj}$ performing slightly better overall."}]

Author rebuttal:

Meta-review: This work presents a technique named YOCO to achieve truly one-shot FL (OFL) in the Multimodal LLM (MLLM) scenario. As pointed by the reviewers and further admitted by the authors the technique is also applicable to unimodal settings. The key idea revolves around imposing directional supervision over LoRA head B with sparse regularization on LoRA head A. The authors demonstrate consistent improvements over the baselines. 

After the rebuttal round, most concerns raised by the reviewers were resolved. However the following were only partially resolved or unresolved:
- evaluation is done on a limited amount of clients (upto 50) due to GPU constraints 
- ablation disentangling the contribution of LLMs and proposed direction supervision is not adequately studied 
- the number of datasets studied are relatively smaller thus affecting generalizability of the claims such as:
   - "A is more sensitive to local data distributions" which is only anecdotal and also observed in some other prior works but not theoretical.

Despite these partially unresolved concerns, most reviewers believe that the experimental evaluation present convincing empirical basis to validate the methods effectiveness. Since the identified limitations can be addressed in a future extension without undermining the core contribution of this work, I recommend an acceptance for this work.
Decision: Accept (poster)