FORLA: Federated Object-Centric Representation Learning with Slot Attention
Authors: Guiqiu Liao
Matjaz Jogan
Eric Eaton
Daniel A Hashimoto
Venue: NeurIPS 2025 poster ()
Keywords: Federated learning
Representation learning
Slot attention
Object-centric learning
Primary area: deep_learning

Abstract: Learning efficient visual representations across heterogeneous unlabeled datasets remains a central challenge in federated learning. Effective federated representations require features that are jointly informative across clients while disentangling domain-specific factors without supervision. We introduce FORLA, a novel framework for federated object-centric representation learning and feature adaptation across clients using unsupervised slot attention. At the core of our method is a shared feature adapter, trained collaboratively across clients to adapt features from foundation models, and a shared slot attention module that learns to reconstruct the adapted features. To optimize this adapter, we design a two-branch student–teacher architecture. In each client, a student decoder learns to reconstruct full features from foundation models, while a teacher decoder reconstructs their adapted, low-dimensional counterpart. The shared slot attention module bridges cross-domain learning by aligning object-level representations across clients. Experiments in multiple real-world datasets show that our framework not only outperforms centralized baselines on object discovery but also learns a compact, universal representation that generalizes well across domains. This work highlights federated slot attention as an effective tool for scalable, unsupervised visual representation learning from cross-domain data with distributed concepts.

FORLA: Federated Object-Centric Representation Learning with Slot Attention
Guiqiu Liao'
Matjaz Jogan' Eric Eaton Daniel A. Hashimoto 1.2
'PCASO Laboratory, Dept. of Surgery, University of Pennsylvania Deet.. of Computer and Information Science, University of Pennsylvamia
guiqiu.liao@pemeeeee www.lliimmmeeiiicaecce..e.... cine. upenn. edu
Abstract
Learning efficient visual representations across heterogeneous unlabeled datasets re- mains a central challenge in federated learning. Effective federated representations require features that are jointly informative across clients while disentangling client- specific factors without supervision. We thus introduce FORLA, novel framework for federated object-centre representation learning and feature adaptation using unsupervised slot attention. At the core of our method is shared feature adapter, trained collaboratively across clients to adapt features from foundation models, and shared slot attention module that learns to reconstruct the adapted features. To architecture. In each
optimize this adapter, we design a two-branch student- client, student decoder learns to reconstruct full features from foundation models, while a teacher decoder reconstructs their adapted, low-dimensional counterpart. The shared slot attention module bridges cross-domain learning by aligning object- level representations across clients. Experiments in multiple real-world datasets show that our framework not only outperforms centralized baselines on object discovery but also learns a compact, universal representation that generalizes well across domains. This work highlights federated slot attention as an effective tool for scalable, unsupervised visual representation learning from cross-domain data with distributed concepts. Our code, data, and pretrained models are available https: / /gi thub. cwm/PCAASlaa/FOOO..
Introduction
Self-supervised vision models such as DINO [7] and MAE [20] have popularized representation learning as transferable pretraining strategy for diverse downstream tasks. However, their joint in federated learning (FL) remains underexplored. Consider a FL scenario where clients hold data from different domains and sub-domains: for example, some contain expert domains like various types of surgical procedures, while datasets on other clients represent natural domains with everyday objects. Despite their differences, these datasets may share certain properties, such being captured with visible light cameras, leading to overlapping textures or visual patterns. In some cases, specific objects, such tissues and surgical instruments, shared only within sub-domains of the expert domain, in this case across different surgical types. This heterogeneity poses challenges for FL using unsupervised methods that leverage consistency capture semantic meaning in the feature space [7, 20, 30].
We propose that self-supervised object-centric learning [17, 38] offers the right inductive bias for learning semantically meaningful representations from independently curated datasets while preserving object-level knowledge across clients in federated settings. Slot Attention [38] leamss object-centric representations by factoring scene into independent slots that specialize individual objects, yielding interpretable, potentially domain-invarian basis. However, training slot attention
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

on pooled data across clients entangles concepts and inflates computation, while training one model per client preserves local structure but leaves feature spaces incompatible.
We thus introduce FORLA (Federated Object-Centric Representation Learning with Slot Attention). that couples object-centric structure learning with collaborative feature adaptation across clients. Each client starts from frozen conndation-mode embeddings. A shared, lightweight adapter maps these embeddings into compact latent space. We train the adapter through a two-branch teacher- student objective: (i) a teacher decoder reconstructs the learnable feature representation; (ii) student decoder reconstructs the frozen foundation model representation; (iii) both teacher and student share the adapter and slot attention modules with other clients. Importantly. FL lets the student inherit the teacher's knowledge without a knowledge distillation (KD) loss, sidestepping the permutation permutation-matching problem posed by slot attention's 's permutation invariance.
The dual-branch architecture of FORLA, with one branch reconstructing reconstructs dynamic features, is crucial for efficient learning. On the one hand, reconstructing raw features anchors the learning objective and enables convergence. On the other hand, learning adaptive features improves object representation in complex scenes and novel domains, as shown by the PCA visualization of adapted versus raw features in Fig. 2.
features while the other
Object-centric learning has not been scaled to large multi-domain datasets, including mixtures of expert and natural domains. This work, to our knowledge, is the first to study and demonstrate the advantage of federated unsupervised object centric learning at scale. Our key contributions are: 1) We consider representation learning first-class problem in FL and motivate object-centric learning as a principled solution. 2) We design an architecture with a communication-efficcal. teacher-student adapter and slot attention modules that harmonizes heterogeneous raw foundation features without sharing data. 3) Through extensive experiments we show that this architecture consistently outperforms centralized training on mixed data, even when using identical foundation models. For instance, federated slot attention achieves a 7.6% higher mIoU in multi-domain object discovery while reducing client-server communication by 85 55.2% through lightweight adaptation layers, compared to standard fine-tuning of foundation models. Furthermore, relative to the mixture- coannddaiiooodeeel approach, the communication cost can be reduced by up 6.7x. This highlights FL's unique ability to exploit distributed datasets when using an object-centric approach.
2
Related work
In this section we provide brief overview of related work for FORLA which we further expand in Supplementary Material A.
Federated Learning (FL) While FL traditionally focuses on parameter aggregation [42], recent work addresses non-lID challenges through prototype exchange [44], contrastive alignment [69]. and knowledge distillation [19, 63). However, these methods focus on classifccation/sgmenttii than learning disentangled representations. Our work bridges this gap by introducing self-supervised object-centric learning FL..
rather
Object-Centrie Learning Slot Attention [38] introduced object-centric learning via iterative slot refinement. Follow-up work improved scalability [56]. applied foundation models [52], and enhanced performance via feature augmentation [39], multimodal inputs [67, 2], or fine-tuning [12]. However, no prior work integrates slot attention with federated learning across heterogeneous data which is setting where FL can mitigate concept entanglement by preserving local structure.
Concept Entanglement in Multi-Domain Learning Concept entanglement persists as a key chal- lenge in multi-domain learning with overlapping semantics. Early solutions like Domain Separation Networks [4] explicitly partition features into shared and private subspaces. Subsequent advances include adversarial Nomain-invarian encoding [37], variational information maximization [24], and label-free latent disentanglement via sparse adapters [9]. Recent work further distills domain-specific representations through adapter alignment [32]. While these methods reduce entanglement, none address object-centric disentanglement gap bridge in both centralized and federated settings.
Adaptation Layers and Feature Harmonization Adaptation layers align feature spaces in transfer [49] and multi-task learning [59]. In FL, adapters support personalization [8]. FT-DINOSAUR [12]
2

---

Staget: representation distilatioo & alignment
Stage2: federated representation rediscovery
Emage
Ciient
Inpul
Client
Input
Foundation
Orocdills
Gooai Fudley
thutent
EMA
Inacher
Global Fachoo
Sludeet
Matther
festure stack
Local Fieskin
sine alteniza
tn sileriion
pioe adlentan
Libol Pecting
FL Server
Sdopter
musdec
adapter
misicc
ocoster
EMM
didaptor
zlot attention
sio attention
Impul
Impie
Stucent
Woacher
Student
Teacher
loss:
EMC
attention
attention
Feddin
Loca Fodiigg
locky
sloc atieniie
alok slioriion
skit alluriiiii
akkt alterrion)
decocrr
Local FaetAny
SA mastee
SA mankks
Student roconstruction
Teacher
Figure 1: Overview of FORLA. Left: Within each client, student and teacher branches are trained to reconstruct raw foundation model features and adapted features, respectively. Right: During each global federated learning (FL) round, the student's s adapter and Slot Attention (SA) modules are aggregated across clients via the server. In later stages of training, the teacher's adapter and SA modules are locally synchronized with the student through local FedAvg update, enabling progressive knowledge distillation.
shows that object-centric learning benefits from cross-domain fine-tuning. Lightweight adapters [16, 70]. bias tuning [6, 68], and low-rank updates [22] offer scalable adaptation. Masking- adaptation for frozen models [60] enables harmonized multi-model input without the need to tune one particular foundation model. Our approach integrates these ideas in a federated setting to align local representations across diverse clients.
Knowledge distillation Knowledge distillation has been used in federated learning in fully supervised setting [71, 61]. Unsupervised self-distillation, as in DINO [7], ensembles knowledge via Exponential Moving Average (EMA) updates from student teacher, akin to Polyak averaging [45]. We adopt a similar strategy: the teacher is updated by averaging with the student, without a conventional KD loss; step necessary due Slot Attention's permutation invariance [38]. Our method also related to co-distillation [1], where student and teacher models mutually distill knowledge perform efficient ensemble learning from large-scale, distributed data. Unlike standard co-distillation, which employs identical models, we share only a subset of modules between student and teacher, and our framework operates entirely in an unsupervised setting.
3 Method
3.1 Overview
Figure 1 illustrates the proposed framework for federated object-centric representation learning and feature adaptation using Slot Attention. Clients share frozen features extracted from multiple vision foundation models. On top of these shared features, FORLA trains two parallel branches with identical fcature-adapter and Slot Attention architectures but with slot decoders of different sizes. Lightweight adapters compress and harmonize the stacked foundation features into compact, low-dimensional representation. The two Slot Attention modules extract object-centric latent slots using the adapted features. The student decoder learns to reconstruct the raw high- nigh-dimensional frozen features, while the teacher decoder reconstructs the adapted features. During training, the adapter and Slot Attention parameters are periodically aggregated and synchronized across clients and within each clients' branches using FedAvg. This architecture enables FORLA to: (i) learn global distilled representation efficiently, and (ii) align slot attention across heterogeneous domains without supervision.
3

---

3.2 Slot Attention with Adapted Foundation Features
Feature stacking and adaptation Given an input image, we extract features from M frozen foundation models (DINO, SAM [28], MAE, CLIP [48]). Let F(m) E Hx xCm be the feature map from model m. We concatenate these into:
F == [ ( ||
F0M E Hx Cio Tox=E
- Cm
To reduce dimensionality and fuse the stacked features, we introduce an adapter module 96 producing adapted features F addupt
IR. x www.wxx xd,
F adaatt = (F), < Coo
We explore three feature adapter designs: MLP adapter [50], mixture-of-experts (MoE) [53, 15], and Attention-based Feature Modulation (AFM) [23]. Details are provided in Supplementary Material B.
Slot attention The adapted features F ualag flattened into N H x W vectors and fed into Slot Attention encoder [38], producing K object-centric slot latent vectors {sx } As in DINOSAUR [52], decoding reconstructs target features and slot attention masks using MLP decoders. Details on Slot Attention and decoding can be found in Supplementary C.
3.3 Feature reconstruction and loss functions
The student branch aims to reconstruct the full stacked feature map. F = DSw( E B2 while the teacher reconstructs the lower-dimensional adapted feature map, F addape = Drea(S) E R H1 x W xd Student and teacher reconstruction losses are defined as:
1
L1
F
F
(3)
HWC
L2 HWd
F Fadapt
F Fadapt
As illustrated in Fig. 1, L1 and L2 independently optimize the student and teacher branches.
prevent collapse (i.e., trivial reconstructions zero target), we adopt progressive training strategy. During early training, gradients from the teacher decoder to its adapter are blocked and the adapter is updated via exponential moving average (EMA) from the student. This stabilizes training, as reconstructing raw foundation features is more reliable when starting from scratch.
Progressive Distillation
3.4 Federated Learning with Two-Stage Knowledge Transfer
Stage 1: Federated representation distillation and alignment When starting from scratch (carly rounds), described Algorithm 1, clients send updated student adapter and slot encoder parameters to the server. The server aggregates these using FedAvg:
(++1))
Ee The Σ n. 0(!)
where θ includes the adapter and slot encoder weights. This enables learning global representation adapter that generalizes across all client datasets, along with global Slot Attention module for object discovery across domains. In this stage only the student will learn to optimize the adapter, while the teacher will learn more adapted, domain domain-specific specific dbject-discovery.
Stage 2: Cross-brande federation. In later rounds, the teacher branch is allowed to further
optimize its own adapter and Slot Attention module. The refined knowledge then transferred student via a self-averaging strategy we term Local FedAvg. For this step, conventional knowledge distillation (KD) is challenging due to Slot Attention's 's permutation invariance. Slot alignment Hungarian matching [38] could make attention-based KD feasible, however adopt simpler

---

local FedAvg Algorithm Federated Two-Branch Slot Attention Training in FORLA
Initialize: Global parameters e (65 0²) for adapter 9e and Slot Attention SAB
2: Initialize: Local teacher branch (O (%a) + (0", ,8") via EMA
3: for each round r 2, ................................ R do
4: for each client c E {L. ..... C} in parallel do
5: Load global model: (ó., 0.) + -Ø) (0("-1) ,(r-11)
synchronize local models
6: for each local epoch
to E do
7:
for each batch B in client c's data do
Extract and concatenate features from M frozen foundation models Compute adapted features F Falage Obtain slots S SAs, (F (Fadaee)
9:
96. g (F)
learnable adapter
10:
# learnable slot attention
11:
Student loss: L1 = || F12 where (S) (Eq. 3)
12:
Teacher loss: L2 = Fatap Fateell 2 where Fatupi
Dus (S) (Eq. 4)
13:
Update student branch (#e. 0c) using Li
# gradient update for student
14:
if in later rounds then
15:
Update teacher branch (Øt 025) using C2
Local FedAvg: (Øe, 0e) ald,, 0.) + (1-a)(0 (e))
17:
else
18:
Update teacher branch 6tth using L2 EMA Update: de <- --M((0e)
19:
early stage: update teacher via EMA
20:
end if end for
21:
22:
end for
23:
Send updated student branch (de, 0s) server
# upload local parameters
24:
end for
25:
Global aggregation: (6()), + FEDAVG(HO,, 0.)
next-round global model
26:
end for
27: Return: Global model (Ø (
approach. We treat the student and the teacher two local clients, sharing knowledge through weight averaging using FedAvg. This allows the teacher to indirectly globalized by synchronizing with the student's Slot Attention module.
In such framework, the KD aspect (the two-branch reconstruction) is tightly integrated with FL: each client's ss training is effectively distilling the knowledge of multiple foundation models into the shared slot representation, and, return, can benefit from the knowledge transfer between teacher and student. By aggregating the adapter and slot encoder, a universal representation adapter and an object discovery module are learned by leveraging knowledge across all domains.
Experiments and Results
We evaluate FORLA a wide range of visual domains. Through extensive experiments, we aim to address the following research questions: 1) Centralized vs. decentralized learning: Can federated object-centric learning match exceed centralized performance? 2) Representation adapters: Which feature adaptation and distillation strategies work best across training regimes? 3) Algorithm compatibility: Can FORLA integrate with various federated optimization algorithms beyond FedAvg, such as FedProx and Fed/Admm 4) Representation quality: How interpretable are FORLA's representations compared frozen foundation features? 5) Impact of domain gap: How does the domain gap across datasets affect object discovery in solo, centralized, and federated training setups? 6) Component contribution: What is the role of each design component-din distillatinnn. adapter, and personalization the overall performance?
We perform rigorous evaluations both surgical and natural datasets using object discovery metrics. FORLA outperforms individual and centralized training, yielding interpretable representations and scene segmentation capabilities without supervision data sharing.
Dataset evaluate our experiment on seven datasets, categorized into two groups: 1) Surgi- cal vision datasets: including the abdominal surgical dataset [72], Cholec80 dataset [58], and proprietary thoracic surgery dataset. 2) Natural vision datasets: including COCO [36], PASCAL VOC 2012 [13], YouTube-VII [66], and YouTube-Object [47]. In total, these datasets comprise
5

---

Table 1: Federated performance on surgical and natural data (stratified evaluation).
Surgioil
Natural
Adaptes Method
Abdieminal
Choloe
Theratie
CDCO
PASCAL
YTVIS
YTORJ
mBO
CorLoc mBO
Corl.oc
EBO
IBO
PG AGG-ARI
IBO
MBD
MBD CocLoo
DINO
Indiviuull
47.33
57.87
52.18
30.96
23.96
29.2
33.09
54.78
SAM
Indicitual
47.00
53,70
25.75
32.62
36.07
54.95
Concot
Indivsdual
48.65
54.07
28.68
37.96
45.77
22.49
35.58
32.97
35.97
50.56
Indivstual
51.85
59,04
33.54
27.63
38.25
00:31
Contralized
55.91
61.31
76.47
31:12
41.12
56.48
27.59
52.2
37.5
57.43
FORLA
36.71
62.81
34.58
48.75
59.8
47.02
26.84
38.32
36,51
40.14
57.52
Indivsdual
51.83
57.54
72.02
34664
34.36
34,82
61.52
MOD
Centralized
54,89
60.66
30.37
24,82
27.72
36.21
53.43
FORLA 56.42
61.95
34.06
57.71
25.7
39.38
62.21
Individual
59,04
27.76
37.54
66.11
Centralized
61.5
32,87
36.4
52.13
FORLA
57.56 64.88
34.2
44.35
54.49
61.26
47.58
75.42
27.22
30.11
64.94
48.83
39.51
64.84
38.51
43.08
64.92
51,92
66.87
Table 2: Federated performance on mixed-domain training with all datasets.
Method
Abdorninal
Cholec
Thoracic
COCO
PASCAL
YTVIS
YTOBJ
Average
mBO Corloc mB0 CorLoe mBO CorLooe mBO CorLoc mBO CorLoe mB0 CorLoe MBO CorLoc mBO CorLoe
Individual AFM
50.34
70.73
32.90
42.36
53.03
56.62
24.15
56.22
34.98
51.10
33.94
54.59
48.98
65.11
40.33
56.82
Contralized + MLP
49,78
58.08
25
34.11
44,83
41 95
24.54
56.54
36.7
53.85
33.71
52.11
44,42
47
37.21
49.34
Cenmalized
53.46
68.92
25.28
50.55
46.05
52.21
Contralizee + AFM
54.16
72.5
31.64
46.42
53.72
57.49
26.68
63.97
38.93
53.85
37.15
60.41
47,84
57.41
41.45
58.81
Centraizedd Individual decoder
51,87
70.88
27.39
37,83
50.91
52.39
25.98
62.20
39.09
59.34
36.30
57.21
47.41
58.62
40.28
56.78
Slot FedAvg+ + MOE
55.77
70.55
30.23
44.27
54.53
59.93
26.37
63.97
39.81
63.19
37.78
61.86
48.98
63.65
41.92
61.20
Slot FodAvg AFM
53.74
69.65
32.63
41.66
52.01
50.42
25.45
63.17
39.1
63,74
37.60
62.30
48.12
61.74
41.24
58.95
FORLA + MLP
56.54
77.68
33.36
50.71
58.36
68.32
24.75
56
37.
51.65
36
57,50
44.25
52. 18
41.53
59.04
FORLA + MOE
$6.03
73.08
31.78
42.43
56.79
62.73
26.41
62.84
38.29
60.44
37:43
57.35
49.33
63.79
42.29
60.37
FORLA MOE + FedProx
54.19
72.62
32.42
44.06
57.24
61.1
26.73
62.52
39.19
60.44
37.87
60.84
46.27
$7.95
42.13
$9.89
FORLA + MOE + FedAdam
55.02
73.50
32.58
43.14
53.63
59.82
26.28
63,00
39.05
57.69
37.27
57.93
47.34
58.99
41.88
59.01
FORLA + APM
61.84
77.32
33,96
44.40
56.74
59.17
26.45
64.46
38.02
68.38
39.33
67.39
49.36
64.02
43.81
63.02
FORLA Surgery Natural
57.86
80.30
34.20
61.86
25.42
27.22
64.94
64.84
38.51
64.92
51.92
66.87
44.63
67.40
approximately 1.4 million images. With the exception of the proprietary thoracie dataset, all data is publicly available.' Further details are provided in Supplementary Material E.
Experiment setup Following other research on object centric representation learning papers (e.g
MONET [5], IODINE [18] and FT-DINO [11]), we evaluate our approach based on the quality of the
slot attention masks using four primary metrics: Foreground Adjusted Rand Index (FG-ARI) [17],
Mean Best Overlap (mBO) [46], and Correct Localization (CorLoc) [3] which are widely adopted in object-centric research [52, 39, 11J. We also compute Mean Best Hausdorff Distance (mBHD),
which captures boundary-level accuracy and is particularly important for clinical data.
We compare three training regimes: (i) Individual, where datasets are trained separately: (ii) Cen- tralized mixed, where related domains are combined; and (iii) Federated, where each dataset acts as an isolated client (up to seven domains). Features are extracted from frozen ViT-B/16 16 encoders of DINO, MAF, CLIP (vision branch only), and SAM (encoder for feature only). Each client trains for 100+ epochs with early stopping after 30 stagnant epochs. FedAvg is performed globally every 100 iterations and locally (student-teacher) every 1000. We use Adam (Ir 4x10- -4 batch size = 16). Please see more implementation details in the Supplementary Material F. Unless noted, evaluation uses the student decoder, while teacher results appear in Supplementary G.
Main results Given learned global representation via a feature adapter and a global object-centric slot attention module, we first investigate how their performance compares to models trained on individual datasets and centralized models trained on mixed datasets assigned to each client. We begin with controlled setting where domain gaps are smaller: using three clients for surgical data and four clients for natural datasets in a federated setup. These results are summarized in Table I.
We explore three types of adapters: MLP, MOE, and AFM, under individualized, centralized, and federated training. We also experiment using single foundation models and a naive feature stack (denoted as "Concat" in Table 1) without adaptation. Among the single foundation models, DINO
An open-acces version of the proprietary thoracic dataset is curated as part of the federated learning benchmark in this work.

---

Abdominal
Cholec
Thoracic
COCO
PASCAL
YTVIS
YTOBJ
Input
Foundation
Feature PCA
APM
Centralized
AFM FORLA
Slot attention mask
Contralized
Figure 2: Visualization of PCA maps and Slot Attention (SA) masks from different methods. The middle three rows show the first three PCA components visualized using RGB channels for frozen foundation model features and adapted features produced by the AFM module trained under central- ized training and under FORLA. The last two rows illustrate the scene decomposition ability of each method via the SA SA-generated masks.
and SAM achieved the best overall performance. Results for CLIP and MAE are provided in the Supplementary Material G. Interestingly, concatenated features slightly outperform others on surgical datasets (e.g. achieving 64.3 CorLoc on Abdominal), and are comparable to SAM and DINO on natural datasets, likely due to their shared pretraining when transferred to new domains. Once adapters are introduced, slot attention shows accuracy gains across all domains. Regardless of adapter type or training approach, performance improves. Among training strategies, the proposed federated method consistently outperforms both individualized and centralized training. Within surgical datasets, performance gains on Abdominal and Thoracic especially prominent. FORLA with AFM reaches 80.3 CorLoc on Abdominal and 75.42 on Thoracic, whereas Cholec achieves smaller gains due to closer domain proximity and distinct instrumentation.
natural images, enabling complementarity
For natural datasets, federated training also yields consistent performance improvements. Interestingly, for some subdomains, centralized training underperforms individualized training, likely because unsupervised slot attention benefits from domain-specific curation [11]. J. For instance, YTOBJ attains 48.98 mBO with individual training but drops to 44.51 mBO under centralized training using the same AFM adapter. Since YTOBJ has only 10 object categories but abundant data, mixing it with other datasets may dilute the inherent saliency, making unsupervised Icarning harder. This may also explain why most state-of-the-an slot attention work avoids mixed training. Among adapters, the attention-based AFM performs best, likely due to its scalability and ability to preserve rich foundation feature channels, achieving top CorLoc scores of 66.87 on YTOBJ and 64. 94 on PASCAL (Table 1).
To better understand how different approaches learn representations, Figure 2 shows the top three PCA projections of features visualized using RGB channels. We observe that FORLA captures more fine-grained and globally coherent semantic structures compared centralized mixed training. For instance, it more effectively separates distinct tissue textures in surgical scenes and groups semantically related object parts holistically, Leveraging these adapted representations, the globally

---

Table 3: Study on the Self-Distillation on Different Datasets and Setups.
Abdominal
Cholec
Thoracie
Method
mBO mHD
FG-ARI CorLoc
mBO
mHD
FG-ARI CorL.cc
mBO
mHD FG-ARI Corl
Individual + distill stage 33.91
80.240
35.41
29.22
35.35
56,349
45,28
48.89
33.25
109.666
22.70
30.07
Individual + distill stages
56.93
36,231
63.52
79.40
32.50
62.133
41.98
46.47
49.44
77.084
41.54
55.14
Centralized distill
54.34
50.635
59.34
68.43
27.82
76.464
36.41
37.14
48.73
77.987
40.61
47.07
FORLA self distill
61.84
33,909
69.49
77.32
33.96
59.801
45.07
44.40
56.74
59.790
45.42
59.17
COCO
PASCAL
YTVIS
YTOBJ
Method
MBO FG-ARI CorLoc
MBO
FG-ARI CorLocc
MBO
FG-ARI
MBO
FG-ARI CorL.nc
Individual distill stage
21.78
24.5
47.66
33.77
34.31
53.85
33.97
36.61
53.57
50.8
46.83
65.39
Individual + distill stages
24.56
27.5
53.8
36.09
35,26
50.00
32.79
36.03
46.87
50.37
46.61
63.97
Centralized distill
25.77
28.99
60.9
37,90
37.34
56.04
35.15
38.6
54.59
46.43
42.67
55.46
FORLA self distill
26.45
30.21
64.46
38.02
39,44
65.38
39.33
45.08
67.39
49.36
45.77
64.02
shared SA module is able to explicitly decompose scenes into meaningful object-level masks, as illustrated in the last two rows of Figure 2.
We further test more challenging setting by mixing all 7 datasets (surgical natural) across 7 federated clients, using the AFM FORLA model trained under lower-heterogeneity (stratified) settings strong baseline to see whether federated fully mixed centralized training can surpass it. As shown in Table 2, only the centralized model with AFM surpasses the individualized baseline; for instance, Centralized+MMLP achieves 49.34 CorLoc versus 56.82 for Individual+AFM, likely due to limited decoder scalability. To test whether limited decoder scalability caused centralized underperformance in comparison to FL, experiment hybrid setup: initializing centralized model with AFM, duplicating it to each client, freezing the adapter and slot attention, and fine-tuning individual decoders per client. While this improves performance on some datasets like Abdominal and PASCAL, it reduces performance on others, lowering the overall average, rejecting our hypothesis.
All federated approaches tend to outperform centralized or individualized training, thanks to the inherent domain disentanglement in object-centric learning. Even we remove the teacher from FORLA and apply standard FedAvg to the student's 's adapter and slot attention (referred to slot FedAvg) we see improvements over centralized and individualized baselines. Our full model (FORLA) with the AFM adapter achieves the highest performance despite the substantial heterogeneity across sub-domains. Interestingly. slot FedAvg performs well on the Natural dataset when using a MoE adapter but struggles Surgical datasets. This discrepancy likely arises because foundation models are typically pretrained on natural data, making teacher supervision and feature adaptation more critical for domains like Surgical, which are underrepresented in pretraining. It is noteworthy that MLP adapts well to surgical domains (e.g.. 79.73 Corl.oc on Abdominal in Table 1), but performance drops on natural domains (e.g., 52.18 on YTOBB in Table 2). MOE lies in between, consistent with its moderate gains. We also evaluated FedProx and FedAdam within FORLA+MOE that help in specific domains but do not yield general performance boosts (Table 2).
In conclusion, FORLA remains most effective when applied separately to groups of clients holding sub-domain data from either surgical natural domains. Nevertheless, even under increased heterogeneity, FORLA remains effective and outperforms both individualized and centralized training, highlighting its strength in learning disentangled, generalizable representations without data sharing.
Effect of Self- Saff-istiilattinn on Different Datasets and Setups As demonstrated, FORLA outper- forms direct FedAvg on adapter and slot attention, mainly due to its teacher-studen design for feature distillation and two-stage reconstruction. To test whether this also benefits individualized or centralized training. we applied the same strategy both. For individualized training. two versions are compared: a one-stage setup (EMA from scratch) and a two-stage setup (EMA followed by distillation). In most datasets, the two-stage version outperforms the one-stage setup. For example, as shown in Table 3, on Abdominal, mBO improves from 33.91 (1-stage) to 56.93 (2-stage), and CorLoc from 29.22 to 79.40. However, for Cholec and YTOBJ, the one-stage setup performs strongly (e.g.. YTOBJ: 65.39 CorLoc), possibly because slot attention in these datasets relies less on foundation features pattern also seen in simple-scene slot attention literature [38].
For centralized training, the two-stage approach on par with no-distillation setups (Table 2) gaining moderate accuracy on natural data, and far below FORLA's federated version. This observation
8

---

Table 4: Further model personalization experiment.
Abdominal
Cholec
Surgical Avg
YTVIS
YTOBJ
Natural Avg
Method mBO FG-ART Corl.oe mBO FG-ARI CortLo mBO
FG-ART Corl mBO CorLoe mBO
CorLoc mBO Corl
Individual
51.85
59,04
69.25
33.54
43.59
53.8
42.7
51.32
61.53
35.16
58.08
48.62
60.31
41.89
59.2
Centralized
54,16
59.54
72.5
31.64
41.5
46.42
42.9
50.52
59.46
37.15
60:41
47.84
57.41
42.5
58.91
No personalization
57.04
63.82
74.75
32.38
42.54
48.44
44.71
53.18
61.6
37,72
59.53
46.03
59.71
42.33
59.62
FORTA
Personalized
60.21
66.75
78.25
45.05
47.13
55.9
64.77
43.49
63.29
Personalized adupter
59.63
66.8
77.63
34,86
45.14
54.14
47.25
55.97
65.89
38,79
65.21
48.45
61.74
43.62
63.48
Segmentattoo mB00 [S]]
Localization CooLoe [N]
Datx Camb2
Data Comb'
Combl
Comb2
Data Combh
Commibbed
Andemil
PANCE
Phoel
1rat
Atdenoal
PAUGLL
Figure 3: mBO and Corloc for centralized training and FORLA across different data combinations, Data Comb: Data combinations. Individualized training is presented as baseline here.
Table 5: Results of combining two surgical Table 6: Results of different data partitions within the domains and distributing across clients.
same domain (one domain with 25% or 10% x4).
Method
Abdominul
Thoracc
Method
PASCAL (25%) Thuracic (25%) PASCAL [10%] Thoracic (10%)
mBO FG ARI CorL.cc mDO FG-ARI CurLoc
mBO
PG ARI
mBO
PG AR-ARI
mBO
FG ARI
mBO
PG ARI
domain (50F)c2) 50
53.0
43.48
Indivihal [25/105]]
33.21
31.52
46.39
30.79
30.71
28.11
27. 14
23.35
Centralized [2x50%x2]
Centralized (25010% x4]
34,98
34,16
53.03
43.48
34,99
33.52
49.17
40.11
FORLA
57.30
63.30
61.55
47.84
71.53
32.94
45.26
PORLA
35,25
34.14
32.09
49.66
further validates the advantage of the proposed framework, which tightly integrates distillation and federated training by treating the teacher and student as independent local clients.
Further Personalization Based on Global Representation We also explored whether further personalization further improves object discovery after learning global features representation and slot attention. Under a stratified setup (excluding Thoracic, COCO, and PASCAL), as shown in Table 4, FORLA (without personalization) already outperforms individualized and centralized models in this configuration. Next, initialize models with the global federated version and personalize them by freezing either the adapter or slot attention, allowing only one of them to be optimized locally. Both setups show that domain heterogeneity can be further countered by personalization while still leveraging shared knowledge. This is particularly effective for surgical data, which foundation models have been rarely trained on.
Robustness to Data Partitioning and Domain Combinations We further evaluate the robustness of FORLA under various data stratification and partitioning schemes. Figure shows that FORLA consistently outperforms individualized and centralized models across two- and four-domain combi- nations. cases with large domain gaps (e.g., surgical vs. natural), centralized training can even underperform individualized setups (e.g., Thoracic in Comb1, PASCAL in Comb2), while FORLA scales effectively across domains. Table 5 presents results where the Abdominal and Thoracic datasets are each split between two clients (50% per client). While the benefit of exploiting structural coherence within a local dataset diminishes when the data is distributed, our federated approach still surpasses centralized training, particularly on the Thoracic domain.
We also test performance under data scarcity by partitioning a single domain across four clients (Table 6). We first split full dataset (25% per client) and then a reduced dataset (10% per client, totaling 40% of the original). The results indicate that the advantage of federated object-centric learning declines as the data per client becomes more limited. This trend is consistent across both FORLA and the simpler Flot-FedAvg. This highlights the importance having sufficiently large and representative local datascts for unsupervised object-centric learning to discover meaningful concepts.

---

Table 7: Comparison to zero-shot segmentation of full SAM models with mask decoders.
PASCAL
Ahdominal
Thoracie
mBO
FG-ARI
CorLoc
mBO
FG-ARI
Corl.oc
mBO
FG-ARI
CorLoc
SAM (NT--B)
23.79
24.55
35.16
45.76
52.
39.15
18.10
16.41
8.35
SAM (NIT-H)
54,90
57_31
79.12
66.66
69.81
65.47
53.57
30.48
23.73
FORLA
40.83
39.51
45.04
57.86
61.88
80,30
61.86
47.58
75.42
Note that Thoracic is less publicly available thus likely not included in the training of SAM.
These findings are consistent with the general behavior of object-centric models, confirming that our framework requires data not to be overly partitioned.
Comparison to Supervised Segmentation Foundation Model We compare the unsupervised segmentation of FORLA to the zero-shot segmentation of full SAM models (using ViT-B and the ViT-H with its mask decoder trained under supervision). We found that SAM VIT-H is able to surpass slot attention models trained with FORLA on the natural images but fails on the very specific surgical domain (Thoracic). The underlying reason could be that the thoracic data is not publicly available and not included in the training data of SAM. In addition, the inference time for SAM ViT-H is 1.1s per image (0.85s for ViT-B), while the slot attention model takes only 3.75ms. Another advantage of OCL over SAM is that training SAM on new domains would require mask supervision and, most importantly, SAM will not capture the objectness (over decompose object parts to tiny masks) unless instructed to do so via annotation.
5 Conclusion
In this paper we proposed FORLA, addressing the critical challenge of learning disentangled visual representations from distributed, non-IlI) datasets. By integrating object-centric inductive biases with federated optimization FORLA enables collaborative learning across heterogeneous domains preserving local data structure. Extensive experiments across seven surgical and natural vision datasets demonstrate FORLA's superiority over both individualized and centralized learning of object representation. By using light adapters to learn from cached foundation features distributively, FORLA is an economical framework for large scale object-centric representation learning without centralized data or heavy compute infrastructure, achieving a 6.7x reduction in communication cost.
Limitations. While FORLA shows strong potential in learning universal object-centric representa- tions from heterogeneous data, several limitations remain. First, real-world visual scenes often exhibit hierarchical compositionality, where objects are nested or composed of multiple parts at varying levels of granularity. Our current framework still assumes that there an optimal decomposition hier- archy according to the given dataset, but such compositionality should be downstream task-specific. Second, our approach is still built on foundation models and struggles to discover objects purely based on image reconstruction [52], as it relies on image properties that have already been captured by foundation models.
Future work. Several research directions can further enhance FORLA. Incorporating dynamic slot mechanism [14, 34] will automatically adjust the number of slots per scene. We also intend explore alternative decoders, such slot-diffusion decoder [25], improve reconstruction and flexibility, as well as high resolution encoders such as DINOv3 [55]. Extending FORLA to support both image and video data [33] would enable unified training from mixed temporal modalities. Our approach could also support a federated object-centric foundation model for downstream tasks such as video action recognition, using slot representations as region-based tokens [54]. Integrating weak supervision signals from classification tasks [35] or language prompts [65] available to only a subset of clients could enhance the semantic alignment of learned slots and bridge the gap between vision and language in federated settings. In addition, incorporating additional learning objectives tailored to dataset subtypes or tasks, while using slots to bridge representations, could lead to more universal, task-agnostic representations [32]. Another promising next step could involve experimentation on robotic manipulation data [27]. These datasets represent a mix of data with specialized actions, similar to robotic surgery, and Natural datasets, they act on everyday objects.
10

---

Acknowledgments and Disclosure of Funding
This work was supported by the Linda Pechenik Montague Investigator Award and the American Surgical Association Foundation Fellowship.
References
[1] Anil, R., Pereyra, G., Passos, A., Ormandi, R., Dahl, G.E., Hinton, G.E.; Large scale distributed neural network training through online distillation. arXiv preprint arXiv: 1eX.iv1804.000 1804.03235 (2018)
[2] Bao, Z. Tokmakov, P., Wang. Y., Gaidon, A., Hebert, M.: Object discovery from motion motion-guided tokens. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023, Vancouver, BC, Canada, June 17-24, 2023. pp. 22972-22981 (2023)
[3] Bilen, H., Vedaldi, A.: Weakly supervised deep detection networks. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30,
2016. pp. 2846-2854 (2016)
[4] Bousmalis, K., Trigeorgis, G., Silberman, N., Krishnan, D., Erhan, D.: Domain separation
networks. Advances in neural information processing systems 29 (2016)
[5] Burgess, C.P., Matthey, I.,, Watters, N., Kabra, R., Higgins, I., Botvinick, M., Lerchner, A.: Monet: Unsupervised scene decomposition and representation. ArXiv preprint abs/1901.11390 (2019)
[6] Cai. H., Gan, C.. Zhu, L., Han, S.: Tinytl: Reduce memory, not parameters for efficient on-device learning. Advances in Neural Information Processing Systems 33, 11285-11297 (2020)
[7] Caron, M., Touvron, H., Misra, I., Jégou, H., Mairal, J., Bojanowski, P., Joulin, A.: Emerging properties in self-supervised vision transformers. In: 2021 IEEE/CVF International Conference on Computer Vision, ICCV 2021, Montreal, QC, Canada, October 10-17, 2021. pp. 9630-9640 (2021)
[8] Collins, L., Rendle, S., Ghosh, J.: Fedadapter: Efficient federated learning with local adapters. In: International Conference on Learning Representations (ICLR) (2023)
[9] Deecke, L.,, Hospedales, T.M., Bilen, H.: Visual representation learning over latent domains. In: Tenth International Conference on Learning Representations 2022 (2022)
[10] Didolkar, A., Zadaianchuk, A., Awal, R., Seitzer, M., Gavves, E., Agrawal, A.: Ctrl-o: language-
controllable object-centric visual representation learning. In: Proceedings of the Computer Vision and Pattern Recognition Conference. pp. 29523-2222 22533 (2025)
[11] Didolkar, A., Zadaianchuk, A., Goyal, A., Mozer, M., Bengio, Y., Martius, G., Seitzer, M.: Zero-shot object-centric representation learning. arXiv preprint arXiv:2408.0916 (2024)
[12] Didolkar, A.R., Zadaianchuk, A., Goyal, A., Mozer, M.C., Bengio, Y., Martius, G., Seitzer,
M.: On the transfer of object-centric representation learning. In: The Thirteenth International Conference Learning Representations (2025)
[13] Everingham, M., Eslami, S.M.A., Van Gool, L., Williams, C.K.I., Winn, J., Zisserman, A.: The pascal visual object classes challenge: retrospective. International Journal Computer Vision 111, 98-136 (2015)
[14] Fan, K., Bai. Z., Xiao, T., He, T., Horn, M., Fu, Y., Locatello, F., Zhang, Z.: Adaptive slot attention: Object discovery with dynamic slot number. In: Proceedings the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 23062-200262
(2024)
[15] Fedus, W., Zoph, B., Shazeer, N.: Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity. Journal of Machine Learning Research 23(120), 1-39 (2022)
[16] Gao, P., Geng, S., Zhang, R., Ma, T., Fang, R., Zhang, Y., Li. H.. Qiao, Y.: Clip-adapter: Better vision-language models with feature adapters. International Journal of Computer Vision 132(2), 581-595 (2024)
[17] Greff, K. Kaufman, R.I., Kabra, R., Watters, N., Burgess, C.,, Zoran, D., Matthey, L., Botvinick, M., Lerchner, A.: Multi-object representation learning with iterative variational inference. In: Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9-15 June

---

2019, Long Beach, California, USA. Proceedings of Machine Learning Research, vol. 97, Pp. 2424 2424-2433 (2019)
[18] Greff, K.. Kaufiman. R.L., Kabra, R., Watters, N., Burgess, C.,, Zoran, D., Matthey, L., Botvinick,
M., Lerchner, A.; Multi-object representation learning with iterative variational inference. In: International conference on machine learning. pp. 2424-2433 2433. PMLR (2019)
[19] Han, S., Park, S., Wu, F., Kim, S., Wu, C.,, Xie, X., Cha, M.: Fedx: Unsupervised federated learning with cross knowledge distillation. In: European Conference on Computer Vision. pp. 691-707. Springer (2022)
[20] He, K., Chen, X., Xie, S., Li, Y., Dollár, P., Girshick, R.B.: Masked autoencoders are scalable vision learners. In: IEEE/CVF Conference Computer Vision and Pattern Recognition, CVPR
2022, New Orleans, LA, USA, June 18-24, 2022. pp. 15979-15988 (2022)
[21] Heinrich, G., Ranzinger, M., Yin, H., Lu, Y., Kautz, J., Tao, A., Catanzaro, B., Molchanov, P.: Radiov2. 5: Improved baselines for agglomerative vision foundation models. In: Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 22487-22497 (2025)
[22] Hu, E.J. Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang. L., Chen, W., et al.: Lora: Low-rank adaptation of large language models. ICLR 1(2), 3 (2022)
[23] Hu, J., Shen, L., Sun, G.: Squeeze-ande und-excitation networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 7132-7141 (2018)
[24] Hwang, H., Kim, G.H., Hong, S., Kim, K.E.: Variational interaction information maximization for cross-domain disentanglement. Advances in Neural Information Processing Systems 33, 22479-22499 (2020)
[25] Jiang, J., Deng, F., Singh, G., Ahn, S.: Object-centric slot diffusion. In: Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 16, 2023 (2023)
[26] Kakogeorgiou, I., Gidaris, S., Karantzalos, K., Komodakis, N.: Spot: Self-training with patch- order permutation for object-centric learning with autoregressive transformers. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 22776-22786 (2024)
[27] Khazatsky, A., Pertsch, K., Nair, S., Balakrishna, A., Dasari, S., Karamcheti, S., Nasiriany, S., Srirama, M.K., Chen, L.Y. Ellis, K., et al.: Droid: A large-scale in-the-wild robot manipulation dataset. arXiv preprint arXiv:2403. 12945 (2024)
(28) Kirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., Xiao, T., Whitehead, S., Berg, A.C., Lo, W.Y., et al.: Segment anything. In: Proceedings of the IEEE/CVF international conference on computer vision. pp. 4015-4022 (2023)
[29] Konečný, J., MeMahan, B., Yu, F.X., Richtárik, P., Suresh, A.T., Bacon, D.: Federated learning: Strategies for improving communication efficiency. arXiv preprint arXiv:1610.088
(2016) [30] Li, D., Xie, W., Wang, Z., Lu, Y.. Li, Y.. Fang, L.: Feddiff: Diffusson model driven federated learning for multi-modal and multi-clients. IEEE Transactions on Circuits and Systems for Video Technology (2024)
[31] Li, T., Sahu, A.K., Zaheer, M.. Sanjabi, M.. Talwalkar, A., Smith, V.: Federated optimization in heterogeneous networks. Proceedings of Machine learning and systems 2, 429 450 (2020)
[32] Li, W.H., Liu, X., Bilen, H.: Universal representations: A unified look multiple task and domain learning. International Journal of Computer Vision 132(5), 1521-1545 (2024)
[33] Liao, G., Jogan, M., Hussing, M., Nakahashi, K., Yasufuku, K., Madani, A., Eaton, E.,
Hashimoto, D.A.: Slot-bert: Self-supervised object discovery in surgical video. arXiv preprint Ttt1:25011.14777 (2025)
[34] Liao, G., Jogan, M., Hussing, M., Zhang, E., Eaton, E.., Hashimoto, D.A.: Future slot prediction for unsupervised object discovery surgical video. In: International Conference on Medical Image Computing and Cosputer-Assisted Intervention. pp. 219-229. Springer (2025)
[35] Liao, G., Jogan, M., Koushik, S., Eaton, E., Hashimoto, D.A.: Disentangling spatio-temporal knowledge for weakly supervised object detection and segmentation in surgical video. ArXiv preprint abs/2407.15794 (2024)
12

---

[36] Lin, T.Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár. P., Zitnick, C.L.: Microsoft COCO: Common objects in context. In: ECCV. pp. 740-755 (2014)
[37] Liu, A.H., Liu, Y.C., Yeh, Y.Y., Wang, Y.C.F.: A unified feature disentangler for multi-domain image translation and manipulation. Advances in neural information processing systems 31 (2018)
(38) Locatello, F., Weissenborn, D., Unterthiner, T., Mahendran, A., Heigold, G., Uszkoreit, J., Dosovitskiy, A., Kipf, T.: Object-centric learning with slot attention. In: Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-1 2020, virtual (2020)
[39] Löwe, S., Lippe, P., Locatello, F., Welling, M.: Rotating features for object discovery. Advances in Neural Information Processing Systems 36, 59606-59633 59666 (2023)
[40] Lubana, E., Tang, C.I., Kawsar, F., Dick, R., Mathur, A.: Orchestra: Unsupervised federated learning via globally consistent clustering. In: Chaudhuri, K., Jegelka, S., Song, L., Szepesvari,
C., Niu, G., Sabato, S. (eds.) Proceedings of the 39th International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 162, pp. 14461-14484. PMLR html
(17-23 Jul 2022), https: http:///pccceenn..//////// mlr. wwweess/vve2//uuuaannaa
[41] Manasyan, A., Seitzer, M., Radovic, F., Martius, G., Zadaianchuk, A.: Temporally consistent object-centric learning by contrasting slots. In: Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 5401-5411 (2025)
[42] McMahan, B., Moore. E., Ramage, D., Hampson, S., Arcas, B.A.; Communication-efficn learning of deep networks from decentralized data. In: Artificial intelligence and statistics. pp.
1 1273-1282. PMLR (2017)
[43] Miao, R., Koyuncu, E.: Federated representation learning through clustering. In: 2023 IEEE 33rd International Workshop on Machine Leannnng for Signal Processing (MLSP). pp. 1-6 (2023). https://doi.or//I 1009/LLLP558442223...000
[44] Michieli, U., Ozay, M.: Prototype guided federated learning of visual feature representations orgo//////a ht////b//221000 08982
(2021), https://aaras
[45] Polyak, B.T., Juditsky, A.B.: Acceleration of stochastic approximation by averaging. SIAM journal on control and optimization 30(4), 838-855 (1992)
[46] Pont-Tuset, J., Arbelaez, P., Barron, J.T., Marques, F., Malik, J.: Multiscale combinatorial grouping for image segmentation and object proposal generation. IEEE transactions on pattern analysis and machine intelligence (9), 128-140 (2016)
[47] Prest, A., Leistner, C., Civera, J., Schmid, C., Ferrari, V.: Learning object class detectors from weakly annotated video. In: CVPR. pp. 128923228922 (2012)
[48] Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., Sutskever, I.: Learning transferable visual models from natural language supervision. In: Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 18-24 July 2021, Virtual Event. Proceedings of Machine Learning (2021)
Research, vol. 139, pp.
[49] Rebuffi, S.A., Bilen, H., Vedaldi, A.: Learning multiple visual domains with residual adapters.
In: Advances in Neural Information Processing Systems (NeurIPS) (2017)
[50] Rebuffi, S.A., Bilen, H., Vedaldi, A.: Learning multiple visual domains with residual adapters.
Advances in neural information processing systems 30 (2017)
[51] Seitzer, M., Horn, M., Zadaianchuk, A., Xiao, T., Biza, O., Schneider, T., van Steenkiste, S., Rätsch, G., Schflkopf, B., Brox, T.: Improving object-centric learning with query optimization. In: International Conference on Learning Representations (ICLR) (2023)
[52] Seitzer, M., Horn, M., Zadaianchuk, A., Zietlow, D., Xiao, T., Simon-Gabriel, C., He, T., Zhang, Z., Schölkopf, B., Brox, T., Locatello, F.: Bridging the gap to real-world object-centric learning. In: The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023 (2023)
[53] Shazeer, N., Mirhoseini, A., Maziarz, K., Davis, A., Le, Q.V., Hinton, G.E., Dean, J.: Outra- geously large neural networks: The sparsely-gane gated mixture-of-expprtt layer. In: International Conference on Learning Representations (ICLR) (2017)
13

---

[54] Saappentokh-Rothhaae M., Blume, A., Xiao, Y., Wu, Y. TV, S., Tao. H., Lee, J.Y., Torres, W., Wang. Y.X.. Hoiem, D.: Region- basied representations revisited. In: Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pp. 17107-17116 (2024)
[55] Siméoni, O., Vo, H. Seitzer, M., Baldassarre, F., Oquab, M.,Jos,, C., Khalidov, V., Szafraniec, M., Yi, S., Ramamonjisoa, M., et al.: Dinov3. arXiv preprint ar Xrx:v:2508. 10104 (2025)
[56] Singh, G., Deng, F., Ahn, S.: Slotformer: Unsupervised dynamic scene decomposition. In:
Advances in Neural Information Processing Systems (NeurIPS) (2022)
[57] Tan, Y., Long, G., Ma, J., Liu, L.., Zhou, T., Jiang, J.: Federated learning from pre-trained models: A contrastive learning approach. Advances in neural information processing systems 35, 19332-19344 (2022)
[58] Twinanda, A.P., Shehata, S., Mutter, D., Marescaux, J., De Mathelin, M., Padoy, N.: Endonet: deep architecture for recognition tasks on laparoscopic videos. IEEE transactions on medical imaging 36)1), 86-97 (2016)
[59] Vandenhende, S., Georgoulis, S., Van Gansbeke, W., Proesmans, M., Dai, D., Van Gool, L.: Multi-task learning for dense prediction tasks: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) (2021)
[60]
Warmerdam, A.T., Caron, M., Asano, Y.M.: Self-masking networks for unsupervised adaptation. In: DAGM German Conference on Pattern Recognition, pp. 86-102. Springer (2024)
[61] Wu, C., Wu, F., Lyu, L., Huang. Y., Xie, X.: Communication--ffiiciann federated learning via knowledge distillation. Nature communications 13(1), 2032 (2022)
[62] Wu, Z.,, Hu, J., Lu, W., Gilitschenski, I., Garg, A.: Slotdiffusion: Object-centric generative modeling with diffusion models. In: Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurlPS 2023, New Orleans, LA, USA, December 10 16, 2023 (2023)
[63] Xiao, H., Dong, S., Xiao, N.: Fedukd: Unsupervised federated learning with bilateral knowledge distillation and domain bias elimination. In: Proceedings of the 2024 International Conference on Intelligent Perception and Pattern Recognition, p. 92-99. IPPR '24, Association for Computing Machinery, New York, NY, USA (2024). http:///60.oo//1111114//70000000 //doi. org/1..011.... 1 1145/ 3700035 3700050
https:
[64] Xie, J.. Zhan, X., Liu, Z., Ong, Y.S., Loy, C.C.: Unsupervised object-level representation learning from scene images. Advances Neural Information Processing Systems 34, 28864- 28876 (2021)
[65] Xu, J., Lan, C., Xie, W., Chen, X., Lu, Y.: Slot-vim: Object-event slots for video- ladgo-language modeling. In: The Thihth- Annual Conference on Neural Information Processing Systems (2024)
[66] Yang, L.., Fan, Y., Xu, N.: Video instance segmentation. In: ICCV. pp. 5188-5197 (2019)
[67] Zadaianchuk, A., Seitzer, M., Martius, G.: Object-centric learning for real-world videos by predicting temporal feature similarities. In: Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans. LA, USA, December 10 16, 2023 (2023)
[68] Zaken, E.B., Ravfogel, S.. Goldberg, Y.: Bitfit: Simple eframeter-efficient fine-tuning for transformer-bant based masked language- models. arXiv preprint arXiv:2106. 10199 (2021)
[69] Zhang, F., Kuang, K., Chen, L.., You, Z., Shen, T., Xiao, J., Zhang, Y., Wu, C., Wu, F,, Zhuang,
Y., et al.: Federated unsupervised representation learning. Frontiers of Information Technology
& Electronic Engineering 24(8), 1181-1193 (2023)
[70] Zhang, R., Fang, R., Zhang, W., Gao, P., Li, K., Dai, J.. Qiao, Y., Li, H.: Tip-adapter: Training- free clip-adapter for better vision- lanoua-aagguae modeling. arXiv preprint arXiv:211 (2021)
[71] Zhu, 7.,, Hong, J., Zhou, Data-free knowledge distillation for heterogeneous federated learning. In: International conference on machine learning. pp.
PMLR (2021)
[72] Zia, A., Bhattacharyya, K., Liu, X., Berniker, M., Wang, Z., Nespolo, R., Kondo, S., Kasai, S., Hirasawa, K., Liu. B., et al.: Surgical tool classification and localization: results and methods from the miccai 2022 surgtoolloc challenge. ArXiv preprint abs/2305.07155 (2023)
14

---

NeurIPS Paper Checklist
The checklist is designed encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No] or [NA]. .
[NA] means either that the question is Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1-2 sentence) justificatin right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.
The reviewers of your paper will be asked to use the checklist as one of the factors in their evaluation.
While "[Yes] " is generally preferable to "[No]" ", is perfectly acceptable to answer "[No] "provided proper justification is given (c.g., "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). ). general, answering "[No] or "[NA] is not grounds for rejection. While the questions are phrased in binary way, we acknowledge that the true answer often more nuanced, so please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS Paper Checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only the provided macros for your answers.
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: This paper introduces an approach of federated Object-centric representation
learning with slot-attention. Section 3 defines the approach and Section 4 shows experiments using the approach.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
is fine include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes]
15

---

Justification: We mention limitations and further directions of improvement and future work in the conclusion section.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors encouraged to create separate "I "Limitations" section in their paper.
The paper should point any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, c.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proof's
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
Justification: We have provided the complete mathematical formulation and the algorithm, but theoretical claims are not made.
Guidelines:
The answer NA means that the paper does not include theoretical results. All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We will be releasing the code, data splits and pretrained models after review. The algorithm for the approach is included in the paper. The experimental settings are also provided in the main paper.
16

---

Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel urchitecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily to reproduce that algorithm.
algorithm, the paper should make it clear how
(b) If the contribution primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g.. large language model), then there should either be way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. the case of closed-source models, may be that access to the model limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, described in supplemental material?
Answer: [Yes]
Justification:
The code and data splits will be released on Github post review.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.. cc/
ciblic/guides/CCe sonddeSubiissionoolo..
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https: / /niiss /twpss.ccppub////// httpcccpccppppcceeeeeeeeppepeeee httooi/i/guddes/oddduuiiiiiiiiii Soodeeubmissiiiiiccccc... ssionPolicy) for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
17

---

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: While the code will be released along with pretrained models, we have mentioned the experimental settings in the paper. We have also provided additional details in the supplementary.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: We have saved testing sample with mean value standard deviation calculated. We use different metries, such as FG-ARI, mBO and Cor-Loc and mHD to report the performance. This helps show the statistical significance of the performance difference between different methods. Considering the large number of experiments, we only present the mean value in the paper.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
It is OK to report 1-sigma error bars, but should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
Question: For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
18

---

Answer: [Yes]
Justification: Provided in the experiment setup.
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make it into the paper).
9. Code ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https: / http/eeurppp..// cc
Answer:
[Yes]
Justification: all guidelines followed.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Introduction and conclusion discuss about our method's positive impacts. There are no apparent negative impacts.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deploymeet of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied partieular applications, let alone deployments. However, if there is direct path any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and funetioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
19

---

Question: Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification: No risk posed by the work.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We used combination of publicly available datasets (which cited) and proprietary clinical data. For the clinical data, all patient information was de-identified in compliance with HIPAA regulations, and the data usage was approved by the Institutional Review Board (IRB).
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g.. website), the copyright and terms of service of that source should be provided.
assets are released, the license, copyright information, and terms use in package should be provided. For popular datasets, paperswi thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re- se-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: Code will be released post review and the novel algorithm described in the paper. A curated dataset by combining our surgical vision data and public natural vision data for bench marking will be released too.
Guidelines:
The answer NA means that the paper does not release new assets.
20

---

Researchers should communicate the details of the dataset/codeemmodel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
No crowdsourcing involved.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: Retrospective research using de-identified data generally does not require IRB review.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification: The core method development this research does not involve LLMSs as any important, original, or non-standard components.
Guidelines:
21

---

The answer NA means that the core method development in this research does not involve LLMs as any important. original, non non-standard components.
Please refer to our LLM policy (https : http///eerppppepeeee /c/Conferencees22255// /LLM) for what should or should not be described.
22

---

Supplementary Materials for FORLA:
Federated Object- Centric Representation Learning with Slot Attention
A Extended Related work
To provide additional context for our approach, we expand on key areas of related work in this section.
A.1 Federated Learning (FL)
FL enables collaborative training across decentralized datasets while preserving privacy [42]. Prior work addresses challenges like non-IID data [31] and communication efficiency [29], but most focus on classification or segmentation tasks without emphasis on learning representations. Early work on unsupervised representation learning for FL showed that naive FedAvg misaligns local feature spaces under non-IID data. Zhang et al. [69] addressed this with FURL/FedCA, which maintains a shared memory bank and contrastive alignment module. Michieli et al. [44] proposed Prototyee-Guided FL. where clients periodically exchange class-agnostic prototypes to pull their feature spaces into common geometry. Tan et al. [57] also leverage prototypes in FedPCL in contrastive learning scheme. This idea is later extended by clustering-based methods such as ORCHESTRA [40] and FedRLC [43]. Knowledge distillation has also emerged as a powerful alignment tool. FedX [19] employs two-sided knowledge distillation with a contrastive objective, allowing clients to share rich representation information without exchanging data while FedUKD [63] further reduces domain bias via bilateral distillation in an unsupervised FL framework. Despite this progress, object-centris representation learning which aims to decompose scenes into entity-specific features has not yet been explored in federated settings, and no prior work explicitly combines these paradigms to date, despite its potential leverage domain -specfic data.
A.2 Object-Centric Learning
Slot Attention (SA) [38] pioneered the use of iterative attention to decompose scenes into object slots. Subsequent work improved its scalability [56] and incorporated transformers [51], but all assume centralized training on mixed data. ORL [64] demonstrated that exploiting object-level correspondences can improve unsupervised learning on multi-object images. Following the intro- duction of DINOSAUR [52], SA began to be applied to real-world scenarios using foundation models. A growing body of work has since built on this approach to improve performance- by augmenting foundation features [39], adding modality [67, 2]. or by fineituning the foundation models themselves [12]. Recent techniques like contrastive learning objective [41], patch permu- tation augmentation [26] and using language supervision signals [10] have been demonstrated to improve the scene decomposition performance. However, no prior work in federated learning has combined object-centric models with distributed training across heterogeneous datasets. We argue that centralized training of object-centric models struggles with concept entanglement, a challenge that federated learning inherently mitigates by preserving local data coherence.
A.3 Concept Entanglement in Multi-Domain Learning
Concept entanglement represents a significant challenge in the field of multi-domain learning, particularly when training machine learning models on datasets that contain overlapping visual semantics. Early work tackles this problem by explicitly splitting features into shared and private subspaces, as in Domain Separation Networks [4]. Subsequent approaches refine the idea: [37] propose Unified Feature Disentangler that learns domain-invariant codes via adversarial training and [24] maximizes interaction information to carve representations into domain- -speaiii- and domain- agnostic parts. -otentt-ommin methods reduce entanglement without domain labels-foe for example using a sparse adapter for Latent Domain Learning [9]. [32] show that distilling representations from multiple task- and domain-speciffe networks using alignment with small domain-specift adapters can lead to efficient universal representations. Despite these advances, object-centric disentanglement has yet to be explored in any centralized federated setting, leaving gap that our work addresses.
23

---

A.4 Adaptation Layers and Feature Harmonization
Adaptation layers are widely used in transfer learning [49] and multi-task learning [59] to align feature spaces. In federated learning (FL), [8] employs adapters for personalized federated learning. Didolkar et al. [12] investigate the transferabilltty of foundation models object-centric representation learning and introduce fine-tuning framework (FT- (FT-DINOSAUR) for the task of object discovery, achieving strong unsupervised transfer. Crucially, their results suggest that object-centtee learning can be improved by training one domain and fine-tuning on another, but that improvements cannot be achieved merely by scaling up data. Our work goes further by demonstrating how federated learning scale to heterogeneous datasets through cross-client feature harmonization, ensuring global model stability despite diverse local inputs. One technique for adapting foundation models is the application of lightweight feature adapters [16, 70J, which introduce trainable components into the network while keeping most of the model frozen. Other methods include fine-tuning only the bias parameters of pretrained models [6, 68], and learning low-rank adaptations [22]. We also experiment with adaptation modules correlated to masking approach based on frozen foundation models [60]. An advantage of this approach is that multiple foundation models can be used in parallel, without any
query on which one to fine-tune.
A.5 Knowledge Distillation
Knowledge distillation has been used to perform federated learning, for instance, when global student calculated with FedAvg learns for local teacher [71, 61]. In large-scale self-supervised learning, such as DINO [7], self-distillation enables the model to implicitly ensemble knowledge over time, with the teacher network being updated through an exponential moving average (EMA) of the student, akin to Polyak-Ruppert averaging [45]. Inspired by this approach, our framework updates certain modules of the teacher by averaging them with the student, but without relying on a conventional knowledge distillation (KD) loss. This is because slot prediction is inherently cannuutationinvvriinn [38], making direct macher-to-student supervision inapplicable. Instead, synchronize the teacher and student Slot Attention modules through periodic averaging-effecttiveee effectively local FedAvg operation between the two branches. This strategy is also related to co-distillation [1], where student-teacher models distill knowledge from one another to efficiently ensemble knowledge from large-scale, distributed data. Unlike co-distillation, where student and teacher models typically identical, we share only certain modules between the two. Moreover, our framework operates in a fully unsupervised setting.
B Feature adapters
We experimented with three types of adapter. MLP adapter [50], mixteue-off-epperts (MoE) [53, 15], and Attention-based Feature Modulation (AFM) [23]. Extended details on those three adapters listed below:
MLP adpater [50]: a simple MLP that maps F to channels. This treats the concatenated features as one vector per location and learns a single set of weights combine and reduce them.
Mfxture-off-expett [53, 15]: set of E expert projection layers {g(e)) )
(each learned linear projection) corresponding to each foundation. An auxiliary gating network produces spatially varying weights (0)) (u, v) for each foundation model at each location (u, v). The adapted feature is then F Padapi(t (1,, v) = Σe (u, v) g(() (F, u)). This allows the model to dynamically select different combinations of foundation features in different images.
Attention- based Feature Modulation [23]: instead of combining all features, the adapter learns to suppress or amplify channels from F. We implement this as a set of learnable mask parameters {m.) for 1, ..... Ciat applied to F. The mask effectively select which foundation model features to pass through for each location. Following the channel masking,
linear projection is applied to obtain d channel feature F Fddapt
24

---

C Slot Attention Models
Slot Attention [38] is an architecture for learning object-centric representations. It maps a set of N input feature vectors (e.g., image pixels or CNN features) to K slot vectors, each aiming to represent an object in the scene. The K slots are initialized (e.g., randomly) and iteratively updated by an attention mechanism that binds slots to parts of the input. At each iteration t = I...T, attention weights Wa are computed between each input feature i; and each slot 8/ (for i = 1.... .N and k = 1, ..... K). For example, using dot-product attention with learned projections W W for queries/keys, one can write:
(s, (x, (ek (orW
exp
Vi
Wi
(6)
EE= exp
(2.W WK)
vd
which is a normalized attention weight indicating how much slot k attends to input i. Using these weights, each slot is updated by aggregating the inputs:
8 + GRU Sks Σ Was (o:W)))
where WV is a learned value projection and GRU a gating recurrent unit that combines the current slot value with the weighted input. This process is repeated T times, and includes an MLP--ased refinement per iteration [38]. The result is output slot vectors {sk 1= that are exchangeable
peermutation-invariin and ideally each slot specializes to one object or background component.
In unsupervised object discovery, Slot Attention typically trained by reconstructing the original image from the slots. A decoder (e.g.. a CNN, spatial broadcast decoder, or transformer decoder) uses the slots to output K component reconstructions and masks, which are combined match the input image. The model thus learns to partition the scene into objects. However, such purely unsupervised training can struggle on complex real-world images. Thus, recent work of applying slot attention to such images is all based on foundational models [52, 39, 67, 11], and the reconstruction target is feature map instead of image.
D
Additional Details on FORLA
D.1 Two-stage Training and Local-FedAvg
We extend the FORLA framework details in Algorithm 1. The FedAvg between clients is performed every 100 iterations, while the FedAvg between teacher and student is performed every 1000 iterations. We use Adam optimizer with learning rate of 4x10-4, and weight decay of 4x10 For all data the batch size is 16.
We set an empirical switching criteria as 90K iterations, as this number allows the reconstruction loss of slot attention models on clients's 's student branch to converge to plateau. We also found that training for a larger number of iterations before switching from EMA (larger than 90K) can lead to better initial convergence of the student branch, but in the long term this advantage is offset by the second stage training of the student, making the additional time spent initial training obsolete. In future, the empirical threshold can be replaced by a dynamic threshold based on tracking of student loss.
D.2 Foundation Model Specifications
In computer vision community, a significant amount of time and resources have been used train foundation models from large image datasets of everyday scenes and objects (Natural vision domain). We provide specifications of four vision foundation models used in FORLA. These four models were selected because of their complementary capabilities in semantic understanding, segmentation, reconstruction, and multimodal aligmment. Table 14 summarizes key technical specifications, while we elaborate on their architectural and functional characteristics below:
DINO (self-Distillation with NO labels) [7]:
25

---

Architecture: Vision Transformer (ViT-B/16, ViT-B/14 for DINO-v2) ) with 12 layers, 768D embeddings
Pretraining Objective: Self-supervised distillation using image augmentations without labels
Training Data: ImageNet- Ik (1.28M images)
Key Strengths: Captures high-level semantic relationships through global self-attention;
produces spatially consistent feature maps ideal for object discovery
SAM (Segment Anything Model) [28]:
Architecture: ViT-B/16 image encoder with mask decoder
Training Objective: Supervised promptable segmentation on IB+ masks
Training Data: SA-1B dataset (1 IM licensed images)
Key Strengths: Specialized in boundary-awar feature extraction.
MAE (Masked Autoencoder) [20]:
Architecture: Asymmetric ViT-B/16 with 75% patch masking
Pretraining Objective: Self-supervised, pixel reconstruction of masked image regions
Truining Data: ImageNer- Ik (1.28M images)
- Key Strengths: Excels at texture/structure recovery; provides complementary low-level features to DINO's semantics.
CLIP (Contrastive Language-Image Pretraining) [48]]
Architecture: ViT-B/16 image encoder (text branch disabled)
Pretraining Objective: Contrastive alignment of 400M image-text pairs, language is used as superviiooii signal
Training Data: Web-crawled multimodal corpus
Key Strengths: Cross-modal concept grounding; robust to distribution shifts
Our experiments use frozen foundation models without fine-tuning: ViT-B/16 variants of DINO, MAE, and CLIP with 14x14 patch size and 224x224 input resolution. The CLIP text branch is excluded. For SAM, use only the image encoder, downsampling positional embeddings to align spatial resolution with other models; the decoder is omitted as our focus is solely on representation learning. The specific hyperparameters of other modules are included in Table 8. We follow DINOSAUR [52] in setting the slot count for COCO and PASCAL to slots, and for other datasets we use 7 slots. Both teacher and student branches trained with the same number of slots. Using teacher and student with a different number of slots would be interesting to explore in future work.
E Dataset details
Here we extend details of all datasets used in this research:
Abdominal dataset is public dataset from animal, phantom, and simulator abdominal surgeries [72]. We utilize 739260 frames for training, and 3000 frames with segmentation masks evaluation.
Cholec dataset [58] consists of 80 laparoscopic cholecystectomy videos. Following [35], we use 15,000 frames for training. 8,000 frames with segmentation annotations for evaluation.
Thoracic dataset includes data from 40 assisted right upper lobectomies (RULs) for lung cancer, performed at Toronto General Hospital between 2014 and 2023. We use total of 51,900 images for training and 800 manually annotated frames for evaluation." 2
COCO (Common Objects in Context) [36] a widely used benchmark for object detection, segmen- tation, and image captioning, consisting of 80 object categories. We use the 2017 split, with 118,000 images for training and 5,000 for validation.
PASCAL VOC 2012 [13] provides 1 1,530 images with segmentation masks for 20 object categories. Following standard protocol, we use 10,582 images for training and 1,449 for validation.
2 An open-access version of this proprietary dataset is integrated as part of the federated learning benchmark in this work. See: https:// :/gs:///itta githhb..e. com/PCASOLaa/FOOLLA
26

---

Table 8: Hyperparameters of different network components.
Name
Type
Model file size
Feature dim
DINO
ViT-B/16
346 MB
768
SAM
VIT-B/16
375 MB
256
Foundation models
MAE
ViT-B/16
327 MB
768
Cimge encoder)
VIT-B/16
437 MB
768
Type
Input dim
Model file size
Feature dim
MLP
2560
56MB
256
Adapters
MoE
2560
107MB
256
AFM
2560
158MB
256
Slot num
Slot
Model file size
Itrratonn
Slot attention
6/7
256
2.3 MB
3
MI.P hidden dim
Input dim
Model file size
Output dim
Teacher decoder
1024
256
13M
262 / 263
Student decoder
1024
256
23M
2566 / 2567
YTVIS (YouTube-VIS) [66] is benchmark for video instance segmentation, containing 8,858 videos spanning 40 object categories, with pixel-level masks across frames. We train on the 2,985-vide0 training split from the 2021 version (78,810 frames) and evaluate 4,210 validation frames.
YTOBJ (YouTube-On Ybbeett-Objectt)) [47] consists of 126 YouTube videos across object categories, annotated with sparse bounding boxes for object tracking. We extract 388,050 frames from 100 videos for training and evaluate on 9,000 frames from 26 held-out videos.
F
Additional Implementation Details
F.1
Additional Details on Metrics
We evaluate our approach based on the quality of the slot attention masks using four primary metries: Foreground Adjusted Rand Index (FG-ARI) [17], Mean Best Overlap (mBO) [46]. Mean Best Hausdorff Distance (mBHD), and CorLoc [3]. FG-ARI, widely adopted in object-centric research, measures the similarity between predicted object masks and ground truth segmentation, specifically focusing on foreground regions. mBO evaluates the overlap between predicted and ground truth masks using intersection-ovee -overuioonover-uiioon (IoU). computes the average IoU after Hungarian matching between each ground truth and its best-matching predicted mask. While FG-ARI emphasizes segmentation quality regardless the permutation, mBO offers broader assessment by channel matched IoU. To further assess spatial precision, we compute mBHD, which captures boundary-level accuracy and is particularly important for clinical applications. CorLoc measures localization accuracy by counting predicted object instances whose bounding boxes achieve IoU > 0.5 with ground truth object.
F.2
Additional Details on Experiment setup
Our experiments are conducted under three training regimes: (i) individual training, where each dataset is trained independently; (ii) centralized mixed training, where data from multiple datasets (surgical or natural) are pooled; and (iii) federated training, where each dataset treated as separate client without data sharing.
As each dataset can be considered as residing on a single client, we have a maximum of seven clients corresponding to seven distinct sub-domains. All images are resized as 224x224 for input (14x14 patches after ViT-B/16). For SAM, only the image encoder is used; positional embeddings are down-samples to match the spatial resolution of the other models, and the decoder is omitted since representation learning is our focus. All adapter variants reduce the input feature dimensionality to 256 and use a slot embedding size of 256, following common practice in recent slot attention models [52, 62]. Each federated client is trained for minimum of 100 epochs, with early stopping triggered if the student's reconstruction loss does not improve over 30 consecutive epochs. This stopping criterion was consistently met, and no late convergence was observed. The teacher's reconstruction loss is not used for carly stopping, it tracks dynamic and more fluctuating target. Individual and
27

---

Table 9: Performance on surgical data using single foundation model's ' raw features for slot attention.
Abdominal
Cholec
Thoracic
Model
mBO
mHDL
FG-ARI
CorLoc
mBO
mHD
FG-ARI
CorLoc
mBO
mHD]
FG-ARI
CorLoc
DINO
47.33
51.497
57.87
52.38
28.7
62.13
38,9
30.4
30.96
112.324
19.3
30.45
SAM
47.0
55.84
53.7
56.2
25.75
79.43
32.62
31.81
50.28
71.94
41.14
54.88
MAE
35.0
71.17
42.6
34.0
14.37
107.21
19.23
18.7
35.08
100.57
36.35
26.53
CLIP
75.14
10.9
97.26
14.99
117.26
8.71
11.75
centralized training baselines are run for 130 epochs. Our experiments were conducted using four NVIDIA RTX 6000 GPUs, with some GPUs assigned multiple clients. Each running client consumes approximately 6 GB of GPU memory after feature caching. Feature caching accelerates training by factor of 10-20 for each client. complete run of federated learning across all mixed sub-domains takes approximately 12 hours.
Centralized training 1.4 million data points (frames) across datasets with 130 epochs and batch size 16 takes 21.6 hours which is 1.8 times slower than FL. As the data can not be distributed across too many clients since unsupervised object centric learning needs a certain amount of data to discover meaningful concept, it would indeed be interesting to explore in more detail how one can achieve an optimal ppeecupp/erformmacce ratio FL. training regime.
For the Natural image datasets we used optimal slot counts reported in the literature [11, 52]. For the surgical datasets tuned the slot count to achieve best score. We've also used some evidence from training SA models with both types of data [34, 33]. Choosing an optimal slot number is indeed important for slot attention algorithms. Our method is also fully compatible with SA approaches that use an adaptive slot count, but we decided to use more traditional approaches in this work for easier evaluation.
G Additional results
In this section, we extend experiments on using single or different combination of frozen/dapted foundation models, distillation dynamics on different dataset using different adapters, evaluation the performance of teacher branch (student reported in the main paper), Comparison on Zero-shot transfer from natural surgical domain, FORLA inference on videos when compared to DINO and SAM backbone, and more qualitative demonstration of feature adaptation and slot attention on different domains.
G.1 Comprehensive Foundation Model Benchmarking
Tables 9 and 10 provide detailed benchmark of four foundation models across surgical and natural domains. In this experiment we use only frozen features and no adapter or tin-ttnnngg applied. Each model is train on single dataset individually. Three key insights emerge: 1) Specialization- utility tradeoff:: DINO's supervueee features excel on Cholec instruments (28.7 mBO) and COCO (23.96 mBO), indicating strong general object semantics. SAM with its segmentation- focused pretrained features achieves 50.28 mBO on Thoracic data (Table 9), outperforming DINO's 30.96 mBO, demonstrating out of domain generalization advantages. Reconstruction- based MAE underperforms on surgical data (-17.2 mBO vs. SAM) but shows unexpected competence on YTVIS (20.2 mBO) which could due YTVIS requiring less high-level semantic features. 2) Modality and domain mismatch: CLIP's text-image alignment provides limited value for surgical domains (14.99 mBO Thoracic), suggesting medical imaging diverges from its web-scale pretraining. MAE trained on natural domain also transfers poorly to surgical scenes. 3) Complementary Strengths: SAM achieves highest CorLoc (56.2) while DINO leads in FG-ARI (57.87) Abdominal data, and no single model dominates all metric on all data.
This analysis confirms that foundation models exhibit specialized capabilities aligned with their pre- training objectives and training data. FORLA's feature integration strategy (Eq. 1) allows synergistic combination of these complementary representations.
28

---

Table 10: Performance on natural data using single foundation model's 'ss raw features for slot attention.
COCO
PASCAL
YTVIS
YTOBI
Model
mB0
mttD 1
FG-ARI
CorLoc
mB0
milD 1
FG-ARI
CorL.ve
mBO
mttDD L
FG-ARI
CorLoc
mBO
CorLoov
DINO
23.96
81.929
29.2
20.13
34.74
78.853
35.17
52.31
33,09
68.238
36,64
54.02
12.77
51.78
SAM
22.61
93.192
25.02
20.5
36.98
73.992
35,59
56,04
52.62
68.704
35.89
53.66
42.86
54.95
MAE
17.08
25.95
91.41
22.93
31.87
20.81
15.94
33.86
29.91
CLI
12.73
109.433
14.14
3.46
14.52
109.054
9.82
27.47
14.37
107.217
19.23
23.15
22.27
9.02
Table 11: Adaptation of features from a single foundation model on Abdominal, YTVIS and YTOBJ.
Adoapier
Model
Abdominal
YTVIS
mBO
mHD]
FG-ARI
CoeLooc
mBO
mHD]
FG-ARI
CorLoc
mBO
CorLoc
Concat
51.85
43.025
59.04
09:25
35.16
67.22
38.25
58.08
45. 48.62
60.31
MLP
DINO
45.35
53,633
50.55
63.6
36.42
66.584
40.91
62.13
46.10
60.02
SAM
35.93
64.650
MAE
40.84
59.286
47.28
52.3
21.94
83.329
22.97
20.67
33.52
26.83
Concat
30:34
44 308
59.04
20.73
53.94
66.637
$7.54
$4.59
48 18.98
65.11
AFM
DINO
41.99
63.738
47.46
56.73
36.52
65.752
41.01
61.18
48.17
62,00
SAM
41.73
66.985
46.05
56.65
36.06
64,939
64,04
51.35
MAE
42.85
59.433
54.37
21.20
22.66
19.36
34.87
27.58
Table 12: Evaluation of Frozen and Adapted models using 3 vs. 4 foundation models. FM: Foundation Model, Surgical:Abdominn Natural: YTOBJ.
Surgical
Natural
Setting
FM number
mBO
RG-ART
Cor-Loc
mBO
PG-ART
Cor-Loc
39,78
51.28
54.26
42.92
33.74
$1.85
Frozen
48.65
54.07
64,30
43.33
38.78
50.56
Adapted
58.52
69.58
47.32
4
50.34
59.04
70.73
48.98
45.32
65.11
G.2 Efficacy of Adaptation Layer for Single Foundation Models
Next, to assess the benefit of feature adaptation, we evaluate the performance when using a single foundation model augmented with lightweight adapter module. In these experiments, we attach either a simple MLP adapter or the proposed AFM adapter on top of the frozen foundation model, then train the adapter (keeping the foundation model weights fixed) for the downstream object- discovery. Here MOE is not used as it is only applicable for multiple foundation models. Table 11 (middle) reveals the performance:All foundation model show improvement with both adapter YTVIS data, particularly AFM boost SAM's CorLoc from 53.66 to 64.65. Adapters recover 54% of MAE'S performance gap SAM on abdominal data (35.0 42.8 mBO). Importantly, the performance of foundation models when used individually lags the performance of concatenated adapted baselines, in particular when applied to domain that is new foundation models (+5 mBO, + 9 FG-ARI, +9 Cor-Loc).
G.3 Using Different Numbers of Foundation Models
We further examine the impact of using different numbers of foundation models within FORLA. When employing four foundation models (including ViT-B/16, the smaller ViT variant), the computational overhead increases modestly from 1.4 ms (using only DINO) to 3.8 ms per image. Considering recent sumain-specific foundation models such as RADIOv2.5 [21], J, one may question whether fewer models could provide sufficient representational power, particularly in the surgical domain.
To investigate this, we evaluated configurations using either three (DINO, SAM, and CLIP) or four foundation models, under both frozen and adapted settings. As shown in Table 12, the configuration with four foundation models consistently achieved the best performance, especially on surgical images, which are typically underrepresented in the pretraining of generic foundation models. Despite the modest increase in computational cost (an additional 2.4 ms per image), the performance gains justify the use of four foundation models for achieving stronger generalization and robustness.
29

---

Table 13: Additional results of self-distillation on the single data with MLP and MOE adapters.
Abdominal
Thoracic
Adapter
Method
mBO
mHD]
FG-ARI
Corl.nc
mB0
mHD)
RG-ARI
Corl .nc
self dicill
59.04
54.00
72.358
43.59
58.00
MLP
self self-distill
56.06
35.005
63.09
79.23
44.86
85.707
36,73
39.28
w/o self ditil
51.83
42.758
57.54
72.02
$1.10
74.670
41.87
55.89
MOE
self-distill
57.05
35.039
63.27
79.62
62.16
52.544
47.96
72.32
Table 14: Decoder performance of the teacher branch in comparison student branch in FORLA.
mBO
mHD
FG-ARI
CorLoc
Domain
Sub-domain
Teacher
Stident
Teacher
Student
Teacher
Student
Teacher
Student
Abdominal
51.29
58.31
77.73
Surgical
Cholec
32.52
34,20
57.247
56, 66.942
42.31
44.35
52.13
54.49
Thoracie
56.86
61.86
43.60
76.04
75.42
Averge
46,89
51.31
49.469
47,36
48.07
52.27
68.63
70.07
COCO
26.72
27.22
93:473
93.541
28.41
30.11
66.88
64.94
PASCAL
39.36
40.83
76.277
75.642
36.41
39.51
65.38
64.84
Natural
YTVIS
37.96
38.51
61.396
62.472
41.63
43.08
65.50
64.92
YTOBJ
51.06
51.92
62.03
66.87
Average
38.78
39.62
77.05
77.22
35.48
37.57
64.95
65.39
G.4
Distillation Dynamics
Table 13 (bottom) shows additional effects of self-distillation on abdominal and thoracic data with MLP and MOE adapter: 1) MLP overfitting: Sclf-distillation improves Abdominal CorLoc (im- proved performance score by 10) but harms Thoracic performance (-8.7 points performance), suggest- ing MLP could over-fit to smaller specialized dataset with less constrains for feature reconstruction. 2) MOE Robustness: MOE adapters gain +16.4 CorLoc on Thoracic with distillation, leveraging expert gates preserve generalizability. This once again confirmed that MLP could be the least suitable adapter choice in our FORLA federated object-centric learning framework, as demonstrated in Table 2.
G.5 Teacher Decoder Analysis
In FORLA, teacher and student branch share the same global adapter and Slot attention module, while having different decoders for reconstructing features and slot attention masks. We analyzed the performance the decoder branches by comparing the teacher and student models across all datasets. As shown in Table 14, the teacher branch performance is consistently competitive and often close to the performance of the student decoder, despite the fact that its adapter is not directly optimized but rather updated through EMA (carly stage) or local FedAvg (later stage).
Table 14 reveals nuanced performance differences between teacher and student decoders in federated learning: 1) Dominance on surgical domain: Student decoder achieves +4.42 average mBO improvement (51.31 vs. 46.89), demonstrating superior object discovery from federated knowledge aggregation; it maintains boundary precision with 4.7% lower mHD (47.36 vs. 49.47), crucial for anatomical structures; 2.44 FG-ARI gain highlights better breerrunndbbccgrooun
separation. 2) Gains on natural domain: Student leads marginally in mBO (+0.84) FG-ARI (+2.09), and CorLoc (0.44). This validates the effectiveness our teacher-student design, where the student benefits from both local and global knowledge transfer via FL and reconstruct more constrained features, while the teacher adapts more aggressively and encourages the student to re-discover more specialized and transferable object-centric representations.
G.6 Compared to Zero-shot transfer performance of slot attention
We performed additional testing that confirmed our hypothesis which was that, if the domain gap is small, the zero-shot and transfer learning will guarantee good performance. However, FORLA FL will outperform transfer learning when the domain gap large (Table 15). We first tested zero- shot transfer of slot attention trained on Natural images (PASCAL and YTOBJ) to surgical images (Abdominal). In this case zeroshot performance is significantly lower compared to FORLA and even compared to models individually trained on abdominal data.
30

---

Table 15: Comparison of zero-shot transfer from Natural to surgical, individual training, and FORLA.
Method
mBO
FG-ARI
Cor-Loc
Zero-shot (PASCAL - Abdominal)
35.61
39.81
35.90
Zero-shot (YTOBJ + Ahdominal)
33.02
37.69
30.65
Individual
50.34
59.04
70.73
FORLA
57.86
64.88
80.30
G.7 Inference on Videos
We demonstrate that slot attention module trained with our FORLA framework can be directly applied to video scene decomposition by leveraging RNN-based slot inference techniques [67]. Specifically, evaluated our FORLA-trained slot attention model the YTOBJ video dataset sampled at 1 frame per second (fps). For comparison, we trained individualized slot attention (SA) models directly YTOBJ using adapted versions of foundation models, including DINO (equivalent to DINOSAUR (52]) and SAM.
As shown in Figures 4 and 5, the slots produced by FORLA maintain strong temporal consistency across different video sequences. The performance of individualized models reveals that SAM- and DINO-based SA models exhibit distinct strengths: the SAM-based model excels at decomposing close-up scenes such as animals or vehicles captured by a near camera, while the DINO-based model performs better in delineating objects from the background in distant wide-angle scenes.
FORLA however consistently tracks objects both near and far scenes in a more fine- mne-grained and semantically meaningful manner. Unlike the SAM or DINO-based models, FORLA is less likely segment objects into implausible parts (e.g.. splitting a cat or car into non-semantic regions). demonstrating stronger object-level coherence and generalization.
H
Visualization of Feature Representation
Figures 6 and 7 provide additional visualizations PCA maps and Slot Attention (SA) masks for surgical and natural image domains, respectively, across different methods. We visualize the first three principal components of the feature representations using RGB channels. These include both frozen features from foundation models and adapted features learned through our FORLA federated learning framework.
In the surgical domain, the frozen foundation model features demonstrate a general understanding of texture-based separation-for example, surgical instruments and tissues often appear as different colors in the PCA map. This indicates some level of semantic separation. However, the foundation features struggle to distinguish between multiple instances similar instruments, as they are often represented with the same color. In contrast, the FORLA- adapted features effectively assign distinct semantic representations to different instrument instances, enabling clearer separation. Additionally, FORLA representations offer greater differentiation between various tissue textures. This is reflected in the PCA maps, where different tissues are represented by homogenous yet distinct colors, aiding the SA module in grouping regions into semantically meaningful categories.
In the natural image domain, foundation model features can delineate some salient objects, due to their pretraining natural image datasets, but the representations remain relatively coarse. With FORLA's unsupervised adaptation, the feature representations become significantly more fine-grained. For example, tiny objects that are otherwise overlooked in foundation features become clearly highlighted in the PCA maps. The model also learns to extract subtle cues from cluttered backgrounds, such the texture of plants, and can distinguish between visually similar objects located close to each other. These propertics the adapted representation enable the downstream SA module to decompose scenes into semantically coherent segments.
Such fine- fine-grained representation and semantically meaningful decomposition suggests that FORLA- learned representations can potentially generalize well to downstream tasks, including semi- supervised or weakly supervised segmentation, as well as variety of prediction tasks that benefit from pre-segmented scene understanding [54].
31

---

Input
Adapted SAM
Adapted DINO
FORLA
Input
Adapted 5AM
atapted DINO
FORLA
Input
Adapted
SAM
Adapted
DINO
FORLA
Figure 4: Inference using RNN like slot initialization [67] on YTOBJ videos. We compared to individualized trained SA models on YTOBJ using adapted single foundation model including DINO (as used in DINOSAUR (52]) and SAM.
32

---

Input
Adapted SAM
Adapted DINO
FORLA
Input
Adapted SAM
Adapted DINO
FORLA
Input
Adapted
SAM
Adapted
DINO
FORLA
Figure 5: Additional results on YTOBJ videos compared individualized trained SA models with single foundation model adaptation.
33

---

PCA
PCA
Input
Foundation
FORLA
SA mask
Input
Foundation
FORLA
SA mask
Abdominal
Cholec
Thoracic
Figure PCA map of raw foundation model features and FORLA representation on surgical image
samples. Slot attention (SA) masks of FORLA shown 4th and 8th columns.
34

---

PCA
PCA
Input
Foundation
FORLA
mask
Input
Foundation
FORLA
SA mask
Figure 7: PCA map of raw foundation model features and FORLA representation on natural image samples. Slot attention (SA) masks of FORLA are shown 4th and 8th columns.
35


Reviews:
[{"review_id": "oy9OJExy1o", "reviewer": "Reviewer_TTbE", "summary": "This paper proposes FORLA, a method for federated representation learning with foundation models. The key components of FORLA include a shared adapter to reduce the dimensionality of features extracted from FMs, along with a slot attention mechanism to learn object-centric representations. These parameters are trained in a two-stage distillation process using a student-teacher architecture. Notably, the proposed federated solution exceeds the performance of centralized training on multiple domains.", "questions": "1. The ‘Slot FedAvg’ baseline seems quite competitive with FORLA, with only ‘FORLA+AFM’ beating ‘Slot FedAvg + MoE’. Could you provide additional context on the comparison here - why does the student-teacher distillation framework lend itself so well to the AFM adapter? \n2. Is this ‘Slot FedAvg’ baseline equivalent to a ‘DINOSAUR FedAvg’ strategy, when we only use features extracted from DINO?\n3. The observation that the federated solution can outperform the centralized one is quite interesting. Does this still occur when the number of clients exceeds the number of datasets (and each user correspondingly holds less data)? Further investigation into this phenomenon would be helpful. \n4. Have you conducted any ablation studies on the quantity of data available and the optimal feature stack + adapter methods? It would be interesting to see if the AFM still obtains the best results in low-sample settings, and if it is still best to use features from multiple pretrained models.", "limitations": "I like the idea of this paper, but I have some concerns regarding the lack of results in more distributed settings (see weakness #4), and I would like to see a clearer discussion of the key results.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n1. The paper is well-written and tackles an interesting problem of domain-invariant federated representation learning. \n2. The connection of slot attention's success in mixed domains and federated learning is quite interesting.\n\nWeaknesses\n1. Some details of the algorithm are difficult to find in the main text and require going into the supplementary material, for example, the description of the two-stage training in Section 3.4.  It would be helpful to bring a more detailed algorithm description to the main text and reserve parts of Section 4 for the appendix instead. \n2. The presentation and discussion of the results require some clarification. The authors provide some guiding research questions, but the corresponding discussion of the experiments is difficult to parse. The definition of baseline methods (e.g., Slot FedAvg in Table 2) should also be made more prominent in the text. \n3. While the selected baselines make sense, it would be helpful to see a comparison to other recent FL techniques that leverage foundation models, if possible.\n4. Some details of the experimental setup are missing and may require some ablation - e.g., the amount of training/validation data on each client. The current results are also limited to 1 client per dataset.", "was_revised": true, "final_justification": "I appreciate the authors’ efforts during the rebuttal. My concerns regarding the clarity and discussion of the results were addressed, and the additional experiments in more distributed and low-sample settings helped validate the proposed method. In the final version, I encourage the authors to benchmark the full set of baseline methods (e.g., naive Slot-FedAvg) in these added ablations.", "rebuttal": "We thank the reviewer for recognizing the novelty of connecting slot attention’s success in mixed domains with federated learning, and for appreciating our approach to the timely problem of domain-invariant federated representation learning. We will clarify the algorithmic details in the main text, improve the presentation of baselines, and provide additional ablations and context to address the insightful questions raised. Below is our response to specific questions: \n\n---\n\n**Q:** *Some details of the algorithm are difficult to find in the main text and require going into the supplementary material, for example, the description of the two-stage training in Section 3.4. It would be helpful to bring a more detailed algorithm description to the main text and reserve parts of Section 4 for the appendix instead.*\n\n**A:** We agree with this proposal to move  details on the two-stage training, including Algorithm 1, from the appendix to the main text. We will move some details on selection of feature adaption modules and parts of datasets description to appendix.\n\n---\n\n**Q:** *The presentation and discussion of the results require some clarification. The authors provide some guiding research questions, but the corresponding discussion of the experiments is difficult to parse. The definition of baseline methods (e.g., Slot FedAvg in Table 2) should also be made more prominent in the text. Is this 'Slot FedAvg' baseline equivalent to a 'DINOSAUR FedAvg' strategy, when we only use features extracted from DINO?*\n\n**A:** The \"Slot FedAvg\" baseline differs from the DINOSAUR + FedAvg in that it uses a feature stack and feature adaptation. Because of that, the reconstruction target is the stacked feature. If the baseline had DINO only and a MLP adapter, they would be equivalent. We will improve the description of this and other baselines (currently paragraph starting at line 204) in the revised version.\n\n---\n\n**Q:** *The 'Slot FedAvg' baseline seems quite competitive with FORLA, with only 'FORLA+AFM' beating 'Slot FedAvg + MoE'. Could you provide additional context on the comparison here - why does the student-teacher distillation framework lend itself so well to the AFM adapter?*\n\n**A:** We thank the reviewer for pointing out this important performance result that we did not elaborate on in the initial submission. While it is true that Slot FedAvg performs well in comparison to FORLA, its performance gains are mostly on Natural datasets while it lags in performance on Surgical datasets. The reason for this is that foundation models are trained on Natural datasets and therefore teacher supervision and feature adaptation are less important for Natural datasets in comparison to Surgical datasets. We think this gap in performance further highlights the flexibility of cross-domain training using FORLA and we'll thus include a stratified average column to Table 2, together with an explanation in the main text.\n\nAs for the reason why MOE appears to be a better choice for fedAvg while AFM works better with FORLA's student-teacher distillation: The AFM is much more adaptive and can easily overfit to certain concepts with FedAvg, while with the student-teacher constrained learning in FORLA, AFM's high adaptivity will not hinder scaling to more data.\n\n---\n\n**Q:** *The observation that the federated solution can outperform the centralized one is quite interesting. Does this still occur when the number of clients exceeds the number of datasets (and each user correspondingly holds less data)? Further investigation into this phenomenon would be helpful.*\n\n**A:** Thank you for this question that made us consider new experiments that show how exploiting structural coherence in local datasets fades when data is distributed. We conducted an additional experiment with 2 clients for each dataset (50% percent of dataset on each) using Abdominal and Thoracic datasets. Indeed, performance is lower in such a setting but federated learning is still able to surpass the centralized learning. \n\n| Region     | Method      | mBO   | HD     | FG-ARI | Cor-Loc |\n|------------|-------------|-------|--------|--------|---------|\n| Abdominal  | Centralized | 54.13 | 41.79  | 60.00  | 74.70   |\n| Abdominal  | FORLA       | 57.30 | 40.07  | 63.30  | 77.08   |\n| Thoracic   | Centralized | 54.52 | 63.82  | 44.37  | 62.67   |\n| Thoracic   | FORLA       | 61.55 | 53.78  | 47.84  | 71.53   |\n\n*Table: Comparison of methods (Centralize and FORLA) across Abdominal and Thoracic using 50% (2 clients each)*\n\nWe also performed an experiment with smaller datasets distributed across four clients. First we used all data for each dataset (25% on each client), and then decreased the amount of data to 40% (10% on each client, see also our answer to request for data quantity ablation). Again, we compare Individual (each client learns own model) to Centralized (all data mixed) and FORLA. We can confirm that the advantage of FORLA slowly fades with less data distributed over nodes, indicating that FL might overfit to spurious structural aspects of smaller training batches, highlighting the need for distributed datasets of same domain to have similar distributions.  \n\n| Method      | Pascal (25%) |        | Thoracic (25%) |         | Pascal (10%) |        | Thoracic (10%) |         |\n|-------------|--------------|--------|----------------|---------|--------------|--------|----------------|---------|\n|             | mBO          | FG-ARI | mBO            | FG-ARI  | mBO          | FG-ARI | mBO            | FG-ARI  |\n| Individual  | 33.21        | 31.52  | 46.39          | 39.79   | 30.71        | 28.11  | 27.14          | 23.35   |\n| Centralized | 34.98        | **34.16** | 53.03        | 43.48   | **34.99**    | **33.52** | 49.17        | **40.14** |\n| FORLA       | **35.25**    | 34.14  | **54.26**      | **44.30** | 32.09        | 29.25  | **49.66**      | 39.21   |\n\n*Table: Comparison of mBO and FG-ARI across training methods under 25% and 10% data splits for Pascal and Thoracic datasets.*\n\nSince unsupervised  object centric learning needs a certain amount of data to discover meaningful concept, these additional results are consistent with OCL and indicate that our framework requires the data not to be over partitioned.\n\n---\n\n**Q:** *While the selected baselines make sense, it would be helpful to see a comparison to other recent FL techniques that leverage foundation models, if possible.*\n\n**A:** Federated foundation model research is mostly on LLMs, for instance \"Towards Federated Foundation Models: Scalable Dataset Pipelines for Group-Structured Learning\". The only vision FL model to our knowledge is \"Evaluating Federated DINO's performance on the segmentation task across diverse domains\"(Marko Harasic, et al., 2024) which is not object centric and thus could not be quantitatively compared. We will mention this in the discussion/related work.\n\n---\n\n**Q:** *Some details of the experimental setup are missing and may require some ablation - e.g., the amount of training/validation data on each client. The current results are also limited to 1 client per dataset. Have you conducted any ablation studies on the quantity of data available and the optimal feature stack + adapter methods? It would be interesting to see if the AFM still obtains the best results in low-sample settings, and if it is still best to use features from multiple pretrained models.*\n\n**A:** In addition to our experiment with 50% of  database per client (see answer above), the experiment with 10% of database per client (40% total data) was  designed to partially answer this question. In this setup, FL matched the performance of centralized learning while still outperforming individual training, albeit with diminished returns.\n\nIn addition, we previously experimented using only three foundation models (DINO,SAM,CLIP). Both without adaptation (39.78 mBO on abdominal and 42.92 mBO on YTOBJ) and with adaptation (49.58 mBO on abdominal and 47.47 mBO on YTOBJ) results were suboptimal to using four foundation models. \n\nAs for the contribution of AFM and feature adaptation in low data regimes, we hypothesize that they would become less stable with increased data heterogeneity and decreased dataset size. We already tested the use of a single foundation model with/without adaptation in the Appendix and the results support our hypothesis. We'll definitely consider a more extensive testing of AFM and other adapters in low data regimes for future work."}, {"review_id": "3liInzAnVk", "reviewer": "Reviewer_bGQ6", "summary": "This paper proposes FORLA, a novel framework for Federated Object-Centric Representation Learning with Slot Attention that couples object-centric structure learning with collaborative feature adaptation across clients. FORLA mitigates conceptual entanglement by preserving local structure—a key challenge in multi-domain learning with semantic overlap. Moreover, while traditional federated learning has primarily focused on classification and segmentation tasks, this work introduces self-supervised object-centric learning into the FL setting, filling an important gap in current research.", "questions": "The current design uses a dual-branch architecture, where the student reconstructs raw features and the teacher reconstructs adapted features. However, the motivation behind this specific design choice is not fully explained. A clearer explanation would improve the overall clarity and justification of the method and may positively influence the evaluation.\n\nFederated learning often accelerates the training process due to parallel updates across multiple clients. While the paper reports the overall training time of the federated setup, it is unclear how this compares to centralized training in terms of speedup. Could the authors report the actual speedup ratio relative to centralized training and clarify whether FORLA achieves any acceleration benefits in practice?\n\nIn the early stages of training, FORLA updates the teacher adapter using the student via EMA. However, it remains unclear how the update boundary between EMA-based updates and gradient-based updates is defined. Could the authors elaborate on how the two update mechanisms are coordinated and whether any specific schedule or switching criteria is applied?\n\nWhile FORLA demonstrates strong performance in the reported experiments, real-world data distributions are often more complex and non-stationary. How sensitive is FORLA to different degrees of data heterogeneity? Specifically, would randomly reshuffling the data partitions under varying heterogeneity levels significantly affect its performance?", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Quality.\n\nThe paper presents a technically sound and well-executed framework that integrates self-supervised object-centric learning into the FL paradigm. FORLA combines Slot Attention with a dual-branch student-teacher architecture and introduces several thoughtful design choices, such as the use of EMA and cross-branch FedAvg, to enable mutual knowledge transfer and reduce conceptual entanglement. The experimental section is comprehensive, with detailed implementation descriptions and reproducibility considerations. Extensive evaluations across multiple datasets and metrics demonstrate consistent improvements over both individual and centralized baselines. However, some architectural decisions—such as the motivation for dual reconstruction targets and the necessity of cross-branch federation—would benefit from deeper theoretical or empirical justification. In real-world scenarios, federated systems often involve a large number of clients and varying degrees of data heterogeneity. The current experiments are limited in client scale and do not assess the framework’s performance under more complex, realistic distributions, which raises concerns about its practical applicability.\n\nClarity.\n\nThe paper is generally clear and well-organized. A more in-depth explanation of how these components contribute to the mitigation of conceptual entanglement would enhance the overall clarity and persuasiveness of the work.\n\nSignificance.\n\nThe paper addresses a timely and important challenge in multi-domain learning with overlapping semantics. By introducing object-centric self-supervised learning into the FL context, the authors tackle a largely unexplored area and open up new directions for future research. The demonstrated performance gains across various benchmarks underscore the practical impact of the method.\n\nOriginality.\n\nThe work is original in both its problem formulation and methodological design. While the components employed are known individually, their integration into a federated setting for object-centric representation learning is novel. The perspective of addressing conceptual entanglement through collaborative object-level learning and cross-client feature adaptation is fresh and contributes meaningfully to the FL literature.", "was_revised": true, "final_justification": "My concerns have almost been addressed, i will raise my score.", "rebuttal": "We are grateful for the reviewer’s appreciation of the challenge we are addressing and for the acknowledgment that our integration of object-centric learning with federated learning is novel and meaningful for the FL literature. In response, we will elaborate on the dual-branch architecture and cross-branch mechanisms, report training efficiency, and provide additional analysis to address the questions regarding sensitivity to heterogeneity and update strategies. In particular, new experiments with heterogeneous datasets further support our hypothesis that FL lends naturally to unified representation learning across domains by exploiting the local structural coherence present in homogeneous datasets.  \n\n---\n\n**Q:** *The current design uses a dual-branch architecture, where the student reconstructs raw features and the teacher reconstructs adapted features. However, the motivation behind this specific design choice is not fully explained. A clearer explanation would improve the overall clarity and justification of the method and may positively influence the evaluation.*\n\n**A:** Thank you for this comment. Our justifications are:  1) the two branch design allows to learn adaptive features which are useful in delineating objects in complex scenes and novel domains(as shown by the PCA of adapted feature versus raw feature in Fig. 2 ). 2) the reason for one branch to reconstruct raw features while the other reconstructs dynamic features is that reconstructing raw features anchors the learning target which makes convergence possible. Reconstructing dynamic features solely would indeed not converge. We will include this motivation to the manuscript. \n\n---\n\n**Q:** *Federated learning often accelerates the training process due to parallel updates across multiple clients. While the paper reports the overall training time of the federated setup, it is unclear how this compares to centralized training in terms of speedup. Could the authors report the actual speedup ratio relative to centralized training and clarify whether FORLA achieves any acceleration benefits in practice?*\n\n**A:** Centralized training on 1.4 million data points (frames) across 7 datasets with 130 epochs and batch size 16 takes 21.6 hours which is 1.8 times slower than FL. As the data can not be distributed across too many clients since unsupervised  object centric learning needs a certain amount of data to discover meaningful concept (please also see the answer and results to Reviewer TTbE's comments on data re-partition), it would indeed be interesting to explore in more detail how one can achieve an optimal speedup/performance ratio in a FL training regime.  \n\n---\n\n**Q:** *In the early stages of training, FORLA updates the teacher adapter using the student via EMA. However, it remains unclear how the update boundary between EMA-based updates and gradient-based updates is defined. Could the authors elaborate on how the two update mechanisms are coordinated and whether any specific schedule or switching criteria is applied?*\n\n**A:** We set an empirical switching criteria as 90K iterations, as this number allows the reconstruction loss of slot attention models on clients's student branch to converge to plateau. We also found that training for a larger number of iterations before switching from EMA (larger than 90K) can lead to better initial convergence of the student branch, but in the long term this advantage is offset by the second stage training of the the student, making the additional time spent on initial training obsolete. We will add this detail to our experiments setup. In future, the empirical threshold can be replaced by a dynamic threshold based on tracking of student loss. \n\n---\n\n**Q:** *While FORLA demonstrates strong performance in the reported experiments, real-world data distributions are often more complex and non-stationary. How sensitive is FORLA to different degrees of data heterogeneity? Specifically, would randomly reshuffling the data partitions under varying heterogeneity levels significantly affect its performance?*\n\n**A:** Thank you for this question. We ran a set of new experiments with data reshuffling and stratification. Based on data stratification in Table 4 of the manuscript, we reshuffle the datasets so that a group of clients (clients count smaller than 7) has random selection from all 7 datasets. We demonstrate experiments under 3 different combinations of datasets. Results for the first two data combinations with four clients show very similar results (when compared to centralized learning) as the initial results on 7 clients. This means that FORLA is quite robust to moderate changes in data heterogeneity. The third data combination used just two clients and here FORLA showed less of an advantage as it could not exploit learning from multiple subdomains.  FORLA's robustness is driven by SA gradually discovering disentangled concepts  across clients. \nA plain mixed centralized training will remove such disentanglement. In centralized training, this problem could be addressed by techniques in supervised learning by leveraging meta data source labels to group training data in batch schedules, but the FL offers an elegant and efficient solution. \n\n| Metric | Method      | Abd (Comb1) | Thor (Comb1) | PASCAL (Comb1) | YTOBJ (Comb1) | Chol (Comb2) | Abd (Comb2) | YTOBJ (Comb2) | YTVIS (Comb2) | Abd (Comb3) | PASCAL (Comb3) |\n|--------|-------------|-------------|--------------|--------------|---------------|--------------|-------------|---------------|---------------|-------------|--------------|\n| mBO    | Individual  | 50.34       | 53.03        | 34.98        | 48.98         | 33.54        | 50.34       | 48.98         | 35.16         | 50.34       | 34.98        |\n|        | Centralized | 53.42       | 50.89        | 35.97        | 45.28         | 31.64        | 54.16       | 47.84         | 37.15         | 56.56       | 36.83        |\n|        | FORLA       | **55.16**   | **57.67**    | **38.27**    | **50.66**     | **34.86**    | **59.63**   | **48.45**     | **38.79**     | **56.84**   | **37.68**    |\n| Corloc | Individual  | 70.73       | 56.62        | 51.10        | 65.11         | 42.36        | 70.73       | 60.31         | 58.08         | 48.98       | 51.10        |\n|        | Centralized | 73.57       | 53.49        | 57.14        | 53.00         | 46.42        | 72.50       | 57.41         | 60.41         | 76.97       | 51.10        |\n|        | FORLA       | **75.52**   | **64.53**    | **57.14**    | **66.44**     | **54.14**    | **77.63**   | **61.74**     | **65.21**     | **79.32**   | **57.69**    |\n\n*Table: mBO and Corloc scores across different methods and data combinations.Abd:Abdominal,Thor:Thoracic,Chol:Cholec, Comb: Data combinations*"}, {"review_id": "ISVKwWzAwp", "reviewer": "Reviewer_mowD", "summary": "This paper extends the object-centric representation learning from the centralized setting the distributed federated setting. Several techniques are assembled to enable this extension. First, a shared feature adapter is introduced and trained collaboratively across clients to adapt features from foundation models. Second, two-branch student–teacher architecture is introduced, a student decoder learns to reconstruct full features from foundation models, while a teacher decoder reconstructs their adapted, low-dimensional counterpart. Experiments show the effectiveness of the proposed method.", "questions": "See the weakness above.", "limitations": "Yes.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths\n- The paper is well presented.\n- The proposed method is technically sound. \n- Experiments and analysis have done to verify the effectiveness of the proposed method. \n\nWeakness\n- The technical contribution is limited. The proposed method essentially ensembles several mature techniques to adapt the object centric representation learning problem from the conventional setting the federated learning setting. \n\n- Adding adapters to finetune a pretrained model for the target task is common.  Feature reconstruction is also a widely used technique is self-supervised learning, and the knowledge distillation between sever model and client model is also mature technique in federated learning. \n\n- Can the proposed method generalizes to other object centric representation learning approaches beyond slot attention? I did not find any  specific design for the slot attention. I guess other object centric representation learning approaches can also be employed and extended to the federated learning setting. It would be interesting to see these results as well.", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for highlighting the technical soundness and clear presentation of our work. We will  further emphasize the main technical result and novelty of representation learning that combines slot attention’s success in mixed domains with federated learning to exploit structural coherence in local datasets. We will also strengthen the discussion of our novel contributions beyond existing techniques and provide insight on how our approach generalizes to other OCL methods.\n\n---\n\n**Q:** *The technical contribution is limited. The proposed method essentially ensembles several mature techniques to adapt the object centric representation learning problem from the conventional setting the federated learning setting.*\n\n**A:** We will clarify the novelty and contribution in the updated manuscript. Object-centric learning has not been scaled to large multi-domain data (e.g. including surgical and natural, or multiple domains of natural images) and ours is the first study of federated unsupervised object centric learning. As SA works by discovering repeated occurrence of objects in large datasets, disentangling the concepts is crucial and we show it can be achieved with FL. We are thus first to propose using FL to scale object centric learning and we clearly demonstrate how to implement representation disentanglement using a dual branch design, feature adaptation and knowledge distillation and why these approaches work. \n\n---\n\n**Q:** *Adding adapters to finetune a pretrained model for the target task is common. Feature reconstruction is also a widely used technique is self-supervised learning, and the knowledge distillation between sever model and client model is also mature technique in federated learning.*\n\n**A:** Thank you for this comment. We acknowledge that adding adapters to a stack of pretrained models is not our core contribution. We tested several classic adapter designs to investigate FORLA's sensitivity to adapter type, and while the data did show it's important to carefully consider the right choice of the adapter (see also reply to reviewer TTbE regarding AFM performance), we will try to hit a right balance of detail and informativeness in the revision. We also agree with another reviewer to move the details of these designs to the Appendix while focusing more on our main contribution. As for knowledge distillation (KD), its use in slot attention is a technique that was proposed only recently and thus still requires specific and innovative solutions. For example, our method not only uses KD for FL (which is a mature technique) but it leverages FL to stabilize the loss for slot attention KD in a dynamic feature selection regime, a contribution that we believe is novel. \n\n---\n\n**Q:** *Can the proposed method generalizes to other object centric representation learning approaches beyond slot attention? I did not find any specific design for the slot attention. I guess other object centric representation learning approaches can also be employed and extended to the federated learning setting. It would be interesting to see these results as well.*\n\n**A:** We agree with the hypothesis that other unsupervised object centric learning like MONET and IODINE will work with our framework, as long as they could be modified to use feature reconstruction instead of image reconstruction.\nIn object centric learning literature slot attention has been shown to be most effective in both synthetic toy data and real-word data. We believe slot attention is the strongest candidate to incorporate within our framework, and also representative of the phenomenon of object centric learning with federated learning.  \n\n1. Klaus Greff, et al.,Multi-Object Representation Learning with Iterative Variational Inference (IODINE)  \n2. Christopher P. Burgess, at al., MONet: Unsupervised Scene Decomposition and Representation"}, {"review_id": "3eyMSqiD6i", "reviewer": "Reviewer_dX9d", "summary": "The paper addresses an interesting question of efficiently learning object-centric features from datasets in various domains. In particular paper proposed to use adapters for each domain that transform general features from a combination of foundational models with a sophisticated way of training each adapter on the client dataset, while using a shared SA module.", "questions": "Please address the weaknesses of the paper. \n\nAdditional questions: \n1. YTOBJ is a storage choice for the multi-object scene decomposition. If I remember correctly, most videos consist of objects and background. Thus, other video domains, such as robotics videos from DROID, could be a much better choice there. \n2. FT-DINOSAUR showed that it is a significant zero-shot transfer is possible from the COCO natural image domain to other domains. Thus zero-shot baseline could be interesting to see how much such training is needed. \n3. How is the number of slots for each dataset determined? Is the whole pipeline sensitive to it?", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "### Strengths: \n\n1. Overall, the paper studies the important problem of how to combine representations from general foundational models with different domains effectively. \n2. I think the paper is one of the first to show how OCL could be applied to different domains simultaneously, showing the importance of using dataset structure (e.g., that different sources of data share some features, but still have different biases). \n3. The paper studies many research questions during its analysis. The future work and limitations \n\n### Weaknesses: \n\n1. (Clarity) The proposed approach is a quite complex combination of several topics that are not trivially connected, thus making the presentation challenging. It would be great if the authors would make emphasize to making take-away messages and reshape the results accordingly. For example, different ways to adapt features are not novel, and while an important analysis of the better choice could be presented in the appendix, if this is not the message of the paper. \n2. The paper states that it is doing Representation Learning; however, only masks (e.g., object discovery) are evaluated. It would be great to include other ways of evaluation representations (class label or attribute prediction). It would be great to see if learned features in such an involved way are indeed more informative other the domain than the naive average training. Also, while SAM is used only for features, it is not clear if the (supervised) masks from the SAM decoder are much better already than those obtained from decomposition. Thus, such zero-shot baseline should also be present. \n3. Use of several foundational models is potentially quite heavy during inference. It would be great to show how much we are losing in inference speed because of such a combination. Could Agglomerative models like RADIOv2.5 [4] (combined DINO, SAM, and CLIP) be enough already? \n4. Some recent related work is potentially missing. For example, SlotContrast [1] and CTRL-O [2], showed that adaptation of dense features with an MLP is helping with the contrastive objective in videos and vision-language accordingly, while SPOT [3] was using student-teacher distillation for images to improve mask quality. \n\n[1] Temporally Consistent Object-Centric Learning by Contrasting Slots, CVPR 2025\n[2] CTRL-O: Language-Controllable Object-Centric Visual Representation Learning, CVPR 2025\n[3] SPOT: Self-Training with Patch-Order Permutation for Object-Centric Learning with Autoregressive Transformers, CVPR 2024\n[4] RADIOv2.5: Improved Baselines for Agglomerative Vision Foundation Models,  CVPR 2025", "was_revised": true, "final_justification": "I appreciate the effort made during the rebuttal. \nAs authors are planning to simplify the presentation and FLORA works better SAM masks (I think this should be highlighted and discussed to show the difference between OCL methods and zero-shot segmentation), I increased my score to weak accept.", "rebuttal": "We appreciate the reviewer’s recognition that our paper is among the first to apply OCL across multiple domains and demonstrate the benefits of leveraging dataset structure via federated learning. We will restructure the results to emphasize key takeaways, expand the evaluation metrics, and address missing baselines, efficiency concerns, and sensitivity analyses as suggested. \n\n---\n\n**Q:** *The proposed approach is a quite complex combination of several topics that are not trivially connected, thus making the presentation challenging. It would be great if the authors would make emphasize to making take-away messages and reshape the results accordingly. For example, different ways to adapt features are not novel, and while an important analysis of the better choice could be presented in the appendix, if this is not the message of the paper.*\n\n**A:** We agree that the clarity of the paper can be further improved, which is also in line with comments from other reviewers. In the revised version we will move content that is not the core contribution, including the description of adapter designs, performance and benchmarks with a single foundation model, and detailed datasets description to the Appendix. The key message we want to highlight is that FORLA surpasses centralized slot attention for multi-domain data and show how the network can exploit local structural coherence while learning a global representation.\n\n---\n\n**Q:** *The paper states that it is doing Representation Learning; however, only masks (e.g., object discovery) are evaluated. It would be great to include other ways of evaluation representations (class label or attribute prediction). It would be great to see if learned features in such an involved way are indeed more informative other the domain than the naive average training.*\n\n**A:** Testing the learned representation on tasks like classification or attribute prediction is definitely valuable and interesting and is part of our ongoing work. We followed other object centric representation papers (e.g MONET, IODINE and FT-DINO) focusing on the object segmentation masks that provide fine-grained evaluation metrics to compare different approaches. \n\n1. Klaus Greff, et al.,Multi-Object Representation Learning with Iterative Variational Inference  2. Christopher P. Burgess, et al., MONet.\n\n---\n\n**Q:** *while SAM is used only for features, it is not clear if the (supervised) masks from the SAM decoder are much better already than those obtained from decomposition. Thus, such zero-shot baseline should also be present.*\n\n**A:** We perform zero-shot evaluation of full SAM models (with ViT-b and the ViT-h). We found SAM ViT-h is able to surpass slot attention models trained with FORLA on the natural images but fails on the very specific surgical domain (Thoracic). The underlying reason could be that the thoracic data is not publicly available and not included in the training data of SAM. The inference time for SAM ViT-h of one image is 1.1s (0.85s for ViT-b), while the slot attention model takes only 3.75ms. Another advantage of OCL over SAM is that training SAM on new domains would require mask supervision and, most importantly, SAM will not capture the objectness (over decompose object parts to tiny masks) unless instructed to do so via annotation. \n\n|          |       |     **PASCAL**    |         | | **Abdominal** |         |  | **Thoracic (+)** |         |\n|----------|------------|---------|---------|---------------|---------|---------|--------------|---------|---------|\n|  Method   | **mBO**    | **FG-ARI** | **Cor-Loc** | **mBO**       | **FG-ARI** | **Cor-Loc** | **mBO**      | **FG-ARI** | **Cor-Loc** |\n| **SAM (ViT-b)** | 23.79      | 24.55   | 35.16   | 45.76         | 52.10  | 39.15   | 18.10        | 16.41   | 8.35    |\n| **SAM (ViT-h)** | 54.90      | 57.31   | 79.12   | 66.66         | 69.81 | 65.47   | 33.57        | 30.48   | 23.73   |\n| **FORLA**      | 40.83      | 39.51   | 45.04   | 57.86         | 64.88 | 80.30   | 61.86        | 47.58   | 75.42   |\n\n*Performance comparison of SAM and FORLA.+Note that thoracic is less publically available*\n\n---\n\n**Q:** *Use of several foundational models is potentially quite heavy during inference. It would be great to show how much we are losing in inference speed because of such a combination. Could Agglomerative models like RADIOv2.5 [4] (combined DINO, SAM, and CLIP) be enough already?*\n\n**A:** Using four foundation models (with VIT-b/16) increases the computation overhead from 1.4 ms (with only DINO) to 3.8 ms. In addition to testing performance with a single foundation model (already in Appendix), we tested a configuration with  3 foundation models (DINO, SAM,CLIP) under both frozen and adapted setting. As shown in the table below, we found that 4 foundation models achieve best performance, especially in surgical images which is mostly not included in the pretraining of foundation models. Nevertheless, RADIOv2.5 provides additional insight on combining foundation models and will be mentioned in the related work.\n\n|              |                          |  |      **Surgical**   |         | |  **Natural**   |         |\n|-------------|------------|-------------|---------|---------|--------------------|---------|---------|\n| **Setting** |  **FM number** | **mBO**                 | **FG-ARI** | **Cor-Loc** | **mBO**            | **FG-ARI** | **Cor-Loc** |\n| Frozen      | 3                      | 39.78                   | 51.28   | 54.26   | 42.92              | 33.74   | 51.85   |\n|             | 4                      | 48.65                   | 54.07   | 64.30   | 43.33              | 38.78   | 50.56   |\n| Adapted     | 3                      | 49.58                   | 58.52   | 69.58   | 47.32              | 41.86   | 58.97   |\n|             | 4                      | 50.34                   | 59.04   | 70.73   | 48.98              | 45.32   | 65.11   |\n\n*Evaluation of Frozen and Adapted models using 3 vs. 4 foundation models.FM: Foundation Model, Surgical:Abdominal, Natural:PASCAL*\n\n---\n\n**Q:** *Some recent related work is potentially missing. For example, SlotContrast [1] and CTRL-O [2], showed that adaptation of dense features with an MLP is helping with the contrastive objective in videos and vision-language accordingly, while SPOT [3] was using student-teacher distillation for images to improve mask quality.*\n\n**A:** We appreciate the pointers to additional literature in this fast growing area, and we will add them to the related work section. A few contributions in these papers can be directly applied to our work in the future. For instance the loss in SlotContrast and the permutation of patches (removing of localization bias) in SPOT. For the CTRL-O, it would be interesting to include clients with language data as well.\n\n---\n\n**Q:** *YTOBJ is a storage choice for the multi-object scene decomposition. If I remember correctly, most videos consist of objects and background. Thus, other video domains, such as robotics videos from DROID, could be a much better choice there.*\n\n**A:** Thank you for the recommendation of a new dataset. We chose YTOBJ as it partially overlaps with other Natural domains and lends well to the federated learning exploiting regularities in sub-domains. That being said, the robotic data is interesting for our problem as it lies in between a specialized dataset such as robotic surgery, and a Natural dataset with actuators acting on everyday objects. We will mention this in the discussion section and hopefully add this dataset to our future experiments.\n\n---\n\n**Q:** *FT-DINOSAUR showed that it is a significant zero-shot transfer is possible from the COCO natural image domain to other domains. Thus zero-shot baseline could be interesting to see how much such training is needed.*\n\n**A:** Thank you for this question. We performed additional testing that confirmed our hypothesis which was that, if the domain gap is small, the zero-shot and transfer learning will guarantee good performance. However, FORLA FL will outperform transfer learning when the domain gap is large.We first tested zero-shot transfer of slot attention trained on Natural images (PASCAL and YTOBJ) to surgical images (Abdominal). In this case zeroshot performance is significantly lower compared to FORLA and even compared to models individually trained on abdominal data.\n\n| **Method**                          | **mBO** | **FG-ARI** | **Cor-Loc** |\n|-------------------------------------|---------|------------|-------------|\n| Zero-shot (PASCAL → Abdominal)      | 35.61   | 39.81      | 35.90       |\n| Zero-shot (YTOBJ → Abdominal)       | 33.02   | 37.69      | 30.65       |\n| Individual                          | 50.34   | 59.04      | 70.73       |\n| FORLA                               | 57.86   | 64.88      | 80.30       |\n\n*Comparison of zero-shot transfer, individual training, and FORLA*\n\nWe additionally compare zero shot and transfer finetuning from COCO to PASCAL where the domain gap is smaller, and show that FORLA has an advantage in this setting, but with a smaller margin.\n\n| **Metric** | **Zeroshot** | **Transfer fine-tune** | **FORLA** |\n|------------|--------------|------------------------|-----------|\n| mBO        | 37.20        | 37.80                  | 40.80     |\n\n*Comparison of zeroshot,tranfer from COCO to PASCAL.*\n\n---\n\n**Q:** *How is the number of slots for each dataset determined? Is the whole pipeline sensitive to it?*\n\n**A:** For the Natural image datasets we used optimal slot counts reported in the literature [10,43]. For the surgical datasets we tuned the slot count to achieve a best score. We've also used some evidence from training SA models with both types of data. Choosing an optimal slot number is indeed important for slot attention algorithms. Our method is also fully compatible with SA approaches that use an adaptive slot count, but we decided to use more traditional approaches in this work for easier evaluation."}]

Author rebuttal:

Meta-review: The paper adapts the Slot Attention-based Object-Centric Representation Learning method to federated settings. In addition, it incorporates a two-stage distillation process and an adapter mechanism into the proposed solution.

Strengths: The paper extends federated learning research into a new application scenario: object-centric representation learning.

Weaknesses: The overall solution is presented in a relatively engineering-oriented manner, with limited theoretical depth.

Reviewer Consensus: All reviewers provided positive scores for this paper.

Recommendation: I recommend accepting this paper, as it opens a promising new sub-domain for federated learning research.
Decision: Accept (poster)