MedGMAE: Gaussian Masked Autoencoders for Medical Volumetric Representation Learning
Authors: Xueming Fu
Fenghe Tang
Rongsheng Wang
Yingtai Li
Lixia Han
Jian Lu
Zihang Jiang
S Kevin Zhou
Venue: ICLR 2026 Poster (2026)
Keywords: 3D Gaussian Representation
Medical Imaging analysis
Volumetric Representation Learning
Primary area: applications to computer vision, audio, language, and other modalities
TL;DR: MedGMAE replaces traditional voxel-level reconstruction in medical volumetric data pre-training with 3D Gaussian primitives prediction, enabling more effective anatomical continuous representation and faster CT reconstruction convergence.

Abstract: Self-supervised pre-training has emerged as a critical paradigm for learning transferable representations from unlabeled medical volumetric data. Masked autoencoder based methods have garnered significant attention, yet their application to volumetric medical image faces fundamental limitations from the discrete voxel-level reconstruction objective, which neglects comprehensive anatomical structure continuity. To address this challenge, We propose MedGMAE, a novel framework that replaces traditional voxel reconstruction with 3D Gaussian primitives reconstruction as new perspectives on representation learning. Our approach learns to predict complete sets of 3D Gaussian parameters as semantic abstractions to represent the entire 3D volume, from sparse visible image patches. MedGMAE demonstrates dual utility across medical imaging applications. For representation learning, sparse Gaussian prediction produces superior encoder representations that outperform traditional MAE baselines on downstream segmentation, classification, and registration tasks. For volumetric reconstruction, the Gaussian decoder leverages pretrained anatomical priors to accelerate 3D CT volume reconstruction convergence. Extensive experiments across multiple medical imaging datasets demonstrate that our approach achieves superior performance, establishing a new paradigm for medical image pre-training. The code will be available in https://github.com/windrise/MedGMAE.

Published as a conference paper at ICLR 2026
MEDGMAE: GAUSSIAN MASKED AUTOENCODERS FOR MEDICAL VOLUMETRIC REPRESENTATION LEARNING
Xueming Fu Fenghe ? Rongsheng Wang",, Yingtai Li!., Lixia Han', Jian Lu
Zihang Jiangl
1.21 & S. Kevin Zhoul 1,2,3,4
School of Biomedical Engineering, Division of Life Sciences and Medicine, USTC *MIRACLE Center, Suzhou Institute for Advance Research, USTC
Jiangsu Provincial Key Laboratory of Multimodal Digital Twin Technology
*State Key Laboratory of Precision and Intelligent Chemistry, USTC
$ NNajjing University of Aeronautics and Astronautics Department of Radiology, Medical School, Zhongda Hospital, Southeast University (jzh0103) 03, skevinzhou Bustc.c.. edu.cn cn
ABSTRACT
Self-supervised pre-training has emerged as critical paradigm for learning trans- ferable representations from unlabeled medical volumetric data. Masked autoen- coder based methods have garnered significant attention, yet their application volumetric medical image faces fundamental limitations from the discrete voxel- level reconstruction objective, which neglects comprehensive anatomical structure continuity. To address this challenge, We propose MedGMAE, novel framework that replaces traditional voxel reconstruction with 3D Gaussian primitives recon- struction as new perspectives on representation learning. Our approach learns predict complete sets of 3D Gaussian parameters as semantic abstractions represent the entire 3D volume, from sparse visible image patches. MedGMAE demonstrates dual utility across medical imaging applications. For representation learning, sparse Gaussian prediction produces superior encoder representations that outperform traditional MAE baselines on downstream segmentation, classi- fication, and registration tasks. For volumetric reconstruction, the Gaussian de- coder leverages pretrained anatomical priors to accelerate 3D CT volume recon- struction convergence. Extensive experiments across multiple medical imaging datasets demonstrate that our approach achieves superior performance, establish- ing new framework for medical image pre-training. The code will be available in https///gituhh.om//wnniis/eeddd
INTRODUCTION
Volumetric medical imaging modalities, such as Computed Tomography (CT) and Magnetic Reso- nance Imaging (MRI), have become indispensable cornerstones of modern clinical practice, provid- ing ftree-dimensional anatomical information crucial for diagnosis, treatment planning, and prog- nostic assessment. The advent of deep learning has heralded a new era in the automated analysis of these data, demonstrating unprecedented performance across a spectrum of tasks (Zhou et al., 2023c; Litjens al., 2017). However, the full potential of these data-hungry models severely constrained by fundamental bottleneck: the scarcity of large-scale, expertly annotated datasets (Willemink et al., 2020; Ravi al., 2016).Recen work explores train-free paradigms that leverage pretrained 2D foundation models extract semantic information from 3D volumes (An et al., 2025), demonstrating an alternative.
"Equal contributions Corresponning authors

---

Published as a conference paper at ICLR 2026
(a) MedGMAE
Downstream transfer
Zero shot capabilitt
3D65
Encoder
Decoder part
part
Mask
Rendering
Rec.
Volume
Reg.
Cls.
Other
MIMs
x
99.25% params
3DGS-based reconstruction
Voxel-based
MIM
(267x238x651x4
Pixels)
(27648x11=3e* Params)
(#4e² Voxels)
(b) Pretrained Encoder for Downstream Transfer Tasks
(c) Zero-shot initialization for Reconstruction
Lung redule
1.39x speed
Picural effusion
Card Candionegely no
Registration
Arterial coleification
Segmentation
Clossification
SeyTiOR
projections
CT volume
Figure 1: MedGMAE overview. (a) our MedGMAE pre-training with 3D Gaussian Splatting recon- struction loverages CT volume sparsity (anatomical organs occupy only 11.8% of space) to achieve 99. 99.25% parameter reduction and superior coherence compared to voxel-based MIM methods. (b) Pre-trained encoder fine-tuning for downstream tasks: our MedGMAE could learn a strong encoder representation for downstream segmentation, registration, and classification tasks across multiple medical datasets. (c) our MedGMAE could bring zero-shot capability for 3DGR-based CT recon- struction with 1.39x speed-up.
This challenge sparks an increasing interest self-supervised pre-training methods that can har- ness unlabeled 3D data improve performance in downstream tasks, such segmentation, reg- istration, and diagnosis. Due to the high anatomical similarity across different medical volumes, Masked Image Modeling (MIM) has emerged powerful 3D pre-training approach for learning local representations by reconstructing masked regions from visible context. Despite its promising results, we identify three fundamental yet underexplored challenges that limit the effectiveness of directly reconstructing masked regions via voxel-level regression: (i) Discrete reconstruction con- flicts with anatomical continuity: conventional MIM methods typically regress discrete intensity voxels of masked regions (He et al., 2022; Chen et al., 2023). While this teaches the model to "fill in blanks" based on immediate spatial context works well for photorealistic data, is ill-suited for capturing the underlying semantic continuity and geometric abstraction of anatomical structures in volumetric space. Discrete voxel regression often fails to model shape-consistent features, which crucial for understanding medical images and transferring knowledge to downstream tasks. (ii) Non-transferable decoder representations: A common yet overlooked issue in voxel-based MIM that the decoder designed purely for reconstructing low-level pixel intensities (Xie et al., 2022b; Tang et al., 2024; Tian et al., 2023). The pre-trained decoder is typically discarded, and the features learns are rarely leveraged for downstream tasks, while its zero-shot capability is inherently con- strained by the reliance on pixel-level reconstruction. (fii) Sparse anatomical distribution leads to parameter inefficiency: Unlike natural 2D images that contain dense textural information through- out, 3D medical images are inherently sparse in both semantic and intensity distributions. Redundant voxel-based representation fails achieve optimal reconstruction efficiency.
To address these limitations, we introduce Medical Gaussian Masked Autoencoder (MedGMAE), a novel self-supervised framework tailored for 3D medical image pretraining grounded in a key insight: learning sparse 3D Gaussian representations instead of reconstructing dense voxel in- tensities. As shown in Fig. (a), our approach leverages 3D Gaussian primitives as an intermediate representation that naturally addresses the aforementioned challenges through three key advantages: (i) Continuous geometric modeling for anatomical coherence: 3D Gaussian primitives provide continuous, differentiable representations that inherently capture geometric abstractions and shape consistency across anatomical structures. Each Gaussian primitive encodes spatial position, orienta- tion, and scale information, enabling the model to learn semantically meaningful geometric features
2

---

Published as a conference paper at ICLR 2026
that align with the continuous nature of anatomical boundaries (as shown in Fig. (ii) Trans- ferable decoder: Our Gaussian based decoder remains useful after pre-training directly serving sophisticated initialization for Gaussian representation 3D medical reconstruction (as shown in Ic). faster x.39x for coverage). (iii) Parameter-efficient representation:Out Gaussian-based approach naturally aligns with the sparse anatomical distribution in medical volumes, achieving superior parameter efficiency (99% reduction in parameters).
The main contributions of this work can be summarized follows:
First, we introduce MedGMAE, the first framework to successfully adapt and extend Gaussian-barte masked autoencoding for self-supervised pre-training on 3D volumetric medical data. Our approach learns parameter-efficte representations that better cap- tures continuous anatomical boundaries, enabling models to develop more structured and anatomically-aware representations.
Second, we demonstrate a novel application for the pre-trained decoder by using zero-shot, geometry-aware initializer for downstream 3D CT reconstruction tasks. The learned anatomical priors from pre-training significantly accelerate 3D Gaussian Representation-basse CT reconstruction convergence, thus bridging self-supervised pre- training with practical medical image reconstruction applications.
Third, extensive experiments across downstream tasks including segmentation, classifica- tion, and registration validate the superiority of our proposed approach compared to voxel- based masked representation methods. Additionally, experiments on low-dose CT recon- struction tasks demonstrate the zero-shot initialization capability of our proposed frame- work, showing significant acceleration in convergence while maintaining reconstruction quality.
(a) MedGMAE
Grid Sompling
3D65 set
Encoder
Decoder
Linear
Rendering
Patchify Masking
pos.
GS tokens
Reconstruction Loss
(b) MedGMAE*
Scaling reducing
Level-0
65 tokens
coarse to fine
Concat
decoder
Level-1 B
Level-2 8
Residual blocks
Extended
Figure 2: MedGMAE architecture. (a) MedGMAE pre-training framework that processes patchi- fied and masked input through encoder-decoder architecture to predict 3D Gaussian parameters, which are then rendered and optimized via reconstruction loss. (b) Extended MedGMAE* with multi-level residual blocks for progressive Gaussian parameters refinement.
2 RELATED WORK
2.1 PATCH-BASED MASKED IMAGE MODELING
Masked autoencoders learn representations by reconstructing masked input regions. MAE (He al., 2022) uses ViT encoder (Dosovitskiy et al., 2020) processing only 25% visible patches and lightweight decoder for reconstruction, enabling efficient pretraining. self-supervised methods focus on different architectural designs and masking strategies improve representation learning (Zhou et al., 2023b; Xie et al., 2022b; Tang et al., 2024: Tian et al., 2023; Tang et al., 2022a; Goncharov et al., 2023). Despite their architectural variations and different mask- ing mechanisms, all these approaches remain fundamentally constrained by the voxel-level recon- struction objective, which encourages local interpolation rather than global structural understanding
medical imaging, existing

---

Published as a conference paper at ICLR 2026
of anatomical features. We propose fundamentally different approach using 3D Gaussian parame- ter prediction instead of voxel-level reconstruction. Unlike discrete intensity prediction, our method reconstructs anatomy through continuous geometric primitives, enabling structured representation learning aligned with anatomical continuity.
2.2 3D GAUSSIAN SPLATTING FOR MEDICAL IMAGING
3D Gaussian Splatting (3DGS) was developed for rendering 3D natural scenes (Kerbl al., 2023).
This approach has since been applied across diverse medical reconstruction scenarios, including 3D CT reconstruction (Li al., 2025; Cai et al., 2024; Zha al., 2024), coronary artery reconstruction (Fu et al., 2024), and 4D CT reconstruction (Fu et al., 2025; Yu et al., 2025).
Limitations and motivation. While Gaussian Masked Autoencoders ((Raaasegran al., 2025)) pioneered this approach for 2D images- the z-axis of 3D Gaussians to infer abstract 2.5D layers for spatial understanding our motivation is fundamentally different. Instead of inferring abstract structure, we leverage the continuous and parameter-effine efficient nature of 3D Gaussians to holistically represent true 3D anatomical volumes, directly addressing the limitations of discrete voxel models for capturing continuous anatomy. This objective is better suited for downstream 3D tasks like segmentation and registration, and also unlocks a novel application in accelerating CT reconstruction. Our key insight is: leverage 3D Gaussian primitives intermediate representations for masked autoencoder pre-training, learning geometric structures rather than discrete voxels. This shifts the objective from local reconstruction to geometric reasoning, encouraging spatial reasoning and anatomical priors while addressing initialization challenges in existing 3DGS medical methods.
METHOD
3.1 PRELIMINARIES
3D Gaussian primitives and volume rendering. In medical imaging domain, each 3D Gaussian primitives is parameterized by a center position j E R and covariance matrix Σ R which jointly define the spatial distribution and morphology of the Gaussian primitive. In addition, each Gaussian carries an intensity value I that represents the intensity at the Gaussian center. our im- plementation, follow the standard practice of decomposing the covariance matrix Σ RSSTT Rd into a scaling matrix = diag(s) E 3x3 represented by scale vector s E R and rotation matrix R 3x3 parameterized by rotation quaternion Φ R.1. Consequently, each Gaussian represented by an 11 11-dimensional parameter vector
{14, 8, Φ T} E R For volumetric rendering, we reconstruct the complete 3D volume by evaluating the Gaussian field discrete grid positions corresponding to the target volumetric dimensions. Each Gaussian contribution to any spatial position X mathematically described by:
Gx(X(gs) = 7; 1xx (X-
(1)
where X E R denotes a position in the 3D space, and g, = {14, $;, 6i, I represents the param- eters of the i-th Gaussian. The exponential term defines the spatial decay of the Gaussian influence based on the Mahalanobis distance from its center, naturally encoding the ellipsoidal shape through the covariance structure. The final volumetric intensity is computed as a spatially aggre- gation of contributions from nearby Gaussians:
V(Xlg:)
Σ G((X)g))
=IIX-MiISS
where d; defines the effective radius of influence for each Gaussian, typically set based the eigenvalues the covariance matrix to ensure computational efficiency while maintaining rendering quality. This localized aggregation strategy enables efficient rendering by avoiding computations for Gaussians with negligible contributions, making the differentiable rendering process tractable for large-scale medical volumes.
3.2 PROPOSED APPROACH
We propose MedGMAE, a framework that replaces voxel-level reconstruction with 3D Gaussian parameters prediction for medical volumetric representation learning in 2(a.)(a).
4

---

Published as a conference paper at ICLR 2026
MedGMAE representation learning: The model consists of Vision Transformer (ViT)-based encoder, a lightweight Transformer decoder, and a differentiable Gaussian renderer specifically de- signed for volumetric medical data reconstruction. For given 3D medical image patch with di- mensions 96 x 96 x 96, we first patchify it into N non-overlapping patches of size 12 x 12 x 12, resulting in N 512 patches. We then randomly mask these patches with masking ratio r, typ- ically set to 0.75, yielding n visible patches where n N x (1-r). The ViT encoder processes only the visible patches and encodes them from raw patch representations to latent embeddings I; E R Rene, where i E {1, ... The decoder employs & learnable query tokens 9j E R Rdaes, j E 1,, ... k 1}, where represents the number of 3D Gaussians to be predicted. Impor- tantly, & can be set to any value independent of the number of masked tokens, providing flexibility in controlling the reconstruction granularity. We project the encoder latent embeddings to the decoder dimension space as 2; E Rdde and construct the decoder input by concatenating three components: the encoder class token, the learnable Gaussian query tokens, and the remaining encoder tokens:
Xdec { U {a)))-11 U (a))i=2
The decoder processes the Xdec tokens through multiple Transformer blocks with multi- self- attention mechanisms. This allows the query tokens to attend to the visible patch embeddings and aggregate spatial-semanta information necessary for accurate 3D Gaussian parameter prediction. The decoder outputs k sets of Gaussian parameters, with each query token predicting one 3D Gaus- sian primitive through dedicated parameter heads. Each predicted 3D Gaussian is an 1-dimennsional vector comprising position coordinates J E R anisotropic scaling factors & E R rotation quater- nion & E R²,, and intensity 7 E R.. The conversion from decoder features Gaussian parameters is accomplished through four specialized linear prediction heads: a Gaussian center head, a scale head, a rotation head, and an intensity head. Each head applies appropriate activation functions to ensure parameter validity: sigmoid activation for positions and densities to constrain values within [0,1], and L2 normalization for rotation quaternions to maintain unit length. To ensure stable train- ing and balanced parameter distributions across the three spatial dimensions, we employ custom initialization strategies for the prediction heads. All heads utilize Xavier uniform initialization for weights, while biases are initialized specifically for each parameter type: position and rotation heads use zero initialization, the scale head employs a constant bias of - -1.886 (resulting in approximately 0.2 after sigmoid activation), and the density head uses bias of -0.405 (yielding approximately 0.5 after sigmoid activation). This initialization scheme promotes consistent scale distributions across x, y, and z dimensions while providing reasonable starting values for Gaussian intensity.
Differentiable volumetric rendering and training: Once we obtain i predicted 3D Gaussians, we employ differentiable volumetric renderer to reconstruct the 3D medical image. The renderer ac- cumulates the contributions of all Gaussians within the volume space, with each Gaussian influence determined by its spatial extent and intensity. During training, we apply the reconstruction loss only to the originally masked regions, computed as the mean squared eтror between the rendered volume and the ground truth image. This masked reconstruction objective encourages the model to learn meaningful 3D representations while maintaining computational efficiency.
Extended MedGMAE for reconstruction: For enhanced reconstruction performance, we further present MedGMAE* with multi-level residual blocks (a hierarchically extended MedGMAE struc- ture (Hyun & Heo, 2024)), utilizing more Gaussians to capture fine-grained volumetric details in Fib. We define a hierarchical structure with levels / E (0, 1,2), from coarse to fine granularity, where each level contains a set of Gaussian parameters. Specifically, we establish dependencies between Gaussian parameters of adjacent levels, where Level 0 contains No base Gaussians, Level expands to mi x No Gaussians, and Level expands to m2 x No Gaussians. We model the 3D representation in coarse-to- to-fine manner by assigning coarser- and finer-level Gaussians for coarser and finer details. For scale parameters, we enforce hierarchical reduction as:
(4) where ssoate 0.1 controls the residual magnitudes, and As' > 0 to ensure monotonic scale duction. A = 0.02 for Level and As² = 0.05 for Level 2 are adopted. For other parameters, we compute new positions as: n¹ = n² + j T.. o.. where j! are the predicted residual position parameters. We define residual transformations as: I' = 1 + f oj. d' = normalize( + I' Gyot) where Tp 01, rot control the residual magnitudes. Note that all residual prediction modules adopt tanh activation functions to ensure bounded residual outputs and stable training dynamics. This hierarchi- cal densification enables coarse-to-fine reconstruction while maintaining spatial coherence through
s' 5 + s' Eecale As'
5

---

Published as a conference paper at ICLR 2026
Table I: Comparison of different methods with different proportions AMOS (Ji et al., 2022),
FLARE'22 (Ma et al., 2024), BTCV (Landman et al., 2015) and SegTHOR (Lambert et al., 2020).
The DSC (%) is reported. val (bold) val (underline) : top method second method. denotes utilize official pre-training weights. denotes the results are copied from VoCo (Wu et al., 2024b).
AMOS
AARE
BTCV
SegTHOK
Pretrain Method
1%
10%
100%
1%
10%
100%
10%
100%
10%
100%
Training from scratch
UNETR
23.67
77.02
22.47
70.81
28.05
42.85
79.82
42.31
72.72
85.82
SwinUNETT
28.94
63.45
82.51
35.89
63.38
75.38
27.71
51.33
80.53
44.82
73.93
87.35
General sepesopeee methods
Spark
36.
71.68
84.07
36.48
71.74
80.67
30.69
51.26
44.76
80.36
88,08
MAE
83.61
62.35
Medical wef-nipervied methods
25.72
62.99
27.30
57.33
29.27
38.04
81.45
36.96
60.16
83,79
TransVWi
18.72
66.91
82.58
4.81
62.07
75.78
5.63
8.42
8.91
31.30
87.46
UniMiSST
29 49
66.34
79.92
24.92
60.90
74.71
32.95
47.08
42.92
76.59
84.34
SUP
82.45
33.72
81.54
PCRLv2"
21.07
39.07
54.14
27.71
42.97
54.29
24.01
30.48
81.74
40.22
74.71
85.77
GVSL'
24.25
63.45
81.38
26.33
59.54
73.27
24.86
41.79
81.87
42.56
77.40
86.98
voN2veef
32.76
62.30
74.78
34.11
70.33
35.29
51.77
47.21
73.98
86.77
HySrakk
34.50
64.32
85.58
37.54
73.60
82.35
35.81
51.54
58.81
83.95
88.74
VoCaa
77.85
67.12
83.87
MedGMAE
58.79
75.65
84.90
62.72
78.72
83.77
56.19
27.11
83.22
70.92
83.91
89.15
base Gaussian constraints, significantly enhancing the model's ability capture fine-grained details
in CT reconstruction.
EXPERIMENTS
4.1 DATASETS
Pre-training datasets. For self-supervised pre-training, we utilize the AbdomenAtlas' ..iin dataset (Li et al., 2024a), which contains 5,195 CT scans. All scans are first resampled spacing of 1.5mmx 1.5mm 1.5mm using trilinear interpolation. The Hounsfield Unit (HU) values are then clipped to the range [-175, 250]. Finally, the intensity values are normalized the range [0, 1].
Downstream datasets. For segmentation tasks, we conduct experiments on four public datasets: AMOS (Ji al., 2022), FLARE'22 22 (Ma et al., 2024), BTCV (Landman et al., 2015), and Seg THOR (Lambert et al., 2020), with official Medical image classification tasks are conducted on the CT-RATE dataset (Hamamci et al., 2024) with official data partition. For registration tasks perform experiments and OASIS (Marcus et al., 2007) with same data split as (Wu et al., 2024a). Also, CT reconstruction experiments are conducted on the low-dose Chest and Abdomen CT: AAPM-Mayo dataset (Moen et al., 2021).
validation split with 1%, 10% and 100% proportions.
IXI (Kim et al., 2021)
Ground Truth
GVSL
HySparK
MAE
SUP
UniMiSS
VoCo
Ours
Figure 3: Visualization of one-shot segmentation results for AMOS (row 1), FLARE'ARR2 22 (row 2) and SegTHOR (row 3).

---

Published as a conference paper at ICLR 2026
4.2 IMPLEMENTATION DETAILS
For pre-training, we sample the pre-training volumes into 96 x 96 x 96 patches using RandCrop- ByPosNeeL.abel with 3 crops per volume. We use the AdamW optimizer, an initial learning rate of le-5, and a cosine-annealing scheduler for all experiments. The pre-training uses a batch size of 8 and trains the model for about 100K steps. All experiments use fixed random seed of 41 to ensure reproducibility. We evaluate our method using task-specific metrics: Dice Similarity Coefficient (DSC) for segmentation, Area Under the Curve (AUC) for classification, DSC for registration, and Peak Signal-to-Noiss Ratio (PSNR) and Structural Similarity Index Measure (SSIM) for reconstruc- tion tasks.
For downstream transfer tasks, we adopt UNETR (Hatamizadeh et al., 2022) as the baseline network followed as (Chen et al., 2023; Tang et al., 2024). For segmentation task, all the pre-processing strategies are the same (Tang et al., 2022b). For classification task, We resample the volume to 1.5 x 1.5 x 3.0 mm, clip the HU range to [-1000, 1000] and rescale it to [0, 1]. The volume size is set be 192 x 192 x 96. The model is trained for 100 epochs with batch size of 96, using AdamW W as the optimizer with a learning rate of 3e-2 and weight decay of 0.05. For registration task, our registration algorithm based on TransMorph (Chen et al., 2022) and all the registration pre- processing and training strategies are the same as (Wu et al., 2024a). All experiments are conducted on NVIDIA H20 GPUs.
For CT reconstruction task, we follow the experimental setup of (Li et al., 2025). The key difference that FBP reconstruction results are cropped using non-overlapping sliding windows and fed into MedGMAE as input. The output Gaussian parameters undergo volume rendering and are concate- nated back to original size. To reduce FBP artifact interference, we conduct experiments with 80, 120, and 160 projections. All experiments are trained for 15,000 iterations on Nvidia 3090 GPUs. We evaluate the original 3DGR, 3DGR with MedGMAE initialization, and 3DGR with MedGMAE** initialization using training time, iterations required to reach PSNR=35 and SSIM-90% and final PSNR and SSIM after complete training.
Comparison with state-of-the-art methods. We select both general and medical self-supervised learning methods for comprehensive comparison. Following (Wu et al., 2024b), we select UNETR (Hatamizadeh et al., 2022), SwinUNETR (Tang et al., 2022a) as compared baseline model. For segmentation tasks, we compare against prominent masked image modeling (MIM) methods, in- cluding MAE (He et al., 2022; Chen et al., 2023) and SparK (Tian et al., 2023), under identical experimental settings. Additionally, we select nine recent and well-known self-supervised methods: Models Genesis (MG) (Zhou al., 2021), Tra (Haghighi et al., 2021), UniMiSS (Xie al., 2022a), Swin UNETR Pretrained method (SUP) (Tang et al., 2022a), PCRLv2 (Zhou et al., 2023a), GVSL (He al., 2023), vox2vee (Goncharov al., 2023), HySparK (Tang et al., 2024). and VoCo (Wu et al., 2024b). For registration tasks, we compare against methods trained from scratch, in- cluding VoxelMorph (Balakrishnan et al., 2019), TrrpsMorph (Chen et al., 2022), and SWinUNETR (Hatamizadeh et al., 2021), as well as methods with pre-training such as SUP (Tang et al., 2022b), SuPreM (Li et al., 2024b), and VoCo (Wu et al., 2024b). To ensure fair comparison, we utilize of- ficial implementations and loaded official pre-trained weights for all medical SSL methods before fine-tuning.
5 RESULTS
5.1 PROMISING DOWNSTREAM TRANSFER RESULTS
Medical image segmentation. Following previous work, we fine-tuned pre-trained models using 1%. 10%, and 100% of the training data on AMOS, FLARE' 22, BTCV, and SegTHOR datasets, respectively. The segmentation results are presented Table 1. MedGMAE achieves the best or second-best DSC scores among all compared methods across different data regimes. Notably, our method demonstrates particularly strong performance in low-data scenarios, outperforming the pre- vious best method VoCo by 2.98% and 5.06% on AMOS and FLARE'22, respectively, with 1% data. Compared to training from scratch baselines (Hatamizadeh et al., 2022; 2021), our pre-trained McdGMAE demonstrates substantial improvements, with gains of 20-35% in 1% data scenarios across all datasets. Even with full training data, pre-training consistently provides meaningful im- provements of 2-8% over the corresponding from-scratch methods. These results demonstrate that
7

---

Published as a conference paper at ICLR 2026
Table 2: Performance comparison CT-RATE Table 3: The DSC(%) of registration on IXI and dataset for classification task. The AUC (%) is OASIS datasets. $ denotes the results are copied shown. val (bold) / val (underline) top method from VoCo (Wu et al., 2024a). The best results / second method. i denotes official pre-training are in bold.
weights.
Method
IXI
OASIS
Method
AUC
Training From Scratch
UNETR
71.43
VoxeIMorph
71.5
78.6
Scratch
SwinUNETR
74.29
TransMorph'
74.5
81.6
VoCo-10K
72.11
SwinUNETR
72.6
81.8
VoCo-
76.02
Fine-luning
Fine-tuning
160KT
SUP
67.7
81.5
SUP
76.04
SuPreM
72.9
81.2
MedGMAE 76.40
VoCot
73.6
84.4
MedGMAE
73.7
85.7
Table Comprehensive reconstruction comparison across different projection views. 3DGR refers to the initialization method employed in the original paper (Li et al., 2025), whereas MedGMAE and MedGMAE* indicate initialization using Gaussian points estimated through zero-shot inference by proposed model. Values are reported as mean + standard deviation. The best results are in bold.
Method
Time(min)
iter(P=35)
iter(S=90%)
PSNR(full)
SSIM(full)
80 projections
3DGR
1:3.5
1371.6
1140
44.6
98.4
MedGMAE
303
1135495.0
1100
43.9
97. 97.5
MedGMAE
19.8
990 137.5
820_60.0
40.17
98.0+0.38
120 projections
3DGR
547.47.8
1660+382.6
1150
45.2
98.7 98.72
MedGMAE
357+22.0
1040+91.7
980_60.0
46.2+1.17
98.5 +0 29
MedGMAE
335+22044
920+74.8
780_32.0
45.8+1.15
98.7
160 projections
3DGR
594+140.5
171 1
11374211.8
45, 45.1 1.63
98.7 00.74
MedGMAE
3734
8857
967 66.9
11.36
98.7 +0.25
MedGMAE
388
960+:66.8
780_33.1
45.8
98.7
98.7.4..1 +0.21
our method could learn a strong anatomical representation by using Gaussian representation. Fig. shows the visualization results.
Medical image classification. Table presents the performance comparison on the CT-RATE dataset. Compared to training from scratch, MedGMAE shows substantial improvements over the best scratch-trained baseline Swin-Bv2 by 2.11%. Among pre-trained methods, MedGMAE surpasses the previous best performers VoCo-160K and SUP by 0.38% and 0.36% respectively, demonstrating the effectiveness of our pre-training approach.
Medical image registration. Table presents the DSC performance comparison on IXI and OA- SIS datasets for medical image registration tasks. MedGMAE achieves the best performance on OASIS and competitive results on IXI. Compared to the best seratch-trained baselines, our method provides substantial improvements of 1.1% on IXI and 3.9% on OASIS. Among me-trained meth- ods, MedGMAE outperforms the previous state-of-the-an VoCo by 1.3% on OASIS, confirming the effectiveness of pre-training approach for medical image registration tasks. It worth noting that both IXI and OASIS are from unseen MRI modality, which demonstrates the generalization ability of MedGMAE.
5.2 GGOMETRY-AWARE ZERO-SHOT INITIALIZATION FOR 3DGS-BAL BESSE MEDICAL IMAGE RECONSTRUCTION
As shown in Table 4 and Fig. 4, MedGMAE demonstrates significant acceleration training con- vergence across all projection settings. For training efficiency, MedGMAE reduces training time
8

---

Published as a conference paper at ICLR 2026
500 900 2.50K3.4k 5.00k
7.50k
10.00k
12.50k
15.0k
Iterations
45
PSNR
MedGMAE MedGMAE'
3DGR
30
900
3.4k
15k
Med
GMAE
Med
GMAE
3DGR
CT reconstruction convergence analysis on AAPM-Mayo dataset. Top: Average PSNR curves with standard eтror bands showing reconstruction quality improvement over training itera- tions for different methods. Bottom: Visual comparison of reconstructed CT slices 500, 900, 3.4k, and 15k iterations for MedGMAF, MedGMAE* and 3DGR methods, demonstrating the faster con- vergence and superior reconstruction quality of our approaches.
Figure 4:
by 31.0%, 35.0%, and 37.2% compared 3DGR baseline with 80, 120, and 160 projections spectively, More importantly, MedGMAE substantially accelerates convergence speed, requiring 39.4% and 28.1% fewer iterations to reach PSNR=35 and SSIM=90% benchmarks on average. The residual-extent MedGMAE* further improves convergence performance, achieving even faster iteration counts for quality thresholds while maintaining comparable final reconstruction quality. These results demonstrate that our pre-training approach significantly enhances training efficiency for 3D Gaussian representation based CT reconstruction without compromising final image qual- ity. Statistical analysis using t-tests revealed that our proposed MedGMAE initialization methods significantly outperformed 3DGR (p 0.001) in traiing efficiency.
5.3 ABLATION STUDY
Table presents the ablation study results on MedGMAE components across three segmentation datasets. Adding voxel-based SSL provides substantial improvements of 6-12% the baseline. Our proposed Gaussian-based SSL further enhances performance by 1-2% compared to voxel-based approaches, confirming the superiority of 3D Gaussian representation over voxel-based reconstruc- tion.
Table Transfer ablation on MedGMAE. The DSC (%) is reported.
Proxy
SSL
AMOS
FLARE'22
SegTHOR
Voxel
Gaussian
77.02
70.81
85.82
83.61
82.56
88.52
84.90
83.77
89.15

---

Published as a conference paper at ICLR 2026
6 CONCLUSION
In this paper, we present MedGMAE, a novel self-supervised pre-training framework that replaces voxel-level reconstruction with 3D Gaussian representation. Leveraging the more efficient and con- tinuous 3D Gaussian primitives, MedGMAE achieves promising encoder transfer performance on diverse downstream tasks including segmentation, classification, and registration. Besies, the trans- ferable decoder enables x.39x acceleration compared to original 3DGR-CT reconstruction meth- ods. Extensive experimental results demonstrate the effectiveness of MedGMAE across multiple medical imaging upplications. However, in CT reconstruction tasks, the result are affected by noise from FBP reconstruction, which could be improved by training a multi-view 3D Gaussian founda- tion model.
ACKNOWLEDGMENTS
Supported by National Natural Science Foundation of China under Grant 62271465, Suzhou Basic Research Program under Grant SYG202338, and Jiangsu Province Science Foundation for Youths (NO. BK20240464).
REFERENCES
Ulzee An, Moonseong Jeong, Simon A Lee, Aditya Gorla, Yuzhe Yang, and Sriram Sankarara- man. Raptor: Scalable train-free embeddings for 3d medical volumes leveraging pretrained 2d foundation models. arXiv preprint arXiv:2507.08254 2025.
Guha Balakrishnan, Amy Zhao, Mert R Sabuncu, John Guttag, and Adrian V Dalca. Voxelmorph: learning framework for deformable medical image registration. IEEE transactions on medical imaging, 38(8): (118):778-1800)) 2019.
Yuanhao Cai, Yixun Liang, Jiahao Wang, Angtian Wang. Yulun Zhang, Xiaokang Yang, Zongwei Zhou, and Alan Yuille. Radiative gaussian splatting efficient x-ray novel view synthesis. In European Conference on Computer Vision, pp. 283-299. Springer, 2024.
Junyu Chen, Eric C Frey, Yufan He. William P Segars, Ye Li, and Yong Du. Transmorph: Trans- former for unsupervised medical image registration. Medical image analysis, 82:102615, 2022.
Zekai Chen, Devansh Agarwal, Kshitij Aggarwal, Wiem Safta, Mariann Micsinai Balan, and Kevin Brown. Masked image modeling advances 3d medical image analysis. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision pp. 1970- 1980, 2023.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition scale. arXiv preprint arXiv:2010.1 11929, 2020.
Xueming Fu, Yingtai Li, Fenghe Tang, Jun Li, Mingyue Zhao, Gao-Jun Teng, and S Kevin Zhou. 3dgr-car: Coronary artery reconstruction from ultra-sparse 2d x-ray views with a 3d gaussians representation. In International Conference Medical Image Computing and Computer-Assiited Intervention, pp. 14-24. Springer, 2024.
Xueming Fu, Pei Wu, Yingtai Li, Xin Luo, Zihang Jiang, Junhao Mci, Jian Lu, Gao-Jun Teng, and S. Kevin Zhou. Dyna3dgr: 4d cardiac motion tracking with dynamic 3d gaussian representation, 2025. URL https: arxiv. wtggabs///// 0tt.//550..166688.8 16608.
Mikhail Goncharov, Vera Soboleva, Anvar Kurmukov, Maxim Pisov, and Mikhail Belyaev. vox2vec: A framework for self-supervised contrastive learning of voxel-level representations in medical images. In International Conference on Medical Image Computing and Conputer-Assisted Inter- vention, pp. 605-614. Springer, 2023.
Fatemeh Haghighi, Mohammad Reza Hosseinzadeh Taher, Zongwei Zhou, Michael B Gotway, and Jianming Liang. Transferable visual words: Exploiting the semantics of anatomical patterns for self-supervised learning. IEEE transactions medical imaging, 40( 40(10):2857-2888 2021.
10

---

Published as a conference paper at ICLR 2026
Ibrahim Ethem Hamamci, Sezgin Er, Furkan Almas, Ayse Gulnihan Simsek, Sevval Nil Esirgun, Irem Dogan, Muhammed Furkan Dasdelen, Bastian Wittmann, Enis Simsar, Mehmet Simsar, et al. A foundation model utilizzig chest et volumes and radiology reports for supervised-level zero-shot detection of abnormalities. CoRR, 2024.
Ali Hatamizadeh, Vishwesh Nath, Yucheng Tang, Dong Yang, Holger R Roth, and Daguang Xu. Swin unetr. Swin transformers for semantic segmentation brain tumors in mri images. In International MICCAI brainlesion workshop, pp. 272-2844 Springer, 2021.
Ali Hatamizadeh, Yucheng Tang, Vishwesh Nath, Dong Yang, Andriy Myronenko, Bennett Land- man, Holger R Roth, and Daguang Xu. Unetr: Transformers for 3d medical image segmentation. In Proceedings of the IEEE/CVF winter conference applications of computer vision. pp. 574- 584, 2022.
Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked au- toencoders are scalable vision learners. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 16000-16009, 2022.
Yuting He, Guanyu Yang, Rongjun Ge. Yang Chen, Jean-Louis Coatrieux, Boyu Wang, and Shuo Li. Geometric visual similarity learning 3d medical image self-supervised pre-training. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9538-9988999 95589957,, 2023.
Sangeek Hyun and Jae-Pil Heo. Gsgan: Adversarial learning for hierarchical generation of 3d gaussian splats. Advances in Neural Information Processing Systems, 37:67987-68012, 2024.
Yuanfeng Haotian Bai, Chongjian Ge, Jie Yang, Ye Zhu, Ruimao Zhang, Zhen Li, Lingyan Zhanng. Wanling Ma, Xiang Wan, et al. Amos: A scage-scale abdominal multi-organ benchmark for versatile medical image segmentation. Advances in neural information processing systems, 35:36722-36732, 2022.
Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. 3d gaussian splat- ting for real-time radiance field rendering. ACM Trans. Graph., 12 2023.
Boah Kim. Dong Hwan Kim, Scong Ho Park, Jieun Kim, June-Goo Lee, and Jong Chul Ye. Cy- clemorph: cycle consistent unsupervised deformable image registration. Medical image analysis, 71:102036, 2021.
7oé Lambert, Caroline Petitjean, Bernard Dubray, and Su Kuan. Segthor: Segmentation of thoracic organs at risk in ct images, 2020 Tenth International Conference on Image Processing Theory, Tools and Applications (IPTA), pp. 1-6. Icce, 2020.
Bennett Landman, Zhoubing Xu, Juan Igelsias, Martin Styner, Thomas Langerak, and Arno Klein. Miccai multi-atlas labeling beyond the cranial vault-workshop and challenge. In Proc. MIC- CAI multi-atlas labeling beyond cranial vault works-workshop challenge, volume 5, pp. 12. Munich, Germany, 2015.
Wenxuan Li, Chongyu Qu, Xiaoxi Chen, Pedro RAS Bassi, Yijia Shi, Yuxiang Lai, Qian Yu, Huimin Xue, Yixiong Chen, Xiaorui Lin, al. Abdomenatlas: large-scale, detailed-annotated, multi- center dataset for efficient transfer learning and open algorithmic benchmarking. Medical Image Analysis, 97: 103285, 2024a.
Wenxuan Li, Alan Yuille, and Zongwei Zhou. How well do supervised models transfer to 3d image segmentation. In The Twelfth International Conference on Learning Representations, volume 1, 2024b.
Yingtai Li. Xueming Fu, Han Li, Shang Zhao, Ruiyang Jin, and S Kevin Zhou. 3dgr-ct: Sparse-view ct reconstruction with 3d gaussian representation. Medical Image Analysis, pp. 103585, 2025.
Geert Litjens, Thijs Kooi, Babak Ehteshami Bejnordi, Arnaud Arindra Adiyoso Setio, Francesco Ciompi, Mohsen Ghafoorian, Jeroen Awm Van Der Laak, Bram Van Ginneken, and Clara I Sánchez. A survey on deep learning in medical image analysis. Medical image analysis, 42: 60-88, 2017.
11

---

Published as a conference paper at ICLR 2026
Jun Ma. Yao Zhang, Song Gu, Cheng Ge, Shihao Mae, Adamo Young, Cheng Zhu, Xin Yang, Kangkang Meng, Ziyan Huang, et al. Unleashing the strengths of unlabelled data in deep learning- assisted pan-cancer abdominal organ quantification: the flare22 challenge. The Lancet Digital Health, 6(1 ((11)=e81--880 2024.
Daniel S Marcus, Tracy H Wang, Jamie Parker, John G Csermansky, John C Morris, and Randy L Buckner. Open access series of imaging studies (oasis): cross-sectional mri data in young. middle aged, nondemented, and demented older adults. Journal of cognitive neuroscience, 19(9):1498- 1507, 2007.
Taylor R Moen, Baiyu Chen, David Holmes III, Xinhui Duan, Zhicong Yu, Lifeng Yu, Shuai Leng, Joel G Fletcher, and Cynthia H McCollough. Low-dose ct image and projection dataset. Medical physics, 4802)9902-911 2021.
Jathushan Rajasegaran, Xinlei Chen, Rulilong Li, Christoph Feichtenhofer, Jitendra Malik, and Shiry Ginosar. Gaussian masked autoencoders, 2025. URL https://arxiv: wwtpeiivv.rrr.eeooooooo /bs/ 2501.03224 03229.
Daniele Ravi, Charence Wong, Fani Deligianni, Melissa Berthelot, Javier Andreu-Perez, Benny Lo. and Guang-Zhong Yang. Deep learning for health informatics. IEEE journal of biomedical and health informatics, 21(1):4 21, 2016.
Fenghe Tang, Ronghao Xu, Qingsong Yao, Xueming Fu, Quan Quan, Heqin Zhu, Zaiyi Liu, and Kevin Zhou Hyspark: Hybrid sparse masking for large scale medical image pre-training. In International Conference on Medical Image Computing and Compuler-Assisted Intervention, pp. 330-340. Springer, 2024.
Yucheng Tang, Dong Yang, Wenqi Li, Holger R Roth, Bennett Landman, Daguang Xu, Vishwesh Nath, and Ali Hatamizadeh. Self-supervised pre-training of swin transformers for medical image analysis. In Proceedings of the IEEE/CVF conference on computer vision and patlern 2022a.
recognition, pp.
Yucheng Tang, Dong Yang, Wenqi Li, Holger Roth, Bennett Landman, Daguang Xu, Vishwesh Nath, and Ali Hatamizadeh. Self-supervised pre-training of swin transformers for 3d medical image analysis. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 20730-20740, 2022b.
Keyu Tian, Yi Jiang, Qishuai Diao, Chen Lin, Liwei Wang, and Zehuan Yuan. Designing bert for convolutional networks: Sparse and hierarchical masked modeling. arXiv preprint wrXiv:2301.03500 2023.
Martin J Willemink, Wojciech A Koszek, Cailin Hardell, Jie Wu, Dominik Fleischmann, Hugh Harvey, Les R Folio, Ronald M Summers, Daniel L Rubin, and Matthew P Lungren. Preparing medical imaging data for machine learning. Radiology, 295(1):4-15, 2020.
Linshan Wu, Jiaxin Zhuang, and Hao Chen. Large-scale 3d medical image pre-training with geo- metric context priors. arXiv preprint arXiv:2410.00 00.9990, 2024a.
Linshan Wu, Jiaxin Zhuang, and Hao Chen. Voco: simple-yet-effectivy volume contrastive learn- ing framework for 3d medical image analysis. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 22873-22883 2024b.
Yutong Xie, Jianpeng Zhang, Yong Xia, and Qi Wu. Unimiss: Universal medical self-supervised learning via breaking dimensionality barrier. In European Conference on Computer Vision, pp- 558-575. Springer, 2022a.
Zhenda Xie, Zheng Zhang, Yue Cao, Yutong Lin, Jianmin Bao, Zhuliang Yao, Qi Dai, and Han Hu. Simmim: simple framework for masked image modeling. Proceedings of the IEFF/CVF conference on computer vision and pattern recognition, pp. 9653-9663, 2022b.
Weihao Yu, Yuanhao Cai, Ruyi Zha, Zhiwen Fan, Chenxin Li, and Yixuan Yuan. X gaasssssn::n:: 4d radiative gaussian splatting for continuous-tima tomographic reconstruction. arXiv preprint arXiv:2503.21779 2025.
12

---

Published as a conference paper at ICLR 2026
Ruyi Zha, Tao Jun Lin, Yuanhao Cai, Jiwen Cao, Yanhao Zhang, and Hongdong Li. R 2- gaussian: Rectifying radiative gaussian splatting for tomographic reconstruction. arXiv preprint arXiv:2405.20693, 2024.
Hong-Yu Yu Zhou, Chixiang Lu, Chaoqi Chen, Sibei Yang, and Yizhou Yu. A unified visual infor- mation preservation framework for self-supervised pre-training in medical image analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(():8020-880) -0350 2023a.
Lei Zhou, Huidong Liu, Joseph Bae, Junjun He, Dimitris Samaras, and Prateek Prasanna. Self pre- training with masked autoencoders for medical image classification and segmentation. In 2023 IEEE 20th international symposium on biomedical imaging (ISBI), pp. 1-6. IEEE, 2023b.
S Kevin Zhou, Hayit Greenspan, and Dinggang Shen. Deep learning for medical image analysis. Academic Press, 2023c.
Zongwei Zhou, Vatsal Sodha, Jiaxuan Pang, Michael B Gotway, and Jianming Liang, Models genesis. Medical image analysis, 67:101840, 2021.
APPENDIX
7.1 THE USE OF LARGE LANGUAGE MODELS
Large language models (LLMs) were employed solely to improve the clarity and readability of the manuscript. They were not involved in the conception the study, the design or execution of experiments, the analysis or interpretation of results, any other scientific aspect of this work.
7.2 ETHICS STATEMENT
This work involves the analysis of human CT scan data exclusively sourced from publicly available datasets. We acknowledge that the representativeness and potential biases present in these public datasets may influence the fairness and generalizability of our proposed model. We encourage fu- ture work to validate our methods on more diverse and representative datasets to ensure equitable healthcare outcomes. All datasets used in this study were previously collected with appropriate ethi- cal approvals and consent procedures as documented by the original data contributors. Beyond these considerations, have identified no additional ethical confficts concerns related to this research.
7.3 REPRODUCIBIILIT STATEMENT
To ensure the reproducibility of our work, we have made the following efforts: All architectural details, hyperparameters, and training procedures for our proposed method are comprehensively de- scribed in Section 3.2 and Section 4.2 of the main paper. For comparative baseline results, we have directly reported performance metrics from the original publications to avoid potential inconsisten- cies that may arise from reimplementation differences, with proper citations provided throughout. Upon acceptance this paper, we commit to releasing the complete source code, including training scripts, model implementations, and evaluation, to facilitate full reproducibility of our results.
7.4 NETWORK ARCHITECTURE CONFIGURATION
Encoder Architecture (VIT Large)
Our MedGMAE employs Vision Transformer (ViT) Large configuration the encoder backbone. The detailed specifications are presented in Table 6.
The encoder processes 3D) medical images through the following pipeline:
Decoder Architecture (Lightweight Design)
The decoder employs lightweight Transformer architecture optimized for Gaussian parameter pre- diction, as detailed in Table 7.
Decoder Input Token Composition
13

---

Published as a conference paper at ICLR 2026
Table 6: ViT Large Encoder Configuration Details
Component
Configuration
Embedding Dimension
1536
Number Attention Heads
Number of Transformer Layers
12
MLP Ratio
4.0
Patch Size
16 16 16
12 x 12
Input Image Size
96 x 96 x 96
Number of Patches
512 (for 12*) or 216 (for 16
Dropout Rate
0.0
Attention Dropout Rate
0.0
Drop Path Rate
0.1
Table 7: Gaussian Decoder Configuration Details
Component
Configuration
Embedding Dimension
528
Number of Attention Heads
16
Number of Transformer Layers
8
MIP Ratio
4.0
Number of Gaussian Query Tokens
512
-eoodee-to-Decoder Projection
1536
- 528 Linear Layer
Dropout Rate
0.0
Attention Dropout Rate
0.0
Drop Path Rate
0.1
The decoder processes a carefully constructed sequence tokens:
Xatec =
1512 U (x
(5)
where:
Xats: Class token from encoder (1 token)
{q 1 Gaussian query tokens (512 tokens)
{=: Remaining visible patch tokens (127 tokens for 75% masking)
Total decoder input length: 1 1-211- 127 640 tokens.
7.5 GAUSSIAN PARAMETER PREDICTION HEADS
Four Specialized Prediction Heads
Each Gaussian is parameterized by an 1l II-dimensional vector comprising position, scale, rotation, and intensity, Four specialized linear heads predict these parameters:
Table 8: Gaussian Parameter Prediction Head Specifications
Parameter
Dimension
Activation
Range
Bias Init
Position (μ)
Sigmoid
[0, 1
0.0
Scale (s)
3
Sigmoid
00. 1]s
-1.386
Rotation (+)
4
L2 Normalize
Unit Quaternion
0.0
Density (a)
Sigmoid
[0, 1]
-0.405
Custom Initialization Strategy
14

---

Published as a conference paper at ICLR 2026
To ensure balanced parameter distributions across spatial dimensions, employ specialized ini-
tialization:
Position Head:
W ~ u(-v6/dd /6/d),
b = b 1.386
(6)
Scale Head:
W U( u6-v6/d, /6dd),
(7)
Rotation Head:
W ~ u( u(l-v6dd, /6dd,
b =
(8)
Density Head:
W u((-6//d 6/d),
b 0.405
The bias initialization ensures reasonable starting distributions:
Scale bias of -1,386 results in (-11.880) x 0.2 after sigmoid activation
Density bias of -0.405 results in a 0.405) 0.5 after sigmoid activation
7.6
DIFFERENTIABLE GAUSSIAN RENDERING ALGORITHM
CUDA Implementation Details
Our CUDA implementation employs several optimization strategies:
CUDA Gaussian Rendering Kernel [1] Gaussian parameters {Mi, 8i; Oi, az) Grid points J. Pixel mask M Rendered intensity grid I Initialize shared memory buffers for covariance 1) < N Load Gaussian parameters into shared memory Compute bounding box using 2o rule: expand, = 2.0 x si,d x grid size, for d {x;, y, bounds, [Mad expand Mi,d expand,) each voxel Xy in bounding box M[j] (masked region) Compute Ax Xj μ4 Compute power -0.5Ax Σ Ax intensity = a; exp(power) atomicAdd([[j),, inteesttttt
matrices and centers gaussian atomicAdd
Sparse Rendering Optimization
For masked regions, we implement sparse rendering that only computes intensities for required pixels:
[h!] Sparse Gaussian Rendering [1] Sparse grid points { LM (only masked pixels) Gaussian
parameters {pi St, d., a, J.V Sparse intensity values {4, { each sparse point in parallel L each Gaussian Check point x, within 2a bounds of Gaussian within bounds Compute intensity contribution and add to
This sparse approach reduces computational complexity from O x HxWa x D) to O(N x M)
where M is the number of masked pixels (typically 0.75 x H X W D).
7.7
TRAINING CONFIGURATION
The training parameters are shown in Table 9,
7.8 ADDITIONAL EXPERIMENTAL RESULTS
Downstream Classification Performance on CT-RATE Dataset
CT Reconstruction Performance Analysis
Figure demonstrates the superior performance of MedGMAE in accelerating CT reconstruction convergence. Our method shows significant improvements across different projection views (80, 120, and 160 projections), with MedGMAE achieving faster convergence and better reconstruction quality compared to the baseline 3DGR method.
15

---

Published as a conference paper at ICLR 2026
Table 9: Training Hyperparameters
Parameter
Value
Batch Size
8
Learning Rate
1 x 10-5
Weight Decay
0.05
Optimizer
AdamW
Learning Rate Schedule
Cosine Annealing
Warmup Steps
2000
Max Training Steps
100000
Gradient Clipping
1.0
Gaussian Parameters
Number of Gaussians
512
Maximum Scale
0.5
Temperature (T)
0.5
Medical moterial
Emphysema 0.75
tymphadenooaatty
Arterial wall calcfication
Perivonchial thissening
0.71
Attlectasis
6/72
0.8
0.e9
035
0,62
6.55
Lung nodule
Hitel hemis
Cardomegily
Mosalc atteruation pattem
a30
0.00
887
Coronary artery
Pericarcial effusion
Pleural
Pulmonery sequela
Unter
VoCo
VoCo 160k
Swinunter
SUP
MedGMME
Figure 5: Classification performance comparison on CT-RATE dataset. The radar charts show the Area Under Curve (AUC) scores for different disease categories.
Table 10: Ablation study on pre-training mask ratio for Segmentation tasks. All models use K=512 Gaussian points. rL2 error computed on reconstruction quality. AMOS and SegTHOR show segmentation Dice scores (%). The best results are in bold.
Mask Ratio
rL2 error AMOS(100%)
SegTHOR(100%)
25%
0.0031
80.03
86.03
50%
0,0068
79.01
86.00
75%
0.0117
84.90
89.15
85%
0.0147
85.17
89.17
Table 11: Ablation study on pre-training Gaussian Numbers (K) for Segmentation tasks. All models use pre-training mask ratio of 75%. rL.2 error is computed on reconstruction quality. AMOS and SegTHOR show segmentation Dice scores (%). The best results are in bold.
K
rL2 error
AMOS(100%)
SegTHOR(100%)
256
0.0143
79.99
85,82
512
0.0117
84.90
89.15
768
0.0144
79.33
85,40
1024
0.0137
78.87
85,89
16

---

Published as a conference paper at ICLR 2026
SDGR
3DGR
MedGMAE MedGMME*
800
McdGMAE
594
MedGMAE
700
Total Training (min)
50
45.2
45,
45.1
46.8
45.8
600
507
PSNR FULL
44.6
43.944.1
397
368
400
303
357
333
300
251
200
35
100
View
View
3DGR
3DGR
2500
MedGMAE
MedGMME
1400-
1150
Iterations (PSNR -35dB)
1711
1140
2000
Iterations (SSIM-90%)
1200
1100
980
967
1000
1500
820
780
1135
800
990
1000
920
600
500
120
View
View
Figure 6: CT reconstruction performance comparison across different projection views. Top row shows PSNR convergence, training time, iterations to reach PSNR=35dB, and iterations to reach SSIM=90% Bottom row presents the convergence curves and visual reconstruction quality dif- ferent iteration stages (500, 900, 3.4k, 15k iterations) for MedGMAE, MedGMAE* and 3DGR methods.
Table 12: Ablation study on pre-training mask ratio for zero-shot inference reconstruction. All models use K=512 Gaussian points. Values are reported as mean standard deviation. The best results in bold.
Mask Ratio
Time(min)
iter(P233)
iter(S290%)
PSNR(full)
SSIM(fulI)
25%
541.5
1250.0+158.1
1050.0=70.7
44.642.02
98.4+0.39
50%
572.4 38..2
1220.0 122.9
1040.0 15..6
45.3
98.5
75%
357.0 =2200
1040.0 191.7
980.0 160.0
46.2
98.5
85%
545.9_56.3
1050.0485.0
1000.0
46.0+
98.3
Table 13: Ablation study on pre-training Gaussian Numbers (K) for zero-shot inference reconstruc-
tion. All models use pre-training mask ratio of 75%. Values are reported as mean + standard deviation. The best results are in bold.
K
Time(min)
iter(P235)
iter(S>90%)
PSNR(full)
SSIM(full)
256
498.8
1220.0 113.5
1080.0
45.8+1.44
98.444..4
512
357.0+22.0
0440.0+91.7
980.0
46.2
98.5
768
488.4 488.
1140.0=107..
1040.0 1040.0-69.9
45.19
98.3+0.32
1024
546.7+78.7
1070.0-133.7
920.0
45.2+1.20
98.5+0.31
7.9
HYPERPARAMETER SENSITIVITY ANALYSIS
To address concerns about parameter selection and validate the robustness of approach across different configurations, we conduct systematic ablation studies on two critical hyperparameters: mask ratio and Gaussian primitive count.
17

---

Published as a conference paper at ICLR 2026
Effect of Mask Ratio. Tables 10 and 12 jointly investigate the impact of mask ratio both down- stream transfer learning and zero-shot CT reconstruction capabilities. We evaluate four mask ratios (25%, 50%, 75%, 85%) while fixing the number of Gaussian primitives at K=512.
For downstream segmentation tasks (Table 10), the reconstruction quality, measured by recon- struction L2 (rl.2) error on masked regions, naturally degrades as the mask ratio increases from 0.0031 at 25% to 0.0147 at 85% masking. However, downstream performance on both AMOS and SeOTHOR tells different story: segmentation scores improve significantly from 25% to 75% mask- ing (AMOS: 80.03%-844 90%% SegTHOR:
15%), then plateau at 85%. This trend demonstrates that higher mask ratios force the model to learn more robust and generalizable repre- sentations rather than merely memorizing local patterns. Notably, while lower mask ratios (25%, 50%) achieve better reconstruction metrics, they fail to learn representations that transfer effectively to downstream tasks.
For zero-shot CT reconstruction (Table 12), 75% masking achieves the best overall balance: it requires significantly shorter training time and fewer iterations to reach quality thresholds (PSNR235dB and SSIM290%% compared to lower mask ratios, while maintaining competitive fi- nal reconstruction quality (PSNR: 46.2.t1.17, SSIM: 9..5... Lower mask ratios require longer training despite achieving marginally better final metrics, suggesting they learn less efficient initial- ization priors. The 85% mask ratio converges slightly faster to SSIM>90% but results in lower final quality, indicating potential under-training of the decoder during pre-training. These results consis- tently demonstrate that 75% masking provides the optimal balance across both transfer learning and reconstruction tasks, aligning with findings in natural image MAE methods.
Effect of Gaussian Primitive Number. Tables 11 and 13 examine how the number of Gaussian primitives K affects both transfer performance and reconstruction initialization. We test four config- urations (K 256, 512, 768, 1024) while maintaining the 75% mask ratio.
For downstream segmentation (Table 11), reconstruction quality (rL2 error) shows relatively sta- ble performance across different K values (0.0137-0.0144), suggesting that even moderate numbers of Gaussian primitives can adequately capture volumetric anatomy. However, downstream perfor- mance exhibits a clear preference for K=512, achieving the best results on both AMOS (84.90%) and SegTHOR (89.15%), outperforming both smaller and larger configurations. This non-monotonic re- lationship suggests that K=512 provides an optimal balance: sufficient capacity to model anatomical complexity without introducing excessive parameters.
For zero-shot CT reconstruction (Table 13), K=512 consistently demonstrates superior efficiency and quality: it achieves the fastest training convergence and reaches quality thresholds with fewer iterations, while delivering the best final reconstruction metrics (PSNR: 46.2 SSIM: 9..5+..29). Smaller configurations (K=256) require longer training and more iterations despite achieving similar final quality, suggesting insufficient geometric prior learning. Larger configura- tions (K=768, 1024) also show degraded efficiency despite having more representational capacity. This counter-intuitive result indicates that excessive Gaussian primitives may introduce redundancy and optimization challenges that outweigh the benefits of increased capacity. The superior per- formance of K=512 across both tasks confirms that our method has learned an effective balance between geometric expressiveness and computational efficiency.
Key Findings. These comprehensive ablation studies validate our design choices and demonstrate the robustness of MedGMAE across different configurations for the datasets that we used during experiments: (1) The 75% mask ratio consistently emerges as near optimal for both transfer learning and zero-shot reconstruction, balancing reconstruction quality with learned representation quality; trade-off, avoiding both
(2) "K=512 Gaussian primitives" provides the best under-paree callerparamettrizaaiinn and rreeparramttrizaaton;; (3) Our method's performance remains stable within reasonable hyperparameter ranges.
7.10 RADIAL POWER SPECTRUM ANALYSIS
Frequency-Domain Analysis. We performed comprehensive spectral analysis on 50 volumes from the BTCV dataset. For each volume: (1) Ground Truth spectrum: 3D FFT followed by radial binning into 80 frequency bins (normalized frequency range 0-0.5); (2) Voxel MAE reconstruc- tion: obtained using our pre-trained voxel- masked autoencoder via sliding window inference
18

---

Published as a conference paper at ICLR 2026
(ROI: 963, overlap: 0.5); (3) Gaussian MAE reconstruction: obtained using pre-trained Gaus- sian decoder with differentiable splatting-bar based rendering. All volumes were center-cropped or zero-padded to 963 resolution before FFT computation ensure consistent frequency binning. We visualized average power spectral density across 10 randomly selected samples with #1 standard deviation error bands (Fig. 7), and computed spectral 1.2 distance averaged across all 50 samples (Mean: Voxel=0. 120:=00.55 Gaussian=0. 129::0000001220000000 129+0.018 p=0.12, paired t-test).
Average Power Spectrum (n=10 samples)
Ground Truth
10"
Visel MAP
Power Spectral Density (log scale)
Gaussian MAE
10-2
10.
10
10:10
10-12
0.0
0.1
0.2
0.3
0.4 0.5
Normalized Frequency (w)
Figure 7: Radial power spectrum comparison across 10 validation samples from BTCV dataset. Average power spectral density curves for Ground Truth (black), Voxel MAE (red), and Gaussian MAE (green) are shown with + standard deviation error bands (shaded regions). Both reconstruction methods accurately preserve the dominant low-frequency components (w I 0.3), which contain over 99% of the spectral energy. the high-frequency regime (w 0.3, marked by the dashed vertical line), Gaussian MAE demonstrates superior stability with 25% lower variance (std: 0.0031) compared to Voxel MAE (std: 0.0041), exhibiting smoother spectral decay that more closely follows the ground truth distribution. This validates that Gaussian splatting- based parameterization naturally suppresses high-frequency noise artifacts while maintaining anatomical structure fidelity.
7.11 ORGAN-SPECIFIC GAUSSIAN PRIMITIVE ANALYSIS
To investigate how our Gaussian-based decoder adapts to different anatomical structures, we ana- lyzed the distribution of learned Gaussian primitives across major organ types. We selected eight representative organs from the BTCV dataset covering diverse anatomical characteristics: large solid organs (liver), hollow organs (stomach), paired organs (kidneys), elongated structures (pancreas), tubular structures (aorta), and small organs (gallbladder). For each organ, Gaussian primitives were assigned based on their center positions relative to the ground truth segmentation masks.
Table 14 presents the statistics of Gaussian primitives across 10 validation samples. Several key observations emerge from this analysis:
Adaptive primitive allocation. The number of Gaussian primitives scales proportionally with organ volume. The liver, occupying 60.9% of total organ volume, is represented by 1665=747 primitives on average, while the gallbladder (1.0% volume) requires only 26 16 primitives. This demon- strates that our decoder automatically allocates representational capacity according to anatomical complexity without explicit supervision.
Radius adaptation to organ geometry. Mean Gaussian radii vary systematically across organ types. Small, compact organs like the gallbladder exhibit larger radii (0.053) enabling ef- ficient coverage with fewer primitives. In contrast, organs with complex morphology such as the spleen (0.097=0.031) employ smaller, more granular Gaussians to capture fine- ine-grained structural details. This self-adaptive behavior emerges naturally from the reconstruction objective without geometric priors.
19

---

Published as a conference paper at ICLR 2026
Table 14: Gaussian primitive statistics organ type across 10 BTCV validation samples. Volume percentage is relative to total organ volume sample. Mcan radius and intensity values are aver- aged across all primitives within each organ region.
Organ
Volume
Gaussians (count)
Mean Radius
Mean Intensity
(% of total)
(normalized)
(0-1)
Liver
60.9%
1665 + 747
0.128 + 0.006
0.739 + 0.052
Stomach
16.6%
502 + 332
0.117 + 0.014
0.666 + 0.109
Spleen
6.3%
134 + 103
0.097 + 0.031
0.469 = 0.079
Right Kidney
4.8%
148 +
0.139 0.013
0.770 0.057
Left Kidney
4.6%
175 73
0.013
0.514 = 0.077
Aorta
3.5%
90 109
0.126 0.025
0.753 0.095
Pancreas
2.4%
82 + 16
0.134 0.020
0.736 + 0.068
Gallbladder 1.0%
26 +
0.153 + 0.047
0.704 0.192
Consistent spatial distribution. Notably, the spatial density of primitives (number per mm3 not shown in table) remains relatively uniform across organs (0.0015-0.0023 primitives/mm"", indicat- ing that our method does not require higher primitive density for smaller organs. This uniform coverage suggests that the decoder has learned an efficient and scale-invariant representation strat- egy.
These findings validate that Gaussian splatting-based decoders learn anatomically meaningful rep- resentations. The automatic adaptation of primitive count, radius, and intensity organ-specific characteristics demonstrates the flexibility of Gaussian parameterization for medical image recon- struction, without requiring architectural modifications organ-specific hyperparameters.
20


Reviews:
[{"review_id": "ZVBLGvc2QZ", "reviewer": "Reviewer_TUT3", "summary": "This paper proposes MedGMAE, a self-supervised pre-training framework for 3D medical images that replaces voxel-level masked reconstruction with prediction of sets of 3D Gaussian primitives followed by differentiable volumetric rendering. The encoder is ViT-based; a lightweight decoder emits k Gaussians which are rendered and compared to masked ground truth. An extended variant (MedGMAE\\*) adds hierarchical residual blocks for coarse-to-fine Gaussian refinement. The authors claim that Gaussian reconstruction better matches anatomical continuity, yields a transferable decoder that can zero-shot initialize 3D Gaussian–based CT reconstruction, and is parameter efficient versus voxel MIM. Experiments show gains on segmentation, classification, registration, and faster convergence for low-dose CT reconstruction when using the decoder as an initializer.", "questions": "Please refer to the weaknesses part, especially weakness 2&3.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. The paper precisely formulates predicting Gaussian sets, detailing tokenization, query tokens, parameter heads, activations/normalization, and rendering, including initialization choices for stability.\n2. The motivation is well-argued, and the limitations of voxel-level objectives for continuous anatomy and the sparsity/efficiency argument for Gaussians are articulated, with a schematic overview (Fig. 1) that connects sparsity to parameter reduction and speedup. \n3. On segmentation with 1% labels, MedGMAE outperforms prior best notably on AMOS and FLARE’22 ; classification and registration show competitive/better results as well.  The ablation studies further support the claim.", "weaknesses": "1. The related work briefly contrasts with 2D Gaussian MAE (Rajasegaran et al., 2025) and several medical 3DGS papers, but head-to-head empirical comparisons versus the most relevant medical Gaussian pretext tasks (if any) or Gaussian-rendered proxy tasks are missing; the comparison set is largely voxel-MIM or contrastive SSL. This makes it hard to attribute the transfer gains specifically to Gaussian reconstruction rather than dataset/implementation choices.\n2. The paper states UNETR fine-tuning with consistent preprocessing, but various baselines load official pre-trained weights. Different pre-training corpora/compute may advantage some methods. I suggest the authors to add an explicit data-scale or compute-matched comparison.\n3. The method fixes yet there is no systematic study of k on transfer and reconstruction. The claim of 99.25% parameter reduction relies on a specific setting; a sensitivity sweep would help validate efficiency-generalization tradeoffs.\n4. The main text lists very large pretraining settings (e.g., “batch size 192, 400K steps” vs. Appendix Table 9 “batch size 8, max steps 100K”), making it unclear which configuration corresponds to the reported results; this needs consolidation.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "9CkOQsMHXP", "reviewer": "Reviewer_w2RZ", "summary": "The paper introduces a method for self-supervised pretraining for medical volumes, based on 3D Gaussian primitives (instead of traditional volumetric representations). The idea is that the Gaussian primitives can capture long-range dependencies better than local voxel-based representations. Results are given for multiple datasets and tasks.", "questions": "- How does the method perform (both in terms of accuracy, and computation time) for larger volumes that can be common in practice?\n\n- How were hyperparameters chosen, and how do hyperparameters affect results?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- Good clear paper, explaining the context and method well.\n\n- The motivation for the method seems reasonable.\n\n- Extensive experiments are included for various datasets and tasks.\n\n- Computational requirements are explicitly discussed.", "weaknesses": "- The authors use 96x96x96 voxel cubes, which is relatively low-resolution. It remains unclear to me how the method would perform on higher resolution data. For example, medical CT data can be 512^3 up to 2048^3 (see e.g. [1]). Non-medical CT datasets can be even larger. One concern is whether, for higher resolutions, the 3D gaussians are still able to capture long-range correlations well, and required computation time does not become prohibitively large.\n\n- It is not clear how specific hyperparameters settings are chosen by the authors, and how results are affected by different choices for the hyperparameters. This is also try for comparison methods -- how are the hyperparameters chosen for these?\n\n- Results are not always presented clearly -- for example, in Table 3 the caption states that the best result is shown in bold, but TransMorph achieves the best Dice result but isn't bold.\n\n[1] Oostveen, L. J., Boedeker, K. L., Brink, M., Prokop, M., de Lange, F., & Sechopoulos, I. (2020). Physical evaluation of an ultra-high-resolution CT scanner. European radiology, 30(5), 2552-2560.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "kQM8yGZTGi", "reviewer": "Reviewer_Nwsv", "summary": "This paper introduces MedGMAE (Medical Gaussian Masked Autoencoder), a novel self-supervised learning framework for volumetric medical image representation that replaces conventional voxel-based masked autoencoding with reconstruction via 3D Gaussian primitives. Traditional 3D masked autoencoders reconstruct missing voxel intensities directly, which the authors argue fails to capture continuous anatomical geometry, produces non-transferable decoders, and wastes parameters due to sparse anatomical occupancy. MedGMAE instead predicts a set of continuous 3D Gaussian parameters—position, scale, rotation, and intensity—to represent anatomical structures compactly and coherently. The decoder serves a dual purpose: it not only supports self-supervised pretraining but also provides a zero-shot, geometry-aware initialization for 3D Gaussian Splatting (3DGS)–based CT reconstruction. The framework achieves 99% parameter reduction relative to voxel-based MAEs and enables a 1.39× speed-up in CT reconstruction convergence. Experimental results on multiple medical imaging benchmarks—AMOS, FLARE’22, BTCV, SegTHOR (segmentation); CT-RATE (classification); IXI/OASIS (registration); and AAPM-Mayo (reconstruction)—show that MedGMAE consistently outperforms state-of-the-art voxel-based and self-supervised baselines (e.g., MAE, HySparK, VoCo, SUP) in both data-efficient and full-data regimes", "questions": "- How does the number of Gaussian primitives (k) affect the trade-off between geometric fidelity and representation compactness? Have you explored adaptive Gaussian selection based on anatomical density?\n- Cite RAPTOR which does some work on volumetric scans using foundation models (https://arxiv.org/abs/2507.08254)\n- Can you provide quantitative evidence linking Gaussian coherence (e.g., overlap or smoothness metrics) to segmentation or registration performance?\n- How much computational overhead does the Gaussian rendering add compared to standard voxel decoders, particularly during pretraining?\n- Did you observe any degenerate Gaussian cases (e.g., extreme scales or vanishing densities), and how are these handled during training?\n- Could the method support partially observed 3D data (sparse-view, partial slices) as a more direct input?\n-", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- MedGMAE is the first to apply 3D Gaussian parameter prediction within a masked autoencoder framework for medical volumetric pretraining. This fundamentally shifts the pretraining objective from discrete voxel regression to continuous geometric modeling, aligning better with the smooth, anatomical nature of medical volumes.\n- The model architecture (Figure 2, p. 3) elegantly integrates a ViT encoder with a lightweight Gaussian decoder and differentiable volumetric renderer. The authors detail parameterization (position, scale, rotation, intensity), initialization schemes, and differentiable rendering—showing a mature engineering of the 3D Gaussian field within a self-supervised context.\n- Unlike typical masked autoencoders whose decoders are discarded post-pretraining, MedGMAE’s Gaussian decoder is explicitly transferable. It serves as a zero-shot initializer for downstream 3D Gaussian Splatting CT reconstruction, bridging pretraining and reconstruction in a unified formulation.", "weaknesses": "- While intuitive, the paper provides little formal reasoning or analysis explaining why Gaussian parameterization yields better anatomical representation. A discussion relating spatial frequency content or anatomical topology to Gaussian smoothness would strengthen the conceptual core.\n- The custom CUDA Gaussian renderer adds substantial implementation complexity, and while Appendix §7.5 details optimizations, performance and stability under different resolutions or anisotropic voxel spacing are not benchmarked.\n- The model produces geometric primitives, yet the paper stops short of analyzing their spatial or anatomical semantics", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "hSLrzfuJqC", "reviewer": "Reviewer_fEUr", "summary": "The authors propose a new representation learning framework based on learning 3D Gaussian primitives, which are then used to render volumes through Gaussian splatting. They argue that this intermediate representation enables the decoder to capture anatomical priors, improving performance on downstream tasks such as CT volume reconstruction, medical image classification, segmentation, and registration. The results are shown on multiple datasets and compared with relevant baselines.", "questions": "The main question i have are :\n\n1. Why do the authors refer to the proposed approach as a “sparse representation”? Please clarify what is meant by “sparse” in this context.\n\n2. How does this method differ from “Gaussian Masked Autoencoders”? From my understanding, the approach appears to be largely identical, except for its application to 3D medical images. If there are substantive differences, the authors should provide a detailed comparison highlighting which components are new and which are retained from the original method.\n\n3. Since the code for “Gaussian Masked Autoencoders” has not been publicly released, did the authors collaborate with the original developers or otherwise verify that their implementation and assumptions are consistent with the original work?\n\n4. Please clarify the rationale behind using the same hyperparameters (e.g., masking ratio = 0.75) and other modeling choices. Were these parameters empirically validated for 3D medical data, or simply adopted from the original paper.", "limitations": "", "rating": 2, "confidence": 4, "soundness": 3, "presentation": 1, "contribution": 2, "strengths": "Strengths of the paper:\n\n1. The study includes a comprehensive evaluation across multiple medical downstream tasks, including CT volume reconstruction, image segmentation, disease classification, and deformable image registration, demonstrating the versatility and generalizability of the proposed representation learning framework.\n\n2. To the best of our knowledge, the use of 3D Gaussian primitives for representation learning in medical volumetric data has not been explored before. This work introduces a novel perspective by leveraging Gaussian primitives as an intermediate representation to capture anatomical structure and spatial context, enabling more effective learning for a range of medical imaging tasks.", "weaknesses": "Weakness of the paper:\n\n1. One major weakness of the paper is the lack of methodological novelty. The proposed approach is essentially identical to that of [1], with substantial portions(equations, query tokens, masking ratio) of the methodology directly adapted from it, the only difference being its application to 3D medical volumes instead of 2D images. Therefore, I assign a score of zero for the novelty of the proposed method.\n\n(a) The reasons mentioned in the introduction line 048, 052 are exactly the same reasons for [1]. \n\n(b) Regarding the term “sparse representation,” it is unclear how the authors justify calling their representation sparse. Even in [1], the learned representation is not sparse by definition. It may be described as disentangled, but not sparse. The use of this term is confusing and should be clarified or corrected by the authors.\n\n\n2. If i consider this a benchmark paper and not a methodology paper there are several things missing:\n\n(a) No where in the paper i found how many Gaussian's are used for representation learning ? What is the impact of number of gaussians? \n\n(b) What is the impact of different masking ratios ? As the authors mentioned in the paper that medical images have less textural changes compared to natural images, then why use the same masking ratio ? Feels illogical and uninformative.\n\n(c) MAE are only compared in Table 1, missing from other tables.\n\nIn my opinion, as a benchmark, the work is not comprehensive and requires further development to provide more meaningful or complete evaluations.\n\n(3) The code could have been released through an anonymized GitHub repository; however, this is a minor issue.\n\n\n[1] Rajasegaran, Jathushan, Xinlei Chen, Rulilong Li, Christoph Feichtenhofer, Jitendra Malik, and Shiry Ginosar. \"Gaussian masked autoencoders.\" arXiv preprint arXiv:2501.03229 (2025).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)