Uncertainty modeling for fine-tuned implicit functions
Authors: Anna Susmelj
Mael Macuglia
Natasa Tagasovska
Reto Sutter
Sebastiano Caprara
Jean-Philippe Thiran
Ender Konukoglu
Venue: ICLR 2025 Poster (2025)
Keywords: uncertainty
implicit functions
3D reconstruction
occupancy networks
Primary area: applications to computer vision, audio, language, and other modalities
TL;DR: We introduce Dropsembles, a method for uncertainty estimation in fine-tuned implicit functions, balancing accuracy and computational efficiency.

Abstract: Implicit functions such as Neural Radiance Fields (NeRFs), occupancy networks, and signed distance functions (SDFs) have become pivotal in computer vision for reconstructing detailed object shapes from sparse views. Achieving optimal performance with these models can be challenging due to the extreme sparsity of inputs and distribution shifts induced by data corruptions. To this end, large, noise-free synthetic datasets can serve as shape priors to help models fill in gaps, but the resulting reconstructions must be approached with caution. Uncertainty estimation is crucial for assessing the quality of these reconstructions, particularly in identifying areas where the model is uncertain about the parts it has inferred from the prior. In this paper, we introduce Dropsembles, a novel method for uncertainty estimation in tuned implicit functions. We demonstrate the efficacy of our approach through a series of experiments, starting with toy examples and progressing to a real-world scenario. Specifically, we train a Convolutional Occupancy Network on synthetic anatomical data and test it on low-resolution MRI segmentations of the lumbar spine. Our results show that Dropsembles achieve the accuracy and calibration levels of deep ensembles but with significantly less computational cost.

Published as a conference paper at ICLR 2025
UNCERTAINTY MODELING FOR FINE-TUNED IMPLICIT FUNCTIONS
Anna Susmelj ETH AI Center
Mael Macuglia ETH Zürich
Nataša Tagasovska Prescient/MLDD, Genentech
Reto Sutter, Sebastiano Caprara Balgrist University Hospital, USZ
Jean-Philippe Thiran LTS5, EPFL
Ender Konukoglu ETH Zürich
ABSTRACT
Implicit functions such as Neural Radiance Fields (NeRFs), occupancy networks, and signed distance functions (SDFs) have become pivotal in computer vision for reconstructing detailed object shapes from sparse views. Achieving optimal performance with these models can be challenging due to the extreme sparsity of inputs and distribution shifts induced by data corruptions. To this end, large, noise-free synthetic datasets can serve as shape priors help models fill in gaps, but the resulting reconstructions must be approached with caution. Uncertainty estimation is crucial for assessing the quality these reconstructions, particularly in identifying areas where the model is uncertain about the parts it has inferred from the prior. this paper, we introduce Dropsembles, a novel method for uncertainty estimation in tuned implicit functions. We demonstrate the efficacy of our approach through series of experiments, starting with toy examples and progressing real-world scenario. Specifically, train a Convolutional Oc- cupancy Network on synthetic anatomical data and test it on low-resolution MRI segmentations of the lumbar spine. Our results show that Dropsembles achieve the accuracy and calibration levels of deep ensembles but with significantly less computational cost.
INTRODUCTION
Recent advancements in neural implicit functions have facilitated their use for 3D object representa- tions and applications in novel views synthesis in computer vision. Neural Radiance Fields (NeRFs) Miidenhall et al.) 202T) have gained recognition for their ability to accurately synthesize novel views of complex scenes and became an important tool in applications of photorealistic rendering (Martin-Bruet Brualla et al., (021), such virtual reality and augmented reality. Signed Distance Func- tions (SDFs) (Park et al. [2019a] Gropp et al., 2020) is shown to be particularly useful in scenarios where precise boundary details are crucial, like industrial design and roboties. Occupancy Networks (Mescheder et al. 2019b),, which model shapes as a probabilistic grid of space occupancy, excel in handling topological variations, making them ideal for medical imaging (Amiranashvili and animation. Advancing this concept, Convolutional Occupancy Networks (Peng et al. 2020) in- tegrate convolutional networks enhance spatial learning, proving effective detailed architectural modeling and complex reconstructions.
2022)
In practice, achieving optimal performance with these methods often requires densely sampled input data and a high degree of similarity between the training data and the target object. However, such ideal conditions are rarely met in practical applications like augmented reality, virtual reality, au- tonomous driving, and medical contexts, where inputs are typically sparse and less precise (Truong cet al., 2023 Müller et al.) 2022a). For example, in medical applications, generating precise 3D representations from sparse inputs is particularly erucial for morphological analysis (Töthová al.) 2020). Patient-speciffe anatomical modeling significantly enhances the assessment of a patient's condition and aids in devising eustomized treatment plans (Turella et al. 2021).
In addition to sparsity, real-world data usually suffers from noise and corruption that leads to dis- tribution shifts with respect to the training data, and thus to significant performance degradation.

---

Published as a conference paper at ICLR 2025
Occlusion, noise, truncation, and lack of depth measurements (Liao et al. 2023) greatly affect recon- struction from monocular observations. Noise in estimations of camera poses inevitably degrades reconstruction from sparse views (Truong al.) 2023 Zhang et al. 2022 Chen al. 202 Un- der input sparsity and distribution shifts, in safety- -afettycritical applications, such as medical (Zou et al.) 2023) or autonomous driving (Kiran al., (02T), it is crucial that inferred information is transpar- ently disclosed to the end user, as it may significantly influence the decision -aaiinnnnmainng process. One approach to this end is to quantify uncertainty in the reconstruction. For example, (Shen et all] (2021) derived variational inference objective to model predictive distribution in NeRFs. However, similar results are yet be established for more generic applications of implicit functions.
To tackle the challenges of sparse and corrupted inputs, we propose to leverage dense, noise-free synthetic data for pre-training. For instance, a pretrained encoder can he shared across both syn- thetic and real datasets, while lightweight implicit decoder is initially trained on synthetic data and subsequently fine-tuned on the sparse, noisy real data. This process introduces two primary sources of uncertainty: from the encoder's latent representation and the fine-tuning of the decoder. In this paper, we focus on the latter and demonstrate that epistemic uncertainty is an important, yet under- studied source of error in implicit functions, which significantly impacts the predictions quality. To the best of our knowledge, this type of uncertainty quantification in 3D reconstruction for fine-tuned neural implicit functions has not been addressed in the literature.
Monte Carlo (MC) dropout (Gal and Ghahramani 2016) and deep ensembles (Lakshminarayanan et al. 2017) two commonly used baselines for estimating uncertainty in computer vision ap- plications (enddall and Gal, 2017), which could be readily applied to our task. MC dropout is simple to integrate and computationally efficient during training. However, it often underestimates uncertainty and requires multiple forward passes during inference, increasing computational cost (Postels al., (021). Deep ensembles involve training multiple neural networks independently and averaging their predictions, capturing a wider range of potential outputs. This method provides improved predictive performance and better-calibrated uncertainty estimates but computationally costly and memory-intensive This becomes especially demanding in our fine-tuning context, where each model in an ensemble needs to be trained on both datasets. Here we introduce Dropsembled method that creates ensembles based the dropout technique. Dropsembles aim moderate the computational demands associated with ensembles while attempting to maintain prediction accu- racy and uncertainty calibration of deep ensembles. Combined with Elastic Weight Consolidation (EWC) KKirpatrick et al., 2016). Dropsembles is able to mitigate distribution shifts between source and target datasets.
Contributions This paper introduces several contributions to address the gap in modeling uncer- tainties in fine-tuned neural implicit functions:
To the best of our knowledge, we are the first to model epistemic uncertainty in the implicit decoder during fine-tuning from synthetic data.
To this end, we introduce Dropsembles (overview in Figure to achieve the performance of vanilla ensembles with significantly reduced computation cost (subsection 4.1).
We introduce EWC-inspired regularization for task-agnostic uncertainty adaptation to take into account the distribution shift in uncertainty modeling (subsection 4.2).
We include a series of experiments to validate the proposed methods, in controlled bench- mark on synthetic data and in a real-world medical application. We demonstrate that it is possible to achieve high reconstruction quality and preserve patient-specific details when using synthetic data in the form of an anatomical atlas (section 5).
2 RELATED WORK
Implicit shape modeling from sparse input Much attention has recently been directed towards the
problem of novel view synthesis from sparse views enhancing this task using NeRFs (Truong et al,
2023 Chen et al. 2021a Deng et al. 2022 Jain al. 2021 Kim al. 2022 Niemeyer al. 2022
Roessle et al. 2022 Lin et al. 2021 Oechsle et al. 2021). SDFs (Yu et al.) 2022 Yarly et al. 2020
https::/gittub..cmm
2

---

Published as a conference paper at ICLR 2025
Datacet (sparse)
Doroert A
Step triningg deander
planse [ssmsa)
CAN vove!
Detaset (spors))
encouer
Sre turling decoder
Doloset
Aveder
Figure 1: Occupancy network training with a dense prior and fine-tuning sparse dataset.
and occupancy networks NNeemeyer et al. 2020). Implicit functions also found their applications
in medical imaging, particularly in 3D shape reconstruction from sparse MRI slices (Amiranashvilii
et al.) [022]] 2024) Gatti et al. 2024) McGinnis et al. 2023 Alblas et al.) 2023 Jacob et (023)). In
this paper, we focus on occupancy networks due to their relevance in medical imaging applications but note that Dropsembles can be used with other neural implicit functions.
Uncertainty modeling Estimating uncertainty deep models has attracted significant interest in recent years (Gal and Ghahramani] [016 Lakshminarayanan et al. 22117] Gawlikowski al. [023) remaining challenge here uncertainty estimation under distribution shifts, which in most real- world scenarios is essential for guaranteeing the reliability and resilience of predictions when fronted with out-of-distribution (OOD) examples (Tran et al.] 2022 Li et al. 2022: Wenzel et al.) 2022). In 3D reconstruction, this important since corruption and sparsity pattern test samples often deviates from those in training data. While numerous methods have been devised to detect
OOD cases (Malinin and Gales 2018 [Tagasovska and Lopez-Paz 2019 Ren et al. [019 Charpen- lier et al. 2020; Osband et al. 2023) or enhance accuracy in unobserved domains (Liu et al. 2021 Hendrycks al. 2022)) the adaptation of uncertainty estimates has been relatively unexplored. In
comprehensive benchmark study, Wenzel et al. (2022) find that from all metrics considered, cal- ibration transfers worst, meaning that models that are well calibrated on the training data are not necessarily well calibrated on OOD data. In an era dominated by L.MS and foundation models, addressing this issue becomes paramount, particularly given the tendency of such models for over- confidence a result of fine-tuning (Guo al.) 2017a Dodge al. 2020). To overcome such issues, [Balabanov and Linander 2024) introduces a method for estimating uncertainty fine-tuned LLMs using Low-Rank Adaptation. In computer vision, (Wortsman et al. 2022) highlights that fine-tuning comes at the cost of robustness, and address this issues by ensembling the weights the zero-shot and fine-tuned models. (Lu and Koniusz, (022) propose uncertainty estimation in one-shot object detection, particularly focusing on fine-tuned models.
Several works focused modeling uncertainty in the encoder of neural implicit representations. (Shen et al. 2022) models uncertainty in the color and density output of a scene-level neural rep- resentation with conditional normalizing flows. [Lio et al. (2023) uses this method in robotics applications. Liao and Waslander (2024) estimates uncertainty for neural object representation from monocular images by propagating it from image space first to latent space, and consequently to 3D object shape. Yang et al. (202)) focused on modeling aleatoric uncertainty in occupancy networks. Dropsembles can complement all of the above approaches by modeling the epistemic uncertainty in the weights of implicit functions throughout fine-tuning, rendering them more robust to distribution shifts. By doing so, improves both the performance and reliability of methods building on neural implicit representations.
BACKGROUND
3.1 IMPLICIT SHAPE REPRESENTATIONS
Implicit functions are widely used in computer vision to represent complex shapes. Assuming a continuous encoding of the object domain X, an SDF for surface S encoding shape is defined as
3

---

Published as a conference paper at ICLR 2025
f: R where: f(x) < 0 "inside" S f(x) = - Another popular way to repre- sent a shape object is an occupancy function g: - defined as g(x) = lishape Objec(2)) where I corresponds to the indicator function. The primary advantage of both approaches lies in their abil- ity to accurately model complex shapes from sparse observations using deep neural networks. This capability underpins methods such as "DeepSDF" (Park et al., 2019b) and "Occupancy Network" (Mescheder et al., 22119)). which approximate functions by non con-linearly regressing observations to the surface encoding the shape. To facilitate learning from training sets, these techniques are often employed in conjunction with latent representations to capture diverse shape variations and at infer- ence time yield continuous shape representations. The latent representations can be trained directly from data using autoencoders (Mescheder et al. (2019)) autodecoders (Park et al. 2019b). Recent research has also explored using large pretrained models like CLIP to obtain latent representations (Cheng et al. 2023). While the specific choice of the latent encoder is beyond the scope of this paper, we will refer to as an "oracle".
Both methods can be described with the same formulation. We consider dataset D ( Z
where N is the number of images, K is the number of available pix- iE[N]
els per image Z is a latent representation of image i given by the "oracle", x] E X' C R$ is the position corresponding to the observed value vj. which are obtained through an SDF voxel occu- pancy following the definitions above. In both settings, the regression function is a neural network fo(2, Z), 0 ONN- At training. networks are trained using the following optimization objective.
RD(0) N.K
Σ ())
JE(K] AE(N)
Here, the loss function / depends on the method. For occupancy networks. is the binary cross- entropy, reflecting the binary nature of occupancy, while the L2 norm for SDFs, aligning with the continuous nature of the distance function.
3.2 UNCERTAINTY MODELING
Deep Ensembles Deep Ensembles (Lakshminarayanan et al. 2017) apply ensemble methods to neural networks (Lee al. 2020 Breiman [200] Schapiree 1990, Saberian and Vasconcelos) 2010).
Multiple independent networks are trained to determine set of optimal weights [0thetar] with the subscript i denoting different networks. Ensembles mitigate the risk of selecting a single set of weights, which may not yield good results particularly when training data consist of fewer samples relative to the size of the parameter space. Instead, multiple solutions with comparable accuracy are determined, allowing the ensemble to average outputs and minimize the selection risk. Training of deep ensembles is associated with high computational demands due to training multiple networks.
Dropout Dropout (Srivastava et al., 2014 is a regularization technique that aims to reduce over- fitting by randomly omitting subsets of features during each training step. By "dropping out" (i.e.. setting to zero) a subset of activations within a network layer, it diminishes the network's reliance on specific neurons, encouraging the development of more robust features. Gal and Ghahramani (Gal and Ghahramani] 2016) demonstrated that dropout can also be interpreted from a Bayesian per- spective and applied toward modeling uncertainty. Dropout at test time (referred as Monte Carlo Dropout) is shown to perform approximate Bayesian inference, essentially through using random- ness in dropout configurations for uncertainty modeling.
3.3 ELASTIC WEIGHT CONSOLIDATION
Elastic Weight Consolidation (EWC) is a regularization technique introduced in continual learning to address catastrophic forgetting (Kirkpatrick et al.) 20166). The underlying principle is to protect parameters crucial for previous tasks while learning new ones. Assume two distinct tasks, A and B, with their respective datasets DA and DB. where D n DB = The tasks are learned sequentially without access to previous tasks' datasets. First, task A is learned by training neural network on DA resulting in a set of optimal weights 9 When learning task B using dataset Dg, EWC regu- larizes the weights so they remain within a region in the parameter space that led to good accuracy for task A.
4

---

Published as a conference paper at ICLR 2025
Next, during learning for task B, approximate posterior of weights obtained during learning for A constrains the optimization. A Gaussian approximation to log p(0) (see App. A for further details), serves the regularizer
B = arg min RD (0) + A(6 (()= 0^)
(2)
where Rog (0) corresponds to the likelihood term log p(D 0). À is hyperparameter, F is the diagonal of Fisher information matrix. Details can be found Appendix A
METHODS
We focus shape reconstruction from sparsely sampled and corrupted inputs using occupancy networks. While the proposed method can be applied to both SDF and occupancy networks, we focus on the latter for demonstration. Given the sparse and corrupted nature of our input, train an occupancy network with qugllty data and then fine-tune on a test sample. Through fine- tuning, expect the model adapt to the input while transferring the prior information captured in training.
Assume access to datasets D and DB, defined in Section 3.1 with the number of points per image KA and KB, and the number of images a and NB. The dataset Da is assumed to be high quality and "dense", while the dataset Dn is "sparse" in terms of the number of points observed per image and potentially contains corruptions in individual images, i.e., KA > KB. Additionally, it is as- sumed that the number of images D is greater than DB, i.e. NA NB (in our experiments, we consider the case of single image NB = 1). The two datasets can only be accessed successively and not simultaneously; that is, DA is accessed first, followed by DB, without further access to DA- Note that the latent representation vectors L³ are assumed to be obtained by the same learning oracle for both datasets D and DB described Section 3.1 Without loss of generality, in this paper, we consider the latent map L X Z to be a "frozen" encoder pretrained on dataset DA.
The underlying parametric function class of model is assumed to be a neural network So : X' Z + y approximating regression function described in Section 3.1 The parameter O represents the set of weight matrices of the network i.c := {Wi where L corresponds to the number layers. The output space y corresponds to [o, 1] for occupancy networks and R for signed distance functions. It is important to note that the versatility of the network model in terms of its output space is tailored to the specific modeling task. However, this flexibility does not limit the proposed method, which remains versatile across different tasks.
The procedures outlined in Sections 4.1 and 4.2 involve two stages of training: initially on dataset DA. denoted as Task A, followed by training dataset D referred to as Task B. The primary ob- jective is to achieve high prediction accuracy in the second stage, while acknowledging the inherent challenges posed by the sparsity and smaller size of DB relative to DA-
4.1 DROPSEMBLES
Given the sparsity and corrupted nature of test samples, adaptation of the prior model is prone to un- certainties. Here we introduce Dropsembles, a technique that combines benefits of both dropout and deep ensembles, to capture and quantify this uncertainty. This approach aims to moderate the com- putational demands associated with ensembles while attempting to maintain reasonable prediction accuracy. Although ensembles are known for their reliable predictions, they are resource-intensive. In real-world applications, involving large training sets, this cost becomes a hindering factor.
In contrast, dropout involves training only single model instance. Applying dropout to neural network involves sampling a "thinned" network, effectively the same as applying a binary mask to the weights. However, unlike ensembles where each model is trained independently, dropout results in networks that are not independent; they share weights. Thus, training a neural network with dropout is akin to simultaneously training a collection of 2" thinned networks (n number of weights), all sharing a substantial portion of their weights. However, often this comes at a price of less accurate predictions.
In Dropsembles, the model is first trained with dropout on dataset D Subsequently, M thinned network (fo,, me[M] instances are generated by sampling binary masks. Each of these "thinned"
5

---

Published as a conference paper at ICLR 2025
instances is then fine-tuned dataset DB independently, effectively creating ensemble of "thinned" networks initialized with correlated weights. For inference and uncertainty estimations, this ensemble is treated as uniformly weighted mixture model, and the predictions aggregated in the same manner traditional ensembles: p(y|æ) M Σm Po.. (y/æ, 0m). Although en- sembles typically benefit from networks being large and independent, our experiments show that this relaxation of independence does significantly diminish prediction performance metries or expected calibration error. The overall training procedure is summarized in Algorithm
In the context of implicit functions, networks are optimized according to the objective defined in Equation (T). However, it is important to emphasize that Dropsembles is versatile framework that can be adapted to train with any objective necessary for specific task.
Algorithm Dropsembles
Di Task A
Require: DA A)) Train fo on DA with dropout
b Task B
Require: DD, (()
2: for m = to M do
3: Ofii < Sample a thinned network initialized from x))
4: fm +- arg ming AD,, (0)
TTrain thinned network on DB
5: end for
6: Obtain predictions and uncertainty estimates <- Ensemble (Ôm [ JME[M]
4.2 ELASTIC WEIGHT CONSOLIDATION REGULARIZATION FOR IMPLICIT SHAPE MODELING
In the basic version of optimization described above, networks start their training initialized from the learned posterior of dataset DA- However, there is no guarantee that during fine-tuning the network will not diverge arbitrarily from initial weights. This is particularly problematic for implicit shape modeling, where DA while adapting to sparse and noisy data DB to avoid overfitting. To address this concern, we borrow developments from the continual learning literature, as they seamlessly fit into the framework described above.
essential retain information from a large and dense prior dataset
In particular, when fine-tuning individual instances of thinned networks on dataset DB, we can apply the same reasoning to each network instance described in EWC. Thus, the learning objec- tive for part B is replaced by the objective described in (Equation 2). which includes an additional regularization term.
5 EXPERIMENTAL RESULTS
Our objective is to provide trustworthy predictions on dataset B without hurting performance. We do so by modeling the uncertainty of the weights of fine-tuned model. In standard prediction tasks, models are evaluated with respect to accuracy and Expected Calibration Error (ECE). However, our specific setup (reconstruction from sparse views) calls for additional evaluation metries suitable to computer vision tasks, such as Dice Score Coefficient (DSC) and Hausdorff distance. Besides ECE, we include reliability diagrams (Guo et al.) 2017b) given our preference for a more conservative modeling approach. Details evaluation metrics can be found in Appendix B
models were evaluated using four network instances. Training details are provided Appendix C The optimal regularization parameter for EWC was determined through an ablation study pre- sented in Appendix D Additional ablation results on the number network instances are included
in Appendix
5.1 CLASSIFICATION UNDER DISTRIBUTION SHIFT
Toy dataset We first demonstrate our method on a toy data set for binary classification. We gener- ated two-dimensional datasets A and B with sinusoidal decision boundary and Gaussian noise. A

---

Published as a conference paper at ICLR 2025
Thaining
MC diopoun
Dropoembles
Ensemble
biundary
Utcersvity
6
MC drapout
1,00
1525
dropout LWC
Dropaemblle EWC
Ena Ensembles Ewc
1250
130
100
Figure 2: Toy classification example. a) Training data for binary classification task ("red" vs "blue") from datasets A (points, light) and (crosses, dark) b) MC dropout trained only on Dataset A. c) Comparison of methods fine-tuned Dataset B. Points are colored by the predicted class. EWC consistently improves both accuracy and uncertainty estimates on both A and B datasets.
moderate distribution shift was modeled between the datasets by adjusting the support values. We created 1000 train samples for dataset A, and train samples for dataset B. All metrics Table were averaged across 3 random seeds. From the evaluations (Table [ Figure 2 it is apparent that EWC improves the performance of Ensembles and Dropsembles on both datasets.
Table Comparison of methods fine-tuned on dataset B toy classification, MNIST and ShapeNet reconstruction experiments. Values shown in percentages [%] for all metrics.
Toy classifiation
MNIST
ShapeNet
Method
Ace-A
ECEA
Acc-B
BEEB 1
DSC
BCB L
mUSC ?
BCB L
MCAmppott
59.0 +: 5.3
40.0 4.3
90.5
0.6
11.5 1.7
54.8 6.9
8.6 + 2.0
82.8 t 2.2
19.31
= 3.97
Diopiembles
59.2 4.3
39.8 4.3
90.5
0.6
9.2 1.3
64.1 +
6.1 2.7
83.5 + 2.1
19.51
4.64
Easembles
MCdropoit
66.8
5.6
33.0 4.8
10.5
56.7
2.2
83.9
16.6
Dropeembles DWC
96.2
4.2
5.3
93.5
= 3.8
2.0
70.3 = 1.7
6.11
84,3
+ 2.1
17.67
= 4.45
Easembles EWC
95.8
: 2.5
7.5 2.4
95.5
1.3
53
69.3 3.8
5.2 2.0
83.4
1
24.59
2.98
5.2 IMPLICIT SHAPE MODELING
MNIST digit reconstruction In next experiment, explore the reconstruction from sparse inputs using the MNIST dataset. To this end, we converted the images into binary masks through thresholding. To simulate sparse input conditions, out every third row and column. This masking strategy was consistently applied across both datasets A and For dataset A, utilized all images for a single digit "7" from the MNIST training
applied grid mask the images, masking
To introduce a moderate distribution shift and mimic common real-world dataset corruptions, we rotated the images in dataset by 15 degrees and obscured approximately 15 percent of the original pixels. In this section we are focusing on shape reconstruction, therefore "fine-tuning" and "testing" are applied per image and not per dataset. Given that the occupancy network is trained at the pixel level, each of these images effectively constitutes an individual "dataset B". We randomly selected 20 distinct images from the MNIST test split, which would give us 20 different variations of "dataset B". Fine-tuning and evaluating the whole test split of MNIST dataset would be computationally demanding, as each occupancy network fine-tuned on an individual image.
Our occupancy network comprises 8-layer MLP, designed to process the latent representation along with the 2-dimensional coordinates each pixel, thereby facilitating pixel-wise predictions. For configurations utilizing dropout, we incorporated dropout layer with a probability of p 0.3 following each linear layer in the network.
We demonstrate qualitative predictions and associated uncertainties for single-image example in Figure 3 Visual inspection of the uncertainty estimates reveals that EWC-regularized ensembles exhibit the desired behavior: they not only deliver accurate predictions but also provide conservative

---

Published as a conference paper at ICLR 2025
Drsprembles Fred ctinn
put
Doigo
Uncortaiity
Predicion
Unserisitly
Prediction
Uncortaney
Unoetaary
Prediction
Uncerraliyy
7
7
7
7
Datases
Sparse
Dence
Expe
Figure 3: Corrupted MNIST reconstruction example. a) Example of training images. b) Comparison of fine-tuned methods dataset B. "perfectly calibrated" method would have reliability diagrams aligned on the diagonal. A "good conservative" method would have all bars above the diagonal.
uncertainty estimates, particularly noting high uncertainty in regions with data corruption. Relia- bility diagrams, along with quantitative evaluations presented in Table confirm that elastic reg- ularization enhances the performance across all methods. Notably, EWC-regularized Dropsembles achieve performance comparable that of the Ensembles but with significantly reduced resource usage.
NhtpeNet We demonstrate the applicability of Dropsembles for modeling uncertainty in SDFs a commonly used ShapeNet dataset. For dataset A, we utilized 1,000 randomly selected airplanc shapes. For dataset we selected 10 unseen airplane shapes and introduced distribution shift by applying random erosion as corruption. Encoder inputs were provided as occupancy, with the target SDF estimated from the original meshes. For the corrupted dataset, the SDF was derived by con- verting the corrupted occupancies into mesh and then estimating the SDF. The decoder follows the DeepSDF architecture with dropout probability of = 0.2 where applicable. Quantitative results demonstrate that Dropsembles outperform other methods in terms of DSC, with EWC regularization providing further improvements both DSC and ECE metrics across all methods. Although MC dropout achieves the lowest ECE score, it fails to preserve shape smoothness when used for uncertainty modeling rather than regularization, as highlighted by additional qualitative results in Appendix Figure 11
in Table T
Recently, non- RenL activation functions have gained popularity in neural implicit architectures (Sitzmann al. 2020 Tancik et al., 2020 Müller et al. 2022b Mehta al. 202T Dupont al., 2022). In Appendix Table 4 we demonstrate that Dropsembles can be effectively combined with periodic activation functions without any loss in performance.
Lumbar spine In this section, we adapt the data preparation procedure from (Turella et al. 2021) but replace the high-quality CT dataset with synthetic dataset of anatomical shapes. As anatomi- cal shape prior, we utilize a rigged anatomical model from "TurboSquid".. To model patient-specific variability and variations in poses during MRI acquisition, we generated 94 rigged deformations. Point-cloud models from the atlas were converted to voxels at 256 voxel resolution in order to corre- spond to the resolution of the target MRI dataset. We further applied random elastic deformations to mimic patient-specing shape variability. We created paired "sparse" "dense" dataset by selecting a consistent set of 17-21 sagittal slices and additionally applied two iterations of connected erosions to simulate patient-specific automatic MR segmentations (Turella et al., 2021). A bicubic upsam- pling was employed on the "sparse" inputs to adapt them for the encoder. We trained ReconNet Turella et al. (222I) the entire training split of the atlas dataset to obtain subsequently kept frozen. For the implicit decoder, we employed only one the rigged samples to accurately quantify performance
encoder, which
synthetic data and demonstrate the method's robustness to medium and strong distribution shifts. For real-world applications, we recommend using the entire rigged transformations dataset.
For the occupancy network, we employed the MLP architecture described by (Amiranashvili et al. 2022), which features eight linear layers, each of 128 dimensions and dropout p 0.2. The initial
8

---

Published as a conference paper at ICLR 2025
training of the occupancy network was conducted on anatomical atlas. This intensive train- ing process required 48 hours on a single NVIDIA RTX A6000 GPU, equipped with 48 GB of memory. This pretrained network models. Additionally, we trained four distinct instances of Ensembles without dropout, with each instance requiring 48 hours of training. A key distinction between Dropsembles and Ensembles is that Dropsembles leverage a single pretrained model from dataset A, significantly reducing compu- tational demand by a factor equal the number of thinned networks. This results substantial difference in total training time, the pre-training on dataset A is much more time-consuming than the fine-tuning on dataset (see Table Appendix(E). Additionally, this training time on dataset A remains constant for Dropsembles as the number of thinned networks increases, while it grows linearly for Ensembles.
then used to fine-tune both Dropsembles and MC dropout
As the target dataset B, we use publicly available dataset of MR+CT images from 20 subjects (Cai et al., 2015). We employed the same segmentation network as (Turella al. (02T) to obtain segmentations of 5 vertebrae, 5 discs, and the spinal canal (Pang et al. 2020). These automatically generated segmentations of high-quality MRI samples were used as the ground truth for the sparse 3D reconstruction task. To create sparse inputs, removed the same set of sagittal slices as in the atlas dataset. We randomly selected 3 subjects for the consequent fine-tuning and testing.
The MR dataset described above lacks ground truth segmentations, which complicates numerical evaluation. The goal of the model is to impute missing or misclassified parts using anatomical atlas
Uncerteinty
Samples
Spanse input
Dense
Dropsemblee EWC prodiction
MC dropout
Dropsembles a EWC
MC dropout
Dropsembles + EWC
Uncertainty histogram
dropout
Dropsembles EWC
10.0
75
5.-
2.5
055
1.0
009 Sants
1.2
unsercartty
Fmage
Figure 4: Lumbar spine reconstruction example on Subject 2. a) 3D-rendered views of sparse inputs (bicubic upsampling), dense ground truth (GT), and predictions by our method. b) Histograms of uncertainty (entropy) values, truncated between 0.1 and I for visibility. c) Examples of uncertainty estimates for two different sagittal slices of the 3D volume. d) For each method, network predictions randomly sampled and the corresponding reconstructions are depicted.
priors, which, while enhancing reconstruction, are typically marked as incorrect in standard seg- mentation metrics. To facilitate more accurate assessment of reconstruction metrics, we introduce an intermediate benchmark. For this purpose, we utilize three random rigged deformations from the atlas that were not exposed during training and apply a doubled level of erosions. This strong aug- mentation provides a challenging input for dataset B and allows to model significant distribution shift aimed at rigorously testing the model under adverse conditions.
Numerical evaluation in Table 2 demonstrates consistent improvement of our method upon MC- dropout and comparable performance to Ensembles. In order to investigate the performance of the proposed method on MR dataset, perform detailed qualitative analysis in Figure A consistent and distinct pattern not captured by the metrics alone, stands out from the qualitative assessment: MC-dropout tends to produce noisy reconstructions, illustrated in Figure 4--d, and generally

---

Published as a conference paper at ICLR 2025
yields predictions that lack coherent, continuous shapes. This behaviour is partially captured in metric "DSC avg" in - dice score evaluated on individual samples of the network instances. This observation is notable because, although models trained with conventional dropout generate reasonable predictions, their use uncertainty estimation undermines the fundamental objective of 3D modeling. Samples drawn from the dropout distribution do not yield plausible shapes, as evidenced in Figuregi. In contrast, our model, which draws from deep ensembles, does not exhibit these limitations. Furthermore, increasing the number of samples in MC-dropout does not resolve this issue but rather leads to higher computational demands, as documented in Table 2
EWC demonstrates only minor improvements in terms of Dice score, while the difference Haus- dorff distance is more noticeable. We believe this is because Hausdorff distance better captures small discrepancies that the Dice score overlooks. Dice less sensitive to small differences in reconstruc- tion, as the majority of structures are well reconstructed by both methods. Qualitative inspection in Figure 5 confirms these subtle variations, which are crucial in medical applications where even small differences can significantly impact treatment planning.
Table 2: Comparison of methods fine-tuned dataset B of lumbar spine experiment. Evaluations are performed on the corrupted atlas. Metrics are reported for dataset B. Baseline is a MC-dropout model trained dataset A (w/o fine-tuning).
Matoed
DSC [
DSC
Tmin-A
[b]
Baseline
65.0 1 2.0
63.9 1.8
17.5 +: 1.2
25.2 2: 0.4
48/-
MCdropout
Dropsembles
86.8 1: 2.4
86.3 =: 2:3
11.4 +:
4.6 1.7
48/ 10
Dropsembles + EWC
$6.9 1: 2.4
86.4 1 2.3
10.9 2.1
4.5 1.7
48/ 11
Ensembles
2.9
10.5 + 2.8
2.2
192/
DISCUSSION AND CONCLUSION
Strengths In this study, advanced sparse 3D shape reconstruction for high-precision ap- plications by introducing uncertainty model- ing. We developed flexible framework that fa- cilitates uncertainty-aware fine-tuning of mod- els and showcased its utility in reconstructing the lumbar spine from sparse and corrupted MRI data. Our observations suggest that tra- ditional uncertainty methods like MC dropout not ideal for implicit shape reconstruction, as they tend to undermine the basic principles of implicit functions. However, our Dropsem- ble method effectively addresses these limita- tions, providing promising alternative for un- certainty modeling. Additionally, Dropsembles are significantly more efficient than Ensembles during the resource-heavy pre-training phase, allowing for substantial reductions in resource usage without significant loss in performance.
Dropsembles
Dropsembles
Dropsombles
Dropaembles
Figure 5: Example the performance difference between Dropsembles w/ and w/o EWC. Grey segmentations are produced by Dropsembles w/ w/o EWC regularization overlayed with color- ful ground-truth segmentations. EWC better cap- tures subtle details in modeling thin structures, such as vertebra processes.
Limitations During the fine-tuning stage, Dropsembles face the same computation de- mands as Ensemble models.
Future directions In this work. we focus on the epistemic uncertainty of the decoder, assuming the encoder to be frozen. An interesting future direction to combine Dropsembles with the uncertainty in the encoder to comprehensively cover all aspects of uncertainty modeling in sparse 3D shape reconstruction.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENTS
This study was financially supported by: Personalized Health and Related Technologies (PHRT), project number 222 and 643, ETH domain, 2. ETH AI Center.
REFERENCES
Dieuwertje Alblas, Marieke Hofman, Christoph Brune, Kak Khee Yeung, and Jelmer Wolterink. Implicit neural representations for modeling of abdominal aortic aneurysm progression. In Inter- national Conference on Functional Imaging and Modeling of the Heart, pages 356-365. Springer, 2023.
Tamaz Amiranashvili, David Lüdke, Hongwei Bran Li, Bjoern Menze, and Stefan Zachow. Learning shape reconstruction from sparse measurements with neural implicit functions. In International Conference on Medical Imaging with Deep Learning, pages 22-34. PMLR, 2022.
Tamaz Amiranashvili, David Lüdke, Hongwei Bran Li, Stefan Zachow, and Bjoern H Menze. Learn- ing continuous shape priors from sparse data with neural implicit functions. Medical Image Anal- ysis, page 103099, 2024.
Oleksandr Balabanov and Hampus Linander. Uncertainty quantification in fine-tuned Ilms using lora ensembles. arXiv preprint S.K::2402.12264, 2024.
Breiman. Random forests.
Machine Learning, 45:5-32, 10 2001. doi: 10.1023/A:
1010950718922.
Yunliang Cai, Said Osman, Manas Sharma, Mark Landis, and Shuo Li. Multi-modality vertebra recognition in arbitrary views using 3d deformable hierarchical model. IEEE transactions on medical imaging, 34(8):1676-1622. 2015.
Bertrand Charpentier, Daniel Zügner, and Stephan Günnemann. Posterior network: Uncertainty estimation without ood samples via density-based pseudo-counts. Advances in neural information processing systems, 33:1356: 1113--136677 2020.
Anpei Chen, Zexiang Xu, Fugiang Zhao, Xiaoshuai Zhang, Fanbo Xiang, Jingyi Yu, and Hao Su. Mvsnerf: Fast generalizable radiance field reconstruction from multi-view stereo. In Proceedings of the IEEE/CVF international conference computer vision, pages 14124-14133, 2021a.
Kefan Chen, Noah Snavely, and Ameesh Makadia. Wide-baseline relative camera pose estimation with directional learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3258 3268, 2021b.
Yen-Chi Cheng. Hsin-Ying Ying Lee, Sergey Tulyakov, Alexander G Schwing, and Liang Yan Gui. Sd- fusion: Multimodal 3d shape completion, reconstruction, and generation. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages
2023.
Kangle Deng, Andrew Liu, Jun-Yan Zhu, and Deva Ramanan. Depth-supervised nerf: Fewer views and faster training for free. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12882-12891, 2022.
Jesse Dodge, Gabriel Ilharco, Roy Schwartz, Ali Farhadi, Hannaneh Hajishirzi, and Noah Smith. Fine-tuning pretrained language models: Weight initializations, data orders, and early stopping. arXiv preprint arXiv:2002.06300. 2020.
Emilien Dupont, Hrushikesh Loya, Milad Alizadeh, Adam Golinski, Yee Whye Teh, and Arnaud Doucet. Coin++: Neural compression across modalities. arXiv preprint arX1v:2201.12904, 2022.
Yarin Gal. Uncertainty in Deep Learning. PhD thesis, University of Cambridge, 2016.
Yarin Gal and Zoubin Ghahramani. Dropout as bayesian approximation: Representing model uncertainty in deep learning. In Maria Florina Balcan and Kilian Q. Weinberger, editors, Pro- ceedings of The 33rd International Conference Machine Learning, volume 48 of Proceedings of Machine Learning Research, pages 1050 1059, New York, New York. USA, 20-22 Jun 2016. PMLR. URL https: htt.:sss///rrccediigg..... p.esssppreeeeeeee. v48
html

---

Published as a conference paper at ICLR 2025
Anthony A Gatti, Louis Blankemeier, Dave Van Veen, Brian Hargreaves, Scot Delp, Garry E Gold, Feliks Kogan, and Akshay S Chaudhari. Shapemed-kneee A dataset and neural shape model benchmark for modeling 3d femurs. medRxiv, pages 2024-05 2024.
Jakob Gawlikowski, Cedrique Rovile Njieutcheu Tassi, Mohsin Ali, Jongseok Lee, Matthias Humt, Jianxiang Feng, Anna Kruspe, Rudolph Triebel, Peter Jung, Ribana Roscher, et al. A survey of uncertainty in deep neural networks. Artificial Intelligence Review, 56(Supp1 1):1513-1589 2023.
Amos Gropp, Lior Yariv, Niv Haim, Matan Atzmon, and Yaron Lipman. Implicit geometrie regu- larization for learning shapes. arXiv preprint arXiv:2002.10099 2020.
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration of modern neural networks. In International conference on machine learning, pages 1321-1330. PMLR, 2017a
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q. Weinberger. On calibration of modern neural networks. In Doina Precup and Yee Whye Teh, editors, Proceedings of the 34th International Conference on Machine Learning, volume 70 of Proceedings of Machine Learning Research, pages 1321-1330. PMLR, 06-11 Aug 2017b. htLLttsss//prccednngg .ml...ssmlr..oo www.mml.p..... press/ V V7C/ gww.gul17. V7C/guo172 7a. html
Dan Hendrycks, Steven Basart, Norman Mu, Saurav Kadavath, Frank Wang, Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, Mike Guo, al. The many faces of robustness: A criti- cal analysis of out-of-distribution generalization. In Proceedings of the IEEE/CVF international conference on computer vision, pages 2340-8349 2021.
Athira J Jacob, Puneet Sharma, and Daniel Ruckert. Deep conditional shape models for 3d cardiac image segmentation. In International Workshop on Statistical Atlases and Computational Models of the Heart, pages 44-54. Springer, 2023.
Ajay Jain, Matthew Tancik, and Pieter Abbeel. Putting nerf on a diet: Semantically consistent few- shot view synthesis. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 5885-5894, 2021.
Alex Kendall and Yarin Gal. What uncertainties do we need bayesian deep learning for computer vision? Advances in neural information processing systems, 30, 2017.
Mijeong Kim, Seonguk Seo, and Bohyung Han. Infonerf: Ray entropy minimization for few-shot neural volume rendering. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, pages 12912-12921, 2022.
B Ravi Kiran, Ibrahim Sobh, Victor Talpaert, Patrick Mannion, Ahmad A Al Sallab, Senthil Yoga- mani, and Patrick Pérez. Deep reinforcement learning for autonomous driving: A survey. IEEE Transactions Intelligent Transportation Systems, (366):4909-4928 2021.
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei Demis Hass- Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Overcoming catastrophic forget- abis, Claudia Clopath, Dharshan Kumaran, and Raia Hadsell. ting in neural networks. Proceedings of the National Academy of Sciences, 114, 12 2016. doi: 10.1073/pnas. 1611s351111
Balaji Lakshminarayanan. Alexander Pritzel, and Charles Blundell. Simple and scalable predictive uncertainty estimation using deep ensembles. Advances in neural information processing systems, 30, 2017.
Tae-Hwy Lee, Aman Ullah, and Ran Wang. Bootstrap Aggregating and Random Forest, pages 389 429. 01 2020. ISBN 778-3-030-31149. doi: 10.1007/978-3-000-311555
Xiaotong Li, Yongxing Dai, Yixiao Ge, Jun Liu, Ying Shan, and Ling-Yo Yu Duan. Uncertainty mod- eling for out-of-distribution generalization. ICLR, 2022.
Ziwei Liao and Steven L Waslander. Multi-view 3d object reconstruction and uncertainty modelling with neural shape prior. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 3098-3107, 2024.
12

---

Published as a conference paper at ICLR 2025
Ziwei Liao, Jun Yang, Jingxing Qian, Angela P. Schoellig, and Steven L. Waslander. Uncertainty- aware 3D object-level mapping with deep shape priors, 2023.
Chen-Hsuan Lin, Wei-Chiu Ma, Antonio Torralba, and Simon Lucey, Barf: Bundle-adjusting neural radiance fields. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 5741-5751 2021.
Jiashuo Liu, Zheyan Shen, Yue He, Xingxuan Zhang, Renzhe Xu, Han Yu, and Peng Cui. Towards out-of-distribution generalization: survey, arXiv preprint arXiv:2108. 1362, 2021.
Changsheng Lu and Piotr Koniusz. Few-shot keypoint detection with uncertainty learning for unseen species. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recogni- tion, pages 19416-19426, 2022.
Andrey Malinin and Mark Gales. Predictive uncertainty estimation via prior networks. Advances in neural information processing systems, 31, 2018.
Ricardo Martin-Brualla, Noha Radwan, Mehdi SM Sajjadi, Jonathan T Barron, Alexey Dosovitskiy, and Daniel Duckworth. Nerf in the wild: Neural radiance fields for unconstrained photo collec- tions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7210-7219 2021.
Julian McGinnis, Suprosanna Shit, Hongwei Bran Li, Vasiliki Sideri-Lampretsa, Robert Graf, Maik Dannecker, Jiazhen Pan, Nil Stolt-Ansó, Mark Mühlau, Jan S Kirschke, et al. Single- subject multi-contrast mri super-resolution via implicit neural representations. In International Conference on Medical Image Computing and Computer-Asiisted Intervention, pages 173-183.
Springer, 2023.
Ishit Mehta, Michaël Gharbi, Connelly Barnes, Eli Shechtman, Ravi Ramamoorthi, and Manmohan Chandraker. Modulated periodic activations for generalizable local functional representations. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 14214-
14223, 2021.
Lars Mescheder, Michael Oechsle, Michael Niemeyer, Sebastian Nowozin, and Andreas Geiger. Occupancy networks: Learning 3d reconstruction in function space. In 2019 IEEE/CVF Con- ference on Computer Vision and Pattern Recognition (CVPR), pages 4455-4465, 2019a. doi: 10.1109/CVPR..0190000
Lars Mescheder, Michael Oechsle, Michael Niemeyer, Sebastian Nowozin, and Andreas Geiger. Oc- cupancy networks: Learning 3d reconstruction in function space. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4460-4470, 2019b.
Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. Communications of the ACM, 65(1)999-106, 2021.
Norman Müller, Andrea Simonelli, Lorenzo Porzi, Samuel Rota Bulò, Matthias Nießner, and Peter Kontschieder. Autorf: Learning 3d object radiance fields from single view observations. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pottern Recognition, pages 3971-3980, 2022a.
Thomas Müller, Alex Evans, Christoph Schied, and Alexander Keller. Instant neural graphics prim- itives with a multiresolution hash encoding. ACM transactions on graphics (TOG), 41 (15: 2022b.
Michael Niemeyer, Lars Mescheder, Michael Oechsle, and Andreas Geiger. Differentiable volumet- ric rendering: Learning implicit 3d representations without 3d supervision. In Proceedings of the IEEF/CVF conference on computer vision and pattern recognition, pages 3504-3515, 2020.
Michael Niemeyer, Jonathan T Barron, Ben Mildenhall, Mehdi SM Sajjadi, Andreas Geiger, and Noha Radwan. Regnerf: Regularizing neural radiance fields for view synthesis from sparse inputs. In Proceedings of the JEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5480-5490, 2022.
13

---

Published as a conference paper at ICLR 2025
Michael Oechsle, Songyou Peng, and Andreas Geiger. Unisurf: and radiance fields for multi- view reconstruction. In Proceedings of the IEEE/CVF International Conference Computer Vision, pages $589-5599 2021.
Unifying neural implicit surfaces
lan Oshand, Zheng Wen, Seyed Mohammad Asghari, Vikranth Dwaracherla, Morteza Ibrahimi, Xiuyuan Lu, and Benjamin Van Roy. Epistemic neural networks. Advances Neural Information Processing Systems, 36, 2023.
Shumao Pang. Chunlan Pang. Lei Zhao, Yangfan Chen, Zhihai Su, Yujia Zhou, Meiyan Huang, Wei Yang. Hai Lu, and Qianjin Feng. Spineparsenet: spine parsing for volumetric image by two-stage segmentation framework with semantic image representation. IEEE Transactions on Medical Imaging. 40(1):262 2733, 2020.
Jeong Joon Park, Peter Florence, Julian Straub, Richard Newcombe, and Steven Lovegrove. Deepsdf: Learning continuous signed distance functions for shape representation. In Proceed- ings of the IEEE/CVF conference on computer vision and pattern recognition, pages 165-174, 2019a.
Jeong Joon Park, Peter R. Florence, Julian Straub, Richard A. Newcombe, and Steven Lovegrove. Deepsdf: Learning continuous signed distance functions for shape representation. IEEE Conference Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019, pages 165-174. Computer Vision Foundation IEEE, 2019b. doi: URL httt::/
10.1109/CVPR.200 wwwp:CvPP.2219.0000 209..0025.
thecvf com/content Ptaps//eerrreeeeDeeeFee _ttp__eee___eeeiinnneeeee Batta__eaaaannnnanaaaaat_e Caap:e_oooonnnouuoou _iignee_Distttacceeeee..e_ Distance Functions for___or_______
CVPR Fow_Shape___oeee heappppeppreettttioonn.ooo CVPR__CCRR__1_0 2tt__2220_ppppehhaa__ paper.
Songyou Peng. Michael Niemeyer, Lars Mescheder, Mare Pollefeys, and Andreas Geiger. Con- volutional occupancy networks. In Computer ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part III 16, pages 523-540. Springer, 2020.
Janis Postels, Mattia Segu, Tao Sun, Lue Van Gool, Fisher Yu, and Federico Tombari. On the calibration of deterministic epistemic uncertainty.
Janis Postels, Mattia Segu, Tao Sun, Luca Sieber, Luc Van Gool, Fisher Yu, and Federico Tombari.
On the practicality of deterministic episteemi uncertainty. arXiv preprint arXiv:2107 10700000,,, 2021.
Jie Ren, Peter J Liu, Emily Fertig, Jasper Snoek, Ryan Poplin, Mark Depristo, Joshua Dillon, and Balaji Lakshminarayanan. Likelihood ratios for out-t--distribution detection. Advances in neural information processing systems, 32, 2019.
Barbara Roessle, Jonathan T Barron, Ben Mildenhall, Pratul P Srinivasan, and Matthias Nießner. Dense depth priors for neural radiance fields from sparse input views. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12892-12901, 2022.
Mohammad Saberian and Nuno Vasconcelos. Boosting classifier cascades, pages 2047-2055, 01 2010.
Robert E. Schapire. The strength of weak learnability, Machine Learning, 197-222,, 1990. doi: 10. 111100//BF00110033
doi http://.........00/0000001//1110
Jianxiong Shen, Adria Ruiz, Antonio Agudo, and Francese Moreno-Noguer. Stochastic neural radi- ance fields: Quantifying uncertainty in implicit 3d representations. In 2021 International Confer- ence on 3D Vision (3DV), pages 972-981. IEEE, 2021.
Jianxiong Shen, Antonio Agudo, Francesc Moreno- Noguer, and Adria Ruiz. Conditional-BF nerf: Accurate 3d modelling with reliable uncertainty quantification. In European Conference on Com- puter Vision, pages 540-557. Springer, 2022.
Vincent Sitzmann, Julien Martel, Alexander Bergman, David Lindell, and Gordon Wetzstein. Im- plicit neural representations with periodic activation functions. Advances in neural information processing systems, 33:7462-7473, 2020.
14

---

Published as a conference paper at ICLR 2025
Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, 1lya Sutskever, and Ruslan Salakhutdinov. Dropout: a simple way to prevent neural networks from overfitting. The journal of machine learning research, 15(1):1929-1958, 2014.
Natasa Tagasovska and David Lopez-Paz. Single-model uncertainties for deep learning. Advances in neural information processing systems, 32, 2019.
Matthew Tancik, Pratul Srinivasan, Ben Mildenhall, Sara Fridovich-Keil, Nithin Raghavan, Utkarsh Singhal, Ravi Ramamoorthi, Jonathan Barron, and Ren Ng. Fourier features let networks learn high frequency functions in low dimensional domains. Advances in neural information processing systems, 33:7537-7547, 2020.
Katarína Tóthová, Sarah Parisot, Matthew Lee, Esther Puyol-Antón, Andrew King, Marc Polle- feys, and Ender Konukoglu. Probabilistic 3d surface reconstruction from sparse mri information. In Medical Image Computing and Computer Assisted Intervention-MICCAl 2020: 23rd Inter- national Conference, Lima, Peru, October 4-8, 2020, Proceedings, Part I 23, pages 813-823. Springer, 2020.
Dustin Tran, Jeremiah Liu, Michael W Dusenberry, Du Phan, Mark Collier, Jie Ren, Kehang Han, Zi Wang, Zelda Mariet, Huiyi Hu, et al. Plex: Towards reliability using pretrained large model extensions. arXiv preprini arXiv: M20i::207..7411,, 00411, 2022.
Prune Truong, Marie-Julis Rakotosaona, Fabian Manhardt, and Federico Tombari. Sparf: Neural radiance fields from sparse and noisy poses. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4190-4200, 2023.
Federico Turella, Gustav Bredell, Alexander Okupnik, Sebastiano Caprara, Dimitri Graf, Reto Sut- ter, and Ender Konukoglu. High-resolution segmentation of lumbar vertebrae from conventional thick slice mri. In Medical Image Computing and Computer Assisted Intervention-Me MICCAAI 2021: 24th International Conference, Strasbourg, France, September 27-October 1, 2021, Proceedings, Purt 24, pages 689-698. Springer, 2021.
Florian Wenzel, Andrea Dittadi, Peter Gehler, Carl-Johann Simon-Gabriel, Max Horn, Dominik Zietlow, David Kernert, Chris Russell, Thomas Brox, Bernt Schiele, et al. Assaying out-of- distribution generalization in transfer learning. Advances in Neural Information Processing Sys- fems, 35:7181-7198, 2022.
Mitchell Wortsman, Gabriel Ilharco, Jong Wook Kim, Mike Li, Simon Kornblith, Rebecca Roelofs, Raphael Gontijo Lopes, Hannaneh Hajishirzi, Ali Farhadi, Hongseok Namkoong, et al. Robust fine-tuning of zero-shot models. In Proceedings of the IEEE/CVF conference on computer vision and pallern recognition, pages 7959-7971, 2022.
Xueting Yang, Yihao Luo, Yuliang Xiu, Wei Wang, Hao Xu, and Zhaoxin Fan. D-if: Uncertainty- aware human digitization via implicit distribution field. In Proceedings of the IEEE/CVF Inter- national Conference Computer Vision, pages 9122 9132, 2023.
Lior Yariv, Yoni Kasten, Dror Moran, Meirav Galun, Matan Atzmon, Basri Ronen, and Yaron Lip- man. Multiview neural surface reconstruction by disentangling geometry and appearance. Ad- vances Neural Information Processing Systems, 33:2492 2502, 2020.
Zehao Yu, Songyou Peng, Michael Niemeyer, Torsten Sattler, and Andreas Geiger. Monosdf: Ex- ploring monocular geometric cues for neural implicit surface reconstruction. Advances in neural information processing systems, 35:25018 1-55022,, 2022.
Jason Y Zhang, Deva Ramanan, and Shubham Tulsiani. Relpose: Predicting probabilistic relative rotation for single objects in the wild. In European Conference on Computer Vision, pages 592- 611. Springer, 2022.
Ke Zou, Zhihao Chen, Xuedong Yuan, Xiaojing Shen, Meng Wang, and Huazhu Fu. A review of uncertainty estimation and its application medical imaging. Meta- Radioload,, page 100003, 2023.
15

---

Published as a conference paper at ICLR 2025
A APPENDIX: ELASTIC WEIGHT CONSOLIDATION
Elastic Weight Consolidation (EWC) was introduced to address catastrophic forgetting in continual learning (Kirkpatrick et al., 2016). EWC is regularization that protects crucial parameters network when learning new task, in order to avoid catastrophic forgetting.
Assume two distinct tasks. A and B. with their respective datasets DA and DB. where DANDn 0. The tasks are learned sequentially without access to previous task datasets. First, task A is learned by training neural network on AA, resulting in set of optimal weights 0A. When learning task B using dataset Dg, EWC regularizes the weights so they remain within a region the parameter space that led to good accuracy for task A. The justification of the method is based on probabilistic principles. Given a combined dataset D == DA U DD, applying Bayes' rule yields:
log p(0) = log ((D + log p(0) - log p(D)
(3)
where p(D corresponds to the likelihood over the entire dataset and p(0) is the user-defined prior over the weights of the network. Rearranging equation (3) yields
((g log ((D)(0) + log p(0 (Dn)
(4)
where it can be observed that all information from D is contained in the posterior p(0)DA), which usually intractable. Therefore, the EWC method employs the Laplace approximation to the pos- terior, a process conducted during the training of task A. The resulting approximated posterior modeled as a Gaussian distribution, with its mean represented by 0 and covariance matrix EA (F(Ox) I) where F(Oa) denotes the Fisher information matrix evaluated over data set D at A.
Next, in stage B, the optimization of parameter 0 incorporates the approximation of the posterior obtained during task A as a constraint in the optimization process. Following the equation this approximation, log p(0) DA), serves regularizer in the learning objective:
0 = arg min RD, (0) + A(e (A)T(F(())) [) 0x)
(5)
where R(() corresponds to the likelihood term log p(Dz) ()), À is hyperparameter, / represents the identity matrix, and describes the Hadamard product.
B APPENDIX: METRICS
Uncertainty estimates for regression tasks involve using unbiased estimates of the modes of the approximated posterior predictive distribution. neural networks, this corresponds to performing multiple stochastic forward passes. For classification tasks, three main methods for uncertainty estimates are variational ratios, predictive entropy, and mutual information (Gal, 2016). our experiments, opt for predictive entropy.
Dice score The Dice score, also known as the Dice Similarity Coefficient (DSC), measures the similarity between two data sets, commonly used in medical imaging to evaluate segmentation ac- 2x Xny|
curacy. Defined as DSC = + YT where X and Y represent the ground truth and predicted segmentation sets, respectively. The score ranges from to 1, with indicating perfect agreement and 0 representing no overlap.
Reliability diagrams and Expected Calibration Error Reliability diagrams are graphical tools used in uncertainty modeling to assess the calibration of probabilistic predictions. They plot pre- dicted probabilities against empirical frequencies, allowing for visual inspection of how well the predicted probabilities of model correspond to the actual outcomes. A perfectly calibrated model would align closely with the diagonal line from the bottom left to the top right of the plot, indicating that the predicted probabilities match the observed probabilities. Reliability diagrams are computed by binning predicted probabilities into intervals. For each bin, the mean predicted probability is plotted against the observed frequency of the corresponding outcomes. This involves calculating the
16

---

Published as a conference paper at ICLR 2025
proportion of positive outcomes in each bin and plotting these against the average predicted proba- bility for the bin. The closer the points lie to the diagonal line from the bottom left to the top right, the more calibrated the model is considered. Reliability diagrams Calibration Error (ECE), which quantitatively assesses a model's calibration. ECE is computed as a weighted average of the absolute differences between the predicted probabilities and the actual out- come frequencies across different bins used in reliability diagrams. Each bin's weight corresponds the number of samples it contains. Thus, while reliability diagrams provide visual interpretation of model calibration, ECE offers single numerical value summarizing the calibration error across all bins.
closely related to the Expected
C APPENDIX: TRAINING DETAILS
Toy experiment We generated two-dimensional datasets A and B with a sinusoidal decision boundary and Gaussian noise. A moderate distribution shift was modeled between the datasets by adjusting the support values of I1: I1 E [-0.75, 0.7] for dataset A and Iι E [-0.5, 2.0] for dataset B. We created 1000 training samples and 500 test samples for dataset A, and 50 train and 500 test samples for dataset B.
We used a consistent model architecture across experiments a straightforward 3-layer MLP with 256 hidden units in each layer. For methods using dropout. dropout layer (p 0.3) followed each linear layer. For ensemble methods, 4 separate networks were trained on dataset A. For ensembles and Dropsembles we used network instances for fine-tuning and inference. For MC dropout used 100 samples at inference. We trained for 800 epochs for training with learning rate le 3 and 600 epochs for tuning with a learning rate 5e- 3.
MNIST experiment First, we trained small autoencoder, consisting of three convolutional layers in the encoder, only on dataset A. The encoder does not have dropout layers. The encoder was trained with cross-entropy loss for 50 epochs with a learning rate 0.01 and cosine warmup scheduler. After this initial training, the encoder was kept fixed (frozen) for all subsequent experiments, and the decoder was discarded. This encoder now serves to generate a latent representation of the input data, which is then supplied to an occupancy network.
The convolutional encoder was trained on Dataset A and consequently frozen for all the rest of the experiments. Pooling layer was applied to produce same latent code for all 2D coordinates in shape.
The 8-layer MLP occupancy network was trained with cross-entropy loss for 50 epochs on dataset A with a learning rate 0.005 and a cosine warmup scheduler. For ensemble methods, 4 separate networks were trained dataset A. For fine-tuning dataset B we used same learning rate but tuned the networks for 30 epochs only. For ensembles and Dropsembles we used 4 network instances for fine-tuning and inference.
Shape experiment For Dataset A, we selected 1,000 random samples from the ShapeNet dataset's 's airplane category. For Dataset B, we randomly selected 10 unseen shapes and applied ran- dom erosion to simulate real-world distribution shifts. Examples of eroded and high-quality target shapes are shown in Figure 12 For both datasets, occupancies at resolution of 128 were provided to the encoder. SDF values for Dataset A were estimated from ground-truth high-quality meshes. For Dataset B, meshes were reconstructed from the eroded shapes and subsequently converted into SDF representations.
The convolutional encoder was trained on Dataset A and consequently frozen for all the rest of the experiments. Bi-linear upsampling was applied to query latent vectors for specific 3D coordinates proposed in convolutional occupancy networks architecture.
The 8-layer DeepSDF MLP network was trained on Dataset A for 100 epochs using a clipped L1 loss (with clipping parameter 8 0.1), a learning rate of 0.001, and step scheduler as proposed in (Park et al. 2019a). When applicable, dropout was added after each activation layer, as suggested in Dee. For ensemble methods, four separate networks were trained on Dataset A. Fine-tuning on Dataset B was performed using a non non-clipped L1 loss with a slower learning rate of 0.0001 and limited to 75 epochs. For ensembles and Dropsembles, four network instances were employed
17

---

Published as a conference paper at ICLR 2025
during fine-tuning and inference. In the SIREN experiment, ReLU activations replaced with periodic activation functions (Sitzmann et al. 2020), with dropout applied only after the 4th and 6th layers in the decoder network.
Lumbar spine experiment All experiments in this section were performed on NVIDIA RTX A6000 GPU, equipped with 48 GB of memory. The networks were trained at 16-mixed precision due to memory constraints.
First ReconNet encoder was trained with cross-entropy loss full training split of rigged atlas dataset to obtain a "frozen" encoder. We used a learning rate 0.01 and trained for 100 epochs with early stopping. Bi-linear upsampling was applied query latent vectors for specific 3D coordinates as proposed in convolutional occupancy networks architecture.
The occupancy network architecture incorporates skip connections and ReLU activations, with dropout layers (p = 0.2) following each linear layer except the last. To effectively model the entire lumbar spine, we adapted the strategy from (Peng et al. 2020) by replacing the learnable latent vector with an output from a pretrained convolutional encoder. Specifically, we performed bilinear upsampling on the output of this frozen encoder to generate detailed latent representation for each voxel. This representation. coupled with 3-dimensional voxel coordinates, was provided as input to the MLP. We used cross-entropy loss for training and fine-tuning occupancy network. The network was trained for 100 epochs dataset A with early stopping applied after 68 epochs. Learning rate 0.001 and batch size 32 were used for training, where each batch we used only [64, 64, 64] random voxels. For fine-tuning methods: learning rate 0.001 and tuned for 50 epochs without early stopping. Results evaluated at the checkpoint of the last epoch are presented throughout the paper.
dataset B used the same of parameters for all the
D APPENDIX: SELECTING OPTIMAL REGULARIZATION STRENGTH
dropout
Ensembles
Dropsembles
13
AL12
Accursecy
68
lagt0 Lembod
logt0
Ing10 bembda
Figure 6: Toy classification example ablation for EWC regularization strength. Averages and confi- dence over 3 random seeds. Red dot represents the optimal lambda selected for the main figure.
MC dropout EWC
Ensembles + EWC
Dropsembles EWC
0.75
0.750
0.725
Kroreyy
0.62
0.62
0.675
0.59
0,500
0.50
6.5
log00 lembida
Ieg00 lamiidaa
log20 tamitia
Figure 7: Corrupted MNIST reconstruction example ablation for EWC regularization strength. Dots correspond to different samples.
18

---

Published as a conference paper at ICLR 2025
MC dropout EWC
Dropsembles EWC
0.88
0.90
example
example_ 1
Accuracy
Accuracy
example
0.86
0.80
1
-2
-1
leg10 lambda
log10 lambda
Figure 8: Corrupted atlas reconstruction example ablation for EWC regularization strength. Dots correspond to different samples. Due to high computational demand we were not able to evaluate all examples on the full grid.
E APPENDIX: ADDITIONAL EXPERIMENTAL RESULTS
Toy example
MNIST
0.72
Acc B
DSC
0.70
0.86
0.69
0.84
0.68
100
thinned networks
# thinned networks
Example in the performance difference between Dropsembles w/ and w/o EWC: tiny details in satient-specific variability are not captured well DSC metrics, but are apparent on the qualitative examples and Hausdorff distance (HD). Grey segmentations are produced by Dropsem- bles w/ or w/o EWC regularization overlayed with colorful segmentations of the HQ ground-truth.
Figure 9:
19

---

Published as a conference paper at ICLR 2025
a
Non-corrupted input (as occupancy)
High quality target Used for training
b
Corrupted Input (as occupancy)
Corrupted target SDF used for tuning
High quality Gr
Figure 10: Examples of ShapeNet samples. a) Dataset A; b) Dataset B.
a
High quality
Average prediction MC-dropout
Average prediction Dropsembles EWC
High quality GT
Average prediction of MC-dropout
Average prediction Dropsembles EWC
Figure 11: Comparison of average predictions b/w MC Dropout and Dropsembles. Dropout predic- tions even after averaging are not smooth. a) Meshes produced by averaged predictions; b) A slice of SDF representation produced by averaging predicted samples.
20

---

Published as a conference paper at ICLR 2025
Dataset A:
Dataset B:
Sparse atlas
Sparse and corrupted atlas
Input
Ground Truth
Input
Ground Truth
Figure 12: Examples of Corrupted Atlas dataset.
Dataset A:. Sparse atlas
Dataset B:
Patient MRI segmentations
Input
Ground Truth
Input
Ground Truth
Figure 13: Example of Atlas to MRI segmentations datasct.
Table 3: ShapeNet: Results for models trained on dataset A only. All metrics are reported in per-
centages [%].
Method
mDICE [%]
mIoU [%] 1
ECE [+]]% 4
Dropout
78.4 + 2.4
64.5 + 3.2
31.66 = 3.37
Dropout- SIREN
78.5 + 3.3
64.8 + 4.4
31.94 + 3.89
Ensembles
76.4 2.9
61.9
36.33 3.11
Ensembles SIREN
75.9 + 3.7
61.3 + 4.8
36.44 + 3.85
21

---

Published as a conference paper at ICLR 2025
Table 4: ShapeNet with SIREN activations: Results for models trained dataset A and B (fine-
tuned on dataset B). All metrics are reported in percentages [%]]
Method
mDICE [%]
mloU [%]]
ECE [% +
Dropout
80.2 + 2.4
67.0 3.3
27.99 2.85
Dropsembles
83.8 = 2.2
72.2 3.2
21.04 3.04
Ensembles
83.4 = 2.5
3.6
2.04
Dropout EWC
83.3 = 2.2
71.5
19.27 + 2.74
Dropsembles + EWC
84,5 + 2.3
73.2 3.4
17.73 3.05
Ensembles EWC
80.9 + 2.5
68.0 3.5
28.95 +
Table 5: Detailed comparison of training time on lumbar spine experiment. We report the results in
GPU-hours [h].
Method
Training on atlas [h]
Tuning on corrupted atlas lumbar spine MRI [b]
Total [h]
Baseline
48 /48
MCdropout
48
3.5/22
51.5 99.55
Dropsembles
48
2.5 (per network) x4
58 / 132
Dropsembles EWC
48
2.75 22 (per network) x4
58 / 136
Ensembles
48 (per network)
2.5 (per network) x4
202 / 276
Table 6: Comparison of methods tuned dataset of lumbar spine MR dataset. Dice Score Coef-
ficient (DSC), Hausdorff distance (HD), Dice Score Coefficient average per network sample (DSC
avg), Expected Calibration Error (ECE), and total inference time (time) reported for dataset B.
Baseline is MC dropout trained on dataset A.
Method
DSC [%] t
DSC avg [%]
HD
ECE [%]
time [sec]
Baseline
75.8
74.0 - 0.0
18.2
4.7
161
Subject
MCdropout
93.5
92.3 1.. 0.0
6.3
3.6
159
Dropsembles
93.9
9.1
6.2
4.4
20
Dropsembles + EWC
93.9
93.5
6.0
4.2
21
Ensembles
93.9
93.5 1.2 0.2
24.3
4.3
21
Baseline
35.9
0.1
34.2
33.6
30
Subject 2
MCdropout
91.3
0.0
15.2
4.9
160
Dropsembles
92.0
91.5
9.4
6.1
19
Dropsembles + EWC
92.0
91.5
9.5
6.1
19
Ensembles
91.9
9..4
25.2
6.0
21
Baseline
76.5
0.0
19.5
3.7
159
Subject 3
MCdropout
92.5
9.0
7.1
4.9
158
Dropsembles
92.6
92.2
8.2
5.3
19
Dropsembles + EWC
92.6
92.2 0.0
6.7
5.2
19
Ensembles
92.8
92.5
6.5
5.6
21
22

---

Published as a conference paper at ICLR 2025
Table 7: Comparison of methods tuned dataset B of lumbar spine experiment. Evaluations performed on the corrupted atlas. Dice Score Coefficient (DSC), Hausdorff distance (HD), Dice Score Coefficient average per network sample (DSC avg), Expected Calibration Error (ECE), and total inference time (time) are reported for dataset B. Baseline is MC dropout trained on dataset A.
Method
DSC [%]
DSC avg [%]] f
HD J
ECE [%] +
time [sec] I
Baseline
65.5
64.4 + 0.0
16.8
24.7
172
Corrupted Atlas 1
MCdropout
83.5
81.7 0.0
14.3
4.6
160
Dropsembles
85.0
84.7 0.2
13.2
5.7
21
Dropsembles EWC
85.0
84.7 0.2
13.3
5.6
21
Ensembles
84.5
84.2
13.4
5.0
21
Baseline
67.2
65.8 + 0.0
0.0
16.5
25.6
164
Corrupted Atlas 2
MCdropout
88.6
85.7 0.0
9.4
0.8
170
Dropsembles
90.2
89.6 0.0
8.9
2.3
22
Dropsembles + EWC
90.3
89.6 0.0
8.1
2.1
22
Ensembles
90.0
89.3 0.7
7.8
1.3
22
Baseline
62.3
61.4 0.0
23.5
25.4
163
Corrupted Atlas 3
MCdropout
83.7
81.5
15.4
4.8
162
Dropsembles
85.2
12.0
5.9
22
Dropsembles + EWC
85.3
84.8 : 0.2
11.2
5.8
22
Ensembles
84.9
84.5 0.7
10.2
5.3
22
23


Reviews:
[{"review_id": "ygZRvPrEPl", "reviewer": "Reviewer_6vSX", "summary": "This paper introduces 'Dropsembles', an approach to uncertainty modeling in fine-tuned neural implicit functions. 'Dropsembles' combines dropout-based methods with ensemble learning to balance computational efficiency and robust uncertainty estimation.\nThe authors conduct experiments on synthetic anatomical datasets and MRI segmentation tasks, demonstrating that 'Dropsembles' maintains predictive accuracy comparable to deep ensembles while reducing computational demands.", "questions": "**Q1**: Why is EWC necessary if Tasks A and B are similar?\n\nCould you clarify why EWC is used in this context, given that Task A (dense dataset) and Task B (sparse/noisy dataset) do not appear to be significantly different? Is the intent to quantify uncertainty in a continual learning setting, or is EWC applied simply as a regularization tool?\nWhat makes Dropsembles particularly suited to neural implicit representations?\n\n**Q2**: Dropsembles seems like a standard dropout-ensemble approach. Are there specific adaptations that make it uniquely effective for implicit functions in 3D reconstructions, especially regarding spatial consistency or grid-based predictions?\nConsidering Dropsembles combines well-known methods, why weren’t additional baselines, like Bayesian neural networks or variational inference-based [1], tested to provide a clearer comparison?\n\n**Q3**: How does Dropsembles handle spatial dependencies?\n\nNeural implicit representations often require spatial coherence across grid or voxel predictions. \nDoes Dropsembles include any mechanism to maintain this spatial consistency in its uncertainty estimation, like [2]?\n\nCould you discuss the limitations of Dropsembles in more detail?\n\n**Q4**: Given that Dropsembles simply ensembles thinned models from dropout, what are its limitations in terms of computational cost, reliability, or adaptability to different 3D tasks?\n\n**Minor**: The term \"thinned network\" isn’t a standard term in the field and may lead to confusion. It could be misinterpreted, as it doesn't clearly indicate that dropout is being applied.\n\n**References**:\n\n[1] Stochastic Neural Radiance Fields: Quantifying Uncertainty in Implicit 3D Representations, 3DV 2021\n\n[2] Stochastic Segmentation Networks: Modelling Spatially Correlated Aleatoric Uncertainty, NeurIPS 2021", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "Key contributions include:\n\n(1) A method for modeling epistemic uncertainty in fine-tuned implicit functions.  \n\n(2) The application of Elastic Weight Consolidation (EWC) to address distribution shifts. \n\n(3) Experimental validation across toy and real-world datasets.", "weaknesses": "(1) **Questionable motivation for EWC**:\n\n**Lack of task distinction**: The dense and sparse datasets (Tasks A and B) are similar in content, which doesn’t align well with the typical use case for EWC in continual learning. EWC is usually applied to preserve knowledge across distinct tasks, so its application here appears unjustified or artificial.\n\n(2) **Limited novelty in dropsembles**:\n\n**Lack of specificity to NIR**:\nDropsembles combines dropout with ensembling in a straightforward way without tailoring it for neural implicit representations. \nThis method may not provide significant innovation for uncertainty estimation in 3D reconstruction, particularly if it doesn’t leverage the unique characteristics of implicit functions.\n\n(3) **Insufficient baseline comparisons**:\n\nDropsembles is not compared to other standard uncertainty estimation techniques such as Bayesian neural networks, or other methods tailored to implicit representations [1]. This limits the evidence for Dropsembles’ effectiveness.\n\n(4) **Limited dataset diversity**:\n\n**Narrow generalizability**: The experiments are focused on synthetic anatomical data and sparse MRI segmentation, which may not fully demonstrate Dropsembles' robustness across various domains or types of data shifts.\n\nReferences:\n\n[1] Stochastic Neural Radiance Fields: Quantifying Uncertainty in Implicit 3D Representations, 3DV 2021", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ddMOOfC0Jh", "reviewer": "Reviewer_iePp", "summary": "The paper presents Dropsembles, a novel uncertainty quantification technique for neural implicit functions such as Neural Radiance Fields (NeRFs) and Occupancy Networks. This method aims to achieve the high accuracy and calibration of deep ensembles while significantly reducing computational overhead. By integrating Monte Carlo dropout with ensemble strategies, and leveraging Elastic Weight Consolidation (EWC), Dropsembles effectively handles distribution shifts and sparse, noisy data.\n\nThe primary focus is on addressing the challenges of 3D shape reconstruction from sparse and corrupted inputs. The method leverages high-quality, densely sampled synthetic datasets as shape priors to fill gaps and correct corruptions in the target data, even when distribution shifts occur due to noise and corruption. The premise is that these priors can guide the reconstruction process, ensuring more reliable outputs.\n\nInitially, a Convolutional Occupancy Network is trained on high-quality synthetic data. During testing, the network is fine-tuned on individual sparse and noisy samples using EWC, which regularizes the adaptation process to prevent forgetting the original training data and to transfer prior knowledge effectively. This method involves creating ensembles through dropout, where binary masks generate multiple network instances. Each instance is then fine-tuned independently on the testing sample, forming an ensemble of networks initialized with correlated weights.\n\nThe paper validates the method through extensive experiments on toy datasets and low-resolution MRI segmentations of the lumbar spine. Results demonstrate that Dropsembles maintains accuracy and reliable uncertainty estimation and efficiently handles computational constraints and distribution shifts.", "questions": "- Could you provide more details on how the EWC regularization strength was selected? Is there a rationale beyond the empirical ablation studies, or could a more systematic method be developed?\n\n- Have you tested Dropsembles on other types of neural implicit functions or datasets beyond those presented? How do you anticipate the method will perform in domains with significantly different characteristics?\n\n- While the method reduces training costs, inference demands remain comparable to traditional deep ensembles. Are there potential strategies to also optimize inference efficiency?\n\n- Given that synthetic data might not capture all real-world complexities, how do you ensure the generalization of the model to highly variable or unseen real-world data?\n\n- Can you elaborate on the effectiveness of Dropsembles in handling extreme distribution shifts? Are there specific limitations or edge cases where the method struggles?\n\n- How easily can Dropsembles be integrated into existing neural implicit networks? Are there any specific prerequisites or limitations that practitioners should be aware of?", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- The focus on uncertainty modeling in fine-tuned neural implicit functions addresses an underexplored area, contributing valuable insights into this challenging problem space.\n\n- Dropsembles combines dropout and deep ensembles, offering a computationally efficient  training method for uncertainty estimation specifically tailored to neural implicit functions. This application of established uncertainty quantification and continual learning techniques to neural implicit functions is considered a novel contribution to neural implicit functions.\n\n- The paper provides a rigorous experimental evaluation across synthetic and real-world spine datasets, effectively demonstrating that Dropsembles maintain prediction accuracy and uncertainty calibration under varying conditions.\n\n- The method achieves significant reductions in computational costs during training compared to traditional deep ensembles, without compromising performance. However, computational demands during inference remain comparable to those of ensemble methods.\n\n- The integration of Elastic Weight Consolidation (EWC) effectively mitigates the impact of distribution shifts, enhancing the model's robustness to noisy and sparse inputs.\n\n- The approach is straightforward and versatile, capable of being integrated into various neural implicit networks and adapted to different task-specific training objectives, making it broadly applicable.\n\n- The strategy of training on synthetic data while addressing distribution shifts between synthetic and real data is a pragmatic solution, particularly in scenarios where real-world data is scarce and noisy.\n\n- The paper is clearly written, well-structured, and easy to follow.", "weaknesses": "- While the method shows promise in specific tasks like lumbar spine reconstruction, broader application beyond medical imaging is not thoroughly explored or validated.\n- Despite reduced costs compared to deep ensembles, Dropsembles still require significant resources during fine-tuning, especially on high-resolution data.\n- Although Dropsembles handle sparse inputs well, the effectiveness on highly varied real-world scenarios with extreme data corruption could have been examined more comprehensively.\n- While Dropsembles generally outperform baselines, the improvements in some scenarios, particularly when using EWC, are not consistently significant across all metrics.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "EkbdwzxG12", "reviewer": "Reviewer_BMvL", "summary": "This paper explores the topic of uncertainty modeling for decoder fine-tuning in implicit neural representations for occupancy representations. The presented method employs a combination of widely used uncertainty modeling concepts (i.e., dropout, ensembling) and the elastic weight consolidation concept stemming from continual learning, with the notion of fine-tuning, allowing to obtain multiple (fine-tuned) models at less training cost, with partially improved or comparable performance in modeling the uncertainty. This is demonstrated in experiments with toy examples (Sinusoidal decision boundary, MNIST) to medical shapes with the perseverance of reconstruction accuracy.", "questions": "Importance of encoder design / Application in auto-decoder setting:\n\nQ1: Given the suitability of INRs for non-grid data, it is sometimes challenging to design an encoder for non-grid data. How was the encoder used/designed for the MNIST case, where some pixels are occluded? Did the encoder see the entire image (e.g., with a CNN-based encoder)? Sorry if I missed this; I would really appreciate an answer, even if it does not change the comparability of the different methods/baselines.\n\nQ2: In the INR community, auto-decoder settings are quite common as well (see DeepSDF or Mehta et al.). Could the same method also be used in this case, given that the encoding of, e.g., the shape or instance stems from the same MLP, or are there any theoretical limitations that prohibit a meaningful uncertainty in this case?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 2, "strengths": "Originality: While the paper builds upon technical concepts from the uncertainty modeling community and does not introduce novel technical components, I believe the explored application and conclusions are novel and impactful.\n\nQuality: The paper is very well structured and intuitive. The presentation and writing of the method are very clear, and in combination with the experiments, the method is sound.  \n\nClarity: The paper is well written in all sections, especially in related work and methods, making it a charm to read. It really flows. I also really enjoyed how well the method was put into the context of related work (i.e. very thoroughly research in different areas) and how the method was described in a simple and comprehensive way. Moreover, the authors detail every aspect of the experimental design in the main paper, and together with the supplementary this makes the evaluation of the presented method easy to follow. This, together with the accessibility of the source code, facilitates reproducability of the experiments. \n\nSignificance: While implicit neural representations are often trained - individually - for (single) images and other signals, cohort-based training is quite common for shape completion in SDFs and occupancy representations. Given the prevalence of encoder-/decoder in these settings, this paper constitutes a meaningful exploratory study in the modeling of decoder uncertainty.", "weaknesses": "1) Related Work:\n\nAs detailed above, the paper is very well written and conducts a very thorough related work section. However, I think it would be important to touch upon one more area - i.e. the more recent architectures used in cohort-based training that also use encoder frameworks. Specifically, it would be interesting to discuss the advancements in [1,2], given that these constitute more recent alternatives to the DeepSDF architecture. As both works [1,2] use encoder-/decoder settings in combination with weight modulation approaches, this would also add emphasis to the relevance of this work. \n\n2) Experimental Design: While the experimental design is sound, I believe the paper could have explored a higher variation of datasets and application scenarios.\n\n(a)  Experiments not only for Re-LU-based networks but also for other activation functions:\nWhile ReLU networks were routinely used in [3,4,5] medical applications, the INR community has shifted towards other activation functions [6] or embedding/projection functions [7,8] (particularly for images). While not all (novel) shape modeling applications use these (yet?), I believe it would have been interesting to explore if the same holds true for, e.g., SIREN [6]. This ablation would have been simple, as it merely requires a change in the MLP and DeepSDF [3], which also works well with sinusoid activation functions [6]. Moreover, it has been comparatively less explored if dropout and other anomaly detection frameworks are useful in this context. Thus, for decoder-based uncertainty modeling, for example, in the context of [2], such an ablation study would be ultimately interesting and relevant.\n\n(b) Experiments on more common (shape) datasets, especially with SDFs: The authors state that their method is relevant for both SDF and occupancy representations but only conducts experiments for occupancy networks. Given that the training setup would not change much, I believe it would have been interesting to show experiments in an established shape dataset as well (e.g. (Med-) ShapeNet). \n\nReferences:\n\n[1] Dupont E, Loya H, Alizadeh M, Goliński A, Teh YW, Doucet A. Coin++: Neural compression across modalities. arXiv preprint arXiv:2201.12904. 2022 Jan 30.\n\n[2] Mehta I, Gharbi M, Barnes C, Shechtman E, Ramamoorthi R, Chandraker M. Modulated periodic activations for generalizable local functional representations. InProceedings of the IEEE/CVF International Conference on Computer Vision 2021 (pp. 14214-14223).\n\n[3] Park JJ, Florence P, Straub J, Newcombe R, Lovegrove S. Deepsdf: Learning continuous signed distance functions for shape representation. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition 2019 (pp. 165-174).\n\n[4] Mescheder L, Oechsle M, Niemeyer M, Nowozin S, Geiger A. Occupancy networks: Learning 3d reconstruction in function space. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition 2019 (pp. 4460-4470).\n\n[5] Amiranashvili T, Lüdke D, Li HB, Menze B, Zachow S. Learning shape reconstruction from sparse measurements with neural implicit functions. InInternational Conference on Medical Imaging with Deep Learning 2022 Dec 4 (pp. 22-34). PMLR.\n\n[6] Sitzmann V, Martel J, Bergman A, Lindell D, Wetzstein G. Implicit neural representations with periodic activation functions. Advances in neural information processing systems. 2020;33:7462-73.\n\n[7] Tancik M, Srinivasan P, Mildenhall B, Fridovich-Keil S, Raghavan N, Singhal U, Ramamoorthi R, Barron J, Ng R. Fourier features let networks learn high frequency functions in low dimensional domains. Advances in neural information processing systems. 2020;33:7537-47.\n\n[8] Müller T, Evans A, Schied C, Keller A. Instant neural graphics primitives with a multiresolution hash encoding. ACM transactions on graphics (TOG). 2022 Jul 22;41(4):1-5.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "A4EhMTnF6g", "reviewer": "Reviewer_Yvca", "summary": "In this work, authors introduce Dropsembles, a novel method for uncertainty estimation in tuned implicit functions. They demonstrate the efficacy of this approach through a series of experiments, starting with toy examples and progressing to a real-world scenario. Our results show that Dropsembles achieve the accuracy and calibration levels of deep ensembles but with significantly less computational cost.", "questions": "Please refer to the weakness part.", "limitations": "", "rating": 3, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1.  The topic of this work is really interesting. By utilizing an implicit shape model, points with sparse shapes can be densified with refined shape representations.\n2.  This is the first work to model epistemic uncertainty in the implicit decoder during finetuning from synthetic data.", "weaknesses": "1.  The core concept of this work is uncertainty modeling. There are two innovative designs proposed in this paper. For the Elastic weight consolidation strategy, it seems that no remarkable improvements are done. Authors simply employ EWC to adapt pretrained models from dataset A. The other component is Dropsembles, please provide a more detailed description on its contribution.\n2.  Since Dropsembles is aimed for the tradeoff between computational costs and accuracy. However, no quantitative results are given in the experimental part. Authors should carefully list out the comparison between dropsembles and other methods on uncertainty estimation.\n3.  The improvement on lumbar spine seems to be marginal, maybe a detailed analysis is required.\n4.  The effectiveness of this method is highly required for the evaluation on more challenging datasets, such as Medshapenet, etc.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)