Multiple Noises in Diffusion Model for Semi-Supervised Multi-Domain Translation
Authors: Tsiry Mayet
Simon Bernard
Romain HÉRAULT
Clement Chatelain
Venue: Accepted by TMLR ()

Abstract: In this work, we address the challenge of multi-domain translation, where the objective is to learn mappings between arbitrary configurations of domains within a defined set (such as $(D_1, D_2)\rightarrow{}D_3$, $D_2\rightarrow{}(D_1, D_3)$, $D_3\rightarrow{}D_1$, etc. for three domains) without the need for separate models for each specific translation configuration, enabling more efficient and flexible domain translation.
We introduce Multi-Domain Diffusion (MDD), a method with dual purposes: i) reconstructing any missing views for new data objects, and
ii) enabling learning in semi-supervised contexts with arbitrary supervision configurations. MDD achieves these objectives by exploiting the noise formulation of diffusion models, specifically modeling one noise level per domain.
Similar to existing domain translation approaches, MDD learns the translation between any combination of domains. However, unlike prior work, our formulation inherently handles semi-supervised learning without modification by representing missing views as noise in the diffusion process.
We evaluate our approach through domain translation experiments on BL3NDT, a multi-domain synthetic dataset designed for challenging semantic domain inversion, the BraTS2020 dataset, and the CelebAMask-HQ dataset.

Published in Transactions on Machine Learning Research (09/2025)
Multiple Noises in Diffusion Model for Semi-Supervised Multi-Domain Translation
Tsiry Mayet
INSA Rouen Normandie, LITIS UR 4108, 2-76000 Roven, France
Simon Bernard
Université Rouen Normandie, LITIS UR 4108, P-76000 Rouen, France
Romain Hérault
Université Caen Normandie, ENSICAEN, CNRS, Normandie Unio, GREYC UMR 6072,
P-14000 Coen, France
Clément Chatelain
INSA Rouen Normandie, LITIS UR 4108, F-76000 Rouen, France
Reviewed on OpenReview:
https: // hptp///epprr..e..e//m///e. net/ form?? id= YYTT26DbN
Abstract
In this work, we address the challenge of multi-domain translation, where the objective is to learn mappings between arbitrary configurations of domains within defined set (such as (Di, + D3, D2 + (D), D3 D1, etc. for three domains) without the need for separate models for each specific translation configuration, enabling more efficient and flexi- ble domain translation. We introduce Multi- Domtan Diffusion (MDD), method with dual purposes: i) reconstructing any missing views for new data objects, and ii) enabling learn- ing in semi-supervised contexts with arbitrary supervision configurations. MDD achieves these objectives by exploiting the noise formulation of diffusion models, specifically model- ing one noise level per domain. Similar to existing domain translation approaches, MDD learns the translation between any combination of domains. However, unlike prior work, our formulation inherently handles semi-supervised learning without modification by repre- senting missing views as noise in the diffusion process. We evaluate our approach through domain translation experiments on BL3NDT, multi-domain synthetic dataset designed for challenging semantic domain inversion, the BraTS 2020 dataset, and the CelebAMask- -QQ dataset. The code for MDD and all data are publicly available
1 Introduction
A domain is a set of tensors drawn from the same probability distribution p(x), characterized by both shared features, common across related domains, and domain-specific features, that distinguish it from other domains. We define domain translation as function fs..s, :S Sy that projects the data representations from set of source domains S, to set of target domains Sy-
In scenario with L domains denoted by D (Di,..,, DrJ, aim to obtain model performing all translations such that fs,S, S; + S with S; E P(D), where P(D) represents the power set of D, and 8, D S; is the complement of S, within D. Our goal is to develop model that not limited a

---

Published in Transactions on Machine Learning Research (09/202))
D3
D1
D2
Figure 1: This example considers three domains: photographs of faces, sketches, and semantic segmentation. Supervision scenarios include full supervision (all domain samples available) and semi-supervised (some samples missing). As the number of domains increases, achieving full supervision becomes more challenging. This task can be challenging and tedious, especially when human intervention is required, such as obtaining a sketch and segmentaaiio for each face. Semi-supervised multi-domain translation aims to reduce the data collection burden by allowing missing samples.
specific translation direction, either during training or inference. Any domain should be available serve as a condition, while all remaining domains must be part of the generation. Given that S, can be any subset of DD there are 2 possible translation functions. Figure illustrates this scenario when L= 3 using the Cel (Karras et al. 2018) dataset, where D1, D2, and Da represent image, sketch, and its segmentation map, respectively. Specific instances 2(1), such as face x(1), are referred to as view, and combinations of related views as a data point = [z("), (2) .33))
Some translation configurations can viewed as conditional generation tasks (e.g. face generation, where multiple faces can be considered valid given a unique sketch). In contrast, others can be viewed as regression or classification tasks (e.g. a unique semantic segmentation expected given face). This paper primarily focuses generation and conditional generation configurations.
The MDD framework leverages the noise-removal property of diffusson models (Ho et al. 2020 Song et al. 2021 Rombach et al. 2022) and, like other frameworks, concatenate the domains in the input Wolleb et al. 2022 1 In contrast to other frameworks, MDD uniquely models semi-supervised information using higher noise levels for unavailable views and further models different noise levels per domain. During training, views with higher noise levels, indicating less information, will encourage the model to rely more on less noisy views to enhance its reconstruction capabilities. This approach transforms the task from simple reconstruction to complex domain translation learning based on the joint data distribution.
Our main contributions are summarized as:
We introduce the MDD framework, which incorporates multiple noise levels for each domain. This diffusion-based framework enables learning in a semi-supervised setting, allowing mapping between multiple domains.
We investigate how the noise formulation in MDD can be used condition the generation process on missing modalities, given set of available views.
2

---

Published in Transactions on Machine Learning Research (09/2025)
We conduct comprehensive evaluation of MDD's performance on different datasets with different modal- ities, using both quantitative and qualitative assessments.
2
Related Works
2.1
Domain Translation
Domain translation research has explored various generative models, including GANs
Goodfellow et al.
2014),
VAEs (Kingmal 2013), normalizing flows (Rezende Mohamod 2015 Grover et al. 2020; Sun ct al.
2019
and diffusion models (Sohl-Dickstein et al. 2015). Although approaches like Pix2Pix (Isola et al.
2.17),
CycleGAN (Zhu et al. 2017), and others (Mayet et al., 2023 Liu et al. 2017 Huang et al. 2018 Lee
2019) have demonstrated promising results, they face limitations in multi-modal settings and exhibit reduced scalability as the number of domains increases. Moreover, both CycleGAN and StarGAN are designed for unsupervised settings, neglecting the potential benefits of supervised examples. StarGAN (Choi et al. 2018) enables translation between domain pairs using a single network for each configuration. However, it does not address multi-modal scttings, where multiple modalities can be utilized as conditions simultaneously.
ct al. 2018
2.2 Domain Translation Using Diffusion Models
Diffusion models offer various conditional generation paradigms. These methods can be broadly categorized into two groups: those operating in the original high-dimensional pixel space and those operating in the latent space. Models working in pixel space project the condition onto the target manifold of a pre-trained model using forward diffusion on the condition (Li et al.) 2023 Meng et al., 2022a). These approaches leverage the fact that, for specific translation tasks, condition and generation domain can be visually close (e.g. CBCT to CT LL et al., 2023)). Methods operating in the pixel space face several limitations, including the need to have conditions and largets close together in the input space, not allowing multiple conditions, and requiring a careful balance between condition fidelity and generation realism (Meng et al. 2022a). Models working in
latent space use similar approach. They embed the condition into the latent space of the target domain,
allowing the use of a pre-trained diffusion model (Wang et al. 2022 Ramesh et al. 2022 Lin et al. 2023
However, they do not address the issue of multiple conditions and target domains. Recent work on guided
diffusion (Dhariwal & Nichol 2021 Ho & Salimans 2021 Wang et al. 2023 Cross- Zamirskk et al. 2023 has explored ways to enhance adherence to condition semantics, but shares similar limitations in multi-modal settings.
2.3 Conditional Diffusion Models Using Concatenation
Recent approaches have attempted to address the use of multiple conditionn or largels simullany through their concatenation (Xie al. 2024 Cross-Zamirski et al. 2023 Saharia al. 2022a Lyu & Wang 2022; Saharia et al. 2022b). However, they primarily focus on one-way translation with fixed domains. They do not address the defined multi-domain translation setting, where any domain serve as input or output during generation. Two main approaches have emerged to overcome these limitations: noisy condition and clean condition methods. Noisy condition: To allow a unified framework without a fixed configuration of condition and target domains, this class of approaches (Lugnayr et al. 2022 Sasaki et al. 2022 Mariani et al. 2024) introduces the concept of adding noise to the condition. During training, the condition is degraded using the same forward diffusion process, cnabling the model to learn to reverse the diffusion process for all domains using the joint data distribution. For example, RePaint (Lugmayr et al. 2022) applies a matching noise level between condition and generation and designs jumping mechanism to maintain generation faithful to the condition while significantly increasing the generation time.
Multi-Source Diffusion Models for Simultaneous Music Generation and Separation (MSDM) MSDM (Mariani et al. 2024) presents an innovative approach utilizing noisy conditions. The method proposes applying an equivalent amount of noise to both the condition domain and the generated target domain, enabling multi-domain translation learning in a supervised setting. While this formulation has
demonstrated success in music generation, related applications (Lugmayr et al. 2022 Meng et al. 2022a
3

---

Published in Transactions on Machine Learning Research (09/2025)
Chung et al., 2022 Corneanu al. 2024
Mayet et al. 2025 have highlighted the limitations of noisy conditions and the necessity for additional mechanisms to synchronize the condition and target domains.
These limitations motivate the exploration of alternative approaches, such as UMM-CSGM, which investi- gates the use clean conditioning.
Clean Condition: In contrast, clean condition approaches Xie et al. 2024 Cross-Zamirski al., 2023
Saharia al. 2022a Lyu Wang 2022 Saharia al. 2022bg keep the condition clean during both training and generation. This scheme allows for learning one-way translation conditioned on specific domain and produces successful results. However, it falls short in multi-domain translation setting, where any domain can become an imput or an output at generation time. Unified Multi-Modal Conditional Score-based Generative Model (UMM-CSGM): Recently, UMM-CSGM (Meng el al. 2022b) has alleviated the problems of clean and noisy conditioning. It aims to learn a multi-domain medical image completion task using a multi-in multi-out conditional score network. UMM-CSGM adopts the concept of clean conditioning and incorporates a code to indicate the conditional configuration by partitioning the domain into noisy targets or clean conditions.
While this formulation enables multi-domain translation in fully supervised context, our work adopts a different approach by embedding information about conditions and target domains directly into the noise level modeling during training. Our proposed method inherently facilitates the configuration of missing modalities and aims to overcome the limitations of previous methods in addressing flexible multi-domain translation scenarios.
3
Multi-Domain Diffusion (MDD) Method
3.1 Diffusion Model
Diffusion models learn a data distribution from training dataset by inverting a noising process. During training, the forward diffusion process transforms data point 2o into Gaussian noise Pr I) in steps by creating series of latent variables T1,.., ZT using the following equation
((2)(x111) N((z /1 B,5x-1,,111
(1)
Where B, is the defined variance schedule. With a = 1 - Br, a IT as and (O I), 2 can be marginalized at step t from using the roparametrization trick
It = Vā,zo + /1 - ã.C.
(2)
The reverse denoising process po(#.-1 1,,, from IT ~ NO I) and iteratively reducing the noise in the sequence IT. To. The model c, t) is trained to prodict the added noise to produce the sample o, at time step t using mean square error (MSE):
() allows generation from the data distribution by first sampling
L = Ee (loa + /1 āzc, t) - c112-
(3)
3.2 Existing Issues With Noisy Conditional Diffusion Models
We focus on noisy conditional diffusion models that concatenate different modalities as input to cg, as detailed in Sec. 2.3 These models face inherent challenges due to the shared noise level during forward and backward diffuuson processes across all domains. The fundamental issue arises from the discrepancy bet noise levels of available and unavailable views during training and inference. During training, unavailable views are replaced with pure noise, while other domains contain varying levels of noise. Consequently, t no longer accurately represents the true noise distribution across domains.
This discrepancy propagates to the backward diffusion process during inference (illustrated in Figs. and g where the conditional inputs must be degraded to match the target domain's noise level (as shown in the initial steps in (a) Fig. 2 where the sketch and segmentation mask contain significant noise). When the timestep approaches T. the condition becomes extremely noisy, retaining minimal semantic information. This
4

---

Published in Transactions on Machine Learning Research (09/2025)
(a) Noiay Condition Translation
(b) Multi-Domall Diffusion (MDD)
x
facn geperatiom
nnisy condition sletcch and mask
clean cundition slesch and mask
Figure 2: Schematic illustration of Noisy Conditional Diffusion Models vs MDD, for the same timesteps on SelebAMMask-HO for face generation given sketch and mask conditions. The generation proccods from left right, where & represents the model's (co) prediction at noise level [o. T). (a) MSDM' (Mariani al., (024)) employs noisy conditions. At each timestep, the model receives as input both the current generation with reinjected noise and the noisy condition. Initially, the highly noisy condition leads the model to predict blurred images and converge toward data manifold that may not preserve the conditional features (represented by the blue). As the generation progresses and noise in the condition decreases, the model gains access to the underlying conditional information and corrects its trajectory, moving toward the appropriate data manifold (represented by the red arca). The diffusion model must contimuously adjust its trajectory to accommodate new information that becomes available as noise decreases. (b) Unlike noisy conditional models, MDD utilizes clean conditions throughout the generation process (conditional domains are shown only at the initial timestep for clarity, though they are utilized at cach step). This enables the condition to guide the generation from the initial diffusion steps, resulting in intermediate generated images that are more strongly influenced by the conditional information. Zoom in for better details.
degradation causes the generation of the target domain to deviate from the intended semantics, resulting in the generation traversing different modes as it atlempts to align with the gradually revealed condit information as noise levels decrease. This phenomenon is demonstrated Fig. 3 where the noisy condition produces inconsistent outputs, exhibiting variations key facial features such as gender, hair color, and hair length. RePaint (Lugmayr et al. 2022) encounters similar issue, which it addresses by implementing jumping mechanism. This solution involves looping through the same generation steps multiple times to increase the semantic consistency, but at the cost of increased time and computational complexity.
In contrast, MDD's approach of modeling distinct timesteps domain enables the preservation of clean con- ditional imputs during inference (illustrated in Fig. 2 where the conditions remain noise-free). Consequently, the gencration process maintains greater consistency and is more effectively guided by the conditional in- formation throughout the diffusion process, as demonsttated in Fig. 3 where the intermediate steps exhibit stability and manifest the final image features from the early stages generation. This allows the diffusion model to focus refining the sample rather than correcting its trajectory.
3.3 Noise Modeling for Semi-Supervised Multi- Motii-Domain Translation
To address these interconnected problems, MDD introduces an augmented forward and backward diffusion process. It employs vector T of size L,, assigning separate ((0) for each of the L domains. This approach
5

---

Published in Transactions on Machine Learning Research (09/202))
Conditions
MSDM
MDD
MSDM
MDD
D
DO
DO
DO
DO
DO
MSDM
MDD
Figure Illustration comparing (top) noisy condition with MSDM* (Mariani et al., 2024) and (bottom) clean condition with MDD generation at equivalent timesteps on CelebAMask HQ for face translation, conditioned on sketch and semantic segmentation inputs. The noisy condition approach exhibits mode switching during generation, resulting in inconsistent facial attributes such gender, hair color, and length across limesteps. This occurs the noisy condition model must adjust its trajectory to accormmo- date emerging features from the condition, as illustrated in Fig. 2 In contrast, MDD maintains consistency throughout the generation process, leading to higher-quality domain translation with coherent feature preser- vation.
allows for more accurate modeling of noise levels across different domains, removing the discrepancy bet ween noise levels in training and issues of semantic deviation during inference.
The goal of MDD is to remove constraints predefined conditional domains for semi-supervised domain translation. During training, represents the ground truth data point, where @ denotes the available supervised views and æ @ (1 - m) denotes the unavailable views, and O denotes the Hadamard product (A @ B) = (Arj x Baz). The binary supervision mask m indicales the presence or absence of domain supervision. For instance, Fig. 4 the mask values are m = [1, 0, 1], indicating that the second domain is missing during training. During generation, x O m represents the conditional views, while æ & (1 m) represents the views to be generated. This illustrated in Fig. 5 where the mask values are m |0, 10, 1). indicating that the first domain is being generated from the second and third domains.

---

Published in Transactions on Machine Learning Research (09/2025)
(0) ((0)
(0)
0
2 10 (lep)
NO(II)
(1) 2 6/ ((< (iont -t (eal) [t(0) , ,t(2)))
11)
0
20
(2) ((2)
(2) Lo
Figure 4: Training procedure of MDD. This example illustrates the training process across three domains: Do (face domain), D1 (sketch domain), and D2 (semantic face segmentation). For given data point æ where D1 (sketch) is missing, substitute it with Gaussian noise (0, I) and set its corresponding time step (() to 7. In this example, the supervision mask is defined as Meap = (1,0, 1], and the noise levels are initially sampled as T = [100, 500, 200] (Eq. 4 or Algorithm Line 5).
After applying Eq. (6) (Algorithm) Line
these values become T = [100, T, 200).
A Algorithm MDD training process
1: repeat
2: 2o ~ q(x))
3: Msup the supervision mask
me 10.1)* a random conditional mask
Tsup Uniform ((),..,,))
1)
7:
2 ~ I(O,I)
8:
T = O (1 Maup) + Toup @ Maup
9.
T = 0 @ (1 mc) + T ④ me
10:
TT vā + v1 - -ÔTE
11:
IT z @ Msup) + ZT ④ Msup
12: Training with loss: L = ||c - c T)
13:
until convergence
During training, different noise levels are sampled for each domain:
Tsup Uniform((1..... T
(4)
encouraging learning the joint data distribution by using less noisy domains to reconstruct noisier ones. In semi-supervised settings, missing domains are handled by replacing their views with Gaussian noise and masking their corresponding time steps in T, where is replaced with T:
FT ④ mssp) + FT ④ msup with z NoO,I)
(5)
T = 7 @ (1 maup) + Teup @ maup
(6)
The forward diffusion process applied independently for each view according
IT VaTTo + vI-are
(7)
The loss is computed only on available domains. The training process illustrated in Fig. $ using three domains from the CelchAMask-HHl dataset with sketches. Missing samples are replacod with Gaussian noise, and their corresponding time steps {()) are T. Meanwhile, the supervised domains receive independently sampled noise levels.

---

Published in Transactions on Machine Learning Research (09/2025)
Unless specified otherwise, in addition to Eq. 6), MDD training includes additional step where subset
of domains is randomly selected as clean, with their associated I set to O (ef- Algorithm line
Algorithm 2 MDD generation process
1: oT ~ I)
2: for = T.... do
3: 6~ 10,
2~ ~~,(0,,I) if 1, else 2 0
Feond, = - (Ooltt + v Op(C)
= (1 m) + (()) @
7: 2T = It O (1 - m) + ((t)d 3 m
27 a ca (IT, T)) 0T2
3T
21-1 var
9: end for
10: return 20
This modification allows the network to recognize unavailable views as having a maximum noise level (mod- eled by T') and replaced by noise. During training, the model sees views with different noise levels, encour- aging exploit more inter-domain dependencics for reconstruction. The training procedure dctailed in Algorithm [ This modification also allows MDD to take input with different noise levels for direct gen- eration using clean condition, differing from other work where conditions must. have the same noise level the target domain or where condition and target domain are predefined. The new generation process is described in Algorithm 2 and illustrated in Fig. 5 where @(t) controls the amount of information in the condition during backward diffusion. Unless specified otherwise, @(t) is set to ø(t) = 0 in our results. We investigate different functions for p(t) in Appendix [.1] Intuitively, if ((t) results in a function producing a higher noise level, the generation will drift further away, allowing to produce more diverse results at the cost of less fidelity to the condition.
Similar to previous works image translation (Meng et al. 2022a), different strategies enable achioving trade-off between image diversity and faithfulness to the conditioning signal. Applying higher noise levels to the condition allows the generation process to deviate from a specific unique output, producing more diverse images in generative tasks where multiple distinct outputs are desired. Conversely, for discriminative tasks such as semantic segmentation, it is preferable to maintain a low noise level on the condition, idcally 0, to achieve higher precision the predicted outpuls.
The MDD approach addresses the multi-domain suni-supervised translation task by modeling an indepen- dent noise level for each domain. The design of sampling independent noise levels aims to transform the reconstruction task of noisy condition models into a translation task, wherein cleaner views are utilized to predict noisier views. This formulation inherently accommodates the semi-supervised setting by represent- ing missing views as noise. As all views are concatenated in the model input and the model simultaneously generates all modalities, it eliminates the need for duplications across various input or output configuratioss.
4 Experiments
4.1 Datasets
We validate MDD on three datasets, each containing more than two domains: 1) our proposed BL3NDT
synthetic dataset, 2) the BraTS 2020 (Menze 2014 Spyridon Bakas et al.) 2017 et al.
2019
2017 medical
dataset with missing modality completion task, and 3) CelebAMask-HO (Lee et al. 2020 dataset of face photos and masks, augmented with generated sketches.
Each dataset is evaluated under varying levels of supervision. Model performance assessed by removing a specific number of views from data point and subsequently regenerating them.
8

---

Published in Transactions on Machine Learning Research (09/2025)
cond
oo)
q
(c) X m
forward diffusion
-t-1
denoising
m = [0, 1, 1]
m71
x
Lt
Next Iteration
Figure Conditional generation procedure of MDD. This example illustrates the training process across three domains: Do (face domain), Dj (sketch domain), and D2 (semantic face segmentation). The first domain (face) is missing and is considered target domain to generate, while the other two domains (sketch and semantic face segmentation) are available and are considered condition. m is the mask indicating missing and available domains, and (tt) controls the noise added to the condition.
4.1.1 BL3NDT Synthetic Dataset
The Blender 3 Domain Translation (BL3NDT) dataset provides a lestbed for multi-domain translation frameworks with deterministic mapping. Generated using the open-source 3D software Blende it consists of 64x64 image triplets across three domains (cube, pyramid, iscosphere), each containing domain-specific and common features (Fig. 6 Each domain represents an object (cube, pyramid, icosphere) placed before two walls and floor, with controllable viewing angles. Generation parameters include object type, 3D position, camera angle, object color, floor color, and wall colors. Some parameters are common across domains, while others (position, camera angle, object color) exhibit semantic inversion between domains (Tab. D. By swapping generative parameters between domains, texel-to-pixes mapping is eliminated, compelling the model learn underlying generative parameters and increasing task difficulty. The dataset comprises 40,500 randomly generated image triplets.
We study the BL3NDT setting with different amounts of supervised data. A percentage of the dataset, denoted as N%, is considered supervised data points, with the remaining (100-N)% divided equally between pairs of domains, with E {100, 70, 10.0). For example, if N = 40%, 40% of the data is fully supervised, 20% is (cube, pyramid), 20% is (pyramid, icosphere), and 20% is (cube, icosphere). In addition, the bridge selling divides the dataset into 50% (cube, pyramid) and 50% (pyramid, icosphere) pairs, by removing the icosphere from half the data and removing the cube from the other half of the data. We call it bridge translation there never cube and a corresponding icosphere logether. Therefore, producing an icosphere from cube requires using bridge domain: the pyramids. Generation is evaluated using the Mean Average Error (MAE).
Table 1: Parameters controlling BL3NDT dataset generation. Position, Camera angle, and Object color semantics are swapped between domains, while floor and wall colors are shared.
Position
Camera angle
Object color a, colorl, color2 colorl, color2, a color2, a, colorl
Floor color
Wall 1 color
Wall 2 color
Cube Pyramid
a 1-a a pl, p2, p3
rl, gl, bl rl, gl, bl
r2, g2, b2
r3, g3, b3
p2, p3, pl
r2, g2, b2
13, g3, b3
Icosphere
p3, pl, p2
(a + %1
rl, gl, bl
r2, g2, b2
r3, g3, b3
httpps//www... www.://wender.co org/
9

---

Published in Transactions on Machine Learning Research (09/2025)
Figure 6: Two data points from BL3NDT dataset which provides a synthetic domain translation setup.
4.1.2 CelebAMask-HQ Dataset Augmented With Sketches
Evaluations conducted on the CelebAMask-HO dataset at 256x256 resolution. To oblain setting with more than two domains, each face is augmented with a corresponding sketch computed using pre-trained model (Chen et al. 2018)
The evaluation focuses on face-conditioned generation, as MDD designed for generation rather than dis- criminative tasks (e.g. classification, regression, segmentation etc.).
The same supervision settings BL3NDT is used with N-100% and N-0%. The quality of face generation is evaluated using established metrics from image-lo-inage translation, the Learned Perceptual Image Patch Similarity (Zhang et al. 2018 (OII(L and the Structural Similarity Index Measure (Wang et al. 2004 (SSIM).
4.1.3 BraTS 2020 Medical Dataset
Evaluations are conducted on the BraTS 2020 (Menze 2014 Spyridon Bakas et al. 2017 et al. 2019 2017 dataset comprising four MRI modalities (FLAIR, T1, Tlce, and T2), and 3D tumor segmentations. We use 2D slices (256x256) with linear normalization [0,1] and binary semantic segmentation.
As for CelebAMask-HOO, the evaluation focuses scans generation conditioned on other modalities, since MDD is designed for generation rather than discriminative tasks.
For BraTS 2020, we evaluate three supervision settings: P100% (fully supervised), P80% (80% probability of keeping each view =f()), and P50% (50% probability of keeping each view z/{)), and minimum of two views are retained for each data point. Removing random scans simulates realistic scenario where not all paticnts have complete scan sets. In the P50% setting, only 3% of the data retain all five modalities, 16% miss one modality, and 81% lack two or more modalities. This heterogeneous set setling presents a more complex and realistic challenge compared to the BL3NDT dataset. Missing modality completion on the BraTS 2020 dataset evaluated using the standard metrics for this task (Meng et al., [022b Li et al.) 2023 Xie et al. 2024): Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index Measure (Wang al. 2004) (SSIM). For segmentation generation, the Jaccard Index (Jaccard) is reported.
10

---

Published in Transactions on Machine Learning Research (09/2025)
4.2 Implemented Models
Comparison with State-of-the-Art Methods (sota) For evaluation, MDD is compared with two multi- domain translation paradigms, clean and noisy conditions, which do not speeify condition domain. For the clean condition paradigm, UMM-CSGM (Meng et al. 2022b) training scheme is used as described in Sec. 2.3 The MSDM (Mariani et al. 2024) training scheme is used for the noisy condition paradigm, which has shown competitive quantitative results in music generation and separation. UMM-CSGM uses binary vectors for flexible conditions and target domain definitions, applying forward diffusion only to the target domain and keeping the condition domain clean. MSDM applies the forward diffussin process to all domains during training, bringing them to the same noise level. In MSDM, conditional generation is performed by applying noise to the condition to match the noise level in the target domains. Since MSDM was initially designed for music generation, we kept only the noise scheduling for cach domain and referred to it as MSDM*. We refer to these models as 'sota' in the different results tables. In a semi-supervised setling, 'sola' methods are only trained on the supervised examples, ignoring semi-supervised data.
Comparison with Uni-Dircctional Domain Translation Methods In addition to multi-domain meth- ods, we conduct comparisons with state-of-the-arl diffusion translation models that are restricted to a specifie translation direction (fs,.5, St - 5t) using ControlNet (Zhang et al. 2023b) in Appendix [B6] While Con- trolNet does not strictly fit the introduced multi-domain translation setting, as it operates only on fixed, predefined translation directions, we include these comparisons for the sake of completeness.
Adaptation of State-of-the-Ar Art Methods to Semi-Supervised (ad.) To our knowledge, only UMM- CSGM, MSDM, and the like, are capable of addressing the defined multi-modal domain translation setting, To evaluate performance in both supervised and semi-supervised scenarios, we note that UMM-CSGM, MSDM, and ControlNet do not inherently support semi-supervised training. Therefore, we propose modified variants of cach model (UMM[N], MSDM* [N], and ControlNet(NT) that handle missing views by substituting (hem with noise. The different results tables refer to these adapted models as 'ad..
Multi-Domain Diffusion Ablations (ours) To extensively test MDD capabilities, further ablation is provided by removing the contribution of Eq. (named MDD MDD[NOISY], T is set as T = [4, 1, during training. During generation, missing domains are replaced with noise and their with T as specified in Eq. (6) Ablation of the additional step in Algorithm| []] line is also provided (named (RAN)). The different results tables refer to these adapted models as 'ours' and are analyzed in Appendix B.2
MDD training scheme is adapted to noisy conditions
Additional training details are provided in Appendix A
4.3 BL3NDT Results
In this section, we demonstrate that MDD effectively performs domain translation on datasets without strong pixel-to-pixel correspondence between domains. We examine three distinct supervision settings: (1) the fully- supervised setting, where all data views are available; (2) the semi-supervise setting, where some views are missing from data points; and (3) the bridge data setting, where certain domains are never simultaneously present in the training samples.
In Tab. 2 MDD is compared to the different baselines. For all amount of supervision considered, MDD outperforms every bascline. The lower the supervision (lower N), the more MDD outperforms other basclines.
Supervised performances: For N 100%, UMM-CSGM, UMM[N, and MDD exhibit comparable low MAE errors. In contrast, MSDM* and MSDM*[N] demonstrate notably higher MAE error. The similarity in results for UMM-CSGM, UMM[N), and MDD is expected, as they all utilize clean condition during the generation process, and the full supervision allows learning the translation lask. The higher MAE errors observed in MSDM* and MSDM'NN can be attributed to their noisy condition: the condition is degraded to the same level as the generation. This creates disharmony between condition and generation, as the model cannot leverage the condition during carlier steps. This shows how using a clean condition guides the generation in the right direction from the start of the diffusion process, resulting in more faithful generations.
11

---

Published in Transactions on Machine Learning Research (09/2025)
Table 2: MAEJ error (values are given in e-1 order) for different amounts of supervision on the BL3NDT dataset for cube-<(yyramd, icosphere) translation. 'sota' refers the original state-of-the-art methods without modifications, while 'ad'' refers to our adapted versions of these methods modified to handle semi- supervised scenarios.
BL3NDT MAE
100%
70%
40%
10%
0%
Bridge
ad. sota
MSDM
1.962
2.028
2.186
2.925
UMM-CSGM
0.359
0.503
0.852
4.601
M
1.976
1.944
1.971
2.060
2.321
3.036
UMM
0.353
0.422
0.443
0.544
3.324
3.444
MDD
0.316
0.338
0.361
0.399
0.434
1.199
Semi-supervised performances: As N decrcases, increase in MAE is observed across all models. This trend is particularly pronounced for the supervised models UMM-CSGM and MSDM* which do not utilize samples with missing views. In contrast, it is worth noting that MDD performances only slightly decrease for N E {70, 40, 10). The semi-supervised adaptation [SDM*[N] shows similar results to its original formulation as N decreases, indicating that the noisy formulation not well-suited for producing highly faithful results. For UMM(N], the performance decrease is more substantial than for MDD, suggesting that the clean condition is not the sole factor contributing to MDD performances. At N 0%, all methods except MDD fail to learn the task. It is possible that sampling t according to Eq. may serve additional form of data augmentation.
translation proves challenging, as can be seen in the qualitative images in Fig. 7 where the L1-map shows some shift of the object position, and the Tab. 2 where the Bridge column has the highest error of all supervision setting, with an MAE of 1. 1.199 compared to 0.316 for = 100%. Two factors may explain this: in the carly stages of the backward diffusion process, the pyramid translation is suboptimal and subject to signifieant noise. In addition, the semantic inversion makes the generation more sensitive to noise, e.g. wrong prediction in the pyramid color will also affect the prediction of the camera angle (sec a in Tab. 1.
Bridge Translation:
The (pbe-
The intermediate bridge domain (pyramid) exhibits imperfect generation, with errors propagating to the subsequent icosphere translation. This challenge parallels the noisy condition issues discussed in Sec. 3.2 as the pyramid is unavailable for icosphere construction, similar to noisy translation models that lack access the condition at the diffusion process initiation.
This challenge mirrors issues encountered in diffusion- based inpainting, where the condition is initially noisy (Chung et al. 2022 Luginayr et al. 2022 Zhang et al.) 2023a). Potential solutions from this field could be adapted to our context, such as resampling steps for domain synchronization (Lugmayr et al. 2022) or
additional regularization costs (Zhang et al. 2023a Chung et al. 2022), both approaches introduce additional computational overhead.
While this issue impacts the MAE on BL3NDT, the usual domain translation tasks do not expect a one-to-one mapping (e.g. facet>sketch translation), which mitigates the problem in real-world applications.
4.4 CelebAMask-HQ Results
To assess MDD's capability in generating realistic views for challenging domain-to-domain translation tasks, evaluated domain translation between face, sketch, and segmentation mask on the CelebAMask-HH dataset. This section considers the (sketch, maak) translation and shows that MDD can generate realistic faces.
The diversity of face generation, sketch generation, and segmentation generation is evaluated in Appendix B.8 along with additional quantitative and qualitative results.
As shown by Tab. 3. MDD closely matches LPIPS of UMM[N] in the different supervision settings and improves SSIM. MDD significantly outperforms all other methods on both LPIPS and SSIM. In some
12

---

Published in Transactions on Machine Learning Research (09/2025)
Figure Bridge translation on (pyramdy icosphere). The left row shows the condition green and the generation in red. The right column shows the ground truth. Each row represents, from top to bottom, for each time in the backward diffusion process, the cube condition, the current pyramid translation, the current pyramid L1 map with the ground truth, the current iscophere translation, and the current icosphere L1 map with the ground truth.
instances, the UMM[N] method seems to produce oversaturated results but generally performs better than MSDM N]. This aligns with findings from other datasets, where the noisy condition paradigm consistently underperforms compared the clean condition paradigm.
GT
Conditions
ControlNet(N] MSDM*[N
UMM[N]
MDD
VOU
OOL
UU
Figure 8: Generated faces given sketch, mask condition, and the corresponding GT for N
13

---

Published in Transactions on Machine Learning Research (09/2025)
Table 3: E'aluation metrics for (sketch, segmentation) on the CelebAMask-HiQ dataset. 'ad.' refers
to our adapted versions of state-of-the-art methods modified handle semi-supervised scenarios.
CelebAMask-HOC
LPPPS4
SSIM1
Face
100%
0%
100%
0%
[NDM[[N]
0.3814
0.3785
0.1483
0.1306
ad.
UMM[N]
0.2930
0.3500
0.2902
0.2177
MDD
0.2305
0.3023
0.4385
0.3381
Table 4: Evaluation metrics for T1, Tlce, T2, and Flair, modalities completion on the BraTS 2020 dataset for different levels of supervision PN%, where N% indicates the probability of keeping a view, 'sota' refers to the original state-of- state-of-the-art methods without modifications, while 'ad.' refers our adapted versions of these methods modified to handle semi-supervised scenarios.
T1
Tice
PSNR
SSIM
PSNR
SSIM
P100%
P80%
P50%
P100%
P80%
P50%
P100%
P80%
P50%
P100%
P80%
P50%
MSDM
21.623
18.583
11.077
0.883
0.735
0.146
27.159
25.971
19.067
0.911
0.840
0.135
UMM-CSGM
22.806
17.769
13.280
0.894
0.680
0.064
26.947
26.488
26.558
0.910
0.898
0.892
MSDM
21.486
21.608
21.456
0.883
0.886
0.822
27.056
24.513
16.943
0.887
0.699
0.060
ad.
UMM[N
22.467
21.720
22.188
0.880
0.891
0.874
26.956
27.136
26.680
0.892
0.888
0.880
MDD
23.004
23.373
22.928
0.902
0.903
0.892
27.052
27.567
27.285
0.894
0.909
0.892
T2
Flair
PSNR
SSIM
PSNR1
SSIM
P100%
P80%
P50%
P100%
P80%
P50%
P100%
P80%
P50%
P100%
P80%
P50%
MSDM
24.679
21.485
15.844
0.824
0.775
0.090
22.801
18.829
17.764
0.872
0.785
0.229
UMM-CSGM
24.706
24.241
23.980
0.813
0.874
0.805
22.809
22.603
22.059
0.871
0.860
0.833
MSDM*NN
25.249
20.895
12.321
0.891
0.701
0.029
23.491
21.649
12.366
0.839
0.753
0.042
UMM[N]
24.726
24.843
24.036
0.882
0.887
0.861
23.110
23.384
22.726
0.845
0.840
0.827
MDD
25.383
25.716
24.788
0.875
0.894
0.877
24.173
24.103
23.663
0.876
0.879
0.868
4.5
BraTS 2020 Results
4.5.1 Missing Modalities Completion
To demonstrate MDD's flexibility, we evaluate its performance in regenerating each modality on the BraTS 2020 dataset, establishing that a single trained MDD model successfully handles the generation of all modalities.
Table presents quantitative results for PSNR and SSIM metrics, while Fig. provides qualitative visual results. Additional quantitative analysis using other metries (MSE, MAF) is provided in Appendix [B.7 We focus primarily on Tl scan generation analysis and observe that our findings generalize to other modalities (Tab. ! using only one model, highlighting the flexibility of multi-domain diffusion models compared to fixed diffusion models, which would require at least. one model for each domain.
The existing formulations, MSDM* and UMM-CSGM, demonstrate significant performance degradation under reduced supervision conditions. Specifically, UMM-CSGM's PSNR on T1 generation decreases sub- stantially from 22.81 to 13.28 when supervision is reduced from P100% to P50%, highlighting the inherent difficulty of this task with limited training data (Table 4 As illustrated in Fig. g these methods fail to gen- erate clinically acceptable images outside the fully supervised setling. Our adaplation of these formulations (MSDM* [N] and UMM[N) improves their resilience to supervision reduction, with both methods maintain- ing more stable performance metrics when supervision decreases, as evidenced in Tab. 4. Notably, MDD consistently outperforms the adapted UMM[N), despite the latter being specifically designed for missing scan completion tasks.
14

---

Published in Transactions on Machine Learning Research (09/2025)
GT
P50%
P80%
P100%
MSDM
UMM-CSGM [ontrolNet[N]
MSDM [N]
UMM[N]
MDD
Figure 9: Generated T1 scan given all remaining modalities and the corresponding GT for different super- vision levels.
4.5.2 Flexibility in the Number of Inputs
MDD demonstrates fiexibility in handling both missing modality completion and varying numbers of con- dition domains during inference. We explore this adaptability through two experiments: missing modalities completion (focusing on Tl scan generation results here) and semantic segmentation (Appendix B.7 each with a gradually docreasing number of input modalitics.
We gradually remove modalities the order [Segmentation, Tlce, T2, Flair] and regenerate the missing domains, focusing on T1 scan generation. Qualitative results are presented in Fig. 10 and quantitative rosults in Tab. 5
Models with noisy conditions demonstrate limitations in this setup. MSDM (N's PSNR decreases by ap- proximately 13% when the number of domains used drops from 4 to 1, compared to only a 4% decrease for MDD (Tab. 5) Generated Tl scans from MSDM*[N] are inconsistent when varying the number of condition domains, as observed in Fig. 10
Conversely, models with clean conditions perform well. MDD not only outperforms UMM[N] but also maintains consistency the number of available scans decreases. While UMM[N]'s PSNR shows slight drop between four scans and one scan, its generation can be less consistent than MDD when the number of condition domains becomes very small.
The superior performance of MDD demonstrates how using different per domain allows for learning better data fusion compared to using a binary code (UMM[N] or no code at all (MSDM*NN). MDD effectively learns the relative information different domains at each step, extracting data from less noisy domains reconstruct the more noisy ones. These results indicate that MDD adapts well varying numbers of inputs, particularly when condition domains become scarce. This suggests that MDD could be effectively used for
15

---

Published in Transactions on Machine Learning Research (09/2025)
multimodal data fusion in situations where modalities may be missing during inference, offering a robust solution for diverse medical imaging scenarios.
Table 5: Evaluation metric for Ti scan generation on the BraTS 2020 dataset while varying the number of input domains. 'ad.' refers to our adapted versions of state-of-the-art methods modified to handle semi- supervised scenarios.
Bra 2020
PSNR1
SSTM
T1 P50%
4 dom
dom
dom
1 dom
4 dom
dom
2 dom
1 dom
MSDM [N]
21.456
21.336
20.615
18.711
0.822
0.817
0.806
0.752
मे. MDD
UMM[N]
22.188
22.175
22.055
21.758
0.874
0.878
0.870
0.852
22.928
22.934
22.503
22.075
0.892
0.896
0.887
0.864
5 Limitations and Future Directions
In this section, discuss some of the potential areas for improvement of MDD.
While MDD effectively handles the fusion of information from multiple domains (Sec. 4.5.2 the scalability of combining information from an increasing number of domains within a single latent vector could become a bottleneck. As the number of domains grows, the efficient, integration and representation of multi-lomain information may present computational and architectural challenges. The current architecture could benefit from more advanced data, fusion strategies, particularly those designed for semi-supervised and imbalancod settings involving numerous domains (Han et al. 2024a). Such advanced fusion mechanisms could enhance the model's ability to handle heterogeneous data distributions and varying levels of annotation across do- mains.
Future work could explore the factorization of encoders and decoders across different modalities to improve scalability. Specifically, leveraging a powerful, general-purpose pretrained encoder could enable the extrac- tion of domain representations without domain-specific training. This unified encoder could be conditioned on the specific domain is processing through mechanisms similar to positional embeddings used in trans- former architectures (Vaswani et al., 2017). Similarly, single decoder could be employed generate outputs across all domains using the processod latent code from domain fusion, given gencral-purposo pretrained architecture. However, unlike the encoder, the decoder must be explicitly conditioned on the larget domain since its input (the latent code) is Nomain--gnostic This dual approach of shared encoder-decoder architec- ture with domain conditioning would facilitate scaling to multiple domains without a corresponding lincar increase model parameters at the encoder and decoder levels. While this approach might be well-suited for domain translation tasks where each domain can be represented the same representation space (e.g. all domains of BL3NDT, BraT'S 2020, and CelebA can be represented as images), it would present significant challenges for more heterogencous domains such image and text. In such cases, where the modalities differ fundamentally in their structure and dimensionality, this approach would not be directly applicable and would remain an open research challenge.
When working with fundamentally different modalities, MDD requires different encoders and decoders for each domain type, inherently limiting scalability to a very large number of domains. promising dircction lies in the strategic sharing of parameters across different components of the model, leveraging the inherent similarities between different domains. This could be achieved through parameter sharing techniques, particularly through mixture-of-experts architectures (Wu et al. 2024 Han et al.) 2024b Li ct al. 2025 Such approaches have the potential to efficiently scale to multiple domains while maintaining parameter efficiency through strategic sharing of weights across different domains. These methods could significantly reduce the computational overhead typically associated with multi-domain processing while preserving domain-specific expertise.
Another limitation of this study lies in the evaluation methodology. Although we employed diverse metrics that are widely accepted in the field, the incorporation of human preference evaluations could have provided
16

---

Published in Transactions on Machine Learning Research (09/2025)
4 dom
3 dom
2 dom
1 dom
GT
MSDM
UMM
MDD
MSDM
UMMNN
MDD
Figure 10: Generated Tl scans given multiple numbers of scans as input for P50%. Zoom in for better details.
additional valuable insights. However, such studies can be challenging to implement due to the need for humain annotators and standardized evaluation protocols.
6 Broader Impact Statement
As MDD a generative framework requiring training datasets, raises several important ethical consider- ations. These include concerns regarding image privacy, the inclusiveness of generated images which may exhibit. biases, and the potential for memorization and reproduction of training examples.
17

---

Published in Transactions on Machine Learning Research (09/2025)
containing geometric forms, which inherently circumvents issues related to data privacy, biases, and inclusivenes.. The additional datasets employed in this research are publicly available, thereby mitigating certain data rights and privacy concerns. Nevertheless, several critical aspects require careful attention, inchuding potential biascs in image generation, demographic representation, and training data memorization and reproduction.
MDD leverages synthetic datasets (Sec. 4.1.1
To address these challenges, we recommend the following measures: First, it crucial to ensure proper data usage rights and permissions for all training images. Second, the training dataset should be carefully curated to represent diverse populations and scenarios, avoiding demographic analysis of the generated image distribution should be conducted to detect and address any learned biases or potential privacy breaches. Additionally, regular auditing of the model's outputs and systematic evaluation of its societal impact should be performed to ensure responsible deployment of the technology.
contextual biases. Third, thorough
7 Conclusions
We demonstrate that the MDD framework effectively learns multi-modal, multi-domain translation in a semi- supervised setting by modeling domain-specific noise levels. This approach facilitates domain (ranslation learning without requiring the definition of a specific condition domain.
Unlike existing frameworks, MDD can handle a significant number of modalities without substantially in- creasing model size. Moreover, it exhibits superior information fusion from different modalities, making it particularly suitable for tasks with large number of domains.
Our rescarch clucidates how MDD unifies various approaches to applying noise to the condition in domain translation tasks. We found that maintaining a clean condition during generation yields excellent results, as it allows leveraging the condition from the start of the diffusion process. Using a noisy condition is particularly detrimental when tasks involve deterministic mapping, especially when the target domain is a semantic segmentation map. We attribute this issue to the diffusion model's need generate mean image without condition information at the start of the generation process, and subsequently correct erroneous predictions.
Our work integrates scamlessly with existing literature on diffusion frameworks that aim to learn multi-modal domain translation without defining a specific translation path (Meng et al.] 2202bb Bao et al. 2023 Mariani
ct al. 2024) and by applying different noise levels per modality (Meng et al. 2022b Mariani et al. 2024).
Extending these existing works, we demonstrate that the proposed formulation allows for semi-supervinn conditional domain translation. This can reduce the data burden in settings where data is difficult to acquire, such the medical field, and allow for flexible translation with different inputs.
References
Fan Bao, Shen Nic, Kaiwen Xue, Chongxuan Li, Shi Pu, Yaole Wang, Gang Yuc, Yue Cao, Hang Su, and Jun Zhu. One transformer fits all distributions in multi-modal diffusion at scale. In International Conference on Machine Learning (ICML), pp. 1692 1717. PMLR, 2023.
Chaofeng Chen, Wei Liu, Xiao Tan, and KKY Wong. Semi-supervised learning for face sketch synthesis in the wild. In Asian Conference on Computer Vision (ACCV), 2018.
Yunjey Choi, Minje Choi, Munyoung Kim, Jung- Woo Ha, Sunghun Kim, and Jaegul Choo. Stargan: Unified generative adversarial networks for multi-domain image-to toaaae translation, 2018.
Hyungjin Chung, Byeongsu Sim, Dohoon Ryu, and Jong Chul Ye. Improving diffusion models for inverse problems using manifold constraints. Advances in Newral Information Processing Systems, 0-255633-56000 2022.
Ciprian Corncamu, Raghudeep Gadde, and Alcix M Martinez. Latentpaint: Image inpainting in latent space with diffusion models. In Winter Conference on Applications of Computer Vision (WACV), pp. 4334 4334-4343, 2024.
18

---

Published in Transactions on Machine Learning Research (09/2025)
Jan Oscar Zaoss-Zamirski, Praveen Anand, Guy Williams, Elizabeth Mouchet, Yinhai Wang, and Carola- Bibiane Schönlieb. Class-guided image-to-inage diffusion: Cell painting from brightfield images with class labels. In International Conference on Computer Vision (ICCV), pp. 3800 3809, 2023.
Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances in Newral Information Processing Systems (NeurIPS), 84:8780-8794 2021.
Spyridon Bakas et al. Segmentation labels and radiomic features for the pre-operative scans the tega-gbm collection, 07 2017.
Spyridon Bakas et al. Identifying the best machine learning algorithms for brain tumor segmentation, progression assessment, and overall survival prediction in the brats challenge, 2019.
Ian J Goodfellow, Jean Pouget-Abadie, Mchdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. Advances in neural information processing systems, 27, 2014.
Aditya Grover, Christopher Chute, Rui Shu, Zhangjie Cao, and Stefano Ermon, Alignflow: Cycle consistent learning from multiple domains via normalizing flows. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pp. 4028-4033 2020.
Xing Han, Huy Nguyen, Carl William Harris, Nhat Ho, and Suchi Saria. Fusemoe: Mixture-of-experts trans- formers for fleximodal fusion. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024a. URL
Xing Han, Huy Nguyen, Carl William Harris, Nhat Ho, and Suchi Saria. Fusemoe: Nixture-of-experts trans- formers for fleximodal fusion. In The Tiirty-eighth Annual Conference on Newral Information Processing Systems, 2024b.
Jonathan Ho and Tim Salimans. Classificr-free diffusion guidance. In Advances in Neural Information Processing Systems (NewrIPS) Workshop on Deep Generalive Models and Downstream Applications, 2021.
URL https://openreviies http://ew.....e.oooooo////////aa
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Aduances in neural information processing systems, 33:6840-6851, 2020.
Xun Huang, Ming- Yu Liu, Serge Belongie, and Jan Kautz. Multimodal unsupervised image-to-image trans- lation. In European Conference on Computer Vision (ECCV), pp. 172-189, 2018.
Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexci A. Efros. Image-to-image translation with conditional adversarial networks. In Conference on Computer Vision and Pallern Recognition (CVPR), July 2017.
Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. Progressive growing of gans for improved quality, stability, and variation. In International Conference on Learning Representations, 2018.
DP Kingma. Auto-encoding variational bayes. arXio preprint 1rXi::1312.6114, 2013.
Cheng- Lee, Ziwci Liu, Lingyun Wu, and Ping Luo. Maskgan: Towards diverse and interactive facial image manipulation. In JEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
Hsin-Ying Ying Lee, Hung- Yu Tseng, Jia-Bin Huang, Maneesh Singh, and Ming-Hsuan Yang. Diverse image-to- image translation via disentangled representations. In European Conference on Computer Vision (ECCV), pp. 35-51, 2018.
Hsin-Ying Ying Lee, Hung-Yu Tseng, Qi Mao, Jia-Bin Huang, Yu-Ding Lu, Maneesh Singh, and Ming-Hsuan Yang. Drit++: Diverse inage-to-inage translation via disentangled representations, 2019.
Yunxiang Li, Hua-Chich Shao, Xiao Liang, Liyuan Chen, Ruiqi Li, Steve Jiang, Jing Wang, and You Zhang. Zero-shot medical image translation via frequency-guided diffusion models. IEEE transactions on medical imaging, 2023.
19

---

Published in Transactions on Machine Learning Research (09/2025)
Yunxin Li, Shenyuan Jiang, Baotian Hu, Longyue Wang, Wanqi Zhong, Wenhan Luo, Lin Ma, and Min Zhang. Uni-moe: Scaling unified multimodal llms with mixture of experts. IEFE Transactions on Pattern Analysis and Machine Intelligence, 2025.
Yupei Lin, Sen Zhang, Xiaojun Yang, Xiao Wang, and Yukai Shi. Regeneration learning of diffusion models with rich prompts for zero-shot image translation, 2023.
Ming- Yu Liu, Thomas Breuel, and Jan Kautz. Unsupervised imago-to-image translation networks. Advances in Neural Information Processing Systems (NewrIPS), 30, 2017.
Andreas Lugmayr, Martin Danclljan, Andres Romero, Fisher Yu, Radu Timofte, and Luc Van Gool. Repaint: Inpainting using denoising diffusson probabilistic models. In Conference on Computer Vision and Paltern Recognition (CVPR), pp. 11461-11474 2022.
Qing Lyu and Ge Wang. Conversion between et and mri images using diffusion and score-matching models, 2022.
Giorgio Mariani, Trene Tallini, Emilian Postolache, Michele Mancusi, Luca Cosmo, and Emanuele Rodolà. Multi-source diffusion models for simultaneous music generation and separation. In International Confer- ence on Learning Representations (ICLR), 2024.
Tsiry Mayet, Simon Bernard, Clement Chatelain, and Romain Herault. Domain translation via latent space mapping. In 2023 International Joint Conference on Neural Networks (IJCNN), pp. 1-10. IEEE, 2023.
Tsiry Mayet, Pourya, Shamsolmoali, Simon Bernard, Eric Granger, Romain HÉRAULT, and Clement Chate- lain. TD-paint: Faster diffusion impainting through time aware pixel conditioning. In The Thirteenth International Conference on Learning Represenlations, 2025. URL. htttp:://oeenreviee..eee/// net/forum?id:
erW*BoR591
Chenlin Meng. Yutong He, Yang Song, Jiaming Song, Jiajun Wu, Jun-Yan Zhu, and Stefano Ermon. Sdedit: Guided image synthesis and editing with stochastic differential equations. In International Conference Learning Represendations (ICLR), 2022a.
Xiangxi Meng, Yuning Gu, Yongsheng Pan, Nizhuan Wang, Peng Xue, Mengkang Lu, Xuming He, Yiqiang Zhan, and Dinggang Shen. A novel unified conditional soorebbaed generative framework for multi-modal medical image completion, 2022b.
Bjoern H et al. Menze. The multimodal brain tumor image segmentation benchmark (BRATS). IEEE Trans Med Imaging, 34(10):1993 2024, December 2014.
Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In International conference on machine learning, pp. 8162 8171. PMLR, 2021.
Mang Ning, Enver Sangineto, Angelo Porrello, Simone Calderara, and Rita Cucchiara. Input perturbation reduces exposure bias in diffusion models. In Proceedings of the 40th International Conference on Machine Learning, pp. 22245-26265, 2023.
Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image generation with clip latents, 2022.
Danilo Rezende and Shakir Mohamed. Variational inference with normalizing flows. In International Con- PMLR, 2015.
ference on Machine Learning (ICML), pp.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 10684-10695, 2022.
Chitwan Saharia, William Chan, Huiwen Chang, Chris Lec, Jonathan Ho, Tim Salimans, David Fleet, and Mohammad Norouzi. Palette: Imago-to-image diffusion models. In ACM SIGGRAPH 2022 conference proceedings, pp. 1-10, 2022a.
20

---

Published in Transactions on Machine Learning Research (09/2025)
Chitwan Saharia, Jonathan Ho, William Chan, Tim Salimans, David J Fleet, and Mohammad Norouzi. Image super-resolution via iterative refinement. Transactions on Pollern Analysis and Machine Intelligence (TPAM)) 45(4):4713 4726, 2022b.
Hiroshi Sasaki, Chris G. Willcocks, and Toby P. Breckon. Unit-ddpm: Unpaired image translation with denoising diffusion probabilistic models, 2021.
Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequllibrium thermodynamics. In International Conference on Machine Learning (ICML), pp. 2256-2265. PMLR, 2015.
Jiaming Song, Chenlin Meng, and Stelano Ermon. Denoising diffusion implicit models. In International Conference on Learning Representations, 2021.
Spyridon Bakas et al. Advancing The Cancer Genome Atlas glioma MRI collections with expert segmentation labels and radiomic features. Sci Data, 4:170117, September 2017.
Haoliang Sun, Ronak Mehta, Hao H Zhou, Zhichun Huang, Sterling C Johnson, Vivek Prabhakaran, and Vikas Singh. Dual-glow: Conditional flow-based generative model for modality transfer. In International Conference on Computer Vision (ICCV), pp- 10611-10620, 2019.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural informalion processing systems, 30, 2017.
Qiang Wang, Di Kong, Fengyin Lin, and Yonggang Qi. Diflsketching: Sketch control image synthesis with diffusion models, 2023.
Tengfei Wang, Ting Zhang, Bo Zhang, Hao Ouyang, Dong Chen, Qifeng Chen, and Fang Wen. Pretraining is all you need for image to-image translation, 2022.
Zhou Wang, A.C. Bovik, H.R. Sheikh, and E.P. Simoncelli. Image quality assessment: from error visibility structural similarity. IEEE Transoctions on Image Processing, 13(4):600- 612, 2004. doi: 10.1109/TIP. 2003.819861.
Julia Wolleb, Robin Sandkühler, Florentin Bieder, Philippe Valmaggia, and Philippe C Cattin. Diffusion models for implicit image segmentation ensembles. In International Conference on Medical Imaging with Deep Learning, pp. 1336-1348. PMLR, 2022.
Jialin Wu, Xia Hu, Yaqing Wang, Bo Pang, and Radu Soricut. Omni-smola: Boosting generalist multimodal models with soft mixture of low-rank experts. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 122511 2024.
Taoleng Xie, Chentao Cao, Zhuo-xu Cui, Yu Guo, Caiying Wu, Xuemei Wang, Qingneng Li, Zhanli Hu, Tao Sun, Ziru Sang, et al. Synthesizing pet images from high-field and ultra-high-field mr images using joint diffusion attention model. Medical Physics, 51(8):5250 5269, 2024.
Guanhua Zhang, Jiabao Ji, Yang Zhang, Mo Yu, Tommi Jaakkola, and Shiyu Chang. Towards coherent image inpainting using denoising diffusion implicit models. In International Conference on Machine Learning, pp. 41164-41193. PMLR, 2023a.
Lymin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 3836-3847, 2023b.
Richard Zhang, Phillip Isola, Alexei A. Efros, Eli Shechtman, and Oliver Wang. The unreasonable effective- ness of deep features as a parceptual metric. In Conference on Computer Vision and Pattern Recognition (CVPR), June 2018.
Yan Zhu, Tacsung Park, Phillip Isola, and Alexci A Efros. Unpaired imago-to-image translation using cycle-consistent adversarial networks. In International Conference on Computer Vision (ICCV), pp. 2223- 2232, 2017.
21

---

Published in Transactions on Machine Learning Research (09/2025)
Supplementary Material
A Implementation Details
A.1 Diffusion Model
During training,
linear noise schedule or (le- 2e -22) and the noise prediction function described
in Sec. 3.11 The maximum number of diffusion steps T is 1 1000. During generation, DDIM (Song et al. 2021 with 100 steps is used for BL3NDT, DDPM (Ho et al., 2020) with 1000 steps for BraTS 2020, and DDPM with 1000 steps for CelebA when faces are part the generation; otherwise, DDIM with 250 steps for the other CelebA HQ modalities.
A.2 Training Details
A.2.1
Multi-Domain Translation Models
For all multi-domain translation models (MSDM*, UMM-CSGM, and MDD), we employ consistent hyperpa- rameter settings within each dataset. We utilize the Adam optimizer with an initial learning rate of 7 x 10-5 for BraTS 2020 and 2 x 10-5 for both BL3NDT and MaskkAM datasets. The Adam parameters 31 and 32 are set to 0.9 and 0.999, respectively. For BL3NDT and BraTS 2020, we implement a learning rate decay strategy, multiplying the rate by 0.75 every 10 epochs. We employ batch sizes of 128 for BL3NDT, and 64 for both BraTS 2020 and CalebAMask-HOQ. All models trained with Exponential Moving Average (EMA) model parameters with decay rate of 0.9999. For BL3NDT, we implement early stopping with patience of 40 epochs and maximun computational budget of 240 epochs. For BraTS 2020, all models are trained for 200 epochs, while for CelebAMask--HG HQ, training continues for 2000 epochs.
A.2.2 Uni-Direction Translation Models
For ControlNet implementations (Zhang et al. 2023b), we use the publicly available code repository and select each model based on its best validation metrics. For the BL3NDT dataset, we train with an unlocked decoder for 1000 epochs with a batch size of 128. For BraTS 2020, training proceeds with an unlocked decoder for 700 epochs using a batch size of 128. For CelebAMask-HHQ HHQ we conduct training for 2000 epochs with a batch size of 1024.
A.3 Data Normalization
For BL3NDT, images are linearly normalized between [-1,1], and no data augmentation is applicd.
For BraTS 2020, each 3D scan is independently linearly normalized between [0.1], and the semantic segmen- tation classes are merged to create a binary segmentation and are one hot encoded with one channel. The bottom 80 and top 26 slices are removed. Each resulting 3D volume is sliced in the axial axis and resized to (224, 224).
For CelebAMask-HHO, faces and sketches are linearly normalized in [-1,1], and the 19 classes are one hot encoded. We resize each modality in (256,256) and use random horizontal flips data augmentation.
A.4 Model Architecture and Adaptation to Multiple Domains
All models use the same U-Net architecture based on (Ho et al. 2020) with some modifications to accommo- date multiple domains and times, illustrated in Fig. 11 We call E the U-Net encoder, B its bottleneck, and D its decoder. In a setting with L domains, the encoder and decoder are duplicated L-times and function as in the classical single-domain diffusion setting, i.e. the encoder E() takes as input the modality x!)) and the time T and produces its embedding e(1) and a list of skip connections () The list of emboddings (e()) ...., e(") and times is then processed by a bottleneck adapted for multi-domain multi-time, which we will describe later, produce the processed embedding emb. Each decoder DXU) takes input the processed
22

---

Published in Transactions on Machine Learning Research (09/202))
stip connections
(0)
chennekotonn concal
(0)
(0))
x (1)
(1)
(1)
eggregation network
balllenck
cmb
cmb
D
< (2)
(2)
{(2)
Temb
T Temb
cmob
(2)
Tatan
MLP
Temb
(t(()) (00) 2
Figure 11: Illustration of the MDD architecture with UNet backbone on domains. Each domain has its own encoder E()) and decoder ( that processes modalities individually using their associated timesteps z(0), The embeddings from each modality are concatenated and aggregated for processing through the bottleneck. Subsequently, the processed embeddings are passed through individual decoders D(0), with skip connections from their corresponding encoders g(0 to predict the noise e added to each domain. In the aggregation network and bottleneck, time embeddings are first flattened, then processed through an MLP for scalo-shift operations, following standard practices.
embedding emb, the list of skips connections skips ( and time T and predicts the noise map for the modality x(!) as the classical single-domain diffusion setting.
To accommodate multiple domains, an aggregation network is added before the bottleneck, taking as input the list of embeddings (e()) ..., e..) each of shape (b, h, w) and times T. The embeddings are concatenated the channel dimension before going through ConvNextBlock which reduces their dimension from x L back to They are then processed through the bottleneck layers the original U-Net would.
Each layer that uses the entire T is modified. The T of shape (b, L) is embedded once at the beginning of the forward pass using the initial time-embedding MLP into a tensor shape (b, L, tdim), then reshaped into shape (b, L x tdim). Then, each bottleneck CoxvNextBlock time MLP input dimension is modified to take as input a vector of shape (b, (dim) instead of taking a vector of shape (b, Idim).
B Additional Experiments
B.1 Exploring the Noise on Condition Strategy
During MDD training, there is no domain identified condition domain question is how to use the condition during the generation process: should the condition be kept clean (using Foond,0, see Algorithm [ or should it contain the same level of noise as in generation (using &rond, see Algorithm 2 We experiment on BL3NDT dataset to compare the effect of different noise levels during the generation process and define four noise strategics for the condition described in Fig. T2 We define d, (t) as the function that allows to obtain the noise level in the condition domains according to the noise level present in the generated domains.
target domain, the
23

---

Published in Transactions on Machine Learning Research (09/202))
100
Φe Vanilla noise Φ Skip noise
80
Φe Constant noise
s Constant noise fading
60
40
20
0
0
20
40
60
80
100
Number of transitions
Figure 12: Illustration different o, (t) functions. Evolution of the noise applied to the condition as a function of the number of transitions made for the target domain. Vanilla diffusion steps have the drawback of not properly preserving semantic information of the condition and requiring multiple jumps. Skip Noise diffusion removes information of the condition in the same way as the Vanilla diffusion step, but allows cleaner condition at the start, therefore better preserving semantic information from the condition. Constant noise continuously removes the same amount of information at each step. We found this solution to work best in practice with a low amount of 20% the total number time steps. We also tried applying constant noise to the condition until the noise level of the target domain caught up with the condition.
Approaches that learn the joint distribution apply the same noise level on each modality during training (Marianí et al. 2024) and thus also during generation. This strategy is called Vanilla noise bocause it closoly follows the original backward diffusion process. When a condition domain identified, often kept clean during training and generation (Xie et al.] 2024 Cross-Zamirski al. 2023 Saharia al.] 2022a; Lyu & Wang (2022). We identify this strategy as Constant Noise, which can be parameterized by a noise level, clean condition is identified which is vanilla noise, where the noise applied to the condition is less than that applied to the generation, and Constant Noise Fading, where the noise remains constant until it catches up with the generation.
Constant Noise(0). We also explore two additional strategies: Skip Noise,
For the BL3NDT dataset with the Bridge data setting, we found that the higher the y, the lower the MAE. This is not surprising, as the higher the 7, the less noisy the condition, allowing the model to use information from the condition carly the generation process and avoid drifting too far from the correct semantic. Interestingly, the Skip Noise function has a low MAE even for relatively high level of 7. We speculate that this noise scheme still allows the diffusion model to focus first the low-level frequency that is crased from the condition and, later on, the high-level frequency details.
B.2 Ablation Study
To extensively lest MDD capabilities, further ablation is provided by removing the contribution of Eq. 44
MDD training scheme is adapted to noisy conditions (named MDD[NOISY]. T set as T [t, t, during training. During generation, missing domains are replaced with noise and their with T as specified Eq. (6. Ablation of the additional step in Algorithm [ line g also provided (named MDD/RAND), The different results tables refer to these adapted models fours!
24

---

Published in Transactions on Machine Learning Research (09/2025)
0.4
Vanilla noise
Skip noise
Φ Constant noise
af Constant noise fading
0.3
MAE
0.2
0
20
40
60
80
100
Y Level
Figure 13: Comparison of the MAEJ for different (t) functions according the y parameter on BL3NDT dataset using the Bridge data setting. In general, the smaller the y, the higher the MAE. The smaller % the noisier the condition, and the harder it is for the model to use the information within the condition in the early steps.
Noisy generation: MDD[NOISY). Using a noisy condition instead of a clean condition significantly increases the translation error, shown in Tab. D This is more true for the BL3NDT dataset than for the other dalasets, because the BL3NDT dataset expects specific output with challenging variable in- versions between domains. For other datasets, such as BraTS 2020, MDD also consistently outperforms MDD[NO1SY], as shown in Tabs. to 16
For translation tasks where the larget semantic segmentation, MDD[NOIS]] fails the task as shown Tabs. 17 and 19 by producing realistic segmentation but unaligned with the condition, which is consistent with other noisy condition models such MSDM*[N] (Fig, 22). When generating segmentation maps from other condition domains, MDD[NOISY] (and other noisy condition models) has no mechanism to distinguish which domain is part of the generation and which is part of the condition; therefore, correcting the target domain (in this case, the segmentation) becomes particularly challenging. A similar problem has been reported in other works (Lugmayr et al. 2022 Chung et al. 2022 where the current generation and condition are not well synchronized. Proposed solutions involve resampling mechanisms (Lugmayr et al. 2022 Chung al. 2022), which increases computation time, making prediction of many images impractical for most institutions.
This validates the assumption that using clean condition allows diffusion models to leverage the condition from the early steps of diffusion, leading to more faithful and realistic generations.
Condition vector m.: MDD[RAND). Tweaking the noise scheduling during training allows MDD to better follow a specific set of domains during generation. We found that this formulation beneficial for tasks where a specific output is expected, such as the BL3NDT dataset (Tab. G and scgmentation tasks (Tabs. 17 and 19). For other translation tasks, MDD performs better than RAND), but with smaller
metric difference (Tabs. 13 to 16
25

---

Published in Transactions on Machine Learning Research (09/2025)
Table 6: BL3NDT ablation, MAEJ error (values are given in e-1 order) for different amounts of supervision on the BL3NDT dataset for cube -ube-mmyyamidd icosphere) translation.
BL3NDT MAE4
100%
70%
40%
10%
0%
Bridge
MDD [OISY]
1.977
2.001
2.030
2.158
2.740
4.022
ours
MDD RAND]
0.616
0.652
0.715
0.803
0.565
1.294
MDD
0.316
0.338
0.361
0.399
0.434
1.199
B.3 Sampling Strategy Impact on Diversity
To further validate the flexibility of MDD in modeling different noise levels, we evaluate the noise strategies presented in Appendix B.1 applied to CelebAMask-HO for face translation with sketch and segmentation conditions using full supervision. Results are presented in Fig. [4] For computational efficiency, we employ DDIM with 100 steps instead of DDPM with 1000 steps. While this results in a slight degradation of LPIPS compared to the results presented in Tab. the performance remains superior to other basclincs.
For most Qy strategies, a modest increase in y leads decreased image quality (LPIPS) and, notably, without an increase in image diversity, This phenomenon may be attributed to the initial noise injection removing important features from the sketch that affect LPIPS computation, while being insufficient to introduce meaningful diversity in the generalion process. Increasing noise injection into the condition enables greater image diversity, albeit at the cost of image quality, establishing a clear trade-off between these metrics. Fundamentally, these metrics are inherently antagonistic in domain translation tasks: while we aim to generate samples that closely match the ground truth, perfect fidelity would result in zero diversity, making it challenging to optimize both metrics simultancously.
Further research is needed to develop additional mechanisms for MDD to achieve better control over gen- eration diversity. One potential approach would be to implement a progressive condition relaxation during generation, whereby the condition transitions from a fixed constraint to becoming part of the generation domain. This controlled transition would enable the generated output to gradually deviate from the initial condition, offering a more structured approach to diversity than simple noise injection.
B.4 Sensitivity to Noise Schedulers
We evaluate MDD's adaptability to different noise schedulers by examining its performance across multi- ple scheduling strategies. In our experiments, we employ DDPM (Ho et al. 2020) for BraTS 2020 and CelebAMask-HIQ datasets, while utilizing DDIM (Song et al. 2021 for BL3NDT to optimize computational efficiency. Both DDPM and DDIM implement lincar noise schedules. We further extend our analysis by retraining the model with the cosine scheduler Nichol & Dhariwal 2021). which provides more gradual noise application at higher noise levels. Results are presented in Tab. 7 for the Bridge data setting on BL3NDT and in Tab. g for the P50% data sctting on BraTS 2020, focusing T1 scan generation.
MDD demonstrates robust performance across different noise schedulers, with the cosine scheduler achieving comparable MAE scores to DDIM with linear scheduler on BL3NDT, and even surpassing previous results with fewer sampling steps. Similar behavior is observed for the BraTS 2020 dataset, where the cosine scheduler performs as well as the linear scheduler while requiring only a fraction the sampling steps.
These findings suggest that MDD robust to different scheduler choices and could potentially leverage recent advances in more sophisticated scheduling strategies.
Notably, significantly reducing the number of diffusion steps does not substantially degrade generation per- formance. For BL3NDT, and particularly for BraTS 2020, reducing the steps from 1000 to 50 produces minimal differences in SSIM scores. This resilience might be attributed to the miligation of exposure bias [Ning et al., 2023) at inference time, where the distribution of inputs typically deviates from the training distribution and errors accumulate during diffusion stops. Fewer diffusion steps in DDIM may help alleviate this issue by reducing the cumulative error, while the noise reinjection process in DDPM provides additional regularization during sampling.
26

---

Published in Transactions on Machine Learning Research (09/2025)
20
Diversity
19
Constant noise fading de Vanilla noise Skip noise Constant noise
18
17
0.25
0.3
0.35
0.4
0.45
0.5
LPIPS4
Figure 14: Trade-off between diversity and LPIPS across different noise sampling strategies Φy while varying the noise level parameter 7y. Marker sizes correspond to y values, where larger markers indicate lower y values, representing increased noise injection into the condition. Note that. 7 values cannot be directly compared across different noise sampling strategies by as each strategy implements this parameter distinctly.
Table 7: MDD performance with different noise schedulers on BL3NDT Bridge setting for cube- icosphere) translation.
Sampler
Number Of Steps
MAE
DDIM Cosine
50
0.974
DDIM Cosine
100
0.979
DDIM Linear
100
1.199
B.5 MDD Computational Analysis
We analyze MDD's computational characteristics in terms of generation speed, parameter count, floating- point operations (FLOPs), and muuttply-accumulaa operations (MACs). FLOPs and MACs measured using Calflops for single complete forward pass of the model. Generation time is computed over 50 diffusion steps for 1 image and averaged across 100 runs on NVIDIA A100 GPU. We denote the time required for translating between n domains as nDT seconds. The translation time for n domains using ControlNet is computed by multiplying the time required for single domain generation by n, as domains must be generated sequentially rather than simultaneously. Results are presented Tab.
The multi-domain translation baselines (MDD, MSDM*, UMM-CSGM) demonstrate comparable generation speeds, as they share similar UNet backbone architecture and generale all missing domains simultaneously, Consequently, the generation time for domains remains constant across these models. Tn contrast, the single-domain translation baseline ControlNet, although faster for individual domain generation, may become less efficient in scenarios requiring multiple output domains, as it necessitates separate generation processes for each domain.
For 1, ControlNet achieves generation time of 7.5s, approximately 2.7 times faster than MDD (20.1s). This difference can be attributed to ControlNet operating in the latent space, while MDD operates in the
httpp://gthhu..cc////////lllllld
27

---

Published in Transactions on Machine Learning Research (09/2025)
Table 8: MDD performance with different noise schedulers on BraTS 2020 P50% setting for Tl scan gener- ation.
Sampler
Number Of Steps
PSNRT
SSIM
DDIM Cosine
50
21.699
0.894
DDIM Cosine
100
22.387
0.881
DDPM Linear
1000
22.928
0.892
pixel space. Operating in the latent space could potentially improve MDD's performance, and we plan to extend approach to incorporate latent space operations in future work.
For = 2, MDD (20.1s) is slightly slower than ControlNet (14.9s), and for n 2 3, where MDD is designed to operate, it outperforms ControlNet in terms of generation time. The efficiency gap between MDD and uni-domain generation approaches like ControlNet widens as the number of domains increascs. For instance, with n 5, MDD (20.1s) 1.9 times faster than ControlNet (37.3s).
Additionally, ControlNet's multi-domain generation (n 2 2) relies on using generated domains as conditions for subsequent generations, which may introduce domain adaptation issues and error accumulation across multiple generations, potentially limiting its practicality in such scenarios.
Furthermore, though not quantified in our analysis, the storage and training time requirements for Con- trolNet should be considered, as each translation configuration requires a separate model to be trained and maintained.
Table 9: Computational complexity comparison across different domain translation methods. We report parameter count, FLOPs, MACs, and generation time (nDT) in seconds for translating domains. All measurements are performed on NVIDIA A100 GPU.
Multi-Domate Translation
Model
TFLOPs
MACs
Params
1DT
2DT
3DT
4DT
5DT
MDD
1.4795
733.556
GMACs
266.639 M
20.1
20.1
20.1
20.1
20.1
UMM-CSGM
1.4795
733.556
GMACs
266.639 M
20.6
20.6
20.6
20.6
20.6
MSDM*
1.4795
733.556
GMACs
266.639 M
20.1
20.1
20.1
20.1
20.1
ControlNet
24.8474
12.4135 TMACs
1.2301
7.5
14.9
22.4
29.8
37.3
B.6 Comparison Model with a Fixed Configuration
We compare MDD with ControlNet (Zhang et al. 2023b), diffusion-based domain translation framework that enables translation from a fixed set of conditional domains to specific target domain. While Control- Net limited one specific configuration among the 2" possible translation configurations, making inherently different from our framework's flexibility, we include this comparison to demonstrate the advantages of our approach.
B.6.1 Translation With Complex Semantic Inversion
The translation cube->pyrramd, icosphere) is evaluated. For this purpose, ControlNet requires training two separate models: one for cube-tpyramid and another for cube-ricosphere, In the bridge supervision setting (where 50% of (cubo, pyramid) pairs and 50% of (pyramid, icosphere) pairs are available), two distinct mod- els must be trained: cube-xpyramid and pyramid->cosppeen using
Moreover, for the bridge translation scenario, requires generated pyramid images as conditions, which can introduce domain adaptation challenges due to potential distribution shifts between generated and real images. This demon- strates a significant limitation of ControlNet, as different models must be trained for different configurations, whereas MDD efficiently handles all configurations using single unified model.
28

---

Published in Transactions on Machine Learning Research (09/2025)
Table 10: Evaluation of translation domain with a fixed-coniiuuration using ControlNet. MAE] error (values are given in o-1 order) for different amounts of supervision on the BL3NDT datasct for cube-t(pyramid, icosphere) translation.
BL3NDT MAEJ
100%
70%
40%
10%
0%
Bridge
ControlNet
1.707
1.946
2.116
2.315
2.408
2.390
MDD
0.316
0.338
0.361
0.399
0.434
1.199
Table 11: Evaluation translation domain with a fixed-cofiiguratoon using ControlNet. Evaluation metrics for T1 modality completion on the BraTS 2020 dataset for different levels of supervision PN%, where N% indicates the probability of keeping view.
BraTS 2020
PSNR
SSIMI
T1
P100%
P80%
P50%
P100%
P80%
P50%
Net
24.929
24.665
23.968
0.890
0.869
0.832
MDD
23.004
23.373
22.928
0.902
0.903
0.892
We report the MAE results in Tab. 10 Despite training with an unlocked stable diffusion decoder Con [ro]Net[N] fails effectively solve the translation task. While generates plausible and visually coherent images, struggles significantly with semantic inversion and frequently misassigns colors (Fig. 15). Specif- ically, ControlNet accurately positions the geometric objects but consistently applies incorrect colors to either the objects the surronning walls. Unlocking the full model (encoder, bottleneck, and decoder instead of just the decoder) resolves this issue but requires much more training time and computations; this reveals an inherent limitation in ControlNet's 's domain translation capabilities. These results suggest that, unlike MDD, ControlNet's 's capabilities are limited to tasks with strong pixel-to-pixel correspondences and may not be suitable for more complex domain translation tasks requiring semantic understanding.
Coudition
[ontsolNet([]
GT
Coadition
ControlNet[N]
GT
Figure 15: Evaluation of translation domain with fixed-configuration using ControlNet. cube nube-ipyramid for N = 100 on BL3NDT.
Translation
B.6.2 Missing Completion on BraTS 2020
For the BraTS 2020 datasct, ControlNet[N] demonstrates strong performance (Fig. 9 when adapted to the semi-supervised setting, achieving high PSNR values but lower SSIM scores compared to MDD (Tab. 1 Since SSIM evaluates the similarity in luminance, contrast, and structural information between images, it provides more clinically relevant asscssment for medical scan evaluation, where preservation anatomical structures is critical for diagnostic purposes.
http://:/hhaalleeeeeee//////eeta lacked
29

---

Published in Transactions on Machine Learning Research (09/2025)
B.6.3 Face Generation on CelebAMask-HO
To complement our multi-domain translation evaluation (Sec. 4.4 assess the (sketch, mask) trans- lation by training one ControLNet[N] for each supervision proportion. For N = 100%, ControLNet[[] gen- crates realistic images as evidenced by low LPIPS and SSIM metrics (Tab. 12 It can be observed that MDD produces better metrics than ControlNei[N] despite ControlNet leveraging powerful pre-trained model which, which show MDD capacity to fully utilize the cross- domain information during training. How- ever, for N 0%, despite being trained in the same semi-supervised setting as [SDM and [M] (with missing samples replaced by noise), ControlNet NN] fails to adapt effectively to the translation task, indicated by significantly higher LPIPS values and poor visual quality (Fig. [ This highlights how fixed- domain translation conditioning mechanisms demonstrate reduced robustness in challenging low-supervision settings (when no fully supervised training data cxists, unlike for the BraTS 2020 setting) without specific adaptations. In contrast, our model produces highly realistic images even under these constrained conditions (Fig. 80, demonstrating superior generalization capabilities.
Table 12: Evaluation of translation domain with a conigcooniinaaotoon using ControlNet. Evaluation metries for (sketch, (nnuuenntaiion--fcce on the CelebA dataset.
CelebAMask-Hi
LPIPS
SSIM
Face
100%
0%
100%
0%
NentrolNet[N]
0.2391
0.42780
0.4330
0.1833
MDD
0.2305
0.3023
0.4385
0.3381
B.7 BraTS 2020 Additional Missing Modalities Completion Results
Additional quantitative and qualitative results are provided for all missing modalities BraTS 2020 for a more detailed analysis of the missing modality completion task. Additional results for T1 generation Tab. 13
and Fig. 16 results for Tlce generation Tab. 14 and Fig. 17 T2 generation Tab. 15 and Fig. 18 and Flair generation Tab. 16 and Fig. 19 are consistent with those presented in the main paper, where MDD performs strongly on metries considered.
found that noisy generation strategies (MSDM* and MSDM [N) are unable perform the task even when a lot of supervision is available (Fig, 20 It is possible that at the beginning of the generation, when the condition is noisy, MSDM* and MSDM [N] are unable to predict a meaningful "mean" due to the binary and localized nature of the segmentation maps. However, this effect may be mitigated for sean modalitics as can predict a correct mcan image that better represents the scan modality distribution.
For segmentation generation Tab. 17 and Fig. 20
Table 13:
Evaluation metrics for T1 modality completion on the BraTS 2020 datasot. To save place, SSIM values are given in e+1 order, MAE values are given e-2 order, and MSE values are given in e-3 order. "sota' refers the original state-of-the-art methods without modifications, while 'ad.' refers our adapted versions of these methods modified to handle semi-supervised scenarios.
BraTS 2020
PSNR
SSIM
MAE
MSEJ
T1
P100%
P80%
P50%
P100%
P80%
P50%
P100%
P80%
P50%
P100%
P80%
P50%
MSDM
21.62
18.58
11.08
8.83
7.35
1.46
3.31
5.19
17.46
6.88
13.86
78.03
UMM-CSGM
21.49
21.61
21.46
8.83
8.86
8.22
3.34
3.28
3.54
7.10
6.91
7.15
[NDM
22.81
17.77
13.28
8.94
6.80
0.64
2.92
5.80
14.76
5.24
16.71
46.99
OU1S ad.
UMM[N
22.47
21.72
22.19
8.80
8.91
8.74
3.06
3.34
3.18
5.67
6.73
6.04
MDD(RAND]
23.44
23.15
21.70
8.84
8.83
8.63
2.72
2.79
3.32
4.53
4.84
6.76
MDD[NOISY]
21.85
21.99
21.61
8.85
8.80
8.78
3.19
3.18
3.30
6.52
6.32
6.91
MDD
23.00
23.37
22.93
9.02
9.03
8.92
2.84
2.73
2.87
5.01
4.60
5.10
30

---

Published in Transactions on Machine Learning Research (09/2025)
Table 14: Evaluation metrics for Tice modality completion on the BraTS 2020 dataset. To save place, SSIM
values are given in e-1 order, MAE values are given in c-2 order, and MSE values are given in o-3 order. "sota' refers the original state-of-the-art methods without modifications, while 'ad.' refers our adapted versions of these methods modified to handle semi-supervised scenarios.
BraTS 2020
PSNR
SSIMT
MAEI
MSEJ
Tice
P100%
P80%
P50%
P100%
P80%
P50%
P100%
P80%
P50%
P100%
P80%
P50%
onTs ad. 3012
MSDM
27.16
25.97
19.07
9.11
8.40
1.35
1.69
2.15
7.14
1.92
2.53
12.40
UMM-CSGM
26.95
26.49
26.56
9.10
8.98
8.92
1.72
1.88
1.90
2.02
2.24
2.21
[SDM[[N]
27.06
24.51
16.94
8.87
6.99
0.60
1.83
2.68
10.10
1.97
3.54
20.22
UMM[N]
26.96
27.14
26.68
8.92
8.88
8.80
1.81
1.78
1.89
2.02
1.93
2.15
MDDDRAND]
26.82
27.14
26.92
8.73
8.59
8.92
1.90
1.85
1.76
2.08
1.93
2.03
MDD(NOISY]
26.98
26.80
26.62
9.09
9.01
9.00
1.74
1.81
1.84
2.01
2.09
2.18
MDD
27.05
27.57
27.28
8.94
9.09
8.92
1.81
1.63
1.73
1.97
1.75
1.87
Table 15: Evaluation metrics for T2 modality completion on the BraTS 2020 datasct. To save place, SSIM values are given in e-l order, MAE values are given in e-2 order, and MSE values are given in e-3 order. 'sota' refers the original state-of-the -fettrrrrrrttt methods without modifications, while 'ad?' refers to our adapted versions of these methods modified to handle semi-supervised scenarios.
TSa 2020
PSNR
SSIM
MAEI
MSEJ
T2
P100%
P80%
P50%
P100%
P80%
P50%
P100%
P80%
P50%
P100%
P80%
P50%
ours ad. 501a
MSDM
24.68
21.48
15.84
8.24
7.75
0.90
2.33
3.49
11.92
3.40
7.10
26.03
UMM-CSGM
24.71
24.24
23.98
8.13
8.74
8.05
2.36
2.28
2.59
3.38
3.77
4.00
MSDM'[N
25.25
20.89
12.32
8.91
7.01
0.29
2.11
3.82
16.26
2.99
8.14
58.60
UMM[N
24.73
24.84
24.04
8.82
8.87
8.61
2.24
2.23
2.46
3.37
3.28
3.95
MDD[RAND]
24.86
24.54
24.29
8.64
8.46
8.61
2.20
2.34
2.35
3.26
3.52
3.73
MDD(NOISY]
24.01
24.12
23.59
8.76
8.70
8.42
2.34
2.34
2.59
3.97
3.87
4.38
MDD
25.38
25.72
24.79
8.75
8.94
8.77
2.12
1.98
2.23
2.90
2.68
3.32
B.8 CelebAMask-HO Additional Translation Results
We provide additional quantitative and qualitative results for (face, mask) translation in Fig. 21 and Tab. 18 (lace, mmask in Fig. 22 and Tab. 19 translation and ()=(fface, sketch, mask) generation
in Fig.
Table 16: Evaluation metrics for Flair modality completion on the BraTS 2020 dataset. To save place, SSIM values are given in e-1 order, MAE values are given in e-2 order, and MSE values are given in e-3 order. 'sota' refers the original state-of-the-an methods without modifications, while 'ad.' refers to our adapted versions of these methods modified to handle semi-supervised scenarios.
BraTS 2020
PSNR
SSIM
MAE
MSEJ
Flair
P100%
P80%
P50%
P100%
P80%
P50%
P100%
P80%
P50%
P100%
P80%
P50%
2
MSDM
22.80
18.83
17.76
8.72
7.85
2.29
2.85
5.01
7.24
5.25
13.10
16.74
UMM-CSGM
22.81
22.60
22.06
8.71
8.60
8.33
2.85
3.00
3.32
5.24
5.49
6.22
[SDM
23.49
21.65
12.37
8.39
7.53
0.42
2.79
3.53
16.28
4.48
6.84
57.99
32
UMM[N
23.11
23.38
22.73
8.45
8.40
8.27
2.89
2.85
3.06
4.89
4.59
5.34
RANDR
24.38
23.91
23.09
8.43
8.44
8.38
2.48
2.60
2.87
3.65
4.06
4.91
purs
MDD
22.83
22.86
22.16
8.66
8.66
8.56
2.90
2.90
3.16
5.21
5.18
6.07
MDD
24.17
24.10
23.66
8.76
8.79
8.68
2.45
2.45
2.61
3.83
3.89
4.30
31

---

Published in Transactions on Machine Learning Research (09/2025)
Table 17: Evaluation metrics for Mask modality completion on the BraTS 2020 dataset. 'sota' refers to the original state-of-the-art methods without modifications, while 'ad?' refers to our adapted versions of these methods modified to handle semi-supervised scenarios.
BraTS 2020
Jaccard
Segmentation
P100%
P80%
P50%
ad. sota
MSDM
0.056
0.049
0.000
UMM-CSGM
0.755
0.072
0.002
MSDM'NN]
0.055
0.066
0.071
UMMIN
0.727
0.770
0.713
MDD(RAND
0.752
0.734
0.695
ours
[DDINOISY]
0.061
0.057
0.059
MDD
0.788
0.788
0.763
GT
P50%
P80%
P100%
MSDM*
UMM-CSGM een
MSDM* [N]
UMM[N]
MDD
Examples of generated T1 scan given all remaining modalities, and the corresponding GT for different levels of supervision.
Figure 16:
The diversity of face generation evaluated different settings: (sketch, mask) generation Fig. 24 and (face sketch) Fig. 25 where we calculate the Diversity Score over the generated faces Tab. 20
The translation results for sketch and mask generation consistent with those presented in the main paper, and on the BraTS 2020 dataset, MDD models perform strongly. For the semantic segmentation task, MSDM* N] performs better than on the BraTS 2020 dataset while still achieving the lowest Jaccard. This can be explained by the fact that a correct 'mean' prediction is easier for the CelebAMask-FF dataset, since the segmentations for different images are similar (the faces are centered, often have the same angles etc.).
We found that the noisy condition model, MSDM [N), has the best diversity score for the generated faces but the worst generation quality. Using noisy condition allows the model to avoid using it the beginning
32

---

Published in Transactions on Machine Learning Research (09/2025)
GT
P50%
P80%
P100%
MSDM*
UMM-CSGM
MSDM [N]
UMM[N]
MDD
Figure 17: Examples of generated Tice scan given all remaining modalities, and the corresponding GT for different levels supervision.
of the generation process and use noisy condition later in the diffusion process. This allows the generated faces to drift further away from the condition, producing more diverse images. In the of MSDM* while it has the higher diversity, it also has the worst generation quality according to Tab. 3
Table 18:
Evaluation metrics for (Face, -Skecc the CelebA dataset. 'ad.' refers to adapted versions of state-of-the-art methods modified to handle semi-supervised scenarios.
CelebAMask-HO
PSNR1
SSIM
Skeich
100%
0%
100%
0%
MSDM
17.298
16.652
0.3875
0.3739
UMM
18.576
16.872
0.4371
0.3897
MDDRRAND
17.526
17.071
0.3903
0.3715
MDD
20.402
18.259
0.5055
0.4219
33

---

Published in Transactions on Machine Learning Research (09/2025)
GT
P50%
P80%
P100%
MSDM*
UMM-CSGM
MSDM [N]
UMM[N
MDD
Examples of generated T2 scan given all remaining modalities, and the corresponding GT for different levels of supervision.
Figure 18:
Table 19:
Evaluation metrics for (Face, Sketch)- on the CclebAMask-HO dataset. 'ad.' refers to our adapted versions of stale-of-the-arl methods modified to handle semi-supervised scenarios.
CelebAMask HQ
Jaccard
Mask
100%
0%
MSDM
0.4476
0.4731
onu ad.
UMM
0.7378
0.7131
MDD
0.6978
0.6704
MDD
0.7536
0.7133
Table 20: Diversity of Face generation metrics for (Sketch, Mask)-+Face generation and Mask->(aace, Sketch) on the CelebAMask-HI dataset. 'ad.' refers to our adapted versions of state-of-the-arl methods modified to handle semi semi-supervee supervised scenarios.
CelcbAMaakk-HO
(S,M)-(0
DS1
(M)-(S,F) DS1
Face Diversity
100%
0%
100%
0%
MSDM'N [
19.883
18.625
19.896
18.726
U
19.668
18.213
20.305
18.564
ours
MDDRRAND.
16.518
16.028
17.502
16.614
MDD
19.524
15.842
19.177
15.786
34

---

Published in Transactions on Machine Learning Research (09/2025)
GT
P50%
P80%
P100%
MSDM*
UMM-CSGM
MSDM [N]
UMM[N]
MDD
Figure 19: Examples of generated Flair scan given all remaining modalities, and the corresponding GT for different levels of supervision.
35

---

Published in Transactions on Machine Learning Research (09/2025)
P50%
P80%
P100%
MSDM*
UMM-CSGM
MSDM*[N]
UMM[N]
MDD
Figure 20: Generated segmentation given all remaining modalities. Orange represents true positives, red false negatives, and blue false positives.
36

---

Published in Transactions on Machine Learning Research (09/2025)
GT
Conditions
INSDMTIN
UMM
MDD(RAND]
MDD
FO
es
Figure 21: Examples of generated sketches given a face and a mask, and the corresponding GT for N=0%.
37

---

Published in Transactions on Machine Learning Research (09/2025)
GT
Conditions
MSDM
UMM[N]
MDD(RAND)
MDD
Figure 22: Examples of generated masks given a face and a sketch, and the corresponding GT for N-0%.
38

---

Published in Transactions on Machine Learning Research (09/2025)
Figure 23: Examples of unconditional generation ()~ (Facc,,Skttch,Makk for MDD N=100%
39

---

Published in Transactions on Machine Learning Research (09/202))
GT
Conditions
Figure 24: Examples of diversity face generation (sketch, Baacch.mssk-aaae +face for MDD N=100%
40

---

Published in Transactions on Machine Learning Research (09/2025)
GT
Condition
Figure 25: Examples of diversity face and sketch generation Mask Mask->(Face, Sketch) for MDD N=100%
41


Reviews:
[{"review_id": "CZAWPbmOAv", "reviewer": "Reviewer_BEHa", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Positive points:\n+ The method is based on a simple idea (masking)\n+ The experimental section is comprehensive\n+ The proposed strategy increases performance over baselines/SOTA\n+ Limitation section is good\n\nNegative points:\n- No theoretical justification or analysis of the method\n- Some unclear notation (minor)", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes a new strategy for multi-source domain translation, a problem where one wants to translate inputs (i.e., map) between different domains. The authors do so through diffusin models, semi-supervision and masking, i.e., they produce multiple views of a ground-truth data point through masking, and the diffusion model must reconstruct the masked input. The authors demonstrate, on various image benchmarks, the superiority of their method with respect other diffusion-based domain translation models.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "While I think this paper would benefit from a deeper theoretical analysis, in its current format it is ready for publication at TMLR, especially since:\n\n1. The empirical validation confirms the paper claims,\n2. The paper fits the scope of the journal.\n\nI have only one minor requested change. In section 3.3, the authors mention that they apply a mask $x_{supervised} = x \\odot m$ and $x_{unavailable} = x \\odot (1-m)$. Here, I am supposing $C = A \\odot B \\iff C_{ij} = A_{ij}B_{ij}$, but I would like the authors to: 1) confirm this, 2) include, in the main text, a definition for this operation.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "78zPDUguPK", "reviewer": "Reviewer_Uodi", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths：\n\n1.  The whole idea is pretty novel. The paper proposes a creative formulation that treats each domain with its own noise level, effectively modeling missing data and enabling flexible generation.\n\n2. The method is flexible. MDD supports both full and semi-supervised settings, arbitrary conditional configurations, and can operate with varying numbers of available modalities at inference.\n\n3. Extensive experiments demonstrate its effectiveness.\n\nWeakness:\n\n1. As acknowledged by the authors, the method's scalability with a large number of domains may be limited due to potential bottlenecks in multi-modal fusion within a single latent vector. This needs more further explanation on solutions.\n\n2. While a U-Net backbone is used, more comparative discussion on architectural design choices (e.g., handling of skip connections, time embedding for multi-domain setting) would help clarify contributions over baseline architectures.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes Multi-Domain Diffusion (MDD), a novel framework for semi-supervised multi-domain translation based on diffusion models. The method introduces:\n\n1. A multi-noise-level formulation where each domain has an independent noise level during the diffusion process.\n\n2. An ability to handle missing views naturally during training and inference by treating them as high-noise representations.\n\n3. A flexible approach to semi-supervised learning across arbitrary domain configurations without requiring fixed input-output paths.\n\n4. Extensive experiments on three datasets (BL3NDT, CelebAMask-HQ, and BraTS 2020) showing superior performance over adapted and state-of-the-art baselines.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. Clarify fusion strategy scalability in Sec. 5: Provide a brief discussion or suggestion on how future work could address architectural scaling with >5 domains.This is important because MDD's applicability to large-scale multi-modal tasks (e.g., video, multi-sensor fusion) depends on this.\n\n2. Better motivate the clean vs. noisy condition choice in Sec. 3.2 and App. B.1: While clean condition performs better, provide more theoretical or intuitive reasoning—beyond empirical evidence—about why noisy conditioning degrades generation significantly. Clarify whether clean vs. noisy conditioning generalises across tasks with different generative ambiguities (e.g., deterministic vs. stochastic mappings).", "broader_impact_concerns": "NA", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "AAqITodQFB", "reviewer": "Reviewer_uQXo", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "### Strengths\n\n1. This paper introduces a novel method that assigns independent noise levels to each domain within a noise vector, with missing views set to $t^{(i)} = T$ (white noise). The proposed method elegantly handles missing data in semi-supervised settings by treating absent views as fully noised inputs, outperforming methods like MSDM that use uniform noise levels.\n\n2. By learning the joint data distribution across all domains, MDD supports translations between any domain subset without requiring separate models, a significant improvement over methods like Pix2Pix or CycleGAN. This scalability is evident in its ability to handle $2^L$ mappings with only one model.\n\n3. The supervision mask and noise level masking allow MDD to naturally adapt to semi-supervised settings. For missing views, by setting $t^{(i)} = T$ enables the model to reconstruct absent domains using available ones, with promising performance in low-supervision settings.\n\n4. The paper evalutes MDD across diverse supervision levels, datasets, and baselines. Additional comparisons with ControlNet and noise strategy explorations $\\phi(t)$ in the appendix strengthen the analysis.\n\n### Weaknesses\n\n1. The proposed MDD modified U-Net with duplicated encoders/decoders and an aggregation network, which increases computational cost compared to other methods. Training for 2000 epochs on CelebAMask-HQ and requiring large batch sizes (e.g., 256 for BL3NDT) may limit scalability, a common issue for diffusion models but might be amplified by multi-domain settings. \n\n2. For generative tasks (e.g., face generation from sketches), multiple valid outputs are possible, but MDD’s default clean condition ($\\phi(t) = 0$) may limit diversity. The appendix explores alternative $\\phi(t)$ strategies, but the main results lack an adaptive method, potentially causing conflicts in tasks requiring balanced fidelity and diversity\n\n3. The uniform sampling of noise levels for available domains and fixed at $\\phi(t) = 0$ for generation may not be optimal for all tasks. The BL3NDT bridge setting shows sensitivity to noise in intermediate domains, and the lack of analysis on noise schedule robustness could affect reconstruction stability, especially for missing views.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper introduces an interesting diffusion-based method (MDD) for multi-domain translation that enables mappings between arbitrary combinations of L domains (i.e., $2^{L}$ possible functions) using a single model. Unlike existing methods that train separate models for specific source-to-target translations, MDD can handle two parts: (1) reconstructing missing views (i.e., absent data for specific domains) for new data points, and (2) supporting semi-supervised learning with arbitrary supervision configurations. Missing views, such as an unavailable segmentation mask in a dataset with face images, sketches, and masks, are treated as maximally noisy inputs (i.e., t=T). The authors conduct a wide range of experiments cover fully supervised, semi-supervised, and bridge settings (where certain domain pairs are never seen together), comparing MDD against state-of-the-art methods, and include a detailed ablation study.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. Add a brief example in Section 3.2 to clarify noise level masking for missing views, e.g., for CelebAMask-HQ with a missing segmentation mask. For example, consider a data point with domains $D_1$, $D_2$, and $D_3$, where $D_3$ is missing.\n(1) Show the supervision mask $m = [1, 1, 0]$, sampled noise levels $\\mathcal{T}_{\\text{sup}} = [200, 500, 300]$, and how $\\mathcal{T} = [200, 500, T]$ is computed with $T = 1000$; (2) explain how the input for $D_3$ is replaced with Gaussian noise and used in training (Algorithm 1, line 5).\n\n2. The evaluation shows that MDD struggles in the BL3NDT bridge setting (MAE of 1.199 v.s. 0.316 at 100% supervision), where missing views are reconstructed via intermediate domains. This suggests potential reconstruction conflicts due to error propagation or complex inter-domain dependencies. Add a paragraph in the discussion section to address how MDD could mitigate such conflicts.\n\n3. MDD uses a fixed clean condition ($\\phi(t) = 0$) during generation, which ensures fidelity but may limit diversity in generative tasks.  Please consider adding a discussion in Section 3.3 on the choice of $\\phi(t)$ and propose an adaptive strategy based on task type.\nFor example, for discriminative tasks (e.g., segmentation), use $\\phi(t) = 0$ for precision; while for generative tasks, use a noisy condition (e.g., $\\phi_\\gamma(t)$ with $\\gamma = 0.7$ to increase diversity. Also add a table or figure in the results comparing $\\phi(t) = 0$ and an adaptive $\\phi_\\gamma(t)$ on CelebAMask-HQ, reporting LPIPS and Diversity Score.\n\n4. MDD’s performance relies on uniform sampling of noise levels for available domains and a linear noise schedule. The BL3NDT bridge setting suggests sensitivity to noise in intermediate domains, which raises concerns about reconstruction stability for missing views. Sp, please add a paragraph or subsection in the appendix analyzing noise schedule sensitivity. For example, evaluate MDD with alternative schedules (e.g., cosine schedule, as used in some diffusion models), report the impact on BL3NDT bridge setting MAE or BraTS 2020 PSNR at P50% supervision, and discuss implications for optimizing $\\mathcal{T}_{\\text{sup}}$ sampling (e.g., prioritizing lower noise levels for more informed domains).\n\n5. Please include a computational analysis and compare the proposed MDD with the existing methods.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)