Diffusion Adversarial Representation Learning for Self-supervised Vessel Segmentation
Authors: Boah Kim
Yujin Oh
Jong Chul Ye
Venue: ICLR 2023 poster (2023)
Keywords: Diffusion model
Adversarial learning
Self-supervised learning
Vessel segmentation

Abstract: Vessel segmentation in medical images is one of the important tasks in the diagnosis of vascular diseases and therapy planning. Although learning-based segmentation approaches have been extensively studied, a large amount of ground-truth labels are required in supervised methods and confusing background structures make neural networks hard to segment vessels in an unsupervised manner. To address this, here we introduce a novel diffusion adversarial representation learning (DARL) model that leverages a denoising diffusion probabilistic model with adversarial learning, and apply it to vessel segmentation. In particular, for self-supervised vessel segmentation, DARL learns the background signal using a diffusion module, which lets a generation module effectively provide vessel representations. Also, by adversarial learning based on the proposed switchable spatially-adaptive denormalization, our model estimates synthetic fake vessel images as well as vessel segmentation masks, which further makes the model capture vessel-relevant semantic information. Once the proposed model is trained, the model generates segmentation masks in a single step and can be applied to general vascular structure segmentation of coronary angiography and retinal images. Experimental results on various datasets show that our method significantly outperforms existing unsupervised and self-supervised vessel segmentation methods.

Published as a conference paper at ICLR 2023
DIFFUSION ADVERSARIAL REPRESENTATION LEARN- ING FOR SELF-SUPERVISED VESSEL SEGMENTATION
Boah Kim", Yujin Oh', Jong Chul Ye Korea Advanced Institute of Science and Technology (KAIST), Daejeon, Republic of Korea (boahkim, yuji oh jong. ye
vejek Jēka st ac.kr
ABSTRACT
Vessel segmentation in medical images is one of the important tasks in the di- agnosis of vascular diseases and therapy planning. Although learning-based seg- mentation approaches have been extensively studied, a large amount of ground- truth labels required in supervised methods and confusing background struc- tures make neural networks hard to segment vessels in an unsupervised manner. To address this, here we introduce novel diffusion adversarial representation learning (DARL) model that leverages a denoising diffusion probabilistic model with adversarial learning, and apply it vessel segmentation. particular, for self self-supervised vessel segmentation, DARL learns the background signal using a diffusion module, which lets a generation module effectively provide vessel representations. Also, by adversarial learning based on the proposed switchable spatially-adaptive denormalization, our model estimates synthetic fake vessel im- ages as well vessel segmentation masks, which further makes the model cap- ture vessel-relevant semantic information. Once the proposed model is trained, the model generates segmentation masks in single step and can be applied general vascular structure segmentation of coronary angiography and retinal im- ages. Experimental results various datasets show that method significantly outperforms existing unsupervised and self-supervised vessel segmentation meth- ods.
INTRODUCTION
In the clinical diagnosis of vascular diseases, vessel segmentation is necessary to analyze the vessel structures and therapy planning. In particular, when diagnosing coronary artery disease, X-ray an- giography is taken to enhance vessel visualization by injecting a contrast agent into the blood vessels (Cong al. 2015). However, it challenging to extract vessels accurately due to low contrast, mo- tion artifacts, many tiny branches, structural interference in the backgrounds, etc (Xia et al. 2019
Chen et al. 2014).
To segment vascular structures, various segmentation methods have been explored. Traditional op- timization models (Law Chung 2009 Taghizadeh Dehkordi et al.) (014) typically require com- plicated preprocessing steps and manual tuning. Furthermore, they are computationally expensive to process many images. the other hand, learning- methods (Nasr-Esfahani et al. 2016 Fan al. 2018 Chen et al. 2019) generate segmentation maps in real-time once the models trained. However, supervised methods require a huge amount labeled data for training. which complicates their use in practical applications. Also, existing unsupervised methods designed on natural images are difficult to apply to medical vessel images due to low contrast subtle branches and confusing background structures. Although recent self-supervised method (Ma al.) 2021) is presented to learn vessel representations, this requires two different adversarial networks to segment vessels, which leads increasing training complexity.
Recently, diffusion models such as denoising diffusion probabilistic model (DDPM) (Ho al. 2020) has become one of the main research topies in modeling data distribution and sampling diverse images. By learning the Markov transformation of the reverse diffusion process from Gaussian
cc-first authors

---

Published as a conference paper at ICLR 2023
2t2
D3
real fake?
Generation module, G
(A)
Diffuslon module
s
(B)
x
ResnetBlock
XN
Da real real fake?
Switchable SPADE
g
Figure 1: Our proposed diffusion adversarial representation model for self-supervised vessel seg- mentation. In path (A), given a real noisy angiography image ot our model estimates vessel segmentation masks 8". In path (B), given real noisy background image æ4, and vessel-like fractal mask 8 our model generates synthetic angiography image a.
noise data, DDPM is successfully applied to many low-level computer vision tasks such super- resolution (Chung et al. 2022), inpainting (Lugmayr et al.) 2022). and colorization (Song et al, 2020). For high-level vision tasks, while recent study (Baranchuk al. (022)) shows that DDPM can capture semantic information and be used as image representations, methods applying DDPM in learning semantic segmentation without labeled data have so far not been developed. Also, the sampling process of the diffusion models often takes relatively long time.
In this paper, we introduce a novel concept of diffusion adversarial representation learning (DARL), which is non-iterative version of the diffusion-bast generative model and can be successfully applied to self-supervised vessel segmentation without ground-truit truth labels. As illustrated in Fig- [ our model is composed of a diffusion module and a generation module, which learns semantic information of vessels via adversarial learning. Specifically, based on the observation that the diffu- sion model estimates the noise added to the perturbed input data, and the adversarial learning model generates images for given the noisy vectors, we can naturally connect the diffusion model with the adversarial model. This allows our model not only generate images in real time but also to segment vessels with robustness to noises and various modalities. Here, inspired by the spatially- adaptive denormalization (SPADE) layer (Park et al. 2019) that effective in image synthesis given semantic masks, we present a switchable version of SPADE in the generation module to estimate vessel segmentation maps and mask-based fake angiograms simultancously. This can yield syn- ergy effect in learning vessel representation by extracting proper features for angiogram synthesis.
More specifically, shown in Figure T for given unpaired background images and angiography im- ages that are taken before and after injection of the contrast agent, there are two paths for feeding the inputs into our proposed model: (A) when the real angiography images are given, our model with- out the SPADE estimates vessel segmentation maps; (B) when the background images are given, our model with the SPADE generates synthetic angiograms that composite vessel-like semantic masks with the input backgrounds. Also, as each vessel-like semantic mask in the (B) path be regarded as the pseudo-label for the generated angiography image, by feeding the synthetic angiograms into the (A) path again, we apply the cycle consistency between the segmentation maps and the labels of fractal masks to capture semantic information vessels. In addition, by designing the diffusion module to intensively learn the background signal, we let the module consider vessel structures of angiography images as outlier when estimating the latent feature. Thereby, vessel structures repre- sented in the output of the diffusion module guide the generation module effectively segment the vessels.
We build our model on X-ray coronary angiography using XCAD dataset (Ma al. 2021 and apply to several different blood vessel datasets, including retinal images. Experimental results show that our method outperforms several baseline methods by large margins for vessel segmentation tasks in the absence of labeled data. The main contributions are summarized as:
We propose a diffusion adversarial representation model, a non-iterative version of dif- fusion model for image generation, and apply it for self-supervised vessel segmentation.
2

---

Published as a conference paper at ICLR 2023
Specifically, the latent features of our diffusion module provide vessel information and thus improve the segmentation performance.
2. Through the proposed generation module with switchable SPADE layers, our model not only generates synthetic angiography images but also segments vessel structures.
3. Experimental results verify that model achieves superior segmentation performance by learning vessel representations. In particular, although the model is trained using X-ray coronary angiograms, it provides the state-of-the-art performance for un-/self-supervisee retinal vessel segmentation as well, confirming the generalization capability of the model.
BACKGROUNDS AND RELATED WORKS
Denoising diffusion probabilistic model Diffusion model (Sohl-Dickstein et al., 2015 Ho al. 2020 Song & Ermon 2019) is one of generative models that sample realistic data by learning the distribution of real images. particular, the denoising diffusion probabilistic model (DDPM) (Ho et al. 2020) with a score matching has been shown superior performance in image gener- ation. Specifically, DDPM learns the Markov chain to convert the Gaussian noise distribution ~ (O. I) into the target distribution To- In the forward diffusion process, the noise is gradually added the noise to the data by:
(( = Nax: -1,,,,1,1,,, 3,1),
(I)
where 8, E [0, 1] is a fixed variance. Accordingly, noisy target æ, distribution from the data Do is represented as:
((x (x0) = N (ax; 30 - I))
(2)
where a 111 (1 - 3.). Then, DDPM is trained to approximate reverse diffusion process:
po ar,) N (a: no (are t), o
where or is fixed variance, and p4g is a parameterized mean with the noise predictor eg:
1
Br
(,( VI- 3.
())
VT- at
Thus, in the generative process, the sample can be obtained from the Gaussian noise by the iterative denoising steps: - = p0 t) + diz, where 2 ~ NNO,,) I).
Through this stochastic process, DDPM provides diverse realistic samples and has been exploited in many applications, including super-resolution (Chung et al. 2022 Saharia et al. (02T), inpainting (Lugmayr et al. 2022), and colorization (Song et al., 2020) /ahaaiia et al. (022). However, the application study of semantic segmentation is limited. Although several works (Baranchuk et al.) 2021 Amit et al. 202I) are recently presented to solve high-level vision problems, they require annotated data to train the models.
Self-supervieed vessel segmentation For the vessel segmentation task, is difficult to obtain fine-grained labels for supervised learning, since the vessel has complex structures with numerous tiny branches. While this label scarcity issue can be alleviated by semi- or unsupervised learning, fully unsupervised methods to segment the tiny vessels with reasonable performance relatively scarce. In fact, recent unsupervised learning methods trained with natural images have great gen- eralization capability unseen datasets (Ahn al. 2021 Chen al. 2019 Melas-Kyriazi al.) 2022), thus they can be easily adapted to medical image segmentation tasks. However, due to the unique characteristics of angiography, e.g. confusing background factors and sophisticated vessel structures, any unsupervised methods designed for natural image segmentation may get degraded performance when they are applied to vessel segmentation of noisy angiography images. As type of unsupervised learning, self self-supervised learning also has been introduced to utilize self-generated supervisory labels from data themselves to efficiently learn target representations in various medical image segmentation tasks and has demonstrated its potential (Mahmood et al. 2019 Ma et al.) 2021 Oh Ye 2021). Specifically, Ma al. (202 introduces an end-to-end adversarial learning frame- work for vessel segmentation with the CycleGAN (Zhu et al. 2017) structure, which learns realistic angiogram generation that adds fractal- -giied pseudo labels to the background images. However, the simple arithmetic operation for synthetic vessel generation often fails yield realistic pseudo- vessel images, thus training the adversarial networks using unrealistic synthetic images is difficult to produce optimal segmentation performance.
3

---

Published as a conference paper at ICLR 2023
(A)
(B)
Caiff
s
Ludo
SPADE
Diffusion module, Eg
Generation
3
x
Diffusion module, Eg
Generation module, G
module,
Cycle path
add
Diffusion module, Eg
Generation
Lyy
module,
Training flow of our model. The generation module G' with the switchable SPADE layers takes Eg and the noisy images, and generates desired outputs corresponding to the paths. oi, and <9, denote the noisy angiography and background images, where ta and fs are noise schedules. s" is the generated vessel segmentation, and 2" is the synthetic angiography images. sf is the vessel-like fractal masks. C'tt denotes the concatenation of images channel dimension. Laiff, Lados and Leye the diffusion loss, the adversarial loss, and the cycle loss, respectively.
Figure 2:
3 DIFFUSION ADVERSARIAL REPRESENTATION LEARNING
In this section, we describe our novel diffusion adversarial representation learning (DARL) model, tailored for self-supervised vessel segmentation. We call the images before and after injecting the contrast agents into the blood vessels as background and angiography, respectively. Note that due to the different scanning times, these two images have different contrasts and are not aligned, caused by the movements of patients. Thus, as shown in Figure our DARL model is trained on unpaired angiography images zij and background images x²)
Specifically, our model is comprised of a diffusion module Eg to estimate latent features, a genera- tion module G' to estimate both the vessel segmentation masks 8" and the synthetic angiograms d" and two discriminators (D,. Da) to distinguish real and fake images of the vessel masks and the an- giograms, respectively. Here, in generating angiography images,
provide the vessel-like fractal masks s/ presented by Ma et al. (2021) to the generation module to perform image synthesis based on semantic layouts. Moreover, to estimate the segmentation maps and angiography images effec- tively. we design the generation module with novel switchable SPADE layers, where the SPADE Park et al. (2019) facilitates the semantic image synthesis.
Generation module with switchable SPADE layers As illustrated in Figure I the proposed generation module consists of N residual blocks (ResnetBlock) that have switchable SPADE (S- SPADE) layers. Note that the (A) and (B) paths of our model are implemented simultaneously by sharing the learnable parameters except for the S-SPADE layers. Then, let E MMxxxHxx be the feature map in the ResnetBlock, where M, C, H, and W are the size of batch, channel, height, and width, respectively. The switchable SPADE layer normalizes feature maps differently depending on the existence of an input mask
[SPDEE(,,)) s), if mask s is given, (I(()),
otherwise,
where IN is the instance normalization (Ulanov et al. 2017). So, when model is given the fake vessel mask , the SPADE is computed by:
Tm.e.h.w = (s. Fm.c.h.w Me + (s/),
(6)
where km denotes the (m, c, h w)-th element of the feature tensor U, (Pey T²) are the mean and standard deviation of the feature map in channel e, and (Te,h,ar, Be,n,w) are learned modulation parameters during training.
Thus, in the (A) path, given the noisy angiogram of and the latent feature Eg (x t), the generation module G estimates the vessel segmentation masks &" without the SPADE:
G(Ee( ();
(7)
On the other hand, in the (B) path that provides the fractal mask s/, the generation module taking the noisy background æt and its latent feature G(Eo( (ar;; 1)) synthesizes the fake angiograms ea: G(Eo(ar) 8
_
(8)
4

---

Published as a conference paper at ICLR 2023
3.1 NETWORK TRAINING
In contrast to the DDPM that pretrains the diffusion model, our method trains the diffusion mod- ule, generation module, and discriminators simultaneously using an adversarial learning framework. the detailed training flow of our model. There are two distinct paths: (A) one feeds the real angiograms zij into the model provide vessel masks &", and (B) the other takes the real backgrounds z and the fractal masks sf for the model to generate fake angiograms z". Here, as shown in Figure B),, since the input fractal masks can be regarded as vessel segmentation labels of the fake angiograms, we forward the fake angiograms generated through the (B) path to the (A) path, and apply cycle consistency between the estimated segmentation masks and the fractal masks to capture the vessel information.
3.1.1 LOSS FUNCTION
To train the model, we employ LSGAN (Mao et al. (017) framework, which leads to the alternating application of the following two optimization problems:
min L (EO, D,, Da): 0,G
min Ge,,
(9)
D,D.
where L and _ denotes the losses for the diffusion/generrttor and discriminator, respectively, which are given by:
C ((Ea,,,Dee D.D.. = Lass(EE) + aCC (Ea. G. D., Da) + (EeyolEo, G), PP(Eo,G,D. D.) = C (EO, G, + c (EO, Da),
(10)
(11)
where and B are hyperparameters, Laify is the diffusion loss, Lade is adversarial loss, and Leye is cyclic reconstruction loss. The detailed description of each loss function is as follows.
Diffusion loss Recall that the diffusion module learns the distribution of images to estimate mean- ingful latent features of the inputs. We follow the standard loss for DDPM training (Ho et al. 2020
Lass (Ee) == Er, ||ee ea(V + vI t)12
(12)
where E ~ NNO,,) I). In particular, to let the diffusion module represent the vessels of angiograms effectively, define the diffusion loss the background images, i.c. 2o x in the (B) path and set the sampling schedule in E [o, 7]. Accordingly, the diffusion module is trained intensively to learn the background image signal, allowing the module in the (A) path regard the vessel structures of the angiograms as outlier and represent vessels in the latent features.
Ad Adversarial loss To generate both vessel segmentation masks and synthetic angiograms without the ground-truth labels, the proposed model is trained by adversarial learning using the two discrim- inators D, and D.. As shown in Figure [ the discriminator D, attempts distinguish the estimated segmentation masks &" from the real fractal mask sf (in the (A) path), while the discriminator D. tries to discriminate between the generated angiograms 2a and the real aniography images z6 (in the (B) path). As we employ LSGAN (Mao et al. 2017). the adversarial loss of generator L can be formulated by:
CG (Eo.
(D.(C(ee(()))) 1) + Ex ( ((D((G(e((z"): 1)
(13)
On the other hand, the discriminators are trained compete against the generator with the adversar- ial loss functions, CDd, and LD,, which are defined by:
1
CD (co, D,) Fer [D2
1)3]
2 E (DD((((((==":)0)))
(14)
2
CD (EO, G, Da) = - Enng ( (D) 1)
1
1 Eer ar (Da
)))
(15)
This adversarial loss enables the single generator G to fool the discriminator D, and Das by gen- erating realistic segmentation masks &" = (G(co(2=)) 0) and angiograms aa G(cg (x*): sf). In contrast, the discriminators attempt to distinguish these generated images being fake and the real images of of and ag being real.
gen-
5

---

Published as a conference paper at ICLR 2023
f 180
c. 120
f 60
x cu
Figure 3: Vessel segmentation according to the noise level fa. Our model estimates the segmentation masks 8 using the latent features cg for the noisy angiograms x s" is the ground-truth label.
Cyclic reconstruction loss For the generator G to capture the semantic information of the vessels, we also constrain our model with the cyclic reconstruction loss on the fractal masks. Specifically, as the vessel-like fractal masks sf can be labels for the synthetic angiograms áa generated in the (B) path, we feed the x= into our model and reconstruct the fractal masks by the (A) path. Therefore, the cyclic reconstruction loss computed between the reconstructed segmentation masks and the real fractal masks, which can be written by:
Leye G) = Ex a [[G((e(G(e)((")) s)); s [|1]-
(16)
Here, we solve the segmentation problem as vessel mask image generation, which is why we use L.1 loss in the cyclic loss.
3.1.2 IMAGE PERTURBATION FOR THE MODEL INPUT
Given real images of æg and a..- our diffusion module takes noisy angiograms =;_ in the (A) path and noisy background images æ in the (B) path as the input, in which each noisy image is sampled based on the forward diffusion process (2:
F2 20 vI 046,
(17)
where E ~ NN I). and both of ta and to are uniformly sampled time step in [0,7]. Here, for the diffusion module only to learn the background image signal in the (B) path but also to provide useful information for the generation module to segment the vessel structures under even certain noisy angiogram images in the (A) path, sample ta in the range of [0, Tal where Ta < T. Empirically, we found that this makes our model learn vessel representations robust to the noise.
3.2 INFERENCE OF VESSEL SEGMENTATION
The inference phase of DARL different from the conventional diffusion model in that our model do not require iterative reverse process, similar to the recent diffusion- unsupervised learning method called DiffuseMorph (Kim et al. 2021). Specifically, once the proposed DARL trained, in the inference, we can obtain the vessel segmentation masks of angiograms from the (A) path by one step. For the noisy angiograms æt, given by the forward diffusion process (7). our model provides the vessel segmentation masks using the latent features eg (x tta) estimated from the diffusion module. As shown in Figure 3 our model can generate the segmentation masks for any noise level f., within certain range (i.e. [0, Ta)) Nevertheless, since the angiography image æg can considered as one of the clean target images, the closer fa is to zero, the better the vessel segmentation performance. Therefore, we our model by setting t. 0.
EXPERIMENTS
In this section, we thoroughly evaluate the vessel segmentation performance of our method. We firstly compare the proposed DARL to existing unsupervised and sulf-super baseline models on various angiography datasets, including X-ray coronary angiography and retinal images. Also, we study the noise robustness of our model. Then, we analyze the success our model in vessel representation and conduct ablation study.

---

Published as a conference paper at ICLR 2023
Datasets To realize the self- self-supervised learning framework, train model with the pub- licly available unlabeled X-ray coronary angiography disease (XCAD) dataset obtained during stent placement surgery and generated synthetic fractal masks (Ma et al. 202T). When training the net- work, each angiography and background data is independently sampled. Also, in testing, we utilize external 134 XCA (Cervantes-Sanches et al.] (0199) and 30 XCA (Hao al. 2020) datasets. Further-
more, we evaluate cross-organ generalization capability on retinal imaging datasets; DRIVE (Staal al. 2004 and STARE (Hoover Goldbaum 2003). Details of the datasets are in Appendix B
Our model is implemented by employing the network architectures pro- posed in DDPM (Ho et al. 2020) and SPADE (Park et al. 2219)) for the diffusion module and the generation module, respectively. Also, for the discriminators, we use the network of PatchGAN (Isola et al.; 2017). To train the model, we set the number of time steps T 2000 with the lin- early scheduled noise levels from 10-6 to 10-2. Within this range, we sample the noisy angiograms by setting T. to 200. Also, we the hyperparameters of loss function as Our model is optimized by using the Adam algorithm (Kingma & Ba 2014) with a learning rate of 5 x 10 6 on a single GPU card of Nvidia Quadro RTX 6000. We train the model for 150 epochs. and the model in the epoch with the best performance on the validation set is used for test data. All the implementations are done using the library of Py PyTorch (Paszke et al. 2019) in Python. The details of network structures and hyperparameter setting can be found in Appendix.
Implementation details
0.2 and 3 = 5.
Baseline methods and metrics We compare model to several baseline methods of un-/self- supervised learning, which do not require ground-truth vessel labels. For unsupervised learning methods, utilize Spatial-Guides Clustering (SGC) (Ahn al.) (021), Redrawing (Chen al., 2019), and Deep Spectral (DS) Melas-Kyriazi et al. 2022). For self-supervised learning methods, employ Self-supervised Transformer with Energy-based Graph Optimization (STEGO) (Hamil- ton et al. 2022). Deep Adversarial (DA) (Mahmood et al. 2019), and supe-Supeeeeeed Vessel Seg- mentation (SSVS) (Ma et al. 202I). All these methods are implemented under identical training conditions to our model, unless the method needs no training procedure. For baseline methods that
Image
GT
Unsupervised
Self-supevviedd
SGC
Redrawing
DS
STEGO
DA
SSVS
Ours
XCAD
Noise
(a 25)
Noise
(a 50)
XCA
X0C
DRIVE
STARE
Figure 4: Visual comparison results the vessel segmentation of various angiography images.

---

Published as a conference paper at ICLR 2023
Table 1: Quantitative evaluation results on the vessel segmentation of various angiography images.
Unsupervised
Self-supervised
Data
Metric
SGC
Redrawing
DS
STEGO
DA
SSVS
Ours
IoU
0.060
0.059
0.366
0.410 0.375 0.146.0.000
0..78
XCAD
Dice
0.111 =
0.109
0.526
0.249
0.542=0.078
0.575 00.1
0. 0.636
Precision
0.06220.c34
0.139 00.09 20.091
20.107
0.152
0.557=0.114
0.
0.701
External test: Coronary angiography
IoU
0.04520.005
0.056g0.018
0.256go.110
0.134g0.081
0.318 128 0.190
0.426_0.000
134 XCA
Dice
0.085+0.003
0.105
0,228 0.3
0.291
0.468
0. 0.595=0.058
Precision
0.047
0.058
0.280gn.123
0. 1.56
0.506g0.201
0.5922:.125
0.781
IoU
0.083
0.048 #0.022
0.339 ao.0s
a..022
0.298
0.324 146
0.427=0.184
30 XCA
Dice
0. 0.150
0.091 +0.040
40.19
0.31 4000000
0,468 0.447
193
0.572
Precision
0.09020.0a1
0.144
0.526g0.130
0.200
0.612
0.613 212
0.729
Cruss-odaalit test: Retinal imaging
lol!
0.063ga.055
0.057 20.033
0.217
0.245 4.152
0.314
0.372
DRIVE
Dice 0.115g0.0ws
0.105g0.055
0.333
0.257 10.
0.469 0.386
0.525
Precision
0.06920.001
4.199
0.248
0.169
0.508
0.549 216
0.617
IoU
0.055go.oas
0.074g..0as
g.1
0. 0.12519.075
0.311 0.237
0.368
STARE
Dice
00.0r
0.134
+0.20
0.216
0.367
0.4541
185
00.08
Precision
0.05820.044
0,227 20.187
0.205
g..002
0.427
0.490
0.537 2002000.000
require heuristic thresholds, optimal performance is achieved by selecting data-specific thresholds within the range from 0.2 to 0.8 in increments of 0.1. To quantitatively evaluate the segmentation performance, we compute Intersection over Union (IoU), Dice similarity coefficient, and Precision.
4.1 EXPERIMENTAL RESULTS
Figure shows the vessel segmentation masks from the baseline methods and our proposed method on three different coronary angiography datasets and two retinal imaging datasets. Quantitative evaluation results of the methods are presented in Table The analysis of the results as follows.
Comparison of ours baselines When we compare the proposed method to the baselines, our model segments vessel structures including tiny branches more accurately. Also, as shown in Ta- [ our model consistently achieves the SOTA performance by large margin compared to existing unsupervised and self-supervised methods. In specific, our network shows significantly improved precision scores, which demonstrates advantages of our DARL, that effectively differentiates fore- ground vessel structure and eliminates false positive signals from the noisy backgrounds.
Generalization capability To verify that our trained DARL can be generally used for various vessel images taken from different machines or different anatomic region-of-intered evaluate the generalization capability by applying the models only trained on the XCAD dataset to the other datasets. First, for the external 134 XCA and 30 XCA datasets which have different resolutions and noise distributions those of the XCAD dataset, shown in Figure and Table our model achieves higher performance than the others. Also, with the DRIVE and STARE retinal datasets that have unseen data distributions due to the different modalities from the XCAD, our DARL shows the most promising cross-organ generalization performance. This may come from the proposed framework that reuses the generated angiography images for the segmentation process through the cycle path, diversifying the input data distribution. Also, the diffusion module learning the stochastic diffusion process enables our model to be used in general for vessel segmentation.
(ROI), we
Robustness to noises As X-ray images are often acquired under low-dose radiation exposure to reduce potential risks, we further evaluate the performance of our model on simulated noisy an- giograms. Using the XCAD dataset, we add Gaussian noise the angiogram with different levels of a 100 25, and 50. We show the segmentation results according to the noise levels in Figure Also, report the quantitative evaluation results in Table 2 It noteworthy that our DARL the only method to segment vessel structures with reasonable performance under noise corruption. Since the proposed segmentation method is trained through the diffusion module that perturbs the input images, the model is highly robust to segment vessel structure even from the noisy data.
Latent representation To study the origin of the performance improvement, in Figure 5 we show the latent features eØ (0(x),t) given Bo for (A) the angiography To 26 and (B) the
8

---

Published as a conference paper at ICLR 2023
Table 2: Results of noise robustness test according to the Gaussian noise with o.
Unsupervised
Self-supervised
Metric
SGC
Redrawing
DS
STEGO
DA
SSVS
Ours
IoU
20.03
2.002
0.331 10.10
0.144
2.353
0. 0.25810.079
0.45120.000
Dice
0.122200050
0.487 0.096g0.083
0.245 107
0.519
0.404 099
0. 0.617.0.076
Precision
0.069
0.126700077
0.480
0.157
0.481
0.477
0.710go.115
IoU
0.06920.055
0. 0.03020.021
0. 2.2322000
0. 0.11800.08
0.247
0.389 0.050
25
Dice
0.128 2.128
0. 0.009 20.09
0.36610.132
0.206
0.391 2202
0.109
0.554
Precision
0.072200006
0.095
0. 1.4
0. 0.14
0.371
0. 4..002
0. 0.727=0.119
IoU
2.070
2.00020.012
0.077 20.005
0.060 osa
0. 2..02
10.03
0.269 081
Dice
0.130+0.045
0.0
0.136
0. =0.088
0.041 0.180
0.417 101
Precision
0.07220.028
0.061
0.221 20.168
0.076
0.169
0. 0.060
0.716
backgrounds Do = 2t with t = 100, respectively. In contrast to the (B) path, the latent representation in the (A) path emphasizes the vessel structures. This implies that although there are no ground-truth labels, our model learns the background image representation so that the vessel structure can be captured as outlier, leading to improved segmentation performance.
Xo
Ep
Xo
(A)
(B)
Ablation study Table 3 shows the evaluation results of several ablation studies. Implementation details and visual results are in Appendix [.I] (a) Our model without the diffusion module and Laiss shows lower performance by about 2% for all metrics compared to our model, which suggests that the diffusion module guides the generation module to extract vessel representation accurately. (b) The generation module without the proposed S-SPADE layers is degraded by more than 1% over (a) for all metrics, verifying that our SPADE- based unified generator effectively captures vessel semantic information through the synergy of learning both image segmentation and generation. mentation our model without the proposed cyclic loss Layes we verify that Leye allows our model segment proper vessel regions. (d) When training our model by converting the LI loss for Leye to the cross-entropy (CE) loss, the performance is much worse than ours in all metrics, which implies that our approach using L1 loss for the cycle path is proper to obtain the vessel masks.
Figure 5: Estimated latent features eg in the (A) and (B) paths of our model.
(c) Through the imple-
Table 3: Results of ablation study the proposed model and loss function.
Module
Loss function
Metric
Method
Diffusion
Generation
Lass
Lade
Laye
loU
Dice
Precision
Ours
0.471 =o 078
0.636 072
0.701 =0.
(a)
0.449:20.07
0.6162o.07z
0.646
(b)
w/o S SPADE
0.439 10.090
0.606 aso
0.6201o.111
(c)
0.322220.55
0. 0.485 061
0.580
(d)
LI-CE
0.3466
0.508 004
0.6721o.147
5 CONCLUSION
We present a non-iterative diffusion model called DARL for self-supervised vessel segmentation. Our model composed of the diffusion and generation modules learns vessel representation without labels via adversarial learning, in the guidance of latent features estimated from the diffusion mod- ule. Also, through the proposed switchable SPADE layer, we generate synthetic angiograms as well as vessel segmentation masks, leading to learning semantic information about vessels more effec- tively. Although the diffusion module training is combined with other loss functions, the inference is not iterative but only done in one step, which makes it faster and unique compared to the existing diffusion models. Using various medical vessel datasets, we verify that our model is much supe- learning methods. Moreover, thanks to the diffusion module, our
rior to existing un-/self-superve model is robust to image diversity and noise, suggesting that our model can be an important platform for designing a general vessel segmentation model.
9

---

Published as a conference paper at ICLR 2023
REPRODUCIBILITY
Source code is available aijttpss//gtthub.com////sslll kaiissspp-aiittDaaP.
ACKNOWLEDGMENTY
This work was supported in part by the National Research Foundation of Korea under Grant NRF- 2020R1 A2B511022
in part by Institute of Information & communications Technology Plan- ning Evaluation (IIPP) grant funded by the Korea goVer,ment(MISIT, Ministry of Science and ICT) (No. 2022-0-00984 Development of Artificial Intelligence Technology for Personalized Plug- and-Play Explanation and Verification of Explanation), in part by the MSIT(Ministry of Science and ICT), under the ITRC(Information Technology Research Center) support program (ITTPP-022--202222 0-01461) supervised by the IITP, and in part by the KAIST Key Research Institute (Interdisciplinary Research Group) Project.
REFERENCES
Euijoon Ahn, Dagan Feng, and Jinman Kim. A spatial guided self-supervised clustering network for medical image segmentation. In International Conference Medical Image Computing and Computer-Assisee -ssisste Intervention, pp. 379-388. Springer, 2021.
Tomer Amit, Eliya Nachmani, Tal Shaharbany, and Lior Wolf. Segdiff: Image segmentation with diffusion probabilistic models. arXiv preprint arXiv:2112.00390, 2021.
Dmitry Baranchuk, Andrey Voynov, Ivan Rubachev, Valentin Khrulkov, and Artem Babenko. Label- efficient semantic segmentation with diffusion models. In International Conference on Learning Representations, 2021.
Fernando Servantes-Sancher Ivan Cruz-Aceves, Arturo Hernandez-Aguiree, Martha Alicia Hemandez-Gonzalezz, and Sergio Eduardo Solorio-Meza. Automatic segmentation of coro- nary arteries in x-ray angiograms using multiscale analysis and artificial neural networks. Ap- plied Sciences, 9(24), 2019. ISSN 2076-3417. doi: 10.3390/app9245507. URL https:
// wwwp:/pp...oo/227761111117111121
Mickaël Chen, Thierry Artières, and Ludovic Denoyer. Unsupervised object segmentation by drawing. Advances in neural information processing systems, 32, 2019.
Yang Chen, Luyao Shi, Qianjing Feng, Jian Yang, Huazhong Shu, Limin Luo, Jean-Louis Coatrieux, and Wufan Chen. Artifact suppressed dictionary learning for low-dose et image processing. IEEE transactions medical imaging, 33(12):2271 2292, 2014.
Hyungjin Chung, Bycongsu Sim, and Jong Chul Ye. cone-closer-diffuse--aae Accelerating con- ditional diffusion models for inverse problems through stochastic contraction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12413-12422 2022.
Weijian Cong, Jian Yang, Danni Ai, Yang Chen, Yue Liu, and Yongtian Wang. Quantitative analy- sis deformable madel-based 3-d reconstruction of coronary artery from multiple angiograms. IEEE Transactions Biomedical Engineering, 62(8):2079-21) 2090, 2015.
Jingfan Fan, Jian Yang, Yachen Wang, Siyuan Yang, Danni Ai, Yong Huang, Hong Song, Aimin Hao, and Yongtian Wang. Multichannel fully convolutional network for coronary artery segmen- tation in x-ray angiograms. leee Access, 6:44635 44643, 2018
Mark Hamilton, Zhoutong Zhang, Bharath Hariharan, Noah Snavely, and William T Freeman. Unsupervised semantic segmentation by distilling feature correspondences. arXiv preprint arXiv:2203.084414 2022.
Dongdong Hao, Song Ding, Linwei Qiu, Yisong Lv, Baowei Fei, Yueqi Zhu, and Binjie Qin. Se- quential vessel segmentation via deep channel attention network. Neural Networks, 128:172 187. 2020. ISSN 0893-6080. doi: // 7/www.ac encedi rect.ccom sc httpa///c....eee.a..//////////00
nww.et 20200 URL https:
10

---

Published as a conference paper at ICLR 2023
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp- 770-778, 2016.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge local nash equilibrium. Advances in neural information processing systems, 30, 2017.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 0856840-6851, 2020.
Adam Hoover and Michael Goldbaum. Locating the optic nerve in a retinal image using the fuzzy convergence of the blood vessels. IEEE transactions on medical imaging, 22(8):951-958, 2003,
Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A Efros. Image-to-image translation with conditional adversarial networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1 1125-1134, 2017.
Boah Kim, Inhwa Han, and Jong Chul Ye. Diffusemorph: Unsupervised deformable image reg- istration along continuous trajectory using diffusion models. arXiv preprint 12.054,,,, 2021.
Diederik Kingma and Jimmy Ba. Adam: method for stochastic optimization, arXiv preprint arXiv: 14X2::1412..980, 2014.
Max WK Law and Albert CS Chung. Efficient implementation for spherical flux computation and its application to vascular segmentation. IEEE fransactions on image processing, 18(3):596-612 2009.
Andreas Lugmayr, Martin Danelljan, Andres Romero, Fisher Yu, Radu Timofte, and Luc Van Gool. Repaint: Inpainting using denoising diffusion probabilistic models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11461-11471, 2022.
Yuxin Ma, Yang Hua, Hanming Deng, Tao Song. Hao Wang, Zhengui Xue, Heng Cao, Ruhui Ma, and Haibing Guan. Self-supervised vessel segmentation via adversarial learning. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 7536-7545, 2021.
Faisal Mahmood, Daniel Borders, Richard J Chen, Gregory N McKay, Kevan J Salimian, Alexander Baras, and Nicholas J Durr. Deep adversarial training for multi-organ nuclei segmentation in histopathology images. IEEE transactions medical imaging, 39(1 1):3257-3267, 2019.
Xudong Mao, Qing Li, Haoran Xie, Raymond YK Lau, Zhen Wang, and Stephen Paul Smolley. Least squares generative adversarial networks. In Proceedings of the IEEE international confer- ence on computer vision, pp. 2794-2802, 2017.
Luke Melas- Merris,,riizz,,, Christian Rupprecht, Iro Laina, and Andrea Vedaldi. Deep spectral methods: A surprisingly strong baseline for unsupervised semantic segmentation and localization. In Proceed- ings of the IEEE/CV Conference on Computer Vision and Pattern Recognition, pp. 8364-8375, 2022.
Ebrahim Nasr- fas--Fsahhan, Shadrokh Samavi, Nader Karimi, SM Reza Soroushmehr, Kevin Ward, Mo- hammad H Jafari, Banafsheh Felfeliyan, B Nallamothu, and Kayvan Najarian. Vessel extraction in x-ray angiograms using deep learning. In 2016 38th Anmual international conference of the IEEE engineering in medicine and biology society (EMBC), pp. 643-646. IEEE, 2016.
Yujin Oh and Jong Chul Ye. CXR Segmentation by AdaIN-based based Domain Adaptation and Knowl- edge Distillation. arXiv preprint arXiv:2104.000 05892, 2021.
Taesung Park, Ming- Yu Liu, Ting-Chun Wang, and Jun- Yan Zhu. Semantic image synthesis with aadiiall--daative normalization. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 2337-2346, 2019.
11

---

Published as a conference paper at ICLR 2023
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary De Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang. Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high-performance deep learning library. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché E. Fox, and R. Garnett (eds.), Advances in Neural Information Processing Systems 32, pp. 8024-8035, Cur- ran Associates, Inc., 2019. http:///////ppea.aaaaaaaaa//a///
an an-imperat -eeeiiieesstttttee htt.a//ww..................ooaa.
Olaf Ronneherger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedi- cal image segmentation. In International Conference on Medical image computing and computer- assisted intervention, pp. 234-241. Springer, 2015.
Chitwan Saharia, Jonathan Ho, William Chan, Tim Salimans, David J Fleet, and Mohammad Norouzi. Image super-resolution via iterative refinement. arXiv preprint arXiv:2 2021.
Chitwan Saharia, William Chan, Huiwen Chang, Chris Lee, Jonathan Ho, Tim Salimans. David Fleet, and Mohammad Norouzi. Palette: image-to-image diffusion models. In ACM SIGGRAPH 2022 Conference P Proceedings, pp. 1-10, 2022.
Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics.. In International Conference on Machine Learn- ing. pp. 2256-2265 PMLR, 2015.
Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. Advances in Neural Information Processing Systems, 32, 2019.
Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In Interna- tional Conference on Learning Representations, 2020.
Joes Staal, Michael D Abråmoff, Meindert Niemeijer, Max A Viergever, and Bram Van Ginneken. Ridge-based vessel segmentation in color images of the retina. IEEE transactions on medical imaging, 23(4):501-509, 2004.
Vadim Sushko, Edgar Schönfeld, Dan Zhang, Juergen Gall, Bernt Schiele, and Anna Khoreva. You only need adversarial supervision for semantic image synthesis. arXiv preprint arXiv:2012.04781, 2020.
Maryam Taghizadeh Dehkordi, Ali Mohamad Doost Hoseini, Saeed Sadri, and Hamid Soltanian- zadch. Local feature fitting active contour for segmenting vessels in angiograms. IET Computer Vision, 8(3):161-170, 2014.
Dmitry Ulyanov, Andrea Vedaldi, and Victor Lempitsky. Improved texture networks: Maximizing quality and diversity in feed-forward stylization and texture synthesis. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 6932, 2017.
Shaoyan Xia, Haogang Zhu, Xiaoli Liu, Ming Gong, Xiaoyong Huang. Lei Xu, Hongjia Zhang, and Jialong Guo. Vessel segmentation of x-ray coronary angiographic image sequence. IEEE Transactions on Biomedical Engineering, 1338-133333 2019.
Jun- Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. Unpaired image-to-image translation using cycle-consistent adversarial networks. In Proceedings of the IEEE international conference 2017.
on computer vision, pp.
12

---

Published as a conference paper at ICLR 2023
A DETAILS OF NETWORK ARCHITECTURE
In this section, we provide details of the generator G proposed in our diffusion adversarial repre- sentation learning (DARL) model, which is composed of the diffusion module and the generation module. For the diffusion module, we adapt the network architecture of DDPM (Ho et al. (0000) that has U-Net (Ronneberger al. 2 structure, as described in Table 4 The generation module is composed of four consecutive residual blocks (He et al. 2016) with switchable spatially-adaptin denormalization (SPADE) layers, described in T
Table 4: Detailed network architecture of the diffusion module. For each block (blk), Ct, is the convolution layer with x i kernel and stride length of j. RS; pairs are entry points for residual shortcut path within a block unit, RB is the residual block module, and SA is the self-attention module. GN is the group normalization, and Ch indicates the size of output channel dimension.
Diffusion module
Blk
Downstream
Upstream
Cs.1
RS:
RB
RS
RB
RS:
RSs
RB
RS2
RB
RS.
RB
CB
1
C3.2
RS:
RB
RS₂
RB
RSS
RSg
RB
RS;
RB
RS;
RB
UP
64
Ca.2
RS.
RB
RS2
RB
RSa
RB
RS2
RS.
UP
128
4
C3.2
RS.
RB
RS2
RB
RS:
RSs
RB
RS2
RB
RS,
RB
UP
128
Cs.2
RS.
RB SA
RS
RB
SA
RS;
RSs
RB SA
RS;
RB
RS;
RB
SA
UP
256
Ca.2
RS.
RB
RS
RSa
RB
RS2
RS.
UP
256
Mid
RB
SA
RB
Note:
RB = [RS. GN Swish Ca.1 GN Swish Ca.1 RS...
SA [GN -C- C.
UP = [Upsample- - Ca.1l. CB [GN Swich Cal
Table Detailed network architecture of the generation module. UP is the nearest neighbor upsam- pling function, RS; pairs are entry points for residual shortcut path, C.₁ is the convolution layer with x kernel and stride length jf and IN is the instance normalization layer. S-SPADE the proposed switchable SPADE layer that turns on SPADE if the semantic layout is provided, otherwise turns off SPADE and applies IN. Ch indicates the size of output channel dimension.
Generation module
Stream
Ch
Scale
Conv.
Act.
Norm.
Cony.
Act.
Norm.
C7
IN
ReI.U
DownBlock1
Cs.2
IN
ReLU
128
DownBlock2
Czp
IN
ReLU
256
MidResBlockl
RS;
Cs,1
RcLU
S-SPADE
Cs.1
RcLU
S-SPADE
RS;
256
MidResBlock2
RSa
Ca,1
ReLU
S-SPADE
Cs.1
ReLU
S-SPADE
RSa
256
UpResBlock1
UP
R.S
Cs.1
ReLU
S S-SPADE
Cs.1
ReLUU
S-SPADE
RSs
128
Uloes
UP
RS4
Ca.1
ReLU
S-SPADE
Ca.1
ReLUU
S-SPADE
R54
64
Out
C:
B DETAILS OF DATASET
For training the network, as described in Table 6 we utilize the XCAD dataset (Ma al.) 2021 which provides a total of 1,621 unlabeled X-ray coronary angiography frames. We use each first frame that is taken before the contrast agent injection the real background image. We also generate 1,621 synthetic fractal masks by using the fractal synthetic module proposed by (Ma et al. 2021) The fractal masks are synthesized by drawing rectangles with randomly sampled thickness ranging from 15 to 25 pixels on black background with a size of 512 x 512. Then, local distortions are taken to each rectangle, including affine transformation with a random scale and rotations with a random angle, resulting in generating masks with various shapes and thicknesses. This reduces the effort to match the real vessel thickness distribution, thus, one can simply synthesize such various fractal masks through the fractal synthetic module. Additional 126 angiography images, along with the ground-truth vessel masks annotated by experienced radiologists, are divided into validation and test sets by 10% and 90%, respectively. We subsample all data into 256x256.
For training the baseline methods, we utilize the same amount of angiography images from the XCAD dataset. In specific, method and SSVS utilize angiography images, background images,
13

---

Published as a conference paper at ICLR 2023
and synthetic fractal masks for training both segmentation and generation paths, but each data is randomly sampled independently. DA utilizes angiography images and fractal masks for adversarial training. For inferencing DS, utilize pre-trained network parameters. Redrawing, STEGO and SGC are basically clustering- baastering-based methods, which need only angiography images. For all these methods, we the same 256 256 images ours, without any further image processing such as normalization, but augment data through random flipping and 90-degree rotation.
For external test dataset, we utilize two X-ray coronary angiography (XCA) datasets acquired from different machines. 134 XCA dataset is composed of 134 angiography images with the vessel masks labeled by an expert cardiologist (Cervantes- Sanchee et al. 2019). 30 XCA datset is composed of 30 sequences of angiography images (Hao et al. (020). We utilize one angiography image from each sequence, along with its corresponding ground-truth vessel mask laheled by experts. All the test images are resized to 512x512. Furthermore, for evaluating cross-organ generalization capability, we utilize retinal imaging datasets. We use DRIVE (Staal et al. 2004) and STARE (Hoover & Goldbaum 2003) datasets, each of which is composed of 20 retinal images and the corresponding expert-labeled vessel masks. Since retinal imaging is taken under high-resolution, we resize the image into 768 x 768 and split into 9 patches with 256
Table 6: Detailed dataset for training each (A) segmentation and (B) generation path.
Train
Validation
Test
Input
XCAD
XCAD
XCAD
134 XCA
30 XCA
DRIVE
STARE
Angiogram; an
1,621
12
114
134
30
20
20
(A) Segmentation
Ground-truth mask
12
114
134
30
20
20
Background: x"
1,621
(B) Generation
Fractal mask; sf
1.621
C ADDITIONAL EXPERIMENTAL RESULTS
C.I STUDY ON SYNTHETIC ANGIOGRAM GENERATION
As described in the main paper, single generation module with the switchable SPADE layers in our model provides both the synthetic angiograms and the vessel segmentation masks by one- step inference, compared to iterative inference steps of other diffusion models. To evaluate the angiogram synthesis performance, we compare our model with the methods of DA and SSVS. These baselines generate the segmentation masks and the synthetic angiograms, but unlike ours, they two different networks employing the CycleGAN (Zhu et al. 2017) framework. Furthermore, in this study, we adopt an additional baseline method of OASTS (Sushko al.) 2020), one of the SOTA semantic image synthesis models. Under the same condition as ours with an unpaired dataset setting, we train the OASIS model. For all the baseline methods, total of 1,621 synthetic angiography images are generated using the backgrounds and fractal masks as inputs.
comprers the visual results of synthetic angiograms. Compared to the others, our generation module yields the most realistic images that naturally reflect fractal masks on the background and also contain even tiny branches. This verifies that our model maintains consistency on the vessel-like fractal signals, capturing vessel semantic information effectively and leading to the improvement of segmentation performance. Also, we perform the quantitative evaluation on image generation of angiograms and vessel masks using Fréchet inception distance (FID) (Heusel al. 2017). Table 7 shows that our model achieves much lower FID scores than the comparative methods even though the FID is originally designed for evaluating natural image synthesis, which suggests the superiority of ours in generating angiography images. Also, the proposed DARL model provides the most realistic segmentation masks over the other methods.
C.2 STUDY ON HYPERPARAMETER SETTING
0.2 In the main paper, report the vessel segmentation results from the model trained with
and B based on the study of hyperparameter setting, which yields optimal performance in our experiments. To study the effects of hyperparameters the segmentation performance, using the
14

---

Published as a conference paper at ICLR 2023
Background
Fractal
DA
SSVS
Ours
Figure 6: Visual comparison results of the generated fake angiograms. Yellow boxes in x1 rows are magnified by two or three times in the corresponding bottom rows, respectively. Red triangles indicate remarkable points.
Table 7: Quantitative comparison results using FID score on the image generation of vessel masks and angiograms. Lower FID means that the image generation is more realistic.
Image generation
DA
SSVS
OASIS
Ours
Vessel mask
123.20
149.00
N/A 307.50
93.49
Angiogram
261.64
191.68
177.59
proposed loss funetion, we trained our model with the fixed 5.0 when adjusting the parameter a. Similarly, is fixed with 0.2 when 3 is adjusted. Figure 7 shows graphs of the quantitative evaluation results loU, Dice, and Precision metrics according to the hyperparameters of a and 3. We see that model learn the semantic vessel segmentation when the parameter that controls the adversarial loss Ladv is equal to or more than 0.2, though the performance gradually decreases metrics is achieved when a = 0.2. On the other hand, while our model hardly captures the semantic information of vessels when there is cycle path in network training (i.e. B 0), the model can provide plausible vessel segmentation masks as long as the cycle path exists. Also, when we investigate the segmentation performance according the that weights the cyclic loss Layer the optimal performance is obtained when 3 = 5.0.
increases. Also, the results show that the highest performance for all
C.3
STUDY ON ANGIOGRAM PERTURBATION MODEL TRAINING
When training our model, the background images are perturbed by the forward diffusion process with the uniformly sampled time step to [0, T). whereas the angiograms are perturbed with the time step f, [0. Ta where To T. To investigate the effect of time step size Ta for the angiogram perturbation model performance, we train our model with different time step sizes by setting T, as 100, 200, 500, and 1000. Figure shows the quantitative evaluation results, When T, is less than 200, the vessel segmentation performance on the clean angiograms gets better, but the performance degraded on the simulated low-dose angiograms corrupted by Gaussian noise with levels of 25 and 50. Also, when T, is set over 500, the model shows drastically low performance due the lack of vascular information from the noisy angiograms. These results imply that the diffusion module optimally provide latent features including vascular structures long the model is trained
15

---

Published as a conference paper at ICLR 2023
Io
Dice
Precision
0.40
0.471
0.419
0.60
0,666
0,637
0.515
0.69
(8)
0.20
0.40
0.48
0.378
0.296
0
-4
0.55
6.430
B471
8.61
0.454
BA26
0,629
0.719
0.15
(b)
0.322
0.485
0.500
0.25
0.40
0.50
2.5
7.5
10.0
3.5
5.8
7.5
10.0
3.5
5.8
10.0
Figure 7: Vessel segmentation performance of our model on the XCAD dataset according to the hyper-parameters of the proposed loss function. Each column shows the average values of quanti- tative evaluation results with respect to (a) a for the adversarial loss Lade and (b) / for the cyclic reconstruction loss Leye.
in the setting of Ta < 200. Also, the proposed implementation makes our DARL robust to noise, which suggests that our model can segment vascular structures even on low-dose medical images.
IoU
Dice
Precision
0.8
0.6
0.0
0.0
0.4
-X 7,-100
7,-200
T,-500
- F,-1090
Figure 8: Vessel segmentation performance of our model according to the settings of T. for the angiogram perturbation. Each graph shows the quantitative evaluation results with respect to the Gaussian noise level a.
C.4 STUDY ON ADVERSARIAL LEARNING WITH TWO DISCRIMINATORS
Our proposed model is trained via adversarial learning with two discriminators D2 and D, that distinguish real and fake segmentation masks and angiograms, respectively. To confirm that this discriminator setting is optimal, we additionally conduct experiments to train our model without either D, D.- As reported in Table a)), the model trained without D, slightly increases the precision but degrades the scores for the loU and Dice metrics. Also, the model trained without Da shows inferior segmentation performance as shown in Table [8()), which may be due to the failure to generate realistic angiograms, making the model relatively hard to learn vessel semantic information. These results suggest that model with both D. and D, is optimal to learn vessel representations.
Table Vessel segmentation performance of our model without the discriminator DJDD..
Discriminators
Metric
Method
D.
D.
IoU
Dice
Precision
Ours
/
0.471 07.
0.636
0.701
(a)
/
0.456
0.622
0.728
(b)
0.348 0.348__.061 _.001
0.513
0.496
16

---

Published as a conference paper at ICLR 2023
C.5 STUDY ON DIFFUSION MODEL IN LATENT FEATURE ESTIMATION
Recall that the output of the diffusion module not a simple latent feature of networks but a score function that has spatial information of the data. To show the effect of the diffusion model in our framework, we additionally study our framework by replacing the diffusion module with the autoen- coder model. For fair comparison, We configure the autoencoder by adapting the same DDPM network architecture (Ho et al. 2020 as ours but removing the time embedding vectors and then train without the diffusion loss. Figure shows the latent features and vessel masks of the angiogra- phy images. We can observe that while the autoencoder model estimates latent features that include vessels and other similar confusing structures, our proposed framework with the diffusion module represents vessels better in the latent features and provides more accurate segmentation masks. Ta- blso shows that the autoencoder model achieves inferior performance compared to ours. These results indicate that the latent features from the diffusion module allow the generation module to effectively learn vessel representation.
Autvencoder
Diffusion
Autoencoder
Diffusion
Image
Latent
Image
Latent
Output
Output
GT
GT
Figure 9: Visual results of vessel segmentation according to the latent feature estimation models.
Table 9: Quantitative evaluation results according to the latent feature estimation models.
Latent feature estimation
loU
Dice
Precision
Autoencoder
20.09 0.477
0.566_0.079
0.621
Diffusion (Ours)
0.636
10.115
C.6 STUDY ON MODEL CONTRIBUTION
The contribution of proposed DARL model is further analyzed under supervised, semi-supervised. and self-supervised learning scenarios, and each result on various datasets are described Table 10)
Supervised Firstly, we apply the proposed framework as a supervised model. Here, since there are no labeled pairs in the training dataset, we conduct two supervised segmentation experiments utilizing the 12 pairs of labeled validation data: 1) training our model through only the (A) path, and 2) training
model by giving the labeled data to the (A) path while keeping the (B) path. To evaluate the performance supervised approaches, we report the results the best model among models saves at 10 epoch intervals. In Table I0
observe that in the few-label scenario, DARL framework contributes to the segmentation path to achieve superior performance, leading to getting better than the supervised model only with the (A) path. Moreover, it is noteworthy that the performance of our self-supervised model surpasses the supervised approach only using the (A) path by large margins, even though ours trained without any supervised data.
Semi-supervised As the outputs in the (A) and (B) paths of our DARL model can be used as pseudo labels or inputs for vessel segmentation, we train the segmentation model in a semi- supervised manner. In specific, we prepare the segmentation network that is identical network ar- chitecture to that of our generation module G for fair comparison. Then, we train the network by utilizing the (A) path outputs &" paired pseudo-label for input x" Similarly, we train the network by using the (B) path outputs &" as paired pseudo-inpur for label sf. As reported in Table 10 when compared to the model trained using the data from the (A) path, the performance is slightly higher
17

---

Published as a conference paper at ICLR 2023
than ours. This suggests that the generated masks from our model be used for pseudo-labels for unlabeled data. Also, it is remarkable that although our method estimates the vessel maps and synthetic angiography images simultaneously, the semi-supervised method using the pseudo-labels. On the other hand, the model trained using the data from the (B) path shows mostly lower performance than ours, implying the cycle path in model is more effective to extract vessels.
method achieves comparable performance with
Self-supervised We further test the application of our model an environment that has back- ground images. Our model is trained by replacing the background images a." in the (B) path with the real angiography images x²". Table shows that the segmentation performance of model without non-contrast background images is comparable or even superior to our model. As the gen- eration module of our DARL takes latent features of input images, our model can synthesize images involving the information of input angiography images, based on semantic masks, and learn vessel features. This can be a unique characteristic compared to other semantic image synthesis models that typically take the images directly.
Table 10: Study on model contribution to various learning frameworks.
Supervised (few-label)
Semi-supervised (pseudo-pairy
Self-supervised
Data
Metric
(A) path
(A)+(B) puth
Data from (A)
Data from (B)
Ours
Ours w/o BG
IoU
g0.077
0.548
0.47820.078
0.445=0.072
0.471
0.47920.080
XCAD
Dice
0.590 20590
0.705 0.705
0.643
0.613 0..03
0.63620.072
0.6 0.64420.070
Precision
0.637 10.138
0.746
0. 10.
0.645 1..45_0.117
0.701 11.115
0.700
External test: Coronary angiography
IoU
0.273
0.323
0.358 108810.1T
0.378 L0.179
0.426
0.377 10.177
134 XCA
Dice 0.104
0..54_0.257
0.531
0.522 20.200
0.595_0.058
0.518 21.
Precision
2.439
0.453=202336
0.477 20.213
20.58
0.781:0.118
20205
0.365-g0.022
0,484 00.00
0, 0.42240.004
0. 0.420=00080
0.427 184
0.420=00070
XCA
Diec
0.534
0.6-9 20.081
2558820000
0.592 00009
0.572
0.097
Precision
0.717
0.829
10.088
0.772 124
0.737 10.133
0.729
0.762
Crust-modality test: Retinal imaging
LoU
0.344
0.388
0.365
0.341 10.4177.147
0.372
0.392 2002:=0..0
DRIVE
Dice
0.497
0,544
0.518
0.490
0.525
0.550
Precision
0.617 20.17
0.75440.213
0.585
0.504
0.617
0.65920.247
TnU
0.319
0.332
0.375
0.354 10188
0.368
0.393_1.183
STARE
Dice
0.458
0.464-2..2m 235
0.51 20.210
0. 1..95-5m 20.00 2008
0.508g0:216
0.538 208
Precision
0.529 go
0.591 1. 2815
0.528 260
0.481 20.254
10:200
0.566
BG; background images xx
C.7 STUDY ON IMAGE PROCESSING FOR THE UNSUPERVISED METHODS
For the implementation of comparison methods, we did not perform any image processing except for resizing the images. However, the performance of unsupervised methods would be affected by the processing such as nor- malization, additionally tested the unsupervised meth- ods by applying several normalization methods to the in- put images. As shown in Figure 10, we processed im- ages using histogram equalization (HE) and contrast lim- ited adaptive histogram equalization (CLAHE). Table T1 shows that the performance of unsupervised methods de- grades when using the normalized data. This comes from the angiography images that are hard to visualize only vessel regions due to the confusing background structures though the CLAHE and HE enhance the image con- trast. On the other hand, our method outperforms the other comparative models, suggesting the superiority of our methods.
Buseline
HE
CLAHE
Figure 10: Examples of image process- ing of HE and CLAHE normalization.
18

---

Published as a conference paper at ICLR 2023
Table 11: The Dice scores for the segmentation performance unsupervised methods applying HE and CLAHE to the input images.
Input processing
SGC
Redrawing
DS
Ours
Baseline HE
0.111
0.109
0.526
0.636
0.061
0.107
0,492
CLAHE
0.119
0.099
0.495
C.8 STUDY ON TRAINING COMPLEXITY
Our model has unified generator module which can perform both the segmentation and the gen- eration tasks simultaneously, by efficiently decreasing training complexity compared to CycleGAN structure of other ad adversarial learning framework such like DA or SSVS. We estimate floating point operation per second (FLOPS) of each method in Table and prove the cost-effective characteristic of our model.
Table 12: Training complexity (FL.OP)).
Method
(A) path
(B) path
Cycle path
Discriminator
Total
DA
121.80
121.80
121.80 x 2
6.24 x 2
499.68
SSVS
121.80
121.80
121.80
6.24 x
499.6
Ours
90.66
173.51
90.66 x
6.24 x
367.31
D ADDITIONAL VISUAL RESULTS OF VESSEL SEGMENTATION
D.I RESULTS OF ABLATION STUDY
Implementation detail Section 4.1 of the main paper, we implemented several ablated models of ours. For the methods without the diffusion module (i.e. (a) and (b) in Table 3), the real angiog- raphy images are given for the (A) path, and the real background and synthetic fractal masks are given for the (B) path. Here, there is diffusion module, the input images are not perturbed. In particular, for the methods without S-SPADE layers in the generation module (i.e. (b) in Table 33) there are two independent generation modules for the (A) and (B) paths, with instance normaliza- tion and SPADE as normalization layer for each, so that the input image for each path is given to the corresponding module. On the other hand, for the ablation studies on the loss functions, the data flows are the same as ours.
Visual results Figure shows the qualitative comparison results for the ablation studies in the main paper. (a) and (b) show that our model trained without the diffusion module generates the vessel masks including many false positive regions. When compared ours with (c), we can observe that the cycle consistency on the fake vessels allows our model to segment tiny vessels more ac- curately. Moreover, the comparison of (d) and ours verify that the proposed model achieves better performance, although solve the segmentation problem with the image generation in that use L1 loss for the segmentation masks.
D.2 RESULTS OF OUR DARL MODEL
In this section, we provide additional vessel segmentation results that show the success of our DARL in self-supervised segmentation. Figure 12 shows that our model consistently provides the best performance on XCAD dataset. Also, our model segments vascular structures better than the other baselines even the external angiography datasets (134 XCA and 30 XCA) and the retinal image datasets (DRIVE and STARE). These results suggest that the proposed model can be used as a general vessel segmentation model for various vascular images. Also, Figure 13 shows the vessel segmentation results on the XCAD data that are corrupted by Gaussian noise with different levels of The visual results demonstrate that DARL is the only method which endures harsh corruption and outperforms the baseline methods.
19

---

Published as a conference paper at ICLR 2023
Image
GT
Ours
(a)
(b)
(
(d)
Figure 11: Visual comparison results of the ablation study. (a-d) correspond each case study (a-d) in Table of of the main paper. Yellow boxes denote remarkable parts.
20

---

Published as a conference paper at ICLR 2023
Image
GT
Unsupervised
Self-supervised
SGC
Redmwing
STEGO
SSVS
Ours
XCAD
134XCA
30XCA
DRIVE
STARE
Figure 12: Additional visual comparison results different angiogram and retinal datasets.
21

---

Published as a conference paper at ICLR 2023
Image
GT
Unsupervised
Self-supervised
SGC
Redrawing
DS
STEGO
DA
SSVS
10
25
25
50
10
50
Figure 13: Additional visual comparison results on XCAD datasets with different levels (α) of Gaussian noise.
22


Reviews:
[{"review_id": "_eXdMb1dKvP", "reviewer": "Reviewer_dTPf", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper presents a DDPM based method for vessel image synthesis and segmentation. The idea is to train one base DDPM for both tasks, using a switchable SPADE as a means for incorporating the dfferences between these two tasks. Further, the DDPM is trained in a self-supervised manner. Experiments on the benchmark datasets, including both vessel and non-vessel datasets, demonstrate that the proposed method is effective and achieve better performances that competing SOTA methods.", "summary_of_the_review": "Overall, the paper is of high quality with good motivation, decent novelty, and solid experiments. The experiments should be augmented with new results to make the paper even stronger. \n", "strength_and_weaknesses": "Strength:\nGood writing and easy to follow. Decent novelty. Solid experiments（but with the below weaknesses)\n\nWeaknesses:\n\nThe paper claims a fully self-supervised manner for learning the model. In fact, for the segmentation path A that takes an image as input and outputs the segmentation mask, the adversarial loss is used on the mask output. Practically speaking, this is a bit uncessary as the real segmentation masks are used in the loss function. For each real segmentation mask, it comes with a real input image. Therefore, it is possible to conduct supervised training at least for this path.  I am afraid that the good performances in fact arises from this part.\n\nThere are two experiments that can can be done to verify the effectiveness of the proposed method.\n1) Change the path A to a fully supervised one while keeping the path B unchanges and then  compare with fully supervised segmentation approach\n2) Change the output masks of path A to synthetic masks.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "GtuuN4O3yJQ", "reviewer": "Reviewer_xqnu", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper introduces a diffusion adversarial representation learning (DARL) model, for self-supervised vessel segmentation. The major novelty appears to be the usage of a diffusion module for estimating latent features, which are then used by a generation model to estimate both vessel segmentation masks and synthetic angiograms, through switchable spatially-adaptive denormalization (SPADE) layers.\n\nThe estimated output depends on the input, which admits two paths: Path A accepts a real, noisy angiography image, and estimates the vessel segmentation mask (the main task). Path B accepts a real noisy background image and a vessel-like fractal mask, to generate a synthetic angiography image that attempts to incorporate vessels corresponding to the input fractal mask. The synthetic angiography images can then be forwarded through Path A to apply cycle consistency within the LSGAN framework. Experiments are done on the public XCAD datasets, and external XCA datasets, and also applied to out-of-domain retinal imaging datasets. DARL was shown to outperform both unsupervised baseline methods, and recent self-supervised methods such as STEGO, DA and SSVS.\n", "summary_of_the_review": "The proposed DARL framework introduces a diffusion module towards self-supervised vessel segmentation exploiting cycle consistency. Segmentation results appear superior to other recent approaches, and robust across input noise and specific task.\n", "strength_and_weaknesses": "Strengths:\n\n1. Novel application of diffusion towards task-and-noise robust vessel segmentation.\n\n2. Ablation experiments performed to quantify contributions of various modules/losses.\n\n3. Extensive comparisons against other self-supervised and unsupervised methods.\n\n4. Additional experiments described in appendix.\n\nPossible Weaknesses/Considerations:\n\n5. The nature of the background angiography images might be explained further, as readers may not be familiar with the particular task. Are these background images obtained in the same way as the vessel images, just from areas without blood vessels? If so, should their background distribution be expected to be similar to the areas with blood vessels?\n\n6. Some of the most obvious/natural comparisons might not have been attempted. In particular, the inclusion of the diffusion module appears for the estimation of latent features. However, such an estimation of latent features appears doable with an (auto)encoder, possibly with fully-connected layers for both the encoder and decoder within. The support for the DDPM diffusion model for estimating the latent features thus might not be fully established.\n\n7. Similar to the above, the quality of the synthetic angiography/vessel segmentation images produced by the generation module does not appear benchmarked against state-of-the-art GANs for common metrics such as Inception score etc.\n\n8. Moreover, once synthetic angiography images are generated from corresponding (fractal) vessel masks in Path B, the image-vessel mask pairs should constitute acceptable training data for common image segmentation models (e.g. U-Net and variants; i.e. Type B Ground Truth). The same applies to the estimated vessel masks from Path A (i.e. Type A Ground Truth). The specific contribution of the proposed DARL framework might thus be clarified.\n\n9. For the experiments, it might be clarified as to whether any (global) image preprocessing (e.g. CLAHE) was applied. Unsupervised methods would appear particularly disadvantaged without such normalization.\n\n10. The quality of the vessel segmentations appears possibly sensitive to the fractal masks as generated by the fractal synthetic module (from Ma et al., 2021). In particular, the correlation between the (distribution of) thicknesses of the fractal vessels and the actual vessels in the synthetic angiography images, appears critical towards achieving good IoU scores. Was there any attempt to match true vessel thickness distributions, or to optimize the predicted vessel thicknesses against ground truth?\n\n11. For the ablation study in Section 4, it might be explained in greater detail how the model works without the diffusion module, i.e. what the inputs/data flow is then like, possibly in the appendix.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "EEmRZSj7gE", "reviewer": "Reviewer_qGAg", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes a self-supervised technique for vessel segmentation based on a combination of a denoising diffusion probabilistic model and adversarial learning. The network uses contrast and non-constrast images of vessels and uses the non-contrast images to learn a distribution of the background that is then used to identify the foreground vessels. The approach is evaluated for vessel segmentation in X-ray coronary angiography using two datasets and for retinal vessel segmentation using the DRIVE and STARE datasets in comparison to 6 other recent methods.", "summary_of_the_review": "I have put this as marginally above the acceptance threshold based on the above reasons, but I think the manuscript could be strengthened quite a bit by going over the writing.", "strength_and_weaknesses": "Strengths\n---------\n- Addresses the important and relevant problem of vessel segmentation with limited supervision.\n\n- The approach makes sense and seems to show convincing results.\n\n- Novel use of diffusion models for self-supervised learning.\n\n- Results appear to be a good deal better than state of the art.\n\n\nWeaknesses\n----------\n- The method appears to rely on both contrast and non-contrast images and this could limit the usability of the approach as not all usecases have both. Perhaps the authors could clarify the importance of this requirement and limitation in the paper?\n\n- I find it unclear what amounts of labelled and unlabelled data is used for the self-supervised experiments. Could the authors clarify this? Unsupervised and self-supervised learning is mostly relevant if it can be used to achieve clinically relevant results with fewer labels. The results reported on the DRIVE and STARE datasets do not seem particularly convincing compared to what has been reported by supervised techniques. I would find it relevant to discuss this and compare to fully supervised methods.\n\n- Another potential problem with relying on both contrast and non-contrast images is that differences between the modalities are actually a quite large source of signal for these problems. Of course one can claim that this source of information is in some sense free from supervision, but to make the comparison with the other methods more fair, perhaps it would make sense to comment on the degree the other methods can actually exploit this information as well.\n\n- The increasing training complexity of the two adversarial networks is used as an argument against Ma et al. in the Introduction. Is there any concrete evidence of this being a drawback in comparison to the proposed work? What is the training complexity of these approaches? For comments like that to be supported, I would expect measurements of FLOPS or similar.\n\n- Figure 2 legend do not describe the content with enough detail. What are all the variables, subscripts and superscripts?\n\n- \"Experimental results show that our method outperforms existing unsupervised and self-supervised learning methods in the absence of labeled data for training.\". I find a good argument for including exactly these methods as comparison methods to be lacking. Could the authors justify why these methods are a good representation of the state of the art?\n\n- \"most unsupervised methods fail to endure drastic performance degradation when they are applied to vessel segmentation tasks.\", are you sure this is what you mean to say?\n\n- No code repository makes it hard to reproduce.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}, {"review_id": "2y0Xa7ovUf", "reviewer": "Reviewer_yNNg", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a self-supervised vessel segmentation method with denoising diffusion model and adversarial training. The proposed method separates the image background and vessel by learning the representation of background images with diffusion model. The main contribution is the combination of diffusion model and generation model for self-supervised vessel segmentation.", "summary_of_the_review": "the proposed method seems effective and interesting, but the writing is unclear, and some details are missed.", "strength_and_weaknesses": "Strengths: \n--proposed an effective vessel segmentation method composed of diffusion and adversarial training.\n--Obtained a significant improvement of vessel segmentation both on X-ray coronary angiography and fundus images.\n\n\nWeaknesses: \n--the innovation is limited. Diffusion and adversarial are the common approaches, but the authors have not claimed why they can together? In addition, the main dear of this work is cycle reconstruction. It is also common.\n\n--the motivation and writing are unclear. The authors claim that diffusion is employed to learn background image distribution, but the relationship between background distribution and vessel representation is not presented clearly.\n\n--The details of the model and implements are missed. Such as how to train path A and B simultaneously? How to switch SPADE?\n\n--The method may be hard to implement\n\n--It seems that path A of x^a and path B x^b are trained independently. How to make the training consistent?\n\n--In the experiments, the fundus image is introduced. But for training, how to obtain the real noisy background image?", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: poster