AdaWAC: Adaptively Weighted Augmentation Consistency Regularization for Volumetric Medical Image Segmentation
Authors: Yijun Dong
Yuege Xie
Rachel Ward
Venue: Submitted to ICLR 2023 (2023)
Keywords: Medical image segmentation
Adaptive weighting
Consistency regularization
Subpopulation shift

Abstract: Sample reweighting is an effective strategy for learning from training data coming from a mixture of different subpopulations. However, existing reweighting algorithms do not fully take advantage of the particular type of data distribution encountered in volumetric medical image segmentation, where the training data images are uniformly distributed but their associated data labels fall into two subpopulations---"label-sparse" and "label-dense"---depending on whether the data image occurs near the beginning/end of the volumetric scan or the middle. For this setting, we propose AdaWAC as an adaptive weighting algorithm that assigns label-dense samples to supervised cross-entropy loss and label-sparse samples to unsupervised consistency regularization. We provide a convergence guarantee for AdaWAC by appealing to the theory of online mirror descent on saddle point problems. Moreover, we empirically demonstrate that AdaWAC not only enhances segmentation performance and sample efficiency but also improves robustness to the subpopulation shift in labels.

Under review as a conference paper at ICLR 2023
AdaWAC: ADAPTIVELY WEIGHTED AUGMENTATION CONSISTENCY REGULARIZATION FOR VOLUMETRIC MEDICAL IMAGE SEGMENTATION
Anonymous authors Paper under double-bling blind review
ABSTRACT
Sample reweighting is effective strategy for learning from training data com- ing from a mixture of different subpopulations. However, existing reweighting algorithms do not fully take advantage of the particular type of data distribu- tion encountered in volumetric medical image segmentation, where the training data images are uniformly distributed but their associated data labels fall into two subpopulations- "aaiooosss"llaaspppasaaaa "label-sparse" and "label-dense" dependdnnn
whether the data image occurs near the beginning/end of the volumetric sean or the mid- dle. For this setting, we propose AdaWAC
adaptive weighting algorithm that assigns label-dense samples to supervised cross-entropy loss and label-sparse samples to unsupervised consistency regularization. We provide convergence guarantee for AdaWAC by appealing to the theory online mirror descent on saddle point problems. Moreover, we empirically demonstrate that AdaWAC not only enhances segmentation performance and sample efficiency but also improves robustness the subpopulation shift in labels.
INTRODUCTION
Modern machine learning has been revolutionizing the field of medical imaging, especially in computer-aided diagnosis with Computed Tomography (CT) and Magnetic Resonance Imaging (MRI) scans. While the successes of most classical learning algorithms (e.g., empirical risk mini- mization (ERM)) build upon the assumption that training samples are independently and identically distributed (i.i.d),, real-world volumetric medical images rarely fit into this picture. Specifically for medical image segmentation, as instantiated in Figure [ the segmentation labels corresponding to different cross-sections of organs within given volume tend to have distinct distributions. That is, the slices toward the beginning/end segmentation labels (which refer as "label-sparse"", ;); whereas segmentation labels prolific in the slices toward the middle the volume ("label-dense"). Such discrepancy in labels results in distinct difficulty levels measured by the training cross-entropy (Wang al. 202 and leads to various training schedulers (Tullis & Benjamin, 2011 Tang et al. [2018] Hacohen & Weinshall] 2019). Motivated by the separation between label- -ppars and label-dense samples, we explore the following questions in this work:
the volume that contain no target organs have very few. any,
What is the effect of separation between sparse and dense labels on segmentation? Can leveruge such separation improve the segmentation accuracy?
We first formulate the mixture of label-sparse and label-dense samples as subpopulation shift in the conditional distribution of labels given images P (y|x). As illustrated Figure 1 such subpopulation shift induces separation in supervised cross-entropy between sparse and dense labels despite the uniformity of data images.
Prior works address the subpopulation shift issue by utilizing hard thresholding algorithms such as Trimmed Loss Estimator (Shen Sanghavi 2019), MKL-SGD (Shah et al. 2020). Ordered SGD (Kawaguchi & Lu, 2020), and guantile-based Kacmarz algorithm (Haddock al.) 2020). However, these trimmed Hommeed-los-aae loss-based methods discard the samples from some subpopulations (e.g. samples with label comupptio estimated by their losses) at each iteration, which results in loss of

---

Under review as a conference paper at ICLR 2023
information in the discarded data, leading to reduced sample efficiency. Relaxing the hard thresh- olding operator to soft thresholding is proposed to incorporate the information from both subpopu- lations (Wang et al. 2018; Sagawa et al., 2020). However, lowering the weights assigned to some subpopulations of data according to the properties of their labels reduces the importance of the data and labels simultaneously, suggesting that may further improve the learning efficiency by exploiting the uniformity of data and the separation of labels separately.
cross entropy
FRASS Mirte
loss
322
Figure I: Evolution of cross-entropy losses versus consistency regularization terms for slices across one training volume (Case 40) in the Synapse dataset (Section 5) during training.
Instead of thresholding out or down- nown-weighting the label-sparse samples, we exploit the image inputs of these samples via augmentation consistency regularization. Consistency regularization (Bach-) man et al. 2014 Laine & Aila 2016 Sohn et al. 2020) aims to learn the proximity between data augmentations of the same samples; crucially, this set-up does not involve the data labels, and hence consistency regularization has become an essential strategy for utilizing unlabeled data. For med- ical imaging tasks, consistency regularization has been extensively studied in the semi-supervised
learning setting (Bortsova et al. (2019; /Zhao et al. 2019 Li et al.) 2020; Wang et al. 2021a ZZhag al., 202T; /Zhou et al.] 2021 Basak et al. 2022) By contrast, will explore its potency in the fully supervised setting leverggin the spare information in all image inputs, regardless of their
label subpopulations.
Moreover, in light of the uniformity of unsupervised consistency on different slices throughout each volume, the augmentation consistency of the encoder layer outputs serves as natural reference for separating samples from different subpopulations. Whereby, we introduce the weighted aug- mentation consistency (WAC) regularization-2 minimax formulation that not only incorporates the consistency regularization but also leverages the consistency regularization reweighting the cross-entropy and the augmentation consistency terms corresponding to different samples. At the saddle point, the WAC regularization automatically separates samples from differ- ent label subpopulations by assigning all weight to the consistency terms for label-sparse samples, and all weight to the cross-entropy terms for label-dense samples.
reference for
Furthermore, as an algorithm for solving the minimax problem posed by the WAC regularization, propose Ada WAC adaptive weighting scheme inspired by minror-descent descent-bbbetedd algorithm for distributionally-rohuu
optimization (Sagawa et al., 2020). By adaptively adjusting the weights between the cross-entropy and consistency terms of different samples, AdaWAC comes with both convergence guarantee and empirical success.
Overall, we summarize the main contributions of this work follows:
We cast the discrepancy between the sparse and dense labels within each volume as subpopula-
tion shift in the conditional distribution P (y) x) (Section 2p.
We propose WAC regularization which uses the consistency of encoder layer outputs (in a UNet architecture) as natural reference to incentivize separation between samples with sparse and dense labels (Section 3. along with an adaptive weighting -odrrtthm-AdaACC-aaa
solving the
WAC regularization problem with convergence guarantee (Section 4
We empirically demonstrate the potency of AdaWAC not only in enhancing segmentation perfor- mance and sample efficiency but also in improving distributional robustness (Section 5).
2

---

Under review as a conference paper at ICLR 2023
1.1 RELATED WORK
Sample reweighting. Sample reweighting is a popular strategy for coping with subpopulation shifts in training data where different weights are assigned samples from different subpopula- tions. In particular, the distributionall-rrouss optimization (DRO) (Ben- -een-Tal et al., 2013; Duchi al. 2016, Duchi & Namkoong [0118] [Sagawa et al., 2020) considers a collection of training sam- ple groups from different distributions where, with the explicit grouping of samples, the goal is to minimize the worst-case loss over the groups. Without prior knowledge on sample grouping, impor-
tance sampling (Needell et al. [2014] /Zao & Zhang 2015: Alain et al. [015] Loshchilov & Hutter
2015 Gopal 2016 Katharopoulos & Fleuret] 2018), iterative trimming (Kawauuchi & Lu] 2020]
Shen & Sanghavi| 2019), and ampirical-loss-basse reweighting (Wu et al. (022) are commonly in- corporated in the stochastic optimization process for adaptive reweighting and separation of samples from different subpopulations.
Consistency regularization. Consistency regularization (Bachman et al. 2014 Laine & Aila 2016 [Sohn et al.] 2020 [Berthelot et al., 2019) is popular way to exploit data augmentations that encourage the model to learn the vicinity among augmentations of the same sample, with the assumption that data augmentations generally preserve the semantic information in data.
For medical imaging, consistency regularization is generally leveraged as a semi-supervised learning tool (Bortsova et al. [201 Zhha et al. 2019 Li et al., 2020; Wang et al. 2021a Zhang et al. 2021
Zhou et al. |022 Basak al. (0022). In efforts to incorporate consistency regularization in medical image segmentation with augmentation-sensitee
labels, Li et al. (2020) encourages transformation
consistency between predictions with augmentations applied to the image inputs and the segmen- tation outputs. Basak et al. (2022) penalizes inconsistent segmentation outputs between teacher- student models, with MixUp (Zhang al., 2017) applied image inputs of the teacher model and segmentation outputs of the student model. Instead of enforcing consistency in the segmentation out- put space as above, our algorithm leverages the insensitivity of sparse labels to augmentations and encourages consistent encodings (in the latent space of encoder outputs) on label-sparse samples.
2 PROBLEM SETUP
Notations. We denote [K] = (1...., (1.....K) K} for any K E N. For an arbitrary tensor, we adapt the syntax for Python slicing on the subscript (except counting from 1) to represent its elements and subtensors. For example, Xjs,] denotes the (i,j)-entry of the two-dimensional tensor and [(i.)) denotes its i-th row. Let I be a function onto {0, 1} such that, for any event e, [[e] 1 if e is true and otherwise. For any distribution P and n E N. let pm denote the joint distribution of n samples drawn ii.d from P. We refer to an event as happening with high probability (w.h.p) if it takes place with probability 1 Ω (poly (n))
2.1 PIXEL-WISS CLASSIHICATION WITH SPARSE AND DENSE LABELS
We consider the volumetric medical image segmentation as a pixel-wise multi-class classification problem where we aim to learn pixel-wise classifier h X' > [K]d that serves as good approxi- mation for the ground truth h" X [K]a.
Recall the separation of cross-entropy losses between samples with different fractions of non- background labels during training from Figure [ We refer to sample (x,y) E X' x [K] as label-sparse most pixels y are labeled as the background such that the cross-entropy loss on (x. y) converges rapidly in the early stage of training"" Otherwise, we say that (x, is label-dense. Formally, we describe such variation as a subpopulation shift in the label distribution.
Definition 1 (Mixture of label-sparse and label-dense distributions). Let Po and P1 be the distri- butions of label-sparse and dense- samples with distinct conditional distributions Po (y x)
segmentation label a key feature laber-sparse samples (as the name suggests), the unknown cut-off on such sparsity degenerates it as a sufficient condition for the rapid convergence of cross-entropy loss (Figure []] Instead of making distinction with the sparsity of hackouudd pixels, we formalize natural separation between label-sparse and label-dense samples in Assumption I based on which our algorithm distinguish different samples spontaneously.
AAtthough the sparsity of non- backgroudd pixels
3

---

Under review as a conference paper at ICLR 2023
and Pi (ylx), respectively, but the same marginal distribution (x) such that P. (x,y)
P. (y|x) P (x) (i = 0.1) 1). For & E [0, 1], we define a data distribution Pe where each sample (x Pg is drawn either from P1 with probability or from Po with probability 1 ç.
We aim to learn a pixel-wise classifier from a function class H 3 hig = argmax;e(K] So (x) (x)aa for all j [d] where the underlying function fo E F, parameterized by some 0 E Fo. admits an encoder-decoder structure:
F= {fo - do o v0 Ø6 X + 2, 2h : 2 - [0,1]xxx].
(1)
Here Φo, t/p correspond to the encoder and decoder functions, respectively; (Fo, x)) denotes the inner product space of parameters with the induced norm ||-1|>= and dual norm || F.11 (2, @) is a latent metric space.
To learn from segmentation labels, consider the averaged cross- cross-entropy loss:
tcR (0; (x. - Σ K
d Σ
M [y =k} -log (Jo (x) =
log (Jo (x) (2)
We assume proper learning where there exists 0 E (ej argmingez, Exxy)re ller (0; (x,y)) that is invariant with respect to &.
2.2 AUGMENTATION CONSISTENCY REGULARIZATION
Despite the invariance of fo- to P' on the population loss, in practice we have a finite number of training samples and the predominance of label-sparse samples in the training set introduces difficulties due to the class imbalances. with encoder-decoder structure (Equation (I). when the label-sparse samples are predominant (5 << 1), a decoder function the that predicts every pixel as background can achieve near-optimal cross- entropy loss, regardless of what the encoder function Φe is, considerably compromising the test performance (cf. Table []. To encourage legit encoding even in absence of sufficient dense labels, leverage the unsupervised consistency regularization on the encoder function Φo based data augmentations.
a specific extreme scenario for the pixel-wise classifier
Let A be distribution X' + X) induces an augmentation A (x) of x that perturbs low-level information in x. We aim to learn encoder function dig X - Z that is capable of filtering out low-level information from x and therefore provides similar encodings for augmentations of the same sample. Recalling the metric P on Z, for a given scaling hyperparameter Aac > O. we measure the similarity between augmentations with a consistency regularization term on Φe (-): for any A1, Ag ~ A²,
transformations on X' where for any x E X, each A A (A
fAc (0: x, A1. A2) & AAC e( (A)( de (A2(x)))
(3)
For the n training samples {(x, yz) Ma)heet iE(n) ~ Ph, we consider n pairs of data augmentation transfor- mations {(A.,, A1,)) ~ A In the basic version, we encourage the similar encoding de (-) of the augmentation pairs (Ar, (x:). 4,2 (x:)) for all i E [m]] via consistency regularization:
min 1
Σ 1=1
lce (x., + tac (0: Xx, Ar,1, A42).
(4)
FE
We enforce consistency on de (-) in light of the encoder-decoder architecture: the encoder is gener- ally designed to abstract essential information and filters out low-level non-semantic perturbations (e.g., those introduced by augmentations), while the decoder recovers the low-level information for the wixe classification. Therefore, with different A₁ Ag ~ A, the encoder output Φe (-) tends to be more consistent than the other intermediate layers, especially for label-dense samples.
WEIGHTED AUGMENTATION CONSISTENCY (WAC) REGULARIZATION
As the motivation, begin with key observation about the averaged cross-entropy:
4

---

Under review as a conference paper at ICLR 2023
Remark 1 (Separation of averaged cross-entropy loss Po and Pt). As demonstrated in Figure I the sparse labels from Po tend to much easier to learn than the dense ones from Pt, leading to considerable separation of averaged cross-entropy losses on the sparse and dense labels after a sufficient number of training epochs - tcr (0: (x) & fce (0; (x', y')) for most label-sparse samples (x, y) Po and label-dense samples (x' PI.
Although Equation (4) with consistency regularization alone can boost the segmentation accuracy during testing (cf. Table 4). it does not take the separation between label-sparse and label-dense samples into account. Section[5) we will empirically demonstrate that proper exploitation of such separation, like the formulation introduced below, can bring compelling further improvements.
Coneretely, we formalized the notion of separation between Po and P1 based on the consistency regularization term (Equation 33 with the following assumption
Assumption (NVssparatiin between Po and P.). Given a sufficiently small 7 > O, let Fg- (7) = {0 Fe | 0 1- }} be compact and convex neighborhood of well-trained pixel-wise clas- sifier We say that Po and P1 are N N-separated over Fg. (7) if there exists w >> 0 such that both:
(i) cc (0; (x. < fac (0; x, A1, A2) for all E Fo- (7) given (x.y) Po
i LcE (0; (x.)) fac (0;x., x, A1, A2) for all 0 E Fe. (7) given (x,y) Pi
hold with probability Ω (NI+w) over ((x,y), (A1, A2)) Pg A
This assumption motivated by the empirical observation that the perturbation in do (-) induced by A is more uniform across Po and P1 than the averaged cross-entropy, as instantiated in Figure
Under Assumption | up to a proper scaling hyperparameter AAc. the consistency regularization (Equation (3) can separate the averaged cross-entropy loss (Equation (2) on N label-sparse and label-dense samples with probability 1 Ω (NW)) by the union bound (as explained formally in Appendix A). In particular, the larger N correspond to the stronger separation between Po and Pi.
With Assumption 1 we introduce a minimax formulation that incentivizes the separation of label- sparse and label-dense samples automatically by introducing a flexible weight P1g E [0, 1] that balances ECE (0; (x)) and fac (0; x,, Ai,, Ax2)) for each of the n samples.
@WAC, 3 argmiin argmax (xFor(>) BE(O.1]"
L WAC (0,
1
I
(0,
(5)
T.WAC (0,3)= BH leE (0; (x), + (1 - B(g)) fAc (0: x,, A1,)-
With convex and continuous loss and regularization terms (formally in Proposition I Equation has saddle point where 33 separates the label-sparse and label-dense samples under Assumption
Proposition 1 (Formal proof in Appendix A Assume that ECE (0; (x, and tac (0; x, A A2)
are convex and continuous in 0 for all (x.y) E X x [K] and A,, Ag ~ A2; Fe- (7) C Fo is
compact and convex. If Pu and P1 are n-separated (Assumption 7. then there exists f {0, 1)"
and @WAC argmingGFo-(r) (r) I. W.C (0.3) such that
WWC (0.3)= = WAC (QWAC (B,,,,3 .3) max WWC (owac, .3) 3E(0.1)
min FEF (T)
(6)
Further, G separates the label-sparse and label-dense samples-
Il (x) ~ w..pp.
In other words, for n samples drawn from a mixture of the n-separated Po and Pı, at the saddle point, Equation (5) automatically identifies the label-sparse samples with B 0, learning more from the
unsupervised consistency regularization, and the label-dense ones with 3 = 1, emphasizing more on the supervised averaged cross-entropy loss.
2 We that although Assumption 1 can be rather strong, only required for the separation guarantee of lahel-sparse and lahel-dense samples with high probability in Proposition I but not for the adaptive weighting algorithm introduced in Section orr in practice for the experiments. With pretrained initialization, assume that the optimization algorithm is always probing in Fo- (7)-
5

---

Under review as a conference paper at ICLR 2023
4 ADAPTIVELY WEIGHTED AUGMENTATION CONSISTENCY (AdaWAC)
Remark 2 (Connection to hard thresholding algorithms). The saddle point of Equation is closely related hard thresholding algorithms like Ordered SGD (Kawaguchi & Lu 2020) and iterative trimmed loss (Shen & Sanghavi] 2019). In each iteration, these algorithms update the model only on a proper subset of training samples based on the (ranking of) current empirical risks. Compared to hard thresholding algorithms, (i) Equation ( additionally leverages the unused samples (e.g.. label-sparse samples) for unsupervised consistency regularization on data augmentations, which is known for improving generalization and feature learning even in supervised settings (Yang et al.) 2022 Shen et al.) 2022); (ii) meanwhile, does not require prior knowledge of the sample subpop- ulations (e.g.. for Pt) which is essential for hard thresholding algorithms.
Equation 5 further facilitates the more flexible optimization process. As we will empirically show in Table 2 despite the close relation between Equation (5) and the hard thresholding algorithms (Remark such updating strategies may be suboptimal for solving Equation
Algorithm Adaptively Weighted Augmentation Consistency (AdaWAC)
1: Input:
Training samples ((x,y)))
P augmentations {(Ar,1, A42) ~ A
maximum number of iterations T' E N, learning rates 1ja, V) > O, pretrained initialization for the pixel-wise classifier Ou E Fe. (7).
2: Initialize the sample weights Bo = 1/2 E [0,
3: for t- 1..... T do
4: Sample it ~ [n] uniformly
5: b - (Bc-
- (3.- (3
6:
bij - bij exp (ns LcE (01-1i (xi.)
7:
b b exp (ns lac (0e-1;Xi,: Ai,,, 2i,)))
8:
B. Bi-1, (3r))t) t bu// |bbll
9:
0, + 0.-1 170 (BB))a)) VoLcE (0c-1 Vs))
+ (1 - (Bo))a)) Volac (0 Ar,,, As.2))
10:
end for
Inspired by the breakthrough made by (Sagawa et al. (2020) in the distributionalll--rhuur optimiza- tion (DRO) setting where gradient updating on weights is shown to enjoy better convergence guar- antees than hard thresholding, in Algorithm 0 we introduce adaptive weighting algorithm for solving Equation (5) based on online mirror descent. In contrast to the commonly used stochas- tic gradient descent (SGD), the flexibility of online mirror descent in choosing the associated norm space not only allows gradient updates on sample weights, but also grants distinct learning dynamics to sample weights Br and model parameters O,, which leads the following convergence guarantee.
Proposition 2 (Formally in Proposition 3. proof in Appendix B. assumptions instantiated in Ex- Assume that EeE (0: (xx and Exc (0: x, Al. A2) are convex and continuous in @ for
ample
all
y) X [K]2 and A1, A2 A2; Fo- (7) C Fo is convex and compact. If there
ist
S $G and (ii) Cg.- > 0 such that 1 1= max (fcE (0; (x), lAC (0; (0; A,1, A,,2))² < for all 0 E Fø (7). B E [0, 1)%
(i) Co.- > O such that E Vot.WAC (0,13)
F.P
then with no 173 (/sr)>c8,,+t0 +2nC3..
Algorithm provides
E
max [B((0.))
iWAC (0T,
min BEF
TAC (0, S $215 (,CC,, + +nnC2.)
where 0T = E'= Og and B D
'Following the convention, we use subscription to denote the dual spaces. For instance, recalling the parameter space Fo characterized by the norm || from Section 2.1 we use ||= to denote its dual norm; while Ca.,, Ca,- upper bound the dual norms of the gradients with respect to and (9.

---

Under review as a conference paper at ICLR 2023
In addition the convergence guarantee, Algorithm| alsso demonstrates superior performance hard thresholding algorithms for segmentation problems in practice (Table 2). An intuitive expla- nation is that instead of filtering out all the label-sparse samples via hard thresholding. the adaptive weighting allows the model to learn from some sparse labels at the early epochs, while smoothly down-weighting fcr of these samples since learning sparse labels tends be easier (Remark D- With the learned model tested on mixture of label-sparse and label-dense samples, learning sparse labels at the early stage is crucial for accurate segmentation.
EXPERIMENTS
In this section, we investigate the proposed AdaWAC algorithm (Algorithm [ on different medical image segmentation tasks with different UNet-like architectures. We first demonstrate the perfor- mance improvements brought by AdaWAC in terms of sample efficiency and robustness to sub- population shift (Table D. Then, we verify the empirical advantage of AdaWAC compared to the closely related hard thresholding algorithms discussed in Remark 2 (Table 2). Our ablation study (Table further illustrates the indispensability of both sample rewcighting and consistency regular- ization, the deliberate combination of which leads to the superior performance of AdaWA
Experiment setup. We conduct experiments on two volumetric medical image segmentation tasks: abdominal CT segmentation for Synapse multi-organ dataset (Synapse and cine-MRI seg- mentation for Automated cardiac diagnosis challenge dataset (ACDC) with two UNet-like ar- chitectures: TransUNet (Chen et al.] (021) and UNet Ronneberger et al. 2015) (deferred to Ap- pendix E.1 For the main experiments with UNet in Section 5 we follow the official imple- mentation in (Chen et al. 2021) and use ERM+SGD as the baseline. We evaluate segmentations with two standard metrics- the average Dice-similarity coefficient (DSC) and average 95- 95-percentile of Hausdorff distance (HD95). Dataset and implementation details are deferred to Appendix [ Given the sensitivity of medical image semantics to perturbations, experiments only involve simple augmentations (i.e., rotation and mirroring) adapted from (Chen et al. 2021).
5.1 SEGMENTATION PERFORMANCE OF Ada WITH TRANSUNET
Segmentation on Synapse. Figure visualizes the segmentation predictions on 6 Synapse test slices given by models trained via Ada (ours) and via the baseline (ERM+SGD) with Tran- sUNet (Chen et al. 202T). We observe that AdaWAC provides more accurate predictions on the segmentation boundaries and captures small organs better than the baseline.
backgponrd
qultieetee
ridt licaey
etomarch
ground bruth
DUES
Figure/ Visualization of segmentation predictions against the ground truth (in grayscale) on Synapse. Top to bottom: ground truth, ours (AdaWAC), baseline.
at 'We release our code www.maaiw......../ooooooooooooee "https://www..r http:///wwwww.....o....gghhhhnnn http::/wwww...aaui...ssss//tttta

---

Under review as a conference paper at ICLR 2023
Visualization of Ada As shown in Figure with lce (01; y:)) (Equation 2 of label- sparse versus label-dense slices weakly separated in the early epochs, the model further learns to distinguish EeE (0:: Yz)) of labcl-sparse/labll-dessss slices during training. By contrast, fac (Oe; xs; 1,, A222) (Equatiin (3) remains mixed for all the slices in the entire training process. As result, the CE weights of label-sparse slices are much smaller than those of label-dense ones, pushing Ada to learn more image representations but less pixel classification for slices with sparse labels and learn more pixel classification for slices with dense labels.
CIDss entrapy
CE welght
consibency lose
113
Figure 3: Ece (Ogs (x,,y.) (top), CE weights B, (middle), and fac (0g; X,, A.,, A.2) (bottom) of the entire Synapse training process. The x-axis indices slices 0-2211. The y-axis enumerates epochs 0-150. Individual volumes (cases) are partitioned by black lines; while the purple lines separate slices with/without non-background pixels.
Sample efficiency and robustness. We first demonstrate the sample efficiency of AdaWAC in comparison to the baseline (ERM+SGD) when training only on different subsets of the full Synapse training set ("full" in Table [b. Specifically, (i) half-slice contains slices with even indices only in each volum (ii) half-vol consists of 9 volumes uniformly sampled from the total 18 volumes in full where different volumes tend to have distinct Çs (i.e., ratios of label-dense samples); (iii) half-s- parse takes the first half slices in each volume, most of which tend to be label-sparse (i.e. Çs are made to be small). As shown in Table [ the model trained with AdaWAC on half-slice generalizes as well as baseline model trained full, if not better. Moreover, the half-vol and half-sparse experiments illustrate the robustness of Ada AAC to subpopulation shift. Furthermore, such sample efficiency and distributional robustness of AdaWAC extend the more widely used UNet architec- ture. We defer the detailed results and discussions on UNet to Appendix E.T
Table Ada with TransUNet trained on the full Synapse and its subsets.
Training
Method
DSC r
HD95
Acrta
Gallbladder
Kidney (L)
Kidney (R)
Liver
Pancreas
Spleen
Stomach
full
baseline
75.94
0.68
32.91 + 8.80
87.16
54.70
81.04
74.37
93.99
57.34
84.25
74.66
AdaWAC
78.83
27.50
87.65
93.97
half-stice
AddWAC
baseline
77.37 0.40
74.62 0.78
31.62
29.56 :: 1.09
2: 8.37
86.89
86.14
55.96
44.23
79.09
82.15
78.46
78.63
93.50
94.34
$7.36
55.78
84.54
86.60
75.24
77.05
half-vol
baselinc
71.08
46,83
92.02
AdoWAC
73.81
0.94
35.33
0.92
84.37
48.14
80.32
77.39
93.23
52.78
83.50
70.79
half-sporse
AddWAC
baseline
41.03
31.74 4: 2.78
59.04
69.72 1: 1.37
12.32
71.27
65.71
8.33
8.33
69.14
59.46
63.09
$1.59
51.18
64.29
17.74
10.72
30.77
6.92
3.57
0.00
Comparison with hard thresholding algorithms. Table 2
illustrates the empirical advantage of
Ada C over the hard thresholding algorithms, as suggested in Remark 2 In particular, we con- sider the following hard thresholding algorithms: (i) trim-train learns only from slices with least one non-background pixel and trims the rest in each iteration on the fly; (ii) trim-ratio ranks the cross-entropy loss tce (O,; (x,, y:)) in each iteration (mini-batch) and trims samples with the low- est cross cross-entropy losses at fixed ratio the ratio of all-background slices in the full training set
SSuch sampling is equivalent to doubling the time interval between two consecutive scans or halving the scanning frequency in practice, resulting in the halving of sample size.
8

---

Under review as a conference paper at ICLR 2023
(1 - 1280 22!1 0.42); (iii) ACR further incorporates the augmentation consistency regularization di- rectly via addition of lac (0t; As,1, As,,) without reweighting; (iv) pseudo-Ada WAC simulates the sample weights B the saddle point and learns via lce (0g; Y;)) on slices with at least one non-background pixel while via tac (0,; X,, A;,1 A322) otherwise. We notice that naive incorporation of ACR brings less observable boosts to the hard-thresholding methods. Therefore, the deliberate combination via reweighting in Ada WAC is essential for performance improvement.
Table 2: AdaWAC versus hard thresholding algorithms with TransUNet Synapse.
Method
buseline
trim-train
trim-ratio
peeudo-AdaWAC
AdaWAC
+ACR
+ACR
DSC HD95 4
76.28 29.23
76.01
75.66
74.26
77.98
78.28 29.06
79.12
26.94
35.06
28.59
33.59
25.79
Segmentation ACDC.
Performance improvements granted by AdaWAC also observed
the ACDC dataset (Table
We defer detailed visualization ACDC segmentation to Appendix
Table 3: Ada with Trans trained on ACDC
Method
DSC 1
HD95
RV
Myo
TratsUNet
89.36
3.02
88.36
83.84
95.87
AdaWAC (ours)
90.41
1.29
89.50
85.78
95.95
5.2 ABLATION STUDY
On the influence of consistency regularization. To illustrate the role of consisteney reg- ularization in AdaWAC, we consider the reweight-only scenario with Asc 0 such that lac (0,; Xi; Ai,1, Ai,2) = 0 and therefore b:2j (Algorithm I line 7) remains intact. With consis- tency regularization in AdaWAC, reweighting alone brings little improvement (Table
On the influence of sample reweighting. We then investigate the effect of sample reweighting under different reweighting learning rates 7s (recall Algorithm I (i) ACR-only for 7/s (equivalent to the naive addition of Lac (0,; X,, Ar,,, Ai,2)) (ii) Ada WAC-0.00 for = 0.01, and (iii) Ada WAC-1.0 for de 1.0. As Table 4 implies, when removing reweighting from Ada WAC, augmentation consistency regularization alone improves DSC slightly from 76.28 (baseline) to 77.89 (ACR-only), whereas AdaWAC boosts DSC to 79.12 (AdaWAC-1.0) with a proper choice of 1/3.
Table 4: Ablation study of AdaWAC with TNet trained on Synapse.
Method
HD995
Aorta
Gallbladder
Kidney (1.)
Kidney (R)
Liver
Pancreas
Spleen
Stomach
buseline
76.28
29.23
87.46
55.21
82.06
77.76
94.10
54.06
85.07
74.54
reweight-only
76.68
29.24
86.15
53.98
82.96
80,28
93.42
55.86
85.29
75.49
ACR-only
77.89
31.65
87.96
54,34
81.79
80.21
94.52
60.41
88.07
75.83
AdaWAC-0.01
77.94
27.81
87.58
52.75
82.29
80.22
94,90
55.92
91.63
78.23
AdaWAC-1.0
79.12
25.79
87.23
54.94
84,58
81.69
94.62
58.29
90.63
81,01
6 DISCUSSION
In this paper, we exploit the non-uniformity in labels commonly observed in volumetric medical image segmentation via AdaWAC-a deliberate combination adaptive weighting and augmenta- tion consistency regularization. By casting the separation between sparse and dense segmentation labels as subpopulation shift the label distribution, we leverage the unsupervised consistency regularization on encoder layer outputs (of UNet architectures) as natural reference to distinguish label-sparse and label-dense samples via comparisons against the supervised average cross-entropy weighted augmentation consistency (WAC) regulariza-
losses. We formulate such comparisons tion problem and propose an adaptive weighting scheme Mateme-AdaWAte AAC for iterative and smooth sep- aration of samples from different subpopulations with convergence guarantee. Our experiments demonstrate empirical effectiveness of Ada not only in improving segmentation performance and sample efficiency but also in enhancing robustness to the subpopulation shift in labels.

---

Under review as a conference paper at ICLR 2023
REFERENCES
Guillaume Alain, Alex Lamb, Chinnadhurai Sankar, Aaron Courville, and Yoshua Bengio. Variance reduction in sgd by distributed importance sampling. arXiv preprint arXiv:1511.0648 2015.
Philip Bachman, Ouais Alsharif, and Doina Precup. Learning with pseudo-ensembles. Advances in neural information processing systems, 27:3365-3373, 2014.
Hritam Basak, Rajarshi Bhattacharya, Rukhshanda Hussain, and Agniv Chatterjee. An embarrass- ingly simple consistency regularization method for semi-supervised medical image segmentation. arXiv preprini arXiv:2202.00667, 2022.
Aharon Ben-Tal Dick den Hertog, Anja De Wacgenaere, Bertrand Melenberg, and Gijs Rennen. Ro- bust solutions of optimization problems affected by uncertain probabilities. Management Science, 59(2):341-33) 2013. ISSN 00251900 00251909, 15265501.
David Berthelot, Nicholas Carlini, lan Goodfellow, Nicolas Papernot, Avital Oliver, and Colin A Raffel. Mixmatch: A holistic approach semi-supervised learning. Advances in neural informa- fion processing systems, 32, 2019.
Dimitri Bertsekas. Convex optimization theory, volume 1. Athena Scientific, 2009.
Gerda Bortsova, Florian Dubost, Laurens Hogeweg, Ioannis Katramados, and Marleen de Bruijne. Semi-supervised medical image segmentation learning consistency under transformations. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pp- 810-818. Springer, 2019.
Hu Cao, Yueyue Wang, Joy Chen, Dongsheng Jiang, Xiaopeng Zhang, Qi Tian, and Manning Wang. Swin-unet: Unet-like pure transformer for medical image segmentation. arXiv preprint arXiv:2105.05537, 2021.
Jieneng Chen, Yongyi Lu, Qihang Yu, Xiangde Luo, Ehsan Adeli, Yan Wang, Lu, Alan I. Yuille, and Yuyin Zhou. Transunet: Transformers make strong encoders for medical image segmentation. arXiv preprint arXiv:2102.04300 2021.
John Duchi and Hongseok Namkoong. Learning models with uniform performance via distribution- ally robust optimization. arXiv preprint arXIv:1810.08750, 2018.
John Duchi, Peter Glynn, and Hongseok Namkoong. Statistics of robust optimization: A generalized empirical likelihood approach. arXiv preprint arXiv:1610.03422. 2016.
Siddharth Gopal. Adaptive sampling for sgd by exploiting side information. In Proceedings of the 33rd International Conference on International Conference on Machine Learming Volume 48, ICML'16, pp. 364-372. JMLR.org, 2016.
Guy Hacohen and Daphna Weinshall. On the power of curriculum learning in training deep net- works. In International Conference on Machine Learning, pp. 2535-2544. PMLR, 2019.
Jamie Haddock, Deanna Needell, Elizaveta Rebrova, and William Swartworth. Quantile-based Iter- ative Methods for Corrupted Systems of Linear Equations. arXiv:2009. [cs, math), Septem- ber 2020. arXiv: 2009.08089.
Pavel lakubovskii. Segmentation models pytorch. http:///sthhicooooo/e///e/e/e//e 2019.
Angelos Katharopoulos and François Fleuret. Not all samples are created equal: Deep learning with importance sampling. In International conference machine learning, pp. 2525-2534, PMLR, 2018.
Kenji Kawaguchi and Haihao Lu. Ordered sgd: A new stochastic optimization framework for em- pirical risk minimization. In International Conference Artificial Intelligence and Statistics, pp- 669-679. PMLR, 2020.
Samuli Laine and Timo Aila. Temporal ensembling for sumi-suerriieed learning. arXiv preprint 1611vv1610.02200 2016.
10

---

Under review as a conference paper at ICLR 2023
Xiaomeng Li. Lequan Yu, Hao Chen, Chi-Wing Fu, Lei Xing. and Pheng-Ann Heng. cuunffrmationncoossii consistent self-ensembling model for semisupervised medical image segmenta- tion. IEEE Transactions Neural Networks and Learning Systems, 32(2):523-5344 2020.
Ily Loshchilov and Frank Hutter. Online batch selection for faster training of neural networks. arXiv preprint 151.0435 2015.
Fausto Milletari, Nassir Navab, and Seyed-Ahmad Ahmadi. V-net: Fully convolutional neural net- works for volumetric medical image segmentation. In 2016 Fourth International Conference on 3D Vision (3DV), pp. 565-571, 2016.
Deanna Needell, Rachel Ward, and Nati Srebro. Stochastic gradient descent, weighted sampling, and the randomized kaczmarz algorithm. Advances in neural information processing systems, 27, 2014.
A. Nemirovski, A. Juditsky, G. Lan, and A. Shapiro. Robust stochastic approximation approach to stochastic programming, SIAM Journal on Optimization, 19(4):1574- 1609, 2009.
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedi- cal image segmentation. In International Conference on Medical image computing and computer- assisted intervention, pp. 234 241. Springer, 2015.
Stuart Russell and Peter Norvig. Artificial intelligence: a modern approach. malaysia, 2016.
Shiori Sagawa, Pang Wei Koh, Tatsunori B. Hashimoto, and Percy Liang. Distributionally robust neural networks. In International Conference Learning Representations, 2020.
Vatsal Shah, Xiaoxia Wu, and Sujay Sanghavi. Choosing the sample with lowest loss makes sgd robust. In Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics, volume 108 of Proceedings of Machine Learning Research, pp. 2120-2136 Online, 26-28 Aug 2020. PMLR.
Ruoqi Shen, Sebastien Bubeck, and Suriya Gunasekar. Data augmentation as feature manipulation. In International Conference on Machine Learning, pp. 19773-19808 PMLR, 2022.
Yanyao Shen and Sujay Sanghavi. Learning with bad training data via iterative trimmed loss mini- mization. In International Conference on Machine Learning, pp. 5739-5748. PMLR, 2019.
Maurice Sion. On general minimax theorems. Pacific Journal of Mathematics, 8(1):171 176, 1958.
Kihyuk Sohn, David Berthelot, Nicholas Carlini, Zizhao Zhang, Han Zhang, Colin A Raffel, Ekin Dogus Cubuk, Alexey Kurakin, and Chun-Liang Li. Fixmatch: Simplifying semi-supervised learning with consistency and confidence. Advances in neural information processing systems, 33:596-608, 2020.
Saeid Asgari Taghanaki, Kumar Abhishek, Joseph Paul Cohen, Julien Cohen-Adad, and Ghassan Hamamnh. Deep semantic segmentation of natural and medical images: A review, 2019a
Saeid Asgari Taghanaki, Yefeng Zheng, S Kevin Zhou, Bogdan Georgescu, Puneet Sharma, Daguang Xu, Dorin Comaniciu, and Ghassan Hamarneh. Combo loss: Handling input and out- put imbalance in multi-organ segmentation. Computerized Medical Imaging and Graphics, 75: 24-33, 2019b.
Yuxing Tang, Xiaosong Wang, Adam P Harrison, Le Lu, Jing Xiao, and Ronald M Summers. ditention-guieed curriculum learning for weakly supervised classification and localization of tho- racic diseases chest radiographs. In International Workshop on Machine Learning in Medical Imaging, pp. 249-258. Springer, 2018.
Jonathan G Tullis and Aaron S Benjamin. On the effectiveness self-paced learning. Journal of memory and language, 64(2):109-118, 2011.
Xi Wang, Hao Chen, Huiling Xiang, Huangjing Lin, Xi Lin, and Pheng-Am Ann Heng. Deep virtual adversarial self-training with consistency regularization for semi-supervised medical image clas- sification. Medical image analysis, 10:0020,, 2021a.

---

Under review as a conference paper at ICLR 2023
Xin Wang, Yudong Chen, and Wenwu Zhu. A survey curriculum learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021b.
Yisen Wang, Weiyang Liu, Xingjun Ma, James Bailey, Hongyuan Zha, Le Song, and Shu-Tao Xia. Iterative Learning with Open-set Noisy Labels. arXiv: arXiv:1804.000 [cs], March 2018. arXiv: 1804.00092.
Ken C. L. Wong, Mehdi Moradi, Hui Tang, and Tanveer Syeda-Mahmood. 3d segmentation with exponential logarithmic loss for highly unbalanced object sizes. In Alejandro F. Frangi, Julia A. Schnabel, Christos Davatzikos, Carlos Alberola-López. and Gabor Fichtinger (eds.), Medical Im- age Computing and Computer Assisted Intervention MICCAI 2018, pp. 612-619, Cham, 2018. Springer International Publishing. ISBN 978-3-030-00931-1
Xiaoxia Wu, Yuege Xie, Simon Shaolei Du, and Rachel Ward. Adaloss: A computationally--fficiennt and provably convergent adaptive gradient method. Proceedings of the AAAI Conference on Arti- ficial Intelligence, (1)8:6661-8699 Jun. 2022.
Shuo Yang, Yijun Dong, Rachel Ward, Inderjit S Dhillon, Sujay Sanghavi, and Qi Lei. Sample efficiency of data augmentation consistency regularization. arXiv preprint arXiv:2202. 12230, 2022.
Michael Yeung, Evis Sala, Carola-Bibiane Schönlicb, and Leonardo Rundo. Unified focal loss: Generalising dice and cross entropy- -basd losses to handle class imbalanced medical image seg- mentation. Computerized Medical Imaging and Graphics, 95:102026, 2022. ISSN 0895-6111.
Hongyi Zhang, Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk minimization. arXiv preprint 1rXiv:
2017.
Yi Zhang, Bin Zhou, Lei Chen, Yulin Wu, and Hongchao Zhou. Multi-transformation consistency regularization for semi-supervised medical image segmentation. In 202/ 4th International Con- ference on Artificial Intelligence and Big Data (ICAIBD), pp. 485-489. IEEE, 2021.
Amy Zhao, Guha Balakrishnan, Fredo Durand, John V Guttag, and Adrian V Dalca. Data augmen- tation using learndd transformations for one-shot medical image segmentation. In Proceedings of the JEFF/CVF conference on computer vision and pattern recognition, pp. 8543-8553, 2019.
Peilin Zhao and Tong Zhang. Stochastic optimization with importance sampling for regularized loss minimization. In international conference on machine learning, pp. 1-9. PMLR, 2015.
Hong-Y Zhou, Chengdi Wang, Haofeng Li, Gang Wang, Shu Zhang, Weimin Li, and Yizhou Yu. Ssmd: semi-supervised medical image detection with adaptive consistency and heterogeneous perturbation. Medical Image Analysis, 72:102117, 2021.
12


Reviews:
[{"review_id": "xtIn_wfNSnN", "reviewer": "Reviewer_VPhg", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a new method, namely adaptively weighted augmentation consistency (AdaWAC), by combining adaptive weighting and augmentation consistency regularization. It models the separation between sparse and dense labels as a subpopulation shift in the label distribution. It is demonstrated in the experiments that the segmentation performance has been improved in terms of DSC and HD95.  ", "summary_of_the_review": "The paper presents an interesting idea and good experimental results. However, the evaluation needs improvement.  ", "strength_and_weaknesses": "Strength\n+ An interesting solution to consistency regularization\n+ Comparison with hard thresholding algorithms\n+ Good study on robustness\n\nWeakness\n+ no comparison with the soft thresholding algorithm\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "tWn55oN3Ml", "reviewer": "Reviewer_QgJJ", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The manuscript proposed an adaptive data weighting algorithm for 3D medical image segmentation. The proposed algorithm dynamically adjusts weights between supervised loss and self-supervised consistency loss during model training. The weights are adjusted by the values of loss terms on-the-fly. And the self-supervised loss is based on different augmentations on the same images, and introduced as weighted augmentation consistency (WAC) regularization. Moreover, the experimental results validate the proposed algorithm on two public 3D medical image segmentation datasets. Both fully- and partially-labeled scenarios are studied in the experiments.", "summary_of_the_review": "My concerns about the manuscript are about its novelties and practical impacts. The novelties are limited because the WAC regularization is similar to the existing works like MixMatch or FixMatch and no comparison of such works were conducted. Moreover, only a single 2D backbone neural network is studied in the experiments. The results of the proposed algorithm are not compared to other state-of-the-art segmentation algorithms on the same datasets or public leaderboards.", "strength_and_weaknesses": "<Strength>\n\n•\tThe paper is well-organized and well-written.\n\n•\tThe experimental results supports the claims made in the manuscripts.\n\n•\tThe submitted content is related to the application of deep neural networks in 3D medical image segmentation, which is highly relevant to the ICLR audience.\n\n<Weakness>\n\n•\tThe novelty of the proposed algorithm is limited. The WAC regularization is similar to the existing works like MixMatch or FixMatch.\n\n•\tThe effectiveness and practical values of the proposed algorithm is unconvincing. Only a single 2D backbone neural network is \nstudied in the experiments. And the results of the proposed algorithm are not compared to other state-of-the-art segmentation algorithms on the same datasets or public leaderboards.\n\n•\tIt is lack of comparison of other semi-supervised learning algorithms in medical image segmentation.\n\n•\tDiscussion or ablation studies are not sufficient (see below).\n\n<Comments>\n\n1.\tHow do the (nearly) failure cases look like, especially the cases that the proposed algorithm cannot fix?\n\n2.\tDoes the batch size matter for the performance of Algorithm 1?\n\n3.\tWould using a different random seed in model training give similar results?\n\n4.\tIt would be necessary to conduct ablation studies on the effects of different backbone networks.\n\n5.\tHow are the losses computed in the Table 1 for the baselines with partial annotations? Are the losses calculated only on the annotated \nareas?\n\n6.\tIn Table 2, ACR is able to help improve Dice score for “trim-ratio”. Why does it hurt the Hausdorff distance a lot? Where is the improvement of Dice score from?\n\n7.\tIt would be better to evaluate the algorithm via the public testing leaderboard of the Synapse dataset.\n\n8.\tThe experiments are based on a 2D segmentation networks for 3D image segmentation tasks. It is necessary to conduct the experiments with 3D networks as well.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "R8DepWdroXe", "reviewer": "Reviewer_epcY", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes the AdaWAC as an adaptive weighting algorithm\nthat assigns label-dense samples to supervised cross-entropy loss and label-sparse\nsamples to unsupervised consistency regularization.\nThis paper also proposes a WAC regularization that uses the consistency of encoder layer outputs as a natural reference to incentivize separation between samples with sparse and dense label. ", "summary_of_the_review": "This paper proposed AdaWAC as an adaptive weighting algorithm\nthat assigns label-dense samples to supervised cross-entropy loss and label-sparse\nsamples to unsupervised consistency regularization.\nHowever, this paper is difficult to understand.", "strength_and_weaknesses": "Ads:\nThis paper proposes the AdaWAC as an adaptive weighting algorithm\nthat assigns label-dense samples to supervised cross-entropy loss and label-sparse\nsamples to unsupervised consistency regularization.\nThis paper also proposes a WAC regularization that uses the consistency of encoder layer outputs as a natural reference to incentivize separation between samples with sparse and dense labels. \nExperiments are conducted to validate the results showing the effectiveness of the proposed methods.\n\nDis:\nThe paper organization and the paper writing are poor, which makes the paper difficult to understand.\nIn general, there is a lack of explanation of the methods used in the study.\nFurthermore, an explanation of why the authors did these experiments should be provided.\nIn experimental table 1, the AdaWAC proposed in this paper is still worse than the baseline results in some situations. The authors are advised to provide some explanations and justifications. \nThe results of comparing current SOTA methods used in medical image segmentation should be presented. Now the article only shows the results of the baseline and the proposed AdaWAC. \n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject