Title: LoMix: Learnable Weighted Multi-Scale Logits Mixing for Medical Image Segmentation
Venue: NeurIPS 2025
Accepted: yes
Keywords: LoMix Supervision

Medical Image Segmentation

U-shaped Network

Combinatorial Multi-scale Fusion

Logits Mixing
Abstract: U-shaped networks output logits at multiple spatial scales, each capturing a different blend of coarse context and fine detail. Yet, training still treats these logits in isolation—either supervising only the final, highest-resolution logits or applying deep supervision with identical loss weights at every scale—without exploring mixed-scale combinations. Consequently, the decoder output misses the complementary cues that arise only when coarse and fine predictions are fused. To address this issue, we introduce LoMix (Logits Mixing), a Neural Architecture
Search (NAS)-inspired, differentiable plug-and-play module that generates new mixed-scale outputs and learns how exactly each of them should guide the training process. More precisely, LoMix mixes the multi-scale decoder logits with four lightweight fusion operators: addition, multiplication, concatenation, and attention-based weighted fusion, yielding a rich set of synthetic “mutant” maps. Every original or mutant map is given a softplus loss weight that is co-optimized with network parameters, mimicking a one-step architecture search that automatically discovers the most useful scales, mixtures, and operators. Plugging LoMix into recent U-shaped architectures (i.e., PVT-V2-B2 backbone with EMCAD decoder) on Synapse 8-organ dataset improves DICE by +4.2% over single-output supervision, +2.2% over deep supervision, and +1.5% over equally weighted additive fusion, all with zero inference overhead. When training data are scarce (e.g., one or two labeled scans, 5% of the trainset), the advantage grows to +9.23%, underscoring LoMix’s data efficiency. Across four benchmarks and diverse U-shaped networks, LoMiX improves DICE by up to +13.5% over single-output supervision, confirming that learnable weighted mixed-scale fusion generalizes broadly while remaining data efficient, fully interpretable, and overhead-free at inference. Our implementation is available at https://github.com/SLDGroup/LoMix.

LoMix: Learnable Weighted Multi-Scale Logits Mixing for Medical Image Segmentation

Abstract

U-shaped networks output logits at multiple spatial scales, each capturing a different blend of coarse context and fine detail. Yet, training still treats these logits in isolation-either supervising only the final, highest-resolution logits or applying deep supervision with identical loss weights at every scale-without exploring mixed-scale combinations. Consequently, the decoder output misses the complementary cues that arise only when coarse and fine predictions are fused. To address this issue, we introduce LoMix (Logits Mixing), a Neural Architecture Search (NAS)-inspired, differentiable plug-and-play module that generates new mixed-scale outputs and learns how exactly each of them should guide the training process. More precisely, LoMix mixes the multi-scale decoder logits with four lightweight fusion operators: addition, multiplication, concatenation, and attentionbased weighted fusion, yielding a rich set of synthetic 'mutant' maps. Every original or mutant map is given a softplus loss weight that is co-optimized with network parameters, mimicking a one-step architecture search that automatically discovers the most useful scales, mixtures, and operators. Plugging LoMix into recent U-shaped architectures (i.e., PVT-V2-B2 backbone with EMCAD decoder) on Synapse 8-organ dataset improves DICE by +4.2% over single-output supervision, +2.2% over deep supervision, and +1.5% over equally weighted additive fusion, all with zero inference overhead. When training data are scarce (e.g., one or two labeled scans, 5% of the trainset), the advantage grows to +9.23%, underscoring LoMix's data efficiency. Across four benchmarks and diverse U-shaped networks, LoMiX improves DICE by up to +13.5% over single-output supervision, confirming that learnable weighted mixed-scale fusion generalizes broadly while remaining data efficient, fully interpretable, and overhead-free at inference. Our implementation is available at https://github.com/SLDGroup/LoMix .

1 Introduction

Precise delineation of organs, tumours, and lesions underpins radiotherapy planning, volumetric assessment, and computer-aided diagnosis. State-of-the-art systems almost invariably adopt U-shaped encoder-decoder architectures such as UNet [25], UNet++ [36], Attn-UNet [19], TransUNet [3], and SwinUNet [2]. These models generate logit maps at multiple decoder resolutions: coarse maps offer global anatomical context, whereas fine maps sharpen boundaries and reveal small pathologies.

Surprisingly, the prevailing training protocols ignore most of this multi-scale richness: they either back-propagate loss only from the final logits (single-output supervision) or apply deep supervision (i.e., auxiliary losses on intermediate network outputs) with an identical loss weight for every scale [36, 8, 20, 23]. This uniform treatment presumes that each resolution is equally informative for every anatomy, an assumption contradicted by clinical practice, where minute, high-contrast structures

(e.g. pancreas, gall-bladder) lean heavily on fine-scale details, whereas large homogeneous organs (e.g. liver) profit chiefly from coarse context. The resulting mismatch leaves scale-specific clues under-exploited, and the deficiency grows when labels are scarce.

Deep supervision is also isolationist in nature as it overlooks the synergy that can emerge when coarse- and fine-grain logits are combined . Early 'mutation' methods attempt to bridge this gap by summing up the decoder logits with equal weights, with MERIT [21] being such a prominent example. However, such static fusion fixes the operator, enforces equal contributions from all scales, and demands manual retuning whenever imaging protocols, organ sizes, or data volumes change. General loss-balancing schemes developed for multi-task learning, such as uncertainty weighting [14] or GradNorm [5], do not capture the structured correlations inside a single-task, multi-scale decoder, and thus leave much of this information content untapped.

To address these limitations, we introduce LoMix (Logits Mixing) to convert passive deep supervision into an active, learnable ensemble of mixed-scale predictions. During training every pair of decoder stages is fused by four lightweight, differentiable operators: pixel-wise addition, multiplication, concatenation followed by a 1 × 1 convolution, and attention-based weighted fusion, resulting in a rich family of 'mutant' logits that explicitly blend coarse context with fine detail. Each original or mutant map is modulated by a soft-plus weight optimized jointly with the network, so the model performs a Neural Architecture Search (NAS) style selection of the most informative scales and fusion modes within the main optimization loop; that is, no extra optimizer, no validation-set grid search is needed. The added parameters are negligible and used only during training, leaving the FLOPs, latency, and memory footprint at test-time unchanged, while providing a data-driven fusion that adapts to organ size, image contrast, and label scarcity.

When integrated into a recent U-shaped network, PVT-V2-B2 backbone [34] with EMCAD decoder [24], LoMix improves the mean DICE score on Synapse 8-organ segmentation by +4.2% over single-output supervision, +2.2% over uniform deep supervision, and +1.5% over equal-weight additive fusion; of note, the gains are even larger on the harder Synapse 13-organ segmentation task. Consistent improvements on ACDC cardiac MRI and BUSI breast-tumour ultrasound across both CNN and transformer backbones confirm robustness of LoMix. Even with only 5% of training scans available, LoMix still delivers a +9.23% DICE improvement, underscoring the data efficiency.

In summary, LoMix (i) is the first framework that jointly optimizes which decoder scales to mix and how to mix them for a single-task; (ii) substitutes manual loss weighting with an automatic, interpretable, NAS-inspired weighting mechanism; (iii) introduces zero inference overhead ; and (iv) consistently improves performance across datasets, backbones, and annotation budgets. By allowing networks to learn their own multi-scale fusion strategy, LoMix offers a principled and practical advancement that puts forth a strong baseline for data-efficient medical image segmentation.

The remainder of this paper is structured as follows: Section 2 discusses related literature, Section 3 details the LoMix framework, Section 4 explains experimental evaluations, Section 5 presents several critical ablation studies, and Section 6 concludes our findings and suggests future research directions.

2 Related Work

Medical Segmentation Architectures: U-shaped encoder-decoder networks with skip connections (e.g., U-Net) are the de facto architectures in medical image segmentation [25]; they can capture fine details via multi-scale feature maps, but are limited by the locality of convolutional operations. For example, Chen et al. note that standard U-Net struggles to model long-range dependencies, motivating hybrid designs [3]. To address this, recent work has proposed transformer-based backbones for segmentation. TransUNet [3] combines a CNN encoder with a Vision Transformer to learn the global context, while the decoder recovers the spatial details. Similarly, Swin-Unet [2] uses a hierarchical Swin Transformer in both encoder and decoder, demonstrating that pure-transformer U-shaped models outperform purely convolutional ones on multi-organ tasks.

Other variants adopt the Pyramid Vision Transformer (PVT) [34] as the encoder. For instance, CASCADE [20] and G-CASCADE [22] use PVT encoders with novel attention- or graph-based decoders to progressively refine multi-scale features. Polyp-PVT [7] and SSFormer [33] also leverage PVT backbones for polyp segmentation, incorporating hand-crafted fusion modules (e.g., cascaded fusion, camouflage, and locality decoders) to combine features across scales.

More recently, EMCAD [24] introduces an efficient multi-scale convolutional attention decoder that uses depth-wise convolutions and gated attention to fuse multi-resolution features. While these transformer-based models generate rich multi-scale outputs, they typically use fixed fusion or skip-connection schemes and do not learn explicit weights for combining feature maps.

Deep Supervision and Static Multi-Scale Fusion: Deep supervision has been widely adopted to improve training of segmentation models by attaching auxiliary loss functions to intermediate layers. For example, UNet++ [36] employs nested skip pathways: intermediate decoder outputs are each supervised by ground truth to encourage multi-scale consistency. In practice, however, these supervision losses are typically combined with fixed rules (e.g., simple averaging, summation, etc.).

Likewise, multi-scale fusion in many models is static. Common approaches concatenate or sum feature maps from different depths without learning the fusion weights. For example, Polyp-PVT's [7] design includes fixed modules to merge encoder features across levels, but these components have pre-defined roles and uniform weighting. Such static fusion schemes (even when effective) do not adaptively learn which scales or channels to emphasize, leaving the relative contributions of multi-scale features unchanged during training.

Mutation-Based Training and Its Limitations: Some recent methods aim to exploit multi-scale logits through loss-level ensembling. MERIT [21] is such a notable example: it aggregates original multi-stage logits through an additive MUTATION mixing strategy. This loss aggregation ensembles logits from different scales for final training. However, the MUTATION mixes uniformly all decoder outputs (implicitly giving equal weight of 1 to each scale) using only element-wise addition and does not include learnable parameters for fusion. Similarly, EMCAD's [24] multi-scale decoding produces several parallel predictions, but these are also combined using only additive fusion when computing loss. Hence, while mutation-based training can improve robustness via implicit ensembling, it lacks a trainable mechanism to re-weight or fuse multi-scale outputs using multiple operations dynamically.

Adaptive Weighting and NAS-Inspired Fusion: Learning to balance multi-scale signals has also been studied from the perspective of multi-task learning. Notably, uncertainty-weighted loss functions [14] and GradNorm [5] automatically tune loss weights across tasks. While these methods adaptively adjust the loss terms, they are designed for distinct tasks and do not directly apply to multi-scale outputs of a single task. They also do not explicitly handle feature-level fusion. Some NAS methods have explored learnable combinations of feature maps (e.g. NAS-FPN in detection [9]), but none learns operator-level fusion for multi-scale segmentation outputs. In contrast, to the best of our knowledge, LoMix is the first approach to enable fully learnable multi-scale fusion in segmentation: it synthesizes decoder outputs combinatorially (in the style of NAS cell search [18]) and learns softplus-parameterized weights to aggregate losses. This allows LoMix to adaptively emphasize relevant scales and operations during training, going beyond fixed strategies used in prior work.

3 Method

Next, we first review the background on U-shaped architectures and multi-scale outputs, then formalize the problem, and describe each component (see Figure 1).

3.1 Background: U-shaped Networks and Multi-scale Outputs

U-shaped networks [25, 24, 10], employ an encoder-decoder design with skip connections. The encoder path repeatedly downsamples the input to capture hierarchical features (see Figure 1(a)), while the decoder path upsamples these features to reconstruct the segmentation map (see Figure 1(b)). Skip connections between corresponding encoder and decoder layers ensure that fine spatial details lost during downsampling are recovered at the output.

To further leverage multi-scale context, many networks produce outputs at multiple decoder stages. For example, deeply supervised networks generate intermediate segmentation maps ( P 1 , P 2 , P 3 ,..., P L in Figure 1) at different scales, thus providing richer gradients during training and allowing each decoder stage to capture structures at its respective scale.

In medical imaging, this multi-scale strategy is especially beneficial, since anatomical structures can vary greatly in size. UNet++ [36] is one instantiation of this idea, introducing dense nested skip

Porig = {P, P2,P3 .,PL}

Skip Connections

Ground Truth

(e) Loss

(a)Encoder

(b)Decoder

Aggregation

Stage 1

Stage 1

P

Addition

P2

Stage2

Stage 2

Concatenation

(d)NAS-inspired

Stage3

Stage 3

P3

Multiplication

Mutated

WeightLearning

Inputlmage

PL

Attention

W

Softplus

Stage L

Stage L

OriginalMulti-

WeightedFusion

Weights

scale Logits, Porig

Concatenation

connections and side-output layers that fuse information across scales. Such multi-scale outputs encourage consistency across resolutions and improve medical image segmentation accuracy.

3.2 Problem Definition: Logit Mixing (LoMix)

Let X denote an input image of size H × W and let Y ∈ { 1 , . . . , C } H × W be the ground truth with C classes. A U-shaped network with L decoder stages produces L logit maps { Z ℓ ( X ) } L ℓ =1 , where each Z ℓ ∈ R C × H ℓ × W ℓ has spatial resolution H ℓ × W ℓ . Upsampling each Z ℓ to the full resolution gives logit maps P ℓ ( X ) = σ ( Z ℓ ( X ) ) ∈ [0 , 1] C × H × W , with σ a softmax (for multi-class segmentation) or sigmoid (for binary segmentation), as appropriate. We denote the set of these original logits by P orig . LoMix also synthesizes a collection P mut of 'mutant' maps through fusion operators (addition, multiplication, concatenation, and attention-weighted fusion) in the Combinatorial Mutation Module (see Section 3.3 and Figure 1(c)), forming the combined set P = P orig ∪ P mut .

For each map P u ∈ P , we introduce a scalar α u ∈ R and convert it to a positive loss weight w u = softplus( α u ) = ln ( 1 + e α u ) > 0 by a NAS-inspired weight learning (see Section 3.4 and Figure 1(d)). Given a segmentation loss L seg ( P, Y ) (e.g. Cross-entropy + DICE), we train both the network parameters Θ and the loss-weight parameters { α u } by minimizing min Θ , { α u } ∑ P u ∈P w u L seg ( P u ( X ) , Y ) (see Section 3.5 and Figure 1(e)). Because only the final decoder output P L is used at the test time, LoMix adds zero inference overhead while adaptively learning which scales and fused combinations are most informative for robust segmentation.

3.3 Combinatorial Mutation Module (CMM)

In our LoMix framework, a U-shaped network produces L logit maps P 1 , . . . , P L ∈ R C × H × W , each at a progressively finer scale (all upsampled to a common H × W spatial size). Let P i ( p ) ∈ R C denote the logits of C classes at pixel p ∈ { 1 , . . . , H } × { 1 , . . . , W } from the i -th decoder output. The Combinatorial Mutation Module (CMM) creates additional fused logits by combining subsets of these logits under four operators. Specifically, for every non-trivial subset S ⊆ { 1 , . . . , L } with | S | ≥ 2 , we define fused logit maps P (op) S ( p ) ∈ R C (for op ∈ { add, mult, cat, awf } ) as follows:

Addition (add): We combine each subset S of the original logit maps by element-wise addition to produce fused map P ( add ) S ( p ) as in Eq. 1:

Intuitively, Addition fusion aggregates confidence from each subset of logit maps. Addition fusion will highlight regions where either decoder is confident (acting like an OR operation).

Multiplication (mult): We take the element-wise (Hadamard) product P ( mult ) S ( p ) of each subset S of the original logit maps as in Eq. 2:

Multiplication fusion provides high confidence only where all logit maps agree (analogous to an AND operation). This fusion thus focuses on the intersection of the decoders' predictions, which can enhance precision by reinforcing common correct predictions and canceling out disagreements (if either logit map is uncertain, the product lowers confidence).

Concatenation (cat): We concatenate each subset S of the original logit maps channel-wise and apply a 1 × 1 convolution to fuse them to produce fused map P (cat) S ( p ) as in Eq. 3:

where [ P i ( p ) ] i ∈ S ∈ R | S | C is the channel-wise concatenation and W S ∈ R C × ( | S | C ) is a 1 × 1 convolution weight matrix (with output dimension equal to one logit map). This operation allows the network to learn an optimal pixel-wise linear combination of each subset of logits. The 1 × 1 convolution can be viewed as automatically weighting and combining the two inputs for each output class, potentially learning to trust one decoder more in certain regions and the other decoder elsewhere, based on data.

Attention-Weighted Fusion (awf): We introduce an attention gating to adaptively mix each subset S of original logits. We first compute attention scores and normalize using Eq. 4:

Then, we take the attention-weighted sum P (wf) S ( p ) of each subset of logits as in Eq. 5:

Attention-Weighted Fusion can learn to favor the logit that is likely to be correct at each pixel of the image (for instance, one pixel might be better at fine details, another at coarse structure, so the attention gate can interpolate accordingly). It generalizes the addition fusion by allowing spatially varying weighting instead of a fixed linear mix at each pixel.

These four fusion operations help us produce new segmentation predictions from each subset of original logits without adding significant computation (each is a simple pixel-wise operation or a 1 × 1 conv). They are complementary: addition and multiplication are fixed arithmetic mixes (one expansive, other selective), while concatenation and attention are learnable mixes (one globally learned weight, another dynamically learned per-pixel weight). By supervising all of them, we expose the network to a wide variety of joint-decoder behaviors. The decoder stages are incentivized to cooperate because an error from one decoder stage can be corrected by another in a fused output, thus leading to an overall more accurate ensemble of logits. In our LoMix framework, we apply all four fusion operations to every subset of original logits. Adding all non-empty subsets of the L decoder predictions introduces only 2 L -1 -L extra logit maps (e.g., 11 for L = 4 , 26 for L = 5 ); the total number of fused (mutant) logits is 4 ( 2 L -1 -L ) . Combined with the L original logits, the overall count is upper bounded by L + 4 ( 2 L -1 -L ) , which remains tractable for typical L ≤ 5 since U-shaped networks rarely exceed five stages as in [25, 20, 24, 12].

3.4 NAS-Inspired Weight Learning

To enable the network to learn the relative importance of each logit map, we associate a trainable scalar weight with every original decoder output P i and every fused (mutated) output P ( op ) S . Concretely, let α i ∈ R be the raw parameter for output P i and α ( op ) S ∈ R the parameter for fused output P ( op ) S . We map these parameters through the Softplus function to obtain strictly positive weights as in Eq. 6:

By construction, w i > 0 and w ( op ) S > 0 for all i, op. These weights w i , w ( op ) S serve as learnable scaling factors on the loss contributions of each corresponding logit map. All parameters α i and α ( op ) S are learned jointly with the network weights via backpropagation on the overall training objective.

These design choices allow the model to automatically learn how much emphasis to place on each original and fused logit during training, in a manner reminiscent of architecture weighting in NAS but applied to losses. The learned weights are shown in Appendix A.6 of Supplementary Material.

3.5 Loss Aggregation

Each output logits map is trained with a standard segmentation loss (e.g., Cross-Entropy and DICE losses). For the ground-truth mask Y and the i -th original output P i , we define its per-output loss as in Eq. 7. Similarly, per-output loss for each fused output P ( op ) S is defined in Eq. 8:

Here, L CE is the Cross-entropy loss weighted by β and L DICE is the DICE loss weighted by γ (here, β + γ = 1 and β , γ > 0 ). The total training loss is then formed by weighting each output's loss by the corresponding learned softplus weight and summing as in Eq. 9:

Weighting the loss terms (rather than directly combining logits) provides several benefits:

First, it preserves distinct supervision for each output: each P i and P ( op ) S is individually trained and can receive gradients weighted by its own weights w . If a certain mutated logit P ( op ) S proves to be unhelpful or noisy, then the model can drive its w ( op ) toward zero, effectively ignoring its loss contribution. Conversely, if an output is beneficial, its weight can be increased to emphasize it. This dynamic loss weighting is analogous to multi-task learning schemes where uncertainty or task relevance modulates loss terms.

Second, it avoids the pitfalls of weighting logits directly: mixing logits into a single prediction would blur their individual contributions and could hinder training of underperforming branches. Instead, our weighted loss formulation (Eq. 9) allows the network to automatically focus on useful logit outputs while minimizing the impact of less informative ones, thus leading to more effective training of the ensemble of original and mutated logits.

4 Experimental Evaluation

We evaluate LoMix on several medical image segmentation datasets. Datasets, additional results and analyses including qualitative visualization are provided in the Supplementary Material .

4.1 Implementation details

Our methods are implemented and evaluated using Pytorch 1.11.0, operating on a single NVIDIA RTX A6000 GPU equipped with 48GB of RAM. We use the PVT-EMCAD-B2 as a default model [24] in our experiments with multi-scale kernels [1 × 1 , 3 × 3 , 5 × 5] and four stages unless otherwise mentioned. We consider all four operators (Addition, Multiplication, Concatenation, AttentionWeighted Fusion) with NAS-inspired learnable softplus weights in our LoMix supervision. Only the last-stage prediction from the decoder is used as the final segmentation output. Model optimization is achieved with AdamW [17] optimizer with learning rate and weight decay set to 1 e -4 .

4.2 Comparison with SOTA Methods

Synapse 8-organ Segmentation: Table 1 reports Synapse 8-organ results averaged over at least three runs, comparing single-output supervision (LL), Deep Supervision (DS) [36], MUTATION [21], and

LoMiX across six representative backbones spanning convolutional and transformer families (UNet [25], AttUNet [19], TransUNet [3], UNeXt [31], PVT-CASCADE-B2 [20], and PVT-EMCAD-B2 [24]). LoMiX consistently achieves the highest average DICE and mIoU and the lowest HD95 for all backbones, improving DICE by +3-5% on average over deep supervision (DS) and by up to +13.5% over last-layer (LL) supervision, without any inference time overhead. The gains are especially pronounced on small or challenging organs such as gallbladder and pancreas, indicating that learnable mixed-scale fusion recovers complementary fine detail and coarse context that prior supervision schemes fail to exploit. These results demonstrate that LoMiX is a unified, lightweight, and plug-and-play training module that generalizes across diverse U-shaped decoders and transformer backbones while remaining fully compatible with standard inference pipelines.

ACDC Cardiac Organ Segmentation: Table 2 reports the cardiac organ segmentation on ACDC dataset, averaging over at least three runs. We can see that across a broad set of CNN and transformer models, our LoMix-enhanced models consistently deliver superior performance. Notably, PVT-EMCAD-B2 + LoMix attains a new peak average DICE of 92.51%, surpassing its baseline PVT-EMCAD-B2 model (92.12%) and all prior methods. It achieves the highest scores on every structure: RV 91.41%, Myo 89.96%, and LV 96.15%. Even the lightweight PVT-EMCAD-B0 + LoMix improves over PVT-EMCAD-B0 (91.34% → 91.69%), matching or exceeding more complex cascaded and MERIT-based models. These gains confirm that LoMix's learnable, multi-scale fusion yields more accurate and robust cardiac contours-particularly evident in the challenging myocardium region-while maintaining identical inference complexity.

4.3 Evaluation with Limited Data

Limited Data Setup: We create subsets using 5% (1 scan), 10% (2 scans), 20% (4 scans), and 40% (7 scans) of the Synapse training set to evaluate performance under constrained supervision.

Figure 2 and Table 3 illustrate how LoMix consistently outperforms conventional single-head ('LastLayer') supervision as the amount of labeled data shrinks. In Table 3 and Figure 2(A), with 40% of

the Synapse training set, LoMix improves the mean DICE score +5.1%; at 20% data the gain is +4.8%, and when supervision drops to only 10% and 5% the improvement margins surge to > +9%. The radar plot on the right (Figure 2(B)) shows that these improvements are not confined to a single organ: LoMix raises DICE scores for every organ, with the largest boosts on small, hard-to-segment classes such as gallbladder (GB), pancreas (PC), and stomach (SM), confirming that dynamically weighting complementary scales is especially beneficial where context/details trade-offs are hardest. Hence, LoMix delivers uniform, per-organ gains and turns the U-shaped PVT-EMCAD-B2 architecture into a far more data-efficient and across-organ robust learner without any added inference cost.

5 Ablation Study

This section describes three critical ablation studies. More ablation results and analyses are provided in the Supplementary Material .

5.1 Fusion Operation Ablation

Figure 3 shows that DICE scores increase systematically as we expand the pool of fusion operators. In Figure 3(A), starting from a single operator (AWF, leftmost bar), adding Multiply or Add provides modest gains, while injecting the Concat operation provides the sharpest jump in mean DICE.

(A)AverageDICE(%）↑

(B)Organ-wise DICE(%）↑

80

+5.06

75

LoMix

KL

40%-LastLayer

70

+4.81

KR

GB

40% - LoMix

20%-Last Layer

65

+9.21

20% -LoMix

DICE (%)

60

10%-Last Layer

10% - LoMix

5%-Last Layer

55

Liver

5%-LoMix

50

+9.23

45

40

PC

SM

35

40%

20%

10%

5%

SP

Data Fraction

85.5

(A)AverageDICE(%)↑

(B)Organ-wiseDICE(%)↑

KL

85.0

AWF

84.5

KR

GB

(Mult, AWF)

DICE(%)↑

(Add,Mult)

84.0

Mult

Add

83.5

(Add,AWF)

(Add,Concat)

83.0

Liver

(Add,Mult,AWF)

(Add,Concat,AWF)

82.5

(Mult,Concat)

(Add,Mult,Concat)

82.0

Concat

AWF

(Mult, AWF)

(Add,Mult)

Mult

pp

(Add,AWF)

(Add,Concat)

(Add,Mult,AWF)

(Add,Concat,AWF)

(Mult, Concat)

(Add,Mult,Concat)

Concat

(Mult,Concat,AWF)

(Concat,AWF)

(no) x

(Mult,Concat,AWF)

PC

SM

(Concat,AWF)

LoMix (Ours)

SP

Fusion Operations

Each additional operator gives the softplus search greater freedom to discover complementary scale interactions, and the trend is strictly monotonic: the full LoMix variant that activates all four operators tops the chart at 85.07% DICE, outperforming the best three-operator setting by 0.46%. The radar plot (Figure 3(B)) shows that these improvements are broader, LoMix encloses smaller polygons for every organ, with the largest margins on hard, low-contrast structures (GB, PC, SM) while still improving saturated classes such as liver and aorta toward their performance ceiling. In short, this ablation confirms that diverse operator choice, coupled with NAS-inspired weight learning, is critical: each extra operation opens a new pathway for the model to align coarse and fine cues, and the resultant mixture consistently translates into superior, organ-robust segmentation.

5.2 Effect of NAS-Inspired Weight Learning

Figure 4 contrasts fixed vs. learnable NAS-inspired softplus loss weighting across six supervision types. In Figure 4(A), deep supervision, single-operator fusions (addition, multiplication, concatenation, attention-weighted fusion (AWF)), and LoMix benefit from learning weights online instead of keeping them equal. The absolute gain ranges from +0.21% DICE (Add) to a pronounced +0.7% for Multiply, improving LoMix to 85.07% mean DICE score without changing architecture or inference cost. The radar plot (Figure 4(B)) shows that the learned variant never hurts any organ and delivers the largest jumps on the most scale-sensitive structures: gallbladder (GB) and left kidney (KL), while improving already strong classes (liver, spleen (SP)) toward the performance ceiling. Together, the results confirm that the learnable softplus weighting is a universal add-on: it tightens every supervision strategy but realizes its full potential when paired with LoMix's rich operator pool.

5.3 Effect of LoMix on Backbone Architectures

Figure 5 demonstrates that LoMix is architecture-agnostic: whether the backbone is transformerbased (PVT-v2 [34]) or purely convolutional (ResNet [11]), replacing conventional supervision with

85.5

(A) Average DICE(%）↑

(B)Organ-wise DICE(%）↑

Deep Sup.(Fixed)

Fixed

+0.45

KL

Deep Sup.(Learned)

85.0

Learned

Addition(Fixed)

84.5

+0.31

KR

Aorta

Addition(Learned)

Multiplication (Fixed)

Multiplication (Learned)

84.0

+0.21

Concatenation(Fixed)

(%)

83.5

+0.70

Concatenation(Learned)

+0.27

AWF (Fixed)

DICE

83.0

+0.25

Liver

AWF (Learned)

LoMix (Fixed)

82.5

LoMix(Learned)

82.0

81.5

PC

SM

81.0

DeepSup.

Addition

Multiplication

ConcatenationAWF

LoMix

SP

86

+4.13

(A)Average DICE(%）↑

LastLayer

(B)Organ-wise DICE(%）↑

PVT_V2_B0 -Last Layer

PVT_V2_B0-DeepSup.

KL

PVT_V2_B0 - LoMix (Ours)

84

+3.88

+7.41

LoMix (Ours)

PVT_V2_B1 - Last Layer

82

+6.71

+11.88

KR

GB

PVT_V2_B1-Deep Sup.

PVT_V2_B1 -LoMix (Ours)

PVT_V2_B2-Last Layer

80

PVT_V2_B2-Deep Sup.

PVT_V2_B2-LoMix(Ours)

DICE (%) ↑

78

ResNetl8-Last Layer

ResNet18-Deep Sup.

76

Liver

40506070

ResNet18-LoMix (Ours)

74

72

70

PC

SM

68

PVT_V2_B2

PVT_V2_B1

PVT_V2_B0

ResNet34

ResNet18

SP

our LoMix consistently improves performance. In Figure 5(A), mean DICE score increases with supervision strength: Last Layer < Deep Sup. < LoMix for every network. The absolute gain delivered by LoMix over single-head supervision is generalizable: +7.41-11.88% for ResNet variants and +3.88-6.71% on PVT-v2 variants which shows that our LoMix supervision unlocks benefits that standard decoders leave untapped. Figure 5(B) confirms that the improvements are broader: LoMix dominates other supervisions on all eight organs, with the largest margin gains again on scale-sensitive organs such as gallbladder (GB) and pancreas (PC). Crucially, these gains come at zero inference cost, thus underscoring LoMix's practicality as a universal booster for existing segmentation backbones.

6 Conclusion

This work introduced LoMix, a plug-and-play, NAS-inspired module that unlocks the untapped value of multi-scale decoder logits by (i) generating a rich family of mixed-scale predictions through four differentiable fusion operators and (ii) learning, via softplus gating, how strongly each real or fused map should guide training. Extensive experiments on four medical-image tasks demonstrate that LoMix consistently outperforms both single-output supervision and classical deep supervision baselines while incurring no extra inference cost. Because LoMix exposes its learned weights as explicit scalars, the fusion it learns is interpretable, transferable across backbones, and easy to fine-tune, offering practitioners a principled yet practical path to high quality segmentation without architectural redesign. Our current implementation of LoMix is limited to 2D medical segmentation tasks. Future work will extend the approach to dense prediction tasks beyond segmentation.

Acknowledgments and Disclosure of Funding

This work is supported in part by the NSF grant CCF-2531882, and in part by the iMAGiNE Consortium (https://imagine.utexas.edu/).

References

Al-Dhabyani, W., Gomaa, M., Khaled, H., Fahmy, A.: Dataset of breast ultrasound images. Data in Brief 28 , 104863 (2020)

Cao, H., Wang, Y., Chen, J., Jiang, D., Zhang, X., Tian, Q., Wang, M.: Swin-unet: Unet-like pure transformer for medical image segmentation. In: Proceedings of the European Conference on Computer Vision Workshops. pp. 205-218. Springer (2022)

Chen, J., Mei, J., Li, X., Lu, Y., Yu, Q., Wei, Q., Luo, X., Xie, Y., Adeli, E., Wang, Y., et al.: Transunet: Rethinking the u-net architecture design for medical image segmentation through the lens of transformers. Medical Image Analysis p. 103280 (2024)

Chen, L.C., Papandreou, G., Kokkinos, I., Murphy, K., Yuille, A.L.: Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE Transactions on Pattern Analysis and Machine Intelligence 40 (4), 834-848 (2017)

Chen, Z., Badrinarayanan, V., Lee, C.Y., Rabinovich, A.: Gradnorm: Gradient normalization for adaptive loss balancing in deep multitask networks. In: International Conference on Machine Learning. pp. 794-803. PMLR (2018)

Codella, N., Rotemberg, V., Tschandl, P., Celebi, M.E., Dusza, S., Gutman, D., Helba, B., Kalloo, A., Liopyris, K., Marchetti, M., et al.: Skin lesion analysis toward melanoma detection 2018: A challenge hosted by the international skin imaging collaboration (isic). arXiv preprint arXiv:1902.03368 (2019)

Dong, B., Wang, W., Fan, D.P., Li, J., Fu, H., Shao, L.: Polyp-pvt: Polyp segmentation with pyramid vision transformers. CAAI Artificial Intelligence Research 2 (2023)

Fan, D.P., Ji, G.P., Zhou, T., Chen, G., Fu, H., Shen, J., Shao, L.: Pranet: Parallel reverse attention network for polyp segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. pp. 263-273. Springer (2020)

Ghiasi, G., Lin, T.Y., Le, Q.V.: Nas-fpn: Learning scalable feature pyramid architecture for object detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 7036-7045 (2019)

Hassan, R., Mondal, M.R.H., Ahamed, S.I., Mostafa, F., Rahman, M.M.: An efficient dualline decoder network with multi-scale convolutional attention for multi-organ segmentation. Biomedical Signal Processing and Control 112 , 108611 (2026)

He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 770-778 (2016)

Huang, X., Deng, Z., Li, D., Yuan, X., Fu, Y.: Missformer: An effective transformer for 2d medical image segmentation. IEEE Transactions on Medical Imaging 42 (5), 1484-1494 (2023)

Jha, D., Smedsrud, P.H., Riegler, M.A., Halvorsen, P., De Lange, T., Johansen, D., Johansen, H.D.: Kvasir-seg: A segmented polyp dataset. In: International Conference on Multimedia Modeling. pp. 451-462. Springer (2019)

Kendall, A., Gal, Y., Cipolla, R.: Multi-task learning using uncertainty to weigh losses for scene geometry and semantics. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 7482-7491 (2018)

Kim, T., Lee, H., Kim, D.: Uacanet: Uncertainty augmented context attention for polyp segmentation. In: ACM International Conference on Multimedia. pp. 2167-2175 (2021)

Liu, Y., Zhu, H., Liu, M., Yu, H., Chen, Z., Gao, J.: Rolling-unet: Revitalizing mlp's ability to efficiently extract long-distance dependencies for medical image segmentation. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 38, pp. 3819-3827 (2024)

Loshchilov, I., Hutter, F.: Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101 (2017)

Nekrasov, V., Chen, H., Shen, C., Reid, I.: Fast neural architecture search of compact semantic segmentation models via auxiliary cells. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 9126-9135 (2019)

Oktay, O., Schlemper, J., Folgoc, L.L., Lee, M., Heinrich, M., Misawa, K., Mori, K., McDonagh, S., Hammerla, N.Y., Kainz, B., et al.: Attention u-net: Learning where to look for the pancreas. In: Medical Imaging with Deep Learning (2018)

Rahman, M.M., Marculescu, R.: Medical image segmentation via cascaded attention decoding. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. pp. 6222-6231 (2023)

Rahman, M.M., Marculescu, R.: Multi-scale hierarchical vision transformer with cascaded attention decoding for medical image segmentation. In: Medical Imaging with Deep Learning (2023)

Rahman, M.M., Marculescu, R.: G-cascade: Efficient cascaded graph convolutional decoding for 2d medical image segmentation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. pp. 7728-7737 (2024)

Rahman, M.M., Marculescu, R.: Mk-unet: Multi-kernel lightweight cnn for medical image segmentation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops. pp. 1042-1051 (2025)

Rahman, M.M., Munir, M., Marculescu, R.: Emcad: Efficient multi-scale convolutional attention decoding for medical image segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 11769-11779 (2024)

Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: International Conference on Medical Image Computing and ComputerAssisted Intervention. pp. 234-241. Springer (2015)

Roy, S., Koehler, G., Ulrich, C., Baumgartner, M., Petersen, J., Isensee, F., Jaeger, P.F., MaierHein, K.H.: Mednext: transformer-driven scaling of convnets for medical image segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. pp. 405-415. Springer (2023)

Ruan, J., Xie, M., Gao, J., Liu, T., Fu, Y.: Ege-unet: an efficient group enhanced unet for skin lesion segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. pp. 481-490. Springer (2023)

Silva, J., Histace, A., Romain, O., Dray, X., Granado, B.: Toward embedded detection of polyps in wce images for early diagnosis of colorectal cancer. Int. J. Comput. Assist. Radiol. Surg. 9 (2), 283-293 (2014)

Tajbakhsh, N., Gurudu, S.R., Liang, J.: Automated polyp detection in colonoscopy videos using shape and context information. IEEE Transactions on Medical Imaging 35 (2), 630-644 (2015)

Tang, F., Ding, J., Quan, Q., Wang, L., Ning, C., Zhou, S.K.: Cmunext: An efficient medical image segmentation network based on large kernel and skip fusion. In: IEEE International Symposium on Biomedical Imaging. pp. 1-5. IEEE (2024)

Valanarasu, J.M.J., Patel, V .M.: Unext: Mlp-based rapid medical image segmentation network. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. pp. 23-33. Springer (2022)

Wang, H., Xie, S., Lin, L., Iwamoto, Y., Han, X.H., Chen, Y.W., Tong, R.: Mixed transformer u-net for medical image segmentation. In: IEEE International Conference on Acoustics, Speech and Signal Processing. pp. 2390-2394. IEEE (2022)

Wang, J., Huang, Q., Tang, F., Meng, J., Su, J., Song, S.: Stepwise feature fusion: Local guides global. arXiv preprint arXiv:2203.03635 (2022)

Wang, W., Xie, E., Li, X., Fan, D.P., Song, K., Liang, D., Lu, T., Luo, P., Shao, L.: Pvt v2: Improved baselines with pyramid vision transformer. Computational Visual Media 8 (3), 415-424 (2022)

Wilcoxon, F.: Individual comparisons by ranking methods. In: Breakthroughs in statistics: Methodology and distribution, pp. 196-202. Springer (1992)

Zhou, Z., Rahman Siddiquee, M.M., Tajbakhsh, N., Liang, J.: Unet++: A nested u-net architecture for medical image segmentation. In: International Workshop on Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support, pp. 3-11. Springer (2018)

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: The methodology and experimental results sections justify the claims made in the abstract and introduction. The scope of the contributions is also clear in the abstract and introduction.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: The limitation is discussed in the conclusion as the current evaluations of our proposed LoMix supervision are limited to 2D medical segmentation tasks. In the future, we will extend experiments on 3D segmentation tasks.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory assumptions and proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: The paper explains the theory and ideas in the methods section. The paper explains assumptions made in the methods as well.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental result reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: The paper explains the implementation details setup in the experimental evaluation section. The paper will also publish the source code and weights upon acceptance.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: The data is publicly available online as it uses BUSI, Synapse Multi-organ (BTCV), and ACDC datasets. The citations for these datasets are included. The code will be publicly available to reproduce the results.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental setting/details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: Yes, all of the experimental setups are explained and the same setup was used to evaluate all of the models.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment statistical significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: Including error bars would take too much space. The standard deviation range of 1-3% in different datasets is mentioned.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments compute resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: The paper states that a single Nvidia RTX A6000 GPU with 48GB of RAM was used to run the experiments.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code of ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: Yes, the conducted research preserves anonymity and follows all of NeurIPS Code of Ethics.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: The proposed LoMix is an unparalleled solution for improved high-fidelity medical diagnostics. The paper discusses potential societal impacts on the medical imaging community in the abstract, introduction, and conclusion. There is no known negative societal impact of the proposed research.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: The paper poses no such risks as it focuses on fundamental research on improving medical image segmentation.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: All open source datasets and models used are properly cited and follow the license and terms of those datasets and models.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: No new assets are introduced in the paper.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and research with human subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: The paper does not involve crowdsourcing or any research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional review board (IRB) approvals or equivalent for research with human subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: The paper does not involve crowdsourcing or any research with human subjects. Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

16. Declaration of LLM usage

Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.

Answer: [NA]

Justification: The paper uses the LLM only for writing, editing, or formatting purposes.

Guidelines:

The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.

Please refer to our LLM policy ( https://neurips.cc/Conferences/2025/ LLM ) for what should or should not be described.

A Appendix / Supplementary Material

A.1 Training Algorithm

Algorithm 1 outlines the training procedure for our U-shaped segmentation network with the LoMix module. We train end-to-end with AdamW [17], updating both the usual network parameters and the loss weight parameters α . Pseudocode is given in Algorithm 1 below.

During training, the network learns to minimize the total loss by improving all logits. As training progresses, the softplus( α ) weights will adjust - for example, if the attention-fused output consistently yields lower error than others, its corresponding α awf ij may increase, giving it more influence.

In the end, we have multiple trained decoder branches and fusion modules. For inference, one can either use output of the single best-performing decoder stage or combine the decoder outputs via one of the fusion strategies (e.g., attention fusion or a simple average) to produce the final segmentation.

Thanks to our LoMix supervised training, all these outputs are optimized to be accurate and complementary. The result is a segmentation model that embodies an ensemble of experts, trained jointly in a principled manner to maximize the segmentation quality.

A.2 Advantages of LoMix

Our LoMix-enhanced segmentation has several notable advantages:

Interpretability: The learned weights softplus( α i ) and softplus( α op S ) provide insights into the importance of each decoder output and fusion type. For instance, if the addition-fusion outputs receive a high weight, it means that the simple aggregated prediction is consistently effective; if a particular logits's weight drops close to zero, then the system concludes that that particular logit is not helpful. Examining these weights can thus reveal which combinations of predictions the model finds most useful, thus offering a peek into the ensemble strategy learned by the network.

Adaptability: Because the loss weights are learned, the framework adapts to different datasets and scenarios. The model can allocate more weight to certain predictions if the data benefit from that fusion. For example, on data where one decoder stage consistently outperforms the others, the learning process can put more emphasis to that decoder stage (and possibly its fused outputs) to optimize performance. Conversely, if all decoders are needed (say each captures a different class or scale), then the weights can remain distributed.

Algorithm 1 Training with Logits Mixing (LoMix)

This adaptability alleviates the need for manual hyperparameter tuning of multi-output losses for each new application.

Training Stability and Regularization: Supervising multiple predictions (original and fused) acts as an implicit regularizer and stabilizer. Indeed, it is less likely that the network will overfit or get stuck in a poor local minimum because each decoder stage is guided by its own loss and by the fused losses that tie all decoder stages together. If one decoder begins to make mistakes, the others (and their combinations) still provide correct feedback, preventing the entire model from drifting off. Additionally, the learned weighting further stabilizes training by reducing the impact of any particularly noisy loss term: if a fused output is extremely erroneous at the start, its weight can adjust downward, preventing it from exploding the gradient. Overall, we observed that LoMix yields faster convergence and more robust training than a single-output or manually deep-supervised counterpart, thanks to these effects.

A.3 Why Softplus in NAS-inspired Weight Learning?

We choose Softplus (see Eq. 6) rather than alternatives like Softmax or explicit normalization for several reasons:

No Sum Constraint. Softmax would constrain the weights to sum to one, coupling them and forcing a distribution over outputs. This would prevent the model from independently suppressing a noisy output (as reducing one weight necessitates increasing others). In contrast, Softplus outputs are independent and unbounded, so each weight can shrink toward zero (effectively ignoring that logit) or grow arbitrarily without affecting the sum of other weights.

Strict Positivity. Softplus guarantees α > 0 smoothly, unlike ReLU which could produce exact zeros or sigmoid which would bound weights in (0 , 1) . Positive weights ensure each loss term contributes non-negatively.

Smooth Gradients. Softplus is smooth and has non-vanishing derivatives for all inputs, which stabilizes learning of the weight parameters. A hard normalization or clipping could yield zero gradients for some ranges.

A.4 Datasets

We evaluate the LoMix's efficacy across seven datasets covering six segmentation tasks. Our two multi-class segmentation datasets are Synapse Multi-organs 1 and ACDC cardiac organs 2 . The Synapse multi-organ dataset is used for abdominal organ segmentation and includes 30 abdominal CT scans with 3,779 axial slices of 512 × 512 pixels. Following the TransUNet [3], 18 scans (2,212 slices) are used for training and 12 for validation/testing. We segment eight organs: aorta, gallbladder, left kidney, right kidney, liver, pancreas, spleen, and stomach. For cardiac organ segmentation, the ACDC dataset contains 100 cardiac MRI scans segmented into three sub-organs: right ventricle (RV), myocardium (MYO), and left ventricle (LV). We follow the TransUNet protocol using 70 cases (1,930 slices) for training, 10 for validation, and 20 for testing. Our binary breast cancer segmentation dataset, BUSI [1] contains 647 images: 437 benign and 210 malignant. Our skin lesion segmentation dataset. Our three polyp segmentation datasets are Kvasir [13] (1,000 images), ClinicDB [3] (612 images), CVC-ColonDB [29] (379 images), and ETIS-LaribPolypDB [28] (196 images). Furthermore, we use ISIC2018 [6] (2,594 images) for skin lesion segmentation. We use 80% of the data for training, 10% for validation, and 10% for testing in BUSI, Kvasir, CVC-ColonDB, ETIS-LaribPolypDB, and ISIC2018 datasets.

A.5 Dataset Specific Implementation Details

For multi-class segmentation in Synapse Multi-organs and ACDC datasets, we use an input size of 224 × 224 , and optimize the combined Cross-entropy ( β =0.3) + DICE ( γ =0.7) loss. We train models for 300 and 400 epochs with a batch size of 6 and 12 for Synapse and ACDC datasets, respectively.

(a) Linear scale

(b) Log

10

scale

Operation-wiseSumofSoftplusWeightsperEpoch

Operation-wiseSumofSoftplusWeightsperEpoch(logscale)

1

Original

101

1

Original

Add

(Log-Scale)

Add

Mult

Mult

Sum of Weights

AWF

AWF

6

Concat

100

Concat

fWeights

5

4

10-1

Operation-wise

Sum

3

Operation-wise

10-2

2

1

10-3

0

0

50

100

150

200

250

300

0

50

100

150

200

250

300

Epoch

Epoch

orig_(1) -

(0)6,o

Evolution of Original + Synthesized Softplus Weights Across Epochs

oria,(0)

(1）6,

Log-Normalized Softplus Weights Across Epochs

orig_(2) -

orig_(2) -

d,(0.2)

("0)°pe

orig_(3)

0.6

ad,(0.3 -

add_(0.1) -

add_(0.2)

orig,(3)

0,(0.2)-

ad_,(2.3) -

ad,(1.2) -

add_(0.1.2)

add_(1.3) -

add,(2.3) -

add_(1.2)

10

ad0,(1.2.3)

("0)²ppe

0.5

(e"）°pe

ad,(1.2.3)

mut,(0.1) -

mst,(0.1) -

()rw

(zo)anu

mul,(0.3 -

mur,_10.3 -

mul,(12)-

mul,(1.2)

mul,(1.3) -

mul_,(2.3) -

0.4

mult_(2.3) -

(e't)”w

aqen 1yf9

(2t)p

ufgm srydpos

mul_,(1.2,3)

mul_,(0.1.3)

mul_(0.2.3

10

sttyosj6ol

af_(0.1)

0.3

aM_(0.1 -

w_.10.2)

aw_(0.3)

amf_(1.2)

w_(1.2)

amf_(1.3)

w.(1.3) -

amf_(2.3)

aw_(2.3)

0.2

af_,(0.1.3)

aaf_(0.1.2)

10-3

af_(1.2.3)

f,(1.2,3) -

concat_(0.1) -

concat,(0.2) -

(z0)00)

(E0)0)

concar,(1.2) -

())

concat_(1.3)

concat,(2,3) -

2t')

“t')e30

()0)

concat_(0.2,3)

concat_(1.2.3)

(e''t')0o

Epoch

poci

(a) Linear scale

(b) Log

10

scale

The image dimensions are set to 256 × 256 pixels for the BUSI and ISIC2018 datasets, while the image dimensions are set to 352 × 352 pixels for the polyp datasets (Kvasir, CVC-ColonDB, ETISLaribPolypDB), respectively. We utilize a multi-scale training approach, with scales of {0.75, 1.0, 1.25} and no augmentation. We use a hybrid weighted BinaryCrossEntropy (BCE) with a weighted Intersection over Union (IoU) loss (1:1) and train models for 200 epochs with batches of 16 for BUSI, ISIC2018, Kvasir, CVC-ColonDB, and ETIS-LaribPolypDB. We employ random rotation and flipping as data augmentation methods in all of our experiments except the BUSI dataset. We save the best model based on validation DICE score in all datasets and report DICE score on testsets except the Synapse Multi-organ dataset. Only the last stage prediction is chosen as final segmentation output for Synapse Multi-organ and ACDC datasets, while the predictions from all four stages are summed together to produce the final segmentation map in the BUSI, ISIC 2018, Kvasir, CVC-ColonDB, and ETIS-LaribPolypDB datasets.

(a) Linear scale

(b) Log

scale

OriginalSoftplusWeightsperEpoch

OriginalSoftplusWeightsperEpoch

0.7

orig_{0}

orig_{2}

0.0

orig_{0}

1

orig_{2}

orig_{1}

orig_{3}

0.5

orig_{1}

orig_{3}

0.6

10)

g10)

1.0

0.5

601)

501)

1.5

Weight

0.4

Wei

g

-2.0

0.3

-2.5

oftplus

snic

0.2

Softp

-3.0

3.5

0.1

-4.0

0.0

0

50

100

150

200

250

300

0

50

100

150

200

250

300

Epoch

Epoch

10

(a) Linear scale

(b) Log

scale

AddSyntheticSoftplusWeightsperEpoch

AddSyntheticSoftplusWeightsperEpoch

0.7

0.0

add_{0,1}

add_{0,1,2}

add_{0,1}

add_{0,1,2}

add_{0,2}

add_{0,1,3}

-0.5

add_{0,2}

+

add_{0,1,3}

0.6

add_{0,3}

add_{0,2,3}

add_{0,3}

add_{0,2,3}

g10)

add_{1,2}

add_{1,2,3}

g10)

-1.0

add_{1,2}

add_{1,2,3}

[601)

0.5

add_{0,1,2,3}

[601)

add_{0,1,2,3}

add_{2,3}

1.5

add_{2,3}

Weight

0.4

-2.0

Wei

0.3

Softplus

-2.5

0.2

-3.0

0.1

-3.5

0.0

4.0

0

50

100

150

200

250

300

0

50

100

150

200

250

300

Epoch

Epoch

10

A.6 Weight Dynamics and Interpretability

Figure S.1 shows that LoMix reallocates supervision away from the four original logits toward the far larger pool of fused logits. By epoch 50, the original logits hold < 10% of the total weight, while the learnable fusions, especially attention-weighted fusion (AWF) and multiplication (Mult) remain dominant for the next 250 epochs. Addition and concatenation also preserve non-trivial weight, illustrating that every operator family contributes useful gradients. The log-scale panel highlights near-perfect exponential decay for all groups, with parallel slopes indicating that LoMix balances them proportionally rather than suppressing any single operator outright. Together, these curves confirm that the NAS-inspired optimization discovers a nuanced, multi-operator loss distribution in which learnable fusions drive most of the training signal, yet fixed arithmetic fusions and even the original heads are still retained to provide complementary guidance.

Figure S.2 visualizes the softplus loss weights that LoMix learns for all 48 supervised logits (4 originals + 44 fusions) over 300 epochs. In the linear-scale map (Figure S.2a), the vast majority of weights collapse to closer to 0 after 50 epochs, while a small cluster-mainly the finest (last layer) original logit and a few mutants created by AWF , Add , Mul , and Concat -retain higher weights. The log-scale view (Figure S.2b) makes the hierarchy clearer: the finest grain and attention-weighted fusion (awf) maps dominates over other original or synthetic maps. Thus, the differentiable search automatically reduces 48 supervision maps down to a compact, high-impact subset while discarding redundant logits.

(a) Linear scale

(b) Log

scale

MulSyntheticSoftplusWeightsperEpoch

MulSyntheticSoftplusWeightsperEpoch

0.7

0.0

mult_{0,1}

mult_{0,1,2}

一

mult_{0,1}

mult_{0,1,2}

mult_{0,2}

mult_{0,1,3}

-0.5

mult_{0,2}

mult_{0,1,3}

0.6

mult_{0,3}

mult_{0,2,3}

mult_{0,3}

mult_{0,2,3}

g10)

mult_{1,2}

mult_{1,2,3}

g10)

-1.0

mult_{1,2}

mult_{1,2,3}

601)

0.5

mult_{1,3}

mult_{0,1,2,3}

[601)

mult_{1,3}

mult_{0,1,2,3}

mult_{2,3}

1.5

mult_{2,3}

Weight

0.4

-2.0

Wei

0.3

oftplus

Softplus

-2.5

0.2

-3.0

S

0.1

3.5

0.0

4.0

0

50

100

150

200

250

300

0

50

100

150

200

250

300

Epoch

Epoch

10

(a) Linear scale

(b) Log

10

scale

ConcatSyntheticSoftplusWeightsperEpoch

ConcatSyntheticSoftplusWeightsperEpoch

0.7

0.0

concat_{0,1}

concat_{0,1,2}

concat_{0,1}

concat_{0,1,2}

0.6

concat_{0,2}

concat_{0,3}

concat_{0,1,3}

concat_{0,2,3}

-0.5

concat_{0,3}

concat_{0,2}

concat_{0,2,3}

concat_{0,1,3}

g10)

concat_{1,2}

concat_{1,2,3}

(01601)

-1.0

concat_{1,2}

concat_{1,2,3}

[601)

0.5

concat_{1,3}

concat_{0,1,2,3}

concat_{1,3}

concat_{0,1,2,3}

concat_{2,3}

-1.5

concat_{2,3}

Weight

0.4

Weight(

2.0

Softplus

0.3

Softplus

2.5

0.2

-3.0

3.5

0.1

4.0

0.0

0

50

100

150

200

250

300

0

50

100

150

200

250

300

Epoch

Epoch

Original. Figure S.3 shows how LoMix optimizes the weights to four original decoder outputs during training. Both linear-scale (Figure S.3a)and log-scale (Figure S.3b) reveal that the fine-scale heads ( orig _ 2 / orig 3 ) retaining more influence than the coarser ones during all training epochs. This trend mirrors the full heatmap in Figure S.2: the NAS-inspired optimization quickly reallocates the supervision to a handful of high-utility fused logits, allowing LoMix to concentrate the gradient signal where it is most beneficial.

Addition (Add). Figure S.4 shows that all additive subsets decay smoothly, with the full 4-logit sum ( add 0 , 1 , 2 , 3 ) holding the largest weight throughout training. This indicates that the optimizer values a coarse, resolution-agnostic blending of logits for global consistency even late into training.

Multiplication (Mult). Figure S.5 demonstrates that multiplicative subsets start at the same magnitude as Add, leaving the 3- and 4-branch products dominant after 150 epochs. This pattern suggests that the optimizer relies on multiplicative interactions primarily when multiple scales jointly agree, using them as a selective gating rather than an expansive fusion.

Concatenation (Concat). All Concat curves in Figure S.6 lie almost on top of each other, revealing that concatenation quickly learns to down-weight raw channel stacks. The uniform and steady decay implies that concatenation contributes mainly in the early epochs to speed convergence, with its influence reduces as finer operators take over.

(a) Linear scale

(b) Log

scale

WFSyntheticSoftplusWeightsperEpoch

WFSyntheticSoftplusWeightsperEpoch

0.7

0.0

awf_{0,1}

awf_{0,1,2}

awf_{0,1}

11

awf_{0,1,2}

awf_{0,2}

f

awf_{0,1,3}

-0.5

awf_{0,2}

awf_{0,1,3}

0.6

awf_{0,3}

awf_{0,2,3}

awf_{0,3}

awf_{0,2,3}

g10)

awf_{1,2}

+

awf_{1,2,3}

g10)

-1.0

awf_{1,2}

awf_{1,2,3}

[601)

0.5

awf_{1,3}

awf_{0,1,2,3}

[601)

awf_{1,3}

awf_{0,1,2,3}

awf_{2,3}

-1.5

awf_{2,3}

Weight

0.4

-2.0

Wei

0.3

oftplus

Softplus

2.5

0.2

-3.0

0.1

3.5

0.0

-4.0

0

50

100

150

200

250

300

0

50

100

150

200

250

300

Epoch

Epoch

10

Attention-Weighted Fusion (AWF). Figure S.7 shows that AWF subsets preserve comparatively higher weights for longer-especially the full 4-logit fusion-remaining above other operators. This persistence shows the optimizer's strong preference for spatially adaptive weighting, thus confirming AWF's key role in exploiting complementary decoder resolutions throughout training.

A.7 Learned Loss Weights Comparison of the Best Epoch

At convergence (Figure S.8), the NAS-inspired optimizer concentrates most of the loss weight on attention-weighted fusion (AWF) combinations, followed by a smaller but still meaningful allocation to multiplicative and additive mixes, while concatenation paths receive almost negligible weight. Notably, among the original logits only the fine grain stages (2, 3) receive higher weights, confirming that the network relies primarily on multi-scale mixtures rather than any single head. This distribution echoes our ablation study: the model learns to emphasize operators that can reconcile complementary spatial cues (AWF, Mult, Add), while down-weighting logits that add parameters without clear synergy (Concat), thereby producing the highest DICE performance without manual tuning.

Softplus Weights by Combination at Last Epoch

{0}

{1}

{2}

{3}

add_{0,1}

add_{0,2}

add_{0,3}

add_{1,2}

add_{2,3}

add_{0,1,2}

add_{0,1,3}

add_{0,2,3}

add_{1,2,3}

add_{0,1,2,3}

mult_{0,1}

mult_{0,2}

mult_{0,3}

mult_{1,2}

mult_{1,3}

mult_{2,3}

mult_{0,1,2}

mult_{0,1,3}

mult_{0,2,3}

mult_{1,2,3}

mult_{0,1,2,3}

awf_{0,1}

awf_{0,2}

awf_{0,3}

awf_{1,2}

awf_{1,3}

awf_{2,3}

awf_{0,1,2}

awf_{0,1,3}

awf_{0,2,3}

awf_{1,2,3}

awf_{0,1,2,3}

concat_{0,1}

concat_{0,2}

concat_{0,3}

Group

concat_{1,2}

concat_{1,3}

Original

concat_{2,3}

Add

concat_{0,1,2}

Mult

concat_{0,1,3}

concat_{0,2,3}

AWF

concat_{1,2,3}

Concat

concat_{0,1,2,3}

0000000000000000000000000000000000000000000000

SoftplusWeight

A.8 Results of Synapse 8-organ Segmentation

Table S1 shows that LoMix achieves clearly superior multi-organ segmentation on Synapse compared to prior CNN- and transformer-based methods. In particular, our LoMix variants attain the highest average DICE, lowest HD95, and highest mIoU of any approach. For example, the best LoMix model (85.07%) outperforms PVT-EMCAD-B2 (83.63%), while also exceeding TransCASCADE (82.68%) and MISSFormer (81.96%). The gains are especially large on small, difficult organs: e.g. LoMix

significantly improves pancreas and gallbladder DICE versus previous models. Moreover, integrating LoMix into lightweight PVTv2 backbones yields consistent boosts: the PVT-EMCAD-B0 + LoMix (82.60%) exceeds the DICE of PVT-B0-EMCAD (81.97%), and similarly the PVT-EMCAD-B2 + LoMix (85.07%) outperforms the PVT-EMCAD-B2 baseline (83.63%). Crucially, LoMix is applied only during training (no extra inference cost), yet consistently pushes the SOTA across all key metrics. The reason behind the performance gain is that LoMix's adaptive multi-scale logit fusion produces more accurate and robust abdominal organ segmentation, particularly for small, challenging structures such as the pancreas and gallbladder.

A.9 Results of Synapse 13-organ Segmentation

Table S2 shows that integrating LoMix into the PVT-EMCAD-B2 network provides the strongest 13organ performance reported to date on Synapse. LoMix improves the mean DICE to 76.90%: a gain of +9.8% over single-head supervision and +6.5% over uniform deep supervision, while simultaneously reducing HD95 from 15.56 to 12.42. More importantly, improvements are not limited to one or two easy structures. Indeed, LoMix achieves the best DICE in 10 of 13 organs, including difficult small-volume classes such as the gallbladder (+4.3%), esophagus (+3.1%), and portal and splenic veins (+3.8%). It also revives the previously 'dead' adrenal-gland predictions, pushing DICE from 0 to 51-54%. Larger, context-driven organs such as liver and spleen see further improvement, and aorta performance remains on par with the best prior result. These gains confirm that LoMix's learnable, mixed-scale supervision improves both boundary-sensitive and context-dependent structures, thus delivering a uniformly stronger and more anatomically faithful segmentation without introducing any inference-time overhead.

A.10 Results of Breast Cancer and Skin Lesion Segmentation

Table S3 shows the evaluation of ultrasound breast-tumour (BUSI) and dermoscopic skin lesion (ISIC2018) benchmarks and again demonstrates that LoMix can improve the DICE score of efficient networks without increasing their computations. When integrated onto the PVT-EMCAD-B2 network, LoMix improves DICE scores by +1.07% on BUSI and +0.22% on ISIC 2018, surpassing heavyweight designs such as DeepLabv3+ and SSFormer-L. In all cases, the gains do not require architectural changes at the test time, confirming that LoMix's learnable mixed-scale supervision translates into tangible DICE score improvements, even for small, noise-prone medical datasets, without compromising the compactness or efficiency of the underlying model.

A.11 Results of Polyp Segmentation

Table S4 benchmarks LoMix on three challenging colon-polyp datasets against different CNN, transformer, and lightweight hybrid methods. LoMix improves mean DICE of the PVT-EMCAD-B0 network by +0.4-1.6% on Kvasir, CVC-ColonDB, and ETIS-LaribPolypDB, thus outperforming substantially larger models such as DeepLabv3+ and SSFormer-L. Coupling LoMix with the PVTEMCAD-B2 establishes a new SOTA on all three datasets, thus surpassing the best baseline of PVT-EMCAD-B2 by up to +1.7% while matching its inference parameter count and runtime. Gains are achieved without modifying the network at inference, confirming that LoMix's learnable mixedscale supervision improves polyp delineation accuracy and data efficiency without sacrificing the compactness of the underlying architecture.

A.12 Effect of Input Resolution on Synapse 8-organ Segmentation

Table S5 shows that LoMix capitalizes on every pixel it is given. When the input image is enlarged from 224 × 224 the mean DICE rises from 85.07% to 85.45% while HD95 reduces by 2.7, indicating

crisper boundaries at only a modest memory cost. Doubling the image again to 512 × 512 unlocks a further leap to 87.25% DICE score, setting new highs on six of eight organs: gallbladder gains +4.54%, pancreas +7.22%, stomach +1.51%, and even large structures such as aorta and liver surpass 91.5% and 96.3% DICE, respectively. The pattern confirms that LoMix's learnable multi-scale fusion continues to integrate fine detail without over-fitting, thus scaling with resolution.

A.13 Detailed Results of Fusion Operation Ablation

This section extends Section 5.1 by listing the full numeric results in Table S6. When the search space is restricted to a single operator-e.g., only the attention-weighted fusion (AWF) or only elementwise multiplication-mean DICE remains below 83.6%. Two-operator mixtures raise performance into the mid-84% range, and every additional operator yields a further monotonic gain because the softplus search can explore a richer set of cross-scale interactions. The best three-operator recipe (Concat + Multiply + AWF) reaches 84.71% DICE, but the full LoMix variant, which activates all four operators, pushes the average to 85.07% . Organ-wise scores show that only the complete LoMix combination provides balanced gains across the entire anatomy set, confirming our claim that operator diversity-coupled with NAS-style weight learning-is essential for fully exploiting complementary coarse-to-fine cues.

A.14 Detailed Results of NAS-Inspired Weight Learning

This section extends Section 5.2 by presenting the full numbers in Table S7. Across all five supervision settings, replacing uniform loss weights with our NAS-inspired softplus weights provides a clear net win. For plain deep supervision the learned variant improves mean DICE from 82.90% to 83.17% and, more importantly, reduces HD95 by > 4 point, indicating sharper boundaries. The benefit is modest, but consistent for single-operator fusions- +0.21% DICE for Add, +0.70% for Multiply, +0.39% for Concat, and +0.25% for AWF-while simultaneously lowering or matching HD95 in every case. Crucially, when the full operator pool is active, the learned weights unlock LoMix's headroom: DICE improves from 84.62% to a new high of 85.07% and HD95 falls by over 3 point, with eight-of-eight organs improving or remaining steady, thus confirming that adaptive re-weighting helps the network emphasize whichever resolutions (and fusions) are most informative for each anatomy. Because weights are pruned scalars after training, these improvements come at zero inference cost.

A.15 Detailed Results of Different Supervision Across Backbones

This section augments Section 5.3 with the full numbers in Table S8. Across all five backbones-including three transformer variants (PVT_V2_B0/B1/B2) and two purely-convolutional networks (ResNet18/34)-our LoMix shows the supervision hierarchy: Last-Layer < Deep Supervi-

sion < LoMix . Switching from single-head training to LoMix improves the mean DICE by +6.71% on the lightweight PVT_V2_B0, by +4.13% on the large PVT_V2_B2, and by a striking +11.87% on ResNet18, while simultaneously reducing HD95 by 2.63-18.34. Organ-wise scores echo this trend: LoMix delivers the best DICE for most classes on every backbone, with the sharpest jumps on the most scale-sensitive structures (GB, PC, SM) yet still improving saturated organs such as aorta and liver towards the performance ceiling. These consistent architecture-agnostic gains confirm its value as a plug-and-play supervision method for both CNN and transformer networks.

A.16 Cross-dataset evaluation on polyp segmentation

Clinical deployment demands robustness across scanners and sites. Our evaluation has already addressed overfitting risk by spanning multiple, heterogeneous public datasets and modalities (i.e., abdominal CT, cardiac MRI, breast ultrasound, dermoscopy, colonoscopy), each collected with different protocols and devices. Yet LoMiX improves on every dataset without manual tuning, because it actually behaves like a regularizer. In fact, LoMiX implicitly ensembles diverse multi-scale logits only during training, thus reducing the chance of overfitting to dataset-specific biases.

To make this explicit, we include a cross-dataset experiment (e.g., train on one dataset/hospital, and test on another) as shown in Table S9. LoMiX produces the highest DICE on every dataset which confirm its superior ability to generalize across hospitals and acquisition devices.

A.17 3D Feasibility and Results of Different Supervision

LoMiX is dimension-agnostic: it operates on C-channel class logit maps at the loss level. Extending LoMiX to 3D is straightforward: can be done simply replacing the 2D convolutions and bilinear upsampling with 3D convolutions and trilinear upsampling, everything else remains unchanged. The fusion/weighting logic remains identical, and the cost still scales with the (small) number of decoder stages (rarely exceeding five stages). When GPU memory is low, either in 2D or 3D, standard optimizations such as gradient checkpointing or caching logits on CPU further reduce compute/memory requirements without altering the algorithm, thus keeping LoMiX practical on modest hardware (< 5 GB extra GPU memory required for backpropagation to process a 96 × 96 × 96 volume with a four-stage network and 9 output classes).

To show the feasibility of LoMiX with 3D networks, the new preliminary results of 3D MedNeXt [26] with LoMiX are reported in Table S10. Our results (in bold) demonstrate that LoMiX achieves the best average DICE and HD95 scores among all supervisions.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces LoMix (Logits Mixing), a novel training module designed to improve the performance of U-shaped networks in medical image segmentation. It assigns a learnable weight to every loss term.\n\nNumerous experiments were conducted on datasets such as Synapse Multi-organs, ACDC cardiac organs, and BUSI. They also tested their method using limited data (5%, 10%, etc.), and the experiments demonstrated promising results.\n\nMoreover, a comprehensive ablation study was performed on the fusion operation, the loss term, and the backbone. Show that every module is working.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. The paper is well written, and they conducted many experiments. \n2. I like the ablation part. They did thorough ablation studies: different fusion operations, fixed loss term vs learnable loss term, different backbones.\n\nWeaknesses:\n1. Confusing comparison, the author compares their method with UNet, SwinUNet, TransUNet, PVT-CASCADE, and others (10+ methods) for no specific reason. Since the proposed method is built upon PVT-EMCAD, the only truly relevant baseline is PVT-EMCAD, whose worst-case performance should be equivalent to PVT-EMCAD (i.e., when all irrelevant loss terms become 0). The inclusion of other methods is unnecessary, as these exact comparisons were already published in the original PVT-EMCAD paper. If you check the tables from this paper and PVT-EMCAD, they are the same. \n\n2. The novelty of this paper is limited. There are two modules introduced in this paper: multi-scale feature(logit) fusion and learnable loss weights. The two concepts are widely used in computer vision. The first module looks to me like a late layer fusion module (add, concatenation, multiplication, attention weighted fusion), and the second module is learnable loss weights. All these modules are off-the-shelf. It's hard to say this is the \"first approach to enable fully learnable multi-scale fusion in segmentation\" (line 121).\n\n3. Missing comparison: Since one of the major contributions is introducing learnable weights, they should compare their NAS-based loss weights with other dynamic loss methods. Is there any specific reason or technical barriers that stop doing such a comparison?\n\nQUESTIONS:\n1. Clarify the effectiveness of different modules: how do learnable weights compare with other dynamic loss weights methods? How are your late fusion methods different from others? My opinion would improve if the authors could differentiate their work from methods that use similar individual components in experiments.\n\n2. Carefully choose your comparison method. It is very confusing to include so many irrelevant methods. In fact, the ablation study should be the main result. And the author should write the main table in this way: UNet/UNet+other fusion methods/UNet+ours; PVT-EMCAD/PVT-EMCAD+other fusion methods/PVT-EMCAD+ours. I would improve my rating if the author could improve the main results table. \n\n3. Argue more about the novelty of the module. In my opinion, applying long-standing, off-the-shelf methods to a classical problem can hardly be considered novel research for NeurIPS; It is definitely not the \"first approach to enable fully learnable multi-scale fusion in segmentation\". Please see [1] [2], they are very classical papers in computer vision and were written nearly 8 years ago. I would suggest citing the paper in the related work and arguing more about the difference between the proposed methods and these classical methods. My rating could be improved if the author could convince me of the novelty of this paper.\n\n\n[1] Liu, Shu, et al. \"Path aggregation network for instance segmentation.\" Proceedings of the IEEE conference on computer vision and pattern recognition. 2018.\n\n[2] Zhao, Qijie, et al. \"M2det: A single-shot object detector based on multi-level feature pyramid network.\" Proceedings of the AAAI conference on artificial intelligence. Vol. 33. No. 01. 2019.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-25"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes the LoMix (Logits Mixing) method, which significantly improves the performance of medical image segmentation through a learnable multi-scale feature fusion strategy. Its core innovation lies in the introduction of four lightweight fusion operations (addition, multiplication, concatenation, and attention-weighted fusion) combined with a NAS-style weight learning mechanism to dynamically optimize the contributions of features at different scales. Experiments show that LoMix achieves significant performance gains across multiple datasets and backbone networks, and performs particularly well in data-scarce scenarios. The method does not need to increase the inference overhead, is efficient and versatile, and provides a new technical idea for medical image segmentation.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n1. A learnable multi-scale Logits mixing strategy is proposed for the first time, which dynamically combines the decoder outputs of different scales through four lightweight fusion operations to generate rich \"mutant\" feature maps.\n2. A NAS-style weight learning mechanism is introduced to automatically optimize the loss weights for different fusion methods and scales, without the need for manual parameter tuning or additional validation set search.\n3. The proposed method achieves a plug-and-play design with zero inference overhead, introducing computation only in the training phase without affecting the model inference speed or memory footprint, which is suitable for practical deployment.\n4. The proposed method provides a certain degree of generality and interpretability. It is compatible with CNN and Transformer backbone networks (e.g., PVT, ResNet), and the learned weights can intuitively explain the contributions of different scales.\n5. The effectiveness of the proposed method is fully validated in multiple sets of experiments conducted on multiple datasets. The experimental results show that the proposed method outperforms SOTA methods on several medical image segmentation tasks.\n\nWeaknesses:\n\n1. The paper does not discuss the convergence of multi-scale fusion optimization. For example, are the weights of all fusion operations eventually stable? Are there redundancies?\n2. Different image modalities (e.g., ultrasound vs. CT) may require different fusion strategies, but the paper does not discuss parameter migration or modality adaptation approaches.\n3. Medical image analysis often requires 3D segmentation, but the paper has not yet verified the scalability and capability of LoMix on 3D architectures.\n4. The experiments were based only on publicly available benchmarks, and the generalization ability of LoMix on the data from different acquisition devices or hospitals has not been tested, which may conceal the risk of overfitting.\n\nQUESTIONS:\nSee Weaknesses 1 and 2\n\nLIMITATIONS:\nSee Weaknesses 3 and 4", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors introduce LoMix, a method for enhancing 2D semantic medical image segmentation through mixed-scale fusion of decoder features. LoMix combines multi-scale decoder logits using operations such as addition, multiplication, concatenation, and attention-based fusion within a Combinatorial Mutation Module. A Neural Architecture Search (NAS)-inspired strategy co-optimizes a softplus-weighted loss alongside model parameters to adaptively select effective fusion strategies and scales. The approach is integrated into UNet-like architectures (e.g., PVT-V2-B2 with EMCAD decoder) and evaluated on Synapse and ACDC datasets, including low-data settings and extensive ablations on fusion operations, NAS weighting, and backbone choices.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths: \n1. Data-driven and adaptive fusion strategy: Unlike traditional deep supervision or fixed mutation-based fusion methods, LoMix introduces a dynamic mechanism that learns how to fuse multi-scale decoder logits during training. By leveraging a combinatorial set of operations (addition, multiplication, concatenation, attention) and jointly optimizing them through a NAS-inspired weighting scheme, LoMix adapts the fusion process to the specific characteristics of the data. This makes the method flexible to the semantic granularity and scale variance present in medical images, enabling a more informed, data-driven integration of decoder features instead of relying on manually designed fusion heuristics.\n2. Robustness in limited data regimes: Medical image segmentation often suffers from scarce annotated data due to the high cost of expert labeling. LoMix demonstrates strong performance even under constrained training data conditions. This robustness stems from its ability to dynamically prioritize the most informative scales and fusion paths, effectively regularizing the learning process and mitigating overfitting. This characteristic makes LoMix particularly practical for real-world clinical settings where annotation budgets are limited.\n\nWeaknesses\n1. While the NAS-inspired loss weighting partially addresses the supervision imbalance across the combinatorially generated fused logits, all fusion paths are still computed and backpropagated regardless of their utility. This results in significant computational overhead, especially as the number of decoder stages increases. Moreover, some fusion operators, particularly multiplicative fusion, remain sensitive to low-confidence predictions early in training, potentially introducing unstable gradients before the loss weights converge to suppress them. \n2. The authors state (line 256) that LoMix helps in segmenting small and challenging abdominal organs, yet no analysis is provided to visualize which fusion types or decoder scales contribute most to those improvements. This would strengthen the claim and help explain LoMix’s behavior. Additionally, in Table 2, the performance gain between PVT-EMCAD-B with and without LoMix appears modest, and no statistical significance analysis is presented to support the observed differences. Also, unlike Table 1 (Synapse), Table 2 omits HD95 and mIoU metrics, which would be relevant for assessing organ boundary quality and overall segmentation agreement, especially since table space permits their inclusion.\n3. Although the authors acknowledge that LoMix is designed for 2D images and they recognize the weakness, all datasets used in experiments are intrinsically 3D. This mismatch between method and data dimensionality may limit its applicability in clinical practice unless a 3D-compatible version is developed.\n\nQUESTIONS:\n1. While the NAS-inspired loss weighting helps mitigate supervision imbalance across fusion paths, all paths are still computed and backpropagated during training. Can the authors justify this design in light of the significant computational overhead?\n2. The authors claim LoMix improves small and challenging organ segmentation. Can the authors provide any analysis or visualizations showing which decoder scales or fusion types (e.g., concatenation, multiplication) are most responsible for these improvements?\n3. In Table 2, the performance differences between PVT-EMCAD-B with and without LoMix appear small. Have the authors conducted any statistical tests to confirm that these improvements are significant?\n4. Why are HD95 and mIoU not reported for the multi-organ segmentation results in Table 2, despite being included for Synapse in Table 1? Including them would allow more complete comparisons.\n\nLIMITATIONS:\nThe authors adequately addressed the limitations and potential negative societal impact of their work", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nNeural networks of UNet like shape are widely used in medical image segmentation. These networks include an encoder that processes the input image to a sequence of increasingly abstract representations with lower dimension, and a decoder that combines these representations. Classically when training these neural networks, a signal is generated only by comparing the final layer with the target labels but work on deep supervision has shown that improvements can be made by introducing such a signal for different layers in the network. This work introduces a new kind of supervision method that generates a set of “fusion maps” by up sampling different stages in the decoder and then fuse them together with various operators. The fusion maps are then weighted together and added to the loss. Importantly, the weights are learnable which allows for more flexible supervision compared to previous work.\n\nSTRENGTHS AND WEAKNESSES:\n**Quality:**\nThis work is technically sound.\n\n**Clarity:**\nThis paper is for the most part clearly written and well-structured. However, I think it would be nice to list all of the experiments as tables (some results seem to only be presented in text which makes it hard to find when jumping around in the paper).\n\n**Significance:**\nComparing and evaluating medical image segmentation methods is hard due to lack of a large consensus test suite. This has lead to that very many different papers have been written on architectural improvements of the UNet with claims of improving Dice, but few actually convince the community of improvements good enough to be worth actual implementation. I think this paper unfortunately is likely to fall into this category.\n\n**Originality:**\nFrom what I can tell the leap from [18] doesn’t seem to technically be very high. However, the work is thorough and the results show improvement over the previous results in [18].\n\nQUESTIONS:\n**Question 1:** On line 145: it looks like the number of channels in each decoder layer is constant. Is this a requirement? In that case how would a UNet architecture that increase the number of channels when getting closer to the bottle neck be compatible with the introduced extension?\n\n**Question 2:** Would it be computationally feasible to generalize the method to a 3D setting? Is there some kind of exponential increase in the number of required fusion maps that gets out of hand?\n\n**Question 3:** Are the W_S in Equation (3) normalized to 1? I find it surprising that figure S.8 in the supplementals show that addition fused logit maps get weights so much higher than the concat ones, yet still are shown to be of the most important in the ablation study. Is there an explanation to why this is the case?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-02"}