FuseUNet: A Multi-Scale Feature Fusion Method for U-like Networks
Authors: Quansong He
Xiangde Min
Kaishen Wang
Tao He
Venue: ICML 2025 poster (2025)
Keywords: Feature Fusion; Medical Image Segmentation; Ordinary Differential Equation
Primary area: applications->computer_vision

Abstract: Medical image segmentation is a critical task in computer vision, with UNet serving as a milestone architecture. The typical component of UNet family is the skip connection, however, their skip connections face two significant limitations: (1) they lack effective interaction between features at different scales, and (2) they rely on simple concatenation or addition operations, which constrain efficient information integration. While recent improvements to UNet have focused on enhancing encoder and decoder capabilities, these limitations remain overlooked. To overcome these challenges, we propose a novel multi-scale feature fusion method that reimagines the UNet decoding process as solving an initial value problem (IVP), treating skip connections as discrete nodes. By leveraging principles from the linear multistep method, we propose an adaptive ordinary differential equation method to enable effective multi-scale feature fusion. Our approach is independent of the encoder and decoder architectures, making it adaptable to various U-Net-like networks. Experiments on ACDC, KiTS2023, MSD brain tumor, and ISIC2017/2018 skin lesion segmentation datasets demonstrate improved feature utilization, reduced network parameters, and maintained high performance. The code is available at
https://github.com/nayutayuki/FuseUNet.

FuseUNet: A Multi-Scale Feature Fusion Method for U-like Networks
Quansong He Xiangde Min
Kaishen Wang 3 Tao He
Abstract
neberger et al., 2015) is milestone network architecture in this field, characterized by its symmetric encoder-decoder convolutional network structure and the skip connections for integrating features from different scales.
Medical image segmentation is a critical task in computer vision, with UNet serving as mile- stone architecture. The typical component of UNet family is the skip connection, however, their skip connections face two significant limitations: (1) they lack effective interaction between fea- tures at different scales, and (2) they rely on sim- ple concatenation or addition operations, which constrain efficient information integration. While recent improvements to UNet have focused on enhancing encoder and decoder capabilities, these limitations remain overlooked. To overcome these challenges, we propose novel multi-scale feature fusion method that reimagines the UNet decod- ing process as solving an initial value problem (IVP), treating skip connections as discrete nodes. By leveraging principles from the linear multistep method, we propose an adaptive ordinary differ- ential equation method to enable effective multi- scale feature fusion. Our approach is independent of the encoder and decoder architectures, mak- ing it adaptable to various U-Net-like networks. Experiments on ACDC, KiTS2023, MSD brain tu- mor, and ISIC2017/2018 skin lesion segmentation datasets demonstrate improved feature utilization, reduced network parameters, and maintained high performance. The code is available at https: //github. wtt www.///auuaauut///uuutteooe
The emergence of UNet laid the foundation for segmentation networks, influencing the design of many existing architec- tures. For simplicity, we refer to these networks as "U-Nets". Recently popular U-Nets improvement strategies have pri- marily focused enhancing the information processing capabilities of the encoder and decoder, such as UNETR (Hatamizadeh et al., 2021), Swin-UNet (Cao et al., 2022), and their variants like Swin-UNETH (Tang et al., 2022), MetaUNETR (Lyu et al., 2024) which incorporate Trans- former (Vaswani, 2017); VM-UNet (Ruan & Xiang. 2024), UltraLight VM-Unet (Wu et al., 2024), and LKM-UNet (Wang et al., 2024a) which incorporate Mamba (Gu & Dao, 2023); and Kolling-UNet (Liu et al., 2024), DHMF-MLP (Cheng & Wang, 2023), which improve multi-layer percep- trons (MLP). However, the skip connections in U-Nets are limited to feature fusion within the same scale, lacking in- teraction across different scales, which a clear limitation. Additionally, the feature fusion strategy U-Nets relies solely on simple addition or concatenation, which makes difficult to effectively integrate feature information.
Unfortunately, there has been little research on skip con- nections in recent years. The most recent notable studies addressing this issue are UNet++ (Zhou et al., 2020) and UNet3+ (Huang et al., 2020). UNet++ introduced many intermediate nodes and dense skip connections, using fea- ture summation integrate features from different scales. UNet3+ built UNet++ by proposing full-scale skip connection, where each decoder processes all scales of skip connections. Their experimental results demonstrate the benefits of adding multi-scale information interaction in the UNet structure. In fact, the idea behind U-Nets is quite analogous to the numerical solution of ordinary differential equation (ODE). In U-Nets, the decoder simulates func- tional relationships based multiple known encoder out- put feature nodes to obtain the corresponding results. This process is similar to computing approximate values of a function at discrete points in ODE. However, the aforemen- tioned networks still follow UNet's approach for feature fusion, which relies on simple concatenation or addition. This straightforward method is analogous to the explicit
1. Introduction
Medical image segmentation a crucial branch of computer vision, and with the development of deep learning, many excellent segmentation methods have emerged. UNet (Ron-
"qqual contribution 'College of Computer Science, Sichuan University, Chengdu, China Tonggi Hospital Tongji Medical Col- lege. Huazhong University of Science and Technology, Wuhan, China 'Department of Computer Science, University of Mary- land, College Park, America. Correspondence to: Tao He <tao_he@ scu.edu.cn..
Proceedings of the 42nd International Conference Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

FuseUNet: A Multi-Scale Feature Fusion Method for U-like Networks
Euler method (Euler, 1845) in mathematics, which has only first-order accuracy and fails to fully utilize the available fea- ture information. To better leverage known information for feature fusion, we propose using the linear multistep method (Bashforth & Adams, 1883; Moulton, 1928), a widely used mathematical tool, in this paper.
In the first direction, previous studies primarily focused on increasing connection density. ResUNet (Xiao al., 2018) and DenseUNet (Cai et al., 2020) achieved this by incorpo- rating residual and dense connections within the encoder and decoder. UNet++ (Zhou et al., 2020) introduced numer- ous intermediate nodes and adopted dense skip connections, while + (Huang et al., 2020) further enhanced UNet++ by designing more comprehensive full-scale skip connec- tions. In recent years, research in this area has received less attention due to the rise of advanced modules such as Transformer (Vaswani, 2017), shifting the focus towards the second direction.
In recent years, various discretization methods (He et al., 2024; Wang et al., 2024c; Niu et al., 2024; He et al., 2023) involving neural memory ordinary differential equations (nmODEs) (Yi, 2023) has been developed, showing some potential of the linear multistep method in image segmenta- tion tasks. These methods started by designing the decoder, enabling information interaction between adjacent stages, which helps in feature fusion. However, the discrete meth- ods they use have at most second-order accuracy, rely only on information between adjacent stages, and lack the capa- bility for multi-scale information interaction.
A representative approach in the second direction is the intro- duction of attention mechanisms. For instance, UNet (Chen al., 2024) and UNETR (Hatamizadeh al., 2021) designed encoder architectures based on Transformer, while retaining convolutional decoders. Swin-UNet (Cao et al., 2022), on the other hand, used pure Transformer approach, and Swin-UNETR (Tang et al., 2022) built on UNETR by in- troducing a sliding attention mechanism. MetaUNETR (Lyu et al., 2024) proposed the TriCruci module in an attempt to create a unified segmentation framework. Additionally, the Mamba (Gu & Dao, 2023) architecture gained attention due to its lower complexity compared to Transformer. LKM- UNet (Wang et al., 2024a) demonstrated the feasibility and effectiveness of using large Mamba kernels to achieve a large receptive field, and UltraLight VM-Unet (Wu et al., 2024) introduced a new Parallel Vision Mamba layer on top of VM-UNet (Ruan & Xiang, 2024), significantly re- ducing the number of parameters and computational load in skin lesion segmentation tasks. Furthermore, research on MLP (Liu et al., 2024; Cheng & Wang, 2023) and other mechanisms in segmentation models has been observed, al- though these are not the mainstream methods. However, all these networks share a common limitation: the lack of information interaction across different scales in their skip connections.
According to the mathematical principles of the linear mul- tistep method, we propose a novel multi-scale feature fusion method for skip connections: treating the feature informa- tion of skip connections at various scales as sequence and viewing the decoding process of U-Nets as an initial value problem (IVP). We introduce the nmODEs and adapt them to this problem by designing an adaptive high-order dis- cretization method that discretizes the process. This method processes the sequence of skip connections using discrete nmODEs and ultimately generates the segmentation map. This method is relied on skip connections and is not limited to the types of encoders and decoders, making it theoreti- cally applicable to wide range of U-Net variants.
To demonstrate the effectiveness and generalizability of the proposed method, we applied it to three mainstream U-Nets based on convolution, Transformer, and Mamba architec- tures. Experiments on 3D tasks (ACDC, KiTS2023, MSD brain tumor) as well as 2D tasks (ISIC2017 and ISIC2018 skin lesion segmentation tasks), show that the proposed method significantly improves the utilization efficiency of the information extracted by encoders, while drastically reducing network parameters and maintaining network per- formance.
2.2. Neural Ordinary Differential Equations
Ordinary differential equation (ODE) systems, a key class of dynamical systems, have been widely studied in mathemat- ies and physics. Neural ODEs (NODEs) (Chen et al., 2018) offered mathematical framework to interpret ResNet, trans- forming it from a black box into a comprehensible model by viewing neural networks as ODE representations.
2. Related Work
2.1. UNet and U-Nets
UNet (Ronneberger al., 2015), the foundation of all U- Nets, features a symmetric encoder-decoder structure with skip connections that facilitate information flow between them. Based on its structural characteristics, improvements to UNet generally follow two main directions: optimizing the connection strategy within the network architecture or introducing more efficient modules the encoders and decoders.
However, NODEs face limitations, as models using data as initial values can only learn features within the same topological space (Dupont et al., 2019). Furthermore, attrac- tors in dynamical systems are linked memory capacity (Poucet & Save, 2005; Wills et al., 2005), but conventional NODEs cannot fully exploit this capacity. To address these limitations, nmODEs (Yi, 2023) extended NODEs by en-
2

---

FuseUNet: A Multi-Scale Feature Fusion Method for U-like Networks
hancing nonlinear expression through implicit mapping and nonlinear activation functions. Unlike traditional NODEs, nmODEs treat inputs external parameters, not initial val- ues. They separate the neuron's 's function into learning and memory components: learning happens in the learning part, while the memory part maps inputs to global attractors, link- ing input space to memory space.
X1
V1
X2
Y2
Y Y X1
nmODEs have been successfully applied to various seg- mentation tasks. For example, BiFNN (Niu et al., 2024) employed bidirectional skip connections and a nonpara- metric backward path based on nmODEs, which improved image recognition performance over models such as ResNet and Vision Transformer. ImPLS-Nee (Dong et al., 2023) utilized the nonlinear representation and memory capabili- ties of nmODEs for edge-based decoding, achieving precise lung lobe segmentation. Incorporating nmODEs into UNet through simple discretization has also shown promising results in diabetic kidney (Wang et al., 2024b) and skin cancer lesion segmentation (He et al., 2024; 2023; Wang et al., 2024c). Furthermore, nmODEs have improved the robustness of medical image segmentation (Hu et al., 2023), demonstrating their versatility and effectiveness across di- verse medical segmentation challenges.
Figure I. The traditional architecture of U-Nets, the skip connec- tions only communicate information at the same scale.
method is given as follow:
Theorem 3.1. Linear Multistep Method (Bashforth &
Adams, 1883; Moulton, 1928). Given the derivative g(t) = Ftt, (t)), v(t)) yo, choose a value 5 for the size of every step along t-axis and set tn+2 tn. + i 8, the result is ap- proximations for the value of ((4) Rs ye, multistep methods use information from the previous s steps to calculate the next value:
Σ ajyh+zj 5
> W(
3. Methods
with ag = 1. The coefficients Co...., 0g-1 and bu..... be determine the method.
3.1. High-order Discrete Methods
The traditional structure of U-Nets, as shown in Fig. 1, clearly indicates that the skip connections only communi- cate information within the same stage. Consider U-Net with L stages, for example, in the classic UNet, L 5. The skip connection at the same stage of the encoder is denoted as X,, and the output of the decoder is denoted as Y, The abstracted mathematical model can be expressed as: Y, = [(,,, I)) In this formulation, only the result from the previous step Y and the features from the cur- rent step X, are used. This single-step computation method simple and intuitive, but it fails leverage information from previous stages of X, and the results obtained before Y:-1, and the accuracy of this method is limited. To fully utilize the information from the skip connections and im- prove accuracy, natural next step is to adopt a multistep approach. This would allow the network to incorporate information from both the past and current steps, provid- ing more comprehensive feature integration and ultimately enhancing performance.
In Theorem 3.1, if b, - 0, then the method is called "ex- plicit", since the formula can directly compute ya+a. If bg * 0 then the method is called "implicit", since the value of Un+ depends on the value of F(In+e: Un+s), and the equation must be solved for yn+s-
Table 1. Linear multistep method.
Explicit: Adams Bashforth (AB) Method
step
order
equation
Mas1 Mis F.
2
Boee = Va-1 + $ (3Fu41 - F.)
3
3
Mota = Va-a
(23F - 16F + +5.)
Muss Ma- 31 (55F 50P 37P 9F%)
Implicit:
(AM) Method
step
order
equation
Uoet = Ma + 4- (F.. + Fn-1)
2
3
Vota = Va-1 + (5Fo+a + 8F Fo)
Mais Ma 2 51 (9F 19F 5P.11
Increasing the number of steps (i.c., incorporating more previous time points) raises the order of the linear multistep method, thereby improving its theoretical accuracy. How- ever, it may also reduce stability, cause error accumulation, and increase dependence on initial values. Therefore, the highest order typically used for the linear multistep method is 4. For simplicity, we write F(tn, yn) as Fns the specific coefficients for the linear multistep method are shown in
The linear multistep method is classical numerical method that uses information from the current time step and several previous time steps to predict the solution at the next time step. This method aligns perfectly with the idea of fully uti- lizing the skip connection information X, and the computed values Y, from U-Nets. The theorem of linear multistep
3

---

FuseUNet:: A Multi-Scale Feature Fusion Method for U-like Networks
Table I. Besides, the explicit method is denoted AB, and the implicit method as AM, in section 3.2, where corresponds to the step number in Table 1.
Algorithm Adaptive discrete method for U-Nets
NOTE: Fix represents the sequence (Fi, ......::................... :FF.
indicating all values from F1 to F;-
When the number of steps is the same, implicit methods consistently achieve higher accuracy than explicit methods. Consequently, we strive to use implicit methods whenever possible. However, implicit methods require the derivative values the current node, which are unknown the current step. The wredictor-correcto method provides mathemati- cal solution to this problem. The corresponding theorem is presented as follows:
Input: feature map X, memory statement Y and total stage L
for = to L do
if < 4 then
Predictor: Y441 = AB,(Y, Fis)
Corrector: Y1+1 AM+(V+1, F1s+1)
else
Predictor: Y41
ABa(Y, (Y, F5-33))
Corrector: Y++1 = AMa 22-2141)
Theorem 3.2. Predictor-Correcto Method (Heun, 1900;
end if end for
Gragg& & Stetter, 1964). Consider the differential equation i(t) = (t,, (()), v(tu)= Vo, and denote the step size by 6. First, starting from the current value yo, calculate an initial guess value V64I via an explicit method. Next, improve the initial guess using corresponding implicit method. For example use 1 step methods:
if L < 5 then
Yjinal = (VL, FI.L)
else
Yj finat 11,
end if
Un+1 = Yn + 8. FF(tn,
Return Yfinal
8
Un+1 = Yn + (F(Ln Un)) + F( ((n+1,90++))) Bo+1)),
where yn = y(tn). Dn+I is the predicted median, yn+1 is the final result corrected for yn+1-
Then, we can apply the mathematical ideas of the linear multistep method to process features at multiple scales. To ensure effective interaction between features at different scales, it is crucial to incorporate as much information from previous stages as possible and employ high-accuracy im- plicit methods when deriving the output of a given decoder stage. When the current stage number is greater than 4 and not the last stage, a 4-step implicit method can be straight- forwardly chosen for
Thus, more accurate solutions can be obtained using implicit methods.
3.2. Adaptive Discrete Method for U-Nets
The linear multistep method is a mathematical tool used to solve initial value problem (IVP) in ordinary differen- tial equation (ODE). To relate the architecture of U-Nets with ODE, existing methods typically introduce differential equations starting from the decoder. Following this idca, we define the differential relationship between the skip con- nections X and the corresponding stages Y as F, but we focus on fusing multi-scale information starting from the skip connections. The function will be explained in detail in section 3.3. this section, we first deseribe the proposed discrete method and the overall framework applied in L-stage U-Net ,as shown in Fig. (a).
correction. However, the linear multistep method need to calculates the values of Yi, < i for the previous steps using a single-step method first, and then substitutes the calculated derivatives into the formula. The single-step method the initialization step is still limited to the same scale, which does not align with our goal of multi-scale feature interaction. Therefore, we need to modify the initialization step of the linear multistep method. Our approach is to repeatedly apply the lower- step linear multistep method during the startup phase of the high-step method. Specifically, when calculating I++1, we use the i-step implicit method. For i 4, the 4-step implicit method is applied until the L-th stage, where the final computation is performed using an explicit method. This algorithm is outlined in Algorithm 1, and the detailed computational process is provided in Appendix A.
First, we map the data from each stage of the U-Net to the elements of the linear multistep method: we treat the pro- cess where the decoder reconstructs the features extracted by the encoder into predicted labels as solving an IVP. X; represents series of discrete nodes, and Y, represents the corresponding solution at each node, < i < L. Yfinal is the solution to the IVP. The bottom stage represents the start of the IVP, where we initialize Y 0, representing empty memory stream. As the IVP progresses, we gradu- ally fill in information on the empty memory stream, first mapping the coarse high-level features to the feature map, and then filling in the finer low-level details.
Finally, we apply a convolution layer to map Yrinal to the segmentation map.
3.3. Differential Equations Adapted for U-Nets
nmODEs divided neuron into two parts: learning neuron and memory neuron. The input data passed to the learning

---

FuseUNet: A Multi-Scale Feature Fusion Method for U-like Networks
1
Y
Y
final
Y 1-1
conv
X 1
Y2
encoders
X2
Y1
P20
1
P-C
P-C
Predictor-Corretto
P-C
Calculator
a
(b)
(d)
Y
Corrector: AM
g(X1)
f-uuc
F11141
Final
Transformer! Mamba
X1+1
Y
interpolation
nmODFs block
Calculator: AB
xxconv
Conv/
Y1
M+1
Fiz
X
Fi
Predictor:
F
nmODEs block
x 1-
g-func
Y1
nmODEs block
Y
Yı
Figure 2. (a) The architecture of an staee U-Net incorporating discrete nmODEs. Here, P-C represents the Predictor- Corrector module, with its internal structure detailed in (b). C. denotes the calculator used in the final step, which exclusively employs explicit methods, and its internal structure is illustrated in (c). The number of channels in Y is set to twice the number of target classes, while all other dimensions remain consistent with the original input. The internal structure of the nmODEs block shown in (d), where the function executes the corresponding operations based on the specific network architecture, such as convolution, Transformer, or Mamba.
Predictor-Corrector
neuron to extract features, while the memory carrier records the extracted information in the memory neuron. This de- sign has three advantages. nmODEs inherently avoids the features learned from being isomorphic the input data space. Additionally, we have identified two other advan- tages that particularly relevant to this work:
where X, represents the external input at time 4, Y, repre- sents the memory flow carried by the memory neuron at time t, with the initial condition Y, = O representing an empty memory flow, f()) is a non-linear mapping that com- bines the existing memory and the new external input to update the memory flow using differential equation numer- ical solver, 9(-) is the function that processes the external input X, at each time step. When applying nmODEs to the discrete U-Nets, the following equation holds:
First, the separation of learning and memory neurons in nmODEs allows it to retain continuous memory flow while still maintaining the ability to process external inputs. Therefore, features extracted by the encoders from different scales can be fed into the learning neuron of nmODEs as a sequence nodes, and the memory carrier updated with the numerical solutions corresponding each node.
Y = + -X
(4)
In U-Nets, the X; and Y; in Eq. (4) correspond to those in Section 3.2. The function g(-) uses convolution and interpo- lation align the channels and shape of X; and Y. while the choice of f(-) depends on the network which this equation is applied. By substituting Eq. (3) into the linear multistep method proposed in Algorithm 1, we can obtain the approximate solution for each stage. Following this process, we designed three modules: Predictor- Predictor-Correctorr Calculator, and the nmODEs block, as shown in Fig. (b), (c), and (d), respectively.
Second, the memory carrier ultimately outputs the target data but does not participate in the feature extraction process. As a result, it does not need to retain much information other than the final output. This reduces the required number of channels significantly compared to the decoder of traditional U-Nets, thus greatly reducing computational costs.
The equation for nmODEs is given follows:
v, -Y, + /Y g(x)))
5

---

FuseUNet: A Multi-Scalc Feature Fusion Method for U-like Networks
Table 2. Overview of Used Datasets.
Dataset
Region
Data Type
Number of Samples
Target Classes
ACDC (Bernard et al., 2018)
Heart
3D MRI
100
3 (1-MYO, 2-RV33LVV))
KiTS23 (Heller et al., 2023)
Kidney
3D CT
559
3 (1 (1-Kidney, 2-Cyst, 3-Tumor)
MSD (Antonelli al., 2022)
Brain
3D MRI
484
(1-WT, 2-ET, 3-TC)
ISIC2017 (Codella et al., 2018a)
Skin
2D Dermoscopy
2000
1 (Lesion skin area)
ISIC2018 (Codella et al., 2018b)
Skin
2D Dermoscopy
2594
1 (Lesion skin area)
4. Experiments
backbone network. When using the convolutional nn-UNet as the backbone, FuseUNet achieves a 54.9% reduction in the number of parameters and a 34.3% reduction in GFL.Oss, while maintaining the performance of nn- un-UNet. Although its performance on the ACDC dataset is slightly lower, it surpasses nn- un-UNet on the larger KiTS dataset. Detailed performance data on each fold is provided in Appendix D.
The currently popular U-Nets are primarily based on three architectures: convolution, Transformer, and Mamba. To evaluate the effectiveness and generalizability of the pro- posed method, we select a representative backbone network from each of these architectures and perform experiments on the datasets used in their respective original studies. All ex- periments were conducted on a single RTX 4090. Except for the learning rate, all experimental settings were consistent with those of the backbone networks used for comparison. Due to the significant reduction in the number of parameters, the learning rate was set to 3 times that of the backbone network's setting. The detailed hyperparameter settings are provided in Appendix B.
When using the Transformer-based UNETR as the backbone network, Uus achieves a 13.6% reduction in the num- ber of parameters and a 50% reduction in GFLOPs, while surpassing UNETR's performance by 1.5%. The smaller reduction in parameters compared to nn-UNet is due to UNETR's 's parameters being primarily concentrated in the encoder's attention module, with the decoder contributing a smaller proportion. Since FuseUNet's multi-scale feature interaction method is designed around skip connections and does not modify the encoder, the reduction in the number of parameters is relatively modest.
4.1. Datasets and Backbone Networks
Table 2 presents the details of the datasets used in this paper. In the table, LV denotes the left ventricle, RV represents the right ventricle, MYO corresponds to the myocardium, EC indicates the enhancing tumor, TC refers to the tumor core, and WT denotes the whole tumor.
Fig. illustrates the visual segmentation results of Fuse- UNet, where (a), (b), and (c) correspond to the ACDC, KiTS, and MSD brain tumor datasets, respectively. In Figure (a), nn-UNet frequently makes errors in identifying the right ventricle (RV), sometimes misclassifying unrelated tissues as the RV or missing portions of it entirely. FuseUNet signif- icantly mitigates these issues. Fig. 3(b) shows that nn-UNet occasionally fails to fully recognize the kidneys and often confuses kidney tumors with cysts. This particularly evi- dent the third row, where nn-UNet incorrectly classifies most cysts as tumors, even in the absence of tumor tissue, Fuse greatly improves upon this. In Fig. 3(c), UNETR exhibits challenges such as incomplete tumor recognition and missing enhanced tumor regions, both of which are notably addressed by FuseUNet.
nn-UNet. For CNN-based U-Nets, nn-UNet (Fabian al., 2021; Isensee et al., 2024) remains the backbone due to its superior applicability, outperforming nearly all other U-Nets across reliable datasets. UNETR. In Transformer- based U-Nets, UNETR (Hatamizadeh et al., 2021), from the MONAI framework, is a key foundation for research and a suitable backhone for our study. UltraLight VM- UNet. Mamba's efficiency and linear complexity reduce costs over Transformers. UltraLight VM-UNet (Wu al., 2024) leverages this, achieving strong performance with a lightweight design. We use it to assess our method's s potential for further downsizing.
2D Tasks. For 2D tasks, we report Dice, sensitivity, speci- ficity, and accuracy as percentages, following the protocol of the backbone network. The performance data for the com- pared networks are sourced from (Wu et al., 2024). Table 4 shows the performance of the proposed method on 2D tasks, using the lightweight UltraLight VM-UNet with a Mamba architecture as the backbone network. While FuseUNet reduces the number of parameters by 29.8%, its GFLOPs have increased by 0.075, with performance comparable to that of UltraLight VM-UNet. This increase in GFL.Pss is
4.2. Main Results
3D Tasks. All 3D tasks report Dice scores (%) using five- fold cross-validation, following the protocol of the backbone network. The performance data for the compared networks are sourced from (Hatamizadeh et al., 2021; Isensee et al., 2024; Perera al., 2024). Table 3 presents the performance of the proposed method on 3D tasks. The gray- rows in the table indicate FuseUNet and its corresponding
6

---

FuseUNet: A Multi-Scale Feature Fusion Method for U-like Networks
Table 3. The performance comparison between Fuse the backbone networks, and SOTA on 3D tasks.
Dataset
Model
Params(M)
GFLOPs
Dice1
Dice2
Dice3
Dice avg
CoTT (Xie et al., 2021)
41.9
668.1
89.06
88.56
94.06
90.56
Swin-UNETR (Tang et al., 2022)
62.8
384.2
89.56
89.98
94.33
91.29
U-Mamba (Ma ct al., 2024)
173.5
1255
89.71
89.70
94.25
91.22
ACDC
STU-Net-L (Huang et al., 2023)
440.3
3810
90.02
89.59
94.32
91.31
nn-UNet (lsensee al., 2024)
31.2
402.6
90.11
89.96
94.55
91.54
FuseUNet (Ours)
14.0
264.9
90.18
90.05
94.49
91.57
nnFormer (Zhou et al., 2022)
150.1
425,8
92.27
69.78
65.53
75,86
Swin-UNETR (Tang ct al., 2022)
62.8
384.2
94.48
76.80
72.53
81.27
U-Mamba (Ma al., 2024)
173.5
1255
96.08
82.84
79.77
86.23
KiTS
STU-Net-L (Huang et al., 2023)
440.3
3810
96.11
82.35
79.09
85.85
nn-UNet (Isensee et al., 2024)
31.2
402.6
96.03
82.65
79.44
86.04
FuseUNet (Ours)
14.0
264.9
96.47
83.06
79.04
86.19
UNet (Ronneberger et al., 2015)
13.4
31.1
76.6
56.1
66.5
66.4
UNet3+ (Huang et al., 2020)
12.0
62.2
41.4
47.8
50.5
TransUNet (Chen al., 2024)
116.5
70.6
54.2
68.4
64.4
MSD
Swin-UNETR (Tang et al., 2022)
62.8
384.2
70.0
52.6
70.6
64.4
UNETR (Hatamizadeh et al., 2021)
103.7
40.3
78.9
58.5
76.
71.1
FuscUNG (Ours)
89.2
20.1
79.5
60.1
78.2
72.6
Myo
Kidney
Tumer
WT
TC
ET
IMG
nn-UNet
Fuse-UNel
IMG
GT
UNETR
Fuse-UNet
(a) Visualization on the ACDC
(b) Visualization on the KiTS
(c) Visualization on the MSD
Figure 3. The visual comparison of segmentation results between FuseUNet and the backhone networks 3D tasks.
attributed to the interpolations used by FuseUNet to align the shapes of skip connection X; and the corresponding Y. This issue is less noticeable when the backbone network has larger GFLOPs, but it becomes more pronounced in lightweight networks. However, overall, it only leads to minor increase in GFLOPs at the decimal level.
4.3. Ablation Experiments
The impact the number feature fusion steps. This experiment was conducted on the first fold of all 3D datasets and the full 2D datasets. Figure 5 illustrates the per- formance (Dice) when different maximum orders of the linear multistep method used. Since the baseline varies across datasets, we have normalized the data. As the order increases, more skip connections participate in feature infor- mation interaction. The results indicate strong correlation between the network's performance and the highest order of the applied feature interaction method.
Fig. 4 illustrates the visual segmentation results of Fuse- UNet, where (a) and (b) correspond to the ISIC2017 and ISIC2018 datasets, respectively. FuseUNet addresses the false negative and false positive issues frequently observed in UltraLight VM-UNet. Moreover, it captures lesion bound- details that are significantly closer to the ground truth.
This subsection presents only few visualization results, with additional results provided in Appendix E.
The impact of memory capacity. This experiment was conducted on the first fold of the KITS dataset. Table shows the impact of channel count in Y on performance, where N the number of target classes. Setting Y to 2N

---

FuseUNet: A Multi-Scale Feature Fusion Method for U-like Networks
Table The performance comparison between FuseUNcct, the hackhone networks and SOTA
2DD tasks.
ISIC2017
ISIC2018
Model
Params (M)
GFLOPs
Dice
SE
SP
ACC
Dice
SE
SP
ACC
UNet (Ronneherger et al., 2015)
13.4
31.12
89.89
87.93
98.12
96.13 13
88.51
87.35
97.44
95.47
TransFuse (Zhang al., 2021)
26.16
11.5
79.21
87.14
97.98
96.17
89.27
91.28
95.74
94.66
MALUNet (Ruan et al., 2022)
0.177
0.085
88.24
97.62
95.83
89.31
88.90
97.25
95.48
EGE-UNet (Ruan et al., 2023)
0.053
0.072
90.73
89.31
98.16
96.42
88.19
90.09
96.38
95.10
VM-UNet (Ruan & Xiang. 2024)
27.427
4.112
90.70
88.37
98.42
96,45
88.91
88.09
97.43
95.54
Ugtt VM-UNet (Wu al., 2024)
0.049
0.060
90.64
88.85
98.38
96.63
89.40
86.80
96.38
95.10
FuseUNet (Ours)
0.036
0.095
90.69
89.59
98.20
96.62
89.77
89.10
97.41
95.62
Impact of Order on Performance (Normalized)
1.0
Normalized Performance (0-1)
0.8-
ACDC
MSD-B
ISIC2017
0.6
ISIC2018
0.4
0.2
IMG
GT
ipht VM-UNet
Pise UUNet
IMG
GT
Ultnalaht VM-UNet
Fuse-UNet
0.0
(a) ISIC2017
(b) ISIC2018
Order
Figure The visual comparison of segmentation results between UNse and the backbone networks 2D tasks.
Figure 5. The impact of the number of feature fusion steps on network performance.
significantly outperforms N, suggesting that some redun- dancy in memory flow is beneficial. However, 3.V reduces performance, and 4N offers only a slight gain, indicating the linear multi-
strategy, it is encoder-decoder agnostic and theoretically applicable to any U-Nets. Experiments validate its effec- tiveness across convolution-, Transformer-, and Mamba- based U-Nets, reducing computational costs while preserv- ing backbone performance. Visualizations further highlight improved edge recognition and feature distinction. Our work moves beyond empirical network design, and reveals the underlying connection between skip connections and numerical integration methods. This demonstrates that clas- sical numerical computation theory can provide an inter- pretable mathematical foundation for network structures, offering a perspective the cross-layer information propagation mechanism in U-Nets. However, the method's reliance on the linear multistep principle introduces a limita- tion: the need to store multiple historical solutions, leading to high memory consumption, particularly in large-scale problems with numerous target categories. Addressing this challenge remains key area for future research.
excessive redundancy is unnecessary. step method requires storing multiple intermediate states, memory consumption increases with channel count. Using more than 4N is inefficient, making 2N the optimal balance of performance and cost. Thus, all subsequent experiments adopt 2N for Y.
Table The impact of memory capacity.
Channels
GFLOPs
VRAM(G)
Epoch(s)
Dice
N
264
5.2
125
85,86
2N
265
5.9
128
86.74
3N
266
7.1
135
86.63
4N
267
8.7
86.88
5. Conclusion
This paper reinterprets skip connections in U-Nets from numerical computation perspective for the first time. We propose a multi-scale feature fusion method that integrates the linear multistep method, predictor predictor-correctoo corrector method, and nmODEs, dynamically selecting the optimal discretization order for efficient cross-scale fusion. As a skip connection
A Acknowledgements
This work was supported by the National Natural Sci- ence Foundation of China under Grant 62206189, and the China Postdoctoral Science Foundation under Grant 2023M732427.
8

---

FuseUNet: A Multi-Scale Feature Fusion Method for U-like Networks
Impact Statement
Mishra, N., Kittler, H., and Halpern, A. Skin lesion anal- ysis toward melanoma detection: A challenge at the 2017 international symposium biomedical imaging (isbi), hosted by the international skin imaging collaboration (isic). In 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018), pp. 168-172, 2018a.
This paper proposes a novel perspective for viewing U-Nets and new method for handling skip connections, which can be widely applied to U-Nets. We hope that this research can contribute the advancement of medical image segmenta- tion. We do not think this work has any harmful impact.
Codella, N. C. E., Gutman, D., Celebi, M. E., Helba, B., Marchetti, M. A., Dusza, S. W., Kalloo, A., Liopyris, K., Mishra, N., Kittler, H., and Halpern, A. Skin lesion anal- ysis toward melanoma detection: A challenge at the 2017 international symposium biomedical imaging (isbi), hosted by the international skin imaging collaboration (isic). In 2018 IEEE 1 15th International Symposium on Biomedical Imaging (ISBI 2018), pp. 168-172, 2018b.
References
Antonelli, M., Reinke, A., Bakas, S., Farahani, K., Kopp- Schneider, A., Landman, B., Litjens, G., Menze, B., Ron- neberger, O., Summers, R., Ginneken, B., Bilello, M., Bilic, P., Christ, P., Do, R., Gollub, M., Heckers, S., Huis- man, H., Jarnagin, W., and Cardoso, M. The medical segmentation decathlon. Nature Communications, 13: 4128, 07 2022.
Dong, P., Niu, H., Yi, Z., and Xu, X. nmpls-net: Segmenting pulmonary lobes using nmode. Mathematics, 11(22), 2023. ISSN 2227-7390.
Bashforth, F. and Adams, J. C. An attempt to test the theo- ries of capillary action by comparing the theoretical and measured forms of drops of fluid. University Press, 1883.
Dupont, E., Doucet, A., and Teh, Y. W. Augmented neural odes. Advances in neural information processing systems, 32, 2019.
Bernard, O., Lalande, A., Zotti, C., Cervenansky, F., Yang. X., Heng, P.-A., Cetin, I. Lekadir, K., Camara, O., Ballester, M. A. G., et al. Deep learning techniques for automatic mri cardiac multi-structures segmentation and diagnosis: is the problem solved? IEEE transactions on medical imaging. 37(11):2514 25222225,,,225,-,--- 2018.
Euler, L. Institutionum calculi integralis, volume impen- sis Academiae imperialis scientiarum, 1845.
Fabian, I., Jaeger, P. F., Kohl, S. A. A., Petersen, J., and Maier-Hein, K. H. nnu-net: a self-configuring method for deep learning- based biomedical image segmentation. Nature Methods, 18(2):003-211 Feb 2021. ISSN 1548- 7105.
Cai, S., Tian, Y., Lui, H., Zeng. H., Wu, Y., and Chen, G. Dense-unet: novel multiphoton in vivo cellular image segmentation model based on a convolutional neural net- work. Quantitative imaging in medicine and surgery, 10 6:1275-1285, 2020.
Gragg, W. B. and Stetter, H. J. Generalized multistep predictor-correctt methods. J. ACM, 11(2):188-209, April 1964. ISSN 0004-5411.
Cao, Hu, Wang, Y., Chen, J., Jiang, D., Zhang, X., Tian, Q., and Wang, M. Swin-unet: Unet-like pure transformer for medical image segmentation. In European conference computer vision, pp. 205 205-218,200 2022.
Gu, A. and Dao, T. Mamba: Linear-time sequence modeling with selective state spaces. ArXiv, abs/2312.00755, 2023.
Chen, J., Mei, J., Li, X., Lu, Y., Yu, Q. Wei, Q. Luo, X., Xie, Y., Adeli, E., Wang, Y., et al. Transunet: Rethinking the u-net architecture design for medical image segmen- tation through the lens of transformers. Medical Image Analysis, pp. 103280, 2024.
Hatamizadeh, A., Yang, D., Roth, H. R., and Xu, D. Un- etr: Transformers for 3d medical image segmentation. 2022 IEEE/CVE Winter Conference on Applications of Computer Vision (WACV), pp. 1748-1758, 2021.
He, Q., He, T., and Yi, Medical image segmentation using discretized nmode. In 2023 International Annual Conference on Complex Systems and Intelligent Science (CSIS-IAC), pp. 497-503, 2023.
Chen, R. T., Rubanova, Y., Bettencourt, J., and Duvenaud, D. K. Neural ordinary differential equations. Advances in neural information processing systems, 31, 2018.
Cheng, Z. and Wang, L. Dynamic hierarchical multi-scale fusion network with axial MLP for medical image seg- mentation. Nature-Scientific Reports, 13(1):6342, April 2023.
He, Q., Yao, X., Wu, J., Yi, Z., and He, T. A lightweight u-like network utilizing neural memory ordinary differ- ential equations for slimming the decoder. In Larson, K. (ed.), Proceedings of the Thirty- -hird International Joint Conference on Artificial Intelligence, IJCAI-24, pp. 821-829, 8 2024.
Codella, N. C. F., Gutman, D., Celebi, M. E., Helba, B., Marchetti, M. A., Dusza, S. W., Kalloo, A., Liopyris, K.,

---

FuseUNett: A Multi-Scale Feature Fusion Method for U-like Networks
Heller, N., Isensee, F., Trofimova, D., Tejpaul, R., Zhao, Z., Chen, H., Wang. L.., Golts, A., Khapun, D., Shats, D.,
Ma, J., Li, F,, and Wang, B. U-mamba: Enhancing long- range dependency for biomedical image segmentation. arXiv preprint arXiv:2401 04.00222 2024.
Shoshan, Y., Gilboa-Solomon, F., George, Y., Yang, X., Zhang. J. Zhang, J. Xia, Y., Wu, M., Liu, Z., Walczak, E., McSweeney, S., Vasdev, R., Hornung, C., Solaiman, R., Schoephoerster, J., Abernathy, B., Wu, D., Abdulkadir, S., Byun, B., Spriggs, J., Struyk, G., Austin, A., Simpson, B., Hagstrom, M., Virnig, S., French, J., Venkatesh, N., Chan, S., Moore, K., Jacobsen, A., Austin, S., Austin, M., Regmi, S., Papanikolopoulos, N., and Weight, C. The kits21 challenge: Automatic segmentation of kidneys, renal tumors, and renal cysts in corticomedullary-pasee ct, 2023.
Moulton, F. R. New methods in exterior ballistics. The American Mathematical Monthly, 35(5))26-535 1928. ISSN 00029890, 19300972.
Niu, H., Yi, Z., and Hc, T. bidirectional feedforward neural network architecture using the discretized neural memory ordinary differential equation. International Journal of Neural Systems, 34(04))2450015, 2024.
Perera, S., Navard, P., and Yilmaz, A. Segformer3d: an effi- cient transformer for 3d medical image segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4981 1981-4988, 2024.
Heun, K. Neue methoden zur approximativen integra- tion der differentalgleichungen einer unabhängigen veränderlichen. Z. Math. Phys, 45:23-38, 1900.
Poucet, B. and Save, E. Attractors in memory. Science, 308 (5723):79--800 2005.
Hu, J., Yu, C., Yi. Z., and Zhang, H. Enhancing robustness of medical image segmentation model with neural mem- ory ordinary differential equation. International Journal of Neural Systems, 33(12):2350060 2023.
Ronneberger, O., Fischer, P., and Brox, T. U-net: Con- volutional networks for biomedical image segmentation. International Conference on Medical Image Comput- ing and Computer-Assistel Intervention (MICCAI), pp. 234-241, 2015.
Huang, H., Lin, L., Tong, R., Hu, H., Zhang, Q. Iwamoto, Y., Han, X., Chen, Y.-W., and Wu, J. Unet 3+: A full- scale connected unet for medical image segmentation. In ICASSP 2020 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1055-1059, 2020.
Ruan, J. and Xiang, S. Vm-unet: Vision mamba unet for medical image segmentation. ArXiv, abs/2402.02491, 2024.
Ruan, J., Xiang, S., Xie, M., Liu, T., and Fu, Y. Malunet: A multi-attention and light-weight unet for skin lesion segmentation. In 2022 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), pp. 1150-1156, 2022.
Huang. Z., Wang, H., Deng, Z., Ye, J., Su, Y., Sun, H., He, J., Gu, Y., Gu, L., Zhang, S., and Qiao, Y. Stu-net: Scalable and transferable medical image segmentation models empowered by large-scale supervised pre-training, 2023.
Ruan, J.. Xie, M., Gao, J., Liu, T., and Fu, Y. Ege-unet an ef- ficient group enhanced unet for skin lesion segmentation. International conference on medical image computing and computer-assioe assisted intervention, pp. 481-490, 2023.
Isensee, F., Wald, T., Ulrich, C., Baumgartner, M., Roy, S., Maier-Hein, K., and Jaeger, P. F. nnu-net revisited: call for rigorous validation in 3d medical image segmentation. In International Conference Medical Image Comput- ing and Computer-Assisted Intervention, pp. 488-498, 2024.
Tang, Y., Yang, D., Li, W., Roth, H. R., Landman, B. A., Xu, D., Nath, V., and Hatamizadeh, A. Self-supervised pre-training of swin transformers for 3d medical image analysis. 2022 IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition (CVPR), pp. 20698-20708, 2022.
Liu, Y., Zhu, H., Liu, M., Yu, H., Chen, Z., and Gao, J. Rolling-unet: Revitalizing mip's ability to efficiently extract long-distance dependencies for medical image segmentation. Proceedings of the AAAI Conference on Artificial Intelligence, 38(4):3819 383-38877 Mar. 2024.
Vaswani, A. Attention is all you need, Advances in Neural Information Processing Systems, 2017.
Lyu, P., Zhang, J., Zhang, L., Liu, W., Wang. C., and Zhu,
J. MetaUNETR: Rethinking Token Mixer Encoding for Efficient Multi-Organ Segmentation. In proceedings of Medical Image Computing and Computer Assisted Inter- vention - MICCAI 2024, volume LNCS 15009, October 2024.
Wang. J., Chen, J., Chen, D. Z., and Wu, J. LKM-UNet: Large Kernel Vision Mamba UNet for Medical Image Segmentation In proceedings of Medical Image Comput- ing and Computer Assisted Intervention MICCAI 2024, volume LNCS 15008, October 2024a.
10

---

FuseUNet: A Multi-Scale Feature Fusion Method for U-like Networks
Wang, S., Chen, Y., and Yi, Z. nmode-unet: novel network for semantic segmentation of medical images. Applied Sciences, 14(1), 2024b. ISSN 2076-3417.
Wang, Z., Gu, J., Zhou, W., He, Q. Zhao, T., Guo, J., Lu, L., He, T., and Bu, J. Neural memory state space models for medical image segmentation. International Journal of Neural Systems, 35(01):2450065 2024c.
Wills, T. J.J., Lever, C.,, Cacucci, F., Burgess, N., and O' Jeeeff, Attractor dynamics in the hippocampal representation of the local environment. Science, 308(5723)8833-87 2005.
Wu, R., Liu, Y., Liang, P., and Chang, Q. Ultralight vm-unet: Parallel vision mamba significantly reduces parameters for skin lesion segmentation. ArXiv, abs/2403.20 20035, 2024.
Xiao, X., Lian, S., Luo, Z., and Li, S. Weighted res-unet for high-quality retina vessel segmentation. 2018 9th International Conference on Information Technology in Medicine and Education (ITME), pp. 327-331, 2018.
Xie, Y., Zhang. J., Shen, C., and Xia, Y. Cotr: Effi- ciently bridging enn and transformer for 3d medical im- age segmentation. In Medical Image Computing and Computer Assisted Intervention-MICC 2021, pp. 171- 180, Cham, 2021. ISBN 978-3-030-871992
Yi, Z. mmode: neural memory ordinary differential equation. Artificial Intelligence Review, pp. 1-36, 2023.
Zhang, Y., Liu, H., and Hu, Q. Transfuse: Fusing transform- ers and cnns for medical image segmentation. Medical Image Computing and Computer Assisted Intervention- MICCAI 2021: 24th International Conference, Stras- bourg, France, September 27-October 1, 2021, Proceed- ings, Part 24, pp. 14-24, 2021.
Zhou, H.-Y., Guo, J., Zhang, Y., Yu, I.., Wang, I.., and Yu, Y. informer: Interleaved transformer for volumetric segmentation, 2022.
Zhou, Z., Siddiquee, M. M. R., Tajbakhsh, N., and Liang, Unet++: Redesigning skip connections to exploit multi- scale features in image segmentation. IEEE Transactions Medical Imaging, 396):1856-1866 2020.

---

FuseUNet: A Multi-Scale Feature Fusion Method for U-like Networks
Appendix
A. The detailed process of updating the memory flow Y.
The first time we update the memory stream, i.e., derive 2, we use one-step implicit approach.
Theorem A.1. one-step Adams- method (Moulton, 1928). Given the derivative Y, = F,, lett= trp and choose a step size 8 along the t-axis such that tn+1 = tn + The approximation of Yeuts at the next integration point is given by:
Yeuss = Yi, + 8 (Fi. + Ftatt)
(5)
For U-Net with L stages, let X, and Yi be Xt, and Xt, at different time, respectively, s <1 S L, and set d = 1/L. Taking Xt,, = 11 and Yi, = V1, based on Theorem A.1, we have:
122 Y1+ (Fi+ + F2).
(6)
In the process of calculating Y2, the term F/2 = 2(,,, Y2) is unknown. Therefore, we first need to predict the value of Y2, substitute it into F to compute F2, and then correct the value of Yz. According to the description in Section 3.3, second paragraph, the known values at this stage are X2, X1, Y1, and Fj. When predicting Y2, we can only use one-step explicit method.
Theorem A.2. one-step Adams- AdshsoBashforth method (Bashforth & Adams, 1883). Given the derivative Y, = Fr, = fn- and choose step size along the t-axis such that En+1 tn The approximation of Ytnts at the next integration point given by:
Yr Yr, 8 P
According to Theorem 3.2 and Theorem A.2, we can obtain the predicted value Y2,ie., Yg and calculate F2-
Y22 = Y
Fg F(X,, 12)
At this point, we have obtained all the information necessary to compute Y2. In this process, we obtain Y2 and Fz. The next step is to compute Y3 and F3 using the two-step implicit method.
Theorem A.3. two-step Adams-Moution method (Moulton, 1928). Given the derivative Y. Fr, let = In and choose
step size 8 along the t-axis such that t,+1 = t, + 6, t = t, + 26. The approximation of Y, 1n4z is given by:
8
Ytn Yo +
12 (5F + 8F.4, F.,.).
(9)
In the U-Net specified before Eq. 6, according to Theorem A.3, we have:
Yg = Y2 +
(5 + 8F2 Fi).
(10)
12
Similarly, F3 is unknown, the known values at this stage are X3, X2, X1, Y2, Y1, and F2, Fi. Now we can use a two-step explicit method to predict Yz.
Theorem A.A. two-step Adams-Bashforth method (Bashforth & Adams, 1883). Given the derivative Y, = Fr, let t = tn and choose a step size 5 along the t-axis such that fn+1 = tn + 8, fnt2 = tn + 25. The approximation of Vi is given by:
Yr,12 = Ye. + 2 (3Fe.11 Ft.)-
(11)
According to Theorem 3.2 and Theorem A.4, we can get Y3 and F3.
12

---

FuseUNet: A Multi-Scale Feature Fusion Method for U-like Networks
Y/3 Y/2 +2+ â 2
.33F2 Pi)
(12)
F3 F(X3, Y3).
Then we finish the calculation of Eq. 10, next step is to compute Y4 and F4 using the three-step implicit method.
Theorem A.S. three-step Adams-Moutton method (Moulton, 1928). Given the derivative Yi = Fú, let t = ta and choose a step size 8 along the t-axis such that In+1 = in + 6, lnt2 = in + 26, but3 in + 35. The approximation of V is given by:
8 (9 . + 19 5F,,, + P.,.)
YI 12
(13)
24
In the U-Net specified before Eq. 6, according to Theorem A.5, we have:
Y1 = Ys + 24
(9F) + 19Fs - -5F + Fi).
(14)
F4 is unknown, the known values this stage are X, X4,Y, Yzand Fi F3. Now we can use three-step explicit method to predict Y--
Theorem A.6. three-step Adams-BBahfooe method (Bashforth & Adams, 1883). Given the derivative Y, = Fu. lert tn. and choose step size ó along the t-axis such that bn+1 In lnt2 = ta + 26, 1nt3 t, + 35. The approximation of Y 4n is given by:
Yin4s Vtams + 12
(23F 1.42 - 16F + FF..).
(15)
According to Theorem 3.2 and Theorem A.6, we can get Y4 and F4 as follows:
Y Y 8
(23Fs 16F2 + 5F1)
12 F4 = F(X,,
(16)
Finally, the computation of Eq. 14 is completed. When bn+3 S L. we still use Eq, 13 calculate Yt.4s For example. &
YG = Y1 + 24 (9F3s + 19F; - 5F3 + F2). However, at this stage, we have more known information. When predicting Y,. we can apply a four-step explicit method, unlike the prediction of Y4, where only a three-step explicit method can be used.
Theorem A.7. four-step Adams-Bashfords method (Bashforth Adams, 1883). Given the derivative Y, = Ft, let fn- and choose step size along the t-axis such that tn+1 t, + 6, tnt2 t, 28, tn43 tn + 35, tn+1 fn + 48. The approximation of Ye, given by:
Y Yiues Veuus + 24 (55Fr,,4 59F, k,,2 + 37F bet1 9F...).
(17)
According to Theorem 3.2 and Theorem A.7, we can get Ys and F's as follows:
Yg = Y4 + 24 F; F(X,, Ys).
(55F4 - 59Fs + 37F2 9F1)
(18)
In the subsequent computations, we perform calculations based on Theorem A.5 and A.7. When the computation reaches topmost stage, the known information includes X, Xr,Yi Yr,and Fi Fr. At this point, calculating Yyimal cannot apply the implicit method because XL+1 does not exist for the caleulation. Therefore, we switch four-step explicit method. Then we have:
6
Yfimal Yz +
(55 59F + 37F -9Fz-3)
(19)
24
Finally, we apply a convolution layer with a kernel size of map Yfinal to the prediction map.
13

---

FuseUNet: A Multi-Scale Feature Fusion Method for U-like Networks
Table Workflow and Results. Take network with 6 stages as example for demonstration. P. C, Cal, F stand for Predictor, Corrector, Calculator, nmODEs block, respectively. F. Y, + f(Y; + g(X.)).
Source
Workflow
Result
Y2 = Y1 +
Yi
Y/2
C: Y2 = Yi $ (PI F2)
P: Y3 = Y2 + (3F2 Fi)
X1:2: V1:2
Y33
C: Y3 = Y2 + 12 (5fg 8F2 Fi)
5
P: Y4 = Y3 + 12 (231g 167, + 5F))
X133 V1:3
C: Y2 Y33 + 27 + - -5f2 + R1)
P: Y/ Y4 인 (55F4 597; 9F))
X1:4, V1:4
Y,
C: Y/3 Y4 + (9F +97, 5Pg F2)
X2.5: 12.5
Cal: Y/
Y; + 24
(55F; 59F; + 37Fs 91%)
B. Experimental hyperparameters
Table 7. Initial Learning Rate Scttings
ACDC
KiTS
MSD
ISIC2017
ISIC2018
Backbone
le-2
3e-4
le-4
le-3
le-3
UNet
3e-2
le-3
2e-4
3e-3
3e-3
C. Computational Cost Analysis
Table 8. Computational Cost Analysis for FuscUNcttOri Decoder and Skip Connection
FuseUNet Ori
Decoder
Skip Connection E 2N C. 12-0
Params
L.4N², 12 Eizz 3C² 122
Flops
2L 4N2 12 Ho W. Et 60? 12 H. W. 2 2N C. 19 H. W. + 7H, W. 2N
Taking the L-stage convolutional architecture as an example. In the table, N and the superscript "o" represent the number of target classes and the original, respectively. The reduction parameters and computation is tied to the number of channels in each stage. Flops increase mainly due to interpolation in skip connections, especially when the original network has fewer channels.
Table 9. Detailed comparison of the computational cost data.
VRAM (G) Epoch (s) nn-UNet
UNETR Ultral Light VM-UNet
Backbone FuseUNet
7.33/144
9.4/115
0.87/21
5.91/128
7.8/105
1.27/22
14

---

FuseUNet: A Multi-Scale Feature Fusion Method for U-like Networks
D. Detailed performance
Table 10. Detailed performance data for each fold on 3D tasks
Dataset
Fold
Dicel
Dice2
Dice3
Dice avg
ACDC
90.37
90.46
94.60
91.81
1-Myo
2
91.21
89,40
94.49
91.70
2-RV
89.32
90.11
94.56
90.33
3-LV
91.58
90.28
94.32
92.06
88,41
90.02
94.52
90.98
KiTS23
97.28
83.94
79.51
86.91
I-Kidney
95.49
80.86
79.67
85.34
2-Cyst
97.15
82.83
77.69
85.89
3-Tumor
96.89
87.11
82.82
88.94
5
95.55
84.58
75.48
83.87
MSD
77.92
60.81
76.53
71.75
1-WT
79.72
61.12
79.42
73.42
2-ET
81.40
60.53
78.23
73.40
3-TC
77.92
59.47
79.72
72.37
5
80.56
58.68
76.79
72.01
Table 11. Model Performance Across Datasets of Fig.
Order
KiTS
ACDC
MSD
ISIC2017
ISIC2018
84.8
91.75
71.22
89.25
88.63
2 85.2
91.84
71.49
89.65
89.06
85.7
91.85
71.56
90.15
89.35
4
86.7
92.05
71.75
90.69
89.78
Table 12. Statistical Analysis for FuseUNet Backbone
FuseUNet- Backbone
Mean Differ-
Standard Devia- Differences
Standard Error
95
Confi- dence Interval
t-statistic
Degrees Freedom
p-value
ences
Mean
ACDC
0.03
3.39
0.14
(-0.24, 0.31)
0.25
603
0.80
KITS
0.15
10.18
0.27
(-U.38. 0.67)
0.55
1470
0.58
The data in Table 12 indicate that the performance of FuseUNet the ACDC and KiTS datasets shows no statistically significant difference compared to nn-UNet.
15

---

FuseUNet: A Multi-Scale Feature Fusion Method for U-like Networks
E. Additional Visualization Results
LV
Myo
RV
Fuse- Fuse-UNNd -ie--LNNtt un-UNNl
Figure 6. Visualization on the ACDC
Kidney
Tumor
Cyst
GT
8
nn-UNet
Fuse- UNet
Figure Visualization on the KiTS
WT
TC
ET
GT
FuNe-UNet UNETR
Figure Visualization on the MSD
16

---

FuseUNet: A Multi-Scale Feature Fusion Method for U-like Networks
IMG
GT
Ultraight
Fuse- UNer VM-UNet
(a) ISIC2017
IMG
GT
Fuse-UNNct UltraLight VM-UNet
(b) ISIC2018
Figure Visualization on 2D tasks
17


Reviews:
[{"review_id": "v1nOHrWMpc", "reviewer": "Reviewer_FpbK", "summary": "A new variant of U-Net involving a new way to fuse features across different scales. It achieves half the compute of nnUNet while matching performance evaluation.", "questions": "", "limitations": "", "rating": 5, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank you for the valuable comments and suggestions, which have significantly helped us improve the clarity and depth of the paper. Below, we provide point-by-point responses to each concern.\n\n---\n\n### **Methods and Evaluation Criteria**\n\n**1. Dataset selection and missing benchmarks**  \n\nWe selected datasets based on those used in the original backbone papers to ensure fair comparison and avoid inconsistencies across literature. The MSD dataset was not mentioned in [nnUNet v24](https://arxiv.org/pdf/2404.09556), but we added experiments comparing data from [nnUNet v18](https://arxiv.org/pdf/1809.10486) following your suggestion. ISIC2017 and ISIC2018 were not included as no original data is available, and nnUNet has not been evaluated on them in any relevant paper we reviewed. Therefore, we did not include them.\n\n|Dataset|Model|Dice1|Dice2|Dice3|Dice avg|\n|-|-|-|-|-|-|\n|MSD|nn-UNet|80.71|62.22|79.07|74.00|\n||FuseUNet|80.82|61.32|80.15|74.10|\n\n**2. Justification for dataset choices**\n\nFollowing the principle mentioned above, we balanced data selection with practical compute constraints. For nnUNet, we used two of the three datasets recommended. BTCV, used in UNETR, was reported as unsuitable for comparison in [1], so we ratained MSD. In UltraLight VM-UNet, PH2 was excluded due to its small size (200 images), while ISIC2017 and ISIC2018 were retained.\n\n**3. Reporting only Dice**\n\nSince our primary reference [nnUNet v24] reports only the Dice coefficient, we followed the same format for consistency and fairness. However, we are pleased to provide more detailed metrics of our experiments in later vision.\n\n**4. Compute-related metrics**\n\nWe provide VRAM usage and training time comparisons on RTX4090 between backbone models and FuseUNet. However, we would like to emphasize that compute reduction is not our main contribution; the core innovation lies in the skip connection fusion mechanism, detailed in Response 6.\n\n|VRAM (G) / epoch (s)|nn-UNet|UNETR|UltraLight VM-UNet|\n|-|-|-|-|\n|Backbone|7.33/144|9.4/115|0.87/21|\n|FuseUNet|5.91/128|7.8/105|1.27/22|\n\n**5. On STU-Net comparisons**\n\nIt is important to note that STU-Net relies on pretrained weights. In our experiments, we deliberately excluded models that require pretraining to minimize external factors beyond architecture itself. This helps us ensure that the improvements come solely from the proposed structural changes. We acknowledge the value of pretrained models and plan to explore them as part of our future work.\n\n**6. Comparison with UltraLight VM-UNet**\n\nWhile UltraLight VM-UNet is a strong model, FuseUNet differs in both focus and design advantages.\n\nFirst, the core innovations of them are fundamentally different. UltraLight VM-UNet emphasizes lightweighting with PVM modules, similar to group convolutions, while FuseUNet focuses on multi-scale fusion using a novel view of U-Net stages as discrete ODE nodes and applying techniques like linear multistep and predictor-corrector methods. This enables effective fusion across stages, with lightweighting as a byproduct.\n\nSecond, FuseUNet offers better compatibility with backbones and tasks. UltraLight VM-UNet replaces core modules of others with PVM when applying to them, discarding their core innovations. FuseUNet, on the other hand, enhances skip connections in a way that integrates with existing architectures without altering their structure. Additionally, while UltraLight VM-UNet performs well on 2D single-target tasks, its effectiveness in more complex 3D multi-target segmentation tasks is still unproven.\n\n---\n\n### **Relation to Broader Scientific Literature**\n\nThe goal of FuseUNet is not solely lightweighting. Instead, we propose a general skip connection fusion strategy applicable to both existing and future U-like networks, with practical significance. Besides, major compute reduction would come from lightweighting the encoder and decoder, but we avoided this to isolate the effect of our method. The encoder remains unchanged.  Incorporating ODE-inspired designs into the encoder is part of our future work. We believe that with further development, FuseUNet's efficiency and deployability will improve.\n\n---\n\n### **Other Comments or Suggestions**\n\n**1. Class-wise score source**\n\nFor nnUNet, class-wise results were obtained via direct email communication with the author. We will clarify this in the revised manuscript.\n\n**2. Focus on nnUNet-only**\n\nThank you for recognizing the novelty of our work. While detailed ablation on nnUNet would highlight our method's theoretical effect, generalizability is also key. Since skip connections are common across U-like networks, we prioritize verifying our method's effectiveness across architectures. Deeper ablation studies on nnUNet are planned for future work, as validating generalizability consumed computational resources, limiting our ability to conduct these studies in the current work.", "claims_and_evidence": "Not supported. See below.", "methods_and_evaluation_criteria": "There are multiple evaluation issues for which I have provided a weak reject even though the core technical foundations seem to be strong. \n\n1. nnUNet is missing in the evaluation of MSD and ISIC(s) with no real reason as to why.\n2. There's no real reason provided as to why these datasets are chosen. Very recent (and well defined) literature shows why this is important: https://arxiv.org/pdf/2404.09556 & https://arxiv.org/pdf/2411.03670\n3. Relaying to 2, reporting just one metric is not enough.\n4. If performance in terms of compute is the main innovation (because the scores across model do not seem statistically significant), there should be much more in depth reporting, especially in terms of VRAM usage and run times. These should not be too extravagant to compute in terms of add-on in experimentation times.\n5. Why was STU-L not considered, it achieves comparable performance for almost the same param size.\n6. Overall, it seems to me that UltraLight VM-UNet can achieve everything the proposed model can achieve in terms of performance evaluation and compute costs.", "theoretical_claims": "Did not verify.", "experimental_designs_or_analyses": "Not sound. Explained above why.", "relation_to_broader_scientific_literature": "Important. With the designed steps, it can achieve nnUNet's performance for essentially half the compute. However, at the nnUNet level, it can already run on most commodity hardware and the decrease in compute is not enough to deploy on the edge. This is an important point to be noted in terms of real world use cases.", "essential_references_not_discussed": "STU-Net variant S in terms of adding it in the ablation.", "other_strengths_and_weaknesses": "The primary (and seems to be the only) weakness is in the evaluation setting.", "other_comments_or_suggestions": "1. It is unclear how they achieved the scores for each class. Did they run all the models? If so, it is quite strange that they have matched the exact average values reported here: https://arxiv.org/pdf/2404.09556 where no class -wise scores have been provided. Unless these have been picked from the specific papers (but this is not mentioned anywhere).\n\n2. A personal opinion would be to test only nnUNet and the proposed model on a very strict set of computational ablations to highlight the effectiveness of the method rather than discuss performance scores across various models. I feel those are not even required to highlight the novelty in the work. It also seems that the authors have the necessary compute to do this.", "questions_for_authors": "None.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {"ethical_review_concerns": "None."}}, {"review_id": "4si9OLbirk", "reviewer": "Reviewer_YxK1", "summary": "This paper introduces a new multi-scale feature fusion method for skip connections and for the U-Net framework called FuseUNet, which aims to address the problems of lack the capability for multi-scale information interaction. Specifically, it defines the differential relationship between the skip connections and the corresponding stages. Furthermore, FuseUNet introduces nmODEs for optimization of U-Nets networks, which divides neuron into two parts: learning neuron and memory neuron. Most importantly, the approach proposed by authors is  applicable to any U-like network. Comprehensive experiments are conducted on three datasets to demonstrate its effectiveness.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank you for the constructive and encouraging comments. We appreciate your recognition of our use of Linear Multistep Methods, the clarity of the experiments, and the improvements to U-like networks. Your feedback has helped us refine the paper. Below we provide detailed responses to your comments.\n\n### **Other Comments or Suggestions**\n\n**W1. More detailed ablation experiments are needed for the Predictor-Corrector, Calculator, and nmODEs block**\n\nThe proposed Predictor-Corrector and Calculator modules are implementations of different orders of linear multistep methods. Thus, the ablation study on multistep orders inherently serves as an ablation of these modules. Since they are tightly coupled with established numerical methods, modifying their mathematical formulation would be inappropriate. Apart from varying the step order to explore the trade-off between information depth and complexity, additional experimental variations are difficult to justify.\n\nThe second part of our ablation focuses on the memory space within the nmODEs block. We acknowledge the value of more fine-grained ablations as you suggested. Due to time and computational constraints, we were only able to supplement further experiments on 2D datasets in this round. We plan to explore more comprehensive ablation studies in future work.\n\n| F    | ODE     | Dice17 | SE17 | SP17  | ACC17 | Dice18 | SE18  | SP18  | ACC18 |\n|------|---------|--------|-------|-------|--------|--------|-------|-------|--------|\n| ReLU | nmODEs   | 89.55  | 86.53 | 98.49 | 96.30  | 88.36  | 88.65 | 96.70 | 94.96  |\n| Conv | nmODEs   | 89.14  | 87.04 | 97.73 | 95.43  | 88.52  | 86.16 | 97.66 | 95.18  |\n| PVM  | simple  | 89.18  | 89.59 | 97.46 | 96.01  | 89.25  | 89.50 | 96.96 | 95.35  |\n| PVM  | nmODEs   | 90.69 | 89.59 | 98.20 | 96.62  | 89.77 | 89.10 | 97.41 | 95.62  |\n\nTable: In the ablation study of the nmODEs block, \"ReLU\" refers to the F function used in https://www.ijcai.org/proceedings/2024/0091.pdf. We also replaced the PVM module with a convolutional block to show the importance of preserving core components. The \"simple\" variant removes the implicit mapping in the differential equation based on nmODEs. Results suggest that modifying elements of nmODEs block does not improve performance.\n\n---\n\n**W2. The experiments did not include a comparison with nmODE-Unet**\n\nWe did consider comparing with nmODE-Unet. However, the relevant paper does not provide open-source code or sufficient details about the internal design of the nmODE block, which made reproduction infeasible. For related works that are open-source, their methods are essentially equivalent to our first- and second-order approaches, with the main difference lying in the choice of the function \\( f \\). Based on your suggestion, we have conducted additional experiments on 2D datasets to provide further comparisons and have reported the results accordingly.\n\n---\n\n**W3. The modules (Predictor-Corrector, Calculator, nmODEs) need clearer descriptions, including implementation and workflow**\n\nWhile we have designed the Predictor-Corrector, Calculator, and nmODEs modules in detail, the limited space in the main text prevented us from fully presenting their implementation. To address this, we have included step-by-step derivations in the appendix. However, we acknowledge that the textual explanation may lack clarity in linking the mathematical formulation to the module structure and overall workflow. To improve this, we provide a table to illustrate the workflow and highlight the correspondence between equations and components. Additionally, we have released the [source code](https://anonymous.4open.science/r/FuseUNet-3BA3/README.md) to help readers better understand the implementation.\n\n| source | workflow | result |\n|--------|----------|---------|\n| $X_1,Y_1$ |P: $Y_2 = Y_1 + \\delta \\cdot F_1$||\n||C: $Y_2= Y_1 + \\frac{\\delta}{2}\\cdot (F_1 + F_2)$| $Y_2$ |\n|$X_{1:2},Y_{1:2}$|P: ${Y_3} = Y_2 + \\frac{\\delta}{2}\\cdot (3F_2 - F_1)$||\n|| C: $Y_3 = Y_2 + \\frac{\\delta}{12}\\cdot (5F_3 + 8F_2 - F_1)$|$Y_3$|\n|$X_{1:3},Y_{1:3}$|P: $Y_4 = Y_3 + \\frac{\\delta}{12}\\cdot (23F_3 - 16F_2 + 5F_1)$||\n|| C: $Y_4 = Y_3 + \\frac{\\delta}{24}\\cdot (9F_4 + 19F_3 - 5F_2 + F_1)$|$Y_4$|\n|$X_{1:4},Y_{1:4}$|P: $Y_5= Y_4 + \\frac{\\delta}{24}\\cdot (55F_4 - 59F_3 + 37F_2 - 9F_1)$||\n|| C: $Y_5 = Y_4 + \\frac{\\delta}{24}\\cdot (9F_5 + 19F_4 - 5F_3 + F_2)$|$Y_5$|\n|$X_{2:5},Y_{2:5}$|Cal: $Y_6= Y_5 + \\frac{\\delta}{24}\\cdot (55F_5 - 59F_4 + 37F_3 - 9F_2)$|$Y_6$|\n\nThe process in the table uses a 6-stage U-shaped network as an example.In the table, P, C, Cal, F stand for Predictor, Corrector, Calculator, nmODEs block, respectively. $F_i = -Y_i + f(Y_i+g(X_i))$.\n\n---\n### **Other Comments or Suggestions**\n\nWe have addressed this suggestion in detail across the responses above.", "claims_and_evidence": "The claims in the submission are supported by clear evidence.", "methods_and_evaluation_criteria": "Yes, the proposed methods make sense for the problem.", "theoretical_claims": "Yes, the theoretical claims in this manuscript are all correct.", "experimental_designs_or_analyses": "The experimental results demonstrate performance on both 2D and 3D datasets, but the comparison methods are limited. There is no comparison with other similar methods based on nmODE, and the ablation study is somewhat insufficient.", "relation_to_broader_scientific_literature": "Based on prior related efforts[1, 2], this paper combines them together to reduce network parameters and maintain network performance.\n\n[1] Gragg W B, Stetter H J. Generalized multistep predictor-corrector methods[J]. Journal of the ACM (JACM), 1964, 11(2): 188-209.\n[2] Yi Z. nmODE: neural memory ordinary differential equation[J]. Artificial Intelligence Review, 2023, 56(12): 14403-14438.", "essential_references_not_discussed": "No.", "other_strengths_and_weaknesses": "###Strengths###\nS1: For the related work Linear Multistep Method, Predictor-Corrector Method and nmODEs are introduced relatively clearly, this paper introduces the above methods to optimize U-Net in a reasonable way.\nS2: The experimental results are clear and provide abundant comparative results.\nS3: The paper presents a systematic improvement to the U-Nets architecture, enabling network training with fewer parameters and achieving faster speed.\n\n###Weaknesses###\nW1: More detailed ablation experiments are needed for the three proposed modules in this paper: Predictor-Corrector, Calculator, and the nmODEs block.\nW2: The experiments did not include a comparison with nmODE-Unet[*]. Both approaches improve U-Net using nmODE, and both papers focus on medical image segmentation.\nW3：The modules: Predictor-Corrector, Calculator, and the nmODEs, need to be described in more detail in the paper, including how they are implemented. A comprehensive workflow description is not clear, needing more clarifications.\n\n[*]Wang S, Chen Y, Yi Z. nmODE-Unet: A novel network for semantic segmentation of medical images[J]. Applied Sciences, 2024, 14(1): 411", "other_comments_or_suggestions": "The experimental section needs improvement by including more detailed ablation studies on the Predictor-Corrector, Calculator, and nmODEs block, as well as comparisons with similar methods based on nmODE or Predictor-Corrector.", "questions_for_authors": "The reviewer has no additional questions.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "l2nk5Z9vd3", "reviewer": "Reviewer_n5ho", "summary": "The paper proposes FuseUNet, a multi-scale feature fusion method for U-Net-like networks that enhances skip connection mechanisms by reinterpreting feature fusion as solving an initial value problem (IVP). It employs a linear multistep numerical method with neural memory ODEs (nmODEs) and a predictor-corrector framework, treating skip connections as discrete nodes in an IVP to facilitate effective multi-scale information interaction beyond simple concatenation or addition. Experiments on multiple medical segmentation datasets (ACDC, KiTS2023, MSD brain tumor, ISIC2017, ISIC2018) using CNN, Transformer, and Mamba backbones demonstrate its generalizability, achieving significant reductions in parameters and computational costs while maintaining or surpassing state-of-the-art performance. Ablation studies further analyze the impact of discretization order and memory flow channel numbers, validating the approach’s effectiveness.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate your recognition of our work and your detailed and constructive comments. Below is a summary of our responses.\n\n---\n\n### **Claims and Evidence**\n\n**1. Multi-scale Feature Interaction**\n\nIn the first part of the ablation experiment, we compare the performance of different fused scales, highlighting the effectiveness of multi-scale feature fusion. Specific quantitative metrics have yet to be clearly identified in existing papers. We will explore them in future research.\n\n**2. Generalizability**\n\nWe acknowledge that evaluating only three architectures is insufficient to justify claims of universal generalizability. We will revise the paper to say \"a variety of U-like networks\" and clarify the current scope of validation.\n\n**3. GFLOPs**\n\nHere is a theoretical analysis to explain the computational cost trade-offs. Taking the L-stage convolutional architecture as an example. The reduction in parameters and computation is tied to the number of channels in each stage. Flops increase mainly due to interpolation in skip connections, especially when the original network has fewer channels.\n\n|FuseUNet-Ori| decoder|skip connection|\n|-|-|-|\n|Params|$\\sum_{i=L}^{1}4N^2\\cdot1^2-\\sum_{i=L}^{1}3C_i^2\\cdot k^2$|$\\sum_{i=L}^{1}2N*C_i\\cdot1^2-0$|\n|Flops|$2L\\cdot 4N^2\\cdot1^2\\cdot H_o\\cdot W_o-\\sum_{i=L}^{1}6C_i^2\\cdot k^2\\cdot H_i\\cdot W_i$|$2\\sum_{i=L}^{1}2N*C_i\\cdot1^2\\cdot H_o\\cdot W_o+7H_o\\cdot W_o \\cdot 2N-0$|\n\nN and the superscript 'o' represent the number of target classes and the original, respectively.\n\n---\n\n### **Other Strengths and Weaknesses**\n\n**1. Real-world Significance**\n\nDespite modest performance gains on some benchmarks, FuseUNet offers significant computational savings (up to 50% for standard backbones and 30% for lightweight ones), making it advantageous for compute-constrained applications. Besides, FuseUNet is a general framework, not limited to a fixed model, and can be applied to both existing and future models. We also found that some baselines underperformed in our reimplementation, suggesting FuseUNet's benefits may be understated.\n\nWe view theoretical innovation and practical performance as complementary. Our method introduces a new perspective on feature fusion using numerical ODEs. In future work, we plan to extend ODE theory to the encoder for further improvements.\n\n**2. Hyperparameter Justification**\n\nWe adjusted only the learning rate, keeping other hyperparameters as default. We did not conduct a dedicated ablation study but validated it briefly in the early stage. Hyperparameter tuning is not the focus of this study, and most related works only report them without justification, https://arxiv.org/pdf/2404.09556 simply states \"decreasing the learning rate until convergence\" without specifying a value. \n\n**3. Statistical Significance**\n\nWe added statistical validation as suggested. For Table 4, as metrics were computed from a global confusion matrix over the full test set, per-image scores were unavailable, preventing additional statistical validation. The data shows that FuseUNet performs similarly to the Backbone, which aligns with our claim.\n|FuseUNet-Backbone|Mean of Differences|Standard Deviation of Differences|Standard Error of the Mean|95% Confidence Interval|t-statistic|Degrees of Freedom|p-value|\n|-|-|-|-|-|-|-|-|\n|ACDC|0.03|3.39|0.14|(-0.24,0.31)|0.25|603|0.80|\n|KiTS|0.15|10.18|0.27|(-0.38,0.67)|0.55|1470|0.58|\n\nFig. 5's performance metrics are shown below.\n|oder|KiTS|ACDC|MSD|ISIC2017|ISIC2018|\n|-|-|-|-|-|-|\n|1|84.8|91.75|71.22|89.25|88.63|\n|2|85.2|91.84|71.49|89.65|89.06|\n|3|85.7|91.85|71.56|90.15|89.35|\n|4|86.7|92.05|71.75|90.69|89.78|\n\n**4. Missing Baselines**\n\nWe added suggested baseline where possible, though full evaluation across all datasets may not be feasible within the rebuttal period. \n\n**5. Computational Analysis (Inference & Memory)**\n\nWe provided some theoretical analysis in our previous response, and here we offer a detailed comparison of the computational cost data.\n\n|VRAM (G)/epoch (s)|nn-UNet|UNETR|UltraLight VM-UNet|\n|-|-|-|-|\n|Backbone|7.33/144|9.4/115|0.87/21|\n|FuseUNet|5.91/128|7.8/105|1.27/22|\n\n**6. Limited Modality Diversity**\n\nOur work currently demonstrates the generalizability of the proposed method to some extent through segmentation tasks on three data types. Due to time and compute constraints, we didn’t include more modalities but plan to do in future work to further validate our model’s performance across diverse scenarios.\n\n---\n\n### **Questions for Authors**\n\n**1–5.** These points are addressed in Other Strengths and Weaknesses - 3, 2, 4, 5, and 6, respectively.\n\n**6. Theoretical vs. Practical Focus**\n\nOur primary contribution is theoretical. By linking U-like networks with numerical ODE methods, we propose a mathematically grounded framework for multi-scale fusion. We hope this new perspective will aid both model design and interpretability, serving as a foundation for future architectural research.", "claims_and_evidence": "1. The paper claims that FuseUNet significantly improves multi-scale feature interaction; however, while experiments show performance comparable or marginally superior to baselines, explicit evidence supporting this claim is lacking. The study primarily reports overall Dice metrics without directly demonstrating that enhanced cross-scale feature interaction is the key factor driving these improvements. Detailed quantitative metrics on multi-scale interaction effectiveness would strengthen the argument.  \n\n2. Although the experiments are conducted on diverse architectures, the claim of generalizability across \"any U-like network\" remains insufficiently validated. The study mainly relies on three specific backbones, making the theoretical generalization claim less convincing. Additional empirical support across a broader range of U-Net variants is needed to substantiate this assertion.  \n\n3. The paper highlights a reduction in parameter counts but presents unclear and inconsistent GFLOPs improvements. Notably, for 2D segmentation tasks, GFLOPs slightly increased rather than decreased, contradicting the claimed computational efficiency gains. This discrepancy necessitates a more nuanced explanation or expanded experimental analyses to clarify the impact on computational costs.", "methods_and_evaluation_criteria": "The paper clearly states and conceptually justifies its core methodological innovation: leveraging linear multistep numerical methods, specifically Adams-Bashforth and Adams-Moulton methods, combined with neural memory ordinary differential equations (nmODEs) for multi-scale feature fusion in skip connections. This approach directly addresses the limitations of traditional skip connections in U-Net-based models, making the methodological choice both relevant and theoretically sound for enhancing multi-scale feature interaction.", "theoretical_claims": "The authors conceptualize skip connections as discrete nodes of an initial value problem (IVP), where multi-scale features represent discrete solutions at different timesteps. This analogy is theoretically sound and aligns with established frameworks in neural ordinary differential equations (NODEs) and linear multistep methods. The theoretical background (Section 3.1) on Adams-Bashforth and Adams-Moulton methods, as well as predictor-corrector techniques, is accurate and consistent with classical numerical analysis literature. Additionally, the formulation of neural memory ODEs (nmODEs) in equations (3) and (4) (Section 3.3) aligns with prior theoretical works, correctly capturing the concept of treating neural network decoding steps as discrete ODE solutions. The derivations are presented clearly, with no significant mathematical errors or inconsistencies, demonstrating a careful and rigorous application of established mathematical methods.", "experimental_designs_or_analyses": "The chosen benchmarks and backbone networks—CNN-based nn-UNet, Transformer-based UNETR, and Mamba-based UltraLight VM-UNet—are appropriate and widely recognized, providing a solid basis to verify the generality of the proposed method. The authors utilize well-established, publicly available datasets (ACDC, KiTS2023, MSD brain tumor, ISIC2017, ISIC2018), ensuring reproducibility and comparability of results. Additionally, the evaluation protocol employs standard medical image segmentation metrics, including Dice coefficient, sensitivity, specificity, and accuracy, along with five-fold cross-validation, ensuring a robust and reliable assessment.", "relation_to_broader_scientific_literature": "The paper clearly and accurately positions its contributions within the broader scientific literature, addressing recognized limitations in U-Net-based segmentation architectures (Ronneberger et al., 2015) by enhancing skip connections for improved cross-scale interaction. While existing variants like UNet++ (Zhou et al., 2020) and UNet3+ (Huang et al., 2020) incorporate dense connections and full-scale feature interactions, they primarily rely on simple concatenation or summation, which the authors argue corresponds mathematically to lower-order explicit Euler methods, limiting their information integration capabilities. To overcome this, the authors draw from classical numerical methods—specifically, linear multistep methods such as Adams-Bashforth and Adams-Moulton—and recent advances in neural ordinary differential equations (NODEs, Chen et al., 2018; Yi, 2023). By framing the U-Net decoding process as solving an initial value problem (IVP), the proposed FuseUNet bridges deep learning architecture design with established numerical analysis, enabling higher-order, implicit multi-scale interactions for improved information fusion. This theoretical grounding uniquely situates the work at the intersection of deep learning and classical numerical computation.", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "1. Marginal Practical Performance Gains: Although the proposed FuseUNet reduces the number of parameters and slightly improves performance in some cases, the practical segmentation improvements (Dice scores) are relatively modest (typically under 1%), especially for well-established benchmarks like ACDC and ISIC datasets. This raises questions about the real-world significance of the proposed method compared to existing models.\n\n2. Limited Justification for Hyperparameter Selection: The authors justify the learning rate adjustments (e.g., increasing learning rates by factors of 2 or 3 due to parameter reduction) but do not provide empirical evidence or systematic hyperparameter tuning results. The lack of thorough justification weakens confidence in whether reported performance gains are optimal or incidental.\n\n3. Insufficient Statistical Validation: The paper does not include statistical significance tests (such as paired t-tests or confidence intervals), leaving uncertainty about whether observed minor differences represent genuine improvements or are within experimental variance. Clarity and Visualization Issues: Certain figures (e.g., Fig. 5 with normalized performance) could be improved for clearer interpretability. Non-normalized, explicit performance metrics would better reflect actual impacts and facilitate easier interpretation by readers.\n\n4. Missing Important Baselines: Although the authors benchmark their method against CNN-based, Transformer-based, and Mamba-based architectures, direct comparisons against closely related skip-connection enhancement approaches, particularly UNet++ and UNet3+, are not sufficiently elaborated upon. These models explicitly address multi-scale fusion, and their detailed comparative evaluation would strengthen the justification of FuseUNet’s claimed advantages.\n\n5. Limited Computational Analysis: The reported GFLOPs increase for the lightweight UltraLight VM-UNet backbone is concerning. Although the authors attribute this to interpolation, no comprehensive computational analysis (e.g., inference time or memory consumption) is provided to clearly demonstrate practical efficiency, limiting a comprehensive assessment of the claimed computational benefits.\n\n6. Weak Demonstration of Generalizability: Despite claims of broad applicability, experiments are limited to standard benchmarks. Results across more diverse medical imaging modalities or different medical scenarios are not presented, limiting the generalizability claims.", "other_comments_or_suggestions": "N/A", "questions_for_authors": "1.\tStatistical Significance of Results: Can the authors provide statistical tests (e.g., paired t-tests or confidence intervals) to verify the statistical significance of the performance improvements reported in Tables 3 and 4? Such tests would clarify whether observed improvements over baseline methods are meaningful or could arise from random fluctuations.\n2.\tHyperparameter Justification: Could authors elaborate on how the learning rate and other hyperparameters were selected, ideally presenting ablation studies or grid searches? Clarifying these choices would significantly enhance confidence in the reported performance gains.\n3.\tDirect Comparison to Alternative Multi-scale Methods: Why were direct comparisons to UNet++ and UNet3+—methods explicitly designed for multi-scale feature fusion—not thoroughly presented or discussed? Can the authors include detailed comparative results to clearly demonstrate FuseUNet’s superiority?\n4.\tComputational Efficiency and Practical Deployment: Given that FuseUNet shows a minor increase in GFLOPs in lightweight settings, could the authors further quantify inference speed, GPU utilization, and memory consumption explicitly? Detailed metrics would clarify whether FuseUNet is genuinely advantageous for practical deployments.\n5.\tGeneralizability and Broader Applicability: Have the authors tested or considered their method on modalities beyond CT and MRI segmentation tasks, such as ultrasound or pathology images? Providing additional data or experiments would greatly enhance the strength of claims regarding generalizability.\n6.\tTheoretical Novelty versus Practical Effectiveness: The paper strongly emphasizes theoretical novelty by relating U-Net architectures to numerical ODE methods. Considering the modest practical improvements observed, could the authors clarify whether their primary intent was theoretical innovation (interpretable architecture design) or practical performance improvements?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {"ethical_review_flag": "Flag this paper for an ethics review."}}]

Author rebuttal:

Meta-review: This paper proposes a novel approach to multi-scale feature fusion in U-Net-like networks, grounded in linear multistep methods and neural memory ODEs. The theoretical formulation is elegant and well-founded, offering a fresh interpretation of skip connections through the lens of numerical analysis. Reviewers appreciated the conceptual innovation and broad applicability.

The main concerns brought up by the reviewers are missing comparisons with closely related baselines (e.g., nmODE-UNet, UNet++), lack of runtime/memory analysis, and limited statistical validation of results. Most of these criticisms were addressed by the authors and seemingly accepted by the reviewers. Therefore, the consensus is that the paper presents a meaningful and technically sound contribution that fits well within ICML.
Decision: Accept (poster)