Multiscale Training of Convolutional Neural Networks
Authors: Shadab Ahamed
Niloufar Zakariaei
Eldad Haber
Moshe Eliasof
Venue: Accepted by TMLR ()

Abstract: Training convolutional neural networks (CNNs) on high‑resolution images is often bottlenecked by the cost of evaluating gradients of the loss on the finest spatial mesh. To address this, we propose Multiscale Gradient Estimation (MGE), a Multilevel Monte Carlo‑inspired estimator that expresses the expected gradient on the finest mesh as a telescopic sum of gradients computed on progressively coarser meshes. By assigning larger batches to the cheaper coarse levels, MGE achieves the same variance as single‑scale stochastic gradient estimation while reducing the number of fine mesh convolutions by a factor of 4 with each downsampling. We further embed MGE within a Full‑Multiscale training algorithm that solves the learning problem on coarse meshes first and "hot‑starts" the next finer level, cutting the required fine mesh iterations by an additional order of magnitude. Extensive experiments on image denoising, deblurring, inpainting and super‑resolution tasks using UNet, ResNet and ESPCN backbones confirm the practical benefits: Full-Multiscale reduces the computation costs by  4-16$\times$ with no significant loss in performance. Together, MGE and Full‑Multiscale offer a principled, architecture‑agnostic route to accelerate CNN training on high‑resolution data without sacrificing accuracy, and they can be combined with other variance‑reduction or learning‑rate schedules to further enhance scalability.

Published in Transactions on Machine Learning Research (02/2026)
Multiscale Training of Convolutional Neural Networks
Shadab Ahamed
Deportment of Physics Astronomy, University of British Columbia
Niloufar Zakariaei
vilowzk@Bstddent.c ubc. ca
Department of Earth, Ocean and Atmospheric Sciences, University of British Columbia
Eldad Haber
chaber ubc.ca
Deportment of Earth, Ocean and Atmospheric Sciences, Universily of British Columbia
Moshe Eliasof
me-320ccmm.cc
Department of Applied Mathematics and Theoretical Physics, University of Combridge
Reviewed on OpenReview: https: // owtpr//veeervvee....m net/ /t////ormm? id= HTQuEZWEHH
Abstract
Training convolutional neural networks (CNNs) high-resolution images is often bottle- necked by the cost of evaluating gradients of the loss the finest spatial mesh. To address this, we propose Multiscale Grudient Estimation (MGE), a Multilevel Monte Carlo-inspired estimator that expresses the expected gradient on the finest mesh as a telescopic sum of gradients computed on progressively coarser meshes. By assigning larger batches to the cheaper coarse levels, MGE achieves the same variance as single-scale stochastic gradient estimation while reducing the number of fine mesh convolutions by factor of 4 with each downsampling. We further embed MGE within Full-Mulliscale training algorithm that solves the learning problem coarse meshes first and "hot-starts" the next finer level, cutting the required fine mesh iterations by an additional order of magnitude. Extensive experiments on image demoising, deblurring, inpainting and super-resolution lasks using UNet, ResNet and ESPCN backbones confirm the practical benefits: Full-Multiscale reduces the computation costs by 4-16x with no significant loss in performance. Together, MGE and Full-Multiscale offer a principled, architecture-ae architecture-agnosti route accelerate CNN training on high-resolution data without sacrificing accuracy, and they be combined with other varianco-reduction learning-rate schodules to further enhance scalability
1 Introduction
In this work, consider the task of learning a functional y(x) o(u(x)), where is position (in 2D = (x1,x)) =x,, X2), u(x) 2 and y(x) E y are families of functions. To this end, let us assume that we have discrete samples from 24 and y, that is u uxxx)),y = (xu(x))) for i= 1,... M, associated with some resolution h. common approach to learning the function is to paramcterize the problem, typically by a deep network, and replace é with a function .(...) that accepts the vector n and learnable parameters 0 which leads to the problem of estimating 0 such that y f(u,, 0) for I...,,M. following stochastic optimization problem is formed and solved,
M. To evaluate 0, the
min Bur yth (
(1)
"Cdee available https://ggt thub. wwt.c//wwwaaaa/mmmiiiaaaaaaaaaaa

---

Published in Transactions on Machine Learning Research (02/2026)
where -(.,-) is a loss function (typically mean squared error). Standard approaches use variants of stochastic gradient descent (SGD) to estimate the loss and its gradient for different samples of y²). ). In deep learning with convolutional neural networks, the parameter 0 (the convolutional weights) has identical dimensions, independent of the resolution. Although variants of SGD (e.g. Adam, AdamW, etc.) are widcly used, their computational cost. can become prohibitively high as the mesh-size decreases, especially when evaluating the function f on a fine mesh for many samples u.. This challenge is worsened if the initial guess 0 is far from optimal, requiring many costly iterations for large data sizes M. One way to avoid large meshes is to use small crops of the data where large images are avoided, however, this can degrade performance, especially when a large receptive field is required for learning (Araujo et al. 2019
Background and Related Work. Computational cost reduction can be achieved by leveraging different resolutions, a fundamental concept to multigrid and multiscale methods. These methods have a long history of solving partial differential equations and optimization problems (Trottenberg et al.] 2001 Briggs et al. Nash
2000
2000 Techniques like multigrid (Trottenberg ct al. 2001 and algorithms such as MGopt (Nash
2000
Borzi 2005) and Multilevel Monte Carlo (Giles) 2015 2008
Van Barel & Vandewalle 2019) are widely
used for optimization and differential equations.
In deep learning, multiscale or pyramidal approaches have been used for image processing tasks such as object detection, segmentation, and recognition, where analyzing multiple resolutions is key (Scott & Mjolsness 2019:
Chada et al. 2022: Elizar et al. 2022). Recent methods improve standard CNNs for multiscale computations
by introducing specialized architectures and training methods. For instance, the work by He & Xu (2019)
uses multigrid methods in CNNs to boost efficiency and capture multiscale features, while Eliasof et al,
(2023b) focuses on multiscale channel space learning, and van Betteray et al. (2023) unífies both. Li et al.
(2000) introduced the Fourier Neural Operator, enabling mesh-indepenndett computations, and Wavelet neural
networks were explored to capture multiscale features via wavelets (Fujieda et al.) 2018 Finder et al. 2022
Eliasof et al. 2023ah.
While often overlooked, it is important to note that these approaches, can be divided into two families of approaches that leverage multiscale concepts. The first is to learn parameters for each scale, and a separate set, of parameters that mix scales, as in UNet (Ronneberger
al. 2015). The second, called multiscale training, enables the approximation of finc-scale parameters using coarsc-scale samples (Haber et al.) 2017b] Wang
et al. 2020 Ding et al., 2020 Ganapathy & Liew 2008). The second approach aims lo gain compulational efficiency, as it approximates fine mesh parameters using coarse meshes, and it can be coupled with the first approach, and in particular with UNets.
Our approach. This work advances the multiscale training paradigm by rigorously adapting Multilevel Monte Carlo (MLMC) methods to the non-convex landscape of CNN training. While concepts like MLMC and multigrid are well-established in numerical analysis, their application to stochastic gradient estimation for deep CNNs is non-trivial and requires specific theoretical justification. We distinguish our contributions beyond a simple recombination of existing methods in three key ways.
First, we establish the theoretical bounds for CNN gradient estimation. We explicitly derive the error bounds for the Multiscale Gradient Estimation (MGE) estimator, proving that under standard Lipschitz conditions, the difference between gradients on fine and coarse meshes decays as O(h). This provides the necessary thcoretical guarantee for applying variance reduction to convolutional weights, ensuring that the optimization does not diverge when mixing scales, specificity not addressed in standard MLMC literature.
Second, we provide rigorous analysis of subsampling strategies. Unlike prior empirical works (Haber et al. 2017a), we mathematically demonstrate that the choice of downsampling is critical: we prove that coarsening- based subsampling yields an error that vanishes as resolution increases (O(24))), whereas cropping-based strategies result in a constant error bound (O(1) regardless of resolution. This offers a principled guideline for multiscale training paradigm design.
Finally, we embed these theoretical insights into Full-Multiscale training algorithm. By combining the variance reduction of MGE with coarsc-to-fin "hot-start" initialization, demonstrate an architecture- agnostic framework that accelerates training by orders of magnitude for tasks ranging from denoising to super-resolution, without significant loss in performance over key metrics.
2

---

Published in Transactions on Machine Learning Research (02/2026)
Although method exploits the convolutional operation, it does not explicitly address alternative mechanisms like attention. Consequently, while the framework could, in principle, be adapted to attention, it does not. offer the same theoretical guarantees that we establish for convolutions.
Our main contributions are: (1) we propose new multiscale training algorithm, Multiscale Gradient Estimation (MGE), deriving explicit error bounds for gradient convergence convolutional networks; (ii) we theoretically analyze the limitations of image subsampling strategies, rigorously proving why coarsening outperforms cropping within multiscale framework; and (fii) we validate approach via Full-Multiscale training algorithm on benchmark tasks, showcasing enhanced computational efficiency and scalability across diverse architectures.
2 Multiscale Gradient Estimation
We now present the standard approach of training CNNs, identify its major computational bottleneck, and propose novel approximation to the gradient that can be used for different variants of SGD.
Standard Training of Convolutional Networks. Suppose that we use a gradient descent-based method to train a CNN with input resolution i and with trainable parameters O. Under gradient-based methods, the parameters 0 can be learned iteratively using,
a 0x - Px yh [g(u", yy" 0x)]
(2)
where g(u², . 0&) at iteration represents the gradient with respect to the paramoters of some loss lunction E (e.g., the mean squared error function) given by g(u", y 0s) = ol (J(u", 0x), y*) and ui represents the learning rate. The expectation E is taken with respect to the input-label pairs (u²,y") y"). Evaluating the expected value of the gradient can be highly expensive, especially fine meshes where the value of h is very small. To understand why, consider the estimation of the expected value of the gradient using sample mean of g with batch of N samples,
1 1 g(u
Eygyh [g(u", Oa)]
(3)
The above approximation results in an error in the gradient. Under some mild assumptions on the sampling of the gradient value (Johansen et al. 2010), the error can be bounded by,
E ig(u", ,0,)=))
0x) N Esluf VN
S
C
(4)
for some constant C, where ||-I| represent the L norm. Clearly, obtaining an accurate evaluation of the gradient. (that is, with low variance) requires sampling g( g(u yh, 0&) across many data points i with sufficiently small h reightresolution.. This tradeoff between the sample size N and the accuracy of the gradient estimation, is the costly part of training deep network on high-resolution data.
To alleviate the problem, it is common to use large batches, effectively enlarging the sample size. It is also possible to use various variance reduction techniques (Anschel et al. 2017 Chen et al.] 2017 Alain et al.) 2015). Nonetheless, for high-resolution images, or 3D inputs, the large memory requirement limits the size of the batch. small batch size can result in noisy, highly inaceurate gradients, and slow convergence (Shapiro et al.
2009
2.1
Efficient Training with Multiscale Estimation of the Gradient
To roduce the cost of the computation of the gradients, we use a classical trick proposed in the context of Multilevel Monte Carlo methods Giles 2015). To this end, let h h1 < l22 ... hr, be sequence of mesh
In this paper, we define resolution the pixel size 2D uniform mesh grid, where smaller indicates higher resolution. For simplicity, we assume the same across all dimensions, though different resolutions can be assigned per dimension.
3

---

Published in Transactions on Machine Learning Research (02/2026)
1r
12t
hhth
4h
(1o))(f)) - loss (12h))
(loss (12h)
loss(I2())
(oss(I3))
Ni
N2
N3
loss
Figure 1: Illustration of our Multiscale Gradient Estimation (MGE) algorithm introduced in Section 2 This figure shows a schematic of a 3-level MGE algorithm with resolutions h (finest), 2h, and 4h (coarsest) with batch sizes N3 > N₂ > N1.
sizes, for which the functions u and y are discretized on. We can casily sample (or coarsen) and to some mesh hy, j < L. We consider the lollowing identity, based on the telescopic sum and the linearity of the expectation,
E
b1 (0) E [0) + (0]
(0)- + +... (0) g(²))
(5)
where for shorthand we define the gradient of the loss with respect to 0 with resolution hy by g**(0) g(u", y O). The core idea of our Multiscale Gradient Estimation (MGE) approach, is that the expected value of each term in the telescopic sum is approzimated using different batch of data with different batch. size. This concept is demonstrated in Figure and can be written as,
E [0)]
1 NL
0
Σ j-1 g: (0)
Σ
-0. (0))
(6)
To understand why this concept is beneficial, we analyze the error obtained by sampling each term in Equation (6) Evaluating the first term in the sum requires evaluating the function on the coarsest mesh hy (i.e., the lowest resolution) using downsampled inputs. Therefore, can be efficiently computed, while utilizing large batch size Nr. Thus, following Equation Equation 66) can be bounded by,
(4. the approximation error of the first term in
E 11 (0)]
1 Nr.
(0)
C
VNL
Throughout our discussion, we assume that the per-pixel gradient contribution sp(u²) is uniformly bounded and musschitz-coniinuote in its argument, so that ||<p("") - (p(""]] grows al most linearly with -| addition, when relating gradients across resolutions, we assume that each coarse-grid representative ca (see Appendix A.1 for a detailed discussion) approximates all fine-grid values u within block to within (h), reflecting the smoothness of the underlying image signal. Additionally, we need evaluate the terms of the form,
ry = E - (0) - (0))
The above ry can similarly be approximated by sampling with batch size Nj-1,
f, 1 Σ h (0) g n: (8))
(9)
N,-1
for j 2.....L. L. The key question is: what is the error in approximating r, by the finite sample estimate 2,? Previously, we focused error due to sample size N. However, note that the exact term ry is computed by
4

---

Published in Transactions on Machine Learning Research (02/2026)
evaluating g on two resolutions of the same samples and subtracting the results. The key observation is that if the evaluation of g on different resolutions yields similar results, then g computed on a mesh with step size hj can be utilized to approximate the gradient g on a mesh with finer resolution /1-1,, making the approximation error iy small. Furthermore, for the multiscale estimator we adopt the standard cross- rossstccsoottonn discrepancy condition;
gg (0) -
'. (0)I < Bhj-1
(10)
and p > 0 both independent of the pixel-size h--1- This captures the fact that evaluating the gradient on a slightly coarser mesh yields an approximation whose accuracy improves as the resolution is refined. Then, we can bound the error of approximating rj by fy as follows,
lor some constants B
A 1
||r, - -tyl BC
(11)
VN₃-1
Note that, under the assumption that Equation 10p holds, the gradient approximation error between different. resolutions decreases as the resolution increases (i.e., / - 0). Combining the terms, the sum of the gradient approximation obtained from the telescopic sum in Equation G6 can be bounded by,
( VN
L
hP
c
+
B
(12)
j=2
Sketch of derivation for Equation 12: Starting from the telescoping representation of the fine-scale gradient, we decompose the total error of the multiscale estimator into the sampling error at the coarsest level and the sampling errors of the correction terms across levels. Under our boundedness and Lipschitz assumptions on the per-pixel gradient contribution, the coarsest-level term behaves like a standard Monte Carlo estimator and contributes term proportional to 1/vNN. For each correction term, the cross-resolution discrepancy condition Ilgh:-1 - g*>| S Bhj-1 implies that its sampling error is bounded by factor ht//// /N Summing these contributions over all levels and absorbing constants yiclds the bound in Equation (12) In the two-level case as discussed next, we can then specialize this expression, enforce that the multiscale estirntor matches the single-scale error level, and choose a simple allocation between coarse and fine batches, which lcads directly to the form stated Equation 13
Computational Complexity of using Multiscale Gradient Estimation. Let us look at an exemplary 2-level case (using mesh grids hn and h2 with h = h2 < h2) used for MGE. A standard single-scale gradient estimation (on fine mesh h1) with N samples yields an accuracy of ex = C/VN. If we are to achieve the same accuracy using 2-level method, then, using Equation (2) and choosing N₂ 4N, (that is making the batch size of the coarse mesh h2 four times larger than the one on the fine mesh h1), we see that using MGE the same error is obtain by sampling the fine mesh
Ni = ] (1-2BBP)
(13)
For high-resolution images, as the mesh size h + 0 and B is bounded, Bhd << 1 and therefore the number of computations on high-resolution samples in a 2-level MGE is approximately 1/4 compared to a single mesh algorithm. To see the effect of this sampling on the overall cost, similar to other multiscale algorithms (see Trottenberg et al. (2001)), it is useful to define quantity workunit (#W)). A workunit is defined as the cost. of computation of the convolution operation on the finest mesh. For single-scale estimation of gradients, an error of eN is achieved using N #WU. Let us analyze the same for the level MGE algorithm, which involves the computation of the loss over three different batches: batch of N2 4N1 x N samples over the coarse grid for the first term in Equation 122. a sccond batch of N₃ x N/4 samples over the coarse mesh and finally a third batch of N1 R N/4 over the fine mesh. Since, the cost of convolution of coarse mesh is approximately 1/4 of the cost of the fine scale convolution, the cost of estimating the gradients using a 2-level MGE is N. N x (t) + T x (1 + +) I6 #WU. Thus, even a simple 2-level algorithm can save approximately 43.8% of computations. Detailed caleulations for MGE with more levels is provided in Appendix B When considering the actual wall time for computing multiscale convolutions, note that the two terms (fine and coarse mesh
9N
5

---

Published in Transactions on Machine Learning Research (02/2026)
Algorithm 1 Multiscale Gradient Estimation
Set batch size to Nr, and sample, Nr, samples of nh and yd Pool u R u"1, yh =R,,"" yh
Set loss = "uu,,y"e,, for j= I. do
Set batch size to N, and sample, N, samples of u and yh
Pool u R u yhs = Ryyy and u = R
y = R//
Compute the losses ((u",, y 0) and f(u"s-,, ,,,00,00)))0))
yh ,,, 0) ((u y,y y,y-1,0) loss < loss l(u"s, y
end for
Compute the gradient of the loss.
computations) can, in principle, be executed in parallel. Therefore, with appropriate parallelization of the two processes, the wall time for 2-level MGE algorithm could theoretically be reduced to approximately 50% of the time required for single-scale algorithm. However, it is important to note that in this work, we do not strictly exploit this parallelization potential; our reported experiments are performed sequentially. We leave the engineering of parallel exccution to fully realize these thcoretical wall-time gains for future work.
Beyond these savings, MGE is easy to implement. It simply requires computing the loss at different input scales and batches. Since gradients are linear, the gradient of the loss naturally yields MGE. The full algorithm is outlined in Algorithm
2.2 Multiscale Analysis of Convolutional Neural Networks
We now analyze under which conditions multiscale gradient computation can be applied for convolutional neural network optimization without compromising on its efficiency.
Specifically, for multiscale gradient computation to be effective, the network output and its gradients with respect to the parameters at one resolution should approximate those at another resolution. Here, we explore how & network trained at one resolution h, performs on different resolution 2h. Specifically, let f(u", 0) be a network that processes images at resolution h. The downsampled version 2 = R is generated via the interpolation matrix R, We aim to evaluate f(u2h, 0) using the coarser image 2 A simple approach is reuse the parameters from the fine resolution In Lemma below, we justify such a usage under some conditions.
Lemma 1 (Convergence of standard convolution kernels). Let , be continuously differentiable grid functions, and let 2h = R, and y2th = =RZhyh be their interpolation on a mesh with resolution 2h. Let gh and gah be the gradients of the function in Equation 6 with respect O. Then the difference between g" and 2h is
lg 2 11
As can be seen in the proof - that we present in Appendix A.1 of the above lemma, the convergence of the gradient depends on the amount of high frequencies in the data. This makes sense since the restriction R damps high frequencies. If the signals n and yd contain mainly high frequencies, then the gradients of loss on the coarse mesh cannot faithfully represent those on the fine meshes. We now test the validity of this assumptions for a number of commonly used data set Example
Example 1. Assume that f is a 1D convolution, that is f(u", 0) = u 0, and that the loss is a linear model,
1 (u' 0) T
(x) where 8 is the discretization of some function y on the fine mesh h. We use a 1D convolution with kernel size 3 x 1, whose trainable weight vector is O E TR3 and compute the gradient of the function different.
(14)

---

Published in Transactions on Machine Learning Research (02/2026)
meshes. The loss function the tth coarse mesh of size h can be written as, - Tt;
(R)" 3 u² * 0) (R yy"),
l2 (0)
(15)
where R2,h h is linear interpolation operator that takes the signal from mesh size h to mesh size 2 We compute the difference between the gradient of the loss function on each level and compute its norm,
åg V 0622 V 0l2i1aal
(16)
In our erperiments, add Gaussian noise NOO,I) scaled by different factors to the impuls. We compate the values of ög, and report it in Figure 2 The experiment demonstrates that even for relatively large amounts of noise the difference between the gradients stays rather small. This justified using the approach for a wide range of problems.
Figure 2 also serves as an empirical evaluation of the resolution-induced gradient discrepancy 11 g ||. which the key approximation term used in the analysis of Section [.1] The observed decay of this discrepancy with mesh refinement, as well as its stability under noise perturbations, provides direct support for the cross-resolution consistency assumption underlying the error bounds in Equations (12) and (13). While this
does not measure the full gradient error | VSll. it verifies precisely the gradient behavior required for the multiscale estimator to be theoretically well-founded.
The analysis suggests that for smooth signals, standard CNNs can be integrated into Multilevel Monte Carlo methods, which form the basis of our MGE. The example demonstrates that while for noisy signals, the difference between gradients on different mesh sizes may not decrease, it is still small and thus can be used for estimating the gradients. We also compute the total wall time (in seconds) for the computation of loss under both the single-scale and MGE frameworks for 512 images from the STL10 dataset in Table ! where we show that MGE takes considcrably lower amount of time as compared to the single-scale operations. All computations were performed on a CPU with 48 cores, and a total available RAM of 819 GB. Note that although in terms of FLOP counts, the computation on a coarser mesh is 4 times more effective, using standard hardware and software (PyTorchh) yields more modest gains. This however, can be resolved by designing better hardware and software implementations.
Noise scale u
Table 1: Comparing the time for loss evaluation on
0-0.0
images of different sizes for single images, number of loss function evaluations and total evaluation time for loss computation under single-scale and MGE over 512 images from the STL10 dataset. For this example, all evaluations were performed using a UNet. For the telescopic sum of MGE, for each n, the batch size at the finest possible resolution was set to 16 and subscquent coarser levels were assigned batch sizes in even multiples of 16 (32, 64, etc.) and the coarsest level got all the remaining (out of 512) number of images.
0-0.1
0=0.2
0=0.3
0-0.4
0=0.5
h-
10-2
10 Mesh size
Mesh size (n²) +
256
126?
32a
16a
Wall time for lase computation
Figure 2: The difference between the gradients on different mesh sizes and for different noise levels a. The difference remains small even for large noise levels.
single image (s)
3.338
1.502
0.956
0.682
0.458
Number of loss Singlesscale evaluations
512
512
512
512
512
MGE
624
512
Wall sime 512 inages (s)
Single-scale MGE
1709.06
769.02
489.17
338.91
231.5
527.58
345.98
274.24
245.09
234.50
3 The Full-Multiscale Training Algorithm
While MGE can accelerate the computation of the gradient, solving the optimization problem is still computationally expensive. MGE makes each iteration computationally cheaper, nonetheless, the number of
7

---

Published in Transactions on Machine Learning Research (02/2026)
iterations needed for the solution of the optimization problem is typically very high. Many problems require thousands, if not tens of thousands of iterations, where each iteration sees only a small portion of the data (a batch), due to the large size of the whole data set. Multiscale framework can be leveraged to dramatically reduce the cost of the optimization problem. A common method is to first coarsen the images in the data and solve the optimization problem (that is, estimate the paramntters 0) where the data interpolated (pooled) to coarser mesh. Since the loss and gradients of loss on the coarse mesh approximates the fine mesh problem, the output of the optimization problem on coarse mesh serves a very good initial guess to the solution of the fine scale problem. When starting the solution close to its optimal value, one requires only few iterations on the fine mesh to converge. This process is often referred to called mesh homotopy (Haber et al., 2007). This resolution-dependent approach to training CNNs is summarized in Full-Multiscad in hmg Borzi & Schulz, 2012
Algorithm 2 Full-Multiscale
Randomly initialize the trainable parameters 0.¹
for j j= 1,..... L do
Set mesh size to h2 = 2'-sh and 60' 0." Solve the optimization problem mesh hj for 0
Set 0 - -
end for
Convergence rate for Full-Multiscale To further understand the effect of the Full-Multiscale approach, we recall the stochastic gradient descent (SGD) converge rate. For the case where the learning rate converges to and is smooth and convex, we have that after iterations of SGD, we can bound the error of the loss by,
C
E [[((0)]] E [((+))
CCu Co
(17)
where C' is constant, 0 is the paramcter that optimizes the expectation of the loss, and Co is a constant that depends on the initial error at 0g Let 0 and 0h be the parameters that minimize the expectation of the loss for meshes with resolution H and h, respectively with H and assume that |@0 - 0% S ,H,
where y is a constant independent of h. This assumption is standard (sce Nash (2000) and it justified if the loss converges to a finite value as O. During the Full-Multiscale iterations (Algorithm 2 we solve the problem the coarse mesh H to initialize the fine mesh h solution. Thus, after k steps of SGD on the fine mesh, we can bound the error by,
E [((())]] - E [[(O")] S SyH C
(18)
k
Requiring that the error is smaller than some e, renders bounded number of required iterations
k 0 H
(19)
The above discussion can be summarized by the following theorem.
Theorem 1 (Hotstarting SGD). Let (0) = En nh ((("","", 0)] and let fH) = E, y (l("" y O)}- Let 0 = arrming f# and 8 = arg ming (0). Then, the number of iterations needed to optimize fr. to an error of e is,
= O
In practice, since Algorithm 2 H 24h, h, the number of itcrations for a fixed error e halved at each level, the iterations on the finest mesh are a fraction of those on the coarsest mesh which can speed up training by order of magnitude compared with standard single-scale training.
8

---

Published in Transactions on Machine Learning Research (02/2026)
4 Experimental Results and Discussion
In this section, we evaluate the empirical performance of our training strategies: Multiscale (Algorithm 1 and Full-Mulliscale (Algorithm 2, compared to standard Single-scale CNN training.
Experiments. We demonstrate the broad application of our Multiscale and Full- hall-Multiscale training strategies using architecturos such as RosNet (He ct al.) 2016), UNet (Ronneberger et al. 2015), and ESPCN (Shi et al.) 2016) on a wide variely of lasks ranging from image denoising, deblurring, inpainting, and super-resolution Additional details on experimental settings, hyperparameters, architectures, and datasets are provided in Appendix D Notably, recent architectures employ attention. The application of multiscale techniques to attention are beyond the scope of this paper and will be investigated in future work. We also experiment with different approaches to image subsampling that are based either on cropping, coarsening (pooling) or a combination of both within a multiscale training framework based on MGE.
Research questions. We seek to address the following questions: (i) How effective are standard convo- lutions with multiscale training and what are their limitations? (ii) Can multiscale training be broadly ap- plied to typical CNN tasks, and how much computational savings does it offer compared to standard training? (iii) What is the right image subsam- pling strategy to use, among coars- ening and cropping, within multi- scale training framework?
Comparison of different training strategies, Single-scale, Mul- tiscale and Mull-Multiscale (under various image subsampling strategies like coarsening or cropping for the multiscale training), over different networks and across various tasks such as denoising, deblurring, in- painting, and super-resolution. The training computational costs are measured via # WU and the mean performance of the networks on the test set via MSE or SSIM over various tasks. A paired t-test was performed for Multiscale and Full-Multiscale (for coarsening only) with respect to the Single-scale baseline. Here, u*" indicates performance not statistically different from Single-scale, while "*** indicates perfor- mance statistically different from Single-scale at a significance level of 5%.
Table 2:
Metrics. To address our questions, we focus on performance metrics (e.g,, MSE or SSIM) and the compu- tational effort for training as mea- sured by #WUfor each method. As bascline, train the problem on single-scale (finest) resolution. As explained in Appendix biased to implementation and soft- ware limitations, we define a worku- nit (#WU) as one application of the model on single image the finest mesh (i.c., original resolution). In multiscale training, this unit de- creases by a factor of 4 with each downsampling by a factor of 2. We then compare #WU across train- ing strategies. We also compare a workunit with computational time. For optimal implementation, there is a linear relationship between the two. Finally, we perform paired t- tests bet ween the Single-scale base- line and each of the Multiscale and Full-Multiscale variants on the test set, using 5% significance level to assess whether the differences in their mean performance metrics are statistically significant.
Subsampling strategy
Image Demoisiing, MSE (U)
Training strategy
#WU (4)
Coursen
Crops
UNet
ResNet
Single-scale
480k
0.1472
0.0927
Multiscale (Ours)
74k
0.1417'
0.0943'
Full Multiscale (Ours)
7k
0.1086**
0.0839**
MultiMale (Ours)
X
28.7k
0.2723
0.1599
Full- (Ours)
/
28.7k
0.2632
0.1366
Image Deblurring, MSE (L)
[ to be un-
UNet
ResNet
Single-scale
480k
0.1478
0.1117
Multiscale (Ours)
74k
0.1420
0.1156*
Full-Multiscale (Ours)
/
7k
0.1515**
0.14800*
Tull-Muliiscale (Ours)
X
28.7k
0.4010
0.1852
Multisale (Ours)
/
28.7k
0.3541
0.1849
Iuage Inpainting, SSIM (t)
UNet
ResNet
Single-scale
480k
0.9020
0.9079
Multiscale (Ours)
74k
0.8927**
0.8920"*
Full- Multisalee (Ours)
/
126k
0.9112"
0.9000'
Full Mulliscale (Ours)
X
126k
0.6383
0.8527
Full. Multiscle (Ours)
126k
0.6392
0.8464
Image Super resolution, SSIM (t)
ESPON
ResNet
Singlo-scale
30k
0.8147
0.8294
Multiscale (Ours)
4.6k
0
0.7982
Full-Multiscale (Ours)
/
7.9k
0.79-15**
0
Full-Multiscale (Ours)
X
7.9k
0.7590
0.7501
Full-Multiscas (Ours)
/
7.9k
0.7663
0.7617
9

---

Published in Transactions on Machine Learning Research (02/2026)
Ground truth
Noisy input
UNet
ResNet
Ground truth
Blurred input
UNet
ResNet
Single-scale
Multiscale
|uullMuullti Full-Multiscale
Ground truth
Corrupt input
UNet
ResNet
Ground truth
Low-res Input
ESPON
ResNet
scalle-scale
Multiscale
Full-Mulltisale
Figure Examples of image recovery over different tasks such as image denoising (top-left), deblurring (top-right), inpainting (bottom-left), and super-resolution (bottom-right) under different training strategies (Single-scale, Multiscale, and Full-Multiscale) using various base networks such as UNet, ResNet, and ESPCN.
Image denoising. Here, one assumes data of the form n² = ty + (1 - t)z, where yd some image on a fine mesh h and z ~ NO0, I) is the noise, The noise level E [0, chosen randomly. The goal is to recover yh from u². The loss to be minimized is loss(0) _ Multiscale and Full-Multiscale strategies significantly improve training compute (measured by #WU while maintaining the quality of the reconstructed denoised image on the STL10 test dataset, measured by MSE. Additional experiments on the CelebA dataset for this task are provided in Appendix C.1 (see, Table
_u".,"",dd ff t, 0) - 112 In Table 2 training via
Image deblurring. Here, one assumes data of the form n² = K'y" + z, where yd is some image on mesh h, Kh is the blurring kernel and z is the noise. The goal to recover yh from blurred n². The loss to be minimized is (Os Ey u (u" 0) AI2 In Table [ our Full-Multiscale training strategy significantly accelerates training while maintaining the quality of the reconstructed deblurred image the STL10 dataset, measured by MSE.
Image inpainting. Here, one assumes data of the form n = M + z, where y is complete image on mesh h, M the image corruption operation and z is the noise. The goal is to recover y" from incomplete noised data u². The loss be minimized is loss(0) = B u. f 0) 1 In Table 2 our Mull-sautic training strategy significantly accelerates training while maintaining the quality of the reconstructed inpainted image on the CelebA dataset, measured by SSTM.
10

---

Published in Transactions on Machine Learning Research (02/2026)
Image super-resolution. Here, we aim to predict high-resolution image u" from low-resolution image y', which is a downsampled version of u². The downsampling process is modeled as y' = Du + 2, where D(-) is a downsampling operator (e.g,, bicubic), and z is noise. The goal is to reconstruct u using neural network f(y', .). The loss function is [()) = -By' .rr [SSIM (f(y', .), u²)] In Table 2 our Full-Multiscale training strategy significantly accelerates training while maintaining image quality on the Urban100 dataset, measured by SSIM.
In all our experiments, we noted that within multiscale training framework, a coarsening- based subsampling strategy is better than cropping- gopping-based or combination of both coarsening- and cropping- based strategy. We provide a theoretical justification for this observation in Appendix A.2 The error in the estimation of gradient using the telescopic sum in Equation (5) for a coarsening- based subsampling approach varies as h)2 hence as the mesh resolution h - 0, the error vanishes. On the other hand, for a cropping- based subsampling approach, the error in gradient has an upper bound 2(1 m) (I. 1), where m is the number Nv
of pixels in the cropped patch, N₁, is the total number of pixcls in the entire image, and M represents the upper bound on the norm of gradients for all pixels. Hence, the error in gradients for the cropping- based approach varies as (1) (independent of h) but grows with the number of levels of resolution L in MGE.
Qualitative performance of these training strategies on different tasks have been presented in Figure 3 We discuss the broader impacts of our work in Section 5 Detailed experimental settings and visualizations for each of the above tasks are provided in Appendices D and E respectively. Furthermore, we provide additional experiments with deeper networks, comparison under fixed computational budget, and sensitivity to different number of resolution levels for the Multiscale and Full-Multiscale training in Appendices C.2 to C.4 respectively.
Practical guidelines for hyperparameters (L and batch-size) selection. While the theoretical derivation of MGE allows for arbitrary number of levels, practical implementation requires selecting the number of levels L and batch size multipliers (refer to the calculations in Appendix B based on input
resolution and memory constraints. Based on our ablation studies (see Appendix C.4 and variance analysis in, , we propose the following guidelines for practitioners:
Selecting the number of multiscale levels (L): The choice of involves a tradc-off between computa- lional acceleration and approximation error. As the mesh becomes coarser, the spatial structure may degrade to point where gradients are no longer representative of the fine mesh. We recommend choosing L such that the spatial resolution of the coarsest level h2 remains large enough to capture dominant structural features (typically 2 8 x 8 pixels). For example, with 64 x 64 inputs, we utilized
I. = 4 levels (coarsest. resolution 8 x 8), but for larger inputs (e.g., 256 x 256), T, can be increased to 5 or 6 to maximize speedup.
Sealing batch sizes:
To maintain the variance reduction properties of the telescopic sum (Equation (33 batch sizes at coarser levels should ideally increase to compensate for the approximation noise. Since the computational cost of 2D convolutions decreases by factor of 4 with each downsampling, we recommend increasing the batch size by a factor of 2 to 4 at each subsequent coarser level. This strategy- assigning the largest batches to the cheapest (coarsest) levels- minimizes the variance of the gradient estimator without significantly increasing the wall-clock time or memory footprint.
Potential challenges with extension to attention-based networks. While this work demonstrates the efficacy of MGE for convolutional architectures, extending the framework to attention mechanisms presents distinct theoretical challenges primarily due to the global nature of standard self-attention (Vaswani et al., 2017). Our convergence proofs in Lemma [ rely the locality of the operator, assuming that gradients coarse mesh approximate those on a fine mesh within bounded local neighborhood. Global attention allows every loken to interact. with every other loken, polentially destabilizing the gradient approximation across scales without specific regularization. Future work addressing this would likely benefit from investigating localized window-based attention mechanisms such as Swin Transformers (Liu et al. 2021), which restore the locality assumption required by our theoretical bounds. Furthermore, successfully adapting MGE to attention offers arguably greater potential rewards: since sell-attention computational cost scales quadratically with
11

---

Published in Transactions on Machine Learning Research (02/2026)
resolution, the 4x pixel reduction at each coarser level 2 could theoretically yield up to 16x computational savings per attention layer, significantly amplifying the efficiency gains observed here for convolutions.
5 Conclusions
this paper, we introduced a novel approach to multiscale training for convolutional neural networks, addressing the limitations of high computational costs related to training on single-scale high-resolution data. We theoretically derived error bounds the expected value of gradients of loss within multiscale training framework, proved results the convergence of standard convolutional kernels, discussed the convergence rales of Full-Multiscale algorithm, and provided theoretical justification for why coarsening is a better image subsampling strategy than cropping within multiscale training framework. Empirical findings on mumber of canonical imaging tasks suggest that our proposed methods with coarscning-base can achieve similar or sometimes even better performance than single-scale training with only a fraction of the total training computational costs, as measured by #WUand wall time. While our results indicate that, multiscale training has merit using the existing computational framework, we observe gap between the theoretical complexity to the observed performance. Further gains can be made by using more advanced computational architectures, both in terms of hardware and software. We hope that the theoretical merits discussed in this paper will inspire the development of such efficient implementations that could better utilize multiscale training strategies.
image subsampling
Broader Impact Statement
Our proposed Multiscale Gradient Estimation and Full-Multiscale training algorithms reduce the nunber of expensive fine-resolution convolutions by up 16x, which can potentially significantly cut the energy consumption and carbon footprint associated with training high-resolution convolutional networks while preserving accuracy. By lowering the high expense of large-scale training both in terms of compute hours and electricity consumption this work has the potential to help make high-fidelity deep learning research more attainable for institutions and researchers facing tight budgetary or infrastructure constraints. At the same time, faster and cheaper training could accelerate the development of powerful models for applications ranging from medical-image reconstruction and diagnosis to environmental sensing and weather forecasting, and also lower the barrier for misuse areas like pervasive surveillance or deep-fake generation.
References
Guillaume Alain, Alex Lamb, Chinnadhurai Sankar, Aaron Courville, and Yoshua Bengio. Variance reduction in sgd by distributed importance sampling, arXiv preprint ar arXiv:1511.00/4 2015.
Oron Anschel, Nir Baram, and Nahum Shimkin. Averaged-dqu: Variance reduction and stabilization for deep reinforcement learning. In International conference on machine learning, pp. 176-185. PMLR, 2017.
André Araujo, Wade Norris, and Jack Sim. Computing receptive ficlds of convolutional neural networks. Distill, 4(11)=e21, 2019.
A. Borzi. On the convergence of the mg/opt method. Number 5, December 2005.
Alfio Borzi and Volker Schulz. Computational optimization of systems governed by partial differential equations, volume 8. SIAM, Philadelphia, PA, 2012. ISBN 978-1-611192-044-. URL http:://wwtt ams org/
W. Briggs, V. Henson, and S. McCormick. A Multigrid Tutorial, Second Edition. Society for Industrial and Applied Mathematics, second edition, 2000. doi: 00.1137/1.978008333 org//ol//abs/t..113//11 1ttp:///0..1113/1999999999009990 /t1.://///19999989999995
URL httpps:/eppil siam
Neil K. Chada, Ajay Jasra, Kody H. Law, and Sumeetpal S. Singh. Multilevel bayesian deep neural networks, 2022. URL https: https::///rxvv www.ooggaaao..ooo..o has//bbb/2220/ 12961
12

---

Published in Transactions on Machine Learning Research (02/2026)
Jianfei Chen, Jun Zhu, and Le Song. Stochastic training of graph convolutional networks with variance reduction. Xiv preprint arxin:1711 110.100088. 10568, 2017.
Adam Coates, Andrew Ng, and Honglak Lee. An analysis of single-layer networks in unsupervised feature learning. In Proceedings of the fourteenth international conference on artificial intelligence and statistics, pp. 215-223. JMLR Workshop and Conference Proceedings, 2011.
Lei Ding, Jing Zhang, and Lorenzo Bruzzone. Semantic segmentation large-size vhr remote sensing images using a two-stage multiscale training architecture. IEFE Transactions on Geoscience and Remote Sensing, 58(8):5367 5376, 2020.
Moshe Eliasof, Benjamin J Bodner, and Eran Treister. Haar wavelet feature compression for quantized graph convolutional networks. IEEE Transactions Neurol Networks and Learning Systems, 2023a.
Moshe Eliasof, Jonathan Ephrath, Lars Ruthotto, and Eran Treister. Mgic: Multigrid-in-chaanee channels neural network architecturos. SIAM Journal on Scientifie Computing, 45(3))5307-S328, 2023b.
Elizar Elizar, Mohd Asyraf Zulkifley, Rusdha Muharar, Mohd Hairi Mohd Zaman, and Seri Mastura Mustaza. A review on mulliscale-deep-lernnnng applications. Sensors, 22(19))7384, 2022.
Shahaf E. Finder, Yair Zohav, Maor Ashkenazi, and Eran Treister. Wavelet feature maps compression for imageeto- cnns. Advances in Neural Information Processing Systems, 35:20592 20606, 2022.
Shin Fujioda, Kohci Takayama, and Toshiya Hachisuka. Wavelet convolutional neural networks. arXiv
preprint arXie:1805.088: 08620, 2018.
Velappa Ganapathy and Kok Leong Liew. Handwritten character recognition using multiscale neural network training technique. International Journal of Computer and Information Engineering, 2(3):638-644) 2008.
Michael B Giles. Multilevel monte carlo path simulation. Operations research, 56(3):607 617, 2008
Michael B Giles. Multilevel monte carlo methods. Acla numerica, 24:259 328, 2015.
E. Haber, S. Heldmann, and U. Ascher. Adaplive finite volume method for the solution of discontinuous parameter estimation problems. Inverse Problems, 2007.
Eldad Haber, Lars Ruthotto, and Elliot Holtham. Learning across scales-a multiscale method for convolution neural networks. arXie preprint arXiv: 100.. 103.000030 2017a.
Eldad Haber, Lars Ruthotto, Elliot Holtham, and Seong- Hwan Jun. Learning across scales multiscale method for convolution neural networks. abs/1703.02009.11 8, 2017b. URT.
02009
Juncai He and Jinchao Xu. Mgnet: A unified framework of multigrid and convolutional neural network. Science china mothematics, 62:1331-1354 2019.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770-778, 2016.
Jia-Bin Huang, Abhishek Singh, and Narendra Ahuja. Single image super-rosolution from transformed self-exemplars. In Proceedings of the IEEE conference on compuler vision and pallern recognilion, pp- 5197-5206, 2015.
Adam M Johansen, Ludger Evers, and N Whiteley. Monte carlo methods. International encyclopedia of education, pp. 296-303, 2010.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. Xr preprint 1412. 2014.
13

---

Published in Transactions on Machine Learning Research (02/2026)
Zongyi Li, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar. Neural operator: Graph kernel network for partial differential equations. arXiv preprint Xr1: 03485, 2020.
Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hicrarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference compuler vision, pp. 10012 10022, 2021.
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of International Conference on Computer Vision (ICCV), December 2015.
S.G. Nash. A multigrid approach to discretized optimization problems. Oplimization Melhods and Softwware, 14:99-100 116, 2000.
Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in pytorch. In Advances in Neural Information Processing Systems, 2017.
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. CoRR, abs/1505.04597, 2015. URL
orgg 04597
C. B. Scott and Eric Mjolsness. Multilevel artificial neural network training for spatially correlated learning. SIAM Journal on Scientific Computing, (1(5))5297-S33 2019.
A. Shapiro, D. Dentcheva, and D. Ruszezynski. Leclures on Stochastic Programming: Modeling and Theory. SIAM, Philadelphia, 2009.
Wenzhe Shi, Jose Caballero, Ferene Huszár, Johannes Totz, Andrew P Aitken, Rob Bishop, Daniel Rueckert, and Zehan Wang. Real-time single image and video super-resolttio using efficient sub-pixel convolutional neural network. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp-
1874-1883, 2016.
C. Trottenberg, C. Oosterlee, and A. Schuller. Multigrid. Academic Press, 2001.
Andreas Van Barel and Stefan Vandewalle, Robust optimization of pdes with random coefficients using a multilevel monte carlo method. SIAM/ASA Journal on Uncertainty Quantification, 7(1):174-25) 202, 2019.
Antonia van Betteray, Matthias Rottmann, and Karsten Kahl. Mgiad: Multigrid in all dimensions. efficiency and robustness by weight sharing and coarsening in resolution and channel dimensions. In Proceedings of the IEEE/CVE International Conference on Computer Vision, pp. 1292 1301, 2023.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.
Yating Wang, Siu Wun Cheung, Eric T Chung, Yalchín Efendiev, and Min Wang. Deep multiscale model learning. Journal of Compulational Physics, 406:104 109071, 2020.
14

---

Published in Transactions on Machine Learning Research (02/2026)
A Proofs
A.1 Proof to Lemma 1
Convergence of standard convolution kernels. Let u²,y" yh be continuously differentiable grid functions, and let u R,uu", and y2h. = R yh be their interpolation on mesh with resolution 2h. Let g" and g2t be the gradients of the function in Equation
G with respect to 0. Then the difference between g* and gah is
gg 2h g"1 -hh).
2h
h
k
2h
h
Figure 4: Multiscale Gradient Estimation using coarsening-baser subsampling of images. Here, the left figure shows the discretization of an image on a mesh of resolution h (finest mesh). With subsequent coarsening, the image can be downsampled to a mesh of resolution 2h (and 4h, 8h, and so on). For the sake of proving Lemma 1 we define non-overlapping patches B on the image with resolution 2h (with centers c each of which contains four pixels from the image of resolution h. After coarsening, the image looks like the figure on the right with each pixel of resolution 2h.
Proof. To prove Lemma I let assume mesh with resolution h, shown in Figure (left) containing N₂ pixels. Let the set of all pixels at resolution be denoted by Ω4, and [221 = Nr. While this proof can be generalized for transitions between any two consecutive mesh resolutions 2 + h here we assume j 1 for brevity and restrict our attention to the coarsening + 2h (which is also directly related to the statement of Lemma D. The extension to arbitrary j follows by the same reasoning.
Let us assume the gradient with respect the network parameters of the loss function of the form,
p(u))) b (f(u", 0), ()
(20)
where (f(" 0),y*) represents the contribution to the loss from the pixel j of the image. To prove this lemma, we make the following three assumptions:
(1) The gradient &o is bounded: ||p(())] s M, Va Ωr
(2) The gradient &o is Lipschittz-continuoos:: [|<((x: sp())l < Cl| yll. Va,y E Sr. where C > 0 is the Lipschitz constant.
(3) The error in approximating a pixel h with e² As shown in Figure 4 (left), the image on the grid with resolution h can be coarsened a resolution 2h. At resolution 2h, the overall image consists of Nn/4 non-overlapping patches B with their centers denoted as ct for (1,...,N,, For specific 232h assume that the error || cz < h for 1h B2th and k E (1,...,NN/4),
15

---

Published in Transactions on Machine Learning Research (02/2026)
The gradient g" can be written average over ppuf) for all u} Sh
No
Nr,/4
8
Σ
p
Σ
Σ
(21)
N
N
k=1
Upon coarsening the image to 2h, the gradient Broarsen can be written using the midpoints c of patches B as,
N₂/4
4
2h Scoarsen
N ((cx).
Σ
(22)
k=
Hence, the residual (Equation 9
becomes,
No,/
4sp(cx)) ) |
Teoarsen
l1g
2h Scoarsen
Σ sp(")) En"ER
(23)
Nn
Now, using the triangle incquality, we can write,
N
Σ pp("n) - ap((2))
(24)
Teoarsen
N
x²
But, within specific B2t, En ppll)) 4cp(ce)) = =1 (1((x*)-(p(a) <p((c)))l triangle inequality argument again, we have,
Hence, using the
Σ (pl(')) -((ut)-((a)))ll EA
Σ (pal))(((clll
Σ
Cllu - call s 4Ch,
(25)
a E
a E
where the last two steps in the above expression, we have utilized assumptions (2) and (3). Finally, we have,
Σ
4Ch = Ch.
(26)
Cooarsen
N
1=1
This implies that,
Tooarsen = |1g groarsen
= O(h)
(27)
which completes the proof. Although our proofs use single convolution for simplicity and clarity, the key bound ||g"> /² || O(h) extends directly deep CNNs. Each layer in CNN convolution, nonlinearity, pooling is Lipschitz continuous, with at most multiplicative constant depending the network depth. Thus, the rate of decay of gradient differences across scales remains (hh) for all layers. Consequently, the theoretical guarantees of MGE and Full-Multiscale (variance reduction, telescopic-sum bounds, and convergence behavior) remain valid for deep architectures, with modified but bounded constants.
O(h) perturbation in the input propagates through the network
A.2 Why coarsening is better than cropping under multiscale training framework
invoke similar ideas developed in Appendix A.1 Let us crop patch of size from the image and let wa be the set of all pixels within the cropped patch with [wp = m. The gradient ghop for the cropped patch can be written as,
To compute the residual (Equation (9] for cropping,
gerop
Σ p(ul))
(28)
up Ewn
16

---

Published in Transactions on Machine Learning Research (02/2026)
Let us express the gradient on the finest mesh h in Equation 21 in different way as,
g"
p(u
+
(29)
NR
NA
Eww
where wh represents the of pixels outside the cropped patch and wal Wwil rosidual can be expressed as;
|12p
Ni. Hence, the
Teoop = llg h Scrop
=
1
Σ pp(l)) - (
1
Σ
s(uf))
(30)
Nn uhEwS
No.
up Ewn
Using the triangle inequality, we have,
Terop I Nn E6EWS
Σ || +
Σ
l|o(()))1-
(31)
Nn
And, finally using assumption (1), get,
N₂ - m Nv.
1 )
MM = 2 =2(1- 1- m
Terop
I
M +
M.
(32)
N
This implies that,
Terop = ||" Bcrop || S
2
1
m N₂
M 0(1
(33)
Hence, the upper bound of Terop shrinks with increasing m (as the cropped patch size becomes bigger, m/N, + 1) and is independent of h.
Now, using the telescoping sum for cropping, we have l|g gA>l
S 2(1 No.
M for j = 2..... L. Hence,
1
1. M S Σ 2 M -2-2111 2(1
Σ
lg
M(L-1 -1)
(34)
j=2 Nn
Nn
Therefore, for cropping- based subsampling, have,
(35)
J=2
Hence, when using cropping- based subsampling in MGE, the total error is independent of the resolution h but grows with the number of levels L in MGE.
On the other hand, for the case of coarscning, the telescopic sum becomes,
I
Σ
-g - 1l 5 C(h + 2h 4h + + 21-2 = C(2 1)h.
(36)
2=2
Therefore, for coarsening-basse based subsampling, we have,
1 Σ
g
(37)
/=2
Hence, when using coarsening- boassening-basod subsampling MGE, the total error goes to zero h 0. Hence, coarsening better than cropping as an image subsampling strategy within multiscale training framework.
17

---

Published in Transactions on Machine Learning Research (02/2026)
B Computation of #WUwithin a multiscale framework
Definition 1 (Working Unit (WU)). A single working unit (WU) is defined by the computation of model (neural nelwork) on ingul image on ils original (i.e., highest) resolulion.
Remark 1. To mensure the namber of working wnils (#WUs) required by a neural network and ils training strategy, we measure how many evaluations of the highest resolution required. That is, evaluations at lower resolutions are weighted by the corresponding downsampling factors. In what follows, we elaborate on how # WUUs are measured.
We now show how to measure the computational complexity in terms of #WUfor the three training strategies, Singlo-scale, Multiscale, and Full-Multiscale. For the Single-scale strategy, all computations happen on the finest mesh (size/ h), while for Multiscale and Full-Multiscale, the computations are performed at 4 mesh resolutions (h, 2h, 4h, 8h). The computation running the network on half resolution 1/4 of the cost, and every coarscning step reduccs the work by an additional factor of 4. From Equation
5 we have,
h [0) = E (0) g (0) + + (0) g
(0) + +E g (0) g (0)
+ E g 8h (0)
(38)
With Multiscale, the number of # WWn one iteration needed to compute the Eg" (0) is given by,
#WU Multiscale No 0%
1 21 Ni + 1/1 16
N₂
N2
+
(39)
16
64
where No. Ni. N2 and N3 represent the batch size different scales. With N1 2No. N2 4No and
N₂ 8No #WUfor 1 iterations become;
37 16
W U Multiscale
2.31NO
(40)
Alternatively, seeing an equivalent amount of data, doing these same computations the finest mesh with the Single-scale training strategy, the #WUper iteration is given by, No x 1 N₁ x 1 N₂ x Ng images in one iteration (where each term is computed at the finest scale). With Ni = 2No N₂ = 4No, and N3 = 8No the total #WUfor I iterations in this case, becomes,
WU Singleee-scaale 15N,7
(41)
Thus, using Multiscale is roughly 6.5 times cheaper than Single-scale training.
The computation of # WUfor the Full-Multiscale strategy is more involved due to its cycle taking place at each level. As result, #WUat resolutions h, 2h, 4h and 8h can be computed as,
1
N
WU Pull-Mutticcaie (h) I
N
1
N
Noè
(42)
16
64
64
12hh
1
1
1
N22h
WU Pull-Multiscate
N2h
1
+
N2h
+
(43)
4
4
4
16
16
I4h
1
Nah
# WU Full-Multiacale (4h)
=
x
Nahh
1 + 4
+
(44)
16
4
Ish
WU Full-Multiscale Multiscate (8h)
N78h
(45)
64
(46)
where In. I2h. Tan and Ish represent the number of training itcrations at cach scale and N7 = 2NY No 4Ny. and No. 8Ng for each (h, 2h, 4h, 8h) with Ng 2/ No- For the denoising and deblurring tasks, we
18

---

Published in Transactions on Machine Learning Research (02/2026)
chose I iterations at the coarsest scale with I = Ish 2I 4/2h 81p. for these two tasks is given by,
The total #WUfor Full-Multiscale
37 17 2N
7
1
115
WWU PullMuuttiiceee 16 No.
4No
8No1
No/ 0.90N
(47)
64
64
128
Thus, it is roughly 16 times more effective than using Single-scale training. For both denoising and deblurring tasks, chose No 16 and / 2000.
Similarly, for the impainting and super-resolution tasks, we chose I iterations at the coarsest scale with T= Jsh = Tan = 22 = h We observed that a larger number iterations per level were required for these tasks to achieve a similar accuracy as the Single-scale training. The #WUfor Full-Multiscale for these two tasks is given by,
37
17
7 Not + Nol + Nol = Nol ~ 3.94N 16
63
# W WUFal--Mutiicaaee
No. +
(48)
16
16
16
Thus, it is roughly 3.8 times more effective than using Single-scale training. For inpainting and super-resolution we chose No and 2000, and No 8 and 250, respectively.
C
Additional results
C.1 Experiments for the denoising task on the CelebA dataset
To observe the bchavior of the Full-Multiscale algorithm, we performed additional experiments for the denoising (ask the CelebA dataset using UNet and ResNet. The results have been presented in Table showing that both multiscale training strategies achieved similar better performance to single-scale training for all networks but with considerably lower number of #WU.
Table 3: Comparison of different training strategies using UNet and ResNet for the denoising task on the CelebA dataset. Here, the Multiscale and Full-Multiscale training utilize only the coarsening strategy for image subsampling.
MSE (1)
Training strategy
#WU (1)
UNet
ResNet
Single-scale
480k
0.0663
0.0553
Multiscale (Ours)
74k
0.0721
0.0589
Full-Multiscale (Ours)
28.7k
0.0484
0.0556
C.2 Experiments with deeper networks
In this section, we present the results of our multiscale training strategies for deeper CNNs. To this end, we compare the training to ResNet18 and ResNet50, as well as UNet with levels, for the image denoising task on the STL10 datasct. The results are presented in Table
C.3 Comparison of different training strategies under fixed computational budget
We assessed the performance (as measured by MSE) for both standard convolution under both Single-scale and Multiscale training strategies as a function of the computational budget, measured by # #WU. Table 5 shows that for the same computational budget (#WU), Multiscale training consistently achieves lower error than Single-scale training across all tested budgets. At extremely low budgets (10-20 #WU), Multiscale attains more than 50% lower MSE due to its ability allocate large batches to coarse, cheap levels, producing
19

---

Published in Transactions on Machine Learning Research (02/2026)
Table 4: Comparison of different training strategies using ResNet18, ResNet50, and a deeper UNet for the denoising task on the STL10 dataset. Here, the Multiscale and Full-Mulliscale training utilize only the coarsening strategy for image subsampling.
MSE (4)
Training Strategy
#WU (L)
ResNet18
ResNet50
UNet (5 levels)
Single-scale
480k
0.1623
0.1614
0.1610
Multiscale (Ours)
74k
0.1622
0.161
0.1604
Full-Multiscale (Ours)
28.7k
0.1588
0. 0.1598
0.1597
lower-variance gradient estimates. Even at higher budgets (60-90 900900-00 #WU), Multiscale method still maintains its advantage. These results empirically confirm that Multiscale training makes significantly more efficient use of available compute and that reductions in computational cost do not imply reductions in performance when using our multiscale framework.
Table Comparison of Single-scale and Multiscale training strategies under fixed computational budgets, as measured by #WU.
Computational budget (#W) WU)
MSE (L)
Single-scale
Multiscale
10
0.1000
0.0444
0.0514
0.0179
30
0.0354
0.0231
0.0277
0.0122
60
0.0204
0.0191
70
0.0178
0.0133
0.0159
0.0114
90
0.0151
0.0101
C.4 Ablation over different number of resolution levels within Multiscale training
In this section, we experiment with the number of resolution levels used in our Multiscale and Full-Multiscale training strategies. We had conducted our experiments in the main text in Table wwit levels of resolutions h, 2h, 4h and 8h. The number of levels of resolution is hyperparameter in experiment which can be tuned on held-out validation set. To illustrate this point, Full-Multiscale for the denoising task on the STL10 dalaset in Table t for number of levels 4, and 2. Upon going from 4 levels of resolution to 2 levels of resolution, the performance, in general, slightly degrades for all networks, although it leads to significant gains in computational savings due to results #WUfor lower number of levels of resolution. In fact, the number of iterations and batch size (additional hyperparameters in our experiments, as calculated in Appendix B can also be tweaked further leading to different. (better) values of # Wagainst performance on MSE.
show the performance of Multiscale and
D Experimental setting
For the denoising task, the experiments conducted on STL10 and CelebA datasets using networks, UNet [Ronebergerr et al. 2015) and ResNet (He et al. 2016). For deblurring and inpainting lasks, the experiments were conducted on STLI0 and CelebA datasets, respectively, using the same two networks as the denoising task. For the deblurring experiments, we used a blurring kernel K(x,v) v) S exp (- - with T By 3 blur the input images. For the inpainting task, we introduced up to (randomly chosen for each image) small rectangles with heights and widths sampled uniformly from the range [s/12, 8/6], where
20

---

Published in Transactions on Machine Learning Research (02/2026)
Table 6: Comparison of the performance sensitivity to the number resolution levels for Multiscale and Full-Multiscale training strategies using UNet and ResNet for the denoising task on the STL10 dataset. Here, the Multiscale and Mull-Multiscale training utilize only the coarsening strategy for image subsampling.
MSE (4)
Training strategy
# of Levels
# #WU (4)
UNet
ResNet
Multiscale
74k
0.1975
0.1653
Full-Multiscale
28.7k
0.1567
0.1658
Multiscale
68k
0.2010
0.1741
3
Full-Multiscale
19.5k
0.1644
0.1703
Multiscale
56k
0.2641
0.2081
Full-Multiscale
11k
0.1895
0.1730
s x 8 is the dimension of the image. The key details the training experimental setup for the denoising, deblurring, and inpainting lasks are summarized Table 7 For the super-resolution lask, the experiments were conducted on Urban100 dataset using networks, ESPCN gShi al. 2016) and ResNet (He al. 2016). The key details of the training experimental sctup for the super-resolution task summarized in Table
All our experiments were conducted on system with NVIDIA A6000 GPU with 48GB of memory and Intel(R) (R) Gold 5317 CPU @ 3.00GHz with x86 64 processor, 48 cores, and total available RAM of 819 GB. Upon acceptance, we will release our source code, implemented in PyTorch [Paszke et al. 2017
Table 7: Experimental details for training for denoising, deblurring and inpainting tasks
Component
Details
STL10 (Coates et al. 2011) and CclobA (Liu al. 2015). Images from both datasets were resized dimension
Dataset
of 64 64
UNet: 3-layer network with 32, 64, 128 filters, and residual block (res-block) per layer
Network architectures
ResNet: 2-layer residual network 128 hidden channels
UNet: 2,537,187
Number training parameters
ResNet: 597,699
Single-scale, Multiscale and Full-Multiscale for all networks
Training strategies
Loss function
MSE loss
Optimizer
Adam (Kingma & Ba 2014
Learning rate
5 x 10-4 (with Cosine A Annealing schedular)
Dynamic batch sizing is used, adjusting the batch size upwards during different stages of training for improved efficiency. For details, see Appendix
Batch size strategy
Multiscale levels
4
Single-scale and Multiscale (all tasks): [2000, 2000, 2000, 2000], Mhll-Muutiscall cannosinng/dblurriin [2000, 1000, 500, 250] Full-Multiscale (inpainting): [2000, 2000, 2000, 2000]
Iterations per level
Evaluation metrics
MSE for denoising and deblurring, SSIM for inpainting
21

---

Published in Transactions on Machine Learning Research (02/2026)
Table Experimental details for training for super-resolution task
Component
Details
Urban100 (Huang et al. 2015 consisting of paired low-resolution and high-resolution image patches extracted for training and validation.
Datasel
ESPCN: 5-layer super-resolution network with 64, 64, 32, 32, and 3 filters
Network architectures
ResNet: 9-layer ResNet-like model with
100 hidden channels per layer
ESPCN: 69,603
Number of Training parameters
ResNet: 23,315
Single-scale, Multiscale and Full-Multiscale for all networks
Training strategics
Loss function
Negative SSTM loss
Optimizer
Adam Kingma & Ba 2014)
Learning rate
x 10 (constant)
Dynamic batch sizing used, adjusting the batch size upwards during different stages of training for improved efficiency. For details, see Appendix B
Batch size strategy
Multiscale levels
4
Single-scale, Multiscale, and Full-Multiscale: [250, 250, 250, 250]
Iterations per level
Evaluation metrics
SSIM
E Visualizations
In this section, we provide visualization of the outputs obtained from Singlo-scale, Multiscale and Full- Multiscale training strategies using different networks for various tasks such as image denoising, deblurring, inpainting, and super-resolution. Visualizations for the denoising task on the STL-10 dataset are provided in Figure and on the CelebA dataset are provided in Figure D Visualizations for the deblurring task on the STL-10 dataset are provided in Figure and for the inpainting task on the CelebA dataset are provided in Figure g Visualizations for the super-resolution task the Urban dataset provided in Figure T0
22

---

Published in Transactions on Machine Learning Research (02/2026)
Ground truth
Noisy input
UNet
ResNet
Single Scale
Full-Multiscall Multiscale
Figure 5: A comparison of different network predictions for Single-scale, Multiscale, and Full-Multiscale training for an image from the STL10 dataset for the denoising task. The first two columns display the original image and data (same for rows), followed by results from UNet and ResNet. Here, the Multiscale and Full-Multiscale training utilize only the coarsening stratogy for image subsampling.
Ground truth
Noisy input
UNet
ResNet
Single Scale
Full-Multiscale Multiscale
Figure 6: A comparison of different network predictions for Single-scale, Multiscale, and Full-Multiscale training for an image from the CelebA dataset for the denoising task. The first. two columns display the original image and data (same for all rows), followed by results from UNet and ResNet. Here, the Multiscale and Full-Multiscale training utilize only the coarsening strategy for image subsampling.
23

---

Published in Transactions on Machine Learning Research (02/2026)
Ground truth
Noisy input
UNet
ResNet
Single Scale
Full-Multiscall Multiscale
Figure 7: A comparison of different network predictions for Single-scale, Multiscale, and Full-Multiscale training for an image from the CelebA dataset for the denoising task. The first two columns display the original image and data (same for rows), followed by results from UNet and ResNet. Here, the Multiscale and Full-Multiscale training utilize only the coarsening stratogy for image subsampling.
Ground truth
Blurred input
UNet
ResNet
Single Scale
Multiscale
Full-Multiscale
Figure 8: A comparison of different network predictions for Single-scale, Multiscale, and Full-Multiscale training for an image from the STL.10 dataset. for the deblurring task. The first two columns display the original image and data (same for rows), followed by results from UNet and ResNet. Here, the Multiscale and Full-Multiscale training utilize only the coarsening strategy for image subsampling.
24

---

Published in Transactions on Machine Learning Research (02/2026)
Ground truth
Corrupt input
UNet
ResNet
Single Scale
Full-Multiscale Multiscale
Figure 9: A comparison of different network predictions for Single-scale, Multiscale, and Full-Multiscale training for an image from the STL10 dataset for the inpainting task. The first two columns display the original image and data (same for all rows), followed by results from UNet and ResNet, MFC-UNet. Here, the Multiscalc and Full-Multiscale training utilize only the coarsening strategy for image subsampling.
Ground Truth
Input- Low Res
ESPCN
Res Net
Single Scale
Muiti Scale
Full Multi Scale
Figure 10: A comparison of different network predictions for Single-scale, Multiscale, and Full-Multiscale training for image from the Urban100 dataset for the super-resolution task. The first column displays the low-resolution data (same for all rows), followed by results from ESPCN and ResNet. Here, the Multiscale and Full-Multiscale training utilize only the coarsening strategy for image subsampling.
25


Reviews:
[{"review_id": "AYOqhX8Cff", "reviewer": "Reviewer_ppwV", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "## Summary\n\nThis paper introduces a novel framework for accelerating the training of Convolutional Neural Networks (CNNs) on high-resolution images. The core contributions are two-fold:\n\n- Multiscale Gradient Estimation (MGE): A gradient estimation technique inspired by Multilevel Monte Carlo methods. It expresses the expected gradient on the finest resolution as a telescopic sum of gradients computed on progressively coarser resolutions. By assigning larger batch sizes to cheaper, coarser levels, MGE achieves the same estimation variance as standard single-scale SGD while significantly reducing the number of costly fine-resolution convolutions. The authors provide a theoretical analysis of the error bounds and computational complexity of this estimator.\n\n- Full-Multiscale Training Algorithm: A training schedule that first solves the optimisation problem on coarse meshes and uses the solution to \"hot-start\" training at the next finer level. This approach leverages the idea that coarse-mesh solutions provide good initialisations for fine-mesh problems, and aims to cut the number of iterations required at the finest resolution. A convergence rate for this hot-starting strategy is provided.\n\n## Strengths\n\n- This paper adapts well-established multiscale and multigrid concepts from numerical analysis to the problem of CNN training, offering a principled alternative to ad-hoc methods like cropping.\n- The authors test their method on a range of tasks like denoising, deblurring, inpainting, super-resolution and architectures.\n- The authors derive theoretical support for their method, including a proof of gradient convergence for convolutions across resolutions (Lemma 1), and a convergence rate for the Full-Multiscale algorithm (Theorem 1).\n- The experiments show substantial computational savings (4-16x reduction in work units) without significant loss in performance.\n\n## Weaknesses\n\n- As the authors note, the method and its theoretical guarantees are explicitly tied to convolutional operations. The paper does not address how it would interact with or be applied to attention mechanisms, which are prevalent in modern architectures. While I agree with the authors that addressing attention is outside the scope of the present work, I would appreciate some further discussion in the discussion section. The authors state in the introduction that their approach can be extended to attention layers; a brief paragraph fleshing out this idea would be interesting imho. \n- The paper identifies that coarsening is theoretically and empirically superior to cropping, but the performance of the method is contingent on this choice.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The claims are well-supported by a combination of theoretical analysis and extensive empirical evidence. The paper provides proofs for its key theoretical claims. This theoretical grounding is a significant strength. On the empirical side, Table 2 gives a comprehensive overview over results as as it systematically compares Single-scale, Multiscale, and Full-Multiscale training across four different tasks and multiple architectures. The results consistently show that the proposed methods achieve comparable or better performance (MSE/SSIM) with a dramatically lower computational budget (#WU). The ablation studies in the appendices further strengthen the claims by exploring the robustness and boundaries of the methods.", "audience_yn": "Yes", "audience_explanation": "This paper could be interesting to research in a range of areas, like efficient Deep Learning (particularly on high-resolution data where convolutions are relevant), applied deep learning, and for researches at the intersection of numerical analysis and ML, given the interesting blend of concepts in this paper.", "requested_changes": "The following adjustments are ideas to strengthen the paper; none are critical for acceptance, as the paper is already strong, but they would certainly enhance it.\n\n- The limitation regarding attention mechanisms is briefly mentioned; a discussion on the potential challenges and required modifications to apply MGE to attention layers would be valuable for future work.\n- The paper mentions that the number of levels is a tunable hyperparameter. A more detailed discussion or experiment in the main text (beyond Appendix C.4) on how to choose the number of levels L and the batch size multipliers in practice would be helpful for practitioners.", "broader_impact_concerns": "None", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "9GYFeaLY7B", "reviewer": "Reviewer_QKw5", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This work introduces Multi-Scale Gradient Estimation (MGE) to accelerate CNN training on high-resolution images. Inspired by multilevel Monte Carlo, it expresses fine-resolution gradients as a telescoping sum of coarse-resolution gradients. Coupled with a full-multiscale training scheme, it adopts a coarse-to-fine curriculum that progressively refines initialization at finer scales. Experiments show substantial reductions in computational cost.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The methodology of this work builds on a well-established principle from numerical analysis, namely multilevel Monte Carlo, which offers a principled theoretical underpinning for the technique. For experimental results, the authors validate their method across four image restoration tasks, multiple standard datasets, and several common CNN architectures. The paper further provides a detailed theoretical analysis, and the overall claims are supported by reasonably evidence.", "audience_yn": "Yes", "audience_explanation": "This paper addresses the high computational cost of training CNNs on high-resolution data, a challenge of clear practical value. The proposed method achieves solid cost reductions in experiments and may offer a promising route toward scalable large-resolution training.", "requested_changes": "1. Limited novelty. The core idea appears to be a straightforward refinement of established techniques. MGE is essentially a direct application of multilevel Monte Carlo for variance reduction, and the full-multiscale scheme follows a standard coarse-to-fine paradigm. Although related work (e.g., Haber et al., 2017) is cited, the paper does not clearly articulate contributions that go beyond a recombination of existing methods.\n\n2. Clarity of methodology. The presentation of the core algorithm contains several ambiguities. In Algorithm 1, it is unclear how the variables are aggregated; the loop is declared “parallel,” yet the updates imply sequential accumulation. If a parallel reduction is intended, the current notation is inconsistent with that design.\n\n3. Rigor in language should be improved. Table 2 shows that although computational cost is reduced, task performance (MSE/SSIM) often drops relative to the single-scale baselines, for example U-Net inpainting with SSIM decreasing from 0.8840 to 0.8599 and ESPCN super-resolution with SSIM decreasing from 0.8629 to 0.8274. The statement “without sacrificing accuracy” is therefore insufficiently supported. The manuscript should articulate the cost–accuracy trade-off more explicitly and present the contributions with greater transparency.", "broader_impact_concerns": "This work does not appear to raise any significant ethical concerns.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "UySI8Op9p6", "reviewer": "Reviewer_Y4FE", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This manuscript looks at a multi-scale approach to train convolutional networks.  In contrast to prior work, this framework is based on telescoping sums of expectations, allowing the gradients to be properly estimated by a combination of images at a variety of scales.  This framework is intriguing.  Empirical evidence is provided comparing this strategy compared to standard practice, showing that the proposed strategy uses significantly less compute units.", "claims_and_evidence_yn": "No", "claims_explanation": "There are some slight gaps that make this criterion not fully satisfied.\n\nParticularly, there is the claim that there is \"no significant loss in performance.\"  However, in many cases in Table 2, there is a gap in performance in several of the the results, and it does not seem fully evaluated what is significant here nor statistical analysis.  For example, in the ResNet performance in image deblurring completely eradicates all of the gain from moving from UNet to ResNet, which does seem significant.  In the main draft, there should be a consideration of how much computation is used, including performance as the amount of computation changes.  This is mildly addressed in Table 5, but does not seem comprehensive enough.\n\nTheoretically, there is a small gap that the analysis that is presented is on a single convolution, yet a convolutional neural network has many layers that interact in complex ways.  There needs to be at least a discussion on what happens as you go through deeper layers to the gradient approximations from a theoretical perspective.\n\nThe discussion on how much computation is saved is not clear enough to be, as they are based on a lot of assumptions which were not clear enough.  The details on how much is saved in theory and in practice needs to be more fully elucidated.\n\nWhile there is an analysis of the error on the different level of scales, there doesn't seem to be a clear analysis on the telescoping nature of the gradient. I would encourage the authors to directly calculate error in the gradient as a function of different strategies, to show how the error varies.  It would be feasible for a step or two to calculate the full gradient versus the approximate gradient, and empirically show the errors.", "audience_yn": "Yes", "audience_explanation": "This is an interesting idea that could be used in many cases to reduce computational costs based on traditional statistical techniques.  As it is simple to implement, theoretically sound, and helpful, I think that a reasonably broad audience would be interested.", "requested_changes": "I am requesting for acceptance:\n1. Statistical analysis of the performance to support the claims made in the abstract.\n2. Greater description of what level of performance degradation is trivial.\n3. Discussion of theoretical properties of how bounds would change in a deep neural network (much of the analysis is for a single convolution).\n4. Greater discussion of the impact of different amounts of computation on performance.\n5. Empirical analysis of gradient approximation error.\n6. I don't understand the jump from (24) to (25).  Please elaborate.\n\nAs minor points (suggested):\nThroughout Section 2.1, you are dependent on assumptions, where are largely given in the appendices.  I would add in key assumptions to the main draft.\nFor ease of interpretability, I would provide a derivation of (12) fully along with (13).", "broader_impact_concerns": "None.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)