Subpixel object segmentation using wavelets and multiresolution analysis
Authors: Ray Sheombarsing
Nikita Moriakov
Jan-jakob Sonke
Jonas Teuwen
Venue: ICLR 2022 Submitted (2022)
Keywords: Segmentation
wavelets
contour prediction
multiresolution analysis

Abstract: We propose a novel deep learning framework for fast prediction of boundaries of two-dimensional simply connected domains using wavelets and Multiresolution Analysis (MRA). The boundaries are modelled as (piecewise) smooth closed curves using wavelets and the so-called Pyramid Algorithm. Our network architecture is a hybrid analog of the U-Net, where the down-sampling path is a two-dimensional encoder with learnable filters, and the upsampling path is a one-dimensional decoder, which builds curves up from low to high resolution levels. Any wavelet basis induced by a MRA can be used. This flexibility allows for incorporation of priors on the smoothness of curves. The effectiveness of the proposed method is demonstrated by delineating boundaries of simply connected domains (organs) in medical images using Debauches wavelets and comparing performance with a U-Net baseline. Our model demonstrates up to 5x faster inference speed compared to the U-Net, while maintaining similar performance in terms of Dice score and Hausdorff distance.

Under review as a conference paper at ICLR 2022
SUBPIXEL OBJECT SEGMENTATION USING WAVELETS AND MULTI RESOLUTION ANALYSIS
Anonymous authors Paper under double-bling review
ABSTRACT
We propose novel deep learning framework for fast prediction of boundaries of two-dimensional simply connected domains using wavelets and Multi Resolu- tion Analysis (MRA). The boundaries are modelled as (piecewise) smooth closed curves using wavelets and the so-called Pyramid Algorithm. Our network ar- chitecture is a hybrid analog of the U-Net, where the down-sampling path is a two-dimensional encoder with learnable filters, and the upsampling path is a one- dimensional decoder, which builds curves up from low to high resolution levels. Any wavelet basis induced by a MRA can be used. This flexibility allows for incorporation of priors on the smoothness of curves. The effectiveness of the proposed method is demonstrated by delineating boundaries of simply connected domains (organs) in medical images using Debauches wavelets and comparing performance with a U-Net baseline. Our model demonstrates up to 5x faster in- ference speed compared the U-Net, while maintaining similar performance terms of Dice score and Hausdorff distance.
INTRODUCTION
Semantic image segmentation is a core component of many medical imaging related tasks. Both as part of pipeline to find a region of interest, task by itself, e.g., for measuring tumor volume. Nowadays, almost all segmentation algorithms in medical imaging are replaced by U-Net-like ar- chitectures Ronneberger et al.) (2015) combining an encoder and decoder. Typically, the decoder is an upsampling path, and additional skip connections between the encoding and decoding part are added recover the image's spatial information. While many variants or more exotics methods, such as multi-scale and pyramid based approaches, recurrent networks generative techniques, can be designed, all of these still yield per-pixel classifications. In these settings an image is interpreted as a discrete collection of ordered pixels (or voxels in the three-dimensional case), where the task to assign an appropriate class to each pixel using a probabilistic model. Typically, the pixels are assumed to be independent that the joint- hint-liiel is tractable.
In practice, e.g., in medical imaging, the boundary of a region is annotated and not the region it- ground-truth data is discretization of closed curve. The main motivation
self. Hence the of our paper is to construct a deep learning model which (i) may directly use such raw ground- truth data if available (ii) guaranteed to predict smooth planar curves (iii) improves inference speed by predicting 1d objects (curves) instead of 2d objects. We argue that in traditional pipelines, where pixel-based predictions are constructed, smooth boundaries are faithfully represented. In particular, no prior information about the geometry of planar curves is incorporated. In this paper, we present a hybrid analog of the U-Net, where the sampling path is a two-dimensional en- coder with learnable filters, and the upsampling path is an one-dimensional decoder, which predicts (smooth) representations of curves.
A fundamental component in the setup of our framework is the decision on how to represent (closed) curves. While the Fourier basis a natural candidate at first glance, its global nature may hamper accurate predictions of curves which exhibit highly localized behavior, requiring accurate estimates of small noisy high-frequency modes. For this reason, we have chosen to represent contours using wavelets and Multi Resolution Analysis (MRA) instead. The main idea is to choose single map ip, the so-called scaling function or father wavelet, and to construct subspaces of functions associated to prescribed resolution levels by taking the span of appropriate dilations and translations of yp. This

---

Under review as a conference paper at ICLR 2022
setup provides efficient way decompose and reconstruct contours, from low to high resolution level, using the classical Pyramid Algorithm Mallat (2008) as a decoder. The filters in the decoder not learned but uniquely determined by the chosen wavelet basis. Any wavelet basis induced by a MRA can be used. This flexibility allows for incorporation of priors on the smoothness of curves.
Related work Previous work by Chen et al. (019); Marcos et al. (2018); Hatamizadeh et al. (2020) also proposed models to predict contours by combining Active Contour Models (ACM) with a CNN into an end-to-end model. In these papers the representation of curves is ultimately still based on pixel-based computations. For instance, in |Hatamizadeh et al. 2020) curves are modelled as level sets of distance maps defined on discretization of the domain of the image. In [Cen et al. (2019) similar approach is followed, but a smoothed approximation of an indicator function is used instead of a distance map. The work in Marcos et al. (2018) is perhaps most closely related to ours; they directly construct polygonal approximations of curves and represent them using (pixel) coordinates careful consideration of
of the nodes. The objectives minimized in the cited papers are based mean pixel intensities and geometric properties such as area and are length. These properties are implicitly encoded in an objective function (energy-functional) defined on space of distance maps Hatamizadeh et al. (2020), suitable approximations of indicator functions [Cenn et al. (2019), or family of polygons Marcos al. (2018). However, in contrast to our approach. the above methods all provide maxel-based output. To the best of our knowledge, our work is the first to use MRA and wavelet analysis construct pixel-indeppe -iddependeednt representations of (closed) curves.
This paper is organized as follows. In Section 2 we review the mathematical background needed to construct our model. We explain how contours can be decomposed and reconstructed on different resolution levels. The reconstruction algorithm, the Pyramid Algorithm, forms a core component of our network architecture. Section 3 we set up the model architecture and loss. Subsequently the datasets, training method and performance measures are described in Section + paper with results and a discussion in Section 5 Further mathematical details are provided in the appendix.
We end the
2 BACKGROUND AND MATHEMATICAL SETUP
In this section we describe our mathematical setup and review the theory needed to construct our model. We consider two dimensional gray-valued images x E X == [0. 1)"x", e-g.. slices of MRT scans of size n X n, where E N. We assume that each image contains a (uniquely identifiable) simply connected region R(x) C R²,, e.g., an organ, with boundary 0R(z). It is assumed that 0R(z) can be parameterized by simple closed continuous curve 7(x) We will develop deep learning framework for computing such parameterizations r(z) using Multi Resolution Analysis (MRA). Since wavelets play seminal role in set up, first review the necessary theory in Sections and 2.2 as a subject in its own right in the context of general scalar-valued signals y. In the remainder of the paper, we return to the context in which 7 (1r) is interpreted periodic (planar) curve parameterizing the boundary of region.
2.1 MULTI RESOLUTION ANALYSIS
In this section we briefly review wavelet theory using the framework of a multi resolution analysis (MRA). We closely follow the exposition in Pereyra & Ward (2012) and Montefusco & Puccio 2014). Throughout this section we denote the space of square integrable functions on R, equipped with standard inner product, by (RR.
The uncertainty principle in Fourier analysis dictates that a signal y E 2 cannot be simulta- neously localized in the time and frequency domain, see Pereyra & Ward 2012). Multi resolution analysis aims to address this shortcoming by decomposing a signal on different discrete resolution levels. The idea spanned by integer shifts of localized mapping 9o The level of localization associated to V, is determined by taking an appropriate dilation of prescribed map yo; the so-called scaling function. In the MRA framework the dilation factors are chosen to be powers of two. Formally, we require
construct subspaces Vj C (R)), associated various resolution levels j Z,
that (PP))) is an orthonormal basis for Vj, where (t)(()) = 2 ( k)., see Figures la and
2

---

Under review as a conference paper at ICLR 2022
va(t) - 214(222-
sp((t) 2ha02/
(a) Father wavelet
(b) V
(c) Mother wavelet
(d) W
Figure 1: Example of the Haar MRA: Dilated translation of the Haar scaling map 4 [ 1)-
(b) The approximation subspace level consists of all step-functions with step-size 2- J cc)
Dilated translation of the mother wavelet vA = 10. 11.1) (d) Example of function in the
detail subspace at level j.
1b
A Altogether, this yields an increasing sequence closed subspaces V, C V5+1 C (RR) dense 2 (R). where Vj+1 is the next level up in resolution after V,
The representation of at resolution level j. denoted by "i == Py,, is its orthogonal projection onto Vj. Here P, denotes the orthogonal projection onto Vj- The coefficients of 7/ with respect to the basis for V,, denoted by (7()) = (7))(7)))Ez) are called the approximation coefficients. The associated subspaces V, are referred to as the approximation subspaces (see Figure [bb.. To study the information that is lost when signal in Vj+1 is projected onto Vj. we consider the operator Q2 : Py. The range of Q,, denoted by Wj. is referred to as the detail subspace at level j; it is the orthogonal complement of V, in Vj+1- The detail subspaces (Wj)se2 and orthogonal by construction. A fundamental result, known as Mallat's Theorem, states that the subspaces W,
mutually disjoint
too be spanned by dilating and shifting single map. More precisely, there exists a map t E Wo. the so-called mother wavelet, such that (Win)sez is an orthonormal basis for W,. see Pereyra & Ward (2012). The coefficients of Q;7 with respect this basis, denoted by d,(7)= := (djx(7))kezz are referred to as the detail coefficients of 7 at resolution level j. The detail coefficients store the information needed to go back one level up in resolution, since P,+1 Q, by construction. We often write a7()) aj and dy( = d, for brevity. In practice, we only approximate a finite number of approximation and detail coefficients, see Section 2.3
2.2 THE DISCRETE WAVELET TRANSFORM
In this section we describe how to compute the approximation and detail coefficients given pre- scribed scaling function sp. Many fundamental aspects of MRA's, both theoretical and compu- tational, can be traced back to the following key observation. Since Vo C V/1 there must exist coefficients h (h& )EE such that & Σkez hx4PIk- This equation is referred to as the scaling equation; one of the fundamental properties of a scaling function. The sequence ht the so-called low- pass filter, completely characterizes the scaling function. Similarly, since v Wo C Vi, there exist coefficients g (gk) kez, the so-called high-past fags-passoe filter associated to h, such that & ExeZ Ik4P1k- For Mallat's 'ss mother wavelet, we have gk (-1)*= 1 hn-k- Altogether, in order to define MRA, one only needs to specify appropriate low-pass pass filter h.
The scaling equations can be used to derive an efficient scheme for computing lower order approx- imation coefficients (of any order) given an initial approximation a5-1. Conversely, the orthogonal decomposition Vj+1 = V, < W, can be used to reconstruct aj+1 given the approximation and de- tail coefficients a, and d,, respectively, at resolution level j. The computations summarized in Figure 2 The reconstruction and decomposition formulae together form the well-known Pyramid Algorithm Mallat (2008).
In practice, our signals are periodic and do not directly fit into the MRA framework, since non-zero periodic signals not elements in L (RR. We will address this issue in the next section by using an appropriate cut-off. Here we only remark that the periodicity needs to be carefully taken into account in the decomposition and reconstruction formulae, see Appendix A.2
3

---

Under review as a conference paper at ICLR 2022
*h
aj
aj
*h
aj+1
3j+1
#g
dj dj
(a) Decomposition
(b) Reconstruction
Decomposing approximation coefficients level j + into approximation and detail coefficients at level j. Here h and I are defined by hi == h- and Ds == g-A, respectively, the two-sided discrete convolution and J. downsamples sequence by discarding all terms with odd index. (b) Reconstructing approximation coefficients at level j + from the approximation and detail coclficients at level j. Here f samples sequence up by putting zeros in between every term.
Figure 2: (a)
2.3 WAVELET REPRESENTATION OF PERIODIC CURVES
2-
1
Figure 3: The cut-off signal (t) = (1t) --11) (.) depicted in blue. We only compute approxima- tion coefficients associated to the region 1-2. 2.
In this section explain how scalar-valued periodic signal 7 with period mated using a MRA. Let ji E N be a desired resolution-level. First, we re-parameterize 7 to have period The reason for this rather technical and we refer the reader we require the number of approximation coefficients to be power of two and this parameterization fits nicely with this requirement. Next, address in 1.2(R), we restrict the re-parameterized curve to 11, i.e., set -²(() := 7(1)
be approxi-
Appendix[A.I In essence,
issue that periodic signals are not contained In general, this will introduce discontinuities at the boundary points and 1. This is, however, an
issue, since only need information about " strict subset [4o 41] [-1, 1] of length 1.
shown in Lemma A. I how (and which) approximation coefficients can related to the sam- ple values of y. In particular, if jı is sufficiently large, the approximation coefficients needed to (approximately) cover [-1, 1] are (a,i (7'x(("") k=-2-1 Here 3 > is the support of the underlying wavelet. These coefficients will be close the (scaled) sample values of 7 on {k:2 -i -2 <
k < 2 - 3J}. Motivated by this observation, and the fact that we only need on [- 2 1.1. we
use the coefficients (aj,k(77 ())))) k= 2-1 only, which cover [- 1-21- approximately. see Figure 2)
3 To ensure that 2 1 1 B), require that j1 > [og (8-1) 1og(2)
1
This quantity is
well-defined for all bases considered in this paper, except the Haar-basis.
MODEL
In this section formulate objective and network architecture. Henceforth 7 >(x)) is asso- ciated to an image x and has two components [x)]] and [n(x)]2. All operations from the previous sections understood to be carried out womponent-wise.

---

Under review as a conference paper at ICLR 2022
Bolkwokk
Prediake brandh
0
Rostall conthoce
2
2
Reconstruct (Pyramid algorithm)
conv filters)
Figure 4: A schematic picture of our network. Here a "residual block" corresponds to standard residual convolutional layer. The first residual conv-block uses 32 filters and doubled after every two residual conv-ble blovkb.ocks. Each green block corresponds to the operation depicted in Figure 2b In the decoder, we only predict detail coefficients up to level j1- No detail coefficients are used at levels 1 S S j2- In this example, we have j1 consists of two upsampling paths, one for each spatial component of the curve. We have only drawn for notational convenience. During training, only the approximation coefficients at the lowest and highest resolution levels (most right curves in blue) are supervised.
+ and j2 jo 3. In reality, the decoder
3.1 NETWORK ARCHITECTURE
To formulate our objective, consider image r E X with an associated boundary 0R(z) of simply connected region R(x). We assume that 0R(z) is parameterized by a closed continuous curve (x)) of length ((x) 0. The objective is to compute the relevant approximation coefficients of 7" (ar), see the explanation in Section 2.3 where (x)) denotes the cut-off re-parameterization of (x)) using suitable neural network.
Our network hybrid analog of the U-net. It consists of an encoder, bottleneck and decoder with skip-connections in between. Only the approximation coefficients the lowest resolution-level jo are "directly" computed by the network (in the bottleneck). Afterwards, the Pyramid Algorithm takes over compute approximation coefficients higher resolution levels (the decoder). In prac- tice, the detail coefficients are negligible on sufficiently high resolution levels. For this reason, we only predict detail coefficients up to prescribed level j1- The predictions higher resolution levels j S j2 are computed without detail coefficients. The full architecture is summarized in Figure
3.2 LOss
The loss L consists of two parts: ordinary cross-entropy for optimizing the likelihood of p(z) and a part corresponding to the L2-error between observed and predicted curves on different res- olution levels. More precisely,
otherwise, then Lee (c((p()) r(z)) = r(x) log p(x) + (1 - r(x)) log(1 p(x)). Next, suppose ((l))))))) = (ajo (h> (())(()))))))))))))))())))))(( ...., a...,,;,( (h>" (x)]) are the approximation coefficients of (*(x) on resolution levels jo S j < j2. Define Cj (f, (r),,,(()) ",,a"""(())) (x))) = [x]]
r(x) 1 if R(x)) # and r(x)
(x))+)] for s E {1, and j E [Jo.j2]. Here (z) are the network predictions of the relevant approximation coefficients at level This loss-term corresponds to the error resolution level between the curves with
5

---

Under review as a conference paper at ICLR 2022
approximation coefficients [f,( and ag(l>" (z)].). Finally, define the total loss by ((F()). = wLo(l(()) + r(z)
Σ (1(((((r)),(((((())))
aE(1.2
je(Jo.Ja)
where w is a weight. Notice that L measures the discrepancies between observed and predicted curves on the lowest and highest resolution levels only. In practice, this enforces the approximation
and detail coefficients at intermediate levels to agree as well; see the experiments in Section 5
TRAINING
In this section we describe the dataset on which we test our method. addition, we provide details about preprocessing steps and model development (training).
4.1 DATASETS
Toy dataset The main purpose of the toy-example is to create a setting in which the annotated contours differ substantially from annotations confined to grid. For this purpose, consider piecewise smooth curves having a finite number of non-differentiable points. The toy-dataset con- sists of hypocycloids, up an Euclidian motion and scaling, defined by
rt() = (r1 r2) cost t + cos
11 r2t
(r1 -r)- 72) sin 72 sin
r1 12
r2
T2
where 12 and E R. If ti E N, then vi is closed and has exactly rg points). To casily control the number of cusps, we fix r2 and vary T1. Note that in this case has 71 cusps and period 2n.
cusps (non-differentiable
ra
We construct curves and binary masks of various sizes, orientation and positions, by sampling radius 71 E f3, 4, 5, 6] from 4((3,44 4, 5, 5,)), angle 8 from u([- 2 7D, components q1, Q2 from u([-80, 80)) for shift, and scaling factor from 2(110, 20)). Here (II) denotes random variable uniformly distributed on I, where is an interval of finite length or a discrete finite set.
Next, we evaluate the curve x (R(0)n + [160 + 2 160 160+ ual") on an equispaced grid of [0, 2x]
of size 512. Here R(O) corresponds an anti-clockwise rotation around the origin with angle 0. Finally, the discretized curve is used to construct a binary mask of size 320 x 320 using SKIMAGE.
Medical decathlon The data used to evaluate the performance of our model consists of MRI images of the prostate central gland, henceforth abbreviated as just the prostate, and CT scans of the spleen. The datasets are part of a public dataset made available for the Medical Decathlon Contest Simpson et al. (2019)- The dataset for the prostate consists of T2-weighted MRI images of size 320 x 320, which were cropped to size 224 x 224. The dataset for the spleen consists of CT scans of size 512 x 512 and was cropped to size 256 256. The cropping was based constructing bounding boxes of the form [umin - 8p. Mmax + _pp x [Umin - 6p. Umax + 6pl for the training set, where Umin; Umin; Umux and Umax are the minimal and maximal coordinates of the segmentation in each direction, respectively, using an offset of 5p 65 pixels. A residual CNN (encoder of five blocks) was trained (and validated) on the training set regress the corner and center points of the bounding boxes using a RMSE-llss. This rather crude approach is not meant to produce tight bounding boxes, but serves rough necessary localization step improve performance, and allows us focus on the task of shape- prediction only.
4.2 CONSTRUCTION GROUND-TRUTHE
In this section we describe how the ground-truth data is generated using the Pyramid Algorithm and Lemma A.I Let (Ir u) E X' x R be an image (slice) contour pair, where æ is slice of the CT or MRT scan, u is a finite sequence of points approximating a closed curve and n, = is the number of spatial components. Since we only have access to binary masks (for the public datasets), and not to the raw annotations themselves, we extract u using OPENCV. While not ideal, we stress that u contains "subpixel" information and is not constrainted to integer- grid.

---

Under review as a conference paper at ICLR 2022
Fourier coefficients To initialize the Pyramid Algorithm, compute the approximation coeffi- cients at level j1 using Lemma A.I To accomplish this, we need to compute a Fourier expansion for u. First, parameterize the contour by are length. The are length / is approximated by summing up the Euclidian distances between subsequent points in v. The Fourier coefficients are then computed by evaluating the contour interpolation and the Discrete Fourier Transform. Since the contours are real-valued, we only store the Fourier coefficients (7m) ) E (C**)A. Fourier coefficients that are too small, i.e., have no relevant contribution, are set to zero; see Appendix B.1 for the details.
equispaced grid of [0, of size 2.N 1, where N N, using linear
Consistency To have consistent parameterizations for all slices, ensure that is always tra- versed anti clock-wise (using openov), Furthermore, since the parameterization is only determined up to translation in time, need to pick out specific one. We choose the unique parameteriza- tion such that the contour starts at angle zero at time zero relative to the midpoint c = (c1, (C1,0) E R of R. The implementation details are provided in Appendix B.2
Approximation and detail coefficients Altogether, the above steps yield a contour 7 with Fourier coefficients W and period I. reparameteriz to have period 1, as explained in Section 2.3 and "center" the contour using the average midpoint computed over the training-set, The initial approximation coefficients at level j2 are then computed using Lemma A. Next, the Pyramid Algorithm is used to compute approximation and detail coefficients at levels jo S S j2 1. We set the detail coefficients which are in absolute value below E 5 10 to zero to reduce noise. Subsequently, we reconstruct the approximation coefficients at levels jo + S S j1 using the thresholded detail coefficients. No detail coefficients used compute approximation coefficients at levels ji + 1 S j2. The final approximation and (thresholded) detail coefficients and d, respectively, are used as ground-truth.
The resulting dataset 'D thus consists of tuples (x. a, d). The training and validation set Dirain and Dval= respectively, obtained by randomly omitting subjects from the the full dataset DD. For the toy-example, spleen and prostate we have (|Pmain), [Dval]) = (1650, 250), (|Pmain (Pvall) = (527, 75), and ((Dirin), (Dval) (3148, 502). respectively. Before feeding the images into the model, we linearly rescale the image intensities at each instance to [o. 1). Furthermore, for the spleen and prostate, we use extensive data augmentation: we use random shifts, random rotations, random scaling, elastic deformations, horizontal shearing and random cropping.
4.3 MODEL TRAINING
We use the Adam optimizer Kingma & Ba 2015) to train our network for 150 epochs. In the first five epochs, we use a linearly increasing learning rate from 5- 10 4 to 10-3 which is subsequently decayed by factor of 0.5 each time the loss does not significantly decrease for 10 subsequent epochs (for the remaining 145 epochs). A batch size of 16 samples is used in each descent step. Finally, the model with the lowest loss is selected. The computations were performed in PYTORCH on a Geforce RTX 2080 Ti.
5 RESULTS
In this section we examine the performance of our method and its dependence on the choice of basis. We fix all other hyper-parameters (as much as possible). We consider the Debauches wavelets dbp with E {1, 4, 8, 16) vanishing moments. Roughly speaking, dby corresponds to the unique MRA for which the mother wavelet has minimal support and p vanishing moments. For each basis, we fix appropriate resolution levels Jo, Jı and j2 as follows. We fix jo as the smallest possible resolution level for which the length of our signal (21o) still exceeds the length of the low-pass filter. Since dbp has a low-pass filter of length 2", this forces jo(p)
p. The choice for j1 2 jo is based on the observations that the norm of d, decreases as j increases and small detail coefficients have no significant impact anymore (see Figure [ We fix 2 jo the smallest level for which [[d],>ls| < 5- 10 for all samples in the training set (also see the discussion Section 4.2 The final resolution level je and the support of the mother wavelet determine how much of the contour is covered, see Figure We therefore fix j2 2 ji as the smallest resolution level for which the distance between the end-points of the curves are within 1 pixel distance for all samples in the training set.
7

---

Under review as a conference paper at ICLR 2022
Figure 5: Predicted and observed boundaries colored in red and orange, respectively, for the spleen and prostate for the best models (db8). The last two columns correspond to "hard" examples. Pre- dictions for the other wavelet bases, as well as the toy-example, can be found in Appendix D
We use 64 Fourier coefficients for the the toy-problem, spleen and prostate, respectively to initialize the approximation coefficients.
Baseline For the prostate and spleen use the U-Net in Ronneberger et al. (2015) and recently developed variant in Nikolov et al. (2018) for comparison. We have modified the network archi- tectures to match the parameter count with networks (approximately), while maintaining the structure and idea's presented in the original papers as much as possible, see Appendix We stress, however, that our objective, i.e., parameterizing contours, is different from the U-Net's objective. The binary ground-truth matched by a U-Net is a fundamentally different (often easier) object than the continuous representation of a curve matched by our networks. Subtle curvature and geometry may be accurately presented using our ground truth curves, e.g., by using a sufficiently large number of Fourier coefficients to compute approximation coefficients. Binary ground masks, however, cannot capture such subtle geometry due to their discrete nature.
Performance measures We evaluate accuracy using two-dimentn two-dimensional quantities only, since our models are 2d. We compute the component-wise 1.2-errors between observed and predicted curves on the highest resolution level j2 by taking the {2-norm of the approximation coefficients. Further- more, we compute the dice score and Haussdorf distance between curves using the implementation in SHAPELY. This requires polygonal approximation of the contour, which is easily obtained us- ing the approximation coefficients at level /2. Note that the Hausdorff distance between subsets of general metric space may differ from the Hausdorff distance between the associated boundaries (wavelet models), but coincide for (compact) simply connected subsets of R". The results are shown in Table [ The predictions for the best performing wavelet models with respect the Hausdorff- score are shown in Figure Predictions for other bases, corresponding wavelet decompositions, and detailed visualization statisties (violin plots) can be found in Appendix
Toy problem We observe that all models perform well and are capable of accurately parameter- izing piecewise smooth curves. For db1, however, we observe relatively large gaps between the end-points, since its support is relatively small. In addition, db1 has difficulty with accurately pre- dicting small "densely" sampled cycloids. In general, the predictions associated to the less regular wavelets dbl and db2 sometimes exhibit "small" oscillatory behavior. We found that the latter two issues were caused by a too large resolution level jz. To see why, note that the features extracted from the images only contain information up to a certain resolution level. The subpixel information needed to "fill in the blanks", the chosen wavelet basis. The regularity and support of the wavelet determines to which extent, i.e., up which resolution level, subpixel information be "filled in"" As the regularity (and support) of the wavelet decreases, the maximal achievable resolution level decreases as well.
to speak, is in part provided by the ground-truth data and in part by
Spleen The predictions of our models are accurate and on par with the baseline U-Nets. The more advanced U-Net in Nikolov et al. (2018) performs slightly better; this is mostly due to "edge" cases where the boundary of the spleen is small and about to disappear from our two-dimensional sliced- view. In such cases it may sometimes be ambiguous to define accurate ground-truth contour,
8

---

Under review as a conference paper at ICLR 2022
Table 1: Mean and standard deviation of various performance measures for the toy example, spleen and prostate. The standard deviation is reported in parentheses. The column N₂ is the approximate number of model parameters in millions, T is the inference time per image in milliseconds, and dbp- refers to network trained without detail coefficients. The length of the encoder for the toy-example six and five for the prostate and spleen.
Model
Un J1..2)
Dice
Hausdorff
1 (s 1)
1,2 (a 2)
N₂
T
dbl
0.962 (0. (0.0228)
(1.804)
(0.317)
(0.281)
7.70
15.6i
db2
(2, 6,8)
0.971 (0.021)
2.330 (0.889)
0.377 (0.172)
0.373 (0.161)
5.96
15.1
db1
(0.014)
(0.509)
0.261 (0.122)
(0.124)
5.45
14.5
db8
(4,6,8)
0.978 (0.016)
1.287 (0.520)
0.276 (0.149)
0.281 (0.162)
5.25
14.1
Toy
dh16
(5. 6,8)
0.978 (0.018)
1.489 (0.503)
0.252 (0.104)
0.254 (0.111)
5.26
14.0
db2-
[2, 8)
0.787 (0.036)
13.623 8.8)
2.908 (0.758)
3.015 (0.916)
5.11
14..9
db4-
0.873 (0.165)
7.222 (6.659)
1.230 (0.569)
1.213 (0.541)
5.11
14.3
dbs-
(4,
0.967 (0.013)
2,215 (0.845)
0.432 (0.170)
0.430 (0.177)
5.11
14.0
db
0.977 (0.018)
1.525 (0.516)
0. (0. 104)
0. 10))
5.11
14.0
dbl
(1.6,8)
0.926 (0.034)
6:203 (2.040)
2.643 (0.997)
2.015 (1.276)
9.92
11.6
db2
(2,5,8)
0.943 (0.030)
4,200 (1.680)
1.798 (1.170)
2.113 (1.322)
5.62
10.5
db4
(3.5.8)
0,939 (0.033))
4.302 (1.637)
1.904 (1.096)
2.135 (1.086)
4.48
9.52
Sploen
db8
(4,5.8)
0.940 (0.036)
4.107 (1.572)
1.821 (1.073)
2.058 (1.272)
4.11
9.26
dh16
(5.5.8
0,940 (0.037)
4.175 (1.371)
1,880 (1.025)
1.968 (1.183)
3.98
9.92
db2-
[2. - 8
0.761 (0. (3.02)
14.611 (1.810)
T.341 (2.609)
6.511 (1.861)
3.97
9.77
dbd--
[3,
0.930 (0.041)
4.061 (1.592)
2.000 (0.768)
2 2,429 (1.129)
3.97
9.35
dbs-
0.938 (0.012)
1:205 (2.303)
1.990 (1.455)
2.155 (1.315)
3.98
8.99
db16-
0.939 (0. 035)
4.058 (1.517)
1.798 (0.881)
1.900 (1.007)
3.98
79
Ronneherper
0.952 (0.040)
4.389 (4.913)
7.76
38.9
Nikolov al. 2018
0.948 (0. 038)
3. 584 (1. (17797
6,16
40.7
db1
51.5.6)
0.931 (0. 033)
5,328 (2.387)
2.369 (1.2699)
2. 107 (0.8013
8.38
11.2
db2
(2.
0.930 00.
(2.647)
2.148 (1.300)
2.120 (0.788)
5.07
9.66
db44
(3
0.935 (0.032)
5.333 (2.572)
2.026 (1.242)
2.005 (0.894)
4.17
8.64
db8
(4, 5.
0.931 (0.032)
5.323 (.484)
2.147 (1.381)
2.019 (0.873)
3.87
7.50
Prostate
dh16
(5,5,
0.924 (0.040)
5.583 (2.665)
2.218 (1.250)
2.197 (0.994)
3.74
7.65
db2-
0255
(4.487)
(1.917)
(1.998)
3.73
dh4
(3,
0.921 (0.037)
6,040 (3.264)
2.353 (1.316)
2.328 (1.046)
3.73
7.84
db8-
(0.041)
(3.380)
2.334 (1.485)
2.328 (1.008)
3.73
7.79
db16-
[6.
0.928 (0. (32)
5.199 (2.901)
2.177 (1.273)
2.078
(0.784)
3.71
7.56
Ronneberger 2
0.932 047)
5.673 (2-402)
7.76
37.2
NiKolov al. 2018
0.937 (0.030)
5.175 (2.380)
5.19
32.1
resulting in curves with subtle spurious curvature. Such geometry not (and cannot be) present in the binary ground-truth mask due its discrete nature.
Prostate central gland The wavelet models produce accurate predictions and are on par with the U-Nets. Our models perform slightly better in terms of Hausdorff distance. While all models produce accurate predictions for most examples, there are instances where both the wavelet models and the baseline U-Nets fail to produce accurate predictions; see the last two columns of Figure 5 In these examples, the detail coefficients associated to parts of the curve with high curvature are too small in magnitude to be accurately predicted. While detail coefficients of such small magnitude were less relevant in the latter two examples, they are important for the prostate.
Ablation study (no detail coefficients) To demonstrate the importance of the detail coefficients, we have trained models without them, i.e., without the conneconnetio In this set up, we do not supervise the predictions on the lowest resolution level during training. The results demonstrate, as expected, that the lower-order wavelets db1, db2 perform significantly worse without detail coeffi- cients. In fact, for db1 our model failed to produce any sensible approximations that can be evaluated and were therefore omitted. A small drop performance is observed for db4. In general, for db16 there is not much gain, with respect to accuracy, memory-footprint and inference time, to explicitly model the detail coefficients.
CONCLUSION
We have introduced a novel method to model boundaries of two dimensional simply connected domains using wavelets and MRAs. In effect this allows for subpixel segmentations. The efficacy of the method has been demonstrated by modeling the boundaries of hypercloids (toy-example), spleen and prostate, demonstrating that the results are on par with typical U-Nets, yielding up to five times faster inference speed.

---

Under review as a conference paper at ICLR 2022
REFERENCES
P. Abry and P. Flandrin. On the initialization of the discrete wavelet transform algorithm. 1EEE Signal Processing Letters, 1(2):22-34, 1994. doi: 10.1 109/97.30031.
David Acuna, Amlan Kar, and Sanja Fidler. Devil is in the edges: Learning semantic boundaries from noisy annotations. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recog- nition (CVPR), pp. 11067-11075, 2019. doi: 10. 10.1009/CVPR.211..0113...
Arantxa Casanova, Pedro O. Pinheiro, Negar Rostamzadeh, and Christopher J. Pal. Reinforced ac- tive learning for image segmentation. In International Conference on Learning Representations, 2020. URL https: /ttp:::///peezeviii net htt.//ooummmiii SkgC6TNFVV
Jianxu Chen, Lin Yang, Yizhe Zhang, Mark Alber, and Danny Z Chen. Combin- ing fully convolutional and recurrent neural networks for 3d biomedical image seg- mentation. In D. Lee, M. Sugiyama, U. Luxburg, I. Guyon, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 29. Curran Asso- ciates, Inc., 2016a. URL
neuris...eeepssp.. httppp/.cc.ppppe/21///551 pdf
Liang-Chich Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, and Alan Yuille. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE Transactions on Pattern Analysis and Machine Intelligence, PP, 06 2016b. doi: 10. 2000///A...20112269999000
Siang-Chieh Chen, Yukun Zhu, George Papandreou, Florian Schroff, and Hartwig Adam. Encoder- decoder with atrous separable convolution for semantic image segmentation. In Vittorio Ferrari, Martial Hebert, Cristian Sminchisescu, and Yair Weiss (eds.). Computer Vision - ECCV 2018, pp- 833-851, Cham, 2018. Springer International Publishing. ISBN 978- 978-3-030-01224-2.
Xu Chen, Bryan M Williams, Srinivasa R Vallabhaneni, Gabriela Czanner, Rachel Williams, and Yalin Zheng. Learning Active Contour Models for Medical Image Segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 11632- 11640, 2019.
Jun Fu, Jing Liu, Haijie Tian, Yong Li, Yongjun Bao, Zhiwei Fang, and Hanqing Lu. Dual attention network for scene segmentation. pp. 3p.3144 3141-3149, 06 2019. doi: 10.1109/CVPR.20199 1000900
Loukas Grafakos. Classical Fourier Analysis, volume 249 of Graduate Texts in Mathemat- ics. Springer New York, New York, NY, 2008. ISBN 9780387094311.
doi: 10.
1007/978-0-387-09477 URL htttp:///wwi spr ieppringeee inger ink, com/
978 0 307-388773322
Ali Hatamizadeh, Deblcena Sengupta, and Demetri Terzopoulos. End-to-end trainable deep active contour models for automated image segmentation: Delineating buildings in aerial imagery. In European Conference on Computer Vision, pp. 730-746. Springer, 2020.
Ali Hatamizadeh, Yucheng Tang, Vishwesh Nath, Dong Yang, Andriy Myronenko, Bennett Land- man, Holger Roth, and Daguang Xu. Unetr: Transformers for 3d medical image segmentation, 2021.
Kaiming Hc, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE Computer Society Conference Computer Vision and Pattern Recognition, volume 2016-December. pp. 770-778. IEBE Computer Society, dec 2016. ISBN 9781467388504. doi: 10.1109/CVP.22016.009 URL htttp://imaqennet.
Xiaoling Hu, Fuxin Li, Dimitris Samaras, and Chao Chen. Topology-preserving deep image segmentation. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché- Buc, Fox, and R. Gamett (eds.), Advances in Neural Information Processing Systems, volume 32. Curran As- sociates, Inc., 2019. URL ht http:///eee..........eoeo..eeaap 2
10

---

Under review as a conference paper at ICLR 2022
Fabian Isensee, Paul F Jaeger, Simon A A Kohl, Jens Petersen, and Klaus H Maier- Hein. net: a celffconnigriing method for deep learning based biomedical image segmentation. Nature methods, 18(2):003 211, February 2021. ISSN 7091. doi: 10. 10.1038/44152--00-0000093 URL http::::///ooooogg////.o///
038
Simon Jégou, Michal Drozdzal, David Vazquez, Adriana Romero, and Yoshua Bengio. The one hundred layers tiramisu: Fully convolutional densenets for semantic segmentation. In 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 1175-1183, 2017. doi: D0.1109/VVPR.2017.10
A. Khoreva, R. Benenson, M. Omran, M. Hein, and B. Schiele. Weakly supervised object bound- aries. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 183- 192. Los Alamitos, CA. USA, jun 2016. IEEE Computer Society. doi: 10..100//VP..2211...6 URL https: wwweeeeewteeeoooe............... 1.0///
Diederik P. Kingma and Jimmy Lei Ba. Adam: A method for stochastic optimization. In 3rd International Conference on Learning Representations, ICLR 2015 Conference Track Proceed- ings. International Conference on Learning Representations, ICLR, dec 2015. https:
// //arxiva org/bbs/ 1412. 6980v9
Jeremy Maitin Shepard, Viren Jain, Michal Januszewski, Peter Li, and Pieter Abbeel. Combinato- rial energy learning for image segmentation. In D. Lee, M. Sugiyama, U. Luxburg. I. Guyon, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 29, Curran As- sociates, Inc., 2016. URL https: http:pp:://prrceeed.ngg. neur (wwe:44440077277454880
neur ips. wwwppcer.ppaceee/eecce peaper/2016/f11e. pdf
Stphane Mallat. Wavelet Tour of Signal Processing, Third Edition: The Sparse Way. Academic Press, Inc., USA, 3rd edition, 2008. ISBN 0123743702.
Diego Marcos, Devis Tuia, Benjamin Kellenberger, Lisa Zhang, Min Bai, Renjie Liao, and Raquel Urtasun. Learning deep structured active contours end-to-end. In Proceedings of the IEEE Con- ference Computer Vision and Pattern Recognition (CVPR), pp. 8877-8885, 2018
Kuo Men, Huaizhi Geng, Chingyun Cheng, Haoyu Zhong, Mi Huang, Yong Fan, John P Plastaras, Alexander Lin, and Ying Xiao. Technical note: More accurate and efficient segmentation of
organs-at-risk in radiotherapy with convolutional neural networks cascades. Medical physics, doi: 20.1002//p.132900 URL https::/
46(1):286 292, January 2019. ISSN 0094-2405.
europepme. . org/ aat.acatticcsseeeeeeeeeeeeeeesee PMC6322972
I. Montefusco and L.. Puccio. Wavelets: Theory, Algorithmss, and Applications. ISSN. Elsevier Science, 2014. ISBN 9780080520841. URL htttps://bboks..ooqggt Ln6ABOAAQBAJ
Stanislav Nikolov, Sam Blackwell, Ruheena Mendes, Jeffrey De Fauw, Clemens Meyer, Cían Hughes. Harry Askham, Bernardino Romera-Paredes. Alan Karthikesalingam, Carlton Chu, Dawn Carnell, Cheng Boon, Derek D'Souza, Syed Ali Moinuddin, Kevin Sullivan, Deep- Mind Radiographer Consortium, Hugh Montgomery, Geraint Rees, Ricky Sharma, Mustafa Su- leyman, Trevor Back, Joseph R. Ledsam, and Olaf Ronneberger. Deep learning to achieve clini- cally applicable segmentation of head and neck anatomy for radiotherapy. CoRR, abs/ abss/880.... 009.04430, 2018. URL httt:://p//aiiiii org/ abs 1809. C4430
Sida Peng, Wen Jiang, Huaijin Pi, Xiuli Li, Hujun Bao, and Xiaowei Zhou. Deep snake for real- time instance segmentation. In 2020 IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pp. 8530-8539, 2020. doi: 10.1109/VVR24260022200000
M.C. Pereyra and L.A. Ward. Harmonic Analysis: From Fourier to Wavelets. IAS/Park city mathematical subseries. American Mathematical Society, 2012. ISBN 9780821875667. URL
htttss:/bbooks attps://books..oggee...b//b/
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomed- ical image segmentation. In Nassir Navab. Joachim Homegger, William M. Wells, and Alejan dro F. Frangi (eds.). Medical Image Computing and Computer-Assisted Intervention MICCAI 2015, pp. 234-241, Cham, 2015. Springer International Publishing. ISBN 978-3-319.245744.

---

Under review as a conference paper at ICLR 2022
Amber L. Simpson, Michela Antonelli, Spyridon Bakas, Michel Bilello, Keyvan Farahani, Bram van Ginneken, Annette Kopp-Schneide Bennett A. Landman, Geert Litjens, Bjoern Menze, Olaf Ronneberger, Ronald M. Summers, Patrick Bilic, Patrick F. Christ, Richard K. G. Do, Mare Gollub, Jennifer Golia-Pernicka, Stephan H. Heckers, William R. Jarnagin,, Maureen K. McHugo, Sandy Napel, Fugene Vorontsov, Lena Maier-Hein, and M. Jorge Cardoso. A large annotated medical image dataset for the development and evaluation of segmentation algorithms. 2019.
Nuo Tong, Shuiping Gou, Shuyuan Yang. Dan Ruan, and Ke Sheng. Fully automatic multi-organ segmentation for head and neck cancer radiotherapy using shape representation model constrained fully convolutional neural networks. Medical Physics, 45, 08 2018. doi: 10. 10.1002/mp.133 13147.
Wenxuan Wang, Chen Chen, Meng Ding, Hong Yu, Sen Zha, and Jiangyun Li. Transbts: Multi- modal brain tumor segmentation using transformer. In Marleen de Bruijne, Philippe C. Cattin, Stéphane Cotin, Nicolas Padoy, Stefamic Speidel, Yefeng Zheng, and Caroline Essert (eds.). Med- ical Image Computing and Computer Assisted Intervention MICCAI 2021, pp. 109-119, Cham, 2021. Springer International Publishing. ISBN 978-3-030-87199
Yueyue Wang, Liang Zhao, Manning Wang, and Zhijian Song. Organ at risk segmentation in head and neck ct images using a two-stage segmentation framework based on 3d u-net. IEEE Access, 111444991-444600 2019. doi: 10.1 10.1109/ACCESS.209922 2019..9499999
Yuxin Wu and Kaiming He. Group normalization. In Vittorio Ferrari, Martial Hebert, Cristian Sminchisescu, and Yair Weiss (eds.). Computer Vision - ECCV 2018, pp. 3-19, Cham, 2018. Springer International Publishing. ISBN 978-3-030-01200
Jimei Yang, Brian Price, Scott Cohen, Honglak Lee, and Ming- Hsugn Yang. Object contour detec- tion with fully convolutional encoder-decoder network. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 193-202, 2016. doi: 10.1109/CVPR.201..22
Yundong Zhang, Huiye Liu, and Qiang Hu. Transfuse: Fusing transformers and cnns for medical image segmentation. In MICCAI, 2021.
Zheng, Lu, H. Zhao, X. Zhu, Z. Luo, Y. Wang, Y. Fu. J. Feng, T. Xiang, P. S. Torr, and I.. Zhang. Rethinking semantic segmentation from a sequence-to-sequence perspective with transformers. In 2021 JEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6877-6886, Los Alamitos, CA, USA, jun 2021. IEEE Computer Society. doi: 10.11099CVPPR4443372222222
URL https:/ /opp:://dol: /oi. wec.pppteeeccottttttiittttto.oot 10. N0.1109/CVPR464577 00681
A THEORETICAL RESULTS
In this section we prove theoretical results regarding the computation of wavelet decompositions. In addition, we provide all the implementation details.
A. EXPLICIT EXPRESSION FOR 0.jk
To address the issue that periodic signals are not contained in (2R)R), we consider the cut-off 5 -- this section we derive an explicit formula for (5: (2x)) by exploiting the periodicity of 7. In particular, we will quantify the claim that (scaled) sample values of 7 may be used as approximation coefficients. For this purpose, we first review some facts about the scaling equation. If we take the Fourier transform of both sides of the scaling equation, we obtain the relation (${)) H(S) for E R. where HE)) refinement mask associated to h. Throughout this paper we assume that h is finite, in which case has compact support and H is trigonometric polynomial of period 1. It is beyond the scope of this paper to discuss the properties of Н in detail and refer the reader to Pereyra & Ward (2012); Montefusco & Puccio (2014). Here we only need the following result. Under suitable conditions, one may iterate the equation for &p and show that (E)
+ Etez -12gk The map H is referred to as the
I H( for all & R.
12

---

Under review as a conference paper at ICLR 2022
Lemma A.1 (Initialization approximation coefficients). Let h (Z) be a tow-pass-fiite defining a MRA on (R) and H the associated refinement mask. Assume h is non-zero for only a finite number of coefficients, supp(yp) [0, [] for some > 0, and yp is bounded. Furthermore, suppose that ((E) I (())) for all & R. y (0,,(()))) is a l-periodic map with Fourier coefficients (Tmmmez then
(g))
((,9)))=227 =
Σ Ymm
II
H
(1)
125+a
mEZ
n=1
where w(1) == 25 is the frequency %. any j E Z and k E E[[ [[-221] 1,,,,1221111 -]}. 1
12 - }}} be arbitrary. A change of variables shows that Proof. Let E Z and k E [[-221].....
(, 9pa) 2 / 5 (2 k)) (()) dt, E. 10.8
since supp(y;) C [0, 3). In particular, note that 2 2 + k)) E 1- for t [0, since & E ([-22]]..... (2 - }]}. Therefore, we may plug the Fourier expansion for yy and compute
[ 10.3
7 (2- k)) x0() di 10,81 meZ
ma (1)m t+k (0(1) dt.
Next, note that that series inside the integral converges pointwise to y (2- + +)) ptt) on [0, (]. Furthermore, the partial sums can be bounded from above on [0, 3] by a constant, since y E (10, ()) and to is bounded. Therefore, may interchange the order of summation and integration by the Dominated Convergence Theorem:
1 Emme
e im(lmm Lik D p(t) dt
Eimer ewmm /
culliim pp() xp(t) dt.
10,01 MEZ
[0.00
mEZ
Finally, changing the domain of integration to IR again, that
Tmm (mll)m & 23
jull)m
p(t) dt =
Σ Mm (wl)
(-12) 121
m
10
MEZ
mE
The stated result now follows from the assumption that
- - (1 for any R.
Remark A.2. The bounds [-22t] and 12'1 3 3] are the smallest and largest integer, respectively, for which 2 5- k) E [- for all & 3). The bounds on k are somewhat artificial, however, since the argument may be repeated for any truncation of 7 on [-nl, nll, where n E N. This observation is reflected in the righthand-side of (T), which is well-defined for all k Z (but not snuared-summable The reason for choosing this particular truncation is to identify a minimal number of approximation coefficients needed cover the full signal 7/
H( for & E R is satisfied for all MRA's considered this paper. In particular, we remark that & is continuous at g = and (00) = It follows from these observations and Lemma A.1 that a (7) (5)
The requirement that pp() = II
= a)x(7) z 2 2k(k2 3) for j sufficiently large and k
constrained to [o{[ [[-2
- }]}. -.}
A.2 PYRAMID ALGORITHM IMPLEMENTATION DETAILS
In this section we provide the computational details for how compute approximation and detail coefficients using the Discrete Fourier Transform (DFT). In the following arguments all sequences are assumed to be two-sided. We will frequently abuse notation and write that a finite sequence an element in C" or RR" What we actually mean by this, is that we have two-sided sequence of length n which can be embedded in (Z) by appropriately padding with zeros. In such situation we will explicitly specify the indices of the sequence that the intended ordering is clear. Similarly, all operators in this section are implicitly assumed to be defined on the associated (two-sided) sequence spaces, even though we may write that they are defined on C R..
13

---

Under review as a conference paper at ICLR 2022
Let (ax) RRN and g (gx) M-1 k=1- E R be arbitrary. Here may be interpreted as sech-pass-filtee of length M with N 2 M 2 2. Similarly, we may think of a as the approxima- tion coefficients of 1-periodic signal
specific resolution level with N 2/-1, explained in Section 2.3 In this section, however, we will not emphasize these interpretations, e.g., write Ojk instead of dg. to avoid clutter in the notation. Observe that the righthand-side of (T) is a 2 -Periiodic sequence for 1-periodic signals. Therefore, to properly deal with "boundary terms", we will use the 2N -peroodic extension E R of a to evaluate discrete convolutions.
Convolution and multiplication of trigonometric polynomials To compute the detail and ap- proximation coefficients, we need to evaluate expressions of the form
(ā * g)k
Ike = Σ 1kg SM-1
āk-k29k2
(2)
SM-1 K,E2
for N S k S N 1. Note that although à is an infinite 2.N 2N-periodic sequence, the series in
contains only finite number of nonzero-terme since gk:, = 0 for |K2| 2 M. Furthermore, for k v N - 1, we do not need the full periodic extension à but only a partial (finite) extension (x(a), where Pk: C2N - CK is defined by
ak+2N
1 N - M MSkS --11
(Pk(a))a==
Ok
-NSkSN-1,
OK-2N
N N
and K 2(N M - 1). That is, (ā g)x = (Px(a) g), for -N -
We use standard arguments to compute (a(a) Namely, we interpret Pk as the Fourier coefficients of ur, where u, v R C are the trigonometric polynomials defined by
$
(0) Σ (Pr(a))ackk), v(0) Σ k=1-M M
ghe ik0
The product ue a trigonometric polynomial with K = + 2(M - 1) non-zero coefficients K
K - The coefficients of wv can be characterized by evaluating it on K distinct points in C. After fixing K such points, we may go back and forth between value and coefficient representations of u, v and uv using the isomorphism defined by the evaluation operator.
corresponding to terms of order
2 <
Evaluation at the roots of unity We evaluate and v, in the complex variable 6. the K- roots of unity. To do this, we first extend Px(a) and y to sequences in CK by padding with zeros. More precisely, define Zoren CK - C by (Zoven(b) = by for
K 2 s k - K and zero for 2. Similarly, define Z C2M-1 C by (b)) by for
K < k < - and K
< k < K.
2
|k| < M and zero for - < k < - M and M < k < 2.
K
K We can now evaluate and at the K -tth
roots of unity by computing DFT K o Sg 7 (x(a) and G g DFT & o Sg 2
respectively, where Sg CR Ck is defined by
bu
k 1, 0 05k k < 2
(Sxb)a
b
B < k < K 1
Consequently, ur can be evaluated at the K-th roots of unity by taking the element-wise product of the latter two vectors. Finally, the desired coefficients (Px(a) *g) are obtained by going back to coefficient space using the inverse DFT, i.e.,
(Px(a) * N-1 Is DFT-1
(GR @ DFT Sg o Z a Pre(a))
Here O denotes the Hadamard-produet and II CK - C2N is the truncation operator defined by
II (b) (bx N=
14

---

Under review as a conference paper at ICLR 2022
B PREPROCESSING
In this section we provide the details of our preprocessing steps.
B.1 TRUNCATION FOURIER COEFFICIENTS
The magnitude of the approximated Fourier coefficients will typically stagnate and stay con- stant (approximately) beyond some critical order, since all computations are performed in finite (single) precision. We locate this critical order (s(s) if present, by iteratively fitting the best line, in the least squares sense, through the points m. [[[mm]m)mmm I,) moo S m S N 1 for 1 S mo S N - 1. We iterate this process until the residual is below a prescribed threshold 6y > 0. In practice, we set Ón = 0.1. The Fourier coefficients with index strictly larger than my (s) are set to zero.
for each component s E (1,2),
{
B.2 CONSISTENT PARAMETERIZATIONS
To have consistent parameterizations we enforce that all contours start at angle zero at time zero rela- tive to the midpoint (2), E 22 of the region of interest R. This is accomplished by exploiting the Fourier representation of the curve. More precisely, let l == E MilSN1 c where
(() 2 be the contour with Fourier coefficients T = (5 > 1 The midpoint of the region enclosed by & is given by
C3 X(R) J us dA(u1, u2) = (-1) (T) * [T)2 (T')=) ([T') * [T]2)
S (1,2)
(3)
by Green's Theorem. Here À denotes the Lebesgue measure on IR,, [T), are the Fourier coefficients of [.] and (I'), == im for |m| S 1. We can now compute the desired parameteriza-
tion by determining E [0, such that arccos [5(-T) -11
and defining 71) = 7(1 -))-7).
15(-T) - c|l2
While T can be easily found using Newton's 's method, it suffices in practice to simply re-order from the start, before computing the Fourier coefficients. More precisely, we first define shift y by
[
c/1
np-1
Uk = Yk mod ny for 0 < k S np 1, where 1 == argmin arccos
[18k
c
km0
C U-NET ARCHITECTURES (BASELINES)
In this section we describe our modifications to the networks presented Ronneberger et al. (2015)
and Nikolov et al. (2018). The goal of the modifications is to match the parameter count with our networks (approximately), while maintaining the structure and idea's presented in the original papers as much as possible. Our modifications to the original U-Net in Ronneberger et al. (2015) minimal: we use 32 filters in the first layer instead of 64 and use group normalization with four groups. Our modifications to the network in Nikolov et al. (2018) are follows. We use residual blocks of length two (instead of three), we use a bottleneck with 256 channels (instead of 1024), and do not use 1d convolutions in the "third" direction (to keep model 2d). Furthermore, an encoder and decoder of length six. The number of filters used in the encoder, from top to bottom, 32, 32, 64, 64, 128, and 128, respectively. The number of filters used in the decoder, from bottom to top, is 128, 128, 64, 64, 64 and 64, respectively.
15

---

Under review as a conference paper at ICLR 2022
D FIGURES
In this section we provide additional visualizations of the statistics, wavelet decompositions and predicted contours.
D.1 TOY EXAMPLE
0.90
600
0.80
0.75
d61
602
ob4
ob8
db26
db1
db2
cb4
:b8
db16
Moduls
Models
Dice
Hausdorff
Figure 6: Boxplots and visualization of approximate densities for the dice scores and Hausdorff distances for toy problem.
dbl
db2
db4
db8
(a)
Figure 7: Predicted curve (in red) for binary mask (in white). In order to visualize the predicted curve without too much clutter we have not depicted the ground-truth contour.
16

---

Under review as a conference paper at ICLR 2022
34
lgpne pine
61-
ONLT
5158
64
19
(a) db1
(b) db2
M
13c
2000 P00c
(c) db4
(d) db8
Figure 8: Predicted and observed wavelet decompositions of the first component of the contour depicted the fourth column Figure
17

---

Under review as a conference paper at ICLR 2022
D.1.1 SPLEEN
1.00
0.95
0.85
20-
0.80
10-
0.70
(a) Dice
(b) Hausdorff
Figure 9: Boxplots and visualization of approximate densities for the dice scores and Hausdorff distances for the spleen.
db1
db2
db4
db8
db16
(a)
Figure 10: Predicted and observed boundaries colored in red and orange, respectively, of the spleen. The last two columns correspond few "hard" examples.
18

---

Under review as a conference paper at ICLR 2022
Syme
040 06)
02
(a) dbl
(b) db2
KORTA 2514
0022 4014
(c) db4
(d) dbs
Predicted and observed wavelet decompositions of the first component of the contour depicted in the first column in Figure I0 For db4 and db8 the detail coefficients on higher resolution levels are too small to be accurately predicted, but have little impact on the accuracy of the final approximation.
Figure 11:
19

---

Under review as a conference paper at ICLR 2022
D.1.2 PROSTATE
12-
dhd
cb16
baseline
bastlbassline
(a) Dice
(b) Hausdorff
Figure 12: Boxplots and visualization of approximate densities for the dice scores and Hausdorff distances for the prostate.
dbl
db2
db4
db8
db16
Figure 13: Predicted and observed boundaries colored in red and orange, respectively, of the prostate. The last two columns correspond to "hard" examples.
20

---

Under review as a conference paper at ICLR 2022
Dat de.1
5148
$133
(a) db1, fifth column
(b) db2, fifth column
NETO
290
10
15
15
14
Nprsa an1
-12
(c) db1, sixth column
(d) db2, sixth column
Figure 14: Predicted and observed wavelet decompositions of the first component of the hard exam-
ples depicted in [a] (b) the fifth column and (c) (d) the sixth column of Figure 13
21


Reviews:
[{"review_id": "r-mwLsub9sk", "reviewer": "Reviewer_w2LA", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n1. It is the first work to construct pixel-independent representations of curves instead of pixel-based output, which provides new insight into using wavelet analysis on segmentation tasks.\n\n2. The proposed method achieves up to 5x faster inference speed and competitive scores compared with a U-Net model.\n\nWeaknesses:\n1. The presentation of mathematic is complex and hard to read due to many notations. Also, it can be quite hard for readers to obtain a clear overview of the learning pipeline. It'd be better to demonstrate with figures.\n\n2. It lacks the details of the decoder in the model, especially how the reconstruction process is performed in the decoder. It can be difficult to reimplement the model. It'd be better to demonstrate with figures.\n\n3. The experiment results are not convincing enough. More experiments on other datasets are needed. The improvement of the proposed method is not obvious as compared with the baseline model. It lacks explanations on the reason for using these (j1,j2,j3) settings in the experiments. \n\n4. It lacks explanations on the reason and motivation that it conducts wavelet analysis on the boundary instead of the object itself. \n\n5. Some sentence is ambiguous.\nIn the first paragraph of Section 3, ‘…is henceforth assumed to have two components denoted by [y(x)]1 and [y(x)]2’ how is the two components generated?\n\n6. Some typos are listed below.\nIn the abstract, ‘…the upsampling path is a one-dimensional decoder’, ‘a’ should be ‘an’.\nIn the second paragraph of subsection ‘detail coefficients’, ‘…states that the subspaces Wj can too be spanned by dilating and shifting a single map’, ‘too’ is redundant.\nIn the third paragraph of section 3.2, ‘The encoder is followed by a a bottleneck …’, ‘a’ is redundant.\n", "summary_of_the_paper": "In this paper, the authors propose a novel deep learning framework for the fast prediction of boundaries. The boundary is modeled as smooth closed curves with wavelets and multi-resolution analysis. A U-shape model is trained to predict for coefficients of the curve. The proposed method is evaluated on medical images and achieves similar performances with up to 5x faster inference speed.\nThe main contribution is that it is the first to use wavelet analysis to construct pixel-independent representations of curves.\n", "summary_of_the_review": "The paper provides novel insight into medical image segmentation. The technical introduction is sufficient and makes sense. The experimental results support the theoretical part. \n\nIt can be hard to read and reimplement. The improvement over the baseline is not obvious.\n", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "aMrmr6ihHt", "reviewer": "Reviewer_MBzp", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strength: the use of wavelets and mulit-resolution analysis could be new.\nWeakness: the paper is not well presented. \n1) Section 2 could be simplified.\n2) The motivation of the paper is not clear. Why it is subpixel?\n3) The authors shall highlight their contributions and clarify the difference from the state-of-the-art algorithms.\n4) It is not clear if the use of wavelets and multi-resolution analysis is a significant contribution or not.\n5) I suggest the authors to draw a diagram of their network.\n6) The improvement seems marginal compared with U-net\n7) There are many methods for segmentation that shall be cited and compared.", "summary_of_the_paper": "This paper presents a method for segmentation using wavelets and Multi Resolution Analysis.\n But the presentation of the paper needs to be improved to highlight the technical novelty and difference from the state-of-the-art algorihtms.\n\n", "summary_of_the_review": "Given the weakness stated above, I felt that the paper is not strong enough.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "J18HBYrsAIn", "reviewer": "Reviewer_xHYe", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The article is well written although it is complex to follow.\nThe mathematical introduction is quite dense. Besides, there are some functions that are not defined (such as L(Z) and l(Z)), which makes it difficult to understand the article for those who do not know how Wavelets work. The state of the art is quite good given the space limitations although it would have been nice to go a bit deeper into the differences with respect to previous proposals (apart from the subpixel resolution).\n\n However, the main weaknesses of the paper are:\n1.- Although there are other similar approaches (and even with sub-pixel level segmentation) using wavelets the authors compare with a generic segmentation approach based on a classical encoder-decoder structure.\n\n2.- From the approach with which they compare, they fix the hyperparameters but do not justify why (and do not adjust them to improve the results). \n\n3.- In the comparison, the authors do look for the best hyperparameters of their approximation by comparing the best one with the (prefixed) approximation of the generic method. This dilutes the \"power\" of the results obtained. ", "summary_of_the_paper": "In this paper the authors propose a mixed approach to image segmentation using CNNs and wavelets. Although this approach is not new, to the best of the reviewer's knowledge, it is the first time it has been applied to medical image segmentation. It basically consists of applying an encoder-decoder structure in which the encoder is a CNN and the decoder is based on a traditional (Pyramid Algorithm) Wavelet method. To validate the proposed method, they compare their results with a general and complete CNN method. The results demonstrate that it could be a powerful approximation: fast, accurate and easy to train.", "summary_of_the_review": "Pros:\nThe paper is well written and the proposal is very interesting from the point of view of the medical image segmentation.\n\nCons:\nThe mathematical explanation must be improved to help the understandability of the paper. \nThe validation of the proposal must be compared with other wavelet proposals.\nAuthors should be fairer when adjusting the hyperparameters (not limited to the proposed model). ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "2mZV1l5IDL", "reviewer": "Reviewer_ZJwW", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n1) In contrast to the related literature, where pixel-based segmentations are provided, the output of the proposed method is a closed curve.\n\n\nWeaknesses\n1) The extensive presentation of multi resolution analysis could be avoided as the theory is well known. More experiments could strengthen the paper.\n\n2) A figure clarifying the network layers and the corresponding detail coefficients would be welcome to graphically explain the network architecture in section 3.2 and how the coefficients are associated with the skip connections.\n\n", "summary_of_the_paper": "The paper proposes a method for boundary extraction from images using a U-net type network architecture. The boundaries are considered as closed curves represented by multiresolution analysis (wavelets). The down-sampling part of the network consists of a 2D encoder and the up-sampling part is a 1D decoder that reconstructs the boundaries.", "summary_of_the_review": "Although an important part of the paper presents well-known theory, I like the idea of estimating closed curves and I suggest accepting the paper.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper describes an interesting approach to predicting continuous closed surface segmentations from discretized image data using a wavelet output representation. This is an interesting idea with a lot of potential. Unfortunately, the paper currently suffers from major weaknesses which we encourage the authors to address.

1. While the idea of generating a continuous output representation of a segmentation is technically interesting, what are some applications where this is actually useful?
2. The ground truth annotations in the datasets evaluated are implicitly quite variable. The annotations are not made to sub-pixel precision and there is likely to be large multi-pixel variability across different annotations of the same image. This makes a poor problem to demonstrate the need and potential of a sub-pixel accurate segmentation algorithm.

I encourage the authors to find applications and datasets where reliable sub-pixel ground truth annotations exist, and to demonstrate that their approach to generating sub-pixel segmentations is superior to appropriate baselines which also predict sub-pixel segmentations.
Decision: Reject