Minimum Width for Deep, Narrow MLP: A Diffeomorphism Approach
Authors: Geonho Hwang
Venue: NeurIPS 2025 poster ()
Keywords: Universal Approimation
continuous functions
deep narrow MLPs
deep learning
Primary area: theory
TL;DR: We present a geometric framework that reduces the problem of finding the minimum width for universal approximation by deep, narrow MLPs to a dimension-based function w(d_x,d_y) yielding tight upper and lower bounds under the uniform norm.

Abstract: Recently, there has been a growing focus on determining the minimum width requirements for achieving the universal approximation property in deep, narrow Multi-Layer Perceptrons (MLPs).
Among these challenges, one particularly challenging task is approximating a continuous function under the uniform norm, as indicated by the significant disparity between its lower and upper bounds.
To address this problem, we propose a framework that simplifies finding the minimum width for deep, narrow MLPs into determining a purely geometrical function denoted as $w(d_x, d_y)$. 
This function relies solely on the input and output dimensions, represented as $d_x$ and $d_y$, respectively.
To achieve this, we first demonstrate that deep, narrow MLPs, when provided with a small additional width, can approximate any $C^2$-diffeomorphism.
Subsequently, using this result, we prove that $w(d_x, d_y)$ equates to the optimal minimum width required for deep, narrow MLPs to achieve universality.
By employing the aforementioned framework and the Whitney embedding theorem, we provide an upper bound for the minimum width, given by $\operatorname{max}(2d_x+1, d_y) + \alpha(\sigma)$, where $0 \leq \alpha(\sigma) \leq 2$ represents a constant depending explicitly on the activation function.
Furthermore, we provide novel optimal values for the minimum width in several settings, including $w(2,2)=w(2,3) = 4$.

Minimum Width for Deep, Narrow MLP: A Diffeomorphism Approach
Geonho Hwang
Department of Mathematical Sciences Gwangju Institute for Science and Technology
Gwangju, Buk-gu 61005 hgh21348gist. ac.kr
Abstract
Recently, there has been growing focus determining the minimum width requirements for achieving the universal approximation property in deep, narrow Multi-Layer Perceptrons (MLPs). Among these challenges, one particularly chal- lenging task is approximating continuous function under the uniform norm, as indicated by the significant disparity between its lower and upper bounds. To ad- dress this problem, we propose a framework that simplifies finding the minimum width for deep, narrow MLPs into determining purely geometrical function de- noted as w wdd,,, d.y. This function relies solely on the input and output dimensions, represented as dz and dy, respectively. To achieve this, we first demonstrate that deep, narrow MIPs, when provided with a small additional width, can approximate any C (=-diffeomorphism. Subsequently, using this result, we prove that w (d dy)) equates to the optimal minimum width required for deep, narrow MLPs to achieve universality. By employing the aforementioned framework and the Whitney em- bedding theorem, we provide upper bound for the minimum width, given by max + 1. d,)) + (o)), where < a( < 2 represents a constant depending explicitly the activation function. Furthermore, provide novel optimal values for the minimum width in several settings, including w(2, 2) = wo(2, 4.
Introduction
The universal approximation property (UAP) denotes the capability of neural networks to approximate specific class of functions, forming the foundation for their efficacy and garnering significant interest in the research community. Among the extensively explored subjects is the investigation of the UAP of deep, narrow multilayer perceptrons (MLPs) characterized by restricted width and arbitrary number of layers. Given the practical application of MLPs involving modest widths and more than two layers, comprehending the UAP of such networks has emerged as a pivotal focus.
In this context, a series of papers has aimed to determine the minimum width, representing the neces- sary and sufficient width for achieving the UAP. The minimum width depends input dimension dz. output dimension dy. activation functions, and the employed norm. Out of various norms, one particularly important and challenging task is to determine the minimum width under the uniform norm. In this paper, our focus centers on the universal approximation of continuous functions under the uniform norm. The earlier findings concerning uniform approximation In general, research on determining the minimum width for approximations under the uniform norm, using continuous activation functions, suggests range between max(d,, + 1, d.y)) and d, + dy. This discrepancy highlights a significant gap.
summarized in Table
In this context, present novel upper and lower bounds for the minimum width required by deep. narrow MLPs to possess the UAP. Our support for these bounds unfolds in two steps. Initially, we
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

approximate diffeomorphisms using deep, narrow MLPs, building upon the concept of the UAP of invertible neural networks. Subsequently. we approximate arbitrary continuous functions through composition of linear transformations and a diffeomorphism. We characterize this process by revealing that the minimum width of Leaky-ReI.U MLPss corresponds to a geometrical function denoted as w(dg. d.y) This function represents the required dimension of diffeomorphisms necessary to approximate arbitrary continuous functions given an input dimension of de and an output dimension of dy.
Building upon these statements, we establish upper and lower bounds. By applying results from geometric topology, we prove that MLPs with width max(2d, + 1, dy) can approximate any continu- ous function. Additionally, by utilizing more refined results of topological algebra, we can improve this bound in certain specific cases. For example, when the input dimension is two and the output dimension is three, the optimal minimum width exactly four, and when both the input and output dimensions are two, the optimal minimum width remains four. These findings highlight the practical utility of our framework.
Our contributions can be summarized as follows:
We prove that deep, narrow MLPs with width of employing the Leaky-ReLU activation function, can approximate any Ce-diffcomorphiimm on IR Furthermore, for more general activation functions, we demonstrate that deep, narrow MLPs with width d + using ReLU and d + 2 employing a general activation function, respectively, can approximate any C² -diffeomorphisms uniformly on IR
We propose the purely topological quantity w(dg, dyy, representing the optimal minimum width for achieving the UAP of deep, narrow MLPs employing the Leaky-ReLU activation function.
Building upon the aforementioned results, we prove that deep, narrow MLPs with width of max + 1, dly) + a(r) can approximate any continuous function C(Rd, IR4) uniformly on compact domain, where 0 a(o) < 2 is constant depending the activation function.
We prove that when the input dimension is 2k and the output dimension is 4k 1. the optimal minimum width is 4k.
We demonstrate that a width of 4 is the optimal minimum width for deep, narrow MLPs to approximate arbitrary continuous function mapping [0, 1]2 to R.2.
2
Related Works
In this section, we explore previous research concerning the universal approximation property of neural networks. Initial studies primarily focused on two-layered MLPs. Cybeekk (1989) demon- strated that two-layered MLPs with sigmoidal activation functions possess the UAP for approximating continuous functions. Later, Leshno et al. (1993) expanded the scope of activation functions to more general ones.
Beyond two-layered MLPs, extensive investigations have explored the UAP of deep. narrow MLPs. For instance, [u et al. (2017) showed that deep, narrow MLPs with a width of d, + 4 and ReLU activation functions possess the UAP in Li ((0, (dd,,),, while they do not have the UAP with a width of dz. This early research paved the way for further studies that narrowed down the minimum width range and expanded the application scope of theories. Hanin (2019) extended the study encompass arbitrary output dimensions d,: The minimum width lies between d, + 1 and d, + dy for the ReI.U activation function. JJonson (2019) demonstrated that width of de insufficient to achieve the UAP in continuous function spaces for an activation function that can be approximated by increasing functions. Kidger& & Lyons (2020) proved that dimension of dz + dy + is sufficient for activation functions under weak condition, and d + dy + 2 is sufficient for polynomials. Furthermore, Cai (2023) explored the lower bound max(d,. d,,) of the minimum width for arbitrary activation functions and proved that the UAP can be achieved with max(dz, dy) using the floor function and an activation function having the universal ordering of extrema property. Tabuada & Gharesifard (2022) addressed the UAP of deep, narrow residual networks. Recently, Li et al. (2023) claimed that the upper bound
could be reduced to max (d, + 1, 1, + -d_d+1d,-- On the other hand, (Shen et al. (2022); Hong
& Kratsios (2024); Liu & Chen (024) treat the approximation rates of neural networks with low

---

Table 1: A summary of known results minimum width for universal approximation of continuous functions. denotes a compact domain, and "Conti." " short for continuous.
Reference
Domain
Activation
SipperLLoee Bounds
Hanin 2019
C(K, 12))
ReLU
d₂ 15 Wmin S dz dy Wmin d₂ +
Johnson 2019
C(K,R) IR)
uniformly conti. f conti. nonpoly
C(K,RQ,
Wmnn I d. + dy +
Kidger & Lyons 2020
C(K,
nonaffine poly
Wmni I d₂ + dy + 2
Park et
2021
C(O,, 1J.R)
ReLU
Wmin = 3 > max (d, 1, 1,dy)
Cai 2023
CRK,
Arbitrary
Wmin 2 max d2y. dyy
Kim et al.
2024
C(K, 1Rv)
uniformly conti.
Wmin dy + 1d, <d, S2d,
C(K,R%)
Leaky-ReLU
Wmin S mmmxx(2d 2d,- + dy) Wmin I maxx(2d + 1, dgy +
(R4)))
ReLU
((K,R)))
conti. nonpoly
Wmin max 2d 1, dg) +
Ours
C((0, 1000,,,,2,,-1-- 1R
Leaky-ReLU
Wmin = 4k
C(OO, R2)
ReLU
Wmin = 4
C(O,, R3)
Leaky-ReLU
Wmin 4
uniformly conti.
Wmin 2 4
Requires that a can uniformly approximated by a sequence of one-to-one functions. Requires that a is continuously differentiable at some point with o o'(z) + O.
width. In the recurrent setting (Song et al. 2023), investigations into minimal-widhe deep RNNs have demonstrated that similar universal approximation properties hold, broadening the scope of width-efficient architectures beyond feedforward models.
In addition to the uniform norm, Park et al. 2021) presented the optimal minimum width as max (d 1, dy) for deep, narrow MLPs using ReLO activation functions in Lp space. They demonstrated that this is not applicable for the uniform norm, establishing the optimal width for the UAP in C((o, ))?) as three. Kim al. (2024) proved that the lower bound equals exceeds dy + if dy is less than equal to 2d,, and on a compact domain, the minimum width for Lp space is max(dæ, dy. 2) when using the ReLU activation function. |Rochau al. (2024) provided an alternative constructive proof of the same result. HERNÁNDEZ & ZUAZUA proved that width 2 is sufficient in the classification setting.
In this paper, we prove that width of max(2d, + 1, dly) is sufficient for universal approximation. This provides an advantage in cases where d, is large compared to previous results, which always required an upper bound of approximately dN + dy. We also prove that when both the input and output dimensions are two, the minimum required width is 4, demonstrating that the max (dz + 1, dy -type results obtained in the Lp norm setting do not apply to the uniform norm. See Table for summary.
3 Notation and Definition
In this section, we introduce the notations and definitions utilized throughout this paper: IR represents the set of real numbers. R4, denotes the set of positive real numbers. N stands for the set of natural numbers, and No = N U (0). For a, b E R. ja, bj and (a, b) represent the closed and open intervals from b, respectively. Mn.m denotes the set of Mn,0 represents the set of invertible n X n-matrices. Aff and IAff, stand for the sets of affine transformations from I2 Rm and invertible affine transformations from R" to R respectively. For a d-dimensional vector x E Rd, I; denote the i-th component of æ; in other words, & = (x1, ..,..... Eg). Additionally, Zty represents the (j - i 1)-dimensional vector (zz, 24+1....,
real matrices with real inputs. GL(n) C
3.1 Compact Approximation
Let X and Y be metric spaces. C(X, Y) represents the set of continuous functions from to Y. For function f C(X,)) and non-empty set X' X. flx denotes the restriction of the function to
3

---

the domain X'. For a set of functions A C C(X,Y), Alx is defined as {flxx if E A}. We focus on the uniform approximation of continuous function on a compact set, defined as follows:
Definition 3.1. Given two subsets, A, B C CCR", R,),, say that A compactly approximates B if. for any B, a compact set K C R", and O, there exists g E A such that
fl - gll == supxek llf(x) g(x) <e
(1)
This is denoted as A
> B or B < A.
The compact approximation relation transitive: A B, and C, then, A > Additionally, we use the notation AA indicate that {ff A. For of functions A C(X,Y). A represents the closure with respect to the uniform norm.
3.2 Activation Function
We follow the commonly used condition for activation functions proposed by Kidger & Lyons (2020). Note that this condition permits a linearization of the activation function, as established in Lemma 4.1 of Kidger & Lyons (2020).
Condition 1. An activation function a is a Ca -fffunctiin near a E R, with d'(a) # O.
We define several activation functions that satisfy Condition 1
. ReLU: ReLU(x)>= == =10
if x 2 0
if <
i Br
Leaky-ReLU LL,
if <
Activation functions applied to vectors function as componentwise operators: For Rd, x == (o(2))...., o(T)))
3.3 Deep, Narrow MLP
A set of MLPs, denoted as No dg dx
dx is defined as follows:
No do..d....yy
Rd Rd W, Aff f(z) WN o..... 00.....00000000................. W1}. (3)
For Leaky- additional parameter vary for each layer, resulting in the ALR do.di.....d
MAR dx = WN o LR ................................ o......Rg.......o........... o WT d + Rdx W, E Aff 8 E R..) }. (4)
We define set of deep. narrow MLPs with input dimension d,, output dimension dy. and most intermediate dimensions as follows:
A
U
Ng.
NEÑ, 15d, dy
For natural numbers m N, we define the natural projection Pr.m m 1R - R and the inclusion Im.n as follows:
Pn.m (#1...... ........2..................... Fn) + (21,..., Em)),
(6)
and
Jm, (#1, (#i...., Im) (#1,.... Fny O, .,...,0) 0).
(7)
For any n 2 dzy dy and f A7 dendymn it can be decomposed as:
f = Pn.d, o g Qd,.ns
(8)
where E A nmn.n
Definitions of several geometric concepts including diffeomorphism and embedding, are provided in Appendix[A] We define the set of diffcomorphisms as follows:
Definition 3.2 (Diffcommrphism: D" (U)). Let Rd be open subset, and let be non-negative integer infinity. (U' is the set of C diffeomorphisms from U to Rd

---

Main Theorem
4.1 Problem Formulation
Our primary objective is determine the minimum width wmin such that for any compact set K C R", continuous function f E C(K, RT) can be uniformly approximated by A Mmm. In other words, we aim to determine the value Wmin (n, m, o) such that
Wmin (n, m, o) min (1 E N C(R", R") mm.m.
4.2 Approximating Diffeomorphisms and Continuous Functions
In this subsection, we begin by demonstrating the capability deep, narrow MLPs to approximate diffeomorphisms and aim to prove that any continuous function can be approximated by composing linear transformations and diffeomorphisms.
Lemma 4.1. Let a be a continuous function that satisfies Condition Then, for natural number d E N. the set A ddddd+a(o) compactly approximates D2 (RR"),, where
if o = Leaky-Red ReLU
a(o)= =
1
if a ReLU
(10)
2
if a Otherwise
In other words, we have the relation
(RR)) + D ddddd
(1I)
In Teshima et al. (2020), shown that any diffeomorphism can be approximated by compo- sition of single coordinate transformations. Therefore, it suffices to prove that deep narrow MLPs can approximate any such single coordinate transformation (see Definition A.5 for the formal defini- tion). This is established in Lemma B.I. With the exception of the Leaky-ReL. ReL case, the proof is relatively direct. For the Leaky-ReLU ReLU case, we progressively extend the class of functions that can be approximated. Using Lemma B.3] we show that any increasing scalar function can be approximated by width one Leaky-Rel.U networks. This result implies that any width-1 neural network with increasing activation functions can be approximated by a Leaky-ReL.U network of the same width (see Corollary [B.4). Building on this, we prove in Lemma B.5 that any ACF (see Definition (A.6) can be approximated by deep narrow MLPs. Finally, Lemma B.6 shows that any single coordinate transformation can, turn, be approximated by an ACF,
The detailed proof of Lemma is provided in Appendix
Now, quantify the required geometric width for approximation as wnn, m). Additionally, demonstrate that the network- inderendendyy defined value w(n, m) equals the minimum width of deep, narrow, Leaky-Rell ReLU MLPs.
Let Emb(X, Y) denote the set of smooth embeddings from to Y, and Emb,. (X, Y) represent the set of piecewise linear embeddings from X to Y. For natural numbers d1 2 d2, define Pd, Rd + d2 as the projection to the first d2 coordinates. And define w(n, m) as:
w(n, m)) min (1 E Po.Pt.m Emm([ 1j", R()) C(10, R"
(12)
Intuitively, wnn, m) represents the minimum width required to approximate any arbitrary continuous function using diffeomorphisms.
Remark 4.2. We note that the interval [0, 1] can be replaced with the interval [a, b) for < b. Additionally, Emb([0,1] 10,, RR)) can be replaced with any dense subset of Emb((0, 1]m, R.), such as Embp. (10, 1)nh "", IR') (Munkres 1960
We will prove that the difference between w(n, m) and w(n, m, σ) is bounded by two, and that w(n, m) has the same value as w(n, m, Leaky-Rel.U). The next lemma demonstrates that any smooth embedding can be represented by the composition of an inclusion and smooth diffeomorphism.
5

---

Lemma 4.3 (Theorem C of Palais 1960)). Consider natural numbers n and m where n S m, R... Then, there exists smooth diffeomorphism
and a smooth embedding f 'K= [0,
F Rm - Im such that the following equation holds:
F o Fn.m =
(13)
Using the preceding lemma along with the definition of w(nn, m), we can present the following theorem:
Theorem 4.4. Let a be continuous function satisfying Condition 7
Then, for natural numbers n and m, A (. compactly approximates C(R", R.),, where
if o Leaky-ReLU
a(r) =
if a = ReLU
(14)
2
if a Otherwise
In other words.
C(R",R" x A m, m(mmm,)))))((
(15)
Proof. Without loss of generality, assume that K [0, 1]a In cases where K differs, we achieve this by continuously extending the function to encompass a cube containing K and then rescaling, By the definition of w(n,m) m), for any f C([0, 1]", R and 0, there exists an embedding g E Emb((0, 1]", R=(mm)) such that
Ilf Pw(n.m),, g0.0o,(0,111 <C
(16)
Because w(n, m) > n. by Lemma 4.3 for (n, (I1,..., In) (I1 Iy. ..... 0), there exists smooth diffeomorphism G such that = G o )n) By Lemma 4.1 there exists an
H A (n(,.) ((m.))) such that
| KX (x,0,, <.
(17)
Then.
(P (n.w(n,)) p qn, loo 10.11 <.
(18)
Therefore,
|lf - Pu(n,m),m H In, oe 2e.
(19)
Pm( H. (n,w(n,,)) A (ft
This completes the proof.
O
Furthermore, we can give the lower bound for the minimum width required for the UAP.
Theorem 4.5. Let be an increasing, continuous activation function. For natural numbers and m in N, A (1 does not compactly approximate C(R", IRm)). ). In other words, the following relation holds:
C(R",R")
A A
(20)
The detailed proof is provided in Appendix
C.1
we conclude that the minimum width Wmin (n,, m, Becaky-ReLUj equals w(n, m) for Leaky-ReLU and provides tight inequality for gen- eral increasing activation functions.
Combining Theorem 4.4 with Theorem
4.5
Corollary 4.6. The following equation holds:
Wmin (n, m. Leaky-RelUU) = w(n, m)
(21)
For a general increasing activation function o, which satisfies Condition the following inequality holds:
w(n, m) S Wmin (n,, m, S w(n, m) + a(a),
(22)
where
1
if ReLU
alo)
(23)
2
if a Otherwise

---

4.3 Some Observations about the Upper bound of w(n, m))
In the previous subsection, we demonstrated the fundamental correlation between the minimum width of the deep, narrow MLP and w(n. m.) In this subsection, we will present sufficient condition for w(n, m) be equal m. The following lemma demonstrates that continuous function can be approximated by smooth embedding when the output dimension is greater than twice the input dimension:
Lemma 4.7. Consider natural numbers and where m >> 2n. Let f K [0, R" + R
be a continuous function. Then, for E R_,, there exists smooth embedding K - R" such that
f glo <c
(24)
Proof. Consider connected, open subset U of RR such that K U E". Since K is compact, there exists continuous extension fo of f such that
folk=
(25)
As is a manifold, it satisfies the assumptions of Theorems 3.17 and 3.18 Therefore, there exists an injective immersion g such that
Persson (2014).
|| 9 zo.u <..
(26)
Consequently, the restriction glæ defined on the compact set becomes a smooth embedding.
Now, we present an upper bound in the following theorem.
Theorem 4.8. Let be a continuous function satisfying Condition 7 Then, for any natural numbers n, m E N. the set A max
compactly approximates C(R", ))"), where
0
if a Leaky-ReLU
a(o)
1
if a ReLU
(27)
2
σ Otherwise
In other words. the following relation holds:
C(R" T x A (
(28)
Proof. Lemma 4.7 implies that w(n, m) max (2n 1,m). By Theorem 4.4 we can immediately
get the conclusion.
Remark 4.9. As previously demonstrated by Kim et al. (2024), the minimum width Wiinn Wmin(de dy. o) satisfies the relation Wmin 2 dy + 1d, <d, 52d, for an increasing activation function. indicates that when the output dimension dy twice the input dimension da, Wwin (dg. 2dz, σ) is equals or exceeds dy + 1. In the same configuration, following Theorem 4.8 we derive the relation: Wmin (dz, dy Leaky-ReLU) 2d, +1= dy + 1. By merging these resulis, we arrive at the optimal minimum width Wmin (dz, dy, Leaky-ReLU) = dy + +1 2d, +
Furthermore, we know that Wmin (de, dy σ) 2 max(de, for a general activation function Cai 2023). If dyy >> 2dg, then Wiin (dz, dy Leaky-ReLU) S d, presenting the optimal minimum width as Wminn (dz, dy. Leaky-RelUU)
dly
In addition to the aforementioned relation, there exists an evident upper bound for w(n, w),
wnn, m) S n + m, for n, m E N. This reaffirms the result presented by Hanin 2019 in the case of Leaky-Rel.U.
(29)
Wmin (nt m. Leaky-ReLU)) S n m,
(30)
For ReLU (Hanin 2019) and other general activation functions (Kidger & Lyons 2020), the results are slightly less favorable:
Wmin (n, m. I + + a(a).
(31)
where a0)) 1 for ReLU, and a(o) for other activation functions.
7

---

4.4 Some Novel Upper Bounds of w(n.
The framework of diffeomorphism not only gives the upper bound derived from Whitney embedding. By utilizzing sophisticated techniques of geometric topology, we can give some non-trivial optimal width including u=223) = 4. Note that it is strictly smaller than both of dg + +dy = and 2dz + 1 5.
Proposition 4.10. For even k, wk, 2k- 1)
2k.
Proof. The lower bound w(k: 2 - 1) 2 2k is by the result of Kim et al. 2024). For the upper
bound, consider an arbitrary continuous function f : [0, 1 + R 22--1 By Theorem 5.10 of Hirsch (1959), for any R2, there exists an immersion such that I |0.10.00111 < e. By Corollary 3.2 of Lashof & Smale (1959). there exists an embedding : [0, 1] - IR such that its first 2k- components P24,2k-1 1 o h satisfy ||g - 22k,2k-1 o h|] < e. This completes the proof.
1
4.5 Lower Bound of m))
In this subsection, we offer a nontrivial example of minimum width using the concept of wi(n, m). In particular, we will prove that 222.2) = 4 employing algebraic topological techniques. To provide lower bound, we construct a function f : [0, 1 + R² that cannot be made injective by concatenating an additional one-dimensional output function.
Such a function constructed by designing a function with a winding number of two, meaning that if circle in the domain wraps around the origin once, its image wraps around the origin twice. Specifically, we use a function that maps (r. 0) to (r, 20) where r and @ are the radius and the angle in the polar coordinate system. This ensures that self-intersections always occur at two antipodal points on a certain circle. Then, the Borsuk-Ulam theorem guarantees that any function from a circle S to RR must have the same value at some pair of antipodal points, which implies that the concatenation also attains the same value at some pair of antipodal points. Therefore, the minimum width is at least 4. Topological algebra helps formalize these abstract discussions mathematically by utilizing relationships between homology, the fundamental group, and related concepts. See Hatcher (2000) for further details.
From now on, all homology will refer to singular homology with Z-coefficients, denoted as H, (X; Z). For simplicity, we will write it as (X).) The homology of a topological space provides information about the presence of holes in the space. For example, existence of nonzero H1 indicates the presence of one-dimensional hole.
We use the following lemma, which suggests that the homology of level set of function is robust under the perturbation.
Lemma 4.11 (Theorem 2 of Bendich et al. 2010)) Let % be a compact topological space. For
continuous function f X R. and Ja X IR. defined as fa(x) [f(x) al, we define X, as follows:
X, (fa) = f-rt-aa-a--0 r0
(32)
For h: X + R. such that |lf - lallo0.x < r, h (a) is included in Xx (fa):
- Yrr (fa),
(33)
and this inclusion induces the homomorphism of the homology:
in : H,, (h (a)) + Hn (X, (fa))
(34)
In addition, as f- - r) and f r) are also included in Xxr (fa), we have inclusions:
8: f- + Xx (fa), and 1t fa - r) - X (fa),
(35)
which induce the homomorphisms C, and 1 of the homology:
Q. Hn, ( (a + r)) + Hn (%, (fa)), and i Hn, ( (a r)) - Hn (Xr (fa)).
(36)
Define Bo,r and Bi. as the images of two homomorphism:
Bor := L (Hn (f + r))) and B1, == 1. (Hn (f
(a- - r)))
(37)
8

---

Define (r) as:
Un(r)= n [ |1-I1laast
im(j.).
(38)
Then, the following equation holds:
(n(r) =Bo,r 0 Br.r-
(39)
We employ the well-known theorem as lemma. It gives relationship between the homology and the fundamental group of the space.
Lemma 4.12 (Hurewiez Theorem (Theorem 2A.1 of Hatcher 20000)). By regarding loops singular 1-cycles, we obtain a homomorphism surjective and has kernel, which is the commutator subgroup of क (X). Consequently, h induces an isomorphism, from the abelianization of m (X) onto (x
wı (X, zo) + Hi[2 (X). If X is path-connected, then is
By the preceding lemma, we can ensure that the fundamental group of the self-intersection sets is nontrivial.
Now, we introduce the concept of the winding number in the framework of topological algebra. We can observe that if curve is given as input to the previously defined function that doubles the angle, the winding number of the output curve also doubles that of the input curve.
Definition 4.13 (Winding Number), For O = (0.0) R² and closed curve e [0. 1] + R² - O,
consider as an element of the fundamental group:
[c] E Ti (TR2 0, 0.00)= = =z
(40)
where the fundamental group T (R22 -0 mq) is generated by the curve wi (cos(270), (2=0), sin 2TT)). Then, the winding number of c is the natural number [c] element of m (IR2 0,0)) Z.
As the image curve has winding number greater than one, it must have a self-intersection by the following lemma.
Lemma 4.14. For any closed curve e: S² - {(x, v) E I < x² + x2² < 2} in the annulus with winding number greater than 1,
injective.
Proof. Suppose e is an injective curve. By the Jordan Curve Theorem (See Proposition 2B.1 of Hatcher (2000) for details), injective curve bounds a region homeomorphic to the disk. Therefore, there exists an embedding C': D + R² such that its restriction to the boundary is c:
Cls =
(41)
Because 5t - 2 {O} induces an isomorphism of the fundamental group, the degree should be 1 or -1. Therefore, a curve with a winding number greater than 1 not injective.
Using the lemmas, we can rigorously prove the following theorem.
Theorem 4.15. (2, 2) = 4.
The proof of Theorem 4.15 is provided in Appendix C.2 Then, the theorem yields a direct corollary, stating that the minimum width for universal approximation is 4 when both the input and output dimensions are two.
Corollary 4.16.
Winnn (2,2 ReLU), = W min (2, 2, Leaky-ReLU) = 4.
(42)
Proof. The lower bound Wmin 2, ReLU) > (2, > 4 is direct consequence of Theorem 4.15 and Corollary 4.6 The upper bound provided by Hanin (2019). This completes the proof.
Remark 4.17. In et al. (2023), authors claimed that the optimal minimum width is given by max(d, + +1, dyy) 1 However, as demonstrated in the previous corollary, it is not the case. This may originate from subtle misconception, specifically the assumption that an arbitrury d-dimensional continuous function can be approximated by (d + -)ddimnnsionnl diffeomorphism. which does not hold in general. this paper, we carefally control injectivity and embedding properties rigorously demonstrate that, in some cases, additional width is unavoidable.

---

Limitations
proved that for even the optimal minimum width is w(kk 21: 1) 24. But what about the case when k is odd? It is highly plausible that for odd k, the minimum width wr(k 2k: 1) 2k + 1, since odd k permits the existence of self-intersections with structure [S'; :P'] in the sense of -orddii...m Unfortunately, to the best of our knowledge, there is currently no mathematical tool available for analyzing the self-intersection structure of uniformly perturbed functions in this setting. would particularly interesting if one could rigorously prove that the minimum width indeed depends on the parity of the input dimension. Moreover, our current analysis has primarily focused on cases where the output dimension roughly twice the input dimension, allowing to exploit properties of homology and the fundamental group. Exploring higher-dimensional intersection settings remains a challenging but important direction for determining the minimum width of MLPs.
In Section 4.4
6 Conclusion
In this paper, we introduced novel upper and lower bounds for the minimum width of deep, narrow MLP required achieve the UAP within continuous function spaces. While our derived bound exhibits optimality only when the output dimension is about twice the input dimension, we propose that the strategy of approximating arbitrary functions through diffeommrphisms could potentially lead to achieving optimality across all cases. Exploring this perspective presents an intriguing avenue for future research. Additionally, we anticipate that analyzing the quantitative approximation capacity of general MLPs from the viewpoint of diffcomorphisms might yield valuable insights.
Acknowledgments and Disclosure of Funding
This work was supported by National Research Foundation of Korea(NRF), grant funded by the Korea (Mver BRS-2025-0051524 RS-2024-004061277, and Global University Project grant funded by the GIST in 2025.
References
Bendich, P., Edelsbrunner, H., Morozov, D., and Patel, The robustness of level sets. In European symposium on algorithms, pp. 1-10. Springer, 2010.
Cai, Y. Achieve the minimum width of neural networks for universal approximation. In The Eleventh International Conference on Learning Representations, 2023.
Cybenko, G. Approximation by superpositions sigmoidal function. Mathematics of control, signals and systems, 2(4):303-314, 1989.
Hanin, B. Universal function approximation by deep neural nets with bounded width and relu activations. Mathematics, (10))992, 2019.
Hatcher, A. Algebraic topology, 2000.
HERNÁNDEZ. M. and ZUAZUA, E. Constructive universal approximation and finite sample memorization by narrow deep relu networks.
Hirsch, M. W. Immersions of manifolds. Transactions of the American Mathematical Society, 93(2): 242-276, 1959.
Hong, R. and Kratsios, A. Bridging the gap between approximation and learning via optimal approximation by relu mips of maximal regularity. arXiv preprint S.ti::24409113335 2024.
Johnson, J. Deep, skinny neural networks are not universal approximators. In 7th International Conference on Learning Representations, ICLR 2019, 2019.
Kidger, P. and Lyons, T. Universal approximation with deep narrow networks. In Conference on learning theory, pp. 2306 2327. PMLR, 2020.
10

---

Kim, N., Min, C., and Park, S. Minimum width for universal approximation using relu networks on compact domain. In The Twelfth International Conference on Learning Representations, 2024.
Lashof, R. and Smale, S. Self-intersections of immersed manifolds. Journal of Mathematics and Mechanics, pp. 143-157, 1959.
Leshno, M., Lin, V. Y. Pinkus, A., and Schocken, S. Multilayer feedforward networks with a nonpolynomial activation function can approximate any function. Neural networks, 6(6):861-867, 1993.
Li, L., Duan, Y., Ji, G., and Cai, Y. Minimum width of leaky-ReLU neural networks for uniform uni- versal approximation. In Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pp. 19460-19470, PMLR, 23-29 Jul 2023. U pprttpp://proceeingg.s .ll............. press//202/1123gg 1ttm
Liu, C. and Chen, M. Relu network with width d+ (1) can achieve optimal approximation rate. In International Conference on Machine Learning. 2024.
Lu, Z., Pu, H., Wang, F., Hu, Z., and Wang, L. The expressive power of neural networks: A view from the width. Advances in neural information processing systems, 30, 2017.
Munkres, J. Obstructions to the smoothing of piecewise-difffeenttabb homeomorphisms. Annals of Mathematics, pp. 521-554, 1960.
Palais, S. Extending diffeomorphisms. Proceedings of the American Mathematical Society, 11(2): 274-277, 1960.
Park, S., Yun, C., Lee, J., and Shin, J. Minimum width for universal approximation. International Conference Learning Representations, 2021.
Persson, M. The whitney embedding theorem, 2014.
Rochau, D., Chan, R., and Gottschalk, H. New advances in universal approximation with neural networks of minimal width. arXiv preprint arXiv:2411.08735, 2024.
Shen, Z., Yang, H., and Zhang, S. Optimal approximation rate of relu networks terms of width and depth. Journal de Mathématiques Pures Appliquées, 157:101-135, 2022.
Song, C. h., Hwang, G., Lee. h. and Kang, M. Minimal width for universal property of deep mnn Journal of Machine Learning Research, 24(121):1-41, 2023.
Tabuada, P. and Gharesifard, Universal approximation power of deep residual neural networks through the lens of control. IEEE Transactions Automatic Control, 2022.
Teshima, T., Ishikawa, I., Tojo, K., Oono, K., Ikeda, M., and Sugiyama, M. Coupling- neural networks are universal diffeomorphism approximators. Advances in Neural Information Processing Systems, 33:3362-3373, 2020.
baspling-basedd based invertible
A Definitions
In this section, we define several subsets of the set of diffcomorphisms.
Definition A.1 (Diffeomorphism). For d, r E N and open sets U1, U2 C TRd, function f U1 - U2 called C" diffeeomorphism if and only if bijective, r-times continuously differentiable, and its inverse f is r-times continuously differentiable.
Definition A.2 (Embedding). Let M and N be manifolds. A function f M + N is called an embedding if and only if is an immersion and a homeomorphism onto its image f(M).
Definition A.3. For any natural number d, let G be subset of invertible functions from Id to IR
Then, INNg is defined as:
INNg {Wiogl Wn In W Wnti N. E G, W IAffa
(43)

---

Note that the approximation capability of INNg remains unchanged even if IAffa, in the definition, replaced with Aff
Definition A.4 (Compactly supported diffeomorphism: Diff",(Te.) A diffeomorphism f Rdd - IR is compactly supported if there exists a compact subset K C IRd such that for any x & K, f(x) = x. Diff, is the set of all compactly supported C" difiieeomorrsssss from R2 to Rd.
Definition A.S (Single-coordinate transformations: St (R.)). (T(RR) is the set of all compactly
supported C" -diffeommorphiss defined as follows:
(RR) = = ( 3i E (1....,d] d3 s.t
T(æ); = 2j forj & i and T(x); F(x),F C(R".)). (44)
Definition A.6 (Single-coordinate affine coupling flows). ACF the set of all single-coordinate affine coupling flows defined follows:
+ ACF == ( pan....---,,,cplsnnnnaa-aa- (atp)//=t((=a)))))m)) f( C(R ,R),,
(45)
B Proofs for Approximation
B.1 Proof of Lemma
To prove the lemma, we introduce a lemma that suggests we can focus on approximating St (TR)) to achieve the approximation of diffeomorphisms.
Lemma B.1. For a natural number E N, the following relation holds:
INN 5:00 (R+) > )R
(46)
Proof. This result directly follows from Theorem I(B) in Teshima et al. (220)). As every element of S (Rd) invertible and locally bounded due its continuity, it satishies the conditions outlined in Theorem 1. Given that INN s: (Rd) ((R() we can therefore conclude that INN INNs: (Rd)
(RR)..
Proof of Lemma 4.1 According to Lemma B. 1 suffices to prove that the set of neural networks can approximate
Ag ddd,d+((o) > S (R").
When a = Leaky- ReLU, we need to prove that Aaa > S (Rd). We can accomplish this by
employing Lemma B.2
For a = ReLU, by Theorem in Hanin 2019). for f(x) = (# Fdd (x)), we have f K d Therefore, (I E 22 implying Stc (R2) X L.d
For other continuous activation functions a, Proposition 4.2 of Kidger & Lyons (202 demonstrates that for f(x) = (z1,..., Idd 7 (z)),, we have / < Ag d,4+1,d+2 Consequently, (T1,.... 1. TT)) E
Ag d. concluding that S (Rd) d,d,442
is worth noting that while Theorem 1 of Hanin (2019) and Proposition 4.2 of Kidger & Lyons (2020) do not explicitly state the form of approximated function (31..... Fd, 7 (x)), their proofs
implicilly rely on this form.
Now, the remaining task is to prove the following lemma for the Leaky-ReLU case.
Lemma B.2 Single-Coordinat Transformations to Leaky-ReLU), For a natural number d E N, the following relation holds;
LR A
d, S (R.)).
(47)
The proof of this lemma involves series of lemmas and corollaries that gradually extend the scope of functions approximable using LeaU.-ReLU. The proof is provided in Appendix B.2
12

---

B.2 Proof of Lemma B.2
The following lemma implies that any increasing function can be approximated by composing Leaky-ReLUU and affine transformations.
Lemma B.3 (Increasing Functions to Leaky- ReLU), Define the sets as follows:
Up {az + b R R R., R), Un+1 = {alRR(l) + b R a, B.BBERR D DR
(48)
(49)
Un.
(50)
Then, for any continuous, increasing activation function R - R, the the following relation holds:
x U.
(51)
The proof of Lemma B.3 provided in Appendix B.3
This lemma directly implies the subsequent corollary: deep, narrow MLPs using the Beaky-ReLL activation function can approximate a deep, narrow MLP using an increasing activation function and the same width.
Corollary B.4 (Generalization of Activation). For natural number d E N and any continuous, increasing activation function o, the the following relation holds:
Dd,,,d < AR LR d,d,d"
(52)
Utilizing the corollary above, we can demonstrate that Leaky-ReLU deep, narrow MLPs can approxi- mate any ACF. Lemma B.5 (ACF to Leaky-ReLU). For natural number d E N, the following relation holds:
INNACP. AIR
(53)
d,d,d'
The proof of Lemma B.5 is provided in Appendix B Next, we establish technical lemma serving as the multidimensional counterpart of Lemma B.3 For multidimensional function from Rd to IR that increases with coordinate Id, we can freely change the value when Fa is large, while it remains unaffected when Fa is small.
Lemma B.6. Consider compact K [0, 1] C Rd two distinct real values 0n 02 and single-coordinate transformation F (..,..... Td-1 [(x)) E St where the function f(x)) satisfies the following relation:
f(x) Of aa < 01, and f(x) = 0 Of if = a1
(54)
Assuming that F
LR d.d.d
k" Then,for a funciii dRd-I + R such that b b(x1:d-1) >
for all x. K, there exists single-coordinat transformation G = ( (..1,,,), x A 1.R
d,d.d
K
satisfying the following relation:
[f(x)
if S 01 if = 02
g(x) = (((T)((Z1a)1)
(55)
The proof of Lemma B.6 is provided Appendix B.5
Utilizing the lemma, we can prove Lemma
B.
Proof. Consider an arbitrary single-coordinate transformation F(x)
(II...., Id-1, Fdd- (2,,...., =a)) and a compact set K Rdd. Without loss of generality, assume K [j. Additionally, assume 7 strictly increases with respect Tzt-
Because t is a continuous function defined on compact set, for an e > 0, there exists natural number N e N such that if || x'll < $. then (T(x) r(x') < e. Now, define u; [d-1 - R as follows:
u, (1(dddd11)= = F #1.d-11 x
(56)
13

---

If there exists a single-coordinat transformation G = ( Id-1. x A d,d,a such that
ui (1( - 9 (Ii:d-1, Rt) for E K, then F X A LR
d,d.d' We will demonstrate the existence
of a sequence [== C
such that (2ind--- +) u<(x1dd1)) for S i S n via ALR d
mathematical induction.
By Lemma B.6 there exists a single-coordinate transformation Go = (#1,... 1d 90 (z))
such that Do (o(di: 0) uo( Assume the induction hypothesis holds for no. im- plying the existence of a single-coordinate transformation Gno (I1....., Id Imp (x)) such
that In (F1.d-1: n = w;(21:d11) for S i S no- Then, by Lemma B.5 we can con-
struct Go = (x (21....,, 1d Ina (x) Una (f1:d-1)) A
d.d.d Notably, Gm, satisfies the
K
assumptions of Lemma B.6 with 0n = N and 02 = no+1 By applying Lemma B.6 with
((1:1-1)= =
Mng+ Kng+i
N ) (#i1d-1)" we obtain a single-coordinate transformation G" (mo) =
Knp (2iid-1)
gng (#id-1
(z ( Fd 1: Sn (x:) such that Ih, (Ir:d-1: 3) ux(xi.d-1) un (Z1.a-)) for i S 10 1. Fi-
nally, by Lemma B.5 we can get Gug+1 == (#11.... 1d Ino (2)) + uno (#1=d-1)) E A LR
As
d,d.d
k'
a result, the induction hypothesis is satisfied, and this completes the proof.
B.3 Proof of Lemma
Proof. Because increasing piecewise linear functions are dense in the space of increasing continuous functions defined on a compact interval, it suffices to prove that for any natural number n E N and an increasing piecewise linear function with n breakpoints, we have f E Un. We will proceed with mathematical induction on n. For the base case, n = 0, there is nothing to prove. Assume that the induction hypothesis holds for some n = no. and consider the case of n no + 1, where we have an increasing piecewise linear function f with + 1 breakpoints, denoted as 01 < <... < +1- The function f is affine on each of the intervals (-00, 01), [2), ..., [ahno Onz [Ang+1, 00). Now, let f have values as follows:
( [((no))) + (1 0no+1)
if x [nnn Ono+1]] if E [0mo+ 00)
f(x)
(57)
((ano+)) ) + 72(x - (no+1)
Consider the function fo defined as:
fo(x) f(xx) (f(ano+)) (1 ano+1) if E [0 20)
if x E (-00, Ono+1]
(58)
The function fo coincides with ff on the interval (-o0, [no++1] and affine on the interval [ang oo). This means that the affine function [amn + 1, oo) with the same slope. Therefore, fu has ng breakpoints, and by the induction hypothesis, fo Uno- We can express f in terms of fo follows:
the interval [aizo ano+1] naturally extends to the interval
f(x)
72 LR (Jo(x) f((mo+))) + ((in+11)
(59)
71
Thus, / E Ung+1, and the induction hypothesis is satisfied for no + 1. This completes the proof.
B.4
Proof of Lemma B.5
Proof.
For 3 E R., a, R and b E Rd-1, we define the function follows:
9: I2..,,,
Ia) + (I1, D2,, ..- Id-1,
+ (b Fi:d-1 c))-
(60)
We will prove that g < A d.d.a- b the zero vector, g a constant adding function satisfying the statement. If is not the zero vector and (b,,..., ba-1), there exists an index 1 S i < d - such that b, # O. Let W IAffa be invertible affine transformation defined as:
W (I1, T2, .., Fa) (x1, T2, ................................ I1.1, b. Iid-1 + c, c.
(61)
14

---

Because b, is nonzero, W is invertible. Applying g to the i-th component gives:
(1) 1-11,1111 LRa(b #1:d-1 + c), Ri+1,..., ..d) x A LR
(62)
d.d.d'
By adding times the i-th component to the last component, we have:
za + aLR Ti.d-1) + + ALR (II...... (R 21-d-1 + c), ,,,,.,,,
(63)
d,d.d'
By applying LR to the i-th component and applying W-1, we get:
( Id-1, za + a #1:d-1 c)) - LR d,d,d'
(64)
Next, we will prove that for the function h defined as:
h : (21, ; + (#1, Id- + f(11 1)-
Fa)
(65)
h x LR d. By the UAP of two-layered neural networks (Leshno et al.
1993 for any e >> 0 and
compact set K IRd-1 there exist 14., ag G E, and by E Rd-1
such that:
8(
((11d-1)
Σ La,RR R1dd11 +
<c.
(66)
2=1
00.K
Composing Eq 64 for n different a,, bi, and yields:
F1. Fd- 1, Td
o,LRR g (b, vi + + tco)
+ A d.
(67)
Thus, h A LR
d,d.a'
Finally, by composing the operations described so far, we demonstrate that any ACF can be approxi- d,d,a It is achieved by combining the following four operations:
mated by ALR
Apply the logarithm to the last component.
Add log(s(z)....., xa-1)) to the last component.
Apply the exponential function to the last component.
Add 1d-1) to the last component.
This results in the following transformation:
........dd.1... (21, rr) H (21, Zd-1, exp (log(rs) log(s))+t) = (x1, 22, ................................ #d-1, sta
LR
d,
(68)
This completes the proof.
B.5 Proof of Lemma B.6
Proof. We begin by observing that it is sufficient to consider functions b satisfying b(Z1:g-1) 2
for all E K. Define 3 as B = infzek b(æ1:d-1). We introduce the function F(x) = (#1,.... Fd- Id f(x:), defined as:
F(0) 3LR (f(=)))
(69)
LR If F' E d.d.d K
then E d,d.d K
LR
The value of f(x) can be calculated as:
[(x)
if < 01 if > 01
f(x)
(70)
(z)())
This ratio g(x) Therefore, only need to consider functions b that satisfy b > 1.
اايييا > 1 for all x K, and F also satisfied all the assumptions of the lemma. 3
f())
15

---

Next, we will inductively construct sequence {G; = Id C A that uniformly converges to G when x. - a.- We start with g0(x) = (xx) Define b, R R
d,d,d
b((rad-1): = 0;(
(71)
f(11:d11,02)
for all x E K. Define 74 R as:
:= sup b(x1d-1) (((Iidd1))
I E K
(72)
Next, we define two mutually exclusive sets, denoted as Li, and Li;1;
Li,0= Uiid-1 E 10, 15 b(21:d-1)
(x111d11)
73 f.
(73)
b(11a-1)
L 11:d-1 E 10,1)0-11
S 78
(74)
by (
Define a distance metric D as:
Dxr, C): == inf || ull2,
(75)
And then, we define the function d, RRd-1 + R as:
Dxr, I4,0)
(x) D(x, Ls,u) + ( L,,1)
(76)
The function Φ; satisfies the inequality 0 s (I 1) S 1 for all E K, has value of zero on
Li,0, and value of one on L1 Define h; R - RR as follows:
h1(x1dd11) = (1 6s
(21.a-1,,
(77)
Then, S (= S (c(F1aa1, for all K, has value zero on Li,1 and a value of (1 a2) Lip--
Now, we define (x) E A d.d.d k
as follows:
(i+1(x): = n, LR - (g((x)) - h;(x1:a-1)) + (1(drd-1))
(78)
7.
We have
= gx(x)= =0 0
if as S a1
= 91(x)
if x. = 02 and #1:d-1 E Li,
(1+
(79)
= 7, (x())
if if = 02 and #1:d-1 E L111
< 7/ (x)
if Ir = 02 and #1:d-1 & L,,0 U Li,1
Thus, for x where F2 = ag and 11 E Lins we have
g(x) g(x) b(11d-1)
(80)
1 (x) (x(x) by(01.011)
As S MFLe-1) < 4 b(( % for Fi:d-1 E Lio we deduce that 1 15
gi+1 S 7/
g(z)
For where Ia a2 and Fi:d-1 E Li,1,
g(z) 9(x)
% - b((1rd-1)
(81)
(x) 7/5 (x)))
(x(11dd11)
3 < b,( As 7,
M0rd-1) S 7i for 11 Li,1. we get 1 5 55 - g(z)
S 5:1
())1(())
For x where Fz = a2 and 21:d-1 & LgD0 U Li,1
7; - 6(11ad1)
g(x)
g(x)
g(x) b(I1:d-1)
(82)
7/ (x)
gi+1 (x)
91 (x(x)
b((
16

---

As 7/ S
vi for Fidd1 & Ta,0 Ia,1, we conclude that S b((raa-11]
HoLd-1)
gi+1(2)
g(x)
S
7
We obtain the following results: for all K, where z = a22 we have <
o(w)
S
g(+1
h (iidd
7: This implies I 741 S S%²² Consequently, tends towards infinity, V6 converges to one.
Therefore, a(#1d-1100)) uniformly converges to one as i increases, implying the convergence of 9
G2 G. As result, there exists function G (x1 (.i,,..,, 2a-1 A such that g( 02) (2
d,dd K
To check that is single-coordimate transformation, we observe:
9i+1 (21:d-1, (F > (I va) 91 (21:d-1,
(83)
for va > d. which implies that (( g ( 1, m'd) > go 90( Iid) go (0 a'd) > 0 for all æ K. Therefore, g satisfies the strictly increasing condition, and G becomes single- coordinate transformation.
C Proofs for Topology
C.1 Proof of Theorem 4.5
Proof. For a non-decreasing continuous activation function a, there exist smooth, strictly increasing activation functions on that uniformly converge to a. Therefore, Ad,d, < UneN A d,d,d X D (R4),
making it sufficient to consider only set of smooth, strictly increasing activation functions a.
For f A -m it can be decomposed
f Pw(n,m) 1,m o o (n,
(84)
where 9
A w) 1, (,0(m,)))- Because A L.
Doo [ (Ru(m))-1), g (m.w(n.)-11lo..m Emb((0, 1) Ru(n.m)- Therefore, have:
11 Pro(n,m)=1,m Emb((o, 1j", Ru(),))-1))
(85)
and because f E A m. is arbitrary, we conclude:
a
C PP(n.m)-1.m (Emb((0, 1jm, Rw(n,))-1)
(86)
[0.1]"
Because w(n, m) - 1 w(n,m) by the definition of w(n, m):
Emb((0, 1jmh (w((,.)-11) E C(10, 1[", R"),,
(87)
and consequently,
2 C(0, 1|". RT).
(88)
w.m.u(nm))) [u,1]"
Hence, we conclude that C(R", RT)) A A
C.2 Proof of Theorem 4.15
2 2. Therefore, sufficient prove that w(2,2) 2) > 4. Assume the opposite, w(2, 2) < 3. Then, for an arbitrary continuous function f in C([-2, 2[[_,, RR?,, / is contained in 23.2 emb22 2|,,R*) P3.2 o.r.p.. 2[-2, 2/2, R3). Consider
Proof. Firstly, it obvious that 222 < 2
the piecewise linear map f 1, 2 + R² defined as follows:
-1 (
ir S 22 Æ1,
x2
(2)
11 11-1
-1
11
(89)
2 000
if0 S m $22
-1(2 -21)
if < 0 and 222
f--111 21, x22)
if < 0.
17

---

We can check that f is the piecewise linear a piecewise linear embedding G E Embp,. (1-2.2 22.2) RR3) such that
function. By the assumption, there exists
1
|l/ P3.2 o Gl 20, 4
(90)
Let E: IR22 + be defined as
Σ: (T) x22 + |21| + |22|
(91)
We observe that f conserves the level E: (Eo (z) Ex) for all IR. Therefore,
(I = =1) = { x2) E R2I |21| + | = 1}.
(92)
which is homeomorphic circle S Similarly,
() 2 - ( = {{ (x) (21, E |11 + [22
(93)
And
(3)
(2) {{ (011 E |R
(3)
3
f)
-
[22
(94)
2
are homeomorphic to 8", and
(
1 3 2 -
x
(1
2
3
=f (111)22) E B²2
2
S
+
|122|
S
3
(95)
homeomorphic to a closed annulus S [0, 1].
Define as == P3,2 G. Because
|[ - -lllooo,-2.00
< 1
4. we have
1
Σ - Eogl < 2
(96)
Now, apply Lemma 4.11 to E f. Because (Eo f) (2) = - and (E f) (+)
-1 deformation retractions of (Eo f) ((2 21). , the following equation holds:
U1 ( = H1 (Bu.t) = H1 (B1.1) = Hi (1
[ (() = H4 (2- = z (97)
-11
2
Thus, (1) = E. Note that in HI, (2''g) (1)) (1(+ is surjective.
Because g and Σ are piecewise linear (Eog) -1 (1) consists of finite connected components
A Ag. Then, the first homology H, (Eog)) (1)) decomposed as:
Hi (1oo)) (1) θ Hi (A:) i=1
(98)
And we can express jn as a sum of homomorphisms it. H, (A.) - U₁ (1):
jo(z) = Σ (2.).
(99)
for x = + Ty. As jy surjective, we can choose an index in such that j. is a nonzero ho- momorphism. Set a basepoint xo E Aig. By Lemma 4.12. there exists a surjective Hurewicz homomorphism h : Ti (Aio, 2o) + H, (Atg). Additionally, the Hurewicz homomorphism h2 Th (Eo f) ([), }) .00) H, ((2 f) ([) 2)) is an isomorphism. By compos- ing homomorphism, we obtain:
Ti (Ai,r Ru) h4, H1 (Aio)
(100)
3 Hi
3 )) (1
1 h 1
([1 3
(Eof 1)-
2 22))
U1
-
2
(Σo
2 2
.o
(101)
18

---

resulting in the nonzero homomorphism b/ o j o h1:
h j m (Aigs zo) + n1 (Eof))
( 3 2 .20
(102)
Furthermore, observe that
h2 Jg o h1 = b::
(103)
where 4, represents the homomorphism the fundamental group induced by the inclusion e Aio 4 (Eo f) ([2 2])
Now, we will prove the existence of simple closed curve S - (Eog) - (1), homotopic to the cycle w1 0 > (cos(2 sin (2x0)). Because and E are piecewise linear, (E g) (1) can be realized by simplicial complex, and we can assume that π (2' g) g) (1), 20 is generated by curves with finite segments, where all self-intersection points are breakpoints curves. Choose To πi (Eog) (1). 20) ιi (Ai. zo) such that & ((rol) is nonzero. We iteratively construct closed curve until it has self-intersection points. Suppose has self-intersection point + b: Ta(a) - (b).. Define 7 as Vilaa,6 and Y be defined as st).)-(..)) Then, 7;' and 7; become closed curves again with fewer segments than %. Because the winding number of equals the sum of those of y,, and % at least one of y; or 7 has a nonzero winding number, and we set 7 as the with nonzero winding number. Each % has strictly smaller number of segments and has a winding number not equal to zero. Because "No has finite segments, this process stops in finite sequence. Therefore, can get non-selffinterseethe number. If ạ has a winding number with an absolute value greater than one, by Lemma 14 must have self-intersection point. Thus, has winding number yields a curve with winding number one.
increases
curve y 7/n with nonzero winding
1. Reverse reparametrization
Because is homotopic to f' through linear interpolation and is homotopic to w1, their compositions are homotopic. This implies the same winding number between g o and f' w/ Therefore, the
winding number of g S S = Σ (1) is two. Now consider G. Because G is an embedding, it is injective. Therefore, ((T)) 7(() S x RR is injective. As the image G()(S)) is compact, the image in S x IR can be embedded in the annulus {(11, x2) E R S |xa| + [22 S 1++4 c).
And the map G y has winding number two. However, by Lemma 4.14 any map with winding number two is not injective, leading to contradiction.
19

---

NeurIPS Paper Checklist
Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
No justification
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how be expected to generalize other settings.
much the results
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
No justification
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model sell-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images taken low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question:
For each thcoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
20

---

Justification:
No justification
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [NA]
Justification: The paper does not include experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reprodueibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
21

---

Answer: [NA]
Justification:
The paper does not include experiments.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.cm
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/ ctiipe.cc/pulli//uuiii http:/uwdieeseee CodeeSubeeiieeeee ssion ceiiionPoicyy for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper speciffy all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [NA]
Justification: The paper does not include experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [NA]
Justification: The paper does not include experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, at least for the experiments that support main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
22

---

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of enrors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Answer: [NA]
Justification: The paper does not include experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g.. preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics /tttui://euurpss..
Answer:
[Yes]
Justification: No justification
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is a special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: There is societal impact of the work performed.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to
23

---

generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative socictal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The paper poses such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators original owners of assets (e-g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [NA]
Justification: The paper does not use existing assets.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, haaerssithcoode com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's is creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
24

---

Answer: [NA]
Justification:
The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the methods in this research? Note that if the LLM is used only for writing. editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Answer: [NA]
25

---

The core method development in this research does not involve LLMs as any important, original, or non-standard components.
Justification:
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non non standadd components.
Please refer to our LLM policy hattps://neurips... 200p://ee///0ooeeeeeee/2/22///// for what should or should not be described.
26


Reviews:
[{"review_id": "1SI872m0ob", "reviewer": "Reviewer_VB9C", "summary": "This paper establishes new upper and lower bounds on the minimum width required for deep, narrow MLPs to achieve universal approximation within continuous function spaces. The new upper bounds now depend on $\\max(2d_x + 1, d_y)$, sharpening earlier results. The argument is built on diffeomorphism-based constructions, bridging geometric topology and neural approximation theory.", "questions": "Please see \"Weakness/Questions\"", "limitations": "Yes.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 4, "originality": 3, "strengths_and_weaknesses": "## Strengths\n\n- The new upper bound improves prior results: it now scales with $\\max(2d_x + 1, d_y)$ rather than $d_x + d_y$.\n- The proof is built on diffeomorphism theory, which is original (to my knowledge) in the UAP study and provides new insights.\n* The technical development seems solid. I did not verify every detail, but I checked the main ideas and several key lemmas.\n\nOverall, I am impressed by the work.\n\n## Weaknesses / Questions\n\n1. Lines 153–154 introduce the notation $w(m,n)$ for the first time, and seem to define it as the “required width for approximation” by neural networks. However, Eq. (12) formally defines $w(m,n)$ as the minimum width needed to approximate arbitrary continuous functions using diffeomorphisms. This mismatch is confusing. Please clarify.\n\n2. In Lemma 4.3, does the embedding $f$ need to be piecewise linear, or is smoothness sufficient? Please state explicitly what regularity is required.\n\n3. Lemma 4.1 is crucial. Once it is established, the main result follows fairly directly from known results in topological algebra (if I understood correctly). However, its full proof is scattered throughout the appendix, which makes it difficult to follow. Could you provide a more organized proof outline or a concise roadmap in the main text?", "was_revised": true, "final_justification": "My comments are mostly minor. After discussion with the authors in the rebuttal phase, I decide to keep my scores.", "rebuttal": "**Question 1)** Sorry for the confusion. The statement “required width for approximation” in Lines 153–154 refers to the minimal width required to approximate any arbitrary continuous function using diffeomorphisms, as formulated in Equation (12), not the width of neural networks. We will revise the statement from “Now, we quantify the required width for approximation” to “Now, we quantify the required geometric width for approximation” to clarify this point.\n\n**Question 2)** Lemma 4.3 requires only smoothness.\n\n**Question 3)** Thank you for pointing this out. While we appreciate your observation, we respectfully note that the topological and algebraic components of the proof are not entirely straightforward, as they involve several subtle challenges. We will add the following proof outline to clarify our approach:\n\n*In [1]., it was shown that any diffeomorphism can be approximated by a composition of single coordinate transformations. Therefore, it suffices to prove that deep narrow MLPs can approximate any such single coordinate transformation (see Definition A.3 for the formal definition). This is established in Lemma B.1. With the exception of the Leaky-ReLU case, the proof is relatively direct.*\n\n*For the Leaky-ReLU case, we progressively extend the class of functions that can be approximated. Using Lemma B.3, we show that any increasing scalar function can be approximated by width-1 Leaky-ReLU networks. This result implies that any width-$d$ neural network with increasing activation functions can be approximated by a Leaky-ReLU network of the same width (see Corollary B.4).*\n\n*Building on this, we prove in Lemma B.5 that any ACF (see Definition A.4) can be approximated by deep narrow MLPs. Finally, Lemma B.6 shows that any single coordinate transformation can, in turn, be approximated by an ACF.*\n\n\n\n[1] Teshima, Takeshi, et al. \"Coupling-based invertible neural networks are universal diffeomorphism approximators.\" Advances in Neural Information Processing Systems 33 (2020): 3362-3373."}, {"review_id": "BtuC6ZnqAy", "reviewer": "Reviewer_tyGQ", "summary": "## Emergency Review\n\n### Summary\n\nThe paper establishes the **minimum hidden-layer width** $w(n,m,\\sigma)$ required for a deep, narrow MLP with a given activation function $\\sigma$ to be a universal approximator on $C(\\mathbb{R}^n, \\mathbb{R}^m)$. The main result is that:\n\n$$\nw(n, m, \\sigma) = w(n, m) + \\alpha(\\sigma),\n$$\n\nwhere $w(n, m) = max(2n+1, m)$ is a topological term and $\\alpha(\\sigma) \\in \\{0,1,2\\}$ depends on the smoothness of the activation function.\n\nThe authors prove that this width is both **sufficient** and **necessary**. The proof is constructive for Leaky ReLU networks, involving approximations of diffeomorphisms via affine coupling flows and known geometric theorems. Extensions to ReLU and $C^1$ activations are made using results by Hanin (2019) and Kidger–Lyons (COLT 2020). Lower bounds are proven using tools from differential topology, and a detailed topological construction is provided for the $n = m = 2$ case using winding numbers and Hurwicz theorem.", "questions": "### Questions for the Authors\n\n1. Could you clarify **why affine coupling flows** and **single-coordinate diffeomorphisms** are particularly suited for universal approximation proofs? What’s the intuition behind choosing these building blocks (as also used in Teshima et al., NeurIPS'20)?\n\n2. In what technical sense do your **constructions depart from Teshima et al.**? Do you rely on fundamentally different structural decompositions or just a different universal target?\n\n3. The overhead term $\\alpha(\\sigma)$ appears due to Hanin (2019) and Kidger–Lyons (2020). Are these $+1$ (ReLU) and $+2$ ($C^1$) corrections **provably unavoidable**, or could improved constructions close this gap?\n\n4. Could the results **extend to architectures** that include skip connections, convolutional layers, or residual blocks? Do you expect the minimum width bound to improve or worsen?\n\n5. Could you provide intuition for what the function $w(n,m,\\sigma)$ captures beyond a topological obstruction? Does it have geometric interpretation in network function space?\n\n6. Are there technical obstacles to extending the **lower bound arguments** to the case of odd $k$? Could methods from obstruction theory or embeddings help?\n\n\n### Suggestions for Related Work to Cite\n\nThe following works seem highly relevant but are not cited or discussed:\n- *ReLU Network with Width Can Achieve Optimal Approximation Rate* (ICML 2024)\n- *New Advances in Universal Approximation with Neural Networks of Minimal Width* (arXiv 2024)\n- *Minimal Width for Universal Property of Deep RNN* (JMLR 2023)\n- *Approximating Continuous Functions by ReLU Nets of Minimal Width* (Yarotsky, 2017)\n- *Constructive Universal Approximation and Finite Sample Memorization by Narrow Deep ReLU Networks* (arXiv 2025)\n\nIncluding a discussion of how your results compare with these would clarify your contribution.\n\n### Typos and Clarity\n\n- Line 383, Eq. 44 (Definition A.3): Does $\\tilde{\\tau} \\in C(\\mathbb{R}^d, \\mathbb{R})$ ensure invertibility?\n- Line 232: \"than that both\" $\\rightarrow$ \"than both\"\n- Line 242: \"To provided\" $\\rightarrow$ \"To provide\"", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 4, "originality": 3, "strengths_and_weaknesses": "### Pros\n\n- Closes a known theoretical gap by identifying the minimum width required for universal approximation with deep, narrow MLPs.\n- Covers a broad class of activation functions, including ReLU and other $C^1$ activations.\n- The upper bound is constructive and leverages elegant geometric and topological tools.\n- The lower bounds use rich machinery (immersion theorems, algebraic topology), giving strong evidence of necessity.\n- Clear modular structure in the proof pipeline (diffeomorphisms $\\rightarrow$ composition $\\rightarrow$ universal approximation).\n\n### Cons\n\n- The key assumptions are tailored to cases where the output dimension is roughly twice the input dimension (e.g., $m = 2n - 1$); this limits generality.\n- The exposition could benefit from more explicit motivation for the specific techniques used (e.g., affine coupling flows) and a better connection to the broader literature on UAP.\n- The topological lower bounds, especially in Section 4.5, are difficult to follow and would benefit from an appendix with extended intuition and proof sketches.\n- The role of $\\alpha(\\sigma)$ as a correction term is technically derived but lacks a strong intuitive justification in the main text.\n- The manuscript does not deeply discuss recently concurrent or slightly older related works on minimal width UAP.\n\n\n\n---\n\n### Overall Assessment\n\nThis is a strong theoretical paper that makes **meaningful progress on a long-standing question** regarding the width needed for universal approximation by deep narrow networks. The constructive nature of the proof and use of geometry/topology is elegant and technically non-trivial. However, the **exposition could improve**, particularly in motivating the approach, situating it within the broader UAP literature, and expanding on difficult sections.\n\n### Overall Recommendation\n\n**Score**:  (Weak Accept)  \n**Confidence**: 4 (Expert in this area)  \n**Rationale**: The result is solid and interesting, though the work is **incremental** relative to other recent contributions. A stronger contextualization and more reader-friendly explanations would elevate the impact.", "was_revised": false, "final_justification": "", "rebuttal": "**Question 1 and 2)**  While Teshima et al. use affine coupling flows and single-coordinate transformations as fundamental building blocks, our approach relies solely on MLPs, which highlights the additional difficulty of our setting. Also\n\nThe choice of affine coupling flows and single-coordinate diffeomorphisms is motivated by technical considerations, allowing us to build upon the existing work of Teshima et al. This approach reduces the problem of approximating an entire diffeomorphism to approximating specific functions in which only one coordinate is modified. This structure facilitates approximation by deep narrow neural networks.\n\n**Question 3($\\alpha(\\sigma)$))** We believe that the overhead term $\\alpha$ can be reduced. There is no particular reason to believe that the Leaky ReLU activation function is inherently more powerful than other commonly used activation functions. In fact, for ReLU networks, it has been shown that a width of $d_x + d_y$ is sufficient, whereas our method currently requires an additional $+1$ term.\n\n**Question 4(Other network architectures)** We believe that skip connections can improve the minimum width requirement. Consider a structure of the form $I + n(x)$, where $n \\in \\Delta^{\\sigma}_{d,d,d}$. Suppose we aim to approximate an arbitrary function $f$. Let $f_2$ be a $C^1$ function that is close to $f$. Then, a function of the form $-I + \\varepsilon f_2$ can be a diffeomorphism for sufficiently small $\\varepsilon$, and can be approximated by $n$. Therefore, the summation $I + (-I+ \\varepsilon f_2) =\\varepsilon f_2$ can be approximated by deep narrow MLPs with skip connections. When composed with suitable affine transformations at the final layer, this construction enables the approximation of arbitrary continuous functions. This suggests that skip connections help the universal approximation while potentially reducing the required network width.\n\nFor ResNet blocks, we believe skip connections lead to a similar reduction effect. For instance, under the $L^p$ norm, the optimal minimum width for approximating a function with $d$-dimensional input and output is $d$ for MLPs [3], whereas for ResNet blocks, it can be as low as $1$ [2].\n\nFor convolutional neural networks, defining the notion of minimum width is more subtle. In the CNN literature (e.g., [1]), the number of channels is often used as a proxy for width. In that context, an affine transformation is constructed from an input shape $c_x \\times d_1 \\times d_2$ to an output shape, and the problem effectively reduces to computing the minimum width for an input dimension $d_x = c_x d_1 d_2$ and output dimension $d_y = c_y d_1 d_2$ as the representing power of deep narrow CNN is almost same to that of deep narrow MLPs.\n\nBelieving that the optimal minimum width for MLPs is asymptotically\n$$\n\\min\\left( \\max(2d_x + 1, d_y),\\ d_x + d_y \\right),\n$$\nthen the corresponding width for CNNs would follow the same form:\n$$\n\\min\\left( \\max(2c_x +1, c_y),\\ c_x + c_y \\right),\n$$\nThis implies similar minimum width for the CNNs.\n\nWe expect analogous improvements in representational power for RNNs using similar reasoning. \n\n**Question 5)** The quantity $ w(n, m, \\sigma) $ represents the minimum intermediate dimension required to preserve the information of an arbitrary function with $ n $-dimensional input and $ m $-dimensional output.  \nFor example, consider the identity function $I \\in C(\\mathbb{R}^2, \\mathbb{R}^2) $ and suppose it is decomposed as $I = g \\circ f$, where $f \\in C(\\mathbb{R}^2, \\mathbb{R}) $ and $ g \\in C(\\mathbb{R}, \\mathbb{R}^2) $.  \nSince the identity function is injective, $ f $ must also be injective.  \nThis leads to a contradiction, because the intermediate dimension (in this case, 1) is too small to preserve the complexity of the original function. \n\nMore specifically, the difficulty of diesntagling manifold structure of self-intersecting function is captured by $w(n,m,\\sigma)$.\nThe more self-intersections the target function has, the harder it becomes to disentangle it in the intermediate dimension.\n\n\n**Question 5 & 6)** Yes, the issue is that intersections are not robust under perturbations. We believe that the optimal minimum width satisfies $w(k,2k-1) = 2k+1$ when $k$ is odd because there exists a 2-to-1 self-intersection structure of cycles. However, we are currently unaware of any tools that can preserve such intersection structures under small perturbations in the uniform norm. We are also uncertain whether obstruction theory provides a suitable framework for handling this issue. If the reviewer is aware of any relevant concepts or techniques, we would greatly appreciate any suggestions.\n\n**Related Works)** Thank you for the valuable references. We will add it to related works section.\n\n**Typos)** Thank you for pointing out the typos. We will correct them. Regarding Definition A.3, $\\tau$ should be chosen such that the resulting element is a diffeomorphism.\n\n**References**\n\n[1] Hwang, Geonho, and Myungjoo Kang. \"Universal Approximation Property of Fully Convolutional Neural Networks with Zero Padding.\" arXiv preprint arXiv:2211.09983 (2022).\n\n[2] Lin, Hongzhou, and Stefanie Jegelka. \"Resnet with one-neuron hidden layers is a universal approximator.\" Advances in neural information processing systems 31 (2018).\n\n[3] Shin, Jonghyun, et al. \"Minimum width for universal approximation using squashable activation functions.\" arXiv preprint arXiv:2504.07371 (2025)."}, {"review_id": "mv6Jkrhtr6", "reviewer": "Reviewer_LRhU", "summary": "The paper investigates the minimum width required for deep, narrow Multi-Layer Perceptrons (MLPs) to achieve the universal approximation property (UAP) under the uniform norm. The authors propose a framework that reduces the problem of determining the minimum width to a purely geometric function w(d_x,d_y), dependent solely on the input and output dimensions d_x and d_y​.  \n\nThe work bridges gaps in existing literature by providing tighter bounds and exact values for the minimum width, particularly in cases where the output dimension is roughly twice the input dimension.", "questions": "see weakness.", "limitations": "yes", "rating": 2, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n\n1.The paper is technically rigorous, with well-constructed proofs and a clear methodology. The use of geometric and topological tools to derive bounds is innovative and solidifies the theoretical foundations of the work.\n\n2.The authors provide intuitive explanations for their results, such as the role of diffeomorphisms in approximation, which aids understanding.\n\n3.The introduction of the geometric function and the framework for analyzing minimum width through diffeomorphisms are novel contributions.\n\nWeaknesses:\n\n1.The paper focuses on the uniform norm, which may limit its applicability to other types of norms commonly used in machine learning.\n\n2.The assumptions made about the activation functions (e.g., C^1 near a point with non-zero derivative) may not hold for all types of activation functions used in practice.\n\n3.The paper does not provide extensive empirical validation of the theoretical results, which could be beneficial for demonstrating the practical implications of the findings.", "was_revised": true, "final_justification": "I keep my original rating due to unsatifactory response and the possibility of duplicate submission.", "rebuttal": "**Question 1)**     It is true that our paper focuses on the uniform norm. However, the uniform norm lies at the core of robustness of neural networks and provides a meaningful setting in which to study the universal approximation property. Most existing works on the universal approximation property (UAP) primarily study the uniform norm or the $L^p$ norm.\n\n**Question 2)**  It is not accurate to claim that the assumptions made about the activation functions may not hold. All commonly used activation functions in practice satisfy our assumptions. In fact, the assumption is so mild that it is actually difficult to construct an activation function that violates it. We would appreciate it if you could provide an example of an activation function that does not satisfy the assumption.\n\n**Question 3)**  Our study is aimed at providing a theoretical validation of the minimum width required for universal approximation, and it does not attempt to provide empirical validation."}, {"review_id": "qh5RxlBG4K", "reviewer": "Reviewer_5QuP", "summary": "This paper presents a new framework for developing novel upper and lower bounds on the minimum width for deep MLPs to universally approximating continuous functions under the uniform norm. The paper develops this framework by first proving that a deep narrow MLP can approximate any $C^2$-diffeomorphism and then utilizing that result to show how any continuous function can be approximated by composing linear transformations and diffeomorphisms. Using this framework they provide a novel upper bound on the minimum width of $max(2d_x + 1, d_y) + \\alpha(\\sigma)$. Their results hold for a variety of activation functions including the Leaky-ReLU and ReLU.", "questions": "- How is Definition 3.1 different from saying that A is dense in B?", "limitations": "Yes.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "## Strengths\n- To the best of my knowledge the diffeomorphism approach is a novel framework and this approach may pave the way to apply these tools from geometric topology to other deep learning theory problems.\n- The paper contributes to a clear gap in the literature on universal approximation with deep narrow MLPs. In particular, using their framework they demonstrate that a width of max(2d_x + 1, d_y) always suffices for UAP of deep (narrow) MLPs. This improves upon earlier works which placed the upper bound between max(d_x + 1, d_y) and d_x+d_y.\n- The paper \n- While the paper makes use of very technical math the contributions of the paper are clear and the high level proof strategy is easy to follow.\n\n## Weaknesses:\n- While I understand that this is a theoretical contribution the paper has a very high technical barrier making the proofs inaccessible to most members of the Neurips community. For instance, the term \"diffeomorphism\" is never defined in the paper. \n- The optimal results are for very specific choices of d_x and d_y. It is unclear whether this framework can be useful for obtaining optimal minimum widths for arbitrary d_x and d_y.", "was_revised": true, "final_justification": "I ultimately recommend a 5 for this work. I believe the work provides a strong theoretical contribution and opens the door to a new set of mathematical tools for understanding neural networks. The authors alleviated all of my questions and concerns in the rebuttal.", "rebuttal": "**Regarding Technical Barrier)** Thank you for pointing this out. We will add definitions of several technical terms, including diffeomorphism, embedding, and certain topological and algebraic concepts used in Section 4.5.\n\n**Usefulness of our results)** It is true that, at the current stage, the optimal width is achieved only in specific cases. However, our framework—which reduces the problem of identifying the optimal width to analyzing the quantity $w(n,m)$—is fundamental. Many existing lower bounds (at least for networks with Leaky ReLU activation) can be naturally interpreted and easily rederived within our framework.\n\nFor instance, the lower bound presented in Kim et al. (2024) can be reproduced by constructing a non-injective continuous function with $d$-dimensional inputs and $2d$-dimensional outputs. We believe that future work on width-based universal approximation should build upon this foundational framework.\n\n**Definition 3.1)** Regarding Definition 3.1, note that $B$ is not required to contain $A$, so the definition is slightly different from the standard notion of density."}, {"review_id": "QmkEMjBUdB", "reviewer": "Reviewer_sbdh", "summary": "This works studies the optimal network width, in a minimax sense, for deep but narrow MLPs under the uniform norm, with a primary focus on the regime when the output dimension is (around or at least twice) larger than the input dimension. Toward understanding of the optimal width, the authors draw interesting connections to a geometrical function $w(n,m)$ that describes the minimum dimension needed for restoring a continuous function from a `lifting' embedding. They provide (as the main result) upper and lower bounds on the optimal width, for different classes of activations, using this quantity $w(n,m)$. Then the authors provide a few bounds on this proposed quantity and translate them into the bounds for the MLP optimal widths using the main result.", "questions": "- Please see weaknesses.\n- Can the authors refer to the specific place of misconception and its usage in the proof as mentioned in Remark 4.17?", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n- The proposed bottleneck-like quantity $w(n,m)$ is interesting and insightful, and it closely connects to the minimum UAP width of interest.\n- The paper is nicely written and well explained.\n- The results give a nice addition to this line of research on deep shallow MLPs.\n- The main results are stated in terms of $w(n,m)$ and are general, while later results and remarks in Section 4.3 and 4.4 find nice connections to existing bounds on the minimum width under various settings.\n\nWeaknesses:\n- In Eq.(8) it uses that the projection and the inclusion (or essentially the identity) can be represented by deep narrow MLPs. Is this true for general activations except for ReLU or Leaky-ReLU? Is Condition 1 necessary and/or sufficient? This may deserve an elaboration in the text if I am not missing something trivial. This question also relates to Line 183 in the proof of Theorem 4.4.\n- (minor) The authors should mention that starting from Eq.(8) it assumes $n \\ge d_x, d_y$.\n\nTypos:\n- Line 67 Related Works\n- Eq.(31) should be $\\sigma$ instead of Leaky-ReLU\n- $O$ is not explicitly defined in Definition 4.13\n- $\\ge w(2,2)$ instead of = in Line 300", "was_revised": true, "final_justification": "My questions are addressed in the response. Overall, I believe this is a nice addition to this line of research and is worth being accepted; but I am less familiar with the novelty and the contribution of this work among the existing literature.", "rebuttal": "**Identity function)** Yes, Condition 1 provides a sufficient condition for this, as established in Lemma 4.1 of [1]. We will add this clarification to the paper.\n\n**$n\\ge d_x, d_y$)** Thank you for pointing this out. We will include this condition in the revised version.\n\n**Typos)** Thank you for identifying the errors. We will correct them accordingly.\n\n**Misconception)** In the proof of Lemma 2.4 of [2], the equation $\\det(\\nabla \\Phi) > 0$ does not imply that $\\Phi$ is a diffeomorphism.\n\n\n\n**References**\n\n[1] Kidger, Patrick, and Terry Lyons. \"Universal approximation with deep narrow networks.\" Conference on learning theory. PMLR, 2020.\n\n[2] Li, Li’Ang, et al. \"Minimum width of leaky-ReLU neural networks for uniform universal approximation.\" International Conference on Machine Learning. PMLR, 2023."}]

Author rebuttal:

Meta-review: This paper studies the minimal width required for universal approximation using narrow but deep networks. The paper introduces a relatively new proof technique relying on the ability of DNNs to represent diffeomorphisms, allowing to describe a width that is both necessary and sufficient for universal approximation.

We therefore are happy to accept this paper at NeurIPS, in line with the recommendations of all but one of the reviewers. The only negative review had little concrete criticism and mostly focused on the possibility of a double submission. As pointed by the SAC, this question was settled by the SAC/PC which having found no evidence of a double submission have then decided to disregard the comment.
Decision: Accept (poster)