SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Sibylle Marcotte
ENS - PSL Univ.
sibylle.marcotte@ens.fr
Rémi Gribonval
Univ Lyon, EnsL,UCBL, CNRS, Inria, LIP,
remi.gribonval@inria.fr
Gabriel Peyre
CNRS, ENS - PSL Univ.
gabriel. peyre@ens.fr
Understanding the geometric properties of gradient descent dynamics is a key ingredient in deciphering the recent success of very large machine learning models. A striking observation is that trained over-parameterized models retain some properties of the optimization initialization. This "implicit bias" is believed to be responsible for some favorable properties of the trained models and could explain their good generalization properties. The purpose of this article is threefold. First, we rigorously expose the definition and basic properties of "conservation laws", that define quantities conserved during gradient flows of a given model (e.g. of a ReLU network with a given architecture) with any training data and any loss. Then we explain how to find the maximal number of independent conservation laws by performing finite-dimensional algebraic manipulations on the Lie algebra generated by the Jacobian of the model. Finally, we provide algorithms to: a) compute a family of polynomial laws; b) compute the maximal number of (not necessarily polynomial) independent conservation laws. We provide showcase examples that we fully work out theoretically. Besides, applying the two algorithms confirms for a number of ReLU network architectures that all known laws are recovered by the algorithm, and that there are no other independent laws. Such computational tools pave the way to understanding desirable properties of optimization initialization in large machine learning models.
State-of-the-art approaches in machine learning rely on the conjunction of gradient-based optimization with vastly "over-parameterized" architectures. A large body of empirical [30] and theoretical [5] works suggest that, despite the ability of these models to almost interpolate the input data, they are still able to generalize well. Analyzing the training dynamics of these models is thus crucial to gain a better understanding of this phenomenon. Of particular interest is to understand what properties of the initialization are preserved during the dynamics, which is often loosely referred to as being an "implicit bias" of the training algorithm. The goal of this article is to make this statement precise, by properly defining maximal sets of such "conservation laws", by linking these quantities to algebraic computations (namely a Lie algebra) associated with the model parameterization (in our framework, this parameterization is embodied by a re-parameterization mapping  $\phi$ ), and finally by exhibiting algorithms to implement these computations in SageMath [29].
Over-parameterized model Modern machine learning practitioners and researchers have found that over-parameterized neural networks (with more parameters than training data points), which are often trained until perfect interpolation, have impressive generalization properties [30,5]. This performance seemingly contradicts classical learning theory [25], and a large part of the theoretical deep learning literature aims at explaining this puzzle. The choice of the optimization algorithm is crucial to the model generalization performance [10, 21, 14], thus inducing an implicit bias.
Implicit bias The terminology "implicit bias" informally refers to properties of trained models which are induced by the optimization procedure, typically some form of regularization [22]. For gradient descent, in simple cases such as scalar linear neural networks or two-layer networks with a single neuron, it is actually possible to compute in closed form the implicit bias, which induces some approximate or exact sparsity regularization [10]. Another interesting case is logistic classification on separable data, where the implicit bias selects the max-margin classifier both for linear models [26] and for two-layer neural networks in the mean-field limit [8]. A key hypothesis to explicit the implicit bias is often that the Riemannian metric associated to the over-parameterization is either of Hessian type [10, 3], or can be somehow converted to be of Hessian type [3], which is seemingly always a very strong constraint. For example, even for simple two-layer linear models (i.e., matrix factorization) with more than a single hidden neuron, the Hessian type assumption does not hold, and no closed form is known for the implicit bias [11]. The work of [17] gives conditions on the over-parameterization for this to be possible (for instance certain Lie brackets should vanish). These conditions are (as could be expected) stronger than those required to apply Frobenius theory, as we do in the present work to retrieve conservation laws.
Conservation laws Finding functions conserved during gradient flow optimization of neural networks (a continuous limit of gradient descent often used to model the optimization dynamics) is particularly useful to better understand the flow behavior. One can see conservation laws as a "weak" form of implicit bias: to explain, among a possibly infinite set of minimizers, which properties (e.g. in terms of sparsity, low-rank, etc.) are being favored by the dynamic. If there are enough conservation laws, one has an exact description of the dynamic (see Section 3.4), and in some cases, one can even determine explicitly the implicit bias. Otherwise, one can still predict what properties of the initialization are retained at convergence, and possibly leverage this knowledge. For example, in the case of linear neural networks, certain balancedness properties are satisfied and provide a class of conserved functions [24, 9, 11, 2, 15, 28, 19]. These conservation laws enable for instance to prove the global convergence of the gradient flow under some assumptions. We detail these laws in Proposition 4.1. A subset of these "balancedness" laws still holds in the case of a ReLU activation [9], which reflects the rescaling invariance of these networks (see Section 4 for more details). More generally such conservation laws bear connections with the invariances of the model [16]: to each 1-parameter group of transformation preserving the loss, one can associate a conserved quantity, which is in some sense analogous to Noether's theorem [23, 12]. Similar reasoning is used by [31] to show the influence of initialization on convergence and generalization performance of the neural network. Our work is somehow complementary to this line of research: instead of assuming a priori known symmetries, we directly analyze the model and give access to conservation laws using algebraic computations. For matrix factorization as well as for certain ReLU network architectures, this allows us to show that the conservation laws reported in the literature are complete (there are no other independent quantities that would be preserved by all gradient flows).
We formalize the notion of a conservation law, a quantity preserved through all gradient flows given a model architecture (e.g. a ReLU neural network with prescribed layers). Our main contributions are:
- to show that for several classical losses, characterizing conservation laws for deep linear (resp. shallow ReLU) networks boils down to analyzing a finite dimensional space of vector fields;
- to propose an algorithm (coded in SageMath) identifying polynomial conservation laws on linear / ReLU network architectures; it identifies all known laws on selected examples;
- to formally define the maximum number of (not necessarily polynomial) independent conservation laws and characterize it a) theoretically via Lie algebra computations; and b) practically via an algorithm (coded in SageMath) computing this number on worked examples;
- to illustrate that in certain settings these findings allow to rewrite an over-parameterized flow as an "intrinsic" low-dimensional flow;
- to highlight that the cost function associated to the training of linear and ReLU networks, shallow or deep, with various losses (quadratic and more) fully fits the proposed framework.
A consequence of our results is to show for the first time that conservation laws commonly reported in the literature are maximal: there is no other independent preserved quantity (see Propositions 4.3, 4.2, Corollary 4.4) and Section 4.2.
2 Conservation Laws for Gradient Flows
After some reminders on gradient flows, we formalize the notion of conservation laws.
2.1 Gradient dynamics
We consider learning problems, where we denote  $x_{i} \in \mathbb{R}^{m}$  the features and  $y_{i} \in \mathcal{V}$  the targets (for regression, typically with  $\mathcal{V} = \mathbb{R}^n$ ) or labels (for classification) in the case of supervised learning, while  $y_{i}$  can be considered constant for unsupervised/self-supervised learning. We denote  $X := (x_{i})_{i}$  and  $Y := (y_{i})_{i}$ . Prediction is performed by a parametric mapping  $g(\theta, \cdot): \mathbb{R}^m \to \mathbb{R}^n$  (for instance a neural network) which is trained by empirically minimizing over parameters  $\theta \in \Theta \subseteq \mathbb{R}^D$  a cost
$$
\mathcal {E} _ {X, Y} (\theta) := \sum_ {i} \ell \left(g \left(\theta , x _ {i}\right), y _ {i}\right), \tag {1}
$$
where  $\ell$  is the loss function. In practical examples with linear or ReLU networks,  $\Theta$  is either  $\mathbb{R}^D$  or an open set of "non-degenerate" parameters. The goal of this paper is to analyze what functions  $h(\theta)$  are preserved during the gradient flow (the continuous time limit of gradient descent) of  $\mathcal{E}_{X,Y}$ :
$$
\dot {\theta} (t) = - \nabla \mathcal {E} _ {X, Y} (\theta (t)), \text {w i t h} \theta (0) = \theta_ {\text {i n i t}}. \tag {2}
$$
A priori, one can consider different "levels" of conservation, depending whether  $h$  is conserved: during the optimization of  $\mathcal{E}_{X,Y}$  for a given loss  $\ell$  and a given data set  $(x_{i},y_{i})_{i}$ ; or given a loss  $\ell$ , during the optimization of  $\mathcal{E}_{X,Y}$  for any data set  $(x_{i},y_{i})_{i}$ . Note that using stochastic optimization methods and discrete gradients would break the exact preservation of the conservation laws, and only approximate conservation would hold, as remarked in [16].
2.2 Conserved functions
As they are based on gradient flows, conserved functions are first defined locally.
Definition 2.1 (Conservation through a flow). Consider an open subset  $\Omega \subseteq \Theta$  and a vector field  $\chi \in \mathcal{C}^1 (\Omega ,\mathbb{R}^D)$ . By the Cauchy-Lipschitz theorem, for each initial condition  $\theta_{\mathrm{init}}$ , there exists a unique maximal solution  $t\in [0,T_{\theta_{\mathrm{init}}})\mapsto \theta (t,\theta_{\mathrm{init}})$  of the ODE  $\dot{\theta} (t) = \chi (\theta (t))$  with  $\theta (0) = \theta_{\mathrm{init}}$ . A function  $h:\Omega \subseteq \mathbb{R}^{D}\to \mathbb{R}$  is conserved on  $\Omega$  through the vector field  $\chi$  if  $h(\theta (t,\theta_{\mathrm{init}})) = h(\theta_{\mathrm{init}})$  for each choice of  $\theta_{\mathrm{init}}\in \Omega$  and every  $t\in [0,T_{\theta_{\mathrm{init}}})$ . It is conserved on  $\Omega$  through a subset  $W\subset \mathcal{C}^1 (\Omega ,\mathbb{R}^D)$  if  $h$  is conserved on  $\Omega$  during all flows induced by all  $\chi \in W$ .
In particular, one can adapt this definition for the flow induced by the cost  $(\mathbb{D})$
Definition 2.2 (Conservation during the flow  $\boxed{2}$  with a given dataset). Consider an open subset  $\Omega \subseteq \Theta$  and a dataset  $(X,Y)$  such that  $\mathcal{E}_{X,Y} \in \mathcal{C}^2(\Omega, \mathbb{R})$ . A function  $h: \Omega \subseteq \mathbb{R}^D \to \mathbb{R}$  is conserved on  $\Omega$  during the flow  $\boxed{2}$  if it is conserved through the vector field  $\chi(\cdot) \coloneqq \nabla \mathcal{E}_{X,Y}(\cdot)$ .
Our goal is to study which functions are conserved during "all" flows defined by the ODE (2). This in turn leads to the following definition.
Definition 2.3 (Conservation during the flow  $(\square)$  with "any" dataset). Consider an open subset  $\Omega \subset \Theta$  and a loss  $\ell(z, y)$  such that  $\ell(\cdot, y)$  is  $\mathcal{C}^2$ -differentiable for all  $y \in \mathcal{Y}$ . A function  $h: \Omega \subseteq \mathbb{R}^D \to \mathbb{R}$  is conserved on  $\Omega$  for any data set if, for each data set  $(X, Y)$  such that  $g(\cdot, x_i) \in \mathcal{C}^2(\Omega, \mathbb{R})$  for each  $i$ , the function  $h$  is conserved on  $\Omega$  during the flow  $(\square)$ . This leads us to introduce the family of vector fields:
$$
W _ {\Omega} ^ {g} := \left\{\chi (\cdot): \exists X, Y, \forall i g (\cdot , x _ {i}) \in \mathcal {C} ^ {2} (\Omega , \mathbb {R}), \chi = \nabla \mathcal {E} _ {X, Y} \right\} \subseteq \mathcal {C} ^ {1} (\Omega , \mathbb {R} ^ {D}) \tag {3}
$$
so that being conserved on  $\Omega$  for any dataset is the same as being conserved on  $\Omega$  through  $W_{\Omega}^{g}$ .
The above definitions are local and conditioned on a choice of open set of parameters  $\Omega \subset \Theta$ . We are rather interested in functions defined on the whole parameter space  $\Theta$ , hence the following definition.
Definition 2.4. A function  $h: \Theta \mapsto \mathbb{R}$  is locally conserved on  $\Theta$  for any data set if for each open subset  $\Omega \subseteq \Theta$ ,  $h$  is conserved on  $\Omega$  for any data set.
A basic property of  $\mathcal{C}^1$  conserved functions (which proof can be found in Appendix A) corresponds to an "orthogonality" between their gradient and the considered vector fields.
Proposition 2.5. Given a subset  $W \subset \mathcal{C}^1(\Omega, \mathbb{R}^D)$ , its trace at  $\theta \in \Omega$  is defined as the linear space
$$
W (\theta) := \operatorname {s p a n} \left\{\chi (\theta): \chi \in W \right\} \subseteq \mathbb {R} ^ {D}. \tag {4}
$$
A function  $h\in \mathcal{C}^1 (\Omega ,\mathbb{R})$  is conserved on  $\Omega$  through  $W$  if, and only if  $\nabla h(\theta)\perp W(\theta),\forall \theta \in \Omega$
Therefore, combining Proposition 2.5 and Definition 2.4, the object of interest to study locally conserved functions is the union of the traces
$$
W _ {\theta} ^ {g} := \bigcup \left\{W _ {\Omega} ^ {g} (\theta): \Omega \subseteq \Theta \text {w i t h} \Omega \text {a n e i g h b o r h o o d} \theta \right\}. \tag {5}
$$
Corollary 2.6. A function  $h: \Theta \mapsto \mathbb{R}$  is locally conserved on  $\Theta$  for any data set if and only if  $\nabla h(\theta) \perp W_{\theta}^{g}$  for all  $\theta \in \Theta$ .
It will soon be shown (cf Theorem 2.14) that  $W_{\theta}^{g}$  can be rewritten as the trace  $W(\theta)$  of a simple finite-dimensional functional space  $W$ . Meanwhile, we keep the specific notation. For the moment, this set is explicitly characterized via the following proposition (which proof can be found in Appendix B).
Proposition 2.7. Assume that for each  $y \in \mathcal{V}$  the loss  $\ell(z, y)$  is  $\mathcal{C}^2$ -differentiable with respect to  $z \in \mathbb{R}^n$ . For each  $\theta \in \Theta$  we have:
$$
W _ {\theta} ^ {g} = \operatorname * {s p a n} _ {(x, y) \in \mathcal {X} _ {\theta} \times \mathcal {Y}} \left\{\left[ \partial_ {\theta} g (\theta , x) \right] ^ {\top} \nabla_ {z} \ell (g (\theta , x), y) \right\}
$$
where  $\mathcal{X}_{\theta}$  is the set of data points  $x$  such that  $g(\cdot ,x)$  is  $\mathcal{C}^2$ -differentiable in the neighborhood of  $\theta$ .
Example 2.8. As a first simple example, consider a two-layer linear neural network in dimension 1 (both for the input and output), with a single neuron. For such - admittedly trivial - architecture, the parameter is  $\theta = (u,v)\subseteq \mathbb{R}^2$  and the model writes  $g(\theta ,x) = uvx$ . One can directly check that the function:  $h(u,v) = u^{2} - v^{2}$  is locally conserved on  $\mathbb{R}^2$  for any data set. Indeed in that case  $\nabla h(u,v) = (2u, - 2v)^{\top}\bot W_{\theta}^{g} = \operatorname *{span}_{(x,y)\in \mathbb{R}\times \mathcal{V}}\{(vx,ux)^{\top}\nabla_{z}\ell (g(\theta ,x),y)\} = \mathbb{R}\times (v,u)^{\top}$  given that the gradient  $\nabla_z\ell (g(\theta ,x),y)$  is an arbitrary scalar.
In this example we obtain a simple expression of  $W_{\theta}^{g}$ , however in general cases it is not possible to obtain such a simple expression from Proposition 2.7. We will show that in some cases, it is possible to express  $W_{\theta}^{g}$  as the trace  $W(\theta)$  of a simple finite-dimensional space  $W$  (cf. Theorem 2.14).
2.3 Reparametrization
To make the mathematical analysis tractable and provide an algorithmic procedure to determine these functions, our fundamental hypothesis is that the model  $g(\theta, x)$  can be (locally) factored via a reparametrization  $\phi$  as  $f(\phi(\theta), x)$ . We require that the model  $g(\theta, x)$  satisfies the following central assumption.
Assumption 2.9 (Local reparameterization). There exists  $d$  and  $\phi \in \mathcal{C}^2 (\Theta ,\mathbb{R}^d)$  such that: for each parameter  $\theta_0$  in the open set  $\Theta \subseteq \mathbb{R}^{D}$ , for each  $x\in \mathcal{X}$  such that  $\theta \mapsto g(\theta ,x)$  is  $\mathcal{C}^2$  in a neighborhood of  $\theta_0\mathbb{I}$ , there is a neighborhood  $\Omega$  of  $\theta_0$  and  $f(\cdot ,x)\in \mathcal{C}^2 (\phi (\Omega),\mathbb{R}^n)$  such that
$$
\forall \theta \in \Omega , \quad g (\theta , x) = f (\phi (\theta), x). \tag {6}
$$
Note that if the model  $g(\cdot, x)$  is smooth on  $\Omega$  then (6) is always satisfied with  $\phi := \mathrm{id}$  and  $f(\cdot, x) := g(\cdot, x)$ , yet this trivial factorization fails to capture the existence and number of conservation laws as studied in this paper. This suggests that, among all factorizations shaped as (6), there may be a notion of an optimal one.
Example 2.10. (Factorization for linear neural networks) In the two-layer case, with  $r$  neurons, denoting  $\theta = (U,V) \in \mathbb{R}^{n \times r} \times \mathbb{R}^{m \times r}$  (so that  $D = (n + m)r$ ), we can factorize  $g(\theta,x) := UV^{\top}x$  by the reparametrization  $\phi(\theta) := UV^{\top} \in \mathbb{R}^{n \times m}$  using  $f(\phi,x) = \phi \cdot x$ . More generally for  $q$  layers, with  $\theta = (U_1,\dots ,U_q)$ , we can still factorize  $g(\theta,x) := U_1 \dots U_qx$  using  $\phi(\theta) := U_1 \dots U_q$  and the same  $f$ . This factorization is globally valid on  $\Omega = \Theta = \mathbb{R}^D$  since  $f(\cdot,x)$  does not depend on  $\theta_0$ .
The notion of locality of the factorization (6) is illustrated by the next example.
i.e.,  $x$  belongs to the set  $\mathcal{X}_{\theta_0}$ , as defined in Proposition 2.7.
Example 2.11 (Factorization for two-layer ReLU networks). Consider  $g(\theta, x) = \left( \sum_{j=1}^{r} u_{k,j} \sigma(\langle v_j, x \rangle + b_j) + c_k \right)_{k=1}^n$ , with  $\sigma(t) := \max(t,0)$  the ReLU activation function and  $v_j \in \mathbb{R}^m$ ,  $u_{k,j} \in \mathbb{R}$ ,  $b_j, c_k \in \mathbb{R}$ . Then, denoting  $\theta = (U, V, b, c)$  with  $U = (u_{k,j})_{k,j} := (u_1, \dots, u_r) \in \mathbb{R}^{n \times r}$ ,  $V = (v_1, \dots, v_r) \in \mathbb{R}^{m \times r}$ ,  $b = (b_1, \dots, b_r)^\top \in \mathbb{R}^r$  and  $c = (c_1, \dots, c_n) \in \mathbb{R}^n$  (so that  $D = (n + m + 1)r + n$ ), we rewrite  $g(\theta, x) = \sum_{j=1}^{r} u_j \varepsilon_{j,x} (v_j^\top x + b_j) + c$  where, given  $x, \varepsilon_{j,x} = \mathbb{1}(v_j^\top x + b_j > 0)$  is piecewise constant with respect to  $\theta$ . Consider  $\theta^0 = (U^0, V^0, b^0, c^0) \in \mathbb{R}^D$  where  $V^0 = (v_1^0, \dots, v_r^0)$  and  $b^0 = (b_1^0, \dots, b_r^0)^\top$ . Then the set  $X_{\theta^0}$  introduced in Proposition 2.7 is  $X_{\theta^0} = \mathbb{R}^m - \cup_j \{v_j^\top x + b_j^0 = 0\}$ . Let  $x \in X_{\theta^0}$ . Then on any domain  $\Omega \subset \mathbb{R}^D$  such that  $\theta^0 \in \Omega$  and  $\varepsilon_{j,x}(\theta) := \mathbb{1}(v_j^\top x + b_j > 0)$  is constant over  $\theta \in \Omega$ , the model  $g_\theta(x)$  can be factorized by the reparametrization  $\phi(\theta) = ((u_j v_j^\top, u_j b_j)_{j=1}^r, c)$ . In particular, in the case without bias  $((b,c) = (0,0))$ , the reparametrization is defined by  $\phi(\theta) = (\phi_j)_{j=1}^r$  where  $\phi_j = \phi_j(\theta) := u_j v_j^\top \in \mathbb{R}^{n \times m}$  (here  $d = rmn$ ) using  $f(\phi,x) = \sum_j \varepsilon_{j,x} \phi_j x$ : the reparametrization  $\phi(\theta)$  contains  $r$  matrices of size  $m \times n$  (each of rank at most one) associated to a "local"  $f(\cdot, x)$  valid in a neighborhood of  $\theta$ . A similar factorization is possible for deeper ReLU networks [27], as further discussed in the proof of Theorem 2.14 in Appendix C.
Combining Proposition 2.7 and using chain rules, we get a new characterization of  $W_{\theta}^{g}$ :
Proposition 2.12. Assume that the loss  $\ell(z, y)$  is  $\mathcal{C}^2$ -differentiable with respect to  $z$ . We recall (cf [5]) that  $W_{\theta}^{g} \coloneqq \cup_{\Omega \subseteq \Theta : \Omega \text{ is open and } \Omega \ni \theta} W_{\Omega}^{g}(\theta)$ . Under Assumption [2.9] for all  $\theta \in \Theta$ :
$$
W _ {\theta} ^ {g} = \partial \phi (\theta) ^ {\top} W _ {\phi (\theta)} ^ {f} \tag {7}
$$
with  $\partial \phi (\theta)\in \mathbb{R}^{d\times D}$  the Jacobian of  $\phi$  and  $W_{\phi (\theta)}^f\coloneqq \operatorname *{span}_{(x,y)\in \mathcal{X}_\theta \times \mathcal{Y}}\{\partial f^x (\phi (\theta))^{\top}\nabla_z\ell (g(\theta ,x),y)\}$  where  $f^{x}(\cdot)\coloneqq f(\cdot ,x)$
We show in Section 2.4 that, under mild assumptions on the loss  $\ell$ ,  $W_{\phi(\theta)}^f = \mathbb{R}^d$ , so that Proposition 2.12 yields  $W_\theta^g = \mathrm{range}(\partial \phi(\theta)^\top)$ . Then by Corollary 2.6, a function  $h$  that is locally conserved on  $\Theta$  for any data set is entirely characterized via the kernel of  $\partial \phi(\theta)^\top$ :  $\partial \phi(\theta)^\top \nabla h(\theta) = 0$  for all  $\theta \in \Theta$ . The core of our analysis is then to analyze the (Lie algebraic) structure of  $\mathrm{range}(\partial \phi(\cdot)^\top)$ .
2.4 From conserved functions to conservation laws
For linear and ReLU networks we show in Theorem 2.14 and Proposition 2.16 that under (mild) assumptions on the loss  $\ell(\cdot, \cdot)$ , being locally conserved on  $\Theta$  for any data set (according to Definition 2.4) is the same as being conserved (according to Definition 2.1) on  $\Theta$  through the finite-dimensional subspace
$$
W _ {\phi} := \operatorname {s p a n} \left\{\nabla \phi_ {1} (\cdot), \dots , \nabla \phi_ {d} (\cdot) \right\} = \left\{\theta \mapsto \sum a _ {i} \nabla \phi_ {i} (\theta): (a _ {1}, \dots , a _ {d}) \in \mathbb {R} ^ {d} \right\} \tag {8}
$$
where we write  $\partial \phi (\theta)^{\top} = (\nabla \phi_{1}(\theta),\dots ,\nabla \phi_{d}(\theta))\in \mathbb{R}^{t}\times d$  with  $\nabla \phi_{i}\in \mathcal{C}^{1}(\Theta ,\mathbb{R}^{D})$
The following results (which proofs can be found in Appendix C) establish that in some cases, the functions locally conserved for any data set are exactly the functions conserved through  $W_{\phi}$ .
Lemma 2.13. Assume that the loss  $(z,y)\mapsto \ell (z,y)$  is  $\mathcal{C}^2$ -differentiable with respect to  $z\in \mathbb{R}^n$  and satisfies the condition:
$$
\underset {y \in \mathcal {Y}} {\operatorname {s p a n}} \left\{\nabla_ {z} \ell (z, y) \right\} = \mathbb {R} ^ {n}, \forall z \in \mathbb {R} ^ {n}. \tag {9}
$$
Then for linear neural networks (resp. for two-layer ReLU networks) and all  $\theta \in \Theta$  we have  $W_{\phi (\theta)}^{f} = \mathbb{R}^{d}$ , with the reparametrization  $\phi$  from Example 2.10 and  $\Theta \coloneqq \mathbb{R}^D$  (resp. with  $\phi$  from Example 2.11 and  $\Theta$  consisting of all parameter  $\theta$  of the network such that hidden neurons are associated to pairwise distinct "hyperplanes", cf Appendix C for details).