SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
(i, j) \in \operatorname {s u p p} (I [:, k ] J [ k,: ]) a n d (i, j) \notin \operatorname {s u p p} (I [:, \ell ] J [ \ell , : ]), \forall \ell \neq k.
$$
2. The pair  $(i_2,j_2)$  belongs to  $\mathrm{supp}(I[:,k]J[k,:])$  and to  $\mathrm{supp}(I[:,l]J[l,:])$ .
then  $\mathcal{L}_{\mathbf{I}}$  is non-closed.
Proof. First, it is easy to see that the assumptions of this lemma are equivalent to those of [18, Theorem 4.20] since  $\operatorname{supp}(I(:, k]J[k,:])$  is precisely the  $k$ th rank-one support of the pair  $(I,J)$  [18, Definition 3.1]. Without loss of generality, one can assume that  $i_1,j_1 = 1,i_2,j_2 = 2$  and  $k = 1,l = 2$ . We will prove that  $\mathbf{A} \in \overline{\mathcal{L}_{\mathbf{I}}} \setminus \mathcal{L}_{\mathbf{I}}$  where
$$
\mathbf {A} := \left( \begin{array}{c c} \mathbf {A} ^ {\prime} & \mathbf {0} \\ \mathbf {0} & \mathbf {0} \end{array} \right) \in \mathbb {R} ^ {m \times n}, \text {w i t h} \mathbf {A} ^ {\prime} := \left( \begin{array}{c c} 0 & 1 \\ 1 & 0 \end{array} \right) \in \mathbb {R} ^ {2 \times 2}.
$$
This can be shown in two steps:
680   
681   
682   
683   
684   
685   
686   
687   
688   
689   
690   
691   
692   
693   
694   
695   
696   
697
1. Proof that  $\mathbf{A} \in \overline{\mathcal{L}_1}$ : For any  $\epsilon > 0$ , consider two factors:
$$
\mathbf {X} _ {\epsilon} = \left( \begin{array}{c c} \mathbf {X} _ {\epsilon} ^ {\prime} & \mathbf {0} \\ \mathbf {0} & \mathbf {0} \end{array} \right), \mathbf {Y} _ {\epsilon} = \left( \begin{array}{c c} \mathbf {Y} _ {\epsilon} ^ {\prime} & \mathbf {0} \\ \mathbf {0} & \mathbf {0} \end{array} \right)
$$
where  $\mathbf{X}_{\epsilon}^{\prime}, \mathbf{Y}_{\epsilon}^{\prime} \in \mathbb{R}^{2 \times 2}$  respect the support constraints corresponding to the LU architecture. It is not hard to see that such a construction of  $(\mathbf{X}_{\epsilon}, \mathbf{Y}_{\epsilon})$  satisfies the support constraints  $(I, J)$  (due to the assumption of the lemma and the value of indices). Moreover, we also have:
$$
\left\| \mathbf {A} - \mathbf {X} _ {\epsilon} \mathbf {Y} _ {\epsilon} \right\| _ {F} = \left\| \mathbf {A} ^ {\prime} - \mathbf {X} _ {\epsilon} ^ {\prime} \mathbf {Y} _ {\epsilon} ^ {\prime} \right\| _ {F}
$$
Thus, to have  $\| \mathbf{A} - \mathbf{X}_{\epsilon}\mathbf{Y}_{\epsilon}\|_{F}\leq \epsilon$ , it is sufficient to choose a pair of factors  $(\mathbf{X}_{\epsilon}^{\prime},\mathbf{Y}_{\epsilon}^{\prime})$  respecting the LU architecture of size  $2\times 2$  such that  $\| \mathbf{A}' - \mathbf{X}_\epsilon '\mathbf{Y}_\epsilon '\| _F\leq \epsilon$ . Such a pair exists, since the set of matrices admitting the exact LU decomposition is dense in  $\mathbb{R}^{2\times 2}$ . This holds for any  $\epsilon >0$ . Therefore,  $\mathbf{A}\in \overline{\mathcal{L}_{\mathbf{I}}}$
2. Proof that  $\mathbf{A} \notin \mathcal{L}_{\mathbf{I}}$ : Assume there exist a pair of factors  $(\mathbf{X}, \mathbf{Y})$  whose product  $\mathbf{X}\mathbf{Y} = \mathbf{A}$  and supports are included in  $(I, J)$ . Due to the assumptions on the pairs  $(i_1, j_1), (i_1, j_2), (i_2, j_1)$ , we must have:
$$
\left\{ \begin{array}{l l} \mathbf {X} [ 1, 1 ] \mathbf {Y} [ 1, 1 ] & = \mathbf {A} [ 1, 1 ] = 0 \\ \mathbf {X} [ 2, 1 ] \mathbf {Y} [ 1, 1 ] & = \mathbf {A} [ 2, 1 ] = 1 \\ \mathbf {X} [ 1, 1 ] \mathbf {Y} [ 1, 2 ] & = \mathbf {A} [ 1, 2 ] = 1. \end{array} \right.
$$
It is easy to see that it is impossible. Therefore,  $\mathbf{A} \notin \mathcal{L}_{\mathbf{I}}$ .
Given a pair of support constraints  $\mathbf{I}$ , it is possible to check in time polynomial in  $m, r, n$  whether the conditions of Lemma B.8 hold. A brute algorithm has complexity  $O(m^2 n^2 r)$ . A more clever implementation with careful book-marking can reduce this complexity to  $O(\min(m, n) mn r)$ .
C Proofs for results in Section 4
In fact, Theorem 4.1 is a corollary of Lemma B.1. Thus, we will give a proof for Lemma B.1 in the following.
Proof of Lemma B.1. Since  $\mathbf{A} \in \overline{\mathcal{L}_{\mathbf{I}}} \backslash \mathcal{L}_{\mathbf{I}} \subseteq \mathbb{R}^{N_L \times N_0}$ , we have:
1.  $\mathbf{A}\notin \mathcal{L}_{\mathbf{I}}$
2. There exists a sequence  $\{(\mathbf{X}_i^k)_{i=1}^L\}_{k \in \mathbb{N}}$  such that  $\lim_{k \to \infty} \| \mathbf{X}_L^k \dots \mathbf{X}_1^k - \mathbf{A} \| = 0$  for any norm defined on  $\mathbb{R}^{N_0}$ .
We will prove that the linear function:  $f(x) \coloneqq \mathbf{A}x$  satisfies  $f \in \overline{\mathcal{F}}_{\mathbf{I}} \setminus \mathcal{F}_{\mathbf{I}}$  (where  $\overline{\mathcal{F}}_{\mathbf{I}}$  is the closure of  $\mathcal{F}_{\mathbf{I}}$  in  $(C^0(\Omega), \| \cdot \|_\infty)$ ), that is to say  $f$  is not the realization of any neural network but it is the uniform limit of the realizations of a sequence of neural networks). Firstly, we prove that  $f \notin \mathcal{F}_{\mathbf{I}}$ . For the sake of contradiction, assume there exists  $\theta = (\mathbf{W}_i, \mathbf{b}_i)_{i=1}^L \in \mathcal{N}_{\mathbf{I}}$  such that  $\mathcal{R}_{\theta} = f$ . Since  $\mathcal{R}_{\theta}$  is the realization of a ReLU neural network, it is a continuous piecewise linear function. Therefore, since  $\Omega$  has non-empty interior, there exist a non-empty open subset  $\Omega'$  of  $\mathbb{R}^d$  such that  $\Omega' \subseteq \Omega$  and  $\mathcal{R}_{\theta}$  is linear on  $\Omega'$ , i.e., there are  $\mathbf{A}' \in \mathbb{R}^{N_L \times N_0}$ ,  $\mathbf{b} \in \mathbb{R}^{N_L}$  such that  $\mathcal{R}_{\theta}(x) = \mathbf{A}'x + \mathbf{b}', \forall x \in \Omega'$ . Since  $f = \mathcal{R}_{\theta}$ , we have:  $\mathbf{A}' = \mathbf{A}$  and also equal to the Jacobian matrix of  $\mathcal{R}_{\theta}$  on  $\Omega'$ . Using Lemma B.3 and the fact that  $\mathbf{A} \notin \mathcal{L}_{\mathbf{I}}$ , we conclude that  $f \notin \mathcal{F}_{\mathbf{I}}$ .
There remains to construct a sequence  $\{\theta^k\}_{k\in \mathbb{N}},\theta^k = (\mathbf{W}_i^k,\mathbf{b}_i^k)_{i = 1}^L\in \mathcal{N}_{\mathbf{I}}$  such that  $\lim_{k\to \infty}\| \mathcal{R}_{\theta^k} - f\|_{\infty} = 0$ . We will rely on the sequence  $\{(\mathbf{X}_i^k)_{i = 1}^L\}_{k\in \mathbb{N}}$  for our construction. Given  $k\in \mathbb{N}$  we simply define the weight matrices as  $\mathbf{W}_i^k = \mathbf{X}_i^k,1\leq i\leq L$ . The biases are built recursively. Starting from  $c_1^k\coloneqq \sup_{x\in \Omega}\| \mathbf{W}_1^k x\|_\infty$  and  $\mathbf{b}_1^k\coloneqq c_1^k\mathbf{1}_{N_1}$ , we iteratively define for  $2\leq i\leq L - 1$ :
$$
\begin{array}{l} \gamma_ {i - 1} ^ {k} (x) := \mathbf {W} _ {i - 1} ^ {k} x + \mathbf {b} _ {i - 1} \\ c _ {i} ^ {k} := \sup  _ {x \in \Omega} \| \gamma_ {i - 1} ^ {k} \circ \dots \circ \gamma_ {1} ^ {k} (x) \| _ {\infty} \\ \mathbf {b} _ {i} ^ {k} := c _ {i} ^ {k} \mathbf {1} _ {N _ {i}}. \\ \end{array}
$$
714   
715   
716   
717   
718   
719   
720   
721   
722   
723   
724   
725   
726   
727   
728   
729   
730   
731   
732   
733   
734   
735   
736   
737   
738
The boundedness of  $\Omega$  ensures that  $c_{i}^{k}$  is well-defined with a finite supremum. For  $i = L$  we define:
$$
\mathbf {b} _ {L} ^ {k} = - \sum_ {i = 1} ^ {L - 1} (\prod_ {j = i + 1} ^ {L} \mathbf {W} _ {j}) \mathbf {b} _ {i} ^ {k}.
$$
We will prove that  $\mathcal{R}_{\theta^k}(x) = \left(\mathbf{X}_L^k\ldots \mathbf{X}_1^k\right)x,\forall x\in \Omega$  . As a consequence, it is immediate that:
$$
\begin{array}{l} \lim  _ {k \rightarrow \infty} \| \mathcal {R} _ {\theta^ {k}} - f \| _ {\infty} = \lim  _ {k \rightarrow \infty} \sup  _ {x \in \Omega} \| \mathcal {R} _ {\theta^ {k}} (x) - f (x) \| _ {2} \\ \leq \lim  _ {k \rightarrow \infty} \| \mathbf {X} _ {L} ^ {k} \dots \mathbf {X} _ {1} ^ {k} - \mathbf {A} \| _ {2 \rightarrow 2} \sup  _ {x \in \Omega} \| x \| _ {2} = 0 \\ \end{array}
$$
where we used that all matrix norms are equivalent and denoted  $\| \cdot \|_{2\to 2}$  the operator norm associated to Euclidean vector norms. Back to the proof that  $\mathcal{R}_{\theta^k}(x) = \left(\mathbf{X}_L^k\dots \mathbf{X}_1^k\right)x, \forall x\in \Omega$ , due to our choice of  $c_i^k$ , we have for  $2\leq i\leq L - 1$ :
$$
\gamma_ {i - 1} ^ {k} \circ \dots \circ \gamma_ {1} ^ {k} (x) \geq 0, \forall x \in \Omega
$$
where  $\geq$  is taken in coordinate-wise manner. Therefore, an easy induction yields:
$$
\begin{array}{l} \mathcal {R} _ {\theta^ {k}} (x) = \gamma_ {L} ^ {k} \circ \sigma \circ \gamma_ {L - 1} ^ {k} \circ \dots \circ \sigma \circ \gamma_ {1} ^ {k} (x) \\ = \gamma_ {L} ^ {k} \circ \gamma_ {L - 1} ^ {k} \dots \circ \gamma_ {1} ^ {k} (x) \\ = \mathbf {W} _ {L} ^ {k} (\dots (\mathbf {W} _ {2} ^ {k} (\mathbf {W} _ {1} ^ {k} x + \mathbf {b} _ {1} ^ {k}) + \mathbf {b} _ {2} ^ {k}) \dots) + \mathbf {b} _ {L} ^ {k} \\ = \left(\mathbf {X} _ {L} ^ {k} \dots \mathbf {X} _ {1} ^ {k}\right) x + \sum_ {i = 1} ^ {L - 1} \left(\prod_ {j = i + 1} ^ {L} \mathbf {W} _ {j}\right) \mathbf {b} _ {i} ^ {k} - \sum_ {i = 1} ^ {L - 1} \left(\prod_ {j = i + 1} ^ {L} \mathbf {W} _ {j}\right) \mathbf {b} _ {i} ^ {k} \\ = \left(\mathbf {X} _ {L} ^ {k} \dots \mathbf {X} _ {1} ^ {k}\right) x. \\ \end{array}
$$
C.2 Proof of Theorem 4.2
Given the involvement of Theorem 4.2, we decompose its proof and present it in two subsections: the first one establishes general results that do not use the assumption of Theorem 4.2. The second one combines the established results with the assumption of Theorem 4.2 to provide a full proof.
C.2.1 Properties of the limit function of fixed support one-hidden-layer NNs
The main results of this part are summarized in Lemma C.2 and Lemma C.3. It is important to emphasize that all results in this section do not make any assumption on  $\mathbf{I}$ .
We first introduce the following technical results.
Lemma C.1 (Normalization of the rows of the first layer [26]). Consider  $\Omega$  a bounded subset of  $\mathbb{R}^{N_0}$ . Given any  $\theta = \{(\mathbf{W}_i,\mathbf{b}_i)_{i = 1}^2\} \in \mathcal{N}_{\mathbf{I}}$  and any norm  $\| \cdot \|$  on  $\mathbb{R}^{N_0}$ , there exists  $\tilde{\theta} := \{(\tilde{\mathbf{W}}_i,\tilde{\mathbf{b}}_i)_{i = 1}^2\} \in \mathcal{N}_{\mathbf{I}}$  such that the matrix  $\tilde{\mathbf{W}}_1$  has unit norm rows,  $\| \tilde{\mathbf{b}}_1\|_\infty \leq C := \sup_{x\in \Omega}\sup_{\| u\| \leq 1}\langle u,x\rangle$  and  $\mathcal{R}_{\theta}(x) = \mathcal{R}_{\tilde{\theta}}(x),\forall x\in \Omega$ .
Proof. We report this proof for self-completeness of this work. It is not a contribution, as it merely combines ideas from the proof of [26, Lemma D.2] and [26, Theorem 3.8, Steps 1-2].
We first show that for each set of weights  $\theta \in \mathcal{N}_{\mathbf{I}}$  we can find another set of weights  $\theta' = \{(\mathbf{W}_i', \mathbf{b}_i')_{i=1}^2\} \in \mathcal{N}_{\mathbf{I}}$  such that  $\mathcal{R}_{\theta} = \mathcal{R}_{\theta'}$  on  $\mathbb{R}^{N_0}$  and  $\mathbf{W}_1'$  has unit norm rows. Note that  $\|\mathbf{b}_1'\|_{\infty}$  can be larger than  $C$ . Indeed, given  $\theta \in \mathcal{N}_{\mathbf{I}}$ , the function  $\mathcal{R}_{\theta}$  can be written as:  $\mathcal{R}_{\theta}: x \in \mathbb{R}^{N_0} \mapsto \sum_{j=1}^{N_1} h_j(x) + \mathbf{b}_2$  where  $h_j(x) = \mathbf{W}_2[(:, j] \sigma(\mathbf{W}_1[j,:]x + \mathbf{b}_1[j])$  denotes the contribution of the  $j$ th hidden neuron. For hidden neurons corresponding to nonzero rows of  $\mathbf{W}_1^k$ , we can rescale the rows of  $\mathbf{W}_1^k$ , the columns of  $\mathbf{W}_2^k$  and  $\mathbf{b}_1^k$  such that the realization of  $h_j$  is invariant. This is due to the fact that  $\mathbf{w}_2 \sigma(\mathbf{w}_1^\top x + b) = \| \mathbf{w}_1 \| \mathbf{w}_2 \sigma((\mathbf{w}_1 / \| \mathbf{w}_1 \|)^\top x + (b / \| \mathbf{w}_1 \|))$  for any  $\mathbf{w}_1 \neq \mathbf{0} \in \mathbb{R}^{N_0}$ ,  $\mathbf{w}_2 \in \mathbb{R}^{N_2}$ ,  $b \in \mathbb{R}$ . Neurons corresponding to null rows of  $\mathbf{W}_1^k$  are handled similarly,
739   
740   
741   
742   
743   
744   
745   
746   
747   
748   
749   
750   
751   
752   
753   
754   
755   
756   
757   
758   
759   
760   
761
in an iterative manner, by setting them to an arbitrary normalized row, setting the corresponding column of  $\mathbf{W}_2^k$  to zero, and changing the bias  $\mathbf{b}_2^k$  to keep the function  $\mathcal{R}_{\theta}$  unchanged on  $\mathbb{R}^{N_0}$ , using that  $\mathbf{w}_2\sigma (\mathbf{0}^\top x + b) + \mathbf{b}_2 = \mathbf{0}\sigma (\mathbf{v}^\top x + b) + (\mathbf{b}_2 + \mathbf{w}_2\sigma (b))$  for any normalized vector  $\mathbf{v}\in \mathbb{R}^{N_0}$ . Thus, we obtain  $\theta^\prime$  whose matrix of the first layer,  $\mathbf{W}_1^\prime$ , has normalized rows and  $\mathcal{R}_{\theta} = \mathcal{R}_{\theta^{\prime}}$  on  $\mathbb{R}^{N_0}$ .
To construct  $\tilde{\theta}$  with  $\| \tilde{\mathbf{b}}_1\|_{\infty}\leq C$  we see that, by definition of  $C$ , if  $\| \mathbf{w}_1\| = 1$  and  $b\geq C$  then
$$
\mathbf {w} _ {1} ^ {\top} x + b \geq - C + b \geq 0, \quad \forall x \in \Omega . \tag {20}
$$
Thus, the function  $\mathbf{w}_2\sigma (\mathbf{w}_1^\top x + b) = \mathbf{w}_2(\mathbf{w}_1^\top x + b)$  is linear on  $\Omega$  and
$$
\begin{array}{l} \mathbf {w} _ {2} \sigma \left(\mathbf {w} _ {1} ^ {\top} x + b\right) + \mathbf {b} _ {2} = \mathbf {w} _ {2} \left(\mathbf {w} _ {1} ^ {\top} x + C\right) + ((b - C) \mathbf {w} _ {2} + \mathbf {b} _ {2}) \\ = \mathbf {w} _ {2} \sigma (\mathbf {w} _ {1} ^ {\top} x + C) + ((b - C) \mathbf {w} _ {2} + \mathbf {b} _ {2}) \\ \end{array}
$$
Thus, for any hidden neuron with a bias exceeding  $C$ , the bias can be saturated to  $C$  by changing accordingly the output bias  $\mathbf{b}_2$ , keeping the function  $\mathcal{R}_{\theta}$  unchanged on the bounded domain  $\Omega$  (but not on the whole space  $\mathbb{R}^{N_0}$ ). Hidden neurons with a bias  $b \leq -C$  can be similarly modified. Sequentially saturating each hidden neuron yields  $\tilde{\theta}$  which satisfies all conditions of Lemma C.1.
Lemma C.2. Consider  $\Omega$  a bounded subset of  $\mathbb{R}^{N_0}$ , for any  $\mathbf{I} = (I_2, I_1)$ , given a continuous function  $f \in \overline{\mathcal{F}_{\mathbf{I}}(\Omega)}$ , there exists a sequence  $\{\theta^k\}_{k \in \mathbb{N}}, \theta^k = (\mathbf{W}_i^k, \mathbf{b}_i^k)_{i=1}^2 \in \mathcal{N}_{\mathbf{I}}$  such that:
1. The sequence  $\mathcal{R}_{\theta^k}$  admits  $f$  as its uniform limit, i.e.,  $\lim_{k\to \infty}\| \mathcal{R}_{\theta^k} - f\|_{\infty} = 0$
2. The sequence  $\{(\mathbf{W}_1^k,\mathbf{b}_1^k)\}_{k\in \mathbb{N}}$  has a finite limit  $(\mathbf{W}_1^\star ,\mathbf{b}_1^\star)$  where  $\mathbf{W}_1^\star$  has unit norm rows and  $\mathrm{supp}(\mathbf{W}_1^\star)\subseteq I_1$
Proof. Given a function  $f \in \overline{\mathcal{F}_{\mathbf{I}}(\Omega)}$ , by definition, there exists a sequence  $\{\theta^k\}_{k \in \mathbb{N}}$ ,  $\theta^k \in \mathcal{N}_{\mathbf{I}} \forall k \in \mathbb{N}$  such that  $\lim_{k \to \infty} \| \mathcal{R}_{\theta^k} - f \|_{\infty} = 0$ . We can assume that  $\mathbf{W}_1^k$  has normalized rows and  $\mathbf{b}_1^k$  is bounded using Lemma C.1. We can also assume WLOG that the parameters of the first layer (i.e.  $\mathbf{W}_1^k, \mathbf{b}_1^k$ ) have finite limits  $\mathbf{W}_1^\star$  and  $\mathbf{b}_1^\star$ . Indeed, since both  $\mathbf{W}_1^k$  and  $\mathbf{b}_1^k$  are bounded (by construction from Lemma C.1), there exists a subsequence  $\{\varphi_k\}_{k \in \mathbb{N}}$  such that  $\mathbf{W}_1^{\varphi_k}$  and  $\mathbf{b}_1^{\varphi_k}$  have finite limits and  $\mathcal{R}_{\theta^{\varphi_k}} \to f$  as  $\mathcal{R}_{\theta^k} \to f$ . Replacing the sequence  $\{\theta^k\}_{k \in \mathbb{N}}$  by  $\{\theta^{\varphi_k}\}_{k \in \mathbb{N}}$  yields the desired sequence. Finally, since  $\mathbf{W}_1^\star = \lim_{k \to \infty} \mathbf{W}_1^k$ ,  $\mathbf{W}_1^\star$  obviously has normalized rows and  $\mathrm{supp}(\mathbf{W}_1^\star) \subseteq I_1$ .
Definition C.1. Consider  $\Omega$  bounded subset of  $\mathbb{R}^d$ , a function  $f \in \overline{\mathcal{F}_{\mathbf{I}}(\Omega)}$  and a sequence  $\{\theta^k\}_{k \in \mathbb{N}}$  as given by Lemma C.2. We define  $(a_i, b_i) = (\mathbf{W}_1^\star[i,:], \mathbf{b}_1^\star[i])$  the limit parameters of the first layer corresponding to the  $i$ th neuron. We partition the set of neurons into two subsets (one of them may be empty):
1. Set of active neurons:  $J \coloneqq \{i \mid (\exists x \in \Omega, a_i x + b_i > 0) \land (\exists x \in \Omega, a_i x + b_i < 0)\}$ .
2. Set of non-active neurons:  $\bar{J} = [[N_1]]\setminus J$
For  $i,j\in J$ , we write  $i\simeq j$  if  $(\mathbf{W}_1^\star [j,:],\mathbf{b}_1^\star [j]) = \pm (\mathbf{W}_1^\star [i,:],\mathbf{b}_1^\star [i])$ . The relation  $\simeq$  is an equivalence relation.
We define  $(J_{\ell})_{\ell = 1,\dots,r}$  the equivalence classes induced by  $\simeq$  and we use  $(\alpha_{\ell},\beta_{\ell}):= (a_i,b_i)$  for some  $i\in J_{\ell}$  as the representative limit of the  $\ell$ th equivalence class. For  $i\in J_{\ell}$ , we have:  $(a_i,b_i) = \epsilon_i(\alpha_\ell ,\beta_\ell),\epsilon_i\in \{\pm 1\}$ . We define  $J_{\ell}^{+} = \{i\in J_{\ell}\mid \epsilon_{i} = 1\} \neq \emptyset$  and  $J_{\ell}^{-} = J_{\ell}\setminus J_{\ell}^{+}$ .
For each equivalence class  $J_{\ell}$ , define  $H_{\ell} \coloneqq \{x \in \Omega \mid \alpha_{\ell} x + \beta_{\ell} = 0\}$  the boundary generated by neurons in  $J_{\ell}$  and the positive (resp. negative) half-space partitioned by  $H_{\ell}$ ,  $H_{\ell}^{+} \coloneqq \{x \in \Omega \mid \alpha_{\ell} x + \beta_{\ell} > 0\}$  (resp.  $H_{\ell}^{-} \coloneqq \{x \in \Omega \mid \alpha_{\ell} x + \beta_{\ell} < 0\}$ ). For any  $\epsilon > 0$  we also define the open half-spaces  $H_{\ell}^{(\epsilon, +)} \coloneqq \{x \in \mathbb{R}^d \mid \alpha_{\ell}^\top x + \beta_{\ell} > \epsilon\}$  and  $H_{\ell}^{(\epsilon, -)} \coloneqq \{x \in \mathbb{R}^d \mid \alpha_{\ell}^\top x + \beta_{\ell} < -\epsilon\}$ .
Definition C.1 groups neurons sharing the same "linear boundary" (or "singular hyperplane" as in [26]). This concept is related to "twin neurons" [28], which also groups neurons with the same active zone. This partition somehow allows us to treat classes independently. Observe also that
$$
\operatorname {s u p p} \left(\alpha_ {\ell}\right) \subseteq \bigcap_ {i \in J _ {\ell}} I _ {1} [ i,: ], \forall 1 \leq \ell \leq r. \tag {21}
$$
762   
763   
764   
765   
766   
767   
768   
769   
770   
771   
772   
773   
774   
775   
776   
777   
778   
779   
780   
781   
782   
783   
784   
785   
786   
787   
788   
789   
790   
791   
792   
793   
794   
795   
796   
797   
798   
799   
800
Definition C.2 (Contribution of an equivalence class). In the setting of Definition C.1, we define the contribution of the  $i$ th neuron  $1 \leq i \leq N_{1}$  (resp. of the  $\ell$ th ( $1 \leq \ell \leq r$ ) equivalence class) of  $\theta^{k}$  as:
$$
h _ {i} ^ {k}: \mathbb {R} ^ {N _ {0}} \mapsto \mathbb {R} ^ {N _ {2}}: x \mapsto \mathbf {W} _ {2} ^ {k} [:, i ] \sigma \left(\mathbf {W} _ {1} ^ {k} [ i,: ] x + \mathbf {b} _ {1} ^ {k} [ i ]\right),
$$
$$
g _ {\ell} ^ {k}: \mathbb {R} ^ {N _ {0}} \mapsto \mathbb {R} ^ {N _ {2}}: x \mapsto \sum_ {i \in J _ {\ell}} h _ {i} ^ {k} (x).
$$
Lemma C.3. Consider  $\Omega = [-B, B]^d$ ,  $f \in \overline{\mathcal{F}_{\mathbf{I}}(\Omega)}$  and a sequence  $\{\theta^k\}_{k \in \mathbb{N}}$  as given by Lemma C.2, and  $\alpha_{\ell}, \beta_{\ell}, 1 \leq \ell \leq r, \epsilon_i, i \in J$  as given by Definition C.1. There exist some  $\gamma_{\ell}, \mathbf{b} \in \mathbb{R}^{N_2}$ ,  $\mathbf{A} \in \mathbb{R}^{N_2 \times N_0}$  such that:
$$
f (x) = \sum_ {i = 1} ^ {r} \gamma_ {\ell} \sigma \left(\alpha_ {\ell} x + \beta_ {\ell}\right) + \mathbf {A} x + \mathbf {b} \quad \forall x \in \Omega \tag {22}
$$
$$
\lim  _ {k \rightarrow \infty} \sum_ {i \in J _ {\ell}} \epsilon_ {i} \mathbf {W} _ {2} ^ {k} [:, i ] \mathbf {W} _ {1} ^ {k} [ i,: ] = \gamma_ {\ell} \alpha_ {\ell}, \quad \forall 1 \leq \ell \leq r \tag {23}
$$
$$
\lim  _ {k \rightarrow \infty} \sum_ {i \in J _ {\ell}} \epsilon_ {i} \mathbf {b} _ {1} ^ {k} [ i ] \mathbf {W} _ {2} ^ {k} [:, i ] = \gamma_ {\ell} \beta_ {\ell}, \quad \forall 1 \leq \ell \leq r \tag {24}
$$
$$
\operatorname {s u p p} \left(\gamma_ {\ell}\right) \subseteq \bigcup_ {i \in J _ {\ell}} I _ {2} [:, i ], \quad \forall 1 \leq \ell \leq r \tag {25}
$$
Proof. The proof is divided into three parts: We first show that there exist  $\gamma_{\ell}, \mathbf{b} \in \mathbb{R}^{N_2}$  and  $\mathbf{A} \in \mathbb{R}^{N_2 \times N_0}$  such that Equation (22) holds. The last two parts will be devoted to prove that equations (23) - (25) hold.
1. Proof of Equation (22): Our proof is based on a result of [26], which deals with the case of a scalar output (i.e.,  $N_{2} = 1$ ). It is proved in [26, Theorem 3.8, Steps 3, 6, 7] and states the following:
Lemma C.4 (Analytical form of a limit function with scalar output [26]). In case  $N_{2} = 1$  (i.e., output dimension equal to one), consider  $\Omega = [-B, B]^{d}$ , a scalar-valued function  $f: \Omega \mapsto \mathbb{R}$ ,  $f \in \overline{\mathcal{F}_{\mathbf{I}}(\Omega)}$  and a sequence as given by Lemma C.2, there exist  $\mu \in \mathbb{R}^{N_0}$ ,  $\gamma_\ell, \nu \in \mathbb{R}$  such that:
$$
f (x) = \sum_ {\ell = 1} ^ {r} \gamma_ {\ell} \sigma \left(\alpha_ {\ell} x + \beta_ {\ell}\right) + \mu^ {\top} x + \nu , \quad \forall x \in \Omega \tag {26}
$$
Back to our proof, one can write  $f = (f_{1},\ldots ,f_{N_{2}})$  where  $f_{j}:\Omega \subseteq \mathbb{R}^{N_{0}}\mapsto \mathbb{R}$  is the function  $f$  restricted to the  $j$ th coordinate. Clearly,  $f_{j}$  is also a uniform limit on  $\Omega$  of  $\{\mathcal{R}_{\tilde{\theta}^k}\}_{k\in \mathbb{N}}$  for a sequence  $\{\tilde{\theta}^k\}_{k\in \mathbb{N}}$  which shares the same  $\mathbf{W}_1^k$  with  $\{\theta^k\}_{k\in \mathbb{N}}$  but  $\tilde{\mathbf{W}}_2^k$  is the  $j$ th row of  $\mathbf{W}_2^k$ . Therefore,  $\{\tilde{\theta}^k\}_{k\in \mathbb{N}}$  also satisfies the assumptions of Lemma C.4, which gives us:
$$
f _ {j} (x) = \sum_ {\ell = 1} ^ {r} \gamma_ {\ell , j} \sigma \left(\alpha_ {\ell} x + \beta_ {\ell}\right) + \mu_ {j} ^ {\top} x + \nu_ {j}, \quad \forall x \in \Omega
$$
for some  $\mu_j \in \mathbb{R}^{N_0}, \gamma_{i,j}, \nu_j \in \mathbb{R}$ . Note that  $\alpha_\ell, \beta_\ell$  and  $r$  are not dependent on the index  $j$  since they are defined directly from the considered sequence. Therefore, the function  $f$  (which is the concatenation of  $f_j$  coordinate by coordinate) is:
$$
f (x) = \sum_ {\ell = 1} ^ {r} \gamma_ {\ell} \sigma \left(\alpha_ {\ell} x + \beta_ {\ell}\right) + \mathbf {A} x + \mathbf {b}, \quad \forall x \in \Omega
$$
$\gamma_{\ell} = \left( \begin{array}{c}\gamma_{i,1}\\ \vdots \\ \gamma_{i,N_2} \end{array} \right),\mathbf{A} = \left( \begin{array}{c}\mu_1^\top \\ \vdots \\ \mu_{N_2}^\dagger \end{array} \right),\mathbf{b} = \left( \begin{array}{c}\nu_1\\ \vdots \\ \nu_{N_2} \end{array} \right).$
2. Proof for Equations (23)-(24): With the construction of  $\gamma$ , we will prove Equation (23) and Equation (24). We consider an arbitrary  $1 \leq \ell \leq r$ . Denoting  $\Omega^{\circ}$  the interior of  $\Omega$  and  $H_{\ell} := \{x \in \Omega \mid \alpha_{\ell} x + \beta_{\ell} = 0\}$  the hyperplane defined by the input weights and bias of the  $\ell$ -th class of
801   
802   
803   
804   
805   
806   
807   
808
neurons, we take a point  $x' \in (\Omega^\circ \cap H_\ell) \setminus \bigcup_{p \neq \ell} H_p$  and a fixed scalar  $r > 0$  such that the open ball  $\mathcal{B}(x', r) \subseteq \Omega^\circ \setminus \bigcup_{p \neq \ell} H_p$ . Notice that  $x'$  is well-defined due to the definition of  $J$  (Definition C.1). In addition,  $r$  also exists because  $\Omega^\circ \setminus \bigcup_{p \neq \ell} H_p$  is an open set. Thus, there exists two constants  $0 < \delta < B$  and  $\epsilon > 0$  such that: