Local Identifiability of Deep ReLU Neural Networks: the Theory
Authors: Joachim Bona-Pellissier
Francois Malgouyres
Francois Bachoc
Venue: NeurIPS 2022 Accept (2022)
Keywords: Deep Learning
ReLU networks
Conditions of identifiability
Lifting operator

Abstract: Is a sample rich enough to determine, at least locally, the parameters of a neural network? To answer this question, we introduce a new local parameterization of a given deep ReLU neural network by fixing the values of some of its weights. This allows us to define local lifting operators whose inverses are charts of a smooth manifold of a high dimensional space. The function implemented by the deep ReLU neural network composes the local lifting with a linear operator which depends on the sample. We derive from this convenient representation a geometrical necessary and sufficient condition of local identifiability. Looking at tangent spaces, the geometrical condition provides: 1/ a sharp and testable necessary condition of identifiability and 2/ a sharp and testable sufficient condition of local identifiability. The validity of the conditions can be tested numerically using backpropagation and matrix rank computations.

natural Local Identifiability of Deep ReLU Neural Networks: the Theory
Joachim Bona-Pellissier François Malgouyres", François Bachoc"
Institut de Mathématiques de Toulouse UMR 5219
Université de Toulouse CNRS UTI, F-31042 Toulouse, France UPS, F-31062 Toulouse Cedex 9, France
"Corresponding author: Joachim Bona- Bell-selliss;;r;; joachim. bou.ssppllllessrrrrrri-ttoooooo
Abstract
Is a sample rich enough to determine, least locally, the parameters of a neural network? To answer this question, we introduce a new local parameterization of given deep ReLU neural network by fixing the values of some of its weights. This allows us to define local lifting operators whose inverses are charts of smooth manifold high dimensional space. The function implemented by the deep ReLU neural network composes the local lifting with a linear operator which depends on the sample. We derive from this convenient representation geometric necessary and sufficient condition of local identifiability. Looking tangent spaces, the geometric condition provides: identifiability and 2/ sharp and testable sufficient condition of local identifiability. The validity of the conditions can tested numerically using backpropagation and matrix rank computations.
sharp and testable necessary condition of
Introduction
1.1 Context and motivations
Neural networks are famous for their capacity to perform complex tasks in a wide variety of domains
such as image classification 200 object recognition [33 34 speech recognition [17 36 [6]
language processing (27 26 19 anomaly detection 32 or climate sciences
The following properties of the parameters of neural networks have recently drawn attention: iden- tifiability, inverse stability and stable recovery; from weaker to stronger. Let (X) be the outputs of network parameterized by the parameters 0, for given inputs X. Global identifiability means that if fx(X) (x) then 0 0, up identified invariances, for instance neuron permutation and rescaling for ReLU networks. Local identifiability restricts this analysis for and Ô sufficiently close. Then, inverse stability means that the distance between and 0 (up to invariances) is bounded by function of the distance between fo(X) and fx)).. Finally, stable recovery consists in obtaining algorithm to approximately recover from a noisy version of fX)), ), with quantitative guarantees. In all cases, must distinguish between statements for X being finite list of inputs, in which case would like X to be small, and for infinite X (for instance determining 0 from the entire function fo).
Identifiability from finite X, which is the focus of this paper, is important for different reasons. In the first place, model extraction attacks for neural networks have been a growing topic over the last years. Indeed, some algorithms are able to recover in practice the parameters of a neural network
36th Conference on Neural Information Processing Systems (NeurIPS 2022).

---

from queries [5] This can be concern since neural network providers may wish to keep these parameters secret, for security [2 for privacy [13 g or for intellectual property 42]
A way of preventing such recovery can be by guaranteeing that identifiability does not hold, that is to check that necessary condition of identifiability not met. On the opposite side, guaranteeing that identifiability holds is interesting in the position of an attacker. If the attacker has access to X, to fa and is able to compute t such that fx( fo (x), the question then becomes: does this guarantee that 0 A or shall the attacker expand X with new queries? The attacker needs a sufficient condition of identifiability.
Another important motivation for identifiability is having a better understanding and control of neural networks. Indeed, if the learning sample has the form (X, (o(X),, with 0 the parameters of teaching network, global identifiability from X means that the global minimizer of the empirical risk is unique. In this case, the global minimizer is reached, there will typically be no variability due to the optimization parameters (choice of the algorithm, number of epochs....) and to stochasticity (for stochastic optimizers). Even if very recent works on double descent phenomena, e.g. [4), highlight benefit of overparameteriiation (thus absence of identifiability) for increasing prediction performances, a user may be interested in small enough number of parameters to retain identifiability, if the loss of performance is mild compared to overparameterization.
Note that, of course, global identifiability is more relevant than local identifiability to the above motivations. This work nevertheless focuses on local identifiability, which is necessary condition for global identifiability, and which analysis can be a first step to analyzing global identifiability. Local identifiability is also arguably insightful space of and its image {So (X), 3 varies). J. Note that most existing identifiability, inverse stability and stable recovery results (see the next section) are also local.
the geometry of the relationship between the parameter
1.2 Existing work on identifiability, inverse stability, stable recovery and attacks
Even though has regained interest recently, the question of identifiability for neural networks is not new. Indeed, in the 1990s, some positive results of identifiability for networks with smooth activation functions (tanh, logistic sigmoid or Gaussian for instance) have been established [40 2 [2 [2). These results are mainly theoretical, they concern activation functions which are not the most used nowadays (in particular, they do not apply to ReLU networks), and assume full knowledge of the function fe implemented by the network, which is impossible in practice.
Identifiability:
When it comes ReLUU, for shallow 30 38
well deep 31 5 neural networks, some positive results of identifiability have been recently established. They show that under some conditions on the architecture and parameters of the network, the function implemented by the network uniquely characterizes its parameters, up to neuron permutation and rescaling operations. Although they apply ReLU networks, these results share a limitation with those of previous paragraph: they assume the function implemented by the network to be known on the whole input space, or at least on an open subset of it.
As far as we know, there exists only one identifiability result for deep ReLU networks assuming the knowledge of fe a finite sample only. Stock and Gribonval [9] give theoretical condition for
the existence of a finite set which locally identifies the parameters of a deep neural network. It is an existence result: it does not concretely provide such a finite set, nor does it allow to test local
identifiability for any finite sample, as we propose in this work. The construction in [9 shares
similarities with previous works on deep structured matrix factorization 23 24 25 article also lies in this line of research.
The present
Inverse stability and stable recovery: Closely related to identifiability are the topics of inverse stability and stable recovery of the parameters of a network. Some negative 29] as well as positive
24 25 examine the case of structured networks with the identity as activation function. Only (5 considers a finite X. The
24 25 results of inverse stability exist. The articles 23
authors of [I] consider a general class of networks amongst which ReLU networks, but the result only holds for one-hidden-layer neural networks. Furthermore this result also requires the knowledge of so on whole domain.
Several stable recovery algorithms have also been proposed, for one-hidden-layer neural networks in first place, for smooth activation function [14], as well as ReLU in the fully-connected case
2

---

N(X.9)
((X.6)
((0)
Figure 1: The local intersection between the affine space [XX, 0) (in green) and the smooth manifold Σj (color gradient). We also represent in red the tangent space to Σj at ((0). Left: The identifiable case. The intersection is reduced to {ø(0)} Right: The non identifiable case. The intersection, represented with a dashed white line, is not reduced to (o)0)).
These references consider finite but provide a large sample complexity under which smartly constructed initialization followed by first order algorithm allows to stably recover the parameters of the network.
T 45 46] or in the convolutional case 43
For deep networks, some stable recovery algorithms also exist, for instance for Heavyside activation function [3] for only recovering the first layer with sparsity assumptions [37] in the ReLU case, but to the best of our knowledge there does not exist any algorithm recovering fully a deep ReI.U network from finite sample.
Model inversion attacks: For deep ReLU networks, when one has full access to the function implemented by the network, a practical algorithm (35] sequentially constructs sample X and approximately recovers the architecture and the parameters modulo permutation and rescaling. Simi- larly, formulating the problem as a cryptanalytic problem, [9] reconstructs a functionally equivalent network with fewer requests. As mentioned in Section 1.1. these two references are related to identifiability, but consider a different setting. In this article we consider an arbitrary given X, while they work mostly on its construction.
1.3 Contributions
1/ We establish a necessary and sufficient geometric condition of local identifiability from a finite sample X for deep fully-connected ReLU networks. The condition is that the intersection between an affine space and smooth manifold is reduced to a single point. See Figure] for an illustration.
2/ Considering tangent spaces, we then provide a computable necessary condition of local identi- fiability from finite sample X. Since global identifiability implies local identifiability, it is also computable necessary condition of global identifiability.
3/ We also establish computable sufficient condition of local identifiability, which is close to the necessary condition. To the best our knowledge, these are the first testable conditions of local identifiability for any finite input sample. In particular, [39] provides theoretical condition equivalent to the existence of finite sample for which local identifiability holds, but does not provide the sample explicitly, nor does it characterize local identifiability for any arbitrary sample.
4/ To prove these results, we develop geometric tools which can be of independent interest for theoretically understanding deep ReLU networks as well as for possible applications. Namely, we introduce local reparameterizations Po of the network by fixing some weight values constants. Building on these local parameterizations, we introduce local lifting operators S and we decompose the function implemented by the network fo (z) as composition of vú, which only depends on the parameters, and piecewise constant operator a which depends on 0 and the inputs x' For almost any parameterization 0, the operator linear function to & We show that fact, the operators d are the inverses of coordinate charts of smooth manifold Σ;, contained in a high dimensional space. We find Σf to be of particular interest
constant in a neighborhood of 0 and consists in applying

---

in representing geometrically some properties of the network parameters (in particular to establish 1/, 2/ and above).
1.4 Overview of the article
This work is structured as follows. We start by introducing basic tools and already known results, and we state the definition of local identifiability in Section We then introduce the local parameteriza- tions Pe and the set Σ, and we show that the latter is smooth manifold in Section [ This allows us state our main results in Section that is the geometric and the numerically testable conditions of local identifiability. Finally we discuss in Section S the numerical computations needed to test the latter conditions. All the proofs are provided in the appendices.
ReLU networks, lifting operator and rescaling of the parameters
2.1 ReLU networks
Let us introduce our notations for deep fully-connected ReI.U networks. this paper, a network is graph (E, V) of the following form.
a set of neurons, which is divided L + layers, with L 2 2: V = (Vo)-
Vo is the input layer, Vz the output layer and the layers Vi with < S L - 1 are the hidden layers. Using the notation |C| for the cardinal of finite C, we denote, for all E [0, LJ.
Ni = Vi] the size of the layer Vi.
the set of oriented edges v + v between neurons in consecutive layers, that is
E = {v v Vi, E Vi41, for E [0, 1[}.
network is parameterized by weights and biases, gathered its parameterization 0, with ((w )-bbe (b.)vEB)
E R R
where B = Vi. is also convenient to consider the weights and biases in matrix/vector form: for given 0, we denote, for / [],
Wi = (wo-s))ee, E N and
by = (bo)eEN E R.
When dealing with two parameterizations and 0 E R RB take convention that We-se' and b, well W₁ and by denote the weights and biases associated to 6, and W,- and b, well as Wi and by denote those associated Í.
The activation function, denoted a, is always ReLU: for any E N and any vector (x), IRP, it is defined (xx) = (max(x1) 0), ...., max(Ip, 0))²
For given 0, we define recursively fi IR -> R (we omit the dependency in 0 in the notation for simplicity). for l [0. I.I. by
Vr E Vo,
So(Ø) r;
E [1,
1], Vo E Vo,
1((x) = - (Wif--(()) bb);
Vz V
fL(x) = WLf(-1(x) + by
We define the function fo RV + RV.. implemented by the network of parameter 0 as fo fL.
2.2 The lifting operator o and the activation operator
For fixed x E R., the value of fa( non-linear function of 6. The goal of this section is to obtain a sigher-diiensiona. representation of 0, that will be written o(0), and such that fo(x) is locally linear function of (0)) This will be achieved with Proposition I The function & called lifting operator, wording borrowed from category theory and commonly used in compressed sensing and dictionary learning, for instance in [ The components of ((0) will associated to paths the neural network. Linearity in Proposition I will correspond to summing over these paths.

---

We now introduce the paths notations. For all / E [0, I. I], we define
P V/2 x x VL-1,
which is the of all paths in the network starting from layer I and ending in layer L- 1. We consider an additional element / which can be interpreted as an empty path and whose role will be stated. We define
clear once & has been defined and Proposition [
P =
(UP)) 1=0
Pt
u (3).
In similar way to (3). we can now define the above-mentioned 'lifting operator'
R x _
R
(1)
(on.((0))pPP.eEz
by:
for all / [0, L 1] and all p ,Ut...., (u,...,, E Pt, and for all uy Vz.
I== Way
L-1
if
pp.0 (0)
by II Wey 40044
if 2 1;
for and Uz Vz. Φg.uz (0) bvz.
To define the activation operator, first define, for all E [1, T. all v Vi all 0 E R R
and x E R%,
[1 (Wifiii)))) + bi). 2 0
a2 0)
otherwise,
which is the activation indicator of neuron v. We then define the 'activation operator'
a: R%
RR F
R
RIXP (ap(I, 0))PEP
(2)
(I,0) 0)
by:
for all E [o, L - 1] and all p UL-1) E Pi:
Fuo -1 Onp (x, 0)
I 0
ap(x, 0)
Gor (I, if 2 1;
for p B, a3(xx,0) 1.
We then have the announced linear representation of the function fo implemented by the network. Proposition 1. For all A E R x B and all x E R,, fo(z) = a(z, ())())
This result, which is proven in Appendix B is for instance also stated in [39 Sec. 4] with slightly different notations. Note that each component of the vector fo(x) above is written (very large) number of paths.
over
Let us reformulate Proposition with several inputs. We consider, for some E N some given inputs of E RR Vo, with E [1, nl.. We denote by X E gnx Vo the matrix whose lines are the transpose (x')²' of the inputs. For all 0 R x B we denote by fa(X) E = the matrix whose lines are the transpose fo(x")² of the corresponding outputs. We also denote by axX, 0) RoxP the matrix whose lines are the line vectors a(x', 0). Using Proposition for all the x², we have the relation
fx(X) a ())(
We prove in Appendix B the next proposition, which states that 0 + al X, 0) is piecewise constant.

---

Proposition 2. For all , for all X RRaxVo the mapping
ax X
RaxP
a(X,0)
is piecewise consssttt,,, with a finite number of pieces. Furthermore, the boundary of each piece has Lebesgue measure zero. We call A the union of these bounduries. The set Ax C. closed and has Lebesgue measure zero.
R x R B
is
As discussed before, for a given X E RaxV, when studying the function + (X)), Proposition 2 alongside (3) shows that piece over which ox is constant, [x(X) depends linearly (0)).
Since Ax is closed with measure zero, for almost all 0 RR x B there exists neighborhood of over which ax is constant. As noted for instance by Stock and Gribonval [39 Sec. 2], for any 0 in such a neighborhood, we thus have
fx(X) fa(X) = a(X, (o(0) (x(0) (0(0))
(4)
Hence, studying e will allow us understand better how (X( locally depends on 0.
2.3 Invariant rescaling operations on
Some well-known rescaling operations the parameters 0 do affect the value of @(0). Before detailing them, let us define, for all f E R, the sign indicator sign(t) as 1,0 or -1 depending on whether & 0,
t 0 respectively, For any 0 E 12 x R. we then define
sign (0) = (sign( (wo-o) (sign().))veB) E {-1,0,11] x (-1,0,11)
We can now describe the rescaling operations. Definition 3. Let R B and R² R
We say that 0 is equivalent to 0 modulo rescaling, and we write B 0 iff there exists E (IR') x V2, with \o 1r, and X 1v,
family of vectors (2 .A......4 A²) such that, for all / [1.
Wi = Diag(A W/ bi Diag(X))b
(5)
We say that is equivalent to 0 modulo positive rescaling, and we write 0 ~~ 0 iff
R 0 and sign( sign
For all / [1, LI, satisfy (5) is equivalent satisfy, for all (w E Vi-1 Vi,
>/1 Nop-1
Wm buy Nr, Dug.
Wn-1-9
(6)
The relations R and ~ are equivalence relations on the set of parameters R modulo positive rescaling a well-known invariant for ReLU networks have indeed the following property: f 0, for all x TR. %o
x R The equivalence
38 39
We
fa(x) = fa(x).
(7)
One of the interests of the operator ó is that it captures this invariant, as described by Stock and Gribonval [39] Sec. 2.4]. Propositions 4 and 5 are similar to their results and are restated here and
proven in Appendix for completeness. Indeed, combining the definition of Φ with ( we have the
following property.

---

Proposition 4. For all 0, E RR
x BR_ we have
0 0 0
=
6(0) = =60),
and thus in particular
0~0 0
6 6(0)
The reciprocal of Proposition 4holds provided exclude some degenerate cases. Let denote, for any E [ T- and any v E Vi, by We-vo the vector (ww---) E va- and by Wa->e the E IR Vi+1, We define the following set, which is close to the notion of 'non
vector (W.+ve ) EV+)
admissible parameter' in 33
5 (0 E X 3F "
Wo-s =0 or (wo b,) = (0. 0)).
When Wy-re O, all the outward weights of v are zero. When (w b.) (0, (0,0), all the inward weights as well as the bias of v are zero, so for any input the information flowing through neuron is always zero. In both cases, the neuron does not contribute to the output and could be removed from the network without changing the function fe. Since the set S is a finite union of linear subspaces of codimension larger than 1, it is closed and has Lebesgue measure zero. We can thus exclude the degenerate cases in S without loss of generality. Proposition states that the reciprocal of Proposition
4 holds over (R F x IR S.
Proposition 5. For all E (RE x R) for all R R
( (())
θ R 0.
2.4 Local identifiability
We have now introduced all the concepts used in the formal definition of 'local identifiability. Definition 6. Let X E RaxV and 0 E Eº B.B. We say that is locally identifiable from X there exists > 0 such that for all E g x RB if 110 0 1o0 <
fo(X) fx(X) = 0~ 0.
3 The smooth manifold Σ
We explained the previous section that studying allows to better understand how the output fe (X) locally depends on O. The image of á is of particular interest in this study and is the subject of this section. We define
Σx {o(0), 0 (RF R (S).
The main result of this section, Theorem states that Σ; is smooth manifold. This result is a key element of the article. Indeed, allows consider tangent spaces to Ej, and by doing so, to linearize the geometric characterization of Theorem g illustrated in Figure [ Instead of considering the intersection between a smooth manifold and an affine space as Theorem g this indeed allows to consider the intersection between two affine spaces, which can be characterized with rank computations as in Thcorems and I0
To show this result, we need local injectivity. In this aim, let us consider fixed 0 and analyze the functions u + fo+u(X) (X) and + @(0+ + u) for u around O. We can select N₁ + +... V- scalar scaling parameters (each in neighborhood of 1), and use them to "rescale" 0 + in Definition leaving fe+a(X) (X) and 6( + u) unchanged (77) and Proposition 4p. Locally, first order, this means that there are N1 + ..... NL-1 linear combinations of u which leave fe+u(X) and o u) invariant. order to obtain injectivity with respect to u, locally around 0, we will fix N +.....N..-.
3
components of u as follows.
For each neuron in hidden layer, we choose the outward edge + o' whose weight Wo--or has largest (absolute) value (if there are several such edges, we choose one arbitrarily). We denote by (1) such a neuron v'. For each neuron e in hidden layer Vi, there is exactly one neuron sl (v) in the layer Vi+1, and one corresponding edge - si (v). See Figure 2 for illustration. We will set to 0 the components of u corresponding to all the edges of the form v - g Intuitively,

---

W0-+11
To-+122
Input
Output
U3
Hidden layers
Figure 2: Left: The outward edges of a hidden neuron v and their weights. In this example, 01 s (v), the weight of the edge in red, - v1, has its value fixed Wa- The weights of the remaining edges, Te-+vz and Tu-res are free to vary, Right: In red, all the edges whose weights fixed. The remaining edges, in black, constitute the set Fg.
it will not limit the set of functions fg. in the vicinity of fo; but will permit to obtain one-to-one correspondence between u and fetu.
More precisely, let us denote by Fg C E the set of remaining edges, which is formally defined as
Fo = E\
(v. J (m
1=1
The mapping from the space of restricted parameters R& R the parameter space R R locally around is simply given by the following application
Po Fo x B
R
R
v(u, v') E Fa. Wo-so' = Tu-tod V(u,o') E Fo,,, Wo-so' We-re' Vu E B, by Tv.
(9)
0 such that
In particular, if we define To E Fb x B by (Ta) = Wo-ser and (TO): = bv. we have po( The function Pe is affine and injective. We define
Uo = Po
(TRE x (())
(5),
(10)
which is an open set of Fo x RB We define, for all 0 E (RE x I \,S, the local lifting operator
Uo T
PXV2
(11)
Φ o po())
One can show that vi is C and that it is a homeomorphism from Ug onto its image (see the proofs in Appendix C.. which we denote Va and is thus an open subset of 2; (with the topology induced on Σ; by the standard topology PxV). In particular, since po (To) 8, we have Φ(0) v: (To) Vo. We have the following fundamental result that will allow us to consider and make use the tangent spaces of Dx
Theorem 7. Σj is a smooth manifold of PxV, of dimension |ol + B|
No N₂ + NL-INE NL,
and the family (Ve, (1)9) ) y\s an atlas.
Theorem 7 is proven in Appendix C Besides being key in Section Theorem (both the smooth manifold nature of Σ3 and the explicit atlas (Vo, (()))))))))))X"))))))))))) of more general independent interest. To our knowledge, such a result has not been established elsewhere in the literature. Notice that, as announced, despite the use of restricted parameters in R x IR we can represent the whole tangent space at any point of 27 The only consequence of the restriction is the uniqueness of the representation of the elements of tangent spaces.
may also be considered
Note, in the definition of Fg, the index starting at = and not | 0.

---

4
Main results: necessary and sufficient conditions for local identifiability
The main results of this paper rely on the decomposition (4) introduced Section 2 To reformulate D. let us introduce the linear operator A(X,6) 6), which simply corresponds to the matrix product with or (X;
0( PXV2
RTXV
a(X, 0)7.
where ((X,0) f) is the matrix product between o(X, RR and of E RPV., The operator A(X, 9) inherits the properties oxX,, 0), in particular those stated in Proposition 2 the relation (4) satisfied by in the neighborhood of 0 becomes
Using A(X, f),
(x( fo(X) A(X, 0) (010) 1
(12)
Let us also define the affine space (set-sum of fixed point and a vector space)
0) x(0) + Ker A(X,0)
If a parameterization Ō E RE x RB is such that lx(X)= = (x) and 12 holds, then o(0) x(0) E Ker A(X, 0), so by definition d(0) E NN. 0). Since for & in the neighborhood of we also have (()) E Ej, we see that local identifiability closely related to the nature of the intersection between the smooth manifold Σ and the affine subspace (XX,0) 0).
?l < E) the ball of center (()) and of radius > O. We have the following geometric necessary and sufficient condition of local identifiability, which states that local identifiability of holds if and only if the intersection between Σj and N(X, 0) is locally reduced to the single point {p(0)}
Indeed, let us denote by (ox
{n PXV, [l)(0)
Theorem 8. For any X E R Vo and E (R x R B) \(S U Ax), the two following statements equivalent.
i) 8 is locally identifiable from X.
ii) There exists e >> such that (x() (X (o(0))
Theorem iis proven in Appendix D and is illustrated in Figure [] This geometric condition is crucial for showing the next two results which give testable conditions of identifiability. Theorems and 10 rely on the rank of A(X, 0) and of another linear operator [[X, f), which we define. Since, as we said, the function 16 is C let us denote by D (T) R x R > PxV.. its differential at the point T, for any T E Ug. We define the linear operator r(X, A) R Fo x R + Rax by
0) = A(X,0) D (Ta).
(13)
()) and Rr (a 0)) Since r(X, 0) is defined on RR RBB we have 0 < Rr I |Fo + [B), and the expression (13) shows that we also have 0 < Rr S RA. We now define the two following conditions.
We denote RA (axnn(((XX))))
Condition Cy. Condition Cy is satisfied by (0, iif Rp < RA or R (Fel + [B].
Condition Cs. Condition Cs is satisfied by (0, X) iif Rr = Fe + [B).
The following result states that CN necessary for local and therefore global identifiability. Theorem 9 (Necessary condition of identifiabilitty. Let X Ra and 0 E (R5 x IR. (SUAx). If N is not satisfied, then 0 is not locally identifiable from X (thus not globally identifable).
The following result states that Cs is sufficient condition of local identifiability.
Theorem 10 (Sufficient condition of local identifiability). Let X E Ro and A E (R. R R) VSS Ax). If Cs sutisfied, then is locally identifiable from X.
Both theorems are proven in Appendix D To discuss these two results, let us point out that the output spaces of T(X, 0) and A(X, , 0) have the same dimension, equal to nNN,, Each new input adds N,, to this dimension. One can verify that RA Rp is initially 0 and cannot decrease when new inputs are

---

added. If new input leads to RA > Rr. it can be discarded to preserve RA Rr. Moreover, such an input seems unlikely when RA < [Fg + [B). the equality Rp RA is enforced, the condition Rr = (F + [B] is both necessary and sufficient. Finally, to satisfy Rr [F + | the dimensions must satisfy n.Nr 2 [Pa + [B] The general belief is that the latter is the condition of identifiability since n is the number of scalar measurements and Fal [B] is the number of independent free
parameters, see Theorem 7
Checking the conditions numerically
The key benefit of the conditions CN and Cs, compared the existing literature, is that they can be numerically tested for any fixed finite sample. They need the computation of the rank of two linear operators, namely T(X, 8) and A(X,0) 0). The operator [(X, f) satisfies the following:
Proposition 11. Let X E R= and 0 E (R& x R \(S U Ax). The function T ++ f (X). for E Up is differentiable in a neighborhood TB. and we denote by DrJpo(ro) (X) its differential To- We have
D DrJpo(ro) = T(X,0)
(14)
The proof of Proposition II is in Appendix E Since the reparameterization with po simply consists in fixing the weights of the edges v - <max (v)
to the value W (v). (59 shows that the coefficients
of [(X, f) can be computed by a classic backpropagation algorithm N,. times for each input x', simply omitting the derivatives with respect to the edges of the form v + m (v). An explicit expression of the coefficients of ['X, 0) is given in the Appendix E
To be satisfied, Cs needs the dimensions of I'(X, 0) to satisty n.N 2 F + [B]. One then needs to compute the rank Rr of (XX, 0), which means computing the rank of a n.Nz x (1Fol + [B]) matrix.
Existing algorithms allow to do this with a complexity O(nNz + ( (up to polylog terms), where w is the matrix multiplication exponent and satisfies < 2.38 [10].
When comes to Cv one needs addition to know the rank RA of A(X,0), 0), which, as Proposition 12 states, requires to compute the rank of ox( X, 0).
Proposition 12. Let X E R and E RR F. x R. We have RA = N,, rank (a(X, 0)).
The dimensions of a 0) sensibly larger, with [P] columns and lines, and typically [P] >> However it may have some sparsity properties, as its entries consist in products of activation indicators (with possibly one input o:t any of them being zero causing many entries vanish. The question of the efficient computation of RA still needs to be explored and is left as open for future work.
Conclusion
This paper is the first characterize local identifiability for deep ReL.U networks for any given finite sample, with testable conditions. The practical use of these conditions deserves follow-up research, and so does an extension of our approach inverse stability. The role of ReLU crucial in our approach, especially for the necessary condition of local identifiability and with the linear representation (Proposition [T). In the end, from Theorem 10 and Proposition [T] the sufficient condition for local indentifiability is expressed from the Jacobian matrix of the neural network function with respect its parameters. Extending this to other activation functions than ReLU is an interesting perspective.
Acknowledgments and Disclosure of Funding
The authors would like to thank Pierre Stock and Rémi Gribonval for the fruitful discussions around this work, notably regarding the construction of á and its link to the question of local identifiability.
This work has benefited from the AI Interdisciplinary Institute ANITI. ANITI is funded by the French "Investing for the Future- - PIA3" program under the Grant agreement n A
The authors gratefully acknowledge the support of the DEEL project
htttps://ww...... deel.

---

References
[1] Rilwan A Adewoyin, Peter Dueben, Peter Watson, Yulan He, and Ritabrata Dutta. Tru-net: a deep learning approach to high resolution prediction of rainfall. Machine Learning, 110(8): 2035-2062, 2021.
[2] Francesca Albertini, Eduardo D Sontag, and Vincent Maillot. Uniqueness of weights for neural networks. Artificial Neural Networks for Speech and Vision, pages 115-125, 1993.
[3] Sanjeev Arora, Aditya Bhaskara, Rong Ge, and Tengyu Ma. Provable bounds for learning some deep representations. In Eric P. Xing and Tony Jebara, editors, Proceedings of the 31st International Conference on Machine Learming, volume 32 Proceedings of Machine Learning Research, pages 584-592, Bejing, China, 22-24 Jun 2014. PMLR.
[4] Mikhail Belkin, Daniel Hsu, and Ji Xu. Two models of double descent for weak features. SIAM Journal on Mathematics of Data Science,
2020.
[5] Joachim Dona-Pellissier, François Bachoc, and François Malgouyres. Parameter identifiability of deep feedforward ReLU neural network. arXiv preprint arXiv:2112.12882 2021.
[6] Alon Brutzkus and Amir Globerson. Globally optimal gradient descent for a ConvNet with Gaussian inputs. In Proceedings of the 34th International Conference on Machine Learning- Volume 70, pages 605-614, 2017.
[7] Emmanuel J Candes, Yonina C Eldar, Thomas Strohmer, and Vladislav Voroninski. Phase retrieval via matrix completion. SIAM review, 57(2):225-255 251, 2015.
[8] Nicholas Carlini, Chang Liu, Úlfar Erlingsson, Jernej Kos, and Dawn Song. The secret sharer: Evaluating and testing unintended memorization in neural networks. In 28th {USENIX) Security Symposium (USEENIX) Security 19), pages 267-284, 2019.
[9] Nicholas Carlini, Matthew Jagielski, and Ilya Mironov. Cryptanalytic extraction of neural network models. In Annual International Cryptology Conference, pages 189-218. Springer, 2020.
[10] Ho Yee Cheung, Tsz Chiu Kwok, and Lap Chi Lau. Fast matrix rank algorithms and applications. Journal of the ACM (JACM), 60(5):1-25, 2013.
[11] Dennis Maximilian Elbrächter, Julius Berner, and Philipp Grohs, How degenerate is the parametrization of neural networks with the ReLU activation function? In Advances in Neural Information Processing Systems, volume 32, 2019.
[12] Charles Fefferman. Reconstructing a neural net from its output. Revista Matemática Iberoamer- icana, 10(3):507-555, 1994.
[13] Matt Fredrikson, Somesh Jha, and Thomas Ristenpart. Model inversion attacks that exploit confidence information and basic countermeasures. In Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security, pages 1322-1333, 2015.
[14] Haoyu Fu, Yuejie Chi, and Yingbin Liang. Guaranteed recovery of one-hidden-layer neural networks via cross entropy. IEEE Transactions Signal Processing, 68:3225-3233 2020.
[15] Rong Ge, Jason D Lee, and Tengyu Ma. Learning miehhidde neural networks with landscape design. In 6th International Conference on Learning Representations, ICLR 2018, 2018.
[16] Awni Hannun, Carl Case, Jared Casper, Bryan Catanzaro. Greg Diamos, Erich Elsen, Ryan Prenger, Sanjeev Satheesh, Shubho Sengupta, Adam Coates, et al. Deep speech: Scaling up end-to-end speech recognition. arXiv preprint 1rXiv:1412.5567, 2014.
[17] Geoffrey Hinton, Li Deng. Dong Yu, George E. Dahl, Abdel-rahman Mohamed, Navdeep Jaitly, Andrew Senior, Vincent Vanhoucke, Patrick Nguyen, Tara N. Sainath, and Brian Kingsbury. Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups. IEFE Signal Processing Magazine, 296))82297, 2012.

---

[18] Paul C Kainen, Véra Kúrková, Vladik Kreinovich, and Ongard Sirisaengtaksin. Uniqueness of network parametrization and faster learning. Neural, Parallel & Scientific Computations, 2(4): 459-466, 1994.
[19] Nal Kalchbrenner and Phil Blunsom. Recurrent continuous translation models. In Proceedings of the 2013 conference empirical methods in natural language processing, pages 1700-1709, 2013.
[20] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems, 25:1097- 1105, 2012.
[21] Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Adversarial examples in the physical world. International Conference Learning Representations, 2017.
[22] Véra Kürková and Paul Kainen. Functionally equivalent feedforward neural networks. Neural Computation, 6(3):543-555 558, 1994.
[23] François Malgouyres and Joseph Landsberg. On the identifiability and stable recovery of deep/multi-layee structured matrix factorization, In IEEE, Info. Theory Workshop, Sept. 2016.
[24] François Malgouyres and Joseph Landsberg. Multilinear compressive sensing and an application convolutional linear networks. SIAM Journal on Mathematics of Data Science, 1(3):466 4744,46645, 2019.
[25] Francois Malgouyres. On the stable recovery of deep structured linear networks under sparsity constraints. In Mathematical and Scientific Machine Learning, pages 107-127. PMLR, 2020.
[26] Tomas Mikolov, Martin Karafiát, Lukas Burget, Jan Cemockýj, and Sanjeev Khudanpur. Re- current neural network based language model. In Interspeech, volume 2, pages 1045-1048. 2010.
[27] Tomás Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. Efficient estimation of word representations in vector space. In Yoshua Bengio and Yann LeCun, editors, 1st International Conference on Learning Representations, ICLR 2013, Scottsdale, Arizona, USA, May 2-4, 2013, Workshop Track Proceedings, 2013.
[28] Behnam Neyshabur, Russ R Salakhutdinov, and Nati Srebro. Path-SGD: nath-normalized optimization deep neural networks. Advances neural information processing systems, 28, 2015.
[29] Philipp Petersen, Mones Raslan, and Felix Voigtlaender. Topological properties of the set of functions generated by neural networks of fixed size. Foundations of Computational Mathemat- ics, 21:375 444, 2021.
[30] Henning Petzka, Martin Trimmel, and Cristian Sminchisescu. Notes on the symmetries of 2-layer ReLU-networks. In Proceedings of the Northern Lights Deep Learning Workshop, volume 1, pages 6-6, 2020.
[31] Mary Phuong and Christoph H. Lampert. Functional vs. parametric equivalence of ReLU networks. In International Conference on Learning Representations, 2020.
[32] José Pedro Pinto, André Pimenta, and Paulo Novais. Deep learning and multivariate time series for cheat detection in video games. Machine Learning, 110(1 (10(1))3037-3355 2021.
[33] Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi. You only look once: Unified, real-time object detection. In Proceedings of the IEEE conference computer vision and pattern recognition, pages 779-788, 2016.
[34] Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster R-CNN: Towards real- real-time object detection with region proposal networks. Advances in neural information processing systems, 28:91-99, 2015.
12

---

[35] David Rolnick and Konrad Kording, Reverse-engineering deep ReLU networks. In Hal Daumé III and Aarti Singh, editors, Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pages 8178-8187, 13-18 Jul 2020.
[36] Haşim Sak, Andrew Senior, and Françoise Beaufays. Long short-term memory recurrent neural network architectures for large scale acoustic modeling. In Fifteenth Annual Conference of the International Speech Communication Association, 2014.
[37] Hanie Sedghi and Anima Anandkumar. Provable methods for training neural networks with sparse connectivity. In Deep Learning and representation learning workshop: NIPS, 2014.
[38] Pierre Stock. Efficiency and Redundancy in Deep Learning Models: Theoretical Considerations and Practical Applications. PhD thesis, Université de Lyon, April 2021. URL https://tel
[39] Pierre Stock and Rémi Gribonval. An Embedding of ReLU Networks and an Analysis of their Identifiability. Constructive Approximation, 2022. URL httthhttp::/aall http:/aall..rraieeee-ovvvveee... he/hall0322220
[40] Héctor J Sussmann. Uniqueness of the weights for minimal feedforward nets with given input-output map. Neural networks, 5(4):589-593, 1992.
[41] Mingyang Yi, Qi Meng, Wei Chen, Zhi-ming Ma, and Tie-Yan Liu. Positively scale-invariant flatness of ReLU neural networks. arXiv preprint arXiv: 02237, 2019.
[42] Jialong Zhang, Zhongshu Gu, Jiyong Jang, Hui Wu, Mare Ph Stoecklin, Heqing Huang, and Ian Molloy. Protecting intellectual property of deep neural networks with watermarking. In Proceedings of the 2018 on Asia Conference on Computer and Communications Security, pages
159-172, 2018.
[43] Shuai Zhang, Meng Wang, Jinjun Xiong, Sijia Liu, and Pin- Yu Chen. Improved linear con- vergence of training CNNs with generalizability guarantees: A one-hidden-laye case. IEEE Transactions on Neural Networks and Learning Systems, 32(6):2622 2635, 2020.
[44] Xiao Zhang, Yaodong Yu, Lingxiao Wang, and Quanquan Gu. Learning one- me-hidden-layer ReLU networks via gradient descent. In The 22nd International Conference on Artificial Intelligence and Statistics, pages 1524-1534. PMLR, 2019.
[45] Kai Zhong, Zhao Song, Prateek Jain, Peter L Bartlett, and Inderjit S Dhillon. Recovery guarantees for me-hidden--aaee neural networks. In Proceedings of the 34th International Conference Machine Learning- Volume 70, pages 4140 4149, 2017.
[46] Mo Zhou, Rong Ge, and Chi Jin. A local convergence theory for mildly over-parameterized two-layer neural network. arXiv preprint arXiv:2102.0044 2021.
Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? [Yes] The outline in Section 1.4 ("Overview of the article") provides pointers to where the claimed contributions of the paper are provided.
(b) Did you describe the limitations of your work? [Yes] Section acknowledges the open problem of efficient computation of the rank of oxX, 0) and Section 6 describes other remaining open questions.
(c) Did you discuss any potential negative societal impacts of your work? [N/A] This is theoretical/ffoundatio work that adds the theory and methodology of deep learning. As for any such contributions, the positive or negative societal impact will depend on the application case. We do not promote any harmful use of this theory, but we expand on the existing knowledge.
13

---

(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes] See previous question.
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [Yes] All our results explicitly refer to their required assumptions. Some general assumptions that hold throughout the paper are also stated at the beginning.
(b) Did you include complete proofs of all theoretical results? [Yes] All the proofs are provided in the supplement.
3. If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as a URL)? [N/A] We did not run experiments.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [N/A]
(c) Did you report error bars (e.g.. with respect the random seed after running experi- ments multiple times)? [N/A]
(d) Did you include the total amount of compute and the type of resources used (e.g.. type of GPUs, internal cluster, or cloud provider)? [N/A]
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [N/A] We did not existing assets (code, data or models) nor cure/release new assets (code, data or models).
(b) Did you mention the license of the assets? [N/A]
(c) Did you include any new assets either in the supplemental material or as URL? [N/A]
(d) Did you discuss whether and how consent was obtained from people whose data you'l 're using/curating? [N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if applicable? [N/A] We did not crowdsoureing conducted research with human subjects.
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent participant compensation? [N/A]


Reviews:
[{"review_id": "rKC0QFrHNUn", "reviewer": "Reviewer_jUtV", "summary": "This paper targets an interesting question of neural network basis. The main focus is the uniqueness of parameters in a ReLU-activated NN. Proofs are provided to support the conditions of such uniqueness. The reviewer has the following concerns about the work.", "questions": "Please see the questions and suggestions in the above comments.", "limitations": "Please see the limitations and suggestions in the above comments.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "This work starts with a question to doubt the uniqueness of NN in terms of the parameters, i.e., weights and bias. The authors try to provide the conditions (necessary and sufficient) of the uniqueness for a ReLU-activated NN. To the best of the review’s knowledge, this work is new. For confidential/security concerns, the topic is of great importance and needs more attention in the community.\nWhile the problem is important, the review has some concerns and questions about the work.\n\n1.\tThe motivation and the relevance to existing work need to be clarified. \n\na.\tFor example, the authors claim not solving the inverse stability problem in Sec. 1.1. It is confusing. What is the connection between your work and inverse stability? The review didn’t find a discussion in the following paper. What are the insights generated from this work that can help with inverse stability?\n\nb.\tIn Sec. 1.2, although the authors provide a literature review of the parameter uniqueness of NNs, some details are missing. For example, in Row 48: “interesting results on identifiability of networks,” what are the interesting results? In Row 51: “under some conditions,” what are the conditions that narrow down the implementation. For this entire section, what is the limitation(s) in previous work so that yours is necessary?\n\n\n2.\tConcerns about the general technical contents.\n\na.\tThrough the paper reading, the review would like to know why does the paper focus on ReLU activation? Can the theorems and proofs be generalized to other activation functions?\n\nb.\tFrom this review’s point of view, it’s better to include the discussion of how the theory can be used for the theoretical justification of ReLU-based or other neural networks in massive applications. They are essential and very helpful in related research and could emphasize the broader impact of this work.\n\n\n3.\tDetails of the mathematical derivation and proofs need to be clarified. \n\na.\tThe authors mention tangent spaces in the abstract and introduction for proof. Why should tangent space be used? \n\nb.\tCould the author include some explanations of why graph modeling has to be used for NN representation? Is it easier to represent the weights on the edges, like in Figure 1? \n\nc.\tIt’s easy to get lost in heavy math formulas. For example, it is confusing what is the purpose of defining several notations in Sec.2.1-2.2 until the illustration in Row 145-147 on page 4.\n\nd.\tThen, the last sentence of Sec.2.2 makes it clear that the purpose of \\phi is to show the local dependency of f_{\\theta}(X), and this is related to the local identifiability. The reviewer suggests that the authors should define local identifiability at the beginning of Sec. 2, which should be a mathematical definition other than the one in Sec. 1.1.\n\ne.\tFor Sec. 3, could the authors explain why the smooth manifold of image is essential for proving local identifiability? It seems to be a key intermediate result but has not been mentioned much.\n\nf.\tHow do you generalize the results from fixing weights to free weights? The case now may be too narrow. It is worth more discussion as the freedom of weights combination and function composition are the magic of deep NN. \n\ng.\tThe definition of affine space needs to be clarified. Does “Ker” mean kernel in this paper?\n\nh.\tThe visualization in Figure 2 is intuitive and helpful. I suggested the authors put it around Sec. 1.3, where the intersection is first mentioned in Contribution. Otherwise, the description is confusing to follow.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "aAUfpnWHq5D", "reviewer": "Reviewer_JEdY", "summary": "This study investigates the identifiability of deep ReLU nets, that is, the existence of a tractable inverse map from finite input-response examples {(x,f_\\theta(x))} to the parameter \\theta, and presents necessary and sufficient conditions of the identifiability.", "questions": "Comments and Suggestions:\nI believe that identification is possible by appropriately constraining the solution space. On the other hand, however, the following paper has pointed out the existence of a “null space” in the parameters of neural networks in a constructive manner:\n\n[SII21] S.Sonoda, I.Ishikawa, M.Ikeda, Ghosts in Neural Networks: Existence, Structure and Role of Infinite-Dimensional Null Space, arXiv:2106.04770.\n\nThe null space presented there does not result from the permutation of neurons, but rather construct a nontrivial arrangement where multiple neurons cancel each other out. That is, there exist W_2, W_1, and b_1 such that W_2, W_1, b_1 \\neq 0, but W_2 \\sigma( W_1 x + b_1 ) = 0 for all x. As long as such null space exists, identification is generally hopeless. As such, it would be more practical if the authors did not only describe the conditions under which identification is possible, but also provided a specific discussion on how the non-identifiable cases are avoided. (And my current guess is that either we can show that “(W_2, W_1, b_1) \\sim 0” (in the sense of equivalence relation defined in Eq.5) or Proposition 5 drops such cases. In the former case, I further guess that identification is much harder when the activation function is not ReLU.)\n", "limitations": "Yes they have in Sections 5 and 6.\n", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strength:\nIn general, identification (of parameters) is a challenging problem. This can be expected by considering Fourier series, instead of neural networks, which have been investigated much longer than neural networks. That is, it is difficult to estimate the frequencies (or the parameters) of the original signal from a finite number of irregularly sampled input-output examples. Even in the case of regular sampling, classical issues such as aliasing are known to occur. Therefore, I consider the necessary and sufficient conditions to be nontrivial and interesting.\n\nWeakness:\nIt would be better if the authors have presented a few (simple) numerical examples. Please refer to the questions section below for more details.\n\nI found the notations are sometimes hard to follow. For example, I could not immediately understand that the equation in Proposition 1 is an abbreviation of \\sum_p \\alpha_p(x,\\theta) \\phi_{p,v}(\\theta).\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "UCzyWl3dwOh", "reviewer": "Reviewer_Pq5e", "summary": "This paper is a purely theoretical contribution (no code or experiments) on parameter identifiability for a Deep ReLU network. The main contribution is to give criteria for when the behavior of the function parameterized by the network on a finite set of points induces local identifiability of the parameters.\n\nWhereas (global) identifiability shows that the parameters that generate that function are unique, local identifiability only tells you that the parameters are unique within an arbitrarily small neighborhood (they are the unique parameters in that neighborhood that produce that behavior of the function).\n\nThe motivations come from security, privacy, and intellectual property.\n\nTo summarize the main steps in the argument:\n\n- The paper considers a deep fully-connected ReLU network with parameters $\\theta$ that outputs a function $f_{\\theta}(X)$ on a finite set of inputs $X$.\n- It is well-known that some operations on the parameters do not change the function, including permutation and rescaling. Positive rescaling (when you multiply all the output weights for a single neuron by a positive constant and divide the input weights by the same constant) changes the parameters without changing the function.\n- They decompose the function implemented by the network into a composition of the lifting operator (which only depends on the parameters, not on x), and a piecewise constant operator that depends on both the parameters and the inputs.\n- They exclude a measure zero set they call $S$ from the parameter space. The excluded set S is the finite set of parameters that lie at the edges of the piecewise constants.\n- They show the lifting operator is invariant to rescaling. If the parameters are equivalent, then the lifting operators match. If the lifting operators match, then the parameters are equivalent modulo rescaling.\n- After excluding the set $S$, the lifting operator maps the parameter space (minus S) to a smooth manifold $\\Sigma_1^*$ with a certain dimension.\n- Next they fix some of the weights so that they can’t be rescaled. This removes some of the degrees of freedom for the parameters. (I didn’t fully understand this step.)\n- This allows them to give a geometric condition (that the smooth manifold and an affine space intersect at a single point) that is equivalent (necessary and sufficient) to local identifiability.\n- However, in practice, if I gave you a network and a finite sample, you wouldn’t know whether this geometric criteria was satisfied. So they continue to give computable criteria based on the rank of quantities that depend on both X and $\\theta$.\n- The computable criteria aren’t as tight as the geometric criteria. The geometric criterion is necessary and sufficient for local identifiability. There are two different computable criteria - one is necessary for identifiability and the other is sufficient for local identifiability.\n- They explain how to numerically compute the sufficient criterion, including giving the big-O complexity of the runtimes. They leaving efficient computation of necessary criterion as future work.", "questions": "- Suggestion: It took me a while to figure out what the definition of local identifiability is. You should state this more clearly.\n- Suggestion: I think you should say “geometric” instead of “geometrical”.\n- Question: Why is it called a “lifting operator”? You could explain the intuition for this naming.\n- Suggestion: The section (beginning of Section 3) on fixing some parameters to prevent rescaling was confusing to me. Particularly the sentence that ends in \"and we consider its value to be fixed from now on\". What does it mean to be fixed? What does \"from now on\" mean?", "limitations": "I think the paper is clear about its limitations. I don't have concerns about negative societal impact.", "rating": 5, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The paper is reasonably well-written although I'm not sure I gained a lot of intuition from reading it. There are no experiments to evaluate. I read the main text of the paper carefully but did not read any of the supplementary material and did not check if the proofs are correct.\n\nMy main comment is that I'm unsure of the significance of this paper. I don’t fully understand the motivation for studying identifiability of neural network parameters from their function behavior, since neural networks are overparameterized by design and in practice this appears to be important for optimization. Further, the assumptions required for this theory are so far from practice, and even then only yield a result on *local* identifiability (identifiability within arbitrarily small neighborhoods of the parameter space). The motivations given in the paper around security/privacy/IP don't seem compelling because anyone wanting to protect the exact values of parameters could just use a more complex and more overparameterized architecture. However, my background is biased in that I have worked more on the problem of identifying which function a neural network parameterizes given observed data, along the lines of Nonlinear ICA.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept