Efficiently Learning One Hidden Layer ReLU Networks From Queries

1 Introduction . The problem of learning neural networks given random labeled examples continues to be a fundamental algorithmic challenge in machine learning theory . There has been a resurgence of interest in developing algorithms for learning one hidden layer neural networks , namely functions of the form F ( x ) = ∑k i=1 siσ ( 〈wi , x〉 − bi ) where wi ∈ Rd , si ∈ { ±1 } , bi ∈ R are the unknown parameters . Here we consider the ubiquitous setting where σ is the ReLU activation , the input distribution on x is Gaussian , and the goal is to output a ReLU network F ′ with E [ ( F ( x ) − F ′ ( x ) ) 2 ] ≤ . Despite much recent work on this problem [ JSA15 , ZSJ+17 , GKM18 , GLM18 , GKLW18 , ZYWG19 , BJW19 , DKKZ20 , CKM20 , LMZ20 ] , obtaining a polynomial-time algorithm remains open . All known works take additional assumptions on the unknown weights and coefficients or do not run in polynomial-time in all the important parameters . In fact , there are a number of lower bounds , both for restricted models of computation like correlational statistical queries [ GGJ+20 , DKKZ20 ] as well as under various average-case assumptions [ DV20 , DV21 ] , that suggest that a truly polynomial-time algorithm may be impossible to achieve . In this work , we strengthen the learner by allowing it query access to the unknown network . That is , the learner may select an input x of its choosing and receive the output value of the unknown network F ( x ) . Our main result is the first polynomial-time algorithm for learning one hidden layer ReLU networks with respect to Gaussian inputs when the learner is given query access : Theorem 1.1 . Let F ( x ) = ∑k i=1 siσ ( 〈wi , x〉−bi ) with ‖wi‖2 ≤ R , si ∈ { ±1 } , and bi ≤ B . Given black-box access to F , there exists a ( randomized ) algorithm that will output with probability at least ∗This work was supported in part by an NSF CAREER Award CCF-1453261 and NSF Large CCF-1565235 . †Supported by NSF awards AF-1909204 , AF-1717896 , and the NSF AI Institute for Foundations of Ma- chine Learning ( IFML ) . ‡Supported by NSF CAREER Award CCF-1553605 . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . 1 − δ , a one hidden layer network F ′ such that Ex∼N ( 0 , Id ) [ ( F ( x ) − F ′ ( x ) ) 2 ] ≤ . The algorithm has query complexity and running time that is polynomial in d , B , R , k , 1/ , and log ( 1/δ ) . In light of the aforementioned lower bounds [ GGJ+20 , DKKZ20 , DV20 , DV21 ] , Theorem 1.1 suggests there is a separation between learning from random samples and learning from queries in the context of neural networks . In addition to being a naturally motivated challenge in computational learning theory , the question of learning neural networks from queries has also received strong interest from the security and privacy communities in light of so-called model extraction attacks [ TJ+16 , MSDH19 , PMG+17 , JCB+20 , RK20 , JWZ20 ] . These attacks attempt to reverse-engineer neural networks found in publicly deployed real-world systems . Since the target network is publicly available , an attacker therefore obtains black-box query access to the network — exactly the learning model we are working in . Of particular interest is the work of Carlini et al . [ CJM20 ] ( see also [ JCB+20 ] ) , who gave several heuristics for learning deep neural networks given black-box access ( they also empirically verified their results on networks trained on MNIST ) . Another relevant work is that of [ MSDH19 ] which gave theoretical guarantees for the problem we study under quite strong separation/linear independence assumptions on the weight vectors ; in fact they are able to exactly recover the network , but in the absence of such strong assumptions this is impossible ( see Section 1.2 ) . Since these papers bear strong parallels with our techniques , in Section 1.2 we provide a more detailed description of why these approaches break down in our setting and highlight the subtleties that we address to achieve the guarantee of Theorem 1.1 . 1.1 Our Approach . Here we describe our approach at a high level . Fix an unknown one hidden-layer network F ( x ) =∑k i=1 siσ ( 〈wi , x〉 − bi ) with weight vectors wi ∈ Rd , signs si ∈ { ±1 } , and bias terms bi ∈ R. Consider a line L = { x0 + t · v } t∈R for x0 , v ∈ Rd and the restriction F |L ( t ) , F ( x0 + t · v ) given by F |L ( t ) = k∑ i=1 siσ ( 〈wi , x0〉 − bi + t〈wi , v〉 ) . Note that as a univariate function , F |L ( t ) is a piecewise-linear function . We call a point t ∈ R a critical point of F |L if the slope of the piecewise-linear function changes at t. Our starting point is that if we can identify the critical points of F |L , then we can use estimates of the gradient of F at those critical points to estimate the weights of the hidden units in F . More concretely , suppose we have identified a critical point ti of F |L for a line L. Now , back in the d-dimensional space , if we look at a sufficiently small neighborhood of the point x′ = x0 + ti · v , then the set of activated units around x′ changes by exactly one ( in fact , by this reasoning , for a random L , F |L will have one unique critical point for each neuron in F , provided no two neurons are exact multiples of each other ) . We can exploit this to retrieve the wi ’ s and bi ’ s by finite differencing : for each such x′ , compute ( F ( x′+ δ ) −F ( x′ ) ) /‖δ‖ , for several sufficiently small perturbations δ ∈ Rd , and this will recover siwi ( see Algorithm 2 and 4.9 below for the details ) . We can recover sibi in a similar fashion ( see Algorithm 1 ) . We remark that the approach we have described appears to be quite similar to that of [ CJM20 , JCB+20 , MSDH19 ] . However , the above description leaves out the following key interrelated challenges for getting provable guarantees for arbitrary networks : 1 . How do we identify the critical points of F |L for a given line L ? 2 . If we make no assumptions on how well-separated the weight vectors are , it is actually impossible to recover all of the weight vectors and biases in a bounded number of queries . Using the neurons that we do approximately recover , is it possible to piece them together to approximate F ? A rigorous solution to these issues turns out to be quite tricky and technically involved . For instance , what if some critical points are arbitrarily close to each other ( imagine the piecewise linear function being a tiny peak of exponentially small in d width on the real line ) on the line F |L . In this case , identifying them would not be possible efficiently . We develop several structural results about cancellations in ReLU networks ( culminating in Lemma 3.9 ) that show that such a situation only arises in degenerate cases where some sub-network of F contains a cluster of many similar neurons ( we formalize the notion of similarity in Definition 3.1 ) . We show in Section 3 that much smaller networks can approximate these sub-networks . On the other hand , for pairs of neurons which are sufficiently far from each other , the critical points on a random line L will be well-separated . With the structural results in hand , roughly speaking it suffices to identify one “ representative ” neuron for every cluster of similar neurons in F . To do this , we discretize L into intervals of equal length and look at differences between gradients/biases of F at the midpoints of these intervals . The key result leveraging our structural results is Theorem 4.8 which shows that the set of all such differences comprises a rich enough collection of neurons that we can assemble some linear combination of them that approximates F . To find a suitable linear combination , we run linear regression on the features computed by these neurons ( see Section 4.4 ) . 1.2 Comparison to Previous Approaches . Our general approach of looking for critical lines along random restrictions of F |L is also the approach taken in the empirical works of [ CJM20 , JCB+20 ] and in the theoretical work of [ MSDH19 ] , but we emphasize that there are a variety of subtleties that arise because we are interested in learning worst-case networks from queries . In contrast , the empirical works consider trained networks arising in practice , while [ MSDH19 ] makes a strong assumption that the weight vectors wi are linearly independent and that they are angularly separated . Note that such assumptions can not hold in the practically relevant setting where F is overparameterized . To give a sense for the issues that these existing techniques run up against when it comes to worstcase networks , imagine that the one-dimensional piecewise linear function given by the restriction F |L were simply a “ bump , ” that is , it is zero over most of the line except in a small interval [ a , a+δ ] . For instance , this would arise in the following example : Example 1.2 . Consider the one-dimensional one hidden layer network F : R→ R given by F ( x ) = σ ( x− a ) + σ ( x− a− δ ) − σ ( 2x− 2a− δ ) . This function looks like the zero function except over the interval [ a , a+ δ ] , where it looks like a small “ bump. ” The proposal in the works mentioned above is to run a binary search to find a critical point . Namely , they initialize to some large enough interval [ −τ , τ ] in L and check whether the gradient at the left or right endpoint differs from the gradient at the midpoint tmid = 0 . If the former , then they restrict the search to [ −τ , tmid ] and recurse until they end up with a sufficiently small interval , at which point they return the midpoint of that interval as an approximation to a critical point . The intuition is that n steps of binary search suffice to identify a critical point up to n bits of precision.4 It is clear , however , from the “ bump ” example that such a binary search procedure is doomed to fail : if the bump does not occur in the middle of the starting interval [ −τ , τ ] , then at the outset , we don ’ t know which half to recurse on because the gradients at the endpoints and the gradient at the midpoint are all zero ! Indeed , to locate the bump , it is clear that we need to query a number of points which is at least inverse in the width of the bump to even find an input to the network with nonzero output . We remark that this is in stark contrast to related active learning settings where the ability to query the function can sometimes yield exponential savings on the dependence on 1/ ( see e.g . [ Han09 ] ) . Indeed , if we took the width of the bump to be of size poly ( ) to make it -far in square distance from the zero function , we would still need poly ( 1/ ) , rather than log ( 1/ ) , queries to locate the bump and distinguish the funcction from the zero function . The core reason binary search fails for arbitrary networks is that there can be many disjoint linear pieces of F |L which all have the same slope . Indeed , because the gradient of F at a given point is some linear combination of the weight vectors , if F is overparameterized so that there are many linear dependencies among the weight vectors , it is not hard to design examples like Example 1.2 where there may be many repeated slopes on any given random restriction of F to a line . Apart from these technical issues that arise in the worst-case setting we consider and not in previous empirical or theoretical works on model extraction , we also emphasize that our results are the first 4We remark that the refinement of binary search given in [ CJM20 , JCB+20 ] can speed this up to O ( 1 ) steps for networks that arise in practice , but the issue that we describe poses a challenge for their technique as well . theoretical guarantees for learning general one hidden-layer ReLU networks with bias terms in any learning model , including PAC learning from samples . As will quickly become evident in the sequel , biases introduce many technical hurdles in their own right . To our knowledge , the only other theoretical guarantees for learning networks with biases are the work of [ JSA15 ] which considers certain activations with nonzero second/third-order derivatives , precluding the ReLU , and the recent work of [ ATV21 ] which considers ReLU networks whose weight vectors are well-conditioned . Remark 1.3 . As discussed above , the “ bump ” construction in Example 1.2 shows that unlike in related active learning contexts , poly ( 1/ ) dependence is necessary in our setting . In fact this example also tells us another somewhat counterintuitive fact about our setting . Naively , one might hope to upgrade Theorem 1.1 by replacing the dependence on the scaling parameters R , B with one solely on the L2 norm of the function . To see why this is impossible , consider scaling the function in Example 1.2 by a factor of δ−3/2 to have unit L2 norm . To learn F , we have to figure out the location of the bump , but this requires Ω ( 1/δ ) queries , and δ can be taken to be arbitrarily small . Limitations and Societal Impact The line of work on learning neural networks from black-box access does pose a risk , for instance , that proprietary models offered via publicly-hosted APIs may be stolen by attackers who are only able to interact with the models through queries . The attackers might then use the extracted parameters of the model to learn sensitive information about the data the model was trained on , or perhaps to construct adversarial examples . That being said , understanding why this query learning problem can be easy for theorists is the first step towards building provable safeguards to ensure that it is hard for actual attackers . For instance , it is conceivable that one can prove information-theoretic or computational hardness for such problems if appropriate levels and kinds of noise are injected into the responses to the queries . Furthermore , query complexity lower bounds can inform how many accesses an API should allow any given user .

Summary: This paper provides a polynomial-time algorithm for learning neural networks (NN) with one hidden layer through queries. The result holds for NN with bounded weights and biases under Gaussian distributions.  The authors define a notion of closeness between neurons that allows strong assumptions on the weights (which were used in prior work and didn't translate to the overparametrized setting) to be dropped. Indeed, if a set of neurons are "close" enough, they can be approximated by two neurons or a linear function. This allows the learning algorithm to efficiently represent the target NN, which could be overparametrized.  At a high level, the algorithm recovers neurons by sampling a Gaussian line, separating it into suitably small intervals and computing the gradient (through queries) at their midpoints to recover the neurons' weight vectors.