Efficiently Learning One Hidden Layer ReLU Networks From Queries

1 Introduction . The problem of learning neural networks given random labeled examples continues to be a fundamental algorithmic challenge in machine learning theory . There has been a resurgence of interest in developing algorithms for learning one hidden layer neural networks , namely functions of the form F ( x ) = ∑k i=1 siσ ( 〈wi , x〉 − bi ) where wi ∈ Rd , si ∈ { ±1 } , bi ∈ R are the unknown parameters . Here we consider the ubiquitous setting where σ is the ReLU activation , the input distribution on x is Gaussian , and the goal is to output a ReLU network F ′ with E [ ( F ( x ) − F ′ ( x ) ) 2 ] ≤ . Despite much recent work on this problem [ JSA15 , ZSJ+17 , GKM18 , GLM18 , GKLW18 , ZYWG19 , BJW19 , DKKZ20 , CKM20 , LMZ20 ] , obtaining a polynomial-time algorithm remains open . All known works take additional assumptions on the unknown weights and coefficients or do not run in polynomial-time in all the important parameters . In fact , there are a number of lower bounds , both for restricted models of computation like correlational statistical queries [ GGJ+20 , DKKZ20 ] as well as under various average-case assumptions [ DV20 , DV21 ] , that suggest that a truly polynomial-time algorithm may be impossible to achieve . In this work , we strengthen the learner by allowing it query access to the unknown network . That is , the learner may select an input x of its choosing and receive the output value of the unknown network F ( x ) . Our main result is the first polynomial-time algorithm for learning one hidden layer ReLU networks with respect to Gaussian inputs when the learner is given query access : Theorem 1.1 . Let F ( x ) = ∑k i=1 siσ ( 〈wi , x〉−bi ) with ‖wi‖2 ≤ R , si ∈ { ±1 } , and bi ≤ B . Given black-box access to F , there exists a ( randomized ) algorithm that will output with probability at least ∗This work was supported in part by an NSF CAREER Award CCF-1453261 and NSF Large CCF-1565235 . †Supported by NSF awards AF-1909204 , AF-1717896 , and the NSF AI Institute for Foundations of Ma- chine Learning ( IFML ) . ‡Supported by NSF CAREER Award CCF-1553605 . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . 1 − δ , a one hidden layer network F ′ such that Ex∼N ( 0 , Id ) [ ( F ( x ) − F ′ ( x ) ) 2 ] ≤ . The algorithm has query complexity and running time that is polynomial in d , B , R , k , 1/ , and log ( 1/δ ) . In light of the aforementioned lower bounds [ GGJ+20 , DKKZ20 , DV20 , DV21 ] , Theorem 1.1 suggests there is a separation between learning from random samples and learning from queries in the context of neural networks . In addition to being a naturally motivated challenge in computational learning theory , the question of learning neural networks from queries has also received strong interest from the security and privacy communities in light of so-called model extraction attacks [ TJ+16 , MSDH19 , PMG+17 , JCB+20 , RK20 , JWZ20 ] . These attacks attempt to reverse-engineer neural networks found in publicly deployed real-world systems . Since the target network is publicly available , an attacker therefore obtains black-box query access to the network — exactly the learning model we are working in . Of particular interest is the work of Carlini et al . [ CJM20 ] ( see also [ JCB+20 ] ) , who gave several heuristics for learning deep neural networks given black-box access ( they also empirically verified their results on networks trained on MNIST ) . Another relevant work is that of [ MSDH19 ] which gave theoretical guarantees for the problem we study under quite strong separation/linear independence assumptions on the weight vectors ; in fact they are able to exactly recover the network , but in the absence of such strong assumptions this is impossible ( see Section 1.2 ) . Since these papers bear strong parallels with our techniques , in Section 1.2 we provide a more detailed description of why these approaches break down in our setting and highlight the subtleties that we address to achieve the guarantee of Theorem 1.1 . 1.1 Our Approach . Here we describe our approach at a high level . Fix an unknown one hidden-layer network F ( x ) =∑k i=1 siσ ( 〈wi , x〉 − bi ) with weight vectors wi ∈ Rd , signs si ∈ { ±1 } , and bias terms bi ∈ R. Consider a line L = { x0 + t · v } t∈R for x0 , v ∈ Rd and the restriction F |L ( t ) , F ( x0 + t · v ) given by F |L ( t ) = k∑ i=1 siσ ( 〈wi , x0〉 − bi + t〈wi , v〉 ) . Note that as a univariate function , F |L ( t ) is a piecewise-linear function . We call a point t ∈ R a critical point of F |L if the slope of the piecewise-linear function changes at t. Our starting point is that if we can identify the critical points of F |L , then we can use estimates of the gradient of F at those critical points to estimate the weights of the hidden units in F . More concretely , suppose we have identified a critical point ti of F |L for a line L. Now , back in the d-dimensional space , if we look at a sufficiently small neighborhood of the point x′ = x0 + ti · v , then the set of activated units around x′ changes by exactly one ( in fact , by this reasoning , for a random L , F |L will have one unique critical point for each neuron in F , provided no two neurons are exact multiples of each other ) . We can exploit this to retrieve the wi ’ s and bi ’ s by finite differencing : for each such x′ , compute ( F ( x′+ δ ) −F ( x′ ) ) /‖δ‖ , for several sufficiently small perturbations δ ∈ Rd , and this will recover siwi ( see Algorithm 2 and 4.9 below for the details ) . We can recover sibi in a similar fashion ( see Algorithm 1 ) . We remark that the approach we have described appears to be quite similar to that of [ CJM20 , JCB+20 , MSDH19 ] . However , the above description leaves out the following key interrelated challenges for getting provable guarantees for arbitrary networks : 1 . How do we identify the critical points of F |L for a given line L ? 2 . If we make no assumptions on how well-separated the weight vectors are , it is actually impossible to recover all of the weight vectors and biases in a bounded number of queries . Using the neurons that we do approximately recover , is it possible to piece them together to approximate F ? A rigorous solution to these issues turns out to be quite tricky and technically involved . For instance , what if some critical points are arbitrarily close to each other ( imagine the piecewise linear function being a tiny peak of exponentially small in d width on the real line ) on the line F |L . In this case , identifying them would not be possible efficiently . We develop several structural results about cancellations in ReLU networks ( culminating in Lemma 3.9 ) that show that such a situation only arises in degenerate cases where some sub-network of F contains a cluster of many similar neurons ( we formalize the notion of similarity in Definition 3.1 ) . We show in Section 3 that much smaller networks can approximate these sub-networks . On the other hand , for pairs of neurons which are sufficiently far from each other , the critical points on a random line L will be well-separated . With the structural results in hand , roughly speaking it suffices to identify one “ representative ” neuron for every cluster of similar neurons in F . To do this , we discretize L into intervals of equal length and look at differences between gradients/biases of F at the midpoints of these intervals . The key result leveraging our structural results is Theorem 4.8 which shows that the set of all such differences comprises a rich enough collection of neurons that we can assemble some linear combination of them that approximates F . To find a suitable linear combination , we run linear regression on the features computed by these neurons ( see Section 4.4 ) . 1.2 Comparison to Previous Approaches . Our general approach of looking for critical lines along random restrictions of F |L is also the approach taken in the empirical works of [ CJM20 , JCB+20 ] and in the theoretical work of [ MSDH19 ] , but we emphasize that there are a variety of subtleties that arise because we are interested in learning worst-case networks from queries . In contrast , the empirical works consider trained networks arising in practice , while [ MSDH19 ] makes a strong assumption that the weight vectors wi are linearly independent and that they are angularly separated . Note that such assumptions can not hold in the practically relevant setting where F is overparameterized . To give a sense for the issues that these existing techniques run up against when it comes to worstcase networks , imagine that the one-dimensional piecewise linear function given by the restriction F |L were simply a “ bump , ” that is , it is zero over most of the line except in a small interval [ a , a+δ ] . For instance , this would arise in the following example : Example 1.2 . Consider the one-dimensional one hidden layer network F : R→ R given by F ( x ) = σ ( x− a ) + σ ( x− a− δ ) − σ ( 2x− 2a− δ ) . This function looks like the zero function except over the interval [ a , a+ δ ] , where it looks like a small “ bump. ” The proposal in the works mentioned above is to run a binary search to find a critical point . Namely , they initialize to some large enough interval [ −τ , τ ] in L and check whether the gradient at the left or right endpoint differs from the gradient at the midpoint tmid = 0 . If the former , then they restrict the search to [ −τ , tmid ] and recurse until they end up with a sufficiently small interval , at which point they return the midpoint of that interval as an approximation to a critical point . The intuition is that n steps of binary search suffice to identify a critical point up to n bits of precision.4 It is clear , however , from the “ bump ” example that such a binary search procedure is doomed to fail : if the bump does not occur in the middle of the starting interval [ −τ , τ ] , then at the outset , we don ’ t know which half to recurse on because the gradients at the endpoints and the gradient at the midpoint are all zero ! Indeed , to locate the bump , it is clear that we need to query a number of points which is at least inverse in the width of the bump to even find an input to the network with nonzero output . We remark that this is in stark contrast to related active learning settings where the ability to query the function can sometimes yield exponential savings on the dependence on 1/ ( see e.g . [ Han09 ] ) . Indeed , if we took the width of the bump to be of size poly ( ) to make it -far in square distance from the zero function , we would still need poly ( 1/ ) , rather than log ( 1/ ) , queries to locate the bump and distinguish the funcction from the zero function . The core reason binary search fails for arbitrary networks is that there can be many disjoint linear pieces of F |L which all have the same slope . Indeed , because the gradient of F at a given point is some linear combination of the weight vectors , if F is overparameterized so that there are many linear dependencies among the weight vectors , it is not hard to design examples like Example 1.2 where there may be many repeated slopes on any given random restriction of F to a line . Apart from these technical issues that arise in the worst-case setting we consider and not in previous empirical or theoretical works on model extraction , we also emphasize that our results are the first 4We remark that the refinement of binary search given in [ CJM20 , JCB+20 ] can speed this up to O ( 1 ) steps for networks that arise in practice , but the issue that we describe poses a challenge for their technique as well . theoretical guarantees for learning general one hidden-layer ReLU networks with bias terms in any learning model , including PAC learning from samples . As will quickly become evident in the sequel , biases introduce many technical hurdles in their own right . To our knowledge , the only other theoretical guarantees for learning networks with biases are the work of [ JSA15 ] which considers certain activations with nonzero second/third-order derivatives , precluding the ReLU , and the recent work of [ ATV21 ] which considers ReLU networks whose weight vectors are well-conditioned . Remark 1.3 . As discussed above , the “ bump ” construction in Example 1.2 shows that unlike in related active learning contexts , poly ( 1/ ) dependence is necessary in our setting . In fact this example also tells us another somewhat counterintuitive fact about our setting . Naively , one might hope to upgrade Theorem 1.1 by replacing the dependence on the scaling parameters R , B with one solely on the L2 norm of the function . To see why this is impossible , consider scaling the function in Example 1.2 by a factor of δ−3/2 to have unit L2 norm . To learn F , we have to figure out the location of the bump , but this requires Ω ( 1/δ ) queries , and δ can be taken to be arbitrarily small . Limitations and Societal Impact The line of work on learning neural networks from black-box access does pose a risk , for instance , that proprietary models offered via publicly-hosted APIs may be stolen by attackers who are only able to interact with the models through queries . The attackers might then use the extracted parameters of the model to learn sensitive information about the data the model was trained on , or perhaps to construct adversarial examples . That being said , understanding why this query learning problem can be easy for theorists is the first step towards building provable safeguards to ensure that it is hard for actual attackers . For instance , it is conceivable that one can prove information-theoretic or computational hardness for such problems if appropriate levels and kinds of noise are injected into the responses to the queries . Furthermore , query complexity lower bounds can inform how many accesses an API should allow any given user .

Summary: The paper considers the problem of learning depth-2 ReLU networks w.r.t. Gaussian inputs, where the learner has query (black-box) access to the target network. The problem of learning depth-2 networks (without queries) has been extensively studied in recent years. All known efficient algorithms require assumptions on both the input distribution and the network’s weights (e.g., that the weights matrix has full rank). Moreover, the known efficient algorithms for depth-2 ReLU networks do not handle bias terms. In this paper the authors give a first efficient algorithm for learning depth-2 ReLU networks with bias w.r.t. the Gaussian distribution using black-box access. There are no assumptions on the weights, except for having a bounded norm. The motivation for the problem is two-fold: (1) the model of PAC learning with membership queries is well-studied and natural; (2) there is an interest in model-extraction attacks in the security and privacy communities. Prior works on model-extraction either gave algorithms without theoretical guarantees, or efficient algorithms under strong assumptions.   The proposed algorithms follows the following ideas. For a random line L consider the restriction of the target network F. This restriction is a piecewise linear univariate function, such that each critical point corresponds to a neuron in F. This property can be exploited in order to recover the neurons. However, there are some major issues to overcome in order to implement this idea efficiently. For example, since we do not have any assumptions on the weights, then the critical points might be very close to each other. The authors solve this issue by showing that this situation occurs only if there is a cluster of similar neurons, in which case we can replace them with a small sub-network and do not have to recover all the neurons in the cluster.