Patent Description:
Some neural networks are recurrent neural networks. A recurrent neural network is a neural network that receives an input sequence and generates an output sequence from the input sequence. In particular, a recurrent neural network can use some or all of the internal state of the network from a previous time step in computing an output at a current time step. An example of a recurrent neural network is a long short term (LSTM) neural network that includes one or more LSTM memory blocks. Each LSTM memory block can include one or more cells that each include an input gate, a forget gate, and an output gate that allow the cell to store previous states for the cell, e.g., for use in generating a current activation or to be provided to other components of the LSTM neural network. "SMASH: One-Shot Model Architecture Search through HyperNetworks" (by Andrew Brock et al) relates to use of an auxiliary HyperNet that generates the weights of a main model conditioned on that model's architecture. "Neural Architecture Search with Reinforcement Learning" (by Barret Zoph et al) relates to use of a recurrent network to generate model descriptions of neural networks and train the recurrent network with reinforcement learning. "Efficient Neural Architecture Search via Parameter Sharing" (by Hieu Pham et al) relates to automatic model design.

This specification describes how a system implemented as computer programs on one or more computers in one or more locations can determine, using a controller neural network, an architecture for a neural network that is configured to perform a particular neural network task.

The system can effectively and automatically, i.e., without user intervention, select a neural network architecture that will result in a high-performing neural network for a particular task. The system can effectively determine novel neural network architectures that are adapted for a particular task, allowing the resulting neural network to have an improved performance on the task.

The architecture search techniques described in this specification consume fewer computational resources and less time than existing approaches, while still determining high-performing model architectures. In particular, by limiting the search space to paths within a large model and therefore sharing parameter values between candidate architectures during a given round of search, the system effectively constrains the search space and limits the computational resources required for training while still being able to determine effective architectures that result in high-performing neural networks.

In more detail, other techniques that use a neural network to control a search through a large space of possible neural network architectures (i.e., other "automatic model design" approaches) are extremely expensive in terms of time required to determine a quality architecture and in terms of computational resources, e.g., processing power and memory, consumed by the search process. This is because the other techniques require the neural network to define an entirely new architecture at each iteration and train a neural network from scratch to evaluate each new architecture. Thus, these existing techniques (i) consume large amounts of time and computational resources at each iteration of the search process due to training the neural network and (ii) need a large amount of iterations to determine a quality architecture.

The described techniques, on the other hand, use the controller neural network to search for a path through a large neural network, i.e., search for an optimal subgraph within a large computational graph. This decreases the number of iterations required to find a quality architecture. Additionally, the described techniques employ parameter sharing across iterations of the training of the child networks discovered across iterations. This decreases the time and computational resources consumed by each iteration of the search process.

Accordingly, the described techniques are much faster and much less computationally expensive than existing automatic model design approaches. In some cases, the described techniques can both consume many fewer wall clock hours than existing automatic model design approaches and discover comparable or even better performing architectures while using 1000x fewer computational resources.

This specification describes a system implemented as computer programs on one or more computers in one or more locations that determines, using a controller neural network, an architecture for a neural network that is configured to perform a particular neural network task.

The neural network can be configured to receive any kind of digital data input and to generate any kind of score, classification, or regression output based on the input.

For example, if the inputs to the neural network are images or features that have been extracted from images, the output generated by the neural network for a given image may be scores for each of a set of object categories, with each score representing an estimated likelihood that the image contains an image of an object belonging to the category.

As another example which does not fall under the scope of the claims, if the inputs to the neural network are Internet resources (e.g., web pages), documents, or portions of documents or features extracted from Internet resources, documents, or portions of documents, the output generated by the neural network for a given Internet resource, document, or portion of a document may be a score for each of a set of topics, with each score representing an estimated likelihood that the Internet resource, document, or document portion is about the topic.

As another example which does not fall under the scope of the claims, if the inputs to the neural network are features of an impression context for a particular advertisement, the output generated by the neural network may be a score that represents an estimated likelihood that the particular advertisement will be clicked on.

As another example which does not fall under the scope of the claims, if the inputs to the neural network are features of a personalized recommendation for a user, e.g., features characterizing the context for the recommendation, e.g., features characterizing previous actions taken by the user, the output generated by the neural network may be a score for each of a set of content items, with each score representing an estimated likelihood that the user will respond favorably to being recommended the content item.

As another example which does not fall under the scope of the claims, if the input to the neural network is a sequence of text in one language, the output generated by the neural network may be a score for each of a set of pieces of text in another language, with each score representing an estimated likelihood that the piece of text in the other language is a proper translation of the input text into the other language.

As another example which does not fall under the scope of the claims, if the input to the neural network is a sequence representing a spoken utterance, the output generated by the neural network may be a score for each of a set of pieces of text, each score representing an estimated likelihood that the piece of text is the correct transcript for the utterance.

<FIG> shows an example neural architecture search system <NUM>. The neural architecture search system <NUM> is an example of a system implemented as computer programs on one or more computers in one or more locations, in which the systems, components, and techniques described below can be implemented.

The neural architecture search system <NUM> is a system that obtains training data <NUM> for training a neural network to perform a particular task and a validation set <NUM> for evaluating the performance of the neural network on the particular task and uses the training data <NUM> and the validation set <NUM> to determine an architecture for a neural network that is configured to perform the particular task. The architecture defines the number of layers in the neural network, the operations performed by each of the layers, and the connectivity between the layers in the neural network, i.e., which layers receive inputs from which other layers in the neural network.

Generally, the training data <NUM> and the validation set <NUM> both include a set of neural network inputs and, for each network input, a respective target output that should be generated by the neural network to perform the particular task. For example, a larger set of training data may have been randomly partitioned to generate the training data <NUM> and the validation set <NUM>.

The system <NUM> can receive the training data <NUM> and the validation set <NUM> in any of a variety of ways. For example, the system <NUM> can receive training data as an upload from a remote user of the system over a data communication network, e.g., using an application programming interface (API) made available by the system <NUM>, and randomly divide the uploaded data into the training data <NUM> and the validation set <NUM>. As another example, the system <NUM> can receive an input from a user specifying which data that is already maintained by the system <NUM> should be used for training the neural network, and then divide the specified data into the training data <NUM> and the validation set <NUM>.

Generally, the system <NUM> determines the architecture for the neural network by determining a subset of a plurality of components of a large neural network that should be active during the processing of inputs by the large neural network. The final architecture is then the architecture of the large neural network, with only the components in the final subset active (and, optionally, any inactive components removed).

The large neural network is a neural network that contains many different neural network components, e.g., many different neural network layers, many different activation functions that can be applied by the layers, and many different possible connections between the components that can result in the large neural network generating a network output for a network input. This results in the large neural network having a vast number of parameters (referred to in this specification as "large network parameters"). By selecting a subset of components of the large neural network that should be active during processing, the system <NUM> identifies a high-quality architecture that is computationally feasible and that can be trained to generate high-quality network outputs.

In particular, the system <NUM> maintains large neural network data <NUM> that defines the large neural network as a directed acyclic graph (DAG), i.e., the neural network data <NUM> represents a DAG that defines the architecture of the large neural network and, therefore, the search space for the architecture search process. The DAG includes nodes and edges, where each node represents a computation performed by a neural network component and each edge represents a flow of information, i.e., component inputs and outputs, from one component to another. The local computations at each node have their own parameters, which are used only when the particular computation is designated active during processing. In other words, each edge from one node to another is associated with its own parameters, e.g., a parameter matrix or a kernel, that is only active when the corresponding edge is active in the current architecture, i.e., when the output node of the edge is selected as receiving input from the input node of the edge.

In some cases, the DAG specifies the entire architecture of the large neural network. In other cases, the DAG specifies a portion of the entire architecture that defines the entire architecture. In particular, in some implementations, certain portions of the large neural network architecture are fixed and not adjusted by the search process. For example, the large neural network may always be required to have a specific type of output layer, a specific type of input layer, or both. As another example, a specific type of neural network layer may be automatically inserted at fixed positions within the final architecture, e.g., a batch normalization layer before or after some or all of the layers in the neural network, a certain type of activation function applied before or after some or all of the layers in the neural network, and so on. As yet another example, when the neural network is a convolutional neural network, the neural network may always have as the last two layers of the architecture a global pooling layer followed by a softmax output layer. The global pooling layer can average all the activations of each channel of the input received by the global pooling layer.

Additionally, in some implementations, the DAG specifies a space of possible architectures for one or more types of cells, e.g., one or more types of convolutional cells or one or more types of recurrent cells, made up of multiple components. The cells specified by the DAG can then be arranged within the large neural network in a predetermined pattern to form the complete architecture of the neural network.

For example, a predetermined number of recurrent cells having the same architecture that is generated by the system, i.e., an architecture defined as a subset of the DAG, can be stacked between an embedding layer and an output layer to generate an entire large recurrent neural network architecture.

As another example, while in some implementations the DAG directly specifies the entire architecture of a convolutional neural network (except a predefined output layer), in some other implementations, by selecting a subset of the DAG, the system can can define a resolution-preserving convolutional cell that preserves the spatial resolution of its input and a reduction cell that reduces the spatial resolution of its input. Numerous instances of these two types of cells can be stacked in a predetermined pattern before an output layer to generate the final architecture of a convolutional neural network.

In some implementations, the operations and connectivity specified by the DAG can be automatically augmented with additional operations in the final architecture. For example, at some or all of the nodes in the DAG for a recurrent node, the operations specified by the DAG (and selected by the system <NUM>) can be automatically augmented with a highway connection.

In particular, the system <NUM> determines the architecture, i.e., the final subset, by training a controller neural network <NUM> to generate an output sequence that defines the final subset.

The controller neural network <NUM> is a neural network that has parameters, referred to in this specification as "controller parameters," and that is configured to generate output sequences in accordance with the controller parameters. Each output sequence generated by the controller neural network <NUM> defines a respective subset of a plurality of components of the large neural network that should be active during the processing of inputs by the large neural network. In particular, each output sequence defines a connectivity between nodes in the DAG and the local computation that should be performed at each node.

In particular, each output sequence includes a respective output at each of multiple time steps. Each node in the DAG, i.e., each component represented by the DAG, is associated with a subset of the time steps. The outputs at time steps corresponding to a given node define the inputs to the node and the operations performed by the node (for at least the input node of the DAG, the input may be predetermined). Collectively, the outputs in a given output sequence define a subset of components that are active within the large neural network. Output sequences are discussed in more detail below with reference to FIGS.

Thus, the components specified as active by a given output sequence are (i) any components that are fixed and are not part of the search process and (ii) the active components within the DAG, i.e., the parameter matrices corresponding to the connectivity defined by the output sequence and the components that perform the operations specified by the output sequence. In implementations where the output sequence directly identifies the architecture for a particular type of cell, each instance of that type of cell within the large neural network has the same active components as the instance specified by the output sequence.

The system <NUM> trains the controller neural network <NUM> by repeatedly performing each of two training phases: a controller training phase and a large neural network training phase. For example, the system <NUM> can repeatedly alternate between the controller training phase and the large neural network training phase. During the controller training phase, the system <NUM> updates the controller network parameters while holding the large network parameters fixed and during the large neural network training phase the system <NUM> updates the large network parameters while holding the controller parameters fixed.

In more detail, during the controller training phase, the system <NUM> generates, using the controller neural network <NUM> and in accordance with current values of the controller parameters, a batch of output sequences <NUM>, each output sequence in the batch specifying a respective subset of the plurality of components of the large neural network that should be active during the processing of inputs by the large neural network.

For each output sequence in the batch, a training engine <NUM> determines a performance metric <NUM> of the large neural network on the particular neural network task (i) in accordance with current values of the large network parameters and (ii) with only the subset of components specified by the output sequence active. The architecture of the large neural network with only the subset of components that are specified by a given output sequence active will be referred to in this specification as the architecture defined by the given output sequence. The large network parameters are not updated during the controller training phase. That is, for each output sequence in the batch, the training engine <NUM> evaluates the performance of the architecture defined by the output sequence on the validation set <NUM> without training the large neural network, i.e., without adjusting the parameters of any of the active (or inactive) components, and instead uses the large network parameter values that were determined during the previous iteration of the large network training phase. The controller parameter updating engine <NUM> then uses the results of the evaluations for the output sequences in the batch <NUM> to update the current values of the controller parameters to improve the expected performance of the architectures defined by the output sequences generated by the controller neural network <NUM> on the task. Evaluating the performance of trained instances and updating the current values of the controller parameters is described in more detail below with reference to <FIG>.

A controller parameter updating engine <NUM> then uses the performance metrics <NUM> to determine updated controller parameter values <NUM>.

During the large neural network training phase, the training engine <NUM> holds the values of the controller parameters fixed and samples an output sequence using the controller neural network <NUM>.

The training engine <NUM> then trains the large neural network with the architecture defined by the sampled output sequence active to determine updated large neural network parameter values <NUM> for those components that are active during the training. For example, the training engine <NUM> can train the large neural network for an entire pass through the training data <NUM> or for a specified number of training iterations. The training engine <NUM> can train the neural network using a training technique that is appropriate for the type of large neural network being trained. When the large neural network is a recurrent neural network, the training engine <NUM> can train the large neural network using backpropagation through time. When the large neural network is a convolutional neural network, the training engine <NUM> can train the large neural network using gradient descent with backpropagation.

Thus, the system <NUM> iteratively adjusts the controller parameter values while holding the large network parameters fixed during the controller training phase and iteratively adjusts the large network parameters while holding the controller parameter fixed during the large neural network training phase. By repeatedly performing these two phases, the system <NUM> trains the controller neural network <NUM> to generate output sequences that define high quality architectures without consuming an excessive amount of time and computational resources during the search process.

Once the controller neural network <NUM> has been trained, the system <NUM> can select a final architecture for the neural network, i.e., select a final subset of components to be active. To select the final architecture, the system <NUM> can generate a new output sequence in accordance with the trained values of the controller parameters and use the architecture defined by the new output sequence as the final architecture of the neural network, or can generate multiple new output sequences in accordance with the trained values and then select one of the architectures defined by the multiple new output sequences. In implementations where multiple new output sequences are generated, the system <NUM> can evaluate the performance of the architecture defined by each new output sequence on the validation set <NUM> and then select the highest-performing architecture as the final architecture. Alternatively, the system <NUM> can further train each selected architecture and then evaluate the performance of each of the architectures after the further training.

The neural network search system <NUM> can then output architecture data <NUM> that specifies the final architecture of the neural network, i.e., data specifying the layers that are part of the neural network, the connectivity between the layers, and the operations performed by the layers. For example, the neural network search system <NUM> can output the architecture data <NUM> to the user that submitted the training data.

In some implementations, instead of or in addition to outputting the architecture data <NUM>, the system <NUM> trains an instance of the neural network having the determined architecture, e.g., either from scratch or to fine-tune the parameter values generated as a result of training the large neural network, and then uses the trained neural network to process requests received by users, e.g., through the API provided by the system. That is, the system <NUM> can receive inputs to be processed, use the trained neural network to process the inputs, and provide the outputs generated by the trained neural network or data derived from the generated outputs in response to the received inputs.

In some implementations, the system <NUM> trains the controller neural network in a distributed manner. That is, the system <NUM> includes multiple replicas of the controller neural network. In some of these implementations where the training is distributed, each replica has a dedicated training engine that generates performance metrics for batches of output sequences output by the replica and trains a replica of the large neural network and a dedicated controller parameter update engine that determines updates to the controller parameters using the performance metrics. Once the controller parameter update engine has determined an update, the controller parameter update engine can transmit the update to a central parameter updating server that is accessible to all of the controller parameter update engines. Similarly, once the training engine has determined an update to the large neural network parameters, the training engine can transmit the update to the parameter server. The central parameter updating server can update the values of the controller parameters and large neural network parameters that are maintained by the server and send the updated values to the controller parameter update engine. In some cases, each of the multiple replicas and their corresponding training engines and parameter updating engines can operate asynchronously from each other set of training engines and parameter updating engines.

<FIG> is a diagram <NUM> of an example recurrent cell that can be generated by the architecture search system.

<FIG> shows a DAG <NUM> that represents the possible connectivity of the four nodes <NUM>, <NUM>, <NUM>, and <NUM> of the recurrent cell. The system determines the final connectivity of the DAG <NUM> by determining, for each node <NUM>-<NUM>, which input the node should receive. Each possible edge within the DAG is associated with a different set of parameters, so by determining the connectivity, the system also determines which sets of parameters are active and which are not. The system also determines which operations the node should perform on the received input from a predetermined set of inputs.

<FIG> also shows the architecture <NUM> of the recurrent cell that was generated by the system using the controller neural network and a diagram <NUM> that shows the outputs of the controller neural network that result in the architecture <NUM>.

In particular, the diagram <NUM> depicts the processing performed by the controller neural network <NUM> for seven example time steps <NUM>-<NUM> during the generation of an output sequence. As can be seen from the diagram <NUM>, time step <NUM> corresponds to node <NUM>, time steps <NUM> and <NUM> correspond to node <NUM>, time steps <NUM> and <NUM> correspond to node <NUM>, and time steps <NUM> and <NUM> correspond to node <NUM>.

The controller neural network <NUM> is a recurrent neural network that includes one or more recurrent neural network layers, e.g., layer <NUM>, that are configured to, for each time step, receive as input an embedding of the output generated at the preceding time step in the given output sequence and to process the input to update a current hidden state of the recurrent neural network. For example, the recurrent layers in the controller neural network <NUM> can be long-short term memory (LSTM) layers or gated recurrent unit (GRU) layers. In the example of <FIG>, at time step <NUM>, the controller receives as input the output at the preceding time step <NUM> and update the hidden states of the recurrent layers.

The controller neural network <NUM> also includes a respective output layer for each time step in the output sequence respectively. Each of the output layers is configured to receive an output layer input that includes the updated hidden state at the time step and to generate an output for the time step that defines a score distribution over possible values of the output at the time step. For example, each output layer can first project the output layer input into the appropriate dimensionality for the number of possible output values for the corresponding time step and then apply a softmax to the projected output layer input to generate a respective score for each of multiple possible output values.

Thus, to generate an output for a given time step in an output sequence, the system <NUM> provides as input to the controller neural network an embedding of the output at the preceding time step in the output sequence and the controller neural network generates an output for the time step that defines a score distribution over possible output values at the time step. For the very first time step in the output sequence, because there is no preceding time step, the system <NUM> can instead provide a pre-determined placeholder input. The system <NUM> then samples from the possible values in accordance with the score distribution to determine the output value at the time step in the output sequence. The possible values that a given output can take are fixed prior to training and the number of possible values can be different for different time steps.

As can be seen from the architecture <NUM>, at each time step during the processing of the recurrent cell, node <NUM> receives as input a cell input x_t for the time step and the output of the cell for the previous time step h_t-<NUM>. This can be predetermined, i.e., not generated using the controller neural network. Thus, at the first time step <NUM>, the controller neural network generates a probability distribution over possible activation functions to be applied by node <NUM>. In the example of <FIG>, the system has selected tanh as the activation function for node <NUM> from sampling from the probability distribution, e.g., from a set of possible activations that includes ReLU, tanh, sigmoid, and the identity operation.

For the remainder of the nodes in the graph, the system selects both the input to the node and the activation function to be applied by the node. Thus, for node <NUM>, the system has selected, from the corresponding probability distributions generated by the controller, that the node should receive an input from node <NUM> and apply the ReLu activation function. Generally, the probability distribution is over all of the nodes that are connected to the current node by an incoming edge in the DAG <NUM>, i.e., an edge that goes from another node to the current node.

Similarly, for node <NUM>, the system has selected that the node should receive an input from node <NUM> and apply the ReLu activation function while node <NUM> should receive an input from node <NUM> and apply the tanh activation function.

To generate the output of the cell for time step h_t, the system combines, e.g., averages ("avg"), the outputs of the nodes that were not chosen to provide input to any other node. In the example of <FIG>, the output h_t is the average of the outputs of node <NUM> and node <NUM>. Thus, the overall computation of the cell given the architecture <NUM> can be expressed as follows: <MAT> <MAT> <MAT> <MAT> <MAT> where the Ws are parameter matrices. As can be seen from the equations above, certain components that are possible in the DAG <NUM> are not included in the architecture <NUM>. In particular, parameter matrices corresponding to edges that were not selected are not used in the architecture <NUM>. For example, the parameter matrix that is applied to inputs at node <NUM> to inputs from node <NUM> <MAT> is not active in the architecture <NUM>. Additionally, each node applies only one activation function from the set of possible activation functions.

<FIG> is a diagram <NUM> of an example convolutional neural network architecture that can be generated by the architecture search system.

Like the diagram <NUM> in <FIG>, <FIG> also shows a four-node DAG <NUM>, an architecture <NUM>, and a diagram <NUM> of the processing of the controller neural network to generate the architecture <NUM>. Here, instead of representing components of a single recurrent cell, the nodes in the DAG <NUM> represent layers in a convolutional neural network.

Additionally, like the example of the diagram <NUM>, for the first node of the DAG <NUM>, the system predetermines the inputs to the node and only selects the computation performed by the node, while for each other node, the system selects both the input to the node (from nodes that are before current node in the output sequence) and the computation performed by the node. Instead of selecting activation functions, however, the system instead selects from a different set of possible computations to be performed by the nodes. In particular, the system can select either a particular type of convolution to be performed by the node or a max pooling operation to be performed (and, optionally, an average pooling operation). The types of convolution can include, for example, a set of convolution types that includes convolutions with filter sizes <NUM> × <NUM> and <NUM> × <NUM> and depthwise-separable convolutions with filter sizes <NUM>×<NUM> and <NUM>×<NUM>.

Additionally, unlike the example of the diagram <NUM>, for some or all of the nodes in the DAG, the system can select more than one of the incoming edges to the node to be active in order to form a skip connection. In particular, for each particular node other than the first node, the controller neural network generates a respective independent probability for each of the nodes connected to the particular node by an incoming edge. The system then samples from each probability independently to determine which nodes should provide outputs to the particular node in the final architecture. When a node receives input from more than one other node, the system can depth concatenate, average, or otherwise combine the individual inputs to the node.

While not depicted, as described above, the system can instead generate one or more types of cells and repeat those cells in a predetermined pattern to generate the convolutional neural network architecture, i.e., instead of generating an entire convolutional neural network as described above.

<FIG> is a flow diagram of an example process <NUM> for training the controller neural network. For convenience, the process <NUM> will be described as being performed by a system of one or more computers located in one or more locations. For example, a neural architecture search system, e.g., the neural architecture search system <NUM> of <FIG>, appropriately programmed, can perform the process <NUM>.

The system generates a batch of output sequences using the controller neural network and in accordance with current values of the controller parameters as of the iteration (step <NUM>). In particular, because the system samples from a score distribution when generating each output value in an output sequence, the sequences in the batch will generally be different even though they are each generated in accordance with the same controller parameter values. The batch generally includes a pre-determined number of output sequences, e.g., eight, sixteen, thirty-two, or sixty-four sequences.

For each output sequence in the batch, the system evaluates the performance of the architecture defined by the sequence to determine a performance metric for the trained instance on the particular neural network task (step <NUM>). For example, the performance metric can be an accuracy of an instance of the large neural network having the architecture on the validation set or a subset of the validation set as measured by an appropriate accuracy measure. For example, the accuracy can be based on a perplexity measure when the outputs are sequences or a classification error rate when the task is a classification task.

In order to perform the evaluation, the system uses the values of the large neural network parameters from the completion of the preceding iteration of the large neural network training phase. In other words, the system does not adjust the current values of the large neural network parameters when evaluating the output sequences in the batch.

The system uses the performance metrics for the architectures to adjust the current values of the controller parameters (step <NUM>).

In particular, the system adjusts the current values by training the controller neural network to generate output sequences that result in neural network architectures having increased performance metrics using a reinforcement learning technique. More specifically, the system trains the controller neural network to generate output sequences that maximize a received reward that is determined based on the performance metrics of the generated architectures. In particular, the reward for a given output sequence is a function of the performance metric for the corresponding architecture. For example, the reward can be one of: the performance metric, the square of the performance metric, the cube of the performance metric, the square root of the performance metric, and so on.

In some cases, the system trains the controller neural network to maximize the expected reward using a policy gradient technique. For example, the policy gradient technique can be a REINFORCE technique or a Proximal Policy Optimization (PPO) technique. For example, the system can estimate the gradient of the expected reward with respect to the controller parameters using an estimator of the gradient that satisfies: <MAT> where m is the number of sequences in the batch, T is the number of time steps in each sequence in the batch, at is the output at time step t in a given output sequence, Rk is the reward for output sequence k, θc are the controller parameters, and b is a baseline function, e.g., the exponential moving average of previous architecture accuracies.

The system can repeatedly perform steps <NUM>-<NUM> (the "controller training phase") to train the controller neural network, i.e., to determine trained values of the controller parameters from initial values of the controller parameters.

The system samples an output sequence using the controller neural network (step <NUM>).

The system trains an architecture defined by the sampled output sequence to update the large neural network parameters of the components that are designated as active by the sampled output sequence (step <NUM>). As described above, the system can train the architecture for a specified number of iterations or for one pass through the training data.

The system can repeatedly perform steps <NUM> and <NUM> (the "large neural network training phase") to update the values of the large neural network parameters during the training process. For example, the system can repeatedly alternate between performing steps <NUM>-<NUM> and performing steps <NUM>-<NUM> in order to search for a high performing neural network architecture.

In some implementations, the system trains the controller neural network in a distributed manner. That is, the system maintains multiple replicas of the controller neural network and the large neural network and updates the parameters values of the replicas asynchronously during the training. That is, the system can perform the steps <NUM>-<NUM> asynchronously for each replica and can update the controller parameters and the large neural network parameters using the gradients determined for each of the replicas.

Claim 1:
A computer-implemented method of determining an architecture for a neural network for performing a particular neural network task, the method comprising:
performing a plurality of iterations, wherein each iteration comprises a controller training phase and a large neural network training phase;
wherein the controller training phase comprises:
generating, using a controller neural network having a plurality of controller parameters and in accordance with current values of the controller parameters, a batch of output sequences, each output sequence in the batch specifying a respective subset of a plurality of components of a large neural network that should be active during the processing of inputs by the large neural network, wherein the large neural network has a plurality of large network parameters;
for each output sequence in the batch:
determining a performance metric of the large neural network on the particular neural network task (i) in accordance with current values of the large network parameters and (ii) with only the subset of components specified by the output sequence active; and
using the performance metrics for the output sequences in the batch to adjust the current values of the controller parameters of the controller neural network;
wherein the large neural network training phase comprises:
generating, using the controller neural network and in accordance with the adjusted values of the controller parameters, a new output sequence; and
training the large neural network with only the subset of components specified by the new output sequence active on training data to determine adjusted values of the large network parameters;
wherein the adjusted values of the large network parameters are used in the controller phase of the next iteration; and
wherein the inputs to the large neural network comprises a digital input comprising image data or features extracted from an image; and the neural network task comprises generating a score, classification, or regression output based on the digital input, each score representing an estimated likelihood that the image contains an image of an object belonging to the category.