Augmenting neural networks to generate additional outputs

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for augmenting neural networks to generate additional outputs. One of the systems includes a neural network and a sequence processing subsystem, wherein the sequence processing subsystem is configured to perform operations comprising, for each of the system inputs in a sequence of system inputs: receiving the system input; generating an initial neural network input from the system input; causing the neural network to process the initial neural network input to generate an initial neural network output for the system input; and determining, from a first portion of the initial neural network output for the system input, whether or not to cause the neural network to generate one or more additional neural network outputs for the system input.

BACKGROUND

This specification relates to neural network system architectures.

Some neural networks are recurrent neural networks. A recurrent neural network is a neural network that receives an input sequence and generates an output sequence from the input sequence. In particular, a recurrent neural network can use some or all of the internal state of the network from a previous time step in computing an output at a current time step. An example of a recurrent neural network is a Long Short-Term Memory (LSTM) neural network that includes one or more LSTM memory blocks. Each LSTM memory block can include one or more cells that each include an input gate, a forget gate, and an output gate that allow the cell to store previous states for the cell, e.g., for use in generating a current activation or to be provided to other components of the LSTM neural network.

SUMMARY

This specification describes technologies that relate to augmented neural network systems. In general, an augmented neural network system includes a neural network configured to receive neural network inputs and generate a respective neural network output for each neural network input. The augmented neural network system also includes a sequence processing subsystem that is configured to, for each of the system inputs in a sequence of system inputs, receive the system input and generate an initial neural network input from the system input. The sequence processing subsystem is also configured to cause the neural network to process the initial neural network input to generate an initial neural network output for the system input and to determine, from a first portion of the initial neural network output for the system input, whether or not to cause the neural network to generate one or more additional neural network outputs for the system input.

The subject matter described in this specification can be implemented in particular embodiments so as to realize one or more of the following advantages. By allowing an augmented neural network system to determine when a final system output has been generated for a given system input, the performance of the neural network can be improved by allowing the neural network to generate more accurate outputs for difficult inputs. Additionally, the performance of the neural network is improved without significant increases in processing time needed for or computing resources used by the neural network, either during training or, after training, at run time. Additionally, the augmented neural network system is configured to signal when the output generated by the system is the final system output. By doing this, the system can be given time to accurately produce more complicated system outputs, e.g., to ensure that incomplete variable length outputs are not sent for further processing while still incomplete, improving the performance of the system.

DETAILED DESCRIPTION

FIG. 1shows an example augmented neural network system100. The augmented neural network system100is an example of a system implemented as computer programs on one or more computers in one or more locations, in which the systems, components, and techniques described below are implemented.

The augmented neural network system100is a machine learning system that receives a sequence of system inputs and generates a sequence of system outputs from the system inputs. For example, the augmented neural network system100can receive a system input x as part of an input sequence and generate a system output y that is included in the sequence of system outputs. The augmented neural network system100can store the generated sequence of outputs in an output data repository or provide the sequence of outputs for use for some other immediate purpose.

The augmented neural network system100can be configured to receive any kind of digital data input and to generate any kind of score or classification output based on the input. For example, if the inputs to the augmented neural network system100are images or features that have been extracted from images, the output generated by the augmented neural network system100for a given image may be scores for each of a set of object categories, with each score representing an estimated likelihood that the image contains an image of an object belonging to the category. As another example, if the inputs to the augmented neural network system100are Internet resources (e.g., web pages), documents, or portions of documents or features extracted from Internet resources, documents, or portions of documents, the output generated by the augmented neural network system100for a given Internet resource, document, or portion of a document may be a score for each of a set of topics, with each score representing an estimated likelihood that the Internet resource, document, or document portion is about the topic. As another example, if the inputs to the augmented neural network system100are features of an impression context for a particular advertisement, the output generated by the augmented neural network system100may be a score that represents an estimated likelihood that the particular advertisement will be clicked on. As another example, if the inputs to the augmented neural network system100are features of a personalized recommendation for a user, e.g., features characterizing the context for the recommendation, e.g., features characterizing previous actions taken by the user, the output generated by the augmented neural network system100may be a score for each of a set of content items, with each score representing an estimated likelihood that the user will respond favorably to being recommended the content item. As another example, if the input to the augmented neural network system100is text in one language, the output generated by the augmented neural network system100may be a score for each of a set of pieces of text in another language, with each score representing an estimated likelihood that the piece of text in the other language is a proper translation of the input text into the other language. As another example, if the input to the augmented neural network system100is a spoken utterance, a sequence of spoken utterances, or features derived from one of the two, the output generated by the augmented neural network system100may be a score for each of a set of pieces of text, each score representing an estimated likelihood that the piece of text is the correct transcript for the utterance or sequence of utterances. As another example, the augmented neural network system100can be part of a speech synthesis system. As another example, the augmented neural network system100can be part of a video processing system. As another example, the augmented neural network system100can be part of a dialogue system. As another example, the augmented neural network system100can be part of an auto-completion system. As another example, the augmented neural network system100can be part of a text processing system. As another example, the augmented neural network system100can be part of a reinforcement learning system.

In particular, the augmented neural network system100includes a neural network102and a sequence processing subsystem108. The neural network102may be a feedforward neural network or a recurrent neural network that is configured to receive a neural network input and process the neural network input to generate a neural network output.

The sequence processing subsystem108receives the system input x and generates a neural network input s from the system input x. The sequence processing subsystem108then causes the neural network102to process the neural network input s to generate a neural network output o1for the neural network input. Additionally, from each neural network input generated by the neural network102, the sequence processing subsystem108determines whether to cause the neural network102to generate one or more additional neural network outputs for the current system input. The sequence processing subsystem108then determines, from each neural network output generated by the neural network102for the system input x, the system output y for the system input x.

That is, the sequence processing subsystem108determines, from neural network outputs generated by the neural network102for a given system input, when to generate a final system output for the given system input and to provide a neural network input generated from the next system input to the neural network for processing. Processing a system input to generate a system output is described in more detail below with reference toFIG. 2. Determining whether to cause the neural network to generate additional neural network outputs for a given system input is described in more detail below with reference toFIG. 3.

In some implementations, the neural network102is also augmented with an external memory104and a memory interface subsystem106. In these implementations, the neural network102is configured to generate a neural network output that includes the output o1provided to the sequence processing system108and an output o2that is provided to the memory interface subsystem106. The memory interface subsystem106receives the output o2generated by the neural network102and translates the received output into erase, read, and write operations to be performed on the external memory104. That is, the memory interface subsystem106receives an output o2from the neural network102and, based on the output o2, erases data e from the external memory104, writes data w to the external memory104, and reads data r from the external memory104. The data r read by the memory interface subsystem106can then be provided to the sequence processing subsystem108for inclusion as a portion of a later neural network input, e.g., along with a system input. An augmented neural network system102that includes an external memory and a memory interface subsystem is described in more detail in U.S. patent application Ser. No. 62/064,965, filed Oct. 16, 2014, titled “AUGMENTING NEURAL NETWORKS WITH EXTERNAL MEMORY,” the contents of which are hereby incorporated by reference herein in their entirety.

FIG. 2is a flow diagram of an example process200for generating a system output from a system input. For convenience, the process200will be described as being performed by a system of one or more computers located in one or more locations. For example, an augmented neural network system, e.g., the augmented neural network system100ofFIG. 1, appropriately programmed in accordance with this specification, can perform the process200.

The system receives a sequence of system inputs (step202).

The system generates one or more neural network outputs for each of the system inputs in the sequence (step204). The system generates each of the neural network outputs by processing a respective neural network input using a neural network, e.g., the neural network102ofFIG. 1, that is configured to process the neural network input to generate a neural network output from the neural network input.

Generally, for each system input, the system generates an initial neural network input from the system input and processes the initial neural network input using the neural network to generate an initial neural network output.

The system then determines, from a decision portion of the initial neural network output, whether another neural network output should be generated for the system input. The decision portion of the neural network output is a predetermined portion of the neural network output that has been designated, e.g., by a system administrator, as the portion of the neural network output to be used to make the determination of whether another neural network output should be generated, i.e., so that the same portion of the neural network output is used to make the determination for each output generated by the neural network. Generally, the decision portion of a given neural network output is a value at a predetermined position in the neural network output. As will be described further below, in some implementations, the neural network is configured so that the range of possible values for the decision portion is between zero and one, either inclusive or exclusive. Generating one or more neural network outputs for a system input is described in more detail below with reference toFIG. 3.

The system generates a respective system output for each system input from the neural network outputs for the system input (step206). When only a single neural network output has been generated for a given system input, the system generates the system output for the system input from a system output portion of the single neural network output. The system output portion of the neural network output is a predetermined portion of the neural network output that is different from the decision portion and that has been designated, e.g., by a system administrator, as the portion to be used to generate the system output, i.e., so that the same portion of the neural network output is used to generate the system output for each output generated by the neural network.

In some implementations, the system provides the system output portion as the system output for the system input. In some other implementations, however, the system applies one or more transformations to the system output portion in order to generate the system output. For example, the system output portion may be transformed into a command to control a robot or a different digital interface.

When multiple neural network outputs have been generated for a given system input, the system can generate the system output for the system input in any of a variety of ways.

For example, the system can generate the system output from only the system output portion of the last neural network output that was generated for the system input. That is, the system discards all neural network outputs other than the last neural network output for the system input and then generates the system output from the last neural network output.

As another example, the system can generate the system output from the system output portions of all of the neural network outputs for the system input. In particular, the system can combine the system output portions of all of the neural network inputs to generate the system output. In some implementations, the system computes a weighted sum of the system output portions. That is, the system can multiply each system output portion by the value of the corresponding decision portion to generate a weighted system output portion and then sum the weighted system output portions to generate the system output. Optionally, the system normalizes the weighted sum, e.g., by dividing the sum by a sum of the values of the decision portions. Other ways of combining the neural network outputs to generate the system input are possible, e.g., by computing a measure of central tendency of the neural network outputs.

FIG. 3is a flow diagram of an example process300for generating one or more neural network outputs from a system input. For convenience, the process300will be described as being performed by a system of one or more computers located in one or more locations. For example, an augmented neural network system, e.g., the augmented neural network system100ofFIG. 1, appropriately programmed in accordance with this specification, can perform the process300.

The system receives a system input (step302). The system input is one input of a sequence of system inputs received by the system.

The system generates an initial neural network input from the system input (step304). In implementations where the neural network is not augmented with an external memory, the system can provide the system input as the initial neural network input. In implementations where the neural network is augmented with an external memory, the system can combine, e.g., concatenate, the current read vector or vectors that have been read from the external memory by the memory interface subsystem with the system input to generate the initial neural network input.

The system processes the initial neural network input using the neural network to generate an initial neural network output from the system input (step306). Depending on the implementation, the neural network can either be a feed-forward neural network, e.g., a convolutional neural network or another kind of deep feed-forward neural network, or a recurrent neural network, e.g., an LSTM neural network. If the neural network is a recurrent neural network, the recurrent neural network also uses the internal state of the recurrent neural network from the preceding neural network input processed by the neural network in processing the current neural network input to generate the neural network output. In some implementations, the neural network includes both recurrent and feed-forward layers.

In implementations where the neural network is augmented with an external memory, the system also reads, writes, and erases from the external memory in accordance with designated portions of the neural network output. Reading, writing, and erasing from the external memory is described in more detail in U.S. patent application Ser. No. 62/064,965, filed Oct. 16, 2014, titled “AUGMENTING NEURAL NETWORKS WITH EXTERNAL MEMORY,” the contents of which are hereby incorporated by reference herein in their entirety.

The system determines, from the decision portion of the neural network output, whether or not to generate an additional neural network output for the system input (step308). The system can make this determination from the decision portion of the neural network output in any of a variety of ways.

For example, the system can determine whether the value of the decision portion of the neural network output exceeds a threshold value and, if the value of the decision portion exceeds the threshold value, determine that no more additional neural network outputs should be generated for the system input.

As another example, the value of the decision portion may be configured to be constrained to a range between zero and one, either inclusive or exclusive. In this example, the system can treat the value of the decision portion as a probability. That is, the system can determine not to generate any more additional neural network outputs with a probability equal to the value of the decision portion and determine to generate an additional neural network output with a probability equal to one minus the value of the decision portion. If the value is not constrained to the range, the system can normalize the value and treat the normalized value as a probability.

If the system determines not to generate any more additional neural network outputs, the system proceeds to processing the next system input in the sequence (step310). If the system input is the last system input in the sequence, the system outputs the generated sequence of system outputs for use for some immediate purpose or stores the sequence of outputs in an output data repository.

If the system determines to generate an additional neural network output, the system generates an additional neural network input (step312). In implementations where the neural network is not augmented with an external memory, the system can again provide the system input as the additional neural network input. Alternatively, the system can provide a default input, e.g., an input vector of predetermined default values, as the additional neural network input.

In implementations where the neural network is augmented with an external memory, the system can combine the current read vector that has been read from the external memory, e.g., based on reading from the external memory in accordance with a read portion of the current neural network output, with the system input or the default input.

The system processes the additional neural network input using the neural network to generate an additional neural network output (step314). Because the state of the neural network will generally be different when processing each additional neural network input than the state of the neural network when processing the initial neural network input or when processing each other additional neural network input, each additional neural network output will generally differ from each other additional neural network output and the initial neural network output. For example, in implementations where the neural network is augmented with an external memory, the current read vector being provided to the neural network as part of each neural network input will generally be different. As another example, when the neural network includes one or more recurrent layers, the internal state of the neural network when processing each of the neural network inputs will generally be different.

The system determines, from the decision portion of the neural network output, whether or not to generate an additional neural network output for the system input (returning to step308).

In some implementations, the augmented neural network system can include multiple neural networks that each process each system input to generate a respective neural network output, with the appropriate portions of the neural network outputs generated by each network being combined to generate the system output for the system input. In these cases, the system can determine whether or not to generate an additional neural network output for the system input from the decision portions of each of the neural network inputs, e.g., by computing a measure of central tendency of the decision portions and then making the determination from the measure as described above.

The processes200and300can be performed for each system input in a sequence of system inputs to generate a sequence of system outputs for the sequence of system inputs. The sequence of system inputs can be a sequence for which the desired output, i.e., the output sequence that should be generated by the system for the input sequence, is not known. The system can also perform the processes200and300on inputs in a set of training data, i.e., a set of inputs for which the output that should be predicted by the system is known, in order to train the system, i.e., to determine trained values for the parameters of the neural network and, in implementations where the neural network is augmented with an external memory, any additional parameters of processes used in erasing, writing, and reading from the external memory. In implementations where the components of the system are entirely differentiable, e.g., when the system decides whether or not to generate an additional neural network input by treating the decision portion as a probability, the processes200and300can be performed repeatedly on inputs selected from a set of training data as part of a conventional machine learning training technique to train the neural network, e.g., a stochastic gradient descent with back propagation through time training technique if the neural network is a recurrent neural network.

In some implementations, the objective function that is being optimized includes, in addition to one or more terms that penalize the system for generating incorrect system outputs, one or more terms that penalize the system for spending too much time processing in order to generate a system output. That is, the objective function includes one or more terms that increase the penalty for the system the more time the system takes to generate an output sequence for a given input sequence. The time can be measured, e.g., in terms of number of neural network outputs generated in processing the input sequence. Thus, by training the system using an objective function that offsets the penalty for incorrect system outputs with a penalty for spending too much time processing, the neural network can be trained not to generate excessive additional neural network outputs for any given system input, thereby reducing the increase in processing time and computing resources required for the system to process a given input sequence.