A system comprising one or more computers and one or more storage devices storing instructions that when executed by one or more computers cause the one or more computers to implement: a sequence-to-sequence recurrent neural network configured to: receive a sequence of characters in a particular natural language, and process the sequence of characters to generate a spectrogram of a verbal utterance of the sequence of characters in the particular natural language; and a subsystem configured to: receive the sequence of characters in the particular natural language, and provide the sequence of characters as input to the sequence-to-sequence recurrent neural network to obtain as output the spectrogram of the verbal utterance of the sequence of characters in the particular natural language, wherein the sequence-to-sequence recurrent neural network comprises: an encoder neural network configured to: receive the sequence of characters, and process the sequence of characters to generate a respective encoded representation of each of the characters in the sequence; and an attention-based decoder recurrent neural network configured to: receive a sequence of decoder inputs; and for each decoder input in the sequence: process the decoder input and the encoded representations to generate r frames of the spectrogram, wherein r is an integer greater than one, wherein each of the second and subsequent decoder inputs in the sequence is one or more of the r frames of the spectogram that were generated by processing the preceding decoder input in the sequence. a sequence-to-sequence recurrent neural network configured to: receive a sequence of characters in a particular natural language, and process the sequence of characters to generate a spectrogram of a verbal utterance of the sequence of characters in the particular natural language; and receive a sequence of characters in a particular natural language, and process the sequence of characters to generate a spectrogram of a verbal utterance of the sequence of characters in the particular natural language; and a subsystem configured to: receive the sequence of characters in the particular natural language, and provide the sequence of characters as input to the sequence-to-sequence recurrent neural network to obtain as output the spectrogram of the verbal utterance of the sequence of characters in the particular natural language, wherein the sequence-to-sequence recurrent neural network comprises: receive the sequence of characters in the particular natural language, and provide the sequence of characters as input to the sequence-to-sequence recurrent neural network to obtain as output the spectrogram of the verbal utterance of the sequence of characters in the particular natural language, wherein the sequence-to-sequence recurrent neural network comprises: an encoder neural network configured to: receive the sequence of characters, and process the sequence of characters to generate a respective encoded representation of each of the characters in the sequence; and receive the sequence of characters, and process the sequence of characters to generate a respective encoded representation of each of the characters in the sequence; and an attention-based decoder recurrent neural network configured to: receive a sequence of decoder inputs; and for each decoder input in the sequence: process the decoder input and the encoded representations to generate r frames of the spectrogram, wherein r is an integer greater than one, wherein each of the second and subsequent decoder inputs in the sequence is one or more of the r frames of the spectogram that were generated by processing the preceding decoder input in the sequence. receive a sequence of decoder inputs; and for each decoder input in the sequence: process the decoder input and the encoded representations to generate r frames of the spectrogram, wherein r is an integer greater than one, wherein each of the second and subsequent decoder inputs in the sequence is one or more of the r frames of the spectogram that were generated by processing the preceding decoder input in the sequence.