A speech recognition system, comprising: a computer memory configured to store an encoder configured to convert an input acoustic signal into a sequence of encoder states; an alignment decoder configured to identify locations of encoder states in the sequence of encoder states that encode transcription outputs; a partition module configured to partition the sequence of encoder states into a set of partitions based on the locations of the identified encoder states; and an attention-based decoder configured to determine the transcription outputs for each partition of encoder states submitted to the attention-based decoder as an input; an input interface configured to receive the acoustic signal representing at least a portion of a speech utterance; a hardware processor configured to submit the received acoustic signal to the encoder to produce the sequence of encoder states; submit the sequence of encoder states into the alignment decoder to identify locations of the encoder states that encode the transcription outputs; partition the sequence of encoder states, using the partition module, into the set of partitions based on the locations of the identified encoder states; and submit the set of partitions sequentially into the attention-based decoder to produce a transcription output for each of the submitted partitions; and an output interface configured to output the transcription outputs. a computer memory configured to store an encoder configured to convert an input acoustic signal into a sequence of encoder states; an alignment decoder configured to identify locations of encoder states in the sequence of encoder states that encode transcription outputs; a partition module configured to partition the sequence of encoder states into a set of partitions based on the locations of the identified encoder states; and an attention-based decoder configured to determine the transcription outputs for each partition of encoder states submitted to the attention-based decoder as an input; an input interface configured to receive the acoustic signal representing at least a portion of a speech utterance; an encoder configured to convert an input acoustic signal into a sequence of encoder states; an alignment decoder configured to identify locations of encoder states in the sequence of encoder states that encode transcription outputs; a partition module configured to partition the sequence of encoder states into a set of partitions based on the locations of the identified encoder states; and an attention-based decoder configured to determine the transcription outputs for each partition of encoder states submitted to the attention-based decoder as an input; an input interface configured to receive the acoustic signal representing at least a portion of a speech utterance; a hardware processor configured to submit the received acoustic signal to the encoder to produce the sequence of encoder states; submit the sequence of encoder states into the alignment decoder to identify locations of the encoder states that encode the transcription outputs; partition the sequence of encoder states, using the partition module, into the set of partitions based on the locations of the identified encoder states; and submit the set of partitions sequentially into the attention-based decoder to produce a transcription output for each of the submitted partitions; and submit the received acoustic signal to the encoder to produce the sequence of encoder states; submit the sequence of encoder states into the alignment decoder to identify locations of the encoder states that encode the transcription outputs; partition the sequence of encoder states, using the partition module, into the set of partitions based on the locations of the identified encoder states; and submit the set of partitions sequentially into the attention-based decoder to produce a transcription output for each of the submitted partitions; and an output interface configured to output the transcription outputs.