A speech recognition system comprising: an input device to receive voice sounds; one or more processors; and one or more storage devices storing parameters and program modules including instructions executable by the one or more processors which, when executed, cause the one or more processors to perform operations comprising: extracting, using an acoustic feature extraction module, an acoustic feature sequence from audio waveform data converted from the voice sounds; encoding the acoustic feature sequence into a hidden vector sequence using an encoder network having encoder network parameters; predicting first output label sequence probabilities by feeding the hidden vector sequence to a decoder network having decoder network parameters; characterized by the instructions executable by the one or more processors which, when executed, cause the one or more processors to perform operations comprising: predicting second output label sequence probabilities by a connectionist temporal classification module using CTC network parameters and the hidden vector sequence from the encoder network; and searching, using a label sequence search module, for an output label sequence having a highest sequence probability by combining the first and second output label sequence probabilities provided from the decoder network and the CTC module. an input device to receive voice sounds; one or more processors; and one or more storage devices storing parameters and program modules including instructions executable by the one or more processors which, when executed, cause the one or more processors to perform operations comprising: extracting, using an acoustic feature extraction module, an acoustic feature sequence from audio waveform data converted from the voice sounds; encoding the acoustic feature sequence into a hidden vector sequence using an encoder network having encoder network parameters; predicting first output label sequence probabilities by feeding the hidden vector sequence to a decoder network having decoder network parameters; characterized by the instructions executable by the one or more processors which, when executed, cause the one or more processors to perform operations comprising: predicting second output label sequence probabilities by a connectionist temporal classification module using CTC network parameters and the hidden vector sequence from the encoder network; and searching, using a label sequence search module, for an output label sequence having a highest sequence probability by combining the first and second output label sequence probabilities provided from the decoder network and the CTC module. extracting, using an acoustic feature extraction module, an acoustic feature sequence from audio waveform data converted from the voice sounds; encoding the acoustic feature sequence into a hidden vector sequence using an encoder network having encoder network parameters; predicting first output label sequence probabilities by feeding the hidden vector sequence to a decoder network having decoder network parameters; predicting second output label sequence probabilities by a connectionist temporal classification module using CTC network parameters and the hidden vector sequence from the encoder network; and searching, using a label sequence search module, for an output label sequence having a highest sequence probability by combining the first and second output label sequence probabilities provided from the decoder network and the CTC module.