A speech encoder that encodes a speech signal, comprising: frequency transformation means for transforming the speech signal into a frequency domain; down-sampling means for down-sampling the speech signal and acquiring a low frequency band signal; low frequency band encoding means for encoding the low frequency band signal acquired by the down-sampling means; first to Nth, where N is an integer equal to or larger than two, low frequency band time envelope calculation means for calculating a plurality of time envelopes of low frequency band components of the speech signal transformed into the frequency domain by the frequency transformation means; time envelope information calculation means for calculating, using the time envelopes of the low frequency band components calculated by the first to Nth low frequency band time envelope calculation means, time envelope information necessary to acquire a time envelope of high frequency band components of the speech signal transformed by the frequency transformation means; supplementary information calculation means for analyzing the speech signal and calculating supplementary information for high frequency band generation to be used for generating high frequency band components from the low frequency band signal; encoding means for encoding the supplementary information for high frequency band generation, which is generated by the supplementary information calculation means, and the time envelope information, which is calculated by the time envelope information calculation means; coded sequence construction means for constructing a high frequency band coded sequence from the supplementary information for high frequency band generation and the time envelope information encoded by the encoding means; and multiplexing means for generating a coded sequence in which the low frequency band coded sequence, which is acquired by the low frequency band encoding means, and the high frequency band coded sequence, which is constructed by the coded sequence construction means, are multiplexed, wherein characteristics related to a steepness of a rising edge or a falling edge of the speech signal are detected from the speech signal in the frequency domain, and wherein the time envelope information calculation means generates control information indicating whether or not to perform time envelope calculation according to the characteristics, the control information based on the characteristics is added to the coded sequence. frequency transformation means for transforming the speech signal into a frequency domain; down-sampling means for down-sampling the speech signal and acquiring a low frequency band signal; low frequency band encoding means for encoding the low frequency band signal acquired by the down-sampling means; first to Nth, where N is an integer equal to or larger than two, low frequency band time envelope calculation means for calculating a plurality of time envelopes of low frequency band components of the speech signal transformed into the frequency domain by the frequency transformation means; time envelope information calculation means for calculating, using the time envelopes of the low frequency band components calculated by the first to Nth low frequency band time envelope calculation means, time envelope information necessary to acquire a time envelope of high frequency band components of the speech signal transformed by the frequency transformation means; supplementary information calculation means for analyzing the speech signal and calculating supplementary information for high frequency band generation to be used for generating high frequency band components from the low frequency band signal; encoding means for encoding the supplementary information for high frequency band generation, which is generated by the supplementary information calculation means, and the time envelope information, which is calculated by the time envelope information calculation means; coded sequence construction means for constructing a high frequency band coded sequence from the supplementary information for high frequency band generation and the time envelope information encoded by the encoding means; and multiplexing means for generating a coded sequence in which the low frequency band coded sequence, which is acquired by the low frequency band encoding means, and the high frequency band coded sequence, which is constructed by the coded sequence construction means, are multiplexed, wherein characteristics related to a steepness of a rising edge or a falling edge of the speech signal are detected from the speech signal in the frequency domain, and wherein the time envelope information calculation means generates control information indicating whether or not to perform time envelope calculation according to the characteristics, the control information based on the characteristics is added to the coded sequence.