Audio encoder for generating an audio signal, comprising: a controllable time warper for time warping an audio signal to obtain a time warped audio signal; a time/frequency converter for converting at least a portion of the time warped audio signal or of a non-time-warped version of the audio signal into a spectral representation; a temporal noise shaping stage for performing a prediction filtering over frequency of the spectral representation in accordance with a temporal noise shaping control instruction, wherein the prediction filtering is not performed, when the temporal noise shaping control instruction does not exist; a temporal noise shaping controller (800, 802, 804) for generating the temporal noise shaping control instruction based on the spectral representation, wherein the temporal noise shaping controller is configured for increasing a likelihood for performing the predictive filtering over frequency, when the spectral representation is based on a time warped audio signal or for decreasing the likelihood for performing the prediction filtering over frequency, when the spectral representation is not based on a time warped audio signal; and a processor for further processing an output of the temporal noise shaping stage to obtain the encoded audio signal; wherein the temporal noise shaping controller (800, 802, 804) is configured for estimating a gain in a bitrate or a quality, when the audio signal is subjected to the prediction filtering by the temporal noise shaping stage, for comparing the estimated gain to a decision threshold, and for deciding, in favor of the prediction filtering, when the estimated gain is in a predetermined relation to the decision threshold, wherein the temporal noise shaping controller is furthermore configured for varying the decision threshold so that, for the same estimated gain, the prediction filtering is activated, when the spectral representation is based on a time warped signal, and is not activated, when the spectral representation is not based on a time-warped audio signal. a controllable time warper for time warping an audio signal to obtain a time warped audio signal; a time/frequency converter for converting at least a portion of the time warped audio signal or of a non-time-warped version of the audio signal into a spectral representation; a temporal noise shaping stage for performing a prediction filtering over frequency of the spectral representation in accordance with a temporal noise shaping control instruction, wherein the prediction filtering is not performed, when the temporal noise shaping control instruction does not exist; a temporal noise shaping controller (800, 802, 804) for generating the temporal noise shaping control instruction based on the spectral representation, wherein the temporal noise shaping controller is configured for increasing a likelihood for performing the predictive filtering over frequency, when the spectral representation is based on a time warped audio signal or for decreasing the likelihood for performing the prediction filtering over frequency, when the spectral representation is not based on a time warped audio signal; and a processor for further processing an output of the temporal noise shaping stage to obtain the encoded audio signal; wherein the temporal noise shaping controller (800, 802, 804) is configured for estimating a gain in a bitrate or a quality, when the audio signal is subjected to the prediction filtering by the temporal noise shaping stage, for comparing the estimated gain to a decision threshold, and for deciding, in favor of the prediction filtering, when the estimated gain is in a predetermined relation to the decision threshold, wherein the temporal noise shaping controller is furthermore configured for varying the decision threshold so that, for the same estimated gain, the prediction filtering is activated, when the spectral representation is based on a time warped signal, and is not activated, when the spectral representation is not based on a time-warped audio signal.