An audio decoder for decoding a bitstream of encoded audio data, wherein the bitstream of encoded audio data represents a sequence of audio sample values and comprises a plurality of frames, wherein each frame comprises associated encoded audio sample values, the audio decoder comprising: a determiner configured to determine whether a frame of the bitstream of encoded audio data is an immediate playout frame comprising encoded audio sample values associated with a current frame and additional information, wherein the bitstream of encoded audio data is an MPEG-4 Audio bitstream, wherein the additional information is transported via an MPEG-4 Audio bitstream extension mechanism that is an extension payload element of a new extension payload element type, EXT_AUDIOPRE_ROLL, wherein the extension payload element is located at a first position in the MPEG-4 Audio bitstream, and wherein the extension payload element is contained inside a fill element (ID_FIL), wherein the additional information comprises encoded audio sample values of a number of frames preceding the immediate playout frame, wherein the encoded audio sample values of the preceding frames are encoded using the same codec configuration as the current frame, wherein the number of preceding frames, corresponding to pre-roll frames (2, 3), corresponds to the number of frames needed by the decoder to build up the full signal so as to be in a position to output valid audio sample values associated with the current frame whenever an immediate playout frame is decoded; and an initializer configured to initialize the decoder if the determiner determines that the frame is an immediate playout frame, wherein initializing the decoder comprises decoding the encoded audio sample values comprised by the additional information before decoding the encoded audio sample values associated with the current frame, wherein the initializer is configured to switch the audio decoder from a current codec configuration to a different codec configuration if the determiner determines that the frame is an immediate playout frame and if the audio sample values of the current frame have been encoded using the different codec configuration, and wherein the decoder is configured to decode the current frame using the current codec configuration and to discard the additional information if the determiner determines that the frame is an immediate playout frame and if the audio sample values of the current frame have been encoded using the current codec configuration. wherein the additional information comprises encoded audio sample values of a number of frames preceding the immediate playout frame, wherein the encoded audio sample values of the preceding frames are encoded using the same codec configuration as the current frame, wherein the number of preceding frames, corresponding to pre-roll frames (2, 3), corresponds to the number of frames needed by the decoder to build up the full signal so as to be in a position to output valid audio sample values associated with the current frame whenever an immediate playout frame is decoded; and an initializer configured to initialize the decoder if the determiner determines that the frame is an immediate playout frame, wherein initializing the decoder comprises decoding the encoded audio sample values comprised by the additional information before decoding the encoded audio sample values associated with the current frame, wherein the initializer is configured to switch the audio decoder from a current codec configuration to a different codec configuration if the determiner determines that the frame is an immediate playout frame and if the audio sample values of the current frame have been encoded using the different codec configuration, and wherein the decoder is configured to decode the current frame using the current codec configuration and to discard the additional information if the determiner determines that the frame is an immediate playout frame and if the audio sample values of the current frame have been encoded using the current codec configuration.