Audio decoder for decoding encoded audio data, comprising: an input interface configured for receiving the encoded audio data, the encoded audio data comprising a plurality of encoded audio channels or a plurality of encoded audio objects and compressed metadata related to the plurality of audio objects, and a mode indication; a core decoder configured for decoding the plurality of encoded audio channels and the plurality of encoded audio objects; a metadata decompressor configured for decompressing the compressed metadata; an object processor configured for processing the plurality of decoded audio objects using the decompressed metadata to obtain a number of output audio channels comprising audio data from the audio objects and the decoded audio channels; a mode controller connected to the input interface and configured for analyzing the encoded audio data to detect the mode indication indicating a first mode or a second mode, wherein, in the first mode, the encoded audio data comprise encoded audio channels and encoded audio objects, and wherein, in the second mode, the encoded audio data only comprise the plurality of encoded audio channels; and a post-processor configured for converting the number of output audio channels into an output format, wherein the audio decoder, controlled by the mode controller, is configured to bypass the object processor and to feed a plurality of decoded channels into the post-processor, when the second mode has been detected by the mode controller, and to feed the plurality of decoded audio objects and the plurality of decoded audio channels into the object processor, when the first mode has been detected by the mode controller. an input interface configured for receiving the encoded audio data, the encoded audio data comprising a plurality of encoded audio channels or a plurality of encoded audio objects and compressed metadata related to the plurality of audio objects, and a mode indication; a core decoder configured for decoding the plurality of encoded audio channels and the plurality of encoded audio objects; a metadata decompressor configured for decompressing the compressed metadata; an object processor configured for processing the plurality of decoded audio objects using the decompressed metadata to obtain a number of output audio channels comprising audio data from the audio objects and the decoded audio channels; a mode controller connected to the input interface and configured for analyzing the encoded audio data to detect the mode indication indicating a first mode or a second mode, wherein, in the first mode, the encoded audio data comprise encoded audio channels and encoded audio objects, and wherein, in the second mode, the encoded audio data only comprise the plurality of encoded audio channels; and a post-processor configured for converting the number of output audio channels into an output format, wherein the audio decoder, controlled by the mode controller, is configured to bypass the object processor and to feed a plurality of decoded channels into the post-processor, when the second mode has been detected by the mode controller, and to feed the plurality of decoded audio objects and the plurality of decoded audio channels into the object processor, when the first mode has been detected by the mode controller.