Audio encoder for encoding a multichannel signal, comprising: a linear prediction domain encoder; a frequency domain encoder; a controller for switching between the linear prediction domain encoder and the frequency domain encoder, wherein the linear prediction domain encoder comprises a downmixer for downmixing the multichannel signal to obtain a downmix signal, a linear prediction domain core encoder for encoding the downmix signal and a first joint multichannel encoder for generating first multichannel information from the multichannel signal, wherein the frequency domain encoder comprises a second joint multichannel encoder for generating second multichannel information from the multichannel signal, wherein the second joint multichannel encoder is different from the first joint multichannel encoder, and wherein the controller is configured to perform the switching such that a portion of the multichannel signal is represented either by an encoded frame of the linear prediction domain encoder or by an encoded frame of the frequency domain encoder, wherein the audio encoder further comprises a linear prediction domain decoder for decoding an encoded downmix signal output by the linear prediction domain core encoder to obtain an encoded and decoded downmix signal; and wherein the audio encoder further comprises a multichannel residual coder for calculating and encoding a multichannel residual signal using the encoded and decoded downmix signal, the multichannel residual signal representing an error between a decoded multichannel representation obtained using the first multichannel information and the multichannel signal before downmixing. a linear prediction domain encoder; a frequency domain encoder; a controller for switching between the linear prediction domain encoder and the frequency domain encoder, wherein the linear prediction domain encoder comprises a downmixer for downmixing the multichannel signal to obtain a downmix signal, a linear prediction domain core encoder for encoding the downmix signal and a first joint multichannel encoder for generating first multichannel information from the multichannel signal, wherein the frequency domain encoder comprises a second joint multichannel encoder for generating second multichannel information from the multichannel signal, wherein the second joint multichannel encoder is different from the first joint multichannel encoder, and wherein the controller is configured to perform the switching such that a portion of the multichannel signal is represented either by an encoded frame of the linear prediction domain encoder or by an encoded frame of the frequency domain encoder, wherein the audio encoder further comprises a linear prediction domain decoder for decoding an encoded downmix signal output by the linear prediction domain core encoder to obtain an encoded and decoded downmix signal; and wherein the audio encoder further comprises a multichannel residual coder for calculating and encoding a multichannel residual signal using the encoded and decoded downmix signal, the multichannel residual signal representing an error between a decoded multichannel representation obtained using the first multichannel information and the multichannel signal before downmixing.