An audio signal processor (100;150; 250) for providing ambient signal channels (112a-112c; 162a-162c; 254a-254c; 352; Â p) on the basis of an input audio signal (110;160; 210;310;x(t), x (t), X (m,k)), wherein the audio signal processor is configured to obtain the ambient signal channels and direct signal channels, wherein the ambient signal channels and the direct signal channels are associated with a same set of directions, wherein a number of obtained ambient signal channels comprising different audio content is larger than a number of channels of the input audio signal; wherein the audio signal processor is configured to obtain a direct signal on the basis of the input audio signal; wherein the audio signal processor is configured to obtain an ambient signal on the basis of the input audio signal; wherein the direct signal comprises direct signal components and the ambient signal comprises ambient signal components; wherein the audio signal processor is configured to obtain the ambient signal channels such that ambient signal components are distributed among the ambient signal channels in dependence on positions or directions of sound sources within the input audio signal; wherein the audio signal processor is configured to use spectral weights or panning coefficients, in order to distribute the ambient signal components to the ambient signal channels and the direct signal components to direct signal channels; and wherein the audio signal processor is configured to use the same spectral weights or panning coefficients (332; G p) for distributing the direct signal components (226; D̂;322) of the direct signal to the direct signal channels (252a-252c;342; D̂ p) and for distributing the ambient signal components (230; Â;324) of the ambient signal to the ambient signal channels (112a-112c; 254a-254c; 352; Â p). wherein the audio signal processor is configured to obtain the ambient signal channels and direct signal channels, wherein the ambient signal channels and the direct signal channels are associated with a same set of directions, wherein a number of obtained ambient signal channels comprising different audio content is larger than a number of channels of the input audio signal; wherein the audio signal processor is configured to obtain a direct signal on the basis of the input audio signal; wherein the audio signal processor is configured to obtain an ambient signal on the basis of the input audio signal; wherein the direct signal comprises direct signal components and the ambient signal comprises ambient signal components; wherein the audio signal processor is configured to obtain the ambient signal channels such that ambient signal components are distributed among the ambient signal channels in dependence on positions or directions of sound sources within the input audio signal; wherein the audio signal processor is configured to use spectral weights or panning coefficients, in order to distribute the ambient signal components to the ambient signal channels and the direct signal components to direct signal channels; and wherein the audio signal processor is configured to use the same spectral weights or panning coefficients (332; G p) for distributing the direct signal components (226; D̂;322) of the direct signal to the direct signal channels (252a-252c;342; D̂ p) and for distributing the ambient signal components (230; Â;324) of the ambient signal to the ambient signal channels (112a-112c; 254a-254c; 352; Â p).