An audio signal processing apparatus comprising: a sound source direction estimator configured to estimate and output a plurality of sound source directions based on a plurality of sound collection signals of a plurality of microphones (11-1 to 11-n); a beam forming means configured to receive the plurality of sound source directions output by the sound source direction estimator , form M sound collection beams using the plurality of sound collection signals of the plurality of microphones (11-1 to 11-n) according to the plurality of sound source directions, and output the M sound collection beams as audio signals of M channels, respectively; a selecting means configured to select a channel group of N channels from among the M channels based on a volume level of the audio signal of each of the M channels; and a gain controller configured to control a gain of the audio signal of each channel of the channel group selected by the selecting means , according to a volume level of the audio signal of each channel of the channel group, wherein M is greater than or equal to three, N is greater than or equal to two, and M is greater than N; wherein the sound source direction estimator is configured to estimate the plurality of sound source directions by calculating a cross correlation of the plurality of sound collection signals of the plurality of microphones (11-1 to 11-n), and wherein the selecting means is configured not to simultaneously select channels corresponding to sound collection beams in which sound collection directions thereof are adjacent to each other, among the M sound collection beams. a sound source direction estimator configured to estimate and output a plurality of sound source directions based on a plurality of sound collection signals of a plurality of microphones (11-1 to 11-n); a beam forming means configured to receive the plurality of sound source directions output by the sound source direction estimator , form M sound collection beams using the plurality of sound collection signals of the plurality of microphones (11-1 to 11-n) according to the plurality of sound source directions, and output the M sound collection beams as audio signals of M channels, respectively; a selecting means configured to select a channel group of N channels from among the M channels based on a volume level of the audio signal of each of the M channels; and a gain controller configured to control a gain of the audio signal of each channel of the channel group selected by the selecting means , according to a volume level of the audio signal of each channel of the channel group, wherein M is greater than or equal to three, N is greater than or equal to two, and M is greater than N; wherein the sound source direction estimator is configured to estimate the plurality of sound source directions by calculating a cross correlation of the plurality of sound collection signals of the plurality of microphones (11-1 to 11-n), and wherein the selecting means is configured not to simultaneously select channels corresponding to sound collection beams in which sound collection directions thereof are adjacent to each other, among the M sound collection beams. <EOS>