A method for adaptive control of gain applied to an audio signal, including the steps of: (a) analyzing segments of the signal to identify audio objects indicated by said signal, including by identifying an audio object indicated by each of the segments; (b) storing information regarding each distinct audio object identified in step (a), including an identification of the object and a measure of level of each of at least one segment of the signal indicative of the object; (c) using at least a subset of the information stored in step (b) to determine, independently for each distinct audio object identified in step (a), target gain and a gain change rate for reaching the target gain for the audio object; and (d) applying gain to the segments of the signal indicative of one audio object identified in step (a), such that said gain changes, at the gain change rate for said object, from an initial gain to the target gain for the object, wherein at least some of the audio objects are voices of participants in a voice conference, and the information stored in step (b) includes a data structure regarding each distinct audio object identified in step (a), said data structure including values indicative of: a degree of confidence with which the object is classified as a voice of a conference participant, a degree of confidence with which the object is classified as a nuisance; a direction measure, a distance measure, and a percentage of time during the conference in which the object has been active. (a) analyzing segments of the signal to identify audio objects indicated by said signal, including by identifying an audio object indicated by each of the segments; (b) storing information regarding each distinct audio object identified in step (a), including an identification of the object and a measure of level of each of at least one segment of the signal indicative of the object; (c) using at least a subset of the information stored in step (b) to determine, independently for each distinct audio object identified in step (a), target gain and a gain change rate for reaching the target gain for the audio object; and (d) applying gain to the segments of the signal indicative of one audio object identified in step (a), such that said gain changes, at the gain change rate for said object, from an initial gain to the target gain for the object, wherein at least some of the audio objects are voices of participants in a voice conference, and the information stored in step (b) includes a data structure regarding each distinct audio object identified in step (a), said data structure including values indicative of: a degree of confidence with which the object is classified as a voice of a conference participant, a degree of confidence with which the object is classified as a nuisance; a direction measure, a distance measure, and a percentage of time during the conference in which the object has been active. a degree of confidence with which the object is classified as a voice of a conference participant, a degree of confidence with which the object is classified as a nuisance; a direction measure, a distance measure, and a percentage of time during the conference in which the object has been active.