Audio processor for processing an audio signal, the audio signal comprising a plurality of groups, each group comprising one or more audio elements, and metadata, the metadata indicating which group of the plurality of groups is to be used or is not to be used for determining a loudness compensation gain, the audio processor comprising: an audio signal modifier, wherein the audio signal modifier is configured to modify one or more groups of the plurality of groups of the audio signal in response to a user input to obtain a modified signal; a loudness controller, wherein the loudness controller is configured to determine the loudness compensation gain based, on the one hand, on a reference loudness (L ref) or a reference gain (g i) corresponding to an original audio scene and the metadata indicating which group of the plurality of groups is to be used or is not to be used for determining the loudness compensation gain, and, on the other hand, on a modified loudness (L mod) or a modified gain (h i), wherein the modified loudness (L mod) or the modified gain (h i) depends on the user input, wherein the metadata indicates that a first group of the plurality of groups is not to be used for determining the loudness compensation gain, and wherein the metadata indicates that a second group of the plurality of groups is to be used for determining the loudness compensation gain, and wherein the loudness controller is configured to exclude the first group in a calculation of the reference loudness (L ref) or the reference gain (g i), and to include the second group in the calculation of the reference loudness (L ref) or the reference gain (g i); and a loudness manipulator, wherein the loudness manipulator is configured to manipulate a loudness of the modified signal using the loudness compensation gain. an audio signal modifier, wherein the audio signal modifier is configured to modify one or more groups of the plurality of groups of the audio signal in response to a user input to obtain a modified signal; a loudness controller, wherein the loudness controller is configured to determine the loudness compensation gain based, on the one hand, on a reference loudness (L ref) or a reference gain (g i) corresponding to an original audio scene and the metadata indicating which group of the plurality of groups is to be used or is not to be used for determining the loudness compensation gain, and, on the other hand, on a modified loudness (L mod) or a modified gain (h i), wherein the modified loudness (L mod) or the modified gain (h i) depends on the user input, wherein the metadata indicates that a first group of the plurality of groups is not to be used for determining the loudness compensation gain, and wherein the metadata indicates that a second group of the plurality of groups is to be used for determining the loudness compensation gain, and wherein the loudness controller is configured to exclude the first group in a calculation of the reference loudness (L ref) or the reference gain (g i), and to include the second group in the calculation of the reference loudness (L ref) or the reference gain (g i); and a loudness manipulator, wherein the loudness manipulator is configured to manipulate a loudness of the modified signal using the loudness compensation gain.