BNMusic: Blending Environmental Noises into Personalized Music
Authors: Chi Zuo
Martin B. Møller
Pablo Martínez-Nuevo
Huayang Huang
Yu Wu
Ye Zhu
Venue: NeurIPS 2025 poster ()
Keywords: Music generation
Image inpainting
Primary area: deep_learning

Abstract: While being disturbed by environmental noises, the acoustic masking technique is a conventional way to reduce the annoyance in audio engineering that seeks to cover up the noises with other dominant yet less intrusive sounds. However, misalignment between the dominant sound and the noise—such as mismatched downbeats—often requires an excessive volume increase to achieve effective masking. Motivated by recent advances in cross-modal generation, in this work, we introduce an alternative method to acoustic masking, aiming to reduce the noticeability of environmental noises by blending them into personalized music generated based on user-provided text prompts. Following the paradigm of music generation using mel-spectrogram representations, we propose a Blending Noises into Personalized Music (BNMusic) framework with two key stages. The first stage synthesizes a complete piece of music in a mel-spectrogram representation that encapsulates the musical essence of the noise. In the second stage, we adaptively amplifying the generated music segment to further reduce noise perception and enhance the blending effectiveness, while preserving auditory quality. Our experiments with comprehensive evaluations on MusicBench, EPIC-SOUNDS, and ESC-50 demonstrate the effectiveness of our framework, highlighting the ability to blend environmental noise with rhythmically aligned, adaptively amplified, and enjoyable music segments, minimizing the noticeability of the noise, thereby improving overall acoustic experiences.  Project page: https://d-fas.github.io/BNMusic_page/.

BNMusic: Blending Environmental Noises into Personalized Music
Chi Zuo', Martin B. Moller?, Pablo Martinez-Nuevo', Huayang Huang', Yu Wu² Ye Zhu
Schooo of Computer Science, Wuhan University, China
Bang & Olufsen A/S, Denmark
3 DDepartment of Computer Science, Princeton University, USA
LIX, École Polytechnique, IP Paris, France (zuoc97, wuyucs, byyhuang)@whu edu. cn
{mim, (mim,pmn) poann olufsen dk ye ye.zhu@polytechhiigg edu
Abstract
While being disturbed by environmental noises, the acoustic masking technique is conventional way to reduce the annoyance in audio engineering that seeks to cover up the noises with other dominant yet less intrusive sounds. However, misalignment between the dominant sound and the noise- such as mismatched downbeats- often requires excessive volume increase to achieve effective masking. Motivated by recent advances cross-modal generation, in this work, we introduce an alternative method to acoustic masking, aiming to reduce the noticeability of environmental noises by blending them into personalized music generated based on user-provided text prompts. Following the paradigm of music generation using mel-spectrogram representations, we propose a Blending Noises into Personalized Music (BNMusic) framework with two key stages. The first stage synthesizes complete piece of music a mel-spectrooram representation that encapsulates the musical essence the noise. In the second stage, we adaptively amplify the generated music segment to further reduce noise perception and enhance the blending effectiveness, while preserving auditory quality. Our experiments with comprehensive evaluations MusicBench, EPIC-SOUNDS, and ESC-50 demonstrate the effectiveness of our framework, highlighting the ability blend environmental noise with rhythmically aligned, adaptively amplified, and enjoyable music segments, minimizing the noticeability of the noise, thereby improving overall acoustic experiences. Project page: https: fas fts.gghhfa.tttuuttt github. 10 3o/BilMusccppage/
Introduction
In public environments like subway trains, passengers are often exposed to persistent and irritating noise. While active noise cancellation (ANC) [12 is effective in personal audio devices, its individual- oriented nature limits its practicality in group settings. Equipping every passenger with ANC headphones is unrealistic, and such systems often struggle with high-frequency noise. To address this, we propose new task: rather than eliminating noise for individuals through destructive interference techniques, we aim to blend the environmental noise with correctly designed music in way that reduces its perceptual salience for a group of listeners, as shown Fig (a). This perceptual blending shifts the goal from directly suppressing noise to reducing its impact through harmonious audio masking, enabling a scalable auditory enhancement in shared environments, providing a more comfortable auditory experience without requiring personal devices. Beyond public transportation,
"Corresponinng author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Auditory thresholds with maskers
Noise
Music
Noise Cancelling
Noise Blending
Amplitude
I
Auditory thresholds without maskers
100
200
500
1k 2k 48
8k
Frequency (Hz)
(a)
(b)
Figure 1: (a): Noise cancelling is designed for individual use, requiring proximity to the user, while noise blending aims to reduce noise annoyance for everyone in the room by seamlessly blending a complementary sound with the surrounding noise. (b): This figure demonstrates the principle of auditory masking in psychoacoustics (28). The yellow dashed line indicates the threshold in quiet, ie., the baseline level below which sounds are inaudible in the absence of other stimuli. When a music signal is introduced, it elevates the masking threshold in frequency-deepedden manner, shown by the yellow solid line. As a result, any concurrent noise components that fall below this elevated threshold become imperceptible the listener, effectively masked by the presence of the music.
this blending-basee approach can also be valuable in a variety of other noise-prone environments. For instance, elevators often produce repetitive mechanical sounds during operation. Similarly, household appliances such as washing machines automatic garage doors generate rhythmic, ongoing noise. In such settings, masking the noise with well-aligned music would help improve user comfort and overall auditory experience.
Our proposed task draws inspiration from the theory of auditory masking 7 34 28 a psychoacoustic phenomenon where the perception of one sound is reduced or eliminated by the presence of another sound. This effect is typically modeled in the frequency domain using psychoacoustic principles, which define masking threshold- the minimum intensity below which sound becomes inaudible to the human ear in the presence of a masker [5]. When a new sound is introduced, it elevates the masking threshold making weaker signals in those regions imperceptible
only at its own frequency but also across neighboring frequency bands,
[2 As shown in Fig. (b), such thresholds
are fundamental to many perceptual audio models, and here we use them to guide the blending of
generated music with environmental noise. This task aims to diminish the perception of unwanted noise by introducing supplementary background musical sounds. complex environmental noise within a comfortable loudness range is rarely achievable. Instead, we aim for a more feasible and perceptually effective solution through blending. By generating music that is rhythmically aligned with the underlying noise, our approach enables partial masking while incorporating the residual noise components into the musical texture, thus reducing perceived annoyance without overwhelming the listener.
practice, fully masking
To effectively blend generated music with environmental noise, it is essential that the music can perceptually mask the noise, thereby reducing its distracting impact. Achieving such blending requires that the masking effect remain strong even at relatively low overall loudness levels. To this end, the generated music should align closely with the noise in terms of rhythm or other structural properties, enabling it to integrate naturally without relying on excessive loudness. However, while recent advances in music generation, particularly those based on mel-spectrogram representations in 40, 43 I7]. have demonstrated remarkable progress, most existing models are trained to generate music from clean, structured inputs such as text prompts or orderly music excerpts. These models struggle when conditioned on noisy and unstructured inpuls, as they lack the ability to retain relevant acoustic attributes from chaotic signals. Yet they also provide a natural foundation for leveraging masking principles, which are inherently sigueency-depeneenn Building this emerging paradigm, we explore generating music in the frequency domain whose
the frequency domain 35 T9 14
35
39

---

structure, rhythmic patterns, and statistical properties naturally align with those of the noise, enabling seamless auditory blending, thus in turn diminishes the listeners' awareness of the underlying noise.
In this paper, we propose a novel approach Blending Noises into Personalized Music (BNMusic) that uses adaptive loudness-amplified music to blend with background noise. This approach makes the noise less noticeable while maintaining a balanced overall volume, thereby enhancing the acoustic environment effectively. Our method is designed as a two-stage process that explicitly targets the high-energy regions of noise, which are the most perceptually salient and thus the hardest to mask. In the first stage, we condition on these regions and apply a two-step outpainting -inpanntinng process on the noise meeespectro to generate music that rhythmically and spectrally aligns with the noise. Outpainting extends musical patterns around the dominant noise zones, while inpainting reconstructs coherent content within them, ensuring that the resulting composition inherits the noise's ' most prominent frequency structures in a natural way. This design effectively transforms disruptive noise components into elements that can be musically integrated. In the second stage, we apply adaptive amplification targeted to these same regions. Because the generated music already shares temporal-spectral characteristics with the noise, only modest gain is required to strengthen the masking effect, avoiding excessive loudness while maintaining musical balance. Together, the two stages form tightly coupled system: Stage I embeds the noise into the music in perceptually aligned manner, and Stage 2 consolidates this aligmment by enhancing masking where it matters most. As result, BNMusic achieves robust blending that suppresses the perceptual salience of noise while preserving coherence and listening comfort.
To assess the efficacy of our approach in blending with environmental noise and creating a more pleasant auditory experience, we conduct comprehensive objective and subjective evaluations. We use EPIC-SOUNDS I0] and ESC-50[[]] as noise sources, which together cover wide range of real-world environmental sounds, including various actions, objects, and acoustic scenes, spanning a broad frequency range from 100 Hz to 10,000 Hz. Results show that our complete method achieves the best performance on MusicBench [19].
In summary, we present three major contributions:
We introduce novel multimodal generation task, namely the noise blending with mu- sic, whose primary objective is to reduce the perceptibility of environmental noise with personalized generated music compositions based on user-providential prompts.
We propose BNMusic construct music that integrates musical elements from the noise using a two-step outpainting and inpainting process and then utilize the auditory masking effect to adaptively amplify the generated music segment to blend with ambient noise and minimize its noticeability, creating a more pleasant acoustic environment.
Extensive experiments demonstrated our method effectively generates music that seamlessly blends with environmental noise, minimizing its perception at applicable volume levels.
2 Related work
2.1 Conventional acoustic methods
The conventional acoustic methods of enhancing hearing environments have evolved significantly throughout these years, driven by the demand for better auditory experiences. Early efforts focused on employing physical barriers and materials to block out noise. The introduction of active noise canceling (ANC) 12 marked major breakthrough, which utilizes microphones and electronic circuitry to produce anti-phase sound waves that counteract noise (36]. Researchers have also studied auditory masking for years 34 which describes how louder sound can reduce or eliminate the perception of a quieter one occurring simultancously. Building psychoacoustic models to simulate the human perception of audio (28 24 22] and quantifying how much louder masker needs to be at different frequencies to effectively mask a target sound. Studies have applied auditory masking principles reduce the perceptual impact of noise, designing comfortable soundscapes using chord progressions and melodies that align with the peak frequencies of disruptive noises (3 and investigating the use of natural sounds to mask traffic noise in urban environments [42]. Or utilizing audio masking for audio watermarking [38] Inspired by these masking- masking-based based techniques, we propose the noise blending task that uses the auditory masking effect to reduce the perceptual impact of noise acoustic environments.
3

---

2.2 Generative models for music
Generative models have recently achieved remarkable success in both vision and audio synthe- sis 6 13 35 19
image generation, diffusion models [ 32 have proven especially effective by gradually denoising in the latent space of pre-trained network, leveraging its structure to produce high-quality outputs. Inspired by this success, diffusion- baassd techniques have also been extended to text-to-Music tasks [13 35 19 44), significantly improving both generation quality and efficiency. Contemporary audio generation models generally fall into two categories: those that generate audio directly in the waveform domain, and those that first generate time-frequency repre- sentations (e.g.. mel-sppecrogramm)) and convert them to waveforms using a vocoder. MusicGen B] exemplifies the former, synthesizing music through multiple streams of discrete audio tokens. In contrast, Riffusion 6 follows the latter approach, generating mel-spectrograms via a fine-tuned Stable Diffusion 32 model and converting them into audio. AudioLDM [3] builds on latent-space modeling and CLAP (Contrastive I Language-Audio Pretraining) to generate audio from text. Mus- tango [9] introduces MuNet for fine-grained control, achieving competitive performance even with limited training data. AudioLDM2 [14] proposes a unified "language of audio" (LOA) representation, using GPT-2 bridge multiple modalities and guiding generation with a latent diffusion model. Our work builds on these foundations, aiming to generate music that blends harmoniously with environmental noise using diffusion-based methods. Meanwhile, complementary efforts have focused which may enhance the practical
on accelerating generation for ral-timme applications (26 33 25 2
deployment of our method in future iterations.
2.3
Generation controlling techniques
Recent years have witnessed significant progress in audio editing, with a growing number of works exploring the potential of generative models in transforming and manipulating sound [39 20 I [8
40 43 17 Many approaches adopt a vision-inspired paradigm by converting audio into 2D repre- sentations such as mel-spnetroograms, enabling the application of powerful image editing techniques 44) 43 17]. Following the advances in image generative models like Stable Diffusion (32), audio editing has expanded include tasks such spectrogram inpainting, style transfer, and attribute control. For instance, Audit (39] applied latent diffusion models (LDMs) to edit audio in a controllable manner. Prior audio inpainting methods (20
to audio 6 35| 19
14, 39.
M T8 typically focus on interpolating the gap between two audio clips. In contrast, more recent works [17
[5
leverage spectrogram-bast based image diffusion manipulate high-level features such as genre, in- strument, or mood. Inspired by inpainting tasks in vision and their success in image composition and context-aware filling [6 41]. we propose a new application scenario in which spectrogram inpainting and outpainting are used to mask environmental noise through musical blending. Built upon Riffusion (6) our method adapts image-style spectrogram generation synthesize rhythmically aligned, stylistically coherent music that perceptually reduces the annoyance of background noise. Unlike traditional gap-filling methods, our approach treats the noise-occupied spectrogram as a canvas to be expanded and enhanced, thus offering novel direction in content-aware audio editing.
BNMusic framework: Blending Noises into personalized Music
In this section, we present the details of our BNMusic framework. designed to blend noise ANoise with adaptive amplified music AMusic generated from ANaise and text condition CLexa Our method extends the existing model's application without additional training.
Problem statement. We formalize the noise blending task as an alternative to traditional masking methods, which often require excessive volume to reduce the annoyance background noise. Given repeating noise segment ANoise our goal is to generate a music segment AMusic conditioned on both ANoise and a user- user-provided prompt CLest. When played alongside the ambient noise, AMusic is expected to effectively reduce the noticeability of some major parts of the noise, making the remaining noticeable content less irritating or even being recognized as part of the music, and consequently, enhancing the overall auditory experience. To achieve this, we propose the BNMusic framework. As shown in Fig. 2 the masked noise reveals a regular rhythm or pulse, allowing it to align with the generated music, especially when processed through our two-stage BNMusic method.

---

EEerggtti Rock'
A Noise
XNoise
Noise
Outpainting
XtNid
STFT Mel fiteering
Masked
Noise
Music
Generator
Subtrack
Calculate
T Mask
Energetic Rock'
A Music
Adaptive
X Music
Inpainting
X
S' Music
S Music
Enhance
Music Generator
A Audio signal
Stage
Noise-Aligned Music Synthesis
S Mel-Sppetrrogam
Stage 2: Blending Enhancing by Adaptive Amplification
X Image
Figure 2: Overall pipeline of our proposed BNMusic framework to achieve noise blending with frozen music generators. The two stages of our approach are marked with different background colors. In Stage 1, our approach generates music that aligns with the noise, and in Stage 2 we adaptive amplify the music signal to reach the most ideal and reasonable blending with the noise.
Pre-processing. As the input noise ANoiee E RR where is the length of audio seconds and fs is the sampling rate (i.e., number of samples per second), is given the system, we convert from an audio signal into mel-spectrogram SNoise = McI]STFT(AANsss
E IR XH where Mel stands for the Mel-filterngg process and STFT means Short Time Fourier Transform. Through this conversion, the input one-dimensional signal ANoise can be represented in two-dimensional matrix, namely the -pl-spectro Nosse- Once the mel-spectrogram obtained, its amplitude values are mapped to grayscale pixel intensities in the range [0, 255]. In this mapping, lower pixel values represent louder regions, while higher values indicate quieter ones. This process yields detailed grayscale mel-spectrogram plot, denoted as XNoiee E [0, 255] W x HH11, where the last dimension represents single grayscale channel. With such more perceptually relevant representation of the signal, the evolution of noise frequency content over time becomes easier to analyze and extract. After obtaining
Nooise then apply binary mask M {0, W> that masks out the area representing the higher energy part of the noise. Since frequencies with higher local energy are typically more perceptually salient and more likely interfere with auditory perception, isolating these regions is erucial for subsequent blending. The masked image can be calculated as X XNaise @ M E where represents element-wisg wise multiplication.
[0, 255) B x1
Stage 1: Noise-aligned music synthesis. We adopt a two-step outpainting and inpainting process
guide the generation of music around and within the high-energy regions of the noise spectrogram. This design enables the model to capture and preserve the noise's intrinsic rhythmic cues while diffusing them into coherent musical structure, thereby ensuring that the generated music naturally aligns with the most perceptually salient parts of the noise. First, the mask M isolates the core noise region in the image XNoise. dividing into two parts. During the outpainting stage, the core noise area is preserved, and music is generated to fill the surrounding space, allowing the core information to diffuse outward. The masked mel-spectrogram plot XNoise and the prompt Cres are encoded into latent representations, then fed into the LDM 32 from Riffusion 6 for music generation, which is a modified version of Stable Diffusion-vll fine-tuned for generating music's 's mel- spectrrrro plot. Given a noisy latent representation z, at timestep t, the model predicts the added noise using U-Net conditioned on both the corrupted -pectrrcto XNoise and the associated text prompt Ciest Using the predicted noise ep, the posterior distribution of the previous latent state is computed as:
p(z Ze: XNoise, Cro)) - (z--1: plzz, Ee(Ze, t, XNoise, Cwst)) of1
(I)
5

---

Here, is an analytically derived function that depends on ž₁. Eœ represents the predicted noise, t denotes the diffusion step, and of is determined by a fixed noise schedule. The reverse process proceeds iteratively until / 0. yielding the final latent 20. This latent representation is then passed through the decoder to reconstruct the mel-spectrogram, including the previously masked region XMid (-
The outpainted region of the XMid aligns the rhythmic patterns with the remaining noise content, leading us to invert the mask M to let the model inpaint on the area that represents the higher energy components of the noise:
XMid 3 M1- E [o. 255 Wxx
XMid
After second round of inpainting on the inversely masked area Xvιa. we obtain an image Xussie an image representing the complete musical content. This iterative process extracts rhythmic patterns from the noise and integrates them into the music, replacing noisy elements while eliminating the most distracting parts. The second round of inpainting is crucial for refining the composition, ensuring that the diffused information is reintegrated into the core, ultimately reconstructing piece of music that not only preserves the rhythmic characteristics of the original noise but also aligns with its dominant perceptual features, thereby laying solid foundation for effective blending in the subsequent stage.
Stage 2: Blending enhancing by adaptive amplification. To further enhance blending, we adap- tively applies minimal loudness amplification, exploiting the music's alignment with the noise's ' high-energy regions to achieve effective auditory masking with modest gain, thereby reducing percep- tual intrusiveness without disrupting the overall acoustic balance. To calculate the masking threshold, we first compute the noise's ' magnitude spectrogram using the Short-Time Fourier Transform (STFT), which extracts the magnitude of the complex STFT coefficients. The resulting matrix, SNaise is
real-valued and belongs to R x where W is the number of frequency bins and H is the number of time frames. Based on previous research (28) the minimum signal-to-mast ratio (SMR) for Tune- Saee-Masking-Niss cases is typically between 21 dB. Using the minimum value of 21 dB, we derive the threshold matrix as:
T Mask Mel|10 20- (f
R WxH
These thresholds Task indicate that any sound exceeding these values will trigger auditory masking for the corresponding frequency and time. This allows us to determine when and how sounds can mask the noise. We then use gradient descent to find an optimal amplification factor X, ensuring maximal auditory masking while keeping the total loudness of the music within a reasonable range. The amplified music signal can be represented as S Music = S x E R R H where denotes scalar multiplication. Therefore we put up with an optimization function:
X arg min {SUM(a ) + SUM(max((TMr)) Mask
Suussee @ M,
(4)
The formula optimizes the parameter to minimize the objective function, finding the optimal solution X SUM(a Musssie) represents the sum weighted music signal, where a controls the weight of the music signal in the optimization. SUM(max[(T) Mask S @ M, OJ) is the sum of the noise masking term. This formula ensures amplification occurs only when the masking effect on the core area outweighs the global mel- -pecttrrcto increase, maintaining balance between enhancing core masking and avoiding unnecessary global amplification.
The amplified mel-spectrogram Muusic represents the final music output seek to use for masking the noise. We transform it back into an audio signal AMusic using the following process:
(5) where Griffin-Lim (30 used to estimate and recover the phase information of the signal before performing the ISTFT (Inverse Short-Time Fourier Transform). Mel represents the inverse Mcl-
AMusi = ISTFT (Griffin-LimMMet (S E R
filtering operation. This approach reconstructs the audio signal from the generated mel-spectrogram.
Experiment
4.1 Experiment setup
Dataset. Our dataset consists of three components: noise clips, real music clips, and text prompts. To ensure diverse noise conditions, source 1,000 segments from the EPIC-SOUNDS dataset (10).

---

covering 58 human action categories and 140 object types, and 300 additional segments from the ESC-50 dataset [3T, which includes 50 categories of real-world sounds such as thunderstorms, sea waves, and chirping birds. Together, these samples span wide range of everyday acoustic environments and broad frequency spectrum from 100 Hz to 10,000 Hz. For music data, we use 5,000 five-second clips from the MusicBench dataset 19], derived from 3,413 high-quality tracks across various genres, styles, and instrumentation. These serve as both ground truth and baselines for evaluation. Additionally, we construct a prompt set of 100 text descriptions across seven music genres via LLMS (27], covering Pop, EDM, Rock, Hip-hop, Punk, Jazz, and Classical. By pairing each noise clip with multiple prompts, generate 14,200 music clips using two controllable generation models: Riffusion 66] and MusicGen [3]. Specifically, we pair each of the 1,000 EPIC-SOUNDS noise clips with five different prompts, and each of the 300 ESC-50 clips with seven prompts, resulting in 5,000 and 2,0 generated music pieces, respectively.
Baselines. We compare our results with three baseline methods. The first two use Riffusion's 's audio- to-audio generation (6] and MusicGen's 's melody-conditionns generation B both generated with the same noise and text prompt pair used in our method. The third baseline involves randomly selected real music from MusicBench [9]. All generated music clips are overlaid with their corresponding noise segments to simulate the actual auditory experience perceived by users. This enables both objective and subjective evaluation of how well the music blends with environmental noise in realistic listening conditions.
Implementation details. Since loudness plays critical role in masking perception, our approach aims to blend noise into music without relying on excessive volume, necessitating loudness normaliza- tion to ensure a pleasant and balanced auditory experience. To address this, we apply Pyln-norm RY), which uses the ITU-R 1770- model for loudness normalization, ensuring all audio clips are adjusted appropriately. The noise is consistently normalized to -18 dB LUFS in all evaluations. The Riffusion model [6] was used with default settings to ensure compatibility, and each sample was processed in approximately 5 seconds
Nvidia 4090 GPU. The entire process takes approx- imately 0.28 seconds for preprocessing and amplification, with the majority of time spent on the two-stage generation, while system- induced delay remains minimal. The overall music signal control parameter, a, was set to O. 14 to ensure adaptive amplification remained within a reasonable range. For evaluation, we compare our approach by overlaying music and noise clips simulate real-world scenarios. In the real music baseline, half of the clips were real music paired with noise, and the other half served as ground truth for evaluation.
4.2 Evaluation
Objective evaluation. We use Fréchet Audio Distance (FAD) and Kullback-Leible (KL) divergence as our primary objective metrics. FAD measures how closely generated audio matches reference audio in terms of statistical properties, while KL divergence compares the probability distributions of generated and reference clips. Lower values of both metrics indicate greater similarity. FAD is computed over feature distributions across batches, while KL divergence calculated pairwise between generated and reference audio clips. In our experiment, FAD and KL scores were calculated by comparing the combined noise and music audio to the real music ground truth. We also evaluate these metrics on both direct outputs and those normalized to match the noise's ' loudness. The objective evaluation results in Tab. indicate that our method achieves the best FAD and KL scores across both scenarios, demonstrating effective blending with environmental noise, even when music and noise are presented at equal loudness. The low scores indicate that the combined audio is statistically and perceptually similar to real music, suggesting that distracting noise components are more effectively masked. This supports the effectiveness of our approach in enhancing the auditory experience by rhythmically and structurally aligning music with noise.
Subjective evaluation. Blending noise with generated music to enhance harmony is inherently subjective, as perceptions of auditory harmony can vary between individuals. To evaluate our approach, we conduct human evaluations with 50 samples, each containing five audio clips: the original noise, our result, and three adaptive amplified baselines. To closely approximate real-world conditions, all music clips are mixed with the corresponding noise prior to playback. Adaptive amplification is applied to both our music and the baselines for fair comparison. Testers first listen to the original noise alone, and then evaluate the mixed samples (i.e., our result and three adaptive amplified baselines), then score each on OVL (overall quality) and PER (perceived noise level) using
7

---

Table 1: Objective evaluation results. This table reports the Fréchet Audio Distance (FAD) and Kullback-Leibee (KI.) Divergence scores tested on two scenarios Loudness Normalized (left) and Direct Outputs (right)-on the EPIC-SOUNDS [0] and ESC-50 [31] datasets. As shown, our BNMusic method consistently achieves the best performance, highlighting the robustmess and generalizability of our approach across different acoustic settings and datasets.
Loudness Normalized
Direct Outputs
Methods
EPIC-SOUNDS
TO
ESC-50
31
EPIC-SOUNDS
10
ESC-50
31
FAD4
KI.4
FAD4
K1.
FAD4
KI,
FAD4
KT.
Noise Only
34.17
27.39
34.17
27.39
Random Music
14.22
2.22
8.45
2,49
15.41
2.38
8.32
2.61
MusicGen 3
13.28
2.14
8.62
2.43
10.95
1.85
7.74
2.33
Riff A2A 6
20.06
2.90
12.62
3.26
13.15
2.25
9.11
2.70
BNMusic (Ours)
12.86
2.03
8.09
2.38
7.98
1.67
6.76
2.14
Subjective and objective evaluation on adaptively amplified samples. We report subjective scores for overall quality (OVL) and perceived noise level (PER), along with objective metrics: Fréchet Audio Distance (FAD) and Kullback-Lebler Divergence (KL) evaluated on adaptive amplified audio samples. BNMusic achieves the highest OVL and PER scores and the lowest KL among all comparing methods, demonstrating its effectiveness in blending with noise and reducing the perceptual impact of background interference.
Table 2:
Subjective Metrics
Ohjective Metries
Methods
OVL
PER
FAD,
KI.
Random Music
2.93 + 0.58
2.63 + 0.53
6.84
2.07
MusicGen 3
2.97 + 0.34
2.68 + 0.54
7.08
1.75
Riff A2A 6
2.95 1 0.60
3.24 + 0.67
12.82
2.33
BNMusic (Ours)
3.67 + 0.55
3.84 + 0.63
7.98
1.67
to 5 Likert scale, where indicates the most pleasant experience the least perceived noise. This evaluation provides valuable insights into user perceptions of our method's effectiveness. More details and sample of the user questionnaire included in the Appendix C As shown in Tab. 2 subjective evaluation results indicate that most users find our BNMusic segments provide the best hearing experience alongside environmental noise, outperforming all other baselines. Riffusion's Audio-to-All Audio-to-Audio Audio approach [6] ranks second in noise suppression but compromised musicality, as its outputs closely mimic the noise structure. MusicGen [3] and Real Music achieve lower PER scores, with MusicGen offering marginal noise masking through melody-aware generation, but still falling behind BNMusic and Riffusion.
Visualization-baase comparison. To further illustrate the blending behavior, Fig. presents several representative examples. Each group consists of five plots: the mel-spectrogram of noise sample, followed by four heatmaps showing the difference berween that noise and four types of music: Random Music, MusicGen, Riffusion-A2A, and our BNMusic. All music samples normalized to match the noise before computing the difference to ensure a fair comparison. The heatmaps visualize the energy difference between the music and the noise. Red indicates positive differences, blue indicates negative, and darker colors represent larger magnitudes. These maps reveal how closely each music sample aligns with the noise in terms spectral energy distribution. Among the four, Random Music shows the largest mismatch with the noise, especially in less active frequency bands. MusicGen also differs notably, but lesser extent. In contrast, Riffusion-A22 and our BNMusic demonstrate much closer alignment to the noise across the frequency spectrum. Their differences are more evenly distributed and less extreme, indicating better spectral blending. This suggests that A2A and BNMusic are more effective in matching the energy profile of the noise, which may underlie their superior auditory integration. However, both FAD and subjective evaluation results confirm that our BNMusic significantly outperforms A2A in terms of pleasantness and harmony.
loudness-

---

(Music Noise) Pixel Differences Heatmaps
Nolse
Random Music FAD: 14.22
MusicGen FAD: 13.28
Riffusion-A2A
BNMusic
FAD: 20.06
FAD: 12.86
Noise Type:
Washing
Prampt Type:
Pop
Noise Type:
Slicing
Prompt Type:
Jazz
Noise Type:
Mixing
Prompt Type:
Rock
Figure 3: Visualization of moise-music blending effectiveness across methods. The left images display the mel-spectrograms three types of noise, while the right heatmaps show the differences between the generated music and the noise. The heatmaps illustrate how four music samples blend with the respective noise. Red represents positive values, blue indicates negative values and darker colors correspond to larger magnitudes, highlighting the blending effectiveness of each music type.
Table 3: Ablation study of method components. We evaluate different combinations of method components, and report Fréchet Audio Distance (FAD) and KI. Divergence (KL).
Method Components
Metrics
Outpainting
Inpainting
Adaptive Amplification
FAD4
KL
34.17
8.68
1.89
9.18
1.84
X
8.00
1.78
7.98
1.67
Ablation study. To further evaluate the contribution of each component in framework, conduct an ablation study by systematically disabling parts of our pipeline. Our method consists of three key components: Outpainting, Inpainting Stage and Adaptive Amplification
Stage 2. We test variants where one or more of these components were removed. As shown in Tab. 3 each component contributes the final performance, and the full system, which combines outpainting, inpainting, and adaptive amplification achieves the best objective results. This highlights the importance of all three stages in enabling effective noise-awart coire-aware music blending, As the results shown in Tab, [2] MusicGen performs better in FAD due to amplification making the music more prominent. However, Random Music performs poorly in KI. divergence as amplification fails balance noise-music interaction. Our method, with pairwise adaptive amplification, aligns music and noise effectively and achieves better blending and a more pleasant auditory experience. This is consistent with the user study results, Random Music and MusicGen (3], despite higher music quality, still fail in blending effectively with the noise and making it less annoying. In contrast, our method ensures seamless blending and provides better auditory experience.
Conclusion and discussion
conclusion, our BNMusic demonstrates superior performance in blending music with environmental noise compared to other methods, effectively reducing the annoyance of the noise while enhancing

---

the overall auditory experience. Through series of experiments and ablation studies, we show the effectiveness of our approach, as well as the contribution of each key modeling component. By finding an optimal balance between maximizing the pleasantness of the music, controlling its loudness, and aligning it with the noise for more seamless blending, our method ensures that the combined sound provides the best listening environment.
Limitations. When users deliberately provide prompts that are poorly matched the noise the system struggles to generate a coherent blend, and the influence of the prompt becomes less pronounced. This highlights the need for prompt-noiss compatibility achieve effective blending. While mel- spectrogram representations help reduce inference costs by compressing audio, the conversion between time and frequency domains introduces distortion that slightly affects music quality. Real- time processing is currently limited by slow generation speed, making it infeasible at this stage. Our primary focus demonstrate the effectiveness of the proposed method. For repetitive noise scenarios, practical applications can still be developed through offline recording and post-processing. Looking ahead, integrating our approach with faster generation techniques may enable near real-time performance and broaden the range of potential applications.
Broader impacts. This work aims enhance auditory comfort in noisy environments by blending music with ambient noise in perceptually harmonious way. It has potential applications in public transport, offices, etc, offering more pleasant listening experience. However, overused or applied without user control, such blending systems might unintentionally mask important environmental sounds lead to listener fatigue over time.
10

---

Acknowledgments and Disclosure of Funding
This paper is co-supervised by Prof. Ye Zhu and Prof. Yu Wu. This work was partially supported by the National Natural Science Foundation of China under grant 62372341 to CZ, HH and YW. This research was also partially supported by an Amazon Research Award to YZ. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of Amazon. YZ also acknowledges the travel funding support by the French National Research Ageney (ANR) via the "GraspGNNs" JCJC grant (ANR-24-CE23-36) coordinated by Johannes F. Lutzeyer from École Polytechnique.
References
[1] Zalán Borsos, Matt Sharifi, and Marco Tagliasacchi. Speechpainter: Text-conditioned speech inpainting. arXiv preprint arXiv:2202.0777. 2022.
[2] Marco Comunità, 7hi Zhong, Akira Takahashi, Shiqi Yang. Mengjie Zhao, Koichi Saito, Yukara Ikemiya, Takashi Shibuya, Shusuke Takahashi, and Yuki Mitsufuji. Specmaskgit: Masked generative modeling of audio spectrograms for efficient audio synthesis and beyond. arXiv preprint arXiv: 006006177667 17672, 2024.
[3] Jade Copet, Felix Kreuk, Itai Gat, Tal Remez, David Kant, Gabriel Synnaeve, Yossi Adi, and Alexandre Défossez. Simple and controllable music generation. Advances in Newral Information Processing Systems, 36:47704 47720, 2023.
[4] Zach Evans, Julian D Parker, CJ Car,, 7ack Zukowski, Josiah Taylor, and Jordi Pons. Long-form music generation with latent diffusion. arXiv preprint arXiv:2404. 10301, 2024.
[5] Hugo Fastl and Eberhard Zwicker. Psychoacousticx: Facts and Models. Springer-Verlag. Berlin, Heidelberg, 2006.
[6] Seth* Forsgren and Hayk Martiros. Riffusion Stable diffusion for real-time music generation. 2022.
[7] Donald D Greenwood. Auditory masking and the critical band. The journal of the acoustical society of America, 33(4), 1961.
[8] Bing Han, Junyu Dai. Weituo Hao, Xinyan He, Dong Guo, Jitong Chen, Yuxuan Wang. Yanmin Qian, and Xuchen Song. Instructme: An instruction guided music edit and remix framework with latent diffusion models. Xrv, 2023.
[9] Jonathan Ho, Ajay Jain, and Pieter Abheel. Denoising diffusion probabilistic models. Advances in neural information procesing systems, 33:6840-6851 2020.
[10] Jaesung Huh, Jacob Chalk, Evangelos Kazakos, Dima Damen, and Andrew Zisserman. Epic-sounds: A large-scale dataset of actions that sound. IEEE Transactions Pattern Analysis and Machine Intelligence, 2025.
[11] Kevin Kilgour, Mauricio Zuluaga, Dominik Roblek, and Matthew Sharifi. Fri, 'echet audio distance: metric for evaluating music enhancement algorithms. arXiv preprint
2018
[12] Sen M Kuo and Dennis R Morgan. Active noise control: A tutorial review. Proceedings of the IEEE, 87 (6):943 973, 1999.
[13] Haohe Liu, Zehua Chen, Yi Yuan, Xinhao Mei, Xubo Liu, Danilo Mandic, Wenwu Wang, and Mark D Plumbley. Audioldm: Text-to-audio generation with latent diffusion models. arXiv preprint arXiv:230J. 12503, 2023.
[14] Haohe Liu, Yi Yuan, Xubo Liu, Xinhao Mei, Qiuqiang Kong, Qiao Tian, Yuping Wang, Wenwu Wang. Yuxuan Wang. and Mark D Plumbley. Audioldm 2: Learning holistic audio generation with self-supervised pretraining. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 32:2871- 2283, 2024.
[15] Shansong Liu, Atin Sakkcer Hussain, Qilong Wu, Chenshuo Sun, and Ying Shan. M uuugen: Multi- modal music undcrstanding and generation with the power of large language models. arXiv preprint urXiv:2311.11255 2023.
[16] Andreas Lugmayr, Martin Danelljan, Andres Romero, Fisher Yu, Radu Timofte, and Lue Van Gool. Repaint: Inpainting using denoising diffusion probabilistic models. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pages 11461-11471, 2022.
11

---

[17] Hila Manor and Tomer Michaeli. Zero-shot unsupervised and text-based audio editing using ddpm inversion. arXiv preprini wrXiv:2402.10009 2024.
[18] Andres Marafioti, Piotr Majdak, Nicki Holighaus, and Nathanaël Perraudin. Gacela: A generative adversarial context encoder for long audio inpainting of music. IEEE Journal of Selected Topics in Signal Processing, 150)1110--33,, 2020.
[19] Jan Melechovsky. Zixun Guo, Deepanway Ghosal, Navonil Majumder, Dorien Herremans, and Soujanya Poria. Mustango: Toward controllable text-to mox-tc generation. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 8293-8316, 2024.
[20] Ondřej Mokrý and Pavel Rajmic. Audio inpainting: Revisited and reweighted. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 28:2906- 18-2900-2918, 2020.
[21] Brian CJ Moore, An introduction the psychology of hearing. Brill, 2012.
[22] Brian CJ Moore. Psychoacoustics. Springer handbook of accossiicc 2014.
[23] Masato Nakayama, Takuya Hayashi, Toru Takahashi, and Takanobu Nishiura. Comfortable sound design based on auditory masking with chord progression and melody generation corresponding to the peak frequencies of dental treatment noises. Applied Sciences, 14(22):00677, 2024.
[24] Thibaud Necciari, Peter Balazs, Richard Moonland-Martingg Sølvi Ystad, Bernhard Laback, Sophie Savel, and Sabine Meunier. Auditory time- free-frey masking: psychoacoustical data and application to audio representations. In International Symposium on Computer Music Modeling and Retrieval, pages 171. Springer, 2011.
[25] Zachary Novack, Julian McAuley, Taylor Keng-Kirkpatrick, and Nicholas Bryan. Ditto: diffussio inference-time toptimizaaiinn for music generation. In Proceedings of the 41st International Conference Muchine Learning, pages 38426-38447, 2024.
[26] Zachary Novack, Ge Zhu, Jonah Casebeer, Julian McAuley, Taylor Berg- Kirkpaarktt and Nicholas J Bryan. Presto! distilling steps and layers for accelerating music generation. arXiv preprint arXi:22410.05167, 2024.
[27] OpenAL. Chatgpt: Language model for dialogue, 2024, Accessed: 2024-05-21.
[28] Ted Painter and Andreas Spanias. Perceptual coding digital audio. Proceedings of the IEEE, 88(4):
45 451-515, 2002.
[29] Julian Parker, Janne Spijkervet, Katerina Kosta, Furkan Yesiler. Boris Kuznetsov, Ju-Chiang Wang. Matl Avent, Jitong Chen, and Due Le. Stemgen: music generation model that listens. In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1116-1120. IEEE, 2024.
[30] Nathanaël Perraudin, Peter Balazs, and Peter L Søndergaard. A fast griffin-lim algorithm. In 2013 IEEE workshop on applications of signal processing to audio and acoustics, pages 1 1-4, IEEE, 2013.
[31] Karol J Piczak. Esc: Dataset for environmental sound classification. In Proceedings of the 23rd ACM international conference on Multimedia, pages 1015-1018, 2015.
[32] Robin Rombach, Andreas Blatmannn, Dominik Lorenz, Patrick Esser, and Björ Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684-10695, 2022.
[33] Koichi Saito, Dongjun Kim, Takashi Shibuya, Chieh-Hsin Lai, Zhi Zhong, Yuhtta Takida, and Yuki Mitsufuji. SoundCTM: Unifying score-based and consistency models for full-band text-to-sound generation. In The Thirleenth International Conference on Learning Representations. 2025.
[34] B Scharf. Fundamentals of auditory masking. Audiology, 10(1):30-40, 1971.
[35] Flavio Schneider, Ojasv Kamal, Zhijing Jin, and Bernhard Schölkopf. Mo!" usai: Text-to-music generation with long-contess latent diffusion. arXiv preprint arXiv:2301.1175, 2023.
[36] Ying Song, Yu Gong, and Sen M Kuo. A robust hybrid feedback active noise cancellation headset. IEEE transactions on speech and audio processing, 13(4). 2005.
[37] Christian J Steinmetz and Joshua Reiss. pyloudnorm: A simple yet flexible loudness meter in python. In Audio Engincering Society Convention 150. Audio Engineering Society, 2021.
12

---

[38] Mitchell D Swanson, Bin Zhu, Ahmed H Tewfik, and Laurence Boney. Robust audio watermarking using perceptual masking. Signal processing. 66(3):337-355, 1998.
[39] Yuancheng Wang, Zeqian Ju, Xu Tan, Lei He, Zhizheng Wu, Jiang Bian, et al. Audit: Audio editing by following instructions with latent diffusiin models. Advances in Neural Information Processing Systems, 36:71340-71357, 2023.
[40] Shih-Lun Wu, Chris Donahue, Shinji Watanabe, and Nicholas J Bryan. Music controlnet: Multiple time-varying controls for music generation. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 32:2692 2703, 2024.
[41] Shiyuan Yang. Xiaodong Chen, Jing Liao. Uni-paint: A unified framework for multimodal image impainting with pretrained diffusion model. In Proceedings of the 31st ACM International Conference on Multimedia, pages 3190-3199, 2023.
[42] Shilun Zhang and Lin Chen. Acoustic information masking effects of natural sounds on traffic noise based psychological health in open urban spaces. Frontiers in Public Health, 11:1031501, 2023.
[43] Yixiao Zhang, Yukara Ikemiya, Gus Xia, Naoki Murata, Marco A Martinez-Ramíre. Wei-Hsiang Liao, Yuki Mitsufuji, and Simon Dixon, Musiemagus: Zero-shot eex--oomuss editing via diffusion models. arXiv preprint 0rXiv:2402.06/78. 2024.
[44] Ye Zhu, Yu Wu, Kyle Olszewski, Jian Ren, Sergey Tulyakov, and Yan Yan. Discrete contrastive diffusion for cross-modal music and image generation. In International Conference on Learning Representations (ICLR), 2023.
13

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: I have claimed my contribution and scope of this paper in the abstract and in- troduction of proposing novel task of blending the environmental noise with personalized music and method to address this task.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We've discussed the limitation of our work in the Sec.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sell-speciiiatiinn asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
14

---

Answer:
[NA]
Justification:
We don't 't have any theoretical result in this paper.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof's can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
Justification: The details of experiment and implementation provided in the Sec. 41, and the codes would be provided the supplimentary materials.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, descríbing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e-g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
5. Open access to data and code
15

---

reproducible, they Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
The data we used are open-source, soun-sourced datasets. The names of the datasets and the implementation details are provided in Sec. 4 and our codes in the supplimentary materials.
Justification:
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips- cc/
public//giies//ddebbbbiisss http://oddeuuuisssiooooooooooooo osllllllicy for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" an acceptable answer. Papers cannot be rejected simply for including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
//apps.cc/pullic/uuiiee
swwsssssicyyyyy for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permirted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Our method is training- traening-free, and the test details are provided in Sec.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: In the subjective evaluation results we report error bars based on the standard error of the mean (SEM), which reflects variability due individual differences among participants.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
16

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative еπτог rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information the com- puter resources (type of compute workers, memory, time execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
The compute resources we used for our experiments are discribed in Sec. 4
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code of ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics httpp:://eeurpp::
Answer: [Yes]
Justification: Yes. The research described this paper fully complies with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: The positive and negative potential societal impacts were discussed in the Sec. 55) Broader Impact section.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
17

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
This paper poses no such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification:
The creators of assets, used in the paper, properly mentioned and credited.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
18

---

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com/dataseta
latasets.paperssit
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Question:
Answer: [Yes]
Justification:
The code we used for experiments would be provided in supplimentary materials.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmodel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [Yes]
Justification: The details of human subjective evaluation is provided in Sec. 22, and samples
provided Appendix C
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Question:
Answer: [Yes]
Justification:
We have obtained (equivalent) IRB approval based on our local regulations.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
19

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: The core method development in this research does not involve LLMS as any important, original, non-standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM policy (https://neurips /att::///earrrpsssc//oooerrreee/ cc///oooeeeeeecs//22/////.. for what should or should not be described.
20

---

Technical Appendices and Supplementary Material
This appendix provides additional details to complement the main text and support a deeper un- derstanding of our work. Sec. Asc.AAjin investiggaes how the FAD score varies with changes in loudness.
Sec. B elaborates on the details of Stage 1 in the method described in Sec. 3 including visual representation of the generation process. Sec. Supplementss the subjective evaluation by offering further explanation and presenting representative sample.
A Loudness-FAD relationship
We observed that loudness has a noticeable impact on the Fréchet Audio Distance (I], which evaluates how closely generated audio resembles reference ground truth music in terms of statistical properties. To investigate this effect, we conducted a dedicated experiment examining how FAD varies with changes in loudness. Specifically, we normalized the loudness of outputs from three baseline methods and our approach to a range between -24 and -3 dB LUFS, and then calculated the FAD scores for each loudness level. As shown Fig the results consistently reveal that the FAD score reaches its -18 and -12 dB LUFS. Interestingly, this range coincides with the dominant loudness levels of the reference ground truth, which likely reflects the most comfortable listening range for the human car. This experiment also suggests that while higher loudness can enhance the masking effect of noise, excessively high levels can degrade perceptual quality. Therefore, we aim to keep the overall loudness within an appropriate range to achieve better blending and provide the most comfortable experience for the listener.
the best match to real music-when the loudness is between
FAD with Varying Loudness for Different Types of Music
MusicGen
Riff A2A
10
BNMusic
FAD
-25
-20
-15
-10
5
Loudness (dB WFF)
Figure 4: The lowest FAD values for each type of music are highlighted, and they all appear in the -15 dB LUFS, indicating that for all types of music in our experiment, an optimal loudness level consistently falls between -18 to -12 dB LUFS.
B More details in Stage
This section provides a detailed supplement Stage of the method described in Sec. 3 Through visualizations, we illustrate the motivation for enforcing rhythmical alignment and explain how our Stage achieves this alignment via a combination of outpainting and inpainting. This approach not only ensures temporal coherence but also preserves sufficient musicality in the generated content. Additionally, we describe the strategy used select the core area of the noise mel-spectrogram plot, which serves as the foundation for the alignment process.
21

---

Noise
While Noise
Overlaying
(a)
Noise
Random Music
Overlaying
(b)
Noise
Aligned Music
Overlaying
(c)
Figure 5: Illustration of how conventional auditory masking, often using overly low volume and mismatched rhythm, can disrupt the listening experience. Proper volume adjustment and rhythmic alignment are essential for achieving more harmonious and pleasant blend with background noise.
The significance of alignment. Our approach leverages generative models produce music that rhythmically aligns with the background noise. This aligmment facilitates natural blending, as illustrated in Fig. [ reducing potential conflicts between the two sources. By ensuring temporal coherence from the outset, the combined audio avoids introducing additional disturbances into the soundscape. As result, the subsequent adaptive amplification can be applied more conservatively while still achieving effective masking. Even in frequency regions where complete masking is not possible, the improved alignment enhances perceptual harmony and helps minimize the listener's awareness of the underlying noise.
The significance of all steps. Fig. 6 illlustrates the detailed process of transforming single noise sample, represented as NNoises into the final output music in an image representation, XMusie As shown in Fig. [6] the outpainting step primarily focuses diffusing information from the preserved core region of the noise XNoise into the surrounding areas. This diffusion embeds contextual information into the surrounding music during the generation of XMid- However, this stage, directly converting Xyid into an audio signal AMia would retain noise content from the core region, significantly degrading the listening experience. To address this, subsequent inpainting step required to mask the remaining core noise area and replace it with structured, harmonious music that aligns with the text prompt. During this inpainting process, the information previously embedded in the surrounding music during outpainting diffuses back into the core region, ensuring seamless integration. The result, Xuusic- represents a cohesive and complete musical piece.
Furthermore, as demonstrated in Fig. 7 both our approach and the Riffusion's is audio-to-audio generation (6] exhibit the most effective alignment with the noise. The results of our approach, the Riffusion 6 is, and the MusicGen 's's is are all generated conditioned on the noise, expected to maintain strong rhythmic consistency for more seamlessly blending. In contrast, the result of MusicGen's 's melody-conditionn similar rhythmic synchronization with the noise, as expected. This highlights the superior ability of our method to align the generated music with noise, making it more coherent and seamlessly integrated while maintaining pleasant to the ear.
generation [3]. as well as the randomly chosen music, fail to achieve
The strategy of picking thresholds. The selection of 10%-20% of the area with smaller pixel values the mask region is based on empirical observations. Since the mask is extracted pixel-win axie-wise with a value range of 0-255 while the smaller pixel value indicates the higher energy level of the mel-frequency, small variations pixel intensity can lead to significant differences the mask area, especially in images with relatively low contrast. Our goal is to ensure that the mask region captures the primary high-energy frequency areas while keeping its size minimal. This approach provides the model with greater flexibility generate the desired musical elements. Conversely, during the
22

---

X Noise
Noise
Mask
Mid
1-Mask
Outpainting
Mid
Music
Inpainting
Figure 6: The more detailed illustration for the generation process in Stage 1, transitioning from NNose Musics should emphasize the distinct processing stages and the regions primarily affected during each step. This would include highlighting how the inpainting step serves as the pivotal transformation within the process, where chaotic noise regions are replaced with structured and meaningful music content.
inpainting phase, the preserved core region may sometimes occupy a relatively small proportion of the overall area. In such cases, the limited space can make it challenging for inpainting to generate sufficiently detailed and coherent musical content. To address this, we adjust the threshold to slightly enlarge the mask area, enabling the generation of more complete, harmonious, and cohesive musical result.
C More details about subjective evaluation
This section provides additional details regarding the subjective evaluation process. outlines the evaluation protocol and criteria used to assess perceptual quality, and includes a sample of the questionnaire presented to listeners during the study.
The participants would be seeing these words:
OVL (Overall): Measures the overall quality and pleasantness of the audio.
Perceptibility: Indicates how noticeable the original noise is in the presence of the music.
Both metrics are rated on a scale from to 5, where represents the least pleasant sound or the noise being most perceptible, while 5 denotes the most pleasant sound or the noise being least perceptible.
23

---

Noise
(NMusic
MusicGen
Random
[s]
Figure 7: The waveforms of set of samples, consisting of noise, random music track, and three music segments generated based on the noise, shown. As highlighted, our method achieves one of the best alignment effects, where any impulsive sound from the noise is seamlessly blended with corresponding strong musical sound, ensuring smooth integration between the two.
1
Noise playing simultaneously
in with '1-5' scores, 1 the least pleasant/moas percetible
with different music audios
5- the most pleasant/lesre percetible.
Scoring
Original Noise
Percepoibiitty
(Cverall; Measures overall qualitt pleasantness of the audio.
Perceptibility indicates how noticeable the or original noke the presence the
d
range pleasent
baing most parceptible, and 5 denotes the most pleasant sound the nolse being least perceptible.
Figure 8: A sample page of the subjective evaluation.
Each participant presented with set of audio clips, including the original noise, three music clips generated using the noise, and randomly selected real music piece overlaid with the noise. The participants were asked to the overall quality and perceptibility of each clip. A sample page of the questionaire is presented Fig. g
24


Reviews:
[{"review_id": "JMcVU1yxrJ", "reviewer": "Reviewer_nRgr", "summary": "This paper presents BNMusic, a diffusion-based model for music generation that blends environmental sounds (ENS) and musical notes. Unlike prior works that treat non-musical noise as background, BNMusic integrates ENS as a controllable and creative component of music. The method introduces a two-stage pipeline: first generating ENS audio, then conditioning music generation on it via diffusion. The approach is evaluated on a new dataset combining FSD50K and MusicCaps and demonstrates superior controllability and realism over baselines.", "questions": "- How would BNMusic perform on genres where environmental sounds are less common (e.g., classical)?\n- Can the model adapt to real-time or interactive settings where ENS evolves dynamically?\n- Is there a way to control the temporal alignment between ENS events and musical phrases?\n- Would joint training (instead of two-stage) affect quality or controllability?\n- Could more baselines (e.g., AudioLDM with explicit conditioning) be included?", "limitations": "Yes", "rating": 4, "confidence": 1, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "# Strengths:\n- Novel framing of environmental sounds as a generative music component rather than noise.\n- Effective use of a two-stage diffusion process for better disentanglement and controllability.\n- High-quality samples and a new dataset (BNMusic-50K) contribute to reproducibility.\n- Both objective (FC, FAD) and subjective (MOS) metrics show strong performance.\n\n# Weaknesses:\n- Limited baseline comparison; focuses on Mubert and simple concatenation.\n- Lack of detailed ablation on the contribution of ENS vs music branch in training.\n- Generalization beyond the proposed dataset is not fully explored.", "was_revised": false, "final_justification": "", "rebuttal": "## Rebuttal for reviewer nRgr\nWe appreciate the valuable feedback from the reviewer and the recognition of our *novel framing of environmental sounds as a creative component*, *the effectiveness of our two-stage diffusion process*, and *the high-quality samples produced by BNMusic*.\n\n>Q1: Limited baseline comparison; focuses on Mubert and simple concatenation. Could more baselines (e.g., AudioLDM with explicit conditioning) be included?\n\n**A1:**\nThank you for the suggestion. Following your advice, we additionally tested AudioLDM with explicit noise conditioning as a baseline. When using AudioLDM with noise as the direct audio condition, we observe that the model tends to incorporate excessive elements from the noisy input. This is likely because the audio condition provides a stronger generative prior than the accompanying text prompt, leading the model to focus more on reproducing the structure of the noise rather than generating harmonically coherent music. As a result, the output often resembles a more tonal version of the noise rather than distinct musical content.\n| Methods         |FAD ↓    | KL ↓   |\n|-----------------|---------|--------|\n| AudioLDM        | 27.06   | 3.80   |\n| BNMusic (Ours)  | 7.98    | 1.67   |\n\nIn contrast, our method effectively separates the guidance signal from the raw noise using a two-stage design, which enables more perceptually coherent and better-integrated outputs. These additional baseline results will be included in the revised version to support our design choices.\n\n>Q2: Lack of detailed ablation on the contribution of ENS vs music branch in training. Would joint training (instead of two-stage) affect quality or controllability? \n\n**A2:** \n1. Thank you for the question. We would like to firstly clarify that our method does **not** involve any additional or joint training. Instead, we focus on **re-designing the inference pipeline** in Stage 1 and applying an **adaptive amplification strategy** in Stage 2. Both stages are **training-free** and built to fully utilize the capabilities of the **pre-trained base models**.\n2. As shown in Tab. 3 (p.9) of our original manuscript (also included below for reference), we present an ablation study that evaluates the contributions of each component in our proposed pipeline. The results show that both the **Stage 1 generation** and the **Stage 2 amplification** play crucial roles in improving the perceptual and quantitative quality of the output. This demonstrates that each part of our design contributes meaningfully to the final result.\n    | Outpainting (Stage 1)| Inpainting (Stage 1) | Adaptive Amplification (Stage 2)         |FAD ↓    | KL ↓   |\n    |-----------------|---------|--------|--------|--------|\n    |     ✘     |      ✘      |    ✘    | 34.17   | -   |\n    |     ✓     |      ✘      |    ✘    | 8.68   | 1.89   |\n    |     ✓     |      ✘      |    ✓    | 9.18   | 1.84   |\n    |     ✓     |      ✓      |    ✘    | 8.00   | 1.78   |\n    |     ✓     |      ✓      |    ✓    | 7.98   | 1.67   |\n\n3. Given the nature of our task, we believe the quality and controllability are closely intertwined in our setting. Our objective is to use the environmental noise as a condition to guide music generation, such that the generated music perceptually integrates with the noise and reduces the listener’s awareness of it. In this context, effective controllability means that the noise appropriately influences the music generation, such as alignment, while the music remains sufficiently harmonious and pleasant. Achieving this balance is precisely what ensures the quality of the final blended output.\n\n>Q3: Generalization beyond the proposed dataset is not fully explored.\n\n**A3:** Thank you for raising this point. As described in Section 4.2 of the original manuscript, we evaluated our method on two distinct datasets to assess its generalization ability. We used noise samples from the **EPIC-SOUNDS** dataset, which encompass 58 distinct action types and 140 different object types. We also included the **ESC-50** dataset, which contains 50 different categories of environmental sounds. The strong and consistent performance of our method on both datasets demonstrates its robustness across a wide range of realistic auditory conditions.\n\nTo further address the reviewer’s concern about generalization, we collected 10 types of everyday environmental noises (elevator operation sounds, air conditioner hums, keyboard typing, etc) and used them as new test cases. The results are shown below:\n|Noises|Recorded noise (FAD ↓)|EPIC-SOUNDS (FAD ↓)|ESC-50 (FAD ↓)|\n|-|-|-|-|\n|Real Music|9.56|15.41|8.32|\n|Riff-A2A |8.75|10.95|7.74|\n|MusicGen|9.08|13.15|9.11|\n|BNMusic (Ours)|7.53|7.98|6.76|\n\nThese results further validate the practical generalization ability of our framework. Moreover, since our method is entirely training-free and does not involve any fine-tuning or learning from the input data, it is inherently less susceptible to dataset-specific bias. This design allows our approach to generalize effectively to unseen acoustic conditions without any adaptation or retraining.\n\n>Q4: How would BNMusic perform on genres where environmental sounds are less common (e.g., classical)? \n\n**A4:** \n1. As discussed in **A3**, our method demonstrates strong generalization capabilities and is able to handle a wide variety of environmental sounds, regardless of whether they are commonly seen or relatively rare in real-world settings.\n2. To further address the reviewer’s concern, we conducted additional experiments focusing specifically on less common types of environmental sounds. We selected two subsets: the 30 least frequent categories and the 50 least frequent categories from our test pool. The results are shown below:\n    |Methods|50 least frequent ENS (FAD ↓)|All ENS (FAD ↓)|\n    |-|-|-|\n    |Riff-A2A|13.08|13.15|\n    |MusicGen|11.20|10.95|\n    |BNMusic (Ours)|8.38|7.98|\n\n    These results suggest that even when applied to rarer or less familiar noise types, BNMusic maintains strong performance.\n\n3. Regarding the “*Classical*” genre mentioned in your question, we would like to clarify that in our framework, this refers to the musical style specified by the user prompt, rather than a characteristic of the environmental noise itself. To examine genre sensitivity, we evaluated different music styles,\n   \n    |Genres|Classical (FAD ↓)  |Jazz (FAD ↓) |Pop (FAD ↓) |Rock (FAD ↓) |\n    |-|-|-|-|-|\n    |BNMusic (Ours) |10.53|9.51|7.99|7.89|\n\n    The results suggest that genres like *Classical* and *Jazz*, which tend to have smoother dynamics and slower tempos, may be less effective at perceptually masking noise due to their limited rhythmic overlap with common environmental sounds. In contrast, more upbeat genres like *Pop* and *Rock* tend to blend more naturally with a wider range of noise patterns, making them more robust across diverse scenarios.\n\n>Q5: Can the model adapt to real-time or interactive settings where ENS evolves dynamically? \n\n**A5:** \n1. Our method is particularly well suited for many real-world scenarios where environmental noise exhibits temporal consistency or repetition, such as elevator operation, automatic garage doors, or air conditioning systems. In addition, there are scenarios like subway cars, where the sound patterns during uniform motion are highly similar across different cars. This makes it feasible to use the noise recorded in the front car as input to generate blending music that can be played in another car, achieving effective noise masking without requiring strictly real-time processing.\n2. Moreover, the most time-consuming stage of our method—music generation—can be significantly accelerated using recent fast diffusion inference techniques, such as SoundCTM [1] and Presto [2]. Based on the speed-up ratios reported in these works, a 5-second audio clip can be generated in under 0.1 seconds with single GPU inference. Combined with our preprocessing and Stage 2 adaptive amplification (which take approximately 0.3 seconds in total), the full pipeline can operate within 0.5 seconds. With further advances in hardware and model optimization, even faster performance can be expected, making our method promising for near-real-time or interactive applications where noise evolves gradually rather than abruptly.\n\n>Q6: Is there a way to control the temporal alignment between ENS events and musical phrases? \n\n**A6:** Thank you for the question. In our method, the mel-spectrogram serves as a time-frequency representation, where the horizontal axis corresponds to time and the vertical axis to frequency, with the intensity at each point reflecting the energy at a given time and frequency. When we extract the high-energy region of the ENS spectrogram to serve as the condition for outpainting, we are effectively providing the model with both spectral and temporal cues. As a result, the generative process naturally preserves the temporal structure encoded in the input, allowing the generated musical content to align with salient ENS events over time. This implicit alignment enables the music to respond rhythmically and structurally to the dynamics of the original noise. As shown in Figure 7 in the Appendix (Page 15) of the original manuscript, when the ENS contains salient events—such as sudden onsets or accents—the model tends to generate corresponding musical peaks at aligned time positions, demonstrating strong temporal synchronization. This inherent matching ability of the outpainting-inpainting pipeline distinguishes our approach from other models like MusicGen, which, despite conditioning on noise, do not achieve such precise temporal alignment, thus highlighting the advantage of our method in producing time-aware and contextually coherent musical outputs.\n\n---\n\n[1] SoundCTM: Unifying Score-based and Consistency Models for Full-band Text-to-Sound Generation. ICLR25.\n\n[2] Presto! Distilling Steps and Layers for Accelerating Music Generation. ICLR25."}, {"review_id": "sTt2URceHl", "reviewer": "Reviewer_cD4J", "summary": "This paper proposes a new insightful solution to the social issue of environmental noise in public areas by leveraging generative AI for music. Environmental noise in public areas is one of the major issues affecting the quality of human life.Traditionally, two main approaches have been employed. The first involves individuals using noise-reduction technology in headphones or earphones. However, this approach provides only limited solutions as it requires additional devices for each individual. The second approach involves generating different sounds by leveraging human psychological auditory characteristics to reduce the unpleasantness of the original noise. However, this method faces the challenge of requiring sounds with significantly high energy levels to effectively reduce noise impact. The authors propose a two-stage method consisting of (1) extracting the musical properties (primarily rhythm) of noise and generating music that matches these properties, and (2) equalizing the generated music based on human psychological auditory characteristics and mixing it with the noise. The authors comprehensively investigate the effectiveness of the proposed method from two perspectives: objective indicators and subjective evaluations.", "questions": "This paper proposes a very interesting new use case for generative AI. The paper is very well written and very easy to understand. In particular, the introduction speaks volumes about the appeal of this new application. I actually found myself asking as I was reading the paper, \"How on earth can a music generation AI that takes into account psycho-aural characteristics be constructed?\" and I was very much looking forward to the subsequent contents. Actually, generating music that prompts environmental noise and then suppresses it based on psycho-auditory characteristics seems like a truly magical tool. However, given its conceptually attractive goals, I get the impression that the actual system presented in this paper is somewhat more limited than I had hoped. My guess is that since it is very difficult to modify and improve the generation AI itself, the authors have chosen to use a fixed and pre-prepared generation AI and instead introduce outpainting and inpainting mechanisms to substitute a pseudo-ideal psycho-aural generation AI that takes into account ideal psycho-aural characteristics. I agree that this method of building the system is certainly one of the practical and reasonable solutions. This paper is totally unquestionable as a systems development paper. On the other hand, in a highly competitive and advanced technology presentation venue such as NeurIPS, this practical solution may have room for one more step of improvement in my subjective opinion. More specifically, it seems to me that building a generative AI that directly incorporates psychological auditory characteristics from its learning stage should have been the research subject that this paper was originally intended to address.I understand that these arguments of mine may perhaps seem unreasonable to the authors. So I would like to read the responses with an open mind, including the authors' rebuttals and the comments of other reviewers. Again, I thank the authors for sharing their very fascinating new perspective on the use of generative AI.\n\n[Major concern 1: Validity of outpanting and inpainting mechanism instead of directly incorporating psyco-aural characteristics into generative AI]\n\n- I am questioning whether it is possible to utilize the effects of acoustic masking based on psychological auditory characteristics at the stage of generating music.I understand that the effects of psychological auditory characteristics are utilized in Stage 2 of the authors' method. However, Stage 2 appears to retain only functions similar to an equalizer. To obtain generated music that utilizes psychological auditory characteristics as described in the introduction, it seems necessary to explicitly introduce these effects in Stage 1 of the authors' method.\n\n[Major concern 2: Validity of prompt design (Mask M)]\n\n- I do not fully understand the design guidelines for mask M, which is used to extract high-energy portions from noise. For example, if mask M is intended to extract rhythmic elements from noise, it seems possible to use a method that extracts periodic elements. Alternatively, if the goal is to extract frequency patterns that co-occur simultaneously from noise, it seems more appropriate to use a method that extracts low-rank structures from matrices, such as non-negative matrix factorization.I am unclear about the motivation behind the authors' decision to assign mask M the role of focusing solely on high-energy regions.", "limitations": "Yes. The authors clearly address the limitations.\n\nAs an additional request, if the authors could more clearly discuss the validity and limitations of the authors' use of outpainting and inpainting mechanisms to introduce psychophysiological auditory properties postprocessively instead of introducing them directly into the training of the generative AI, the authors would be happy to discuss the future of this paper. improvements to this paper in the future.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "[Strengths]\n\n- I believe that the focus proposed in this paper is highly insightful as one of the new applications of generative AI in recent years.\n\n- The authors provide video materials as a tool to intuitively convey the effectiveness of the proposed method to readers. This is very useful for easily following the new problem setting assumed by the authors and understanding the practical effects of the proposed method.\n\n- The use of subjective evaluation as an experiment to demonstrate the empirical effectiveness of the proposed method greatly enhances its persuasiveness.\n\n[Weaknesses]\n\n- There are concerns about whether the psychological auditory characteristics of humans are truly being utilized effectively in the generation of music for environmental noise (see question for details).\n\n- I am concerned that the technology used to extract musical characteristics for generating music from noise (masking high-energy areas on the Mel spectrogram) is designed to fit the authors' application examples well.", "was_revised": true, "final_justification": "I am grateful for the authors' strong response and am both surprised and impressed. This is a very strong rebuttal. My concerns have been completely resolved beyond my expectations. I am delighted to raise my rating to 5. I am very grateful for having had the opportunity to read this excellent paper and to have deepened my understanding through this discussion.\n\nInitially, I had doubts about whether the authors' music generation process truly incorporated auditory characteristics. However, the authors' responses completely alleviated my concerns (beyond my expectations). Therefore, I have updated my score to positive.", "rebuttal": "## Rebuttal for reviewer cD4J\n\nWe thank the reviewer for the thoughtful and encouraging feedback. We appreciate the recognition of our work as a *highly insightful* application of generative AI, as well as the positive remarks on the *clarity* of the writing, the *appeal* of the proposed direction, and the *empirical persuasiveness* of the subjective evaluation. We also value the acknowledgment of our system as a *practical and reasonable solution* to a socially relevant challenge. Below, we respond to the reviewer’s concerns in detail.\n\n>Q1: There are concerns about whether the psychological auditory characteristics of humans are truly being utilized effectively in the generation of music for environmental noise (see question for details). [Major concern 1: Validity of outpanting and inpainting mechanism instead of directly incorporating psyco-aural characteristics into generative AI] I am questioning whether it is possible to utilize the effects of acoustic masking based on psychological auditory characteristics at the stage of generating music.I understand that the effects of psychological auditory characteristics are utilized in Stage 2 of the authors' method. However, Stage 2 appears to retain only functions similar to an equalizer. To obtain generated music that utilizes psychological auditory characteristics as described in the introduction, it seems necessary to explicitly introduce these effects in Stage 1 of the authors' method.\n\n**A1:** Thank you for the thoughtful comment. We would like to clarify that the principle of auditory masking is considered **throughout both stages** of our framework, not just in Stage 2.\n\n1. In Stage 1, we explicitly design the conditioning mask *M* based on the high-energy regions of the mel-spectrogram of the noise input. This choice is motivated by the established psychoacoustic theory, that frequencies with **higher local energy** are more perceptually salient and more likely to interfere with auditory perception [1,2].  Since the mel-spectrogram reflects energy distributions in critical bands, extracting the most energetic regions helps us locate the parts of noise that are most audible and thus most important to mask. By using these high-energy regions as input conditions, we guide the model to generate music that naturally aligns with the most perceptually prominent components of the noise, enabling more effective masking of those elements during blending.\n2. Furthermore, to assess the sensitivity of the output to the mask design, we conducted additional experiments with alternative masking strategies:\n\n- Randomly selected 20% region\n- Larger high-energy area (30%–50%)\n- Our original 10%–25% high-energy mask\n\n    The results are summarized below:\n\n    | Masking Strategy   | FAD ↓  | KL ↓   |\n    |--------------------|--------|--------|\n    | Random 20% region | 17.82   | 2.47   |\n    | Top 30%–50% high-energy area     | 13.79   | 2.14   |\n    | Top 10%–25% high-energy (ours) | 7.98   | 1.67   |\n\n    As shown, altering the masking strategy significantly degrades performance. This underscores the critical role that **psychoacoustically informed mask selection** plays in the effectiveness of the first-stage generation.\n3. In Stage 2, our adaptive amplification strategy builds directly upon the foundation laid in Stage 1. By conditioning on the high-energy regions of the noise during generation, the resulting music naturally inherits similar frequency distribution patterns in those regions. This design ensures that the musical signal already contains energy concentrated in the same areas as the noise. As a result, during Stage 2, we can achieve effective auditory masking by applying only modest amplification, since the generated music is **already aligned both temporally and spectrally with the dominant noise components**. This tightly coupled, two-stage design allows our system to perform perceptually effective blending with lower gain, preserving musical coherence while suppressing unwanted noise in a psychologically informed manner.\n4. We agree that incorporating psychoacoustic knowledge directly into the generative model's learning objective is a promising direction and appreciate the reviewer’s encouragement in that regard; we consider it a compelling avenue for future research.\n\n>Q2: I am concerned that the technology used to extract musical characteristics for generating music from noise (masking high-energy areas on the mel-spectrogram) is designed to fit the authors' application examples well. [Major concern 2: Validity of prompt design (Mask M)] I do not fully understand the design guidelines for mask M, which is used to extract high-energy portions from noise. For example, if mask M is intended to extract rhythmic elements from noise, it seems possible to use a method that extracts periodic elements. Alternatively, if the goal is to extract frequency patterns that co-occur simultaneously from noise, it seems more appropriate to use a method that extracts low-rank structures from matrices, such as non-negative matrix factorization.I am unclear about the motivation behind the authors' decision to assign mask M the role of focusing solely on high-energy regions.\n\n**A2:** We appreciate the reviewer’s insightful suggestions regarding alternative ways of designing the conditioning mask 𝑀. We chose the current design based on the following considerations:\n\n1. As mentioned in **A1**, our core goal is to **mask the most perceptually intrusive parts** of the environmental noise. According to psychoacoustic background, among all sounds perceptible on the loudness curve, the strongest (highest-energy) frequency components are the most prominent [1,2], thus, the most critical to address. By selecting the top 10–25% highest-energy regions in the mel-spectrogram as our mask 𝑀, we directly target the parts of the noise that listeners are most likely to notice and therefore most benefit from masking.\n\n2. This masking-oriented design aligns well with our objective: we are not aiming to extract rhythmic or harmonic structure itself, but to generate a musically coherent signal that perceptually suppresses noise. While techniques such as periodicity analysis or matrix decomposition (e.g., NMF) are useful for uncovering latent structures, they *may not focus on the most perceptually relevant aspects* of the noise for our purpose.\n\n3. Furthermore, this energy-based masking scheme naturally aligns with our Stage 2 amplification strategy: the same mask not only guides the music generation in Stage 1, but also **informs gain adjustment** in Stage 2. Because the generated music has already been conditioned to match the dominant energy regions of the noise, it becomes inherently more effective at masking those components. As a result, only a relatively small gain adjustment is needed to achieve perceptually meaningful suppression of the noise, thereby reducing the listener’s awareness of disruptive elements while preserving musical naturalness.\n\n4. In summary, our design of mask *M* is not arbitrarily tailored to our examples but is instead a principled choice for maximizing masking effectiveness in a training-free setting. The high-energy strategy ensures that generation is perceptually aligned with the noise in a way that directly supports our task objective.\n\nWe are glad to integrate the above clarifications into our revised manuscript.\n\n> As an additional request, if the authors could more clearly discuss the validity and limitations of the authors' use of outpainting and inpainting mechanisms to introduce psychophysiological auditory properties postprocessively instead of introducing them directly into the training of the generative AI.\n\n**A3:** Thank you for the valuable comment. We provided the extended discussion below:\n\n- **Validity of outpainting and inpainting mechanisms:** As discussed in **A1** and **A2**, our proposed inpainting and outpainting operations introduce psychophysiological auditory properties via masking strategies. These are applied at the spectrogram level and are designed to align with perceptual cues of salience and continuity.\n\n- **Advantages and limitations of post-sampling techniques versus training-time integration:**\n\n   - Our post-sampling strategy offers strong flexibility and generalization, as it is decoupled from the training data and model. This allows it to be applied across different generative backbones and to a wide range of real-world noise types. Our extended generalization results (please also see **A3** and **A4** to *R-nRgr*) empirically support this advantage.\n\n   - The main trade-off, however, is additional computational overhead during inference, due to the need to extract masking cues from the noise input. However, this overhead can be mitigated with modern fast-inference techniques, such as [3, 4].\n\n   - In contrast, integrating such mechanisms during training would require retraining generative models from scratch, an effort that is both computationally expensive and data-sensitive. Such training-based models often struggle with out-of-distribution (OOD) noise inputs, unfamiliar noise types may lead to incoherent outputs. Moreover, training on **large-scale music data** may raise copyright concerns, whereas our pipeline uses open-access, non-sensitive environmental noise data, avoiding such complications.\n\nWe hope to integrate the above discussions into our revised manuscript if the reviewer find it helpful.\n\n---\n[1] Hugo Fastl and Eberhard Zwicker. Psychoacoustics: Facts and Models. Springer-Verlag, Berlin, Heidelberg. 2006.\n\n[2] Moore, B. C. J.. An Introduction to the Psychology of Hearing (6th ed.). Brill. 2012\n\n[3] SoundCTM: Unifying Score-based and Consistency Models for Full-band Text-to-Sound Generation. ICLR25.\n\n[4] Presto! Distilling Steps and Layers for Accelerating Music Generation. ICLR25."}, {"review_id": "vvKJwtG1fM", "reviewer": "Reviewer_q6Wg", "summary": "This paper introduces BNMusic, a novel framework for blending environmental noise into personalized, rhythmically-aligned music, as an alternative to traditional acoustic masking or noise cancellation. \nRather than suppressing noise, the method reduces its perceptibility by integrating it musically, offering a more comfortable auditory experience in shared spaces. \nThe approach relies on a two-stage spectrogram-level pipeline: (1) noise-aligned music generation via outpainting and inpainting using latent diffusion (Riffusion), and (2) adaptive loudness amplification optimized using psychoacoustic masking thresholds to ensure effective yet subtle blending. \nExperiments on EPIC-SOUNDS and ESC-50 noise sources demonstrate quantitative and subjective improvements over Riffusion, MusicGen, and real music baselines in terms of perceptual quality and masking effectiveness.", "questions": "1. One big question is about the outpainting and inpainting process. From the Figure 6, when you only have a noise for example in the high frequency range, how can the model to outpaint the rest of the region that are related to the noise? These noise regions could be treated as conditional signal, but what is the relationship between these conditional signal with the outpainted regions? For example, how would you link the wind noise that could jam lots of frequency bands with any music signal? In the image inpainting or outpainting, the outpainted region could semantically align with the conditioned region. Can the author provide mroe justifications here?\n2. Why do you choose to use Griffin-Lim algorithm rather than Vocoder for spectrogram to audio converting? It is quite common for audio generation based on the spectrogram to use vocoder to decode the spectrogram back to the audio rather than Griffin-Lim algorithm. What is the concern here?\n3. Visualization issues in Fig3, if you want to directly visualize the sptrogram difference. Would calculate the absolute distance between the generated music and noise while visualizing with a better color scheme like \"jet\" be better?\n4. Other music or audio generation work would also use IS, KL and other metrics to evaluate the performace, could you provide more analysis of why you didn't include this metrics or provide more comparison with different metrics? Using the Audioldm_eval github repo, you can get all of these metrics at once. \n\nI would like to raise my score if my questions are well addressed.", "limitations": "See the weaknesses.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n1. The proposed framework and task is interesting. Blending noise into music and make it less noticeable is quite cute an idea.\n2. Evaluation results show that the proposed method can achieve better noise blending performance in both subjective and objective metrics.\n\nWeaknesses:\n1. The biggest limitation and weakness is the real-time applicability. While I feel the generated audio is quite good, this is still a post-processing method that when you have already recorded the audio and then generate the corresponding music signal to blend the noise together. In the real world scenario, noise can be time-variant signal and it seems that these kind of system need to generate the blending music in real time to adapt to different noisy sound. Can you provide mroe analysis on this aspect?\n2. The description on the second stage is a bit hard to understand. There are lots of equations and descrptions that are confusion. From my understanding, let A denote the nosie and B denote the generated music, and let $\\lambda$ denotes the weights when you blend the music, i.e. A+$\\lambda \\cdot$B. If you want to totally get rid of the noise, theoretically you can make the $\\lambda$ much larger than 1 which I understand is not possible in reality. However, if you put the regions in $\\lambda$ that corresponding to the noise spectrogram close to 0 while mainting other regions (don't contain noise) to 1, the blended results would still be good right? Can you provide more explanations and justifications on this part?", "was_revised": true, "final_justification": "I think most of my concern are addressed and I appreciated the author's discussio on the real-time application for the proposed method. And I think the overal idea of blending the music into noisy to surpass it is cute so I would increase my score to 5.", "rebuttal": "## Rebuttal for reviewer q6Wg\n\nWe thank the constructive and encouraging feedback, and appreciate the recognition of *our framework and task as interesting*, the characterization of our method as *quite a cute idea*, and that our evaluation *shows better noise blending performance in both subjective and objective metrics*. Below, we address the reviewer’s questions and concerns in detail.\n\n>Q1: Main limitation lies in real-time applicability—can the method adapt to time-varying noise instead of relying on offline post-processing?\n\n**A1:** Thank you for raising this important point. We elaborate on our analysis and discussions around different scenarios below.\n\n1. Our method is well-suited for many real-world scenarios **where environmental noise exhibits repetitive or predictable patterns**, such as the operation of elevators, automatic garage doors, or air conditioning systems. In such cases, even though our current pipeline runs as a post-processing step, the **consistency and predictability of the noise** enable the system to **leverage prior statistical patterns**, allowing it to generate perceptually coherent and rhythmically aligned music. This makes the system effectively usable in **real-time or low-latency settings**, without requiring strict online adaptation at every moment.\n2. As for **dynamic noise patterns**, our pipeline is theoretically capable of running with low latency. The current most time-consuming stage is music generation, which can be significantly accelerated using recent advancements in fast diffusion inference techniques, such as SoundCTM [1] and Presto [2]. According to the speed-up ratios reported in these papers, a 5-second audio clip can be generated in under 0.1 seconds on a single GPU. When combined with our preprocessing and Stage 2 adaptive amplification, both of which take approximately 0.3 seconds, the full pipeline can operate in under 0.5 seconds. With continues improvements in hardware and model optimization, this latency could be reduced even more, making the system viable for interactive or near-real-time applications where the noise evolves gradually. In future work, we plan to incorporate these acceleration strategies into our pipeline to support faster and more responsive performance.\n \n>Q2: Stage 2 blending seems unclear—why not simply reduce $\\lambda$ to near zero in noisy regions for a good blending? \n\n**A2:** Thank you for the question. We would like to clarify our motivation behind the adaptive amplification strategy:\n\n1. Our method is grounded in auditory masking theory. The goal is to use music with higher energy to mask the stronger components of noise, rather than expose them. By ensuring the musical signal **exceeds** the noise level in high-energy regions, we effectively suppress the perceptual salience of the noise. In contrast, setting λ close to zero would directly expose those noisy regions without masking, which goes against our core design principle, aiming to make noise less perceptible, not more exposed.\n\n2. Blending does not mean fully exposing the noise. Our goal is to improve the auditory experience by **reducing the listener’s perception of the noise**, which is difficult to achieve if the noise is left entirely unmasked. Instead, we rely on auditory masking to suppress the most salient parts of the noise by amplifying well-matched musical content in those regions. This way, the listener perceives a more music-dominant and harmonious soundscape, where the noise is perceptually embedded rather than directly interfering. Reducing λ to near zero in noisy regions would result in the noise being fully exposed, which undermines the masking effect and ultimately degrades the overall perceptual quality.\n\n\n>Q3: How does the model outpaint meaningful music from noisy regions? What’s the semantic link between the masked input and the outpainted area? How to link noise that jam lots of frequency bands with any music signal?\n\n**A3:**\n1. Our system is built upon a generative model trained to synthesize coherent and harmonious musical spectrograms. Much like how Stable Diffusion can generate a complete and semantically meaningful image from a partial visual region in inpainting or outpainting tasks [3,4,5]—such as generating a human body from a given face image—our model learns to infer the full musical structure based on a partial region and a text prompt. In our case, the high-energy noise region serves as a conditioning signal, and together with the prompt, guides the generation of the remaining music in a way that is both musically plausible and contextually coherent.\n\n2. The relationship between the conditioned signals and the outpainted region is similar to image inpainting/outpainting tasks. The model is compelled to extract meaningful patterns (rhythm, texture, or dynamics) from the input region and use them to inform the surrounding generation. In other words, the conditioned region serves as a semantic and stylistic anchor that influences the generated music, ensuring the output aligns with the perceived \"intent\" of the original noise segment.\n\n3. Our masking strategy focuses only on the top 10%–25% highest-energy regions in the spectrogram, rather than conditioning on all noisy areas. By targeting only the most perceptually dominant components, our method avoids being overwhelmed by the full extent of broadband noise. As a result, even when the noise jam lots of frequency bands, it does not adversely affect the generation process - only the most relevant acoustic features are used to guide the model, ensuring both control and robustness.\n\n\n>Q4: Why choose to use Griffin-Lim algorithm rather than Vocoder for spectrogram to audio converting?\n\n**A4:** We chose to use the Griffin-Lim algorithm for waveform reconstruction due to both practical and methodological considerations:\n\n1. Our method is built upon the Riffusion model, which originally employs Griffin-Lim for audio reconstruction. To maintain consistency with the base model and avoid introducing additional confounding factors, we retained this default setting.\n2. The mel-spectrogram configuration in Riffusion, such as mel scaling, hop size, is not directly compatible with standard neural vocoders like HiFi-GAN. Using such vocoders would require retraining or fine-tuning, which is beyond the scope of our current study.\n3. Our primary focus is on improving the perceptual blending between noise and generated music, rather than optimizing audio fidelity.\n\nThat said, we agree that incorporating a neural vocoder could have the potential to further improve the perceptual quality of the generated audio. This is a very helpful suggestion, and we will consider integrating a vocoder-based decoding pipeline in future revisions to enhance the overall listening experience.\n\n>Q5: Visualizing with color scheme like \"jet\" would be better. \n\n**A5:** Thank you for the suggestion. We agree that using the absolute difference with a better color scheme like jet can improve clarity. Due to the policy of rebuttal this year, we are not able to provide the figures but only text. We will update the visualization accordingly in the revision.\n\n>Q6: Why not include more metrics in Audioldm_eval?\n\n**A6:** Thank you for the suggestion. \n1. In our paper, we primarily use FAD and KL divergence to evaluate our system. These metrics are computed on the **final blended outputs**, which **consist of the output music mixed with conditioned noise** and **represent what the listener would actually hear**. Ideally, our goal is to make the noise perceptually disappear and allow the listener to experience the audio as if it were real music. Both FAD and KL divergence are well-suited for this: they measure distributional distances between generated and real music from a group-wise (FAD) and pairwise (KL) perspective, thus capturing perceptual and statistical closeness to natural music.\n2. Although *Audioldm_eval* includes a wide range of metrics, many of them are **tailored to general audio generation tasks** and are **not designed to evaluate perceptual masking effectiveness or contextual audio blending**. Specifically, metrics such as Inception Score (IS) and CLAP score are not directly appropriate for our use case. IS is intended to assess the diversity and clarity of generated audio, which is useful for general synthesis tasks but does not align with our objective of perceptual integration of music and noise. Similarly, CLAP measures text-audio alignment, which we do not explicitly optimize, as our model relies only on inherited text conditioning from the base model (Riffusion). Other common metrics face similar limitations and may yield evaluations that do not reflect the perceptual goals of our system.\n3. Ultimately, this highlights a broader issue in the field: there is currently no widely accepted metric for quantifying the perceptual effectiveness of auditory masking or audio blending based on psychoacoustic principles. This is especially challenging when the goal is not to evaluate the audio itself in isolation, but its perceptual function in context. As such, we complement our evaluation with human listening studies, which offer a more direct and reliable reflection of perceptual quality. \n\n---\n\n[1] SoundCTM: Unifying Score-based and Consistency Models for Full-band Text-to-Sound Generation. ICLR25.\n\n[2] Presto! Distilling Steps and Layers for Accelerating Music Generation. ICLR25. \n\n[3] Inpainting using Denoising Diffusion Probabilistic Models. CVPR2022.\n\n[4] SmartBrush: Text and Shape Guided Object Inpainting with Diffusion Model. CVPR2023.\n\n[5] Continuous-Multiple Image Outpainting in One-Step via Positional Query and A Diffusion-based Approach. ICLR 2024."}, {"review_id": "tCczxJWwKY", "reviewer": "Reviewer_TZwc", "summary": "This work addresses the task of acoustic masking to reduce the impact of environmental noise. Unlike conventional approaches that rely on active noise cancellation, the authors propose a novel and interesting solution: generating music from the environmental noise itself. The proposed framework uses a diffusion-based music generator to synthesize music conditioned on the input noise and text prompts, which is then blended with the original noise signal to achieve acoustic/noise masking. The evaluation includes quantitative analysis using defined metrics, human evaluation, and qualitative assessment through visualizations of the differences between the generated music and the original noise. An ablation study is conducted to show the contributions of key components in the framework. The supplementary material includes audio demos showcasing music generated from environmental noise.", "questions": "My questions are included in the Strengths and Weaknesses section. I expect the authors' rebuttal to address these points.", "limitations": "Yes", "rating": 5, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strength:\n- The paper is well-written and easy to follow. Most of the questions I had while reading were clarified by the end.\n- The proposed task is novel and interesting, and the authors provide a clear and compelling motivation for it.\n- The experimental results demonstrate that the proposed method performs strongly compared to relevant baselines. The choice of baselines is reasonable, and it is particularly interesting to see that using generated music significantly outperforms using random music.\n\nI did not identify any major weaknesses in the paper. However, there are a few points that may benefit from further clarification:\n- In Stage 1, how is the core noise area selected? Is the final output sensitive to this selection?\n- Regarding the objective evaluation: what is considered the ground-truth music? Since the output music is generated based on both the noise and a text prompt, it’s unclear how ground-truth references are defined or obtained. I may be missing something here.\n- In the qualitative evaluation (Figure 3), Riffusion-A2A appears to produce better results than BNMusic. Why then does Riffusion-A2A perform significantly worse in the quantitative results shown in Tables 1 and 2?\n- In the supplementary video, the generated music clips are quite short. Is it feasible to use longer noise segments to generate extended music tracks? In real-world scenarios, environmental noise often changes over time, rather than repeating the same pattern.", "was_revised": true, "final_justification": "The authors’ rebuttal addressed my concerns, and I will maintain my current rating.", "rebuttal": "## Rebuttal for reviewer TZwc\n\nWe thank the reviewer for the encouraging and constructive feedback. We appreciate the recognition of our work as *well-written*, the *novel* and *interesting* nature of the proposed task, the *clear* and *compelling motivation*, and the *strong performance* of our method against *reasonable baselines*, especially the finding that *generated music significantly outperforms random music*. Below, we address the reviewer’s questions and requests for clarification in detail.\n\n>Q1: In Stage 1, how is the core noise area selected? Is the final output sensitive to this selection? \n\n**A1:** Thank you for the thoughtful question. \n1. In Stage 1, we select the top 10%–25% highest-energy regions of the mel-spectrogram as the conditioning mask. This design is grounded in psychoacoustic background: **energy-dominant components** are the most easily perceived by human listeners [1,2] and thus the most disruptive in noisy environments. By leveraging the auditory masking effect, our goal is to generate music that can perceptually mask these prominent noise components. Therefore, we use these high-energy regions both as conditioning input in Stage 1 to guide the music generation, and in Stage 2 to compute adaptive amplification weights for effective masking.\n2. To assess the sensitivity of the output to the mask design, we conducted additional experiments with alternative masking strategies:\n\n- Randomly selected 20% region\n- Larger high-energy area (30%–50%)\n- Our original 10%–25% high-energy mask\n\n    The results are summarized below:\n\n    | Masking Strategy   | FAD ↓  | KL ↓   |\n    |--------------------|--------|--------|\n    | Random 20% region | 17.82   | 2.47   |\n    | Top 30%–50% high-energy area     | 13.79   | 2.14   |\n    | Top 10%–25% high-energy (ours) | 7.98   | 1.67   |\n\n    As shown, selecting a focused and moderate-sized high-energy region leads to better performance. Enlarging the area dilutes the effectiveness of the condition, and random selection fails to capture perceptually salient noise components. These results validate our design choice as both principled and empirically effective.\n\n>Q2: Regarding the objective evaluation: what is considered the ground-truth music? Since the output music is generated based on both the noise and a text prompt, it’s unclear how ground-truth references are defined or obtained. I may be missing something here. \n\n**A2:** Thank you for raising this important point. In our evaluation, the term “ground-truth” does not refer to a one-to-one matched target for each noise instance. Rather, it represents an idealized reference: in the best-case scenario, the blended result (noise + output music) should be perceptually indistinguishable from real, harmonious music, such that the presence of noise is no longer noticeable to the listener.\n \nTherefore, when computing objective metrics such as FAD or KL divergence, we use a collection of real music tracks from the MusicBench dataset as a reference set. The goal is not to evaluate whether the blended audio match any specific reference track, but rather to assess how closely it resembles natural music at a distributional and perceptual level. This allows us to measure the overall musicality and coherence of the output, without requiring one-to-one correspondence with the reference tracks.\n\n>Q3: In the qualitative evaluation (Figure 3), Riffusion-A2A appears to produce better results than BNMusic. Why then does Riffusion-A2A perform significantly worse in the quantitative results shown in Tables 1 and 2? \n\n**A3:** Thank you for pointing this out. The apparent discrepancy between the qualitative and quantitative results can be explained as follows:\n \n1. Figure 3 in the original manuscript is designed to visualize the spectral difference between the generated music and the input noise, aiming to reflect **how closely the frequency distributions align**. In this regard, Riffusion-A2A indeed shows a smaller difference, which indicating that its output is more similar to the noise mel-spectrogram. However, this comes at a cost: the generated audio tends to **sound like a slightly more melodic version of the noise itself**, rather than coherent music.\n \n2. This trade-off is also reflected in the quantitative results, where Riffusion-A2A shows lower similarity to real music and performs worse in terms of perceptual quality metrics. Its outputs are closer to noise than to music, which **contradicts** the main purpose of our work - to **blend noise with harmonically rich music and reduce listeners' perception of the noise**. As comparison, our method achieves a better balance between alignment with the noise and musicality, leading to higher perceptual quality and better blending.\n\n\n>Q4: In the supplementary video, the generated music clips are quite short. Is it feasible to use longer noise segments to generate extended music tracks? In real-world scenarios, environmental noise often changes over time, rather than repeating the same pattern.\n\n**A4:** Thank you for the insightful comment. While the base model Riffusion supports generating multiple 5-second segments to extend audio length, maintaining coherence across segments is challenging and often results in discontinuities in musical flow. In other words, **long-form music generation remains an open and active research challenge** in the field, particularly when aiming for thematic and structural consistency over time. In our case, this challenge is further compounded by the need to adapt to evolving noise. Improving long-range coherence is thus an important direction for future work, especially for real-world scenarios with dynamic environments.\n\n---\n\n[1] Hugo Fastl and Eberhard Zwicker. Psychoacoustics: Facts and Models. Springer-Verlag, Berlin, Heidelberg. 2006.\n\n[2] Moore, B. C. J.. An Introduction to the Psychology of Hearing (6th ed.). Brill. 2012"}]

Author rebuttal:

Meta-review: The paper proposes BNMusic, a novel method for blending environmental noise into personalized music using a two-stage generative framework. It creatively reframes noise as input rather than interference.

Strengths:
- The idea of treating environmental noise as a creative input for music generation is original and potentially impactful.
- Strong empirical results, including both objective metrics and human evaluations.
- The writing is clear and well-organized, making the methodology and contributions easy to understand.

Although the real-time applicability is still uncertain, I recommend acceptance as the paper presents a novel idea, supported by strong technical execution and thoughtful responses during the rebuttal.
Decision: Accept (poster)