MARS-Sep: Multimodal-Aligned Reinforced Sound Separation
Authors: Zihan Zhang
Xize Cheng
Zhennan Jiang
Dongjie Fu
Jingyuan Chen
Zhou Zhao
Tao Jin
Venue: ICLR 2026 Poster (2026)
Keywords: Universal Sound Separation
Multimodal Learning
Reinforcement Learning
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: Universal sound separation faces a fundamental misalignment: models optimized for low-level signal metrics often produce semantically contaminated outputs, failing to suppress perceptually salient interference from acoustically similar sources. We introduce a preference alignment perspective, analogous to aligning LLMs with human intent. To address this, we introduce MARS-Sep, a reinforcement learning framework that reformulates separation as decision making. Instead of simply regressing ground-truth masks, MARS-Sep learns a factorized Beta mask policy that is steered by a preference reward model and optimized by a stable, clipped trust-region surrogate. The reward, derived from a progressively-aligned audio-text-vision encoder, directly incentivizes semantic consistency with query prompts. Extensive experiments on multiple benchmarks demonstrate consistent gains in Text-, Audio-, and Image-Queried separation, with notable improvements in signal metrics and semantic quality. Our code is available at https://github.com/mars-sep/MARS-Sep. Sound separation samples are available at https://mars-sep.github.io/.

Published as a conference paper at ICLR 2026
MARS-SEP: MULTIMODAL-ALIGNEE REINFORCED
SOUND SEPARATION
Zihan Zhang'; Xize Cheng!*, Zhennan Jiang", Dongjie , Jingyuan Chen', Zhou Zhao',
Tao Jin It
Zhejiang University Institute of Automation, Chinese Academy of Sciences (zihanzhang.al ai, jint zju]@zju.edu.c
ABSTRACT
Universal sound separation faces a fundamental misalignment: models optimized for low-level signal metrics often produce semantically contaminated outputs, fail- ing to suppress perceptually salient interference from acoustically similar sources. We introduce preference alignment perspective, analogous aligning LLMs with human intent. To address this, we introduce MARS-Sep. a reinforcement learning framework that reformulates separation decision making. Instead of simply regressing ground-truth masks, MARS-Sep learns a factorized Beta mask policy that steered by preference reward model and optimized by a stable, clipped trust-region surrogate. The reward, derived from a progressively- aligned audio-text-vision encoder, directly incentivizes semantic consistency with query prompts. Extensive experiments on multiple benchmarks demonstrate con- sistent gains in Text- Audio-, and lauge--ueried separation, with notable im- provements in signal metrics and semantic quality. Our code is available at https::/ http:///ggtuuuuuuoee thub. com ma rs- www.rs-seeppeee wwwpseppat...e MARS Sep. Sound separation samples are available at https::/ marss sep git..gittub... 1o/
INTRODUCTION
Sound separation (Liu et al. 2022 Dong al., 2023 Cheng et al., 2025d, Chen et al. 2022 Mah-
mud & Marculescu 2024 Ma al.
[0244] Huang et al. 2025b) is foundational problem in audio
processing with broad impact on downstream tasks such as speech recognition (Shi ct al., 2022
Cheng et al. 2023 Kalda et al., 2024 Fu et al. 2024), spoken dialogue (Ji al., [024]] Cheng al.
20256 Fu et al. | Cheng et al. 2025a) sound event detection (Turpault et al. [019]] Yin et al,
2025), and acoustic scene analysis (Kim & Chang 2024; Su al. 2023). Beyond its standalone importance, separation also serves as a powerful data engine: by decomposing mixtures into iso- lated sources, enables large-scale augmentation that improves robustness and generalization (Chiu et al., 2021; Yuan et al. 2022; Cheng et al. 2025c; Manilow et al. 2019). In this work, we focus on universal, query-conditioned separation where the query can be audio, text, or image-and and ask how to make the separated output not only signal-clean, but also semantically faithful to the user's intent.
Despite notable progress, prevailing methods are predominantly optimized for distortion or interference-based metrics including SDR, SIR, SAR (Vincent et al.] (2006), SI-SDR (Roux et al.) 2019) and give limited consideration semantic alignment during training. This creates mel- ric dilemma: models optimized for waveform reconstruction can score high on signal-level metrics while leaving perceptually salient interference, thus violating semantic correspondence to the query.
Inspired by RLHF which learns a preference model from human data and uses as reward signal to steer the base LLM,, we conceptualize query-conditioned sound separation ence alignment problem: the user query (audio, text, or image) is the preference, and the goal is to produce output that maximizes semantic alignment with the query.
identical prefer-
Based on this logic, we reformulate sound separation as a meta-rcasoning task. We treat the base separation architecture base policy and reinforcement learning the optimization algo-
"qqual contribution.
1 "orresponning author.

---

Published as a conference paper at ICLR 2026
rithm. The human preference is captured by learned multimodal reward model that provides high-level semantic signal.
To this end, we propose MARS-Sep, reinforcement learning framework that reformulates mask prediction as stochastic decision-making optimized with multimodal rewards. We cast mask gen- eration actor-only trust-region optimization bins. Training uses a clipped surrogate with entropy regularization and normalized advantages, en- suring stable updates. Instead of focusing on low-level sampling details, our approach leverages multimodal rewards that holistically capture signal fidelity, interference suppression, and perceptual quality across audio, text, and visual queries. To provide reliable reward signals and mitigate re- ward hacking, we further introduce progressive alignment strategy that fine-tunes the multimodal encoder to enhance cross- modal discrimination and stabilize policy learning.
factorized Beta policy on time-frequency
We validate MARS-Sep on VGGSOUND-cleamg and MUSIC-clean+ (Dong et al. 2023 across Text-, Audio-, and prior methods, improving PAR/SIR/SARRSI-SDDRR and notably higher CLAP score, while qualitative analyses highlight clearer suppression non-target sources and better category discrimination.
separation. Extensive experiments show consistent gains over
Our contributions are summarized as follows:
We formulate qunry-conditioned sound separation trust-region reinforcement learning problem, which optimizes a factorized Beta mask policy on time-frequency bins.
We introduce progressive alignment strategy that fine-tunes the multimodal encoder to en- hance cross cross-modal discriminability and provide stable, informative reward signals, thereby mitigating reward hacking.
We demonstrate consistent improvements across SDR/SIR/SAR and SI-SDRi, alongside higher CLAP scores and clearer qualitative separation for both synthesized and in mn-the-wild samples, confirming both signal-level and semantic gains.
RELATED WORK
2.1 UNIVERSAL AND QUERY-CONDITIONES SOUND SEPARATION
Research on isolating sources from complex mixtures has progressed from domain-specific set- tings speech separation (Luo & Mesgarani 2018 2019 Zeghidour & Grangier [021]] Subakan
al. 2021 Linhui et al., 2023) and music source separation (Luo et al. 2017 Rouard et 2023a
Luo & Yu 2023) toward Universal Sound Separation (USS) (Kavalerov et al. 2019 Wisdom al.j 2020), which aims to decompose arbitrary mixtures without class constraints. Key enablers
include permutation invariant training (PIT) for resolving label permutations Yuu et al. 2017: Pos-
tolache et al. 2023) and mixture invariant training (MixIT) for leveraging unlabeled mixtures Wis-
dom et al. 2020h; large-scale resources such as AudioSet (Gemmeke et al.) 2017) further catalyzed
progress. Beyond audio-only models, integrating visual context (Majumder et al. [202]] Tan et al.)
2023) or using class labels as queries (Chen et al. 2022 2023) expands separation capability. In par- allel, Query- Sound Extraction (QBSE) reframes the task as extracting user-specified content while suppressing irrelevant sources. By modality, label-queried systems simple yet closed-set (Chen et al. 2022 Liu et al.) 2024); text-queried approaches such as LASS-Net (Liu et al., (022)) enable open vocabulary but face joint-optimization and generalization hurdles; visual queries ex-
ploit images for grounding (Michelsanti et al.] [202T] Gao & Grauman 202T Ye et al., 2024 Pian
al. 2024); and audio queries use exemplars to target abstract or indescribable sounds (Lee al.
2019 Chen et al., (0022). Recent attempts unify modalities via gross-attention (Chen et al.) 2023)
or hybrid encoders (Rouard et al., 20236), though joint training can limit generalization. Repre- sentative systems include CLIPSEP (Dong et al., 2022), which leverages visual data to improve text-queried training, and AudioSep (Liu et al. 2024), which couples CLAP (Wu et al. 2023) with a 14k-hour corpus achieve strong zero-shot performance. Despite these advances, open-vocabulary robustness, multi-polane operation (extraction and removal within one framework), and scalable multimodal composition (e.g., "dog barking in this image") remain challenging; query-mixxt style training (Cheng et al.) (025d) offers a promising direction but still leaves room for improved seman- tic alignment and stability.
2

---

Published as a conference paper at ICLR 2026
Beyond these discriminative and cross-modal systems, several recently proposed models extend query-conditioned separation into generative or flow-based paradigms. More recent generative ap- proaches such as FlowSep Yuan al. 2025) employ rectified-flow matching synthesize query- consistent sources directly in the latent space, and ZeroSep (Huang et al.) 2025b) performs zero- training, text-conditioned separation using a pre-trained audio-language diffusion backbone. In trast to fully synthesis-baat basedd models, DAVIS (Huang et al. (2024) and its successor DAVIS-FIO (Huang et 2025a) adopts video-conditioned rectified-flow trajectory that remains time-aligned with the mixture, enabling evaluation under standard signal-level metrics. These developments col- lectively illustrate a shift toward richer multimodal conditioning and generative reasoning sound separation, further motivating methods that unify signal fidelity with semantic alignment.
2.2 REINFORCEMENT LEARNING FOR LARGE LANGUAGE MODELS AND MULTIMODAL LEARNING
Reinforcement learning from human feedback (RLHF) has become a central paradigm for aligning large language models with human preferences. Early work (Ziegler et al. 2019 Ouyang et al, 2022) established the practice of training reward model from pairwise preferences and optimizing the policy via Proximal Policy Optimization (PPO) (Schulman et al. 2017). Later methods reduced this pipeline's complexity, including Direct Preference Optimization (DPO) (Rafailov al. 2023), which removes the explicit reward model, and more recent Group Relative Policy Optimization (GRPO) approaches (Guo al. 2025 Zheng al. 2025 Yu al. 2025) that improve stability and
reasoning over PPO-based RLHF.
These alignment strategies have also extended multimodal models. Vision-RI (Huang et al. 2025c) and RI-VL (Zhang et al., (2025a) apply structured rewards to improve grounding and chain- of-thought reasoning, while R1-reward (Zhang et al., 2025b) strengthens multimodal reward mod- eling to enhance semantic fidelity and robustness. Together, these works highlight the applicability of RLHFF-ttlle preference alignment beyond text-only L.MSs.
METHOD
3.1
PRELIMINARIES
3.1.1
UNIVERSAL SOUND SEPARATION
Universal Sound Separation (USS) is the task of isolating individual sound sources from arbitrary audio mixture, without prior knowledge of the number or types of sources. Unlike domain -emain-specifi separation exemplified by speech enhancement music source separation, USS aims to generalize across diverse acoustic conditions and sound categories.
Formally, let the observed mixture signal be denoted as zzt) = ΣE (t), where st(t) represents the waveform of the i-th underlying source and N is the (unknown) number of sources. The goal of USS is to estimate set of signals {t) such that each (t) corresponds to one of the true sources (t), up to permutation and possibly scaling. That is, xx1)= Σ (t).
To achieve this, separation models typically operate in the time-frequency domain or directly on the waveform. Let X e CFXT denote the short-time Fourier transform (STFT) of the mixture, where F and T are the frequency and time dimensions. USS methods aim to construct masks (M, E [0, 1FFX such that $, M, @ X with denoting element-wiss multiplication. The inverse STFT is then applied to obtain time-domain estimates (t))).
3.1.2 OMNISEP: UNIFIED OMNI-MODALITY SOUND SEPARATION WITH QUERY-MIXUP
OmniSep Cheng et al., 2025d) provides the base separation architecture: a frozen ImageBind Girdd har et al. 2023) encoder maps audio/image/text inputs to shared feature space and a Separate-Ne (U-Net over STFT magnitudes) predicts masks. Let QA be the audio query, Qy be the visual query, and Qr be the text query. Training in OmniSep mixes queries across modalities via Query-Mixup
Wa QA + W.Qv + w.QT
Way Woy wi E [0,
W. + we + Wt
3

---

Published as a conference paper at ICLR 2026
and combines intermediate masks into final masks M through channel-wise weighting; the super- vised objective is the sum of weighted binary cross-entropy (WBCE) losses on ideal masks. Om- niSep also supports negative queries by introducing negative query weight to adjust the query as Q' = (1 + aq aQN to remove interfering content, and, more broadly, frames omni-modal querying with frozen ImageBind backhone within unified audio/lexx/iideo-quereed separation paradigm.
3.2 MARS-SHE REINFORCEMENT LEARNING FOR MULTI-SOURCE UNIVERSAL SOUND SEPARATION ENHANCEMENT
As established in our introduction, we frame sound separation as preference alignment problem, analogous to the RLHF pipeline for LLMs. To operationalize this framework (see Figure I. we must define three core components:
Base Policy (Tg). The sound separation model that takes the state (mixture spectrogram X and query Q) and produces action (the mask M).
Preference Reward Model (Rt) The model that scores the alignment. We use our progressively- aligned multimodal encoder provide this scalar reward, measuring semantic consistency between the separated audio and the query.
The policy updating algorithm that steers the base policy to maximize the reward. We adapt stable, trust-region policy optimization algorithm for this purpose.
Optimization Process.
During the training stage, the separator produces mask prediction that parameterizes the new policy (Onew, Pnem), while snapshot from the previous step provides the old policy (Catdd Bota). Masks are sampled from the old policy to ensure stable training, and the separated audio compared against audio/text/video queries in shared embedding space using fine-tuned ImageBind encoder with multimodal fusion module. The cosine similarity between the separated audio embedding and the fused query embedding yields scalar reward, which is normalized by running baseline and group- relative scaling to form advantages. These are combined with the log- pyg-robability ratio between old and new policies (i.e., logaooid and log under clipped surrogate objective, supplemented by entropy regularization for exploration and a KL penalty for stability. The current policy is then snapshotted serve the old policy in the next iteration.
3.2. TRUST-REGION-TTEE POLICY OPTIMIZATION FOR STABLE MASK SAMPLING
We formulate conditoond sound separation standard Markov Decision Process (MDP) (S, A, T,, r). The state space consists of the mixture spectrogram and the query Q, while the action space A corresponds to masks M. The transition T is deterministic, 9 s(X, recon- structing the waveform. The reward function R is defined by the similarity between the separated waveform and the multimodal query. Our goal is to train policy TM(M X, Q) that maximizes the cumulative designed reward. Let X be the magnitude spectrogram of mixture and 0 the separator parameters that produce a deterministic mask proposal (x Q) E {o, H| xx xK where H the number of frequency bins, W is the number of time frames and K denotes the number of target sources be separated. We turn this proposal into a stochastic policy over masks by factorized Beta distribution
(M(M X, Q)
II Beta BBeaaM Olhaoo Bn,, a a= a=1+K18 3 1+1+6[1 PP), (2) h,
with concentration scale O. Reparameterizee sampling M To(- X,Q) yields mask that is close to the proposal yet retains exploration. The masked magnitude is combined with the mixture phase to reconstruct waveform structed analogously from ideal masks (only for reward computation during training). The factorized Beta parametrization aligns with the [0, support of masks and offers transparent exploration- exploitation knob via the concentration scale K, which we anneal to avoid degenerate near- binary masks early in training.
s(X M); the ground-truth component y² is recon-
4

---

Published as a conference paper at ICLR 2026
ROCOVVT
Cosine Similurity
Recownne aned Wavefons
Audio
Sampled Mask
Firhoolings
sample
Iuage Query
log Fold
log Knew
Image
Rewand
Ding
Modal Pooling
PPU cli
Quay
Fraed Query
Polley
New Poliey
Einhoskiings
snatsho
build
counpy
Audio Query
backgeop
Proficaal Mosk
poslict
Sopertor
Model
Faul Loss
Fronen Medialz
Leamable Module
Rewar Computation
Policy Optimization Loop
Optimization Signals
The reinforcement learning loop MARS-Sep. The separator generates stochastic mask actions from Beta-distributed policy, while frozen snapshot serves the old policy for stable optimization. Multimodal rewards derived from audio, text, and visual embeddings guide policy updates, with entropy and KI regularization enhancing exploration and stability.
Figure 1:
At each training step we sample from frozen old policy Teaa (a snapshot from the previous step), reconstruct the waveform ý from the sampled mask M, and compute a scalar reward R against the query-conditioned targets. A moving-average baseline b yields the advantage stabilize updates, adopt clipped trust-region style surrogate in the spirit of PPO, using GRPO of the advantage. Concretely, define the importance ratio
R To
TM(M x,Q) exp (log (M( log mama (M)): No (M X,Q)
(M(M)
and let Ā A-p(A) be the group-relative advantage. The clipped surrogate objective with entropy oAA)+s
regularization and KL penalty is
(O) = EM--e min (rs(M) A, dip(ra(MM),11,,1422 A) + Au H((T)) AKL. KL(Te |l mesia
and the loss minimized in training (0) Jatip ( Here H denotes entropy of the factorized Beta policy, is the clipping range, and λи AKL 0 control exploration and the trust region, respectively. In practice, log æ factorizes over bins; we broadcast Á to the mask shape and estimate expectations with Monte Carlo samples per iteration. The old policy Taaia updated current snapshot after each optimization step, yielding single-epoch PPO update that preserves the original training loop while markedly improving stability.
This design preserves the benefits of reinforcement learning while avoiding the instability of plain policy gradients, leading to more reliable convergence for mask-based separation. Importantly, achieves this without introducing additional value networks complex estimators, keeping the optimization efficient while directly tying policy updates to multimodal reward signals.
3.2.2 MULTIMODAL REWARD
To optimize the separation policy, we define a reward that measures how well the separated audio waveform ý semantically matches the target query across modalities in unified embedding space
5

---

Published as a conference paper at ICLR 2026
provided by ImageBind. We project audio waveforms, text queries, and sampled video frames into a shared space via ImageBind encoders (a(:), (t(+), and (-). respectively.
All embeddings are fg-normalized, and similarity is measured with cosine similarity:
sim(u, v) = ||/|| ||i||
(5)
Unimodal rewards. Given separated audio ÿ. ground-truth audio y², a text query embedding t*, and video frames embedding v*, we compute:
Ta-+a = si) Pt-va = (0) To-+a sim().(00) (6)
These terms measure acoustic fidelity, semantic alignment with text, and consistency with visual context, respectively.
Aggregation strategy: Query-pooling. Instead of comparing unimodal similarities separately, fuse the target-side multimodal embeddings into a joint representation using Multi-Modal Low- Rank Bilinear Pooling (MLBP) ((Kim et al. 2017) and compare it directly to the separated audio emhedding. Specifically, 2' MT.P @o yy" (t"), (. and the scalar reward is R si)( =*). Detailed implementation MLBP is described Appendix A
The motivation for pooling is to ensure the reward captures joint multimodal consistency rather than independent unimodal matches. Audio, text, and vision carry complementary cues: audio encodes acoustic details, text conveys semantic categories, and vision provides environmental context. If each compared separately, the reward may overweight a single modality. By applying low-rank bilinear pooling, explicitly model multiplicative interactions between modalities (c.g., a textual query specifying an instrument that also appears visually). This fused target anchor =* encourages the separated audio simultaneously align with all modalities, yielding more semantically faithful and robust rewards. This asymmetric design mirrors the implementation: the separated audio re- mains in its native representation while the target modalities are fused into semantic anchor. This reduces variance from stochastic mask sampling and provides stable training signal.
In the section, describe the progressive fine-tuning curriculum used initialize this policy with robust modal alignment before RL. optimization.
3.3
MULTIMODAL. ENCODER FINE- TUNING VIA PROGRESSIVE ALIGNMENT
"Plapin Congas
-N/ Chapae
Fox Barkingg
-1/pa-- Chepuat
M- (Hleni
(Tee (Feetutiiee
Audio Proj Text Proj è
Image Proj d
Andio Proj Text Ptoj
@ Image Proj ?
Andio Proj
Text Proj
@ Emage Proj
In6NNC
InfoNCE: Triplet Consisency(Partial
InfoNCE Triplan[Partial 51+52)
Positre Fais
Negative
Segnest
Modale
Frunzi Modal:
Figure 2: Progressive fine-tuning strategy for sound source discrimination and separation. Encoders remain frozen while task-specific heads are gradually unfrozen and each stage builds on the best checkpoint from the previous one. The two latter stages are trained with a fraction of the former aligned paired data to avoid catastrophic forgetting.
To enhance discrimination between same- and different-souree signals, we apply multimodal con- trastive fine-tuning objective on ImageBind: for each audio segment, positives pair it with an audio clip, video frame, or label text of the same class, while negatives pair it with content from other classes; by optimizing contrastive loss over these multimodal pairs, the model is encouraged to bring embeddings of semantically consistent sources closer together, while pushing apart those be- longing different sound categories.

---

Published as a conference paper at ICLR 2026
The fine-tuning process, shown in Figure is organized into three sequential stages, each with a distinct training objective, and each stage begins from the best-perforiing checkpoint obtained in the previous one. This curriculum design allows the model to move gradually from semantic grounding to intra-class discrimination and finally to multimodal alignment.
3.3.1 SAMPLE PAIR CONSTRUCTION FOR CONTRASTIVE LEARNING
We build multimodal contrastive pairs with the audio clip as the anchor. Positives pair the anchor with (i) its label text, (ii) another audio instance from the same class, (iii) frames from the tempo- rally aligned video segment. Negatives use labels, audio, or frames from other classes or temporally mismatched segments. This unified scheme pulls matched embeddings together while pushing mis- matched ones apart.
In the first stage, the model is trained to align audio signals with their corresponding textual la- bels. At this point, all modality encoders (audio, text, vision) and postprocessors are kept frozen preserve their pretrained representations, while only the projection heads together with shared temperature parameter are unfrozen and updated. The training objective is a symmetric InfoNCE loss, which encourages paired audio-text embeddings converge while repelling mismatched pairs. Formally, for a batch of size N, with normalized embeddings 2 and z), the loss is defined as
N
(xp((22/241/T)) + log exp((=t, 24) /T) Ej=1 exp((=) (xp((=//=/)//)) =f)/T) Σラ= (xp((=/) =d)
Csi
log
2N
where 7 is learnable temperature scaling factor. This stage establishes the initial semantic ground- ing of audio in the shared embedding space while limiting parameter updates to lightweight layers.
The second stage focuses on audio-audio discrimination. Again, all encoders and postprocessors remain frozen, while the audio projection head and shared temperature are unfrozen to adapt rep- resentations for finer discrimination. Given an anchor audio clip, another clip of the same class is selected as the positive, while clip of different class serves as the negative. The objective com- bines several terms: an InfoNCE loss to enforce alignment between same-class pairs, a triplet loss to guarantee margin between positive and negative similarities, and consistency loss to ensure invariance to perturbations. Specifically,
Cs2 = A LIntNNCE(11 22) + A2 max(0, cos(( - cos(21, +m) + >a| - 22
where ž1 and ž2 are embeddings of audio from the same class, 2, is a negative sample, and m is a margin hyperparameter. To mitigate catastrophic forgetting, fraction of audio-text pairs from stage one are mixed into training, ensuring that semantic alignment is preserved while discrimination improves.
The third stage introduces visual grounding through audio-video pairs. All encoders and postpro- cessors remain frozen, but the audio and vision projection heads unfrozen and trained jointly, along with the shared temperature. Uniformly sampled frames from the corresponding video pro- vide the positive modality, while frames from other videos or temporally misaligned portions serve as negatives. The objective again includes an InfoNCE term between audio and video embeddings, along with a triplet loss that uses mismatched video frames as hard negatives. To maintain previ- ously acquired capabilities, the objectives from stage one and stage two are partially incorporated. The overall loss can expressed as
Csa = μ4l LintNNCE(za z,t) + μ2 LTriplet (Fas x,, 2n) + μ3 Lsi + μ4 Ls2,
where 2, and 2,- embeddings of positive and negative video samples, and the coefficients (4: balance the relative contributions.
At the end of stage and 2, the best checkpoint is used initialize the subsequent stage. Stage one therefore provides a semantic anchor via audio-text alignment, stage two sharpens class discrimina- tion through audio-audio comparison, and stage three consolidates multimodal grounding by linking
7

---

Published as a conference paper at ICLR 2026
audio with vision. This progressive fine-tuning procedure, in which encoders are kept frozen and only rele-aelevant heads and scaling parameters are successively unfrozen, ensures that the model evolves in stable and interpretable manner, acquiring increasingly sophisticated capabilities for sound source discrimination and separation.
4 EXPERIMENTS
4.1 EXPERIMENT SETUP
We evaluate our approach on widely used audio-visual separation benchmarks, VGGSound (Chen et al. 2020) and MUSIC (Zhao et al. 2018). VGGSound is large-scale sealsscae dataset with over 300 sound categories collected from You videos, offering substantial acoustic and visual diver- sity; MUSIC is a smaller dataset of solo and duet music performance videos spanning variety of instruments, which emphasizes structured harmonic signals and thus provides complementary and cross-domain setting. Training and data preprocessing details are provided in Appendix B
We adopt standard separation metrics for evaluation, including Senmalto-Interferenne Ratio (SIR), Signal-to-Distortion Distortion Ratio (SDR), Airtiltto-Ariffctt Ratio (SAR), and Scale-Invariant SDR im- provement (SI-SDRi
We additionally report the CLAP score, which measures the semantic consistency between the sep- arated audio and its textual label using a contrastive language-aude pretraining model. While tra- ditional signal- metrics evaluate separation quality in terms of distortion, interference suppres- sion, and artifact reduction, the CLAP score complements them by capturing whether the separated waveform preserves the intended semantic content. Together, these metrics provide comprehensive assessment of both perceptual signal fidelity and semantic correctness of the separation output.
We select five representative baselines for comparison. CLIPSep-NIT (Dong et al., (023) (the noise-invariant training version released by the authors) employs CLIP embeddings to guide sep- aration with either visual or textual cues. AudioSep (Liu al., 2024) adopts large-scale training with text queries to achieve strong open-domain generalization. OmniSep (Cheng et al. (025d) in- tegrates multiple modalities into unified separation framework, highlighting the potential of multi- modal fusion. LASS-Net (Liu et al. 2022) addresses the Lanuuage-querie audio source separation task via a joint Transformer-based query encoder and ResUNet separation network, conditioned on natural-language descriptions. Second, iQuery (Chen et al. 2023) formulates instruments as learn- able audio-query prototypes and leverages visually-named cross-modal attention to disentangle and separate instrument sounds from videos.
In addition to the above, we further include two generative approach and one zero-shot approach for comparison. FlowSep (Yuan et al. 2025) introduces generative rectified flow matching (RFM) model the VAE latent space to synthesize separated audio from noise under language query guid- ance, thereby enabling cleaner outputs in highly overlapping scenarios. ZeroSep (Huang et al,, 2025b) proposes a zero-training audio separation framework that repurposes pre-trained text-guided audio diffusion models perform open-set language-queried separation without task-specific fine- tuning, We also include DAVIS-Flow (Huang et al.) 2025a).. a visually-guided generative audio- visual separation framework that applies flw-matching synthesize separated spectrograms from noise.
4.2 MAIN RESULTS
We first present results on the VGGSound-clean+ dataset, refined subset of VGGSound that fil- ters out noisy annotations and ensures higher-quality alignment between audio and visual streams. MARS-Sep demonstrates the strongest overall performance across text, audio, image, and composed queries (Table [ It attains the highest SDR and CLAP score across modalities and achieves the best SI-SDRi in three settings (tied for best under audio queries). On subset of measures, the bal- ance shifts toward OmmiSep with notably higher SIR for audio queries and higher SAR for image and composed queries, though the margins are modest. Taken together, these results indicate that
A detailed description of the SI-SDR and SI-SDRi calculation procedure presented in Appendix
8

---

Published as a conference paper at ICLR 2026
mixed
larget
interf
baseline
OLIS
Figure 3: Log-mel spectrograms of separated audio from different query modalities on VGGSOUND-clann+ dataset. The target source is "cattle bovinae cowbell". From left to right: (a) Mixture of "cattle bovinae cowbell" and "tap dancing"; (b) Ground-truth "cattle bovinae cowbell"; (c) Interference "tap dancing"; (d) Separation with text query by the baseline model; (e) Separation with text query by our model.
reinforcement learning with multimodal rewards improves semantic fidelity and signal quality on balance, while remaining competitive or superior on the remaining metrics.
To further validate cross-domain generalizability, we evaluate on MUSIC-cleant, which is derived from the MUSIC dataset and focuses on solo and duet instrumental performances. Compared with VGGSound, MUSIC emphasizes structured harmonic and timbral patterns rather than broad acoustic diversity, making it complementary benchmark. As reported in Table MARS-Sep again achieves clear gains over competing approaches under all query modalities, especially notably higher CLAP scores than all baselines, highlighting that our separated signals preserve semantic consistency with the intended text source, confirming that our method not only handles open-domain separation but also excels in structured, music-centric scenarios.
We provide log-mel spectrograms of a representative sample from the test set of VGGSOUND in Figure 3 for result visualization. Compared with the baseline, MARS-Sep suppresses non-target components more selectively, preserving the target's harmonic ridges and temporal continuity in- stead of the blocky dropouts visible in the baseline spectrogram.
We additionally report CLAP scores for the generative separation frameworks FlowSep and Ze- roSep, which do not produce mask-based or waveform-aligned outputs and are therefore not directly comparable under signal-level metrics. As shown in Table these generative models exhibit sub- stantially higher variance and often inflated CLAP similarity due to their synthesis- based decoding. In contrast, MarsSep provides far more stable and consistent semantic alignment, while also abling evaluation under standard separation metrics. To further assess the quality of the separated audio, introduce CLAP audio scores, which measure the semantic similarity between the sepa- rated audio and the target audio. This additional evaluation highlights MarsSep's ability to maintain strong semantic fidelity to the target, even when compared to models with synthesis-basee outputs, where CLAP scores tend to be more volatile.
As for ablation studies, the impact of hy perparameeee settings, modality query embedding fusion module, progressive finetuning strategy, discrimination ability of multimodal encoder with or without finetuning and other significant factors are discussed in Appendix E.
More samples including synthesized audio and in-the-wild audio are shown in Appendix E.6
CONCLUSION
We present MARS-Sep, multimodal-aligned reinforced sound separation approach that frames sound separation as stochastic decision-making guided by multimodal rewards, enforcing semantic consistency with audio, text, and visual queries rather than optimizing only signal-level metrics. Built on a trust-region style policy with progressive aligmment multimodal encoders, achieves stable training and strong cross-modal discrimination. Experiments VGGSOUND-cleans and MUSIC-cleant show consistent improvements in fidelity and semantic alignment, advancing se- mantically aware separation that better matches perceptual quality.

---

Published as a conference paper at ICLR 2026
Table 1: Comparison of sound separation performance among different methods on VGGSOUND-
cleant dataset. Metrics include SIR, SDR, SAR, and SI-SDRi (all in dB), and CLAP score (%).
VGGSOUND-clean+
Methods
Mean SDRT
Mean SIRT
Mean SAR
Mcan S1-SDRit
Mean CLAP;
Text Query Sound Separation
LASS-Net (Liu ct al. 2022
3.98#1.02
7.63+0.85
4,24+1.00
4.25+0.76
5.12#0.71
CLIPSEP-NII (Dong el al.
2023
2.71
4.58+1.37
13.60=0.68
2.41#0.53
7.97#0.94
AudioSep LLu čt al. 2022
6,26+0.87
8.69=0.90
12,85=0,92
4.01=0.59
8.21=0.96
DAVIS-Flow [Tlunn al.
2025a
6.60#1.02
8,99+0.93
13,48=0.85
4.32+1.03
6.57=0.94
OmniSep Cheng
2025d
6.70#0.66
9.04=0.98
13.61
4.38+0.48
8.98=0.89
MARS-Sep (ours)
6.91=0.68
9.14=1.00
13.73=0.77
4.55+0.44
9.0320.94
Audio Query Sound Separation
OmniSep (Cheng 2025d
7.15+0.65
11.65=1002
11.84=0.81
4.35+0.52
8. 8.60=0.91
MARS-Sep (ours)
7.3320.67
11.63 1.00
12.0020.84
4.3620.50
8.91=0.91
Image Query Sound Separation
CLIPSEP-NIT (Dong et al. 2023
4.61=0.82
8.11+1.32
12.06=0.78
3,4820.60
8.50=0.92
iQuery (Chen al. 2023
0.
9.59+0.85
13.45=1.01
3.77+0.46
1.08
DAVIS-FIow Flow (1 [tanng ml
2025a
6.52+1.01
9,87+0.98
13.54
4.32+0.96
8 1.89
OmniSep Cheng et al. 2025d
6.66+0.65
10.00
13.73=0.76
4.43+0.50
8.79=0.89
MARS-Sep (ours)
6.93=0.67
10.18:1.04
13.41=0.72
4.57=0.47
9.19=0,91
Composed Omni-modal Querv Sound Seporation
OmniSep Cheng et al. 2025d
7.79=0.72
10.76#1.00
14.53=0.93
5. 6.
8.85=0.92
MARS-Sep (ours)
7.93=0.75
10.65=1.00
14.49=0.95
5.20=0.45
9.22=0,90
Table Comparison of sound separation performance among different methods on MUSIC-clean+ dataset.
MUSIC-cleant Mcan SIR
Methods
Mcan SDR
Mean SAR*
Mcan SI-SDRit
Mcan CLAP,
Text Query Sound Separation
LASS-Net (Liu al., [202
9.98+0.99
14.63&1.17
12.24
8.99+0.76
4.92+0.71
CLIPSEP-NIT (Dong ci al.
2023
11.03=0.98
16.40= 1.38
17.37=0.97
7.53+0.90
5.29=0.96
AudioSep [Tuu č7 at. 2022
01.23=0.92
16.90+ 1.31
17.29=0.90
8.56+0.84
5.48=1.02
DAVIS-Flow [Huang al.
2025a
8.97=0.85
17.88=0.95 15.99=1.55
9,23+1.03
5.53+0.75
OmniSep Cheng al.
2025d
12.37=0.85
1.
17. 07.9600.90 90
9.18+ 79
5,41#0.98
MARS-Sep (ours)
12.91=0.93
17.61=1.17
18,28+0.93
9.85+0.82
6.1820.93
Audio Query Sound Separation
OmniSep (Cheng al. 2025d
10.37=0.86
17.76+1.005
14.51
7.18+1.07
5,39
MARS-Sep (ours)
11.73+0.88
19.65+1.14
15.25=0.86
8,38+1.03
5.64=1.06
Image Query Sound Separation
CLIPSEP-NIT (Dong et al. 2023
11.64=0.98
18.4##1.20
17.04=1.05
8.27+0.94
5.97=0.94
i- i-Query (Chen et al. 2023
12.52=0.84
17.05+1.01
17.65=1.23
9,4420.99 99
5.50+1.22 1.2
DAVIS-Flow (Huang et al. 2025a
13.52+1.144
19.00+ 1.02
17.54
10.3240.8
5.99+1.00
OmniSep Cheng el at. 2025d
13.03=0.96
18.97:1.16
17.88+1.00
10.21
6.53+1.03
MARS- Sep (ours)
13.6421.06
19.24
18.05=1.06
10.70:0.89
1.94
Composed Omni-modal Query Sound Separation
OmniSep Cheng al. 2025d]
13.29=0.99
19.55+1.17
17.88=0.99 .6
10.2 10.22+4..0
6.35&1.05
MARS-Sep (ours)
13.8920.98
1..90
17.9920.97
10.78:0.81
6.8220.99
Table 3: CLAP score comparison (Text-queried) across generative separation frameworks on the MUSIC-cleant and VGGSOUND-clean+ datasets.
Method
Dataset
CLAP, score (%)
CLAP, score (%)
ZeroSep
20.02 + 15.14
22.86 + 18.55
FlowSep
MUSIC-clean+
10.67 14.17
39.25 + 29.86
MarsSep (Ours)
6.18 + 0.93
21.56 + 1.08
ZeroSep
15.91 + 14.17
22.65 + 19.98
FlowSep
VGGSOUND-clean+
8.84 + 13.27
56.07 + 19.57
MarsSep (Ours)
9.03 - 0.94
18.70 + 1.23
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
This work was supported by the National Natural Science Foundation of China under Grant No. U25B2064, the "Pioneer" and "Leading Goose" R&D Program of Zhejiang under (Grant No. 2025C02110) 10), Public Welfare Research Program of Ningbo under (Grant No. 2024S062), and Yongjiang Talent Project of Ningbo under (Grant No. 2024A-161-G).

---

Published as a conference paper at ICLR 2026
REFERENCES
Honglie Chen, Weidi Xie, Andrea Vedaldi, and Andrew Zisserman. Vggsound: A large-scale audio- visual dataset. In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 721-725. IEEE, 2020.
Jiaben Chen, Renrui Zhang. Dongze Lian, Jiaqi Yang, Ziyao Zeng, and Jianbo Shi. iquery: Instru- ments queries for audio-visual sound separation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Paltern Recognition, pp. 14675-14686, 2023.
Ke Chen, Xingjian Du, Bilei Zhu, Zejun Ma, Taylor BarggKirkpatrick, and Shlomo Dubnov. Zero- shot audio source separation through query-based learning from weakly-labeled data. In Proceed- ings of the AAAI Conference on Artificial Intelligence, volume 36, pp. 4441 1449, 2022.
Xize Cheng, Tao Jin, Linjun Li, Wang Lin, Xinyu Duan, and Zhou Zhao Opensr: Open-modality speech recognition via maintaining multi-modality alignment, 2023. URL. http::
org/abs/23055 2306 06410
Xize Cheng, Dongjie Fu, Xiaoda Yang, Minghui Fang, Ruofan Hu, Jingyu Lu, Bai Jionghao, Zehan Wang, Shengpeng Ji, Rongjie Huang, Linjun Li, Yu Chen, Tao Jin, and Zhou Zhao. Omnichat: Enhancing spoken dialogue systems with scalable synthetic data for diverse scenarios, 2025a.
URL htttts:://aattttt wttpg///gg/..../o//o/o 01384
Xize Cheng. Ruofan Hu, Xiaoda Yang, Jingyu Lu, Dongjie Fu, Zehan Wang, Shengpeng Ji, Rongjie Huang, Boyang Zhang, Tao Jin, et al. Voxdialogue: Can spoken dialogue systems understand information beyond words? In The Thirteenth International Conference on Learning Represen- tations, 2025b.
Xize Cheng, Slytherin Wang, Zehan Wang, Rongjie Huang, Tao Jin, and Zhou Zhao. Unleashing the power of natural audio featuring multiple sound sources, 2025c. URL https://arxxii
17782
Xize Cheng, Siqi Zheng, Zehan Wang, Minghui Fang, Ziang Zhang, Rongjie Huang, Shengpeng Ji, Jialong Zuo, Tao Jin, and Zhou Zhao. Omnisep: Unified omni-modality sound separation with query-mixup. The Thirteenth International Conference on Learning Representations, 2025d.
Ching-Yos Yu Chiu, Joann Ching, Wen-Yi Hsiao, Yu-Hua Chen, Alvin Wen- Yu Su, and Yi-Hsuan Yang- Source separation-based data augmentation for improved joint beat and downbeat tracking. In 2021 29th European Signal Processing Conference (EUSIPCO), pp. 391-395, 2021. 2391 1091199UUSSC54332222.00009
doi: 10.
Hao-Wen Dong, Naoya Takahashi, Yuki Mitsufuji, Julian McAuley, and Taylor Berg- Kirkpatrick. Clipsep: Learning text-queried sound separation with noisy unlabeled videos. In The Eleventh International Conference on Learning Representations, 2023.
Dongjie Fu, Xize Cheng, Xiaoda Yang, Wang Hanting, Zhou Zhao, and Tao Jin. Boosting speech recognition robustness to modality-distortion with contrast-augmenteee prompts. In Proceedings of the 32nd ACM International Conference on Multimedia, MM '24, pp. 3838-3847, New York, NY, USA, 2024. Association for Computing Machinery. ISBN 9798400706868, 1664647.368 1347. URL http:ttt:://oo.rg/g//// 10 wtt.z///...111///11 111.....111111111 3681347
doi: 10.1145/
Dongjie Fu, Xize Cheng, Linjun Li, Xiaoda Yang, Lujia Yang, and Tao Jin. PACHAT: Persona-aware speech assistant for multi-party dialogue. Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, and Violet Peng (eds.), Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pp. 29325 29342, Suzhou, China, November 2025. Association for Computational Linguistics. ISBN 179-8-89176-3322
main. emn wwwneennmeeeeeee mammlppmmiin. 1492/
doi: 10.186533
1492. URL
Ruohan Gao and Kristen Grauman. Visualvoice: Audio-visual speech separation with cross- modal consistency. In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 15490-15500. IEEE, 2021.
12

---

Published as a conference paper at ICLR 2026
Jort F Gemmeke. Daniel PW Ellis, Dylan Freedman, Aren Jansen, Wade Lawrence, R Channing Moore, Manoj Plakal, and Marvin Ritter. Audio set: An ontology and human-labeles dataset for audio events. In 2017 IEEE international conference acoustics, speech and signal processing (ICASSP), pp- 776-780. IEEE, 2017.
Rohit Girdhar, Alaaeldin El-Nouby, Zhuang Liu, Mannat Singh, Kalyan Vasudev Alwala, Armand Joulin, and Ishan Misra. Imagebind one embedding space bind them all. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 15180-15190, 2023. doi: 10. 109/00PR52229 202.00145..
Daya Guo, Dejian Yang, Haowei Zhang. Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12945 2025.
Chao Huang, Susan Liang, Yapeng Tian, Anurag Kumar, and Chenliang Xu. High-quality visually- guided sound separation from diverse categories. In Proceedings of the Asian Conference on Computer Vision (ACCV), pp. 35 49, December 2024.
Chao Huang, Susan Liang, Yapeng Tian, Anurag Kumar, and Chenliang Xu. High-quality sound separation across diverse categories via visually-guided generative modeling, 2025a. URL
org/ 2509.
Chao Huang, Yuesheng Ma, Junxuan Huang, Susan Liang, Yunlong Tang, Jing Bi, Wenqiang Liu, Nima Mesgarani, and Chenliang Xu. Zerosep: Separate anything in audio with zero training. arXiv preprini 250.::2500..3625. 2025b.
Wenxuan Huang, Bohan Jia, Zijie Zhai, Shaosheng Cao, Zheyu Ye, Fei Zhao, Zhe Xu, Yao Hu, and Shaohui Lin. Vision-rl: Incentivizing reasoning capability in multimodal large language models. arXiv preprint arXiv:2503.00 06749, 2025c.
Shengpeng Ji, Yifu Chen, Minghui Fang, Jialong Zuo, Jingyu Lu, Hanting Wang. Ziyue Jiang, Long Zhou, Shujie Liu, Xize Cheng, Xiaoda Yang. Zehan Wang, Qian Yang, Jian Li, Yidi Jiang, Jingzhen He, Yunfei Chu, Jin Xu, and Zhou Zhao. Wavchat: A survey of spoken dialogue models, 2024. URL https////aaaxi...c///////111/311
Joonas Kalda, Ricard Marxer, Tanel Alumäe, Hervé Bredin, et al. Pixit: Joint training of speaker diarization and speech separation from real-world multi-speaker recordings. In Proc. odyssey 2024, pp. 122,, 2024.
Ilya Kavalerov, Scott Wisdom, Hakan Erdogan, Brian Patton, Kevin Wilson, Jonathan Le Roux, and John R. Hershey. Universal sound separation. In 2019 IEEE Workshop Applications of Signal Processing to Audio and Acoustics (WASPAA), pp. 175, 2019. doi: 10. 00.119//ASPAA.2011 8937253.
Jin Hwa Kim, Kyoung Woon On, Woosang Lim, Jeonghee Kim, Jung Woo Ha, and Byoung Tak Zhang. Hadamard product for low-rank bilinear pooling. In 5th International Conference on Learning Representations, ICLR 2017, 2017.
Yungyeo Kim and Joon-Hyuk Chang- Acoustic-scene-aware target sound separation with sound embedding refinement. IEEE Access, 12:71606-71616, 2024. doi: 10. 10.1109/ACCESS.2024 3402736.
Jie Hwan Lee, Hyeong-Seok Choi, and Kyogu Lee. Audio query-based music source separation. arXiv preprint arXiv: 1908.0653, 2019.
SUN Linhui, LIANG Wenqing, ZHANG Meng, and LI Ping' an. Monaural speech separation using dual-output deep neural network with multiple joint constraint. Chinese Journal of Electronics, 32 (3)44935506, 2023. doi: 10. 222.99//j..20222000ooooooo
URL. https://cjee ejournal .org.
cn/ an/en/aas htt.artiii www.ceeeeelleeooooooooooee /10 2391 9/ jtw..//...../////....ee... 2022. 00. 110
Xubo Liu, Haohe Liu, Qiuqiang Kong. Xinhao Mei, Jinzheng Zhao, Qiushi Huang. Mark D Plumb- ley, and Wenwu Wang. Separate what you describe: Language-queried audio source separation. In Proc. Interspeech 2022, pp. 1801-1805, 2022.
13

---

Published as a conference paper at ICLR 2026
Xubo Liu, Qiuqiang Kong, Yan Zhao, Haohe Liu, Yi Yuan, Yuzhuo Liu, Rui Xia, Yuxuan Wang, Mark D Plumbley, and Wenwu Wang. Separate anything you describe. JEEE/ACM Transactions on Audio, Speech, and Language Processing, 2024.
Yi Luo and Nima Mesgarani. Tasnet: Time-domain audio separation network for real-time, single- channel speech separation. In 2018 IEEE International Conference on Acoustics, Speech and doi: 10. 10.11000CASSP22018346...1.
Signal Processing (ICASSP), pp. 696-700, 2018.
Yi Luo and Nima Mesgarani. Conv-tasnet: Surpassing ideal time- frequency magnitude masking for speech separation. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 27(8): 1256-1266, 2019. doi: 10.1009TAASLP:2192295555
Yi Luo and Jianwei Yu. Music source separation with band-split mnn. JEEE/ACM Transactions on Audio, Speech, and Language Processing, 31:1893-1901, 2023. doi: 10.1 1... 3271145.
Yi Luo, Zhuo Chen, John R Hershey, Jonathan Le Roux, and Nima Mesgarani. Deep clustering and conventional networks for music separation: Stronger together. In 2017 IEEE international conference on acoustics, speech and signal processing (ICASSP), pp. 61-65. IEEE, 2017.
Hao Ma, Zhiyuan Peng, Xu Li, Mingjie Shao, Xixin Wu, and Liu. Clapsep: Leveraging con- trastive pre- trained model for multi-modal coney--oonittooned target sound extraction. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2024.
Tanvir Mahmud and Diana Marculescu. Opensep: Leveraging large language models with textual inversion for open world audio separation. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp. 13244-13266) 2024.
Sagnik Majumder, Ziad Al-Halah, and Kristen Grauman. Move2hear: Active audio-visual source separation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 275-285, 2021.
Ethan Manilow, Gordon Wichern, Prem Seetharaman, and Jonathan Le Roux. Cutting music source separation some slakh: A dataset study the impact of training data quality and quantity. pp. 45-49, 10 2019, doi: 10. 109/AASPAA.20199999777
Daniel Michelsanti, Zheng-Hua Tan, Shi-Xiong Zhang, Yong Xu, Meng Yu, Dong Yu, and Jesper Jensen. An overview of deep-learning-bee based audio-visual speech enhancement and separation. JEEE/ACM Transactions on Audio, Speech, and Language Processing, 19:1368-1396 2021.
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to fol- low instructions with human feedback. Advances in neural information processing systems, 35: 27730-27744, 2022.
Weiguo Pian, Yiyang Nan, Shijian Deng, Shentong Mo, Yunhui Guo, and Yapeng Tian. Continual audio-visual sound separation. Advances in Neural Information Processing Systems, 37:76058- 76079, 2024.
Emilian Postolache, Jordi Pons, Santiago Pascual, and Joan Serrà. Adversarial permutation invariant training for universal sound separation. In ICASSP 2023 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1-5, 2023. doi: 10.1109/CCASSP49337. 2023.1009688 10096847.
Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly reward model. Advances in Neural Information Processing Systems, 1.-33728-53711 2023.
Simon Rouard, Francisco Massa, and Alexandre Défossez. Hybrid transformers for music source separation. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Sig- nal Processing (ICASSP), pp. 1-5. IEEF, 2023a.
14

---

Published as a conference paper at ICLR 2026
Simon Rouard, Francisco Massa, and Alexandre Défossez. Hybrid transformers for music source separation. In ICASSP 2023 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1-5, 2023b. doi: LLLL00/CAASSP9933222211000000990
Jonathan Roux, Scott Wisdom, Hakan Erdogan, and John R. Hershey. Sdr - half-baked or well done? In ICASSP 2019 2019 IEEE International Conference on Acoustics, Speech and Signal doi: 10.1 10.1109/CCASSP2211...6 898888851...
Processing (ICASSP), pp. 626-630, 2019.
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv: 1707.063477 2017.
Jing Shi, Xuankai Chang, Shinji Watanabe, and Bo Xu. Train from scratch: Single-stage joint training of speech separation and recognition. Computer Speech Language, 76:101387, 2022.
Fabian-Robert Stöter, Antoine Liutkus, and Nobutaka Ito. The 2018 signal separation evaluation campaign. In International Conference on Latent Variable Analysis and Signal Separation, pp. 293-305. Springer, 2018.
Yiyang Su, Ali Vosoughi, Shijian Deng, Yapeng Tian, and Chenliang Xu. visible sounds toward universal audiovisual awere sound separation. arXiv:2310.11713. 2023.
Separating in- arXiv preprint
Cem Subakan, Mirco Ravanelli, Samuele Cornlll, Mirko Bronzi, and Jianyuan Zhong. Attention is all you need in speech separation. In ICASSP 202/-202 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 21-25. IEEE, 2021.
Reuben Tan, Arijit Ray, Andrea Burns, Bryan A Plummer, Justin Salamon, Oriol Nieto, Bryan Rus- sell, and Kate Saenko. Language-guided audio-visual source separation via trimodal consistency. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp- 10575-10584, 2023.
Nicolas Turpault, Romain Serizel, Ankit Parag Shah, and Justin Salamon. Sound event detection in domestic environments with weakly labeled data and soundscape synthesis. In Workshop on De- tection and Classification of Acoustic Scenes and Events, New York City, United States, October 2019. URL /ttpps://hal. htt./ahh..iiiiiiiammmmmm fr /r/hal 021 60855
Emmanuel Vincent, Rémi Gribonval, and Cédric Févotte. Performance measurement in blind audio source separation. IEEE transactions on audio, speech, and language processing, 14(4):1462 1469, 2006.
Scott Wisdom, Efthymios Tzinis, Hakan Erdogan, Ron Weiss, Kevin Wilson, and John R. Her- shey. Unsupervised sound separation using mixture invariant training. In Proceedings of the 34th International Conference on Neural Information Processing Systems, NIPS '20, Red Hook, NY, USA, 2020. Curran Associates Inc. ISBN 9781713829546
Yusong WU, Ke Chen, Tianyu Zhang, Yuchen Hui, Taylor KirgKKirkpatiic,, and Shlomo Dubnov. Large-scale contrastive language-audio pretraining with feature fusion and keyword-to-caption augmentation. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1-5. IEEE, 2023.
Feiyang Xiao, Jian Guan, Qiaoxi Zhu, Xubo Liu, Wenbo Wang. Shuhan Qi, Kejia Zhang, Jianyuan Sun, and Wenwu Wang. A reference-free metric for language-queried audio source separation using contrastive language-audio pretraining. arXiv preprint arXiv:2407.04495. 2024.
Yuxin Ye, Wenming Yang, and Yapeng Tian. Lavss: oocation-guided audio-visse visual spatial audio separation. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 5508-5519, 2024.
Han Yin, Jisheng Bai, Yang Xiao, Hui Wang, Siqi Zheng, Yafeng Chen, Rohan Kumar Das, Chong Deng, and Jianfeng Chen. Exploring text-queried sound event detection with audio source sep- aration. In ICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1-5. IEEE, 2025.
15

---

Published as a conference paper at ICLR 2026
Dong Yu, Morten Kolbæk, Zheng-Hur Tan, and Jesper Jensen. Permutation invariant training of deep models for speaker-independen multi-talker speech separation. In 2017 IEEE International Conference Acoustics, Speech and Signal Processing (ICASSP), pp. 241-245, 2017. doi:
10.1000CAASP22217.72255
Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, Haibin Lin, Zhiqi Lin, Bole Ma, Guang- ming Sheng, Yuxuan Tong, Chi Zhang, Mofan Zhang, Wang Zhang, Hang Zhu, Jinhua Zhu, Jiaze Chen, Jiangjie Chen, Chengyi Wang. Hongli Yu, Yuxuan Song, Xiangpeng Wei, Hao Zhou, Jingjing Liu, Wei- Ying Ma, Ya-Qin Zhang, Lin Yan, Mu Qiao, Yonghui Wu, and Mingx- uan Wang. Dapo: An open-source I1m reinforcement learning system at scale, 2025. URL https::/arxiv.orq///
14476
Siyuan Yuan, Zhepei Wang, Umut Isik, Ritwik Giri, Jean-Marc Valin, Michael M. Goodwin, and Arvindh Krishnaswamy. Improved singing voice separation with chromagram pitch-aware remixing. In ICASSP 2022 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 111-115, 2022. doi: 10110000CAASS44322222229999
Yi Yuan, Xubo Liu, Haohe Liu, Mark D. Plumbley, and Wenwu Wang. Flowsep: Language-queried sound separation with rectified flow matching. ICASSP 2025 2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1-5, April 2025. doi:
10.111990CAASP960022200000000000
Neil Zeghidour and David Grangier. Wavesplit: End-to-end speech separation by speaker clustering. IEEE/ACM Transactions on Audio, Speech, and Language Processing,
2021.
Jingyi Zhang, Jiaxing Huang. Huanjin Yao, Shunyu Liu, Xikun Zhang, Shijian Lu, and Dacheng Tao. R1-vl: Learning to reason with multimodal large language models via step-wise group relative policy optimization. arXiv preprint arXiv:2503.12931 2025a.
Yi-Fan Zhang, Xingyu Lu, Xiao Hu, Chaoyou Fu, Bin Wen, Tianke Zhang, Changyi Liu, Kaiyu Jiang, Kaibing Chen, Kaiyu Tang, et al. R1-reward: Training multimodal reward model through stable reinforcement learning. arXiv preprint wxxv:2505.02885 2025b.
Hang Zhao, Chuang Gan, Andrew Rouditchenko, Carl Vondrick, Josh McDermott, and Antonio Torralba. The sound of pixels. In Proceedings of the European conference on computer vision (ECCV), pp. 570-586, 2018.
Chujie Zheng, Shixuan Liu, Mingze Li, Xiong- Hui Chen, Bowen Yu, Chang Gao, Kai Dang, Yuqiong Liu, Rui Men, An Yang, Jingren Zhou, and Junyang Lin. Group sequence policy op- timization, 2025. URL http:tttss///ariiii
18071
Daniel M Ziegler, Nisan Stiemnon, Jeffrey Wu, Tom B Brown, Alec Radford, Dario Amodei, Paul Christiano, and Geoffrey Irving. Fine-tuning language models from human preferences. arXiv preprint Xiv: 1909 1909.0893,, 2019.
16

---

Published as a conference paper at ICLR 2026
A MULTI-MODAL LOW-RANK BILINEAR POOLING (MLBP).
Based on (Kim et al. 2017p. let there be K modalities with input vectors 2: E Rd.. Each is projected to a shared dimension d via linear transformations without bias:
x(*) = W W E Rdxds
(10)
The projected vectors are then fused by an dlement-wiss Hadamard product:
K
(K))
E Rd
(11)
A=1
Finally, an output projection with bias produces the pooled embedding:
= Wop+ b, W. E Raxd be Rd.
(12)
This design captures higher-order modality interactions in compact manner.
B EXPERIMENTAL SETUP
Following (Dong et al. 2023 Cheng et al. 2025d), for all audio samples, we conducted experiments kHz. For spectrum
on samples of length 65535 (approximately seconds) at a sampling rate computation, we employed short-time Fourier transform (STFT) with a filter length of 1024, hop length of 256, and a window size 1024. All images were resized to 224 x 224 pixels. The audio model in this paper is widely used 7-layer U-Net network with k 32, generating 32 intermediate masks. All models were trained with a batch size of 128, using the Adam optimizer with parameters 31 = 0.9. B2 0.999, and 10-8 # for 200,000 steps. Additionally, we employed warm-up and ratio
gradient clipping strategies, following (Dong et al. 2023). We compute the signal-to-distoe (SDR) using museval (Stöter et al. 2018). All experiments were conducted on single A.00 GPU with 40GB display memory.
C SI-SDR AND SI-SDRI IMPLEMENTATION
We evaluate separation quality per utterance in the time domain, reconstructing both mixture and estimates via iSTFT with the same analysis parameters used in training. For each utterance with N sources form 1-D waveforms for the mixture x, references {sk 1.1 and estimates {& | { All signals are cropped to the common minimum length L and zero-meaned prior to scoring; SI-SDR calculations run in "float64" for numerical stability. An energy guard is applied: if the absolute sum of any reference or its matched estimate is < 10-5, the utterance is excluded from SI-SDR/ii aggre- gation and we report the number of skipped items. No temporal delay search performed -ie. assume sample-level alignment from the dataset and the iSTFT pipeline.
For single restratce-estimate pair (s, 8) (zero-mean, length L). SI-SDR follows the standard scale- invariant projection:
- (8.))))/|ss| Suaget as, Surget: SE-SDR($ss) s) 10 (Ilsunges [||slnee112/1ee133
(13)
No filtering beyond the scalar is allowed. In the multi-source case build the matrix S R XN with entries Sk,s = SI-SDR(S,, sk) and obtain the permutation that maximizes the total SI-SDR via the Hungarian algorithm applied to SS; the resulting per-source SI-SDR values under this assignment are recorded alongside the corresponding mixture baselines.
Improvement is measured against the unaltered mixture. For each reference Sy compute SI-SDR(x, sx) using the same preprocessing, and define the sample-level SI-SDRi as the mean per-source gain under the optimal assignment n::
17

---

Published as a conference paper at ICLR 2026
SI-SDR N
(Be) SI-SDR(ar,s))
(14)
We also store per-source SI-SDR and mixture-baseline SI-SDR lists for analysis. Datasct-level scores then obtained by averaging per-utterance values; additionally report either standard deviation or a 95% bootstrap confidence interval. When breaking down by categories (e.g.. query modality), aggregate within category first and macro-average across categories.
D REINFORCEMENT LEARNING TRAINING DETAILS
Policy and sampling. The separator outputs mask proposals parameterizing factorized Beta pol- icy (M X) = IIe, Reta(Oufe Befe) with (a, = (1 + 9m, + 9(1 m)) from the network logits me E [0, 1]. At each iteration we sample M from a frozen old policy Taas (a one-step snapshot) and reconstruct the waveform via iSTFT using the same analysis parameters as training.
Objective. We maximize PPO-style clipped surrogate with entropy regularization and an op- tional KI. penalty:
Jeip (0) EM min (ro(M)A, cl
H( (I (
where (MM ) = exp(log næ - log maa and is the advantage after baseline subtraction and, when enabled, group-relative normalization. We minimize LRL (0) -epp(()) and update Toai T after each step (single-epoch PPO).
Advantages and baselines. Rewards are the cosine similarities between separated audio embed- dings and mnery-conditioned targets (audio/text/video or their mixup/adaptive variants). We use an EMA baseline + 9b + (1 5) E[R] with 3 = 0.92. GRPO normalization (optional) sets = (4 - ((l)-((A))/(((() 10-6) within the current group.
Default byperparameters. Clipping range 0.2: entropy coefficient 'n 0.1; KL coefficient ÅKL. E 0.01) (on by default); one Monte Carlo sample per step; mixed precision (FP16/BF16) for the separator, FP32 for reward computation; Adam W with learning rate 2 10-4, weight decay 0.01; global batch size B as reported in the main text; gradient clipping at 1.0; early stopping on validation reward. Unless otherwise noted, GRPO is enabled.
Reward encoder alignment. We apply progressive alignment of the multimodal encoder prior to RL (staged contrastive objectives with the encoder trunk largely frozen and projection heads train- able), then keep the encoder frozen during RL unless specified. This improves reward faithfulness and stability.
Evaluation protocol. Permutation-invariint matching is used for multi-source cases; SI-SDR/-i are computed per utterance with mixture as the improvement baseline and no delay search, following our metric appendix. All systems share identical evaluation knobs.
E MORE EXPERIMENTS
E. ABLATION STUDIES
This section provides systematic analysis of the key factors influencing MARS-Sep, including hyperparameters, modality fusion strategies, and encoder characteristics. Across all experiments, we observe consistent pattern: stable policy optimization and semantically aligned reward modeling jointly determine final separation quality, and different components contribute in a complementary way.
18

---

Published as a conference paper at ICLR 2026
E.1.1 HYPERPARAMETER SETTINGS
Tables summarize the effect of several hyperparameters in omni-queried cases. The concentra- tion parameter バ of the Beta policy introduces direct trade-off between exploration and stability: small values yield overly noisy samples, while large values cause premature policy collapse. achieves the best balance between semantic reward optimization and signal fidelity. The PPO clip range e exhibits very limited sensitivity, indicating that the trust-region update is already sufficiently stable.
In contrast, the entropy coefficient Aи and the KI. coefficient X noticeably influence exploration behavior. Moderate entropy regularization encourages continued mask diversity, while lightweight KL penalty prevents the policy from drifting too far from its previous snapshot, which stabilizes reward estimation. Overall, these trends align with observations from GRPO-style RL training in large models mild exploration and gentle update constraints are most effective for maintaining semantic consistency.
6
9 (Default)
12
15
Mean SDR
6.99+0.71 8.41
7.84+0.74 8,75+0.92
7.93+0.75
7.74
7.70=0.77
Mean CLAP,
9.22+0.90
9.19#0.94
9.10+0.88
Table 4: Effect of Beta distribution concentration parameter (k).
0.01
0.1
0.2 (Default)
0.3
0.4
Mean SDR
7.91=0.75
7.90+0.77 74
7.93+0.75
7.91 7.91
Mean CLAP;
9.00#0.95
9.08+0.92
9.22+0.90
9.1440.88
9.1240.88
Table 5: Effect of PPO clip range parameter (c).
0.01
0.1
0.2 (Default)
0.3
0.4
Mean SDR
0.70
7.68+0.72
7.93+0.75
7.62+0.72
7.57+0.77
Mean CLAP,
8.50+0.89
8. 8.76+0.93
9.22+0.90
8.7940.92
8.46+1
Table 6: Effeec of Entropy coefficient parameter (Ar)).
E.1.2 MODALITY AGGREGATION MODULE
We compared several query fusion mechanisms beyond MLBP, including Max Pooling, Average Pooling, and Learnable Weighted Sums. Results are displayed in Table and Table Although Learnable Weighted Sums slightly outperform MLBP in the Audio-Query and Omni-Ques QuniyQuery settings of VGGSOUND-cleant, MLBP exhibits more stable behavior in Text- cuttQuery and Image-Quee sce- narios and achieves the strongest and most consistent gains on the cross-domain MUSIC-cleant benchmark.
This advantage arises because MLBP explicitly models multiplicative interactions across modali- ties esspecilly between text and vision allowing the reward model form more reliable cross- modal semantic anchor. As result, MLBP proves particularly effective when semantic cues are complex or span multiple modalities, outperforming simpler pooling or weighting schemes under these challenging conditions.
1.1 EFFECT OF PROGRESSIVE FINE-TUNING ON SOURCE DISCRIMINATION OF IMAGEBIND
To verify that progressive fine-tuning improves ImageBind's ability to discriminate between target and non-target sounds, for each target audio sample in the VGGSOUND test set (with its corresponding text label), we randomly selected an interference audio with a different label and constructed a mixture of the two. We then compared the cosine similarity between the target text embedding and the embeddings of both the clean target and the mixture, using the pretrained and fine-tuned InageBind models. By averaging the similarity differences sim(embrarget emb embmizture)
19

---

Published as a conference paper at ICLR 2026
0.1 (Default)
0.2
Mean SDR 7.43+0.72
7.69#0.70 9.22+0.90
7.93+0.75
Mean CLAP;
9.63#0.90
9.19#0.92
Table 7: Effect of KL Divergence coefficient parameter (AKτ.)
Table Comparison of sound separation performance among different methods on VGGSOUND- cleant dataset.
VGGSOUND-clean+ Mean SARt
Methods
Mean SDRT
Mean SIRT
Mean SI-SDRi1
Mean CLAP, f
Text Query Sound Separation
Pooling
6.8320.68
8.6941.00
14.1940.75
4.35+0.44
8,99=0.92
Learned Weighted Sums
6.79=0.65
8.95+1.00
13.77=0.75
4.34=0.46
8.95=0.95
Average Pooling
6,7540.66
9.20#1.00
13,5420.75
9.1820.79
8.97=0.94
MLBP (ours)
6.91=0.68
9.14 1.00
13.73+0.77
4.55+0.44
9.03=0094
Audio Query Sound Separation
Max Pooling
7.11:0.6
11.21=0.99
11.72+0.79
4.39=0.53
8.67=0.93
Learned Weighted Sums
7.23=0.65
11.57=0.99
11.82+0.82
4.24+0.55
8.75=0.97
Average Pooling
6.69#0.61
11.76=1.02
11.12
3,90=0.60
8.42=0.91
MLBP (ours)
7.33+0.67
11.63=1.00
12.00=0.84
4.36#0.50
8.91=0.91
Image Query Sound Separation
Max Pooling
6,78+0.66
9.52+1.04
13.7640.74
4.43=0.49
8.68=0.90
Learned Weighted Sums
7.15+0.69
10.03 1,07
13.95=0.75
4.80+0.50
9.04=0.92
Average Pooling
7.03+0.68
10.20=1.07
13.95=0.76
4.69=0.48
8.81=0.91
MLBP (ours)
6.93+0.67
10.18
13.41=0.72
4.57=0.47
9.19=0.91
Composed Omnii-oodal Query Sound Separation
Max Pooling
7.8620.72
10.31=1.00
14.5420.92
5.13#0.45
8.9420.92
Learned Weighted Sums
8.08+0.75
10.58=1.00 1.00
14.74=0.95
5.32+0.47
9.07=0.95
Average Pooling
7.87+0.74
10.96+1.03
14.5240.91
5.2020.46
9.18=0.99 94
MLBP (ours)
7.93+0.75
10.65=1.00
14.49#0.95
5.20=0.45
9.22+0.90
across all test samples, obtain robust measure of the model's ability discriminate non-target sounds. The results in Table [odemmnsteeee that the fine-tuned model consistently yields a larger av- erage difference, confirming its improved semantic alignment in the presence of interfering sources.
1.1.4 EFFECT OF REINFORCEMENT LEARNING AND ENCODER FINE-TUNING UNDER DIFFERENT TRAINING PIPELINES.
To disentangle the contributions of reinforcement learning (RL) and progressive fine-tuning (FT) of the ImageBind encoder, compared four training configurations: (i) baseline supervised training with a frozen encoder, (ii) RL with frozen encoder, (iii) FT-only under supervised training, and (iv) RL combined with FT (our full model). Results the test set Table [I reveal consistent trend. The FT-only variant yields higher SAR scores but substantially lower SDR, SIR, SI-SDRi and CLAP score, indicating that the encoder becomes more sensitive semantic cues but the conventional ob- jective fails to enforce clean separation, leading to leakage from interfering sources. By contrast, the RL-only variant achieves improvements over the baseline across all metrics, demonstrating that policy optimization itself enhances separation fidelity even without encoder adaptation. Finally, the RL++FT variant provides the best overall performance, simultaneously improving SDR/SIR and achieving the highest SAR and CLAP scores. These findings confirm that reinforcement learning crucial for harnessing the benefits fine-tuned encoders while avoiding the metric trade-off ob- served in the FT-only setting.
This case further illustrates that while fine-tuning enhances the encoder's 's semantic sensitivity, re- inforcement learning is indispensable to suppress residual noise and achieve clean separation. In combination, RL and FT strike balance between semantic alignment and signal fidelity, yielding perceptually superior outputs.
20

---

Published as a conference paper at ICLR 2026
Table Comparison of sound separation performance among different methods on MUSIC-clean+ dataset.
MUSIC-clean+ Mean SART
Methods
Mean SDR1
Mean SIRT
Mean SI-SDRi
Mean CLAP, t
Text Query Sound Separation
Max Pooling
12.80&0.92
17.37=1.20
18.58+0.89
9.55=0.81
5,34=0.8
Learned Weighted Sums
12.60+0.82
17.79=1.21
18.23+0.8 84
9.47=0.88 80
5.39=0.92
Average Pooling
12.08+0.88
17.31=1.19
17.73=0.87
8.82=0.85
5.14#0.92
MLBP (ours)
12.91
17.61=1.17
18,28+0.93
0.85=0.82
6.18+0.93
Audio Query Sound Separation
Max Pooling
11.13=0.87
18.72=1.14
15.56+0.84
1.81
5.38=0.96
Learned Weighted Sums
10,98:0.89
18.92+1.10
14.6920.90
8.03+1.00
5.7720.98
Average Pooling
10.48 90
18. 13.69:1.00 .03
14. 14.16+0 90
7.62+1 1,08
5,46:1.00
MLBP (ours)
11.73=00.8
19.65=1.14
15.25=0.86
8.38+1.03
5.64+1.06
Image Query Sound Separation
Max Pooling
13.29#1..0
18.69=1.20
18.23#1.00
10.35=0.95
6.44+1.05
Learned Weighted Sums
13.21+0.93
18.75=1.17
18.23+0.95
10.31
6.09=1.01
Average Pooling
12.80+0.96
18.27=1.18
17. 8577
9.82=0.93
6.36+11.03
MLBP (ours)
13.64=1.06
19.24=1.16
18.05=1.06
10.70=0.89
1.94
Composed Owni-modal Query Sound Separation
Max Pooling
13.61
19.61=1.21
18. 10=0.97
10,58=0,88
6.61=0.96
Learned Weighted Sums
13.41=0.90
19.54=1.18
17.99=0.90
10.43+0.8
6.49=0.95
Average Pooling
13.15
19.21=1.23
17.73+0.92
10.09=0.80
6.46+0.94
MLBP (ours)
13.8920.98
19.90=1.18
17.99+0.97
10.78
6.8220.99
Table 10: Average similarity differences (target mixture) relative the target text embedding, eval- uated across the full test set. Larger values indicate stronger discrimination of non-target sources.
Model
Avg. Difference (t)
Pretrained ImageBind
0.0035 +'t 0.0561
Fine-tuned ImageBind
0.0258 + 0.0630
E.2 RESOLVING SEMANTIC AMBIGUITY IN ACOUSTICALLY SIMILAR SOURCES
To qualitatively evaluate our model's ability to address the "metric dilemma", we designed a case study focused on separating acoustically similar sources. We created challenging audio mixture containing both the sound of tap dancing and typewriting simultaneously. These sources are highly confusable both are characterized by sharp. percussive transients with broadband spectral content, lacking the strong, sustained harmonic structures that typically aid in separation.
The task was isolate the tap dancing using the text query "the sound of tap dancing." When this mixture was processed by the baseline OmniSep model (without RL), it achieved high Signal- to-Distortion Ratio (SDR), yet the resulting audio was perceptually contaminated with the distinct, rhythmic clicks of the typewriter. This outcome exemplifies the metric dilemma, where model successfully optimizes a signal-level metric while failing achieve true semantic separation.
In contrast, our proposed MARS-Sep guided by multimodal semantic reward, produced much cleaner separation. While its SDR score was marginally lower, its SIR was substantially higher, indicating superior suppression of the interfering typewriter source. To further quantify this semantic improvement, we computed the CL.AP score ((Xiao al. 2024)), defined as the cosine similarity between the separated audio's embedding and the text query's embedding using the CLAP model. Unlike purely signal-level metrics, the CLAP score directly measures semantic alignment across modalities, offering more reliable indicator of whether the separated source matches the intended textual description. The comparative results are summarized in Table 12
This case study validates that by directly optimizing for semantic consistency, MARS-Sep effec- tively mitigates semantic contamination and delivers perceptually superior results in scenarios where traditional signal-level metrics can be misleading. Moreover, the of CLAP score highlights the
21

---

Published as a conference paper at ICLR 2026
Table 11: Comparison of different training configurations the VGGSOUNDD-leann test set with text queries. RL here stands for reinforcement learning and FT denotes progressive fine-tuning of ImageBind.
Method
Mcan SDR
Mcan SIRT
Mcan SART
Mean SI-SDRit
Mean CLAP,
Buseline (Supervised Frozen Enzoder)
6.70=0.66 66
9,0440.98
13.61=0.77
4. 4.38+0048
8.98+0.89
RI-only (RL.+ Frozen Encoder)
6.71:0.70
9.04x1.02
14.08:0.80
4.5020.75
8.9620.90
Fi-only (Supervised + Fine-tuned Encoder)
0.75=0064
1.41#1.18
87.13+0.15
0.00+0.00
5.48=0.95
RL+FT (Full Model)
6.9120.68
9.14#1.00
13.73=0.77
4.55+0.44
9.03=0.94
Table 12: A quantitative and qualitative comparison for the "tap ""triii"""ttyprrttingg"" separation task. This table presents the results for the baseline OmniSep model and our proposed MARS- Sep-
The CLAP score is the cosine similarity between the separated audio embedding and the text query
('the sound tap dancing") embedding generated by the CLAP model.
Model
Text Query
SDR
SIR
SAR
CLAP score
CLIPSEP-NIT (Dong et al. 2023
11 11.8500
24.10664
16.3873
0.3053
OmniSep CCeen al. 2025d
"tap dancing"
10.8925
24.2579
17.0300
0.4810
MARS-Sep (Ours)
12.0603
24.0055
17.2554
0.4935
advantage of employing cross-modal semantic evaluation, it aligns closely with human perception of whether the separation captures the intended sound concept.
E.3 EFFICACY OF THE PROGRESSIVE ALIGNMENT FINETUNING
To further examine the contribution of the progressive alignment strategy in our MARS-Sep frame- work, replace the progressively fine-tuned ImageRind encoder with
(i) a frozen version without any fine-tuming (no finetuning),
(ii) variant fine-tuned in a single stage on the mixed paired dataset (I-stage finetuning).
Table 13: Comparison of sound separation performance among different fine-tuning strategies YGGSOUND-cleant dataset.
VGGSOUND-clean+ Mean SARt
Methods
Mean SDR1 No fine-tuning
Mean SIR
Mean SI-SDRi
Mean CLAP; 1
Text Query Sound Separation
6.5910.68
8.82+1.01
13.67=0.75
4.23+0.46
8.5620.90
1tsag fine-tuning 6.73+0.68
9.2440.99
13.7240.79
4.40+0.46
9.07=0.91
3-stage fine-tuning 6.91+0.68
9.14+1.00
13.73+0.77
4.55+0.44
9.03+0.94
Audio Query Sound Separation
No fine-tuning
6.85+0.62
11.4620.99
11.3940.77
4.1540.53
8.6920.93
1-stage
6.6910.62
11.35+1.00 1.03
11.40=0.78
3.9820.53
8.64+0.91
3-stage fine-tuning 7.33+0.67
11.63+1100
12.00=0.84
4.36+0.50
8.91+0.91
Image Query Sound Separation
No fine-tuning
7.11+0.68
9.96+1.04 1.04
14.00+0.75
4, 4.68&0.48
8,59+0.91
I-stage fine-tuning 6.54+0.63
9.99+1.05
13.57+0.77
4. 4.1100.50
9.17+0.90
3-stgge finc-tuning 6.93+0.67
1..1
13.41#0.72
4.57+0.47
9.19#0.91
Composed Omni-modal Query Sound Separation
No fine-tuning 7.69+0.75
10.3441.01
14.57=0.94
4, 4.98+0.48
9.05=0.91
1-stage fine-tuning 7.6740.74
10.70:1.03
14.48+0.94
5.8400.50
8.83+0.89
3-stage fine-tuning 7.93+0.75
10.65+1.00
14.49=0.95
5.20:0.45
9.22+0090
As shown by the results in Table 13 MARS-Sep with progressive alignment still demonstrates clear advantages the VGGSOUND-clean+ dataset, though the trends differ slightly from those ob- served on MUSIC-cleant. For text and audio query separation, the three-stage fine-tuning strategy consistently yields the best overall signal-level metrics (SDR, SIR, SAR, and SI-SDRi), indicat-
22

---

Published as a conference paper at ICLR 2026
ing that progressive, stage-wist alignment effectively enhances the multimodal encoder's domain adaptation capability. In particular, the gain audio query separation is most evident, where the three-stage strategy surpasses both the frozen and single-stage variants across all metrics, showing its robustness in cross-modal grounding.
While the improvement from progressive fine-tuning is slightly less pronounced in the image-query setting, this can be naturally explained by the fact that ImageBind's 's visual encoder is pretrained on large-scale visual data and already provides stable embedding space. Thus, image-audio alignment requires less additional adaptation than text-audio or audio-audio alignment. In contrast, the text and audio branches differ more significantly semantic granularity and distribution, making them more sensitive to staged alignment.
Nonetheless, the three-stage strategy consistently provides the most robust performance across text, audio, and omni-modal queries, where semantic compositionality is more complex. These results confirm that progressive alignment is especially beneficial for cross-modal generalization and se- mantic consistency in challenging, in-the-wild mixtures.
E,4 MEMORY CONSUMPTION AND INFERENCE LATENCY
To validate the training time and memory consumption and inference efficiency, we conduct addi- tional experiments to measure training compute. hardware configuration, memory usage, and in- ference latency on standard hardware. Table 14 summarizes the results. Both OmniSep and our MARS-Sep model were trained on an Nvidia A800 40GB GPU, and the memory consumption for all modalities remains comparable at approximately 35.5 GB. In terms of training efficiency, MARS- Sep requires roughly 8 hours per epoch with batch size of 128 and 10k steps, which about 50% of the baseline efficiency reported by OmniSep (about 4 hours per epoch). Despite the increased training cost, our model introduces virtually no additional inference overhead: the real-time factors (RTF) across text, image, audio, and omni-modalit inputs remain on par with OmniSep, with dif- ferences within normal variance. These results demonstrate that MARS-Sep maintains comparable inference-time time efficiency while achieving its improvements through modest increase in training- time computation.
Training Time per Epoch (Batch Size=128, 10k steps)
Inference Latency per Batch(RTF) (Batch Size 4)
Method
Hardware
Memory Consumption
Text: 0.1283s
Image: 0.0841s
OmniSep
35.5GB
Nvidia A800 40GB
hours
Audio: 0.08122
Omni: 0.0801s
Text: 0.1202s
Image: 0.081 1886
MARS-Sep (Ours)
35.5GB
Nvidia A800 40GB
hours
Audio: 0.0814 10814s
Omni: 0.0893s
Table 14: Efficiency reporting on standard hardware. Inference Latency is tested on VGGSOUND dataset.
E.5 USER STUDY FOR EVALUATING CROSS- MROSS-MODAL SEMANTIC ALIGNMENT
To complement the CLAP-based automatic metrics and address concerns about potential biases inherited from CLAP, we conducted human perceptual study to directly evaluate the semantic alignment between user queries and the separated audio. We sampled 100 query-audio pairs from the test set, covering diverse sound events and including both text, image and audio queries. For each pair, presented participants with the query and two audio excerpts-one target audio and one separated by our method or OmniSep-in randomized order.
10 non-expert listeners participated in the study, each providing 100 judgments. Participants rated the semantic consistency between the query and each audio sample on a 1-5 Likert scale, and addi- tionally chose the sample that better matched the query in a pairwise preference setting. As shown
23

---

Published as a conference paper at ICLR 2026
MARS-Se Sep achieves higher semantic-mates matching scores and is preferred against the tar- get audio more often than OmniSep across all three query conditions. These results indicate that improvements are not merely artifacts of CLAP similarities but also perceived by human listeners, supporting the claim that MARS-Sep produces more semantically aligned separations.
in Table 15
Method
Semantic Match (1-5)
Pairwise Preference (%)
Target Audio
4.76 = 0.20
/
OmniSep
3,42 + 0.51
15.4
MARS- Sep (Ours)
3.57 + 0.48
23.2
Table 15: Human user study evaluating semantic alignment between queries and separated audio.
E.6 ADDITIONAL QUALITATIVE RESULTS IN THE TQSS SETTING
Figure iillustrates representative qualitative comparisons under the TQSS scenario. For each ex- ample, we visualize the log-mel spectrograms of the mixed input, the target source, the interference source, as well as the separation outputs from the baseline method and our proposed approach. As can be observed, our method better preserves the structure of the target source while effec- tively suppressing interference components. More examples are available on our project webpage https://ma //mars- sep. sep.github.is.
F THE USE OF LARGE LANGUAGE MODELS (LLMs)
In this work, utilized large language model (GPT-5) for two auxiliary purposes. It was used to generate illustrative images, including those fox barking' and a 'playing congas' for Figure Additionally, we leveraged its search capabilities assist with our literature review.
24

---

Published as a conference paper at ICLR 2026
mixed
interf
bascline
ours
mixed
target
interf
bascline
mixed
target
interf
baseline
OLITS
Figure 4: Qualitative comparison of separation results in the TQSS setting, Each group contains spectrograms: mixed input, target source, interference source, (Onnnine(OmmiSep)) separation, and our method separation.
25


Reviews:
[{"review_id": "sXrdcgSxta", "reviewer": "Reviewer_AB1X", "summary": "The paper presents a Dynamic Semantic Routing Framework (DSRF) for the MSA task. A hierarchical semantic factorization module disentangles each modality into four functionally independent representations: primary emotion, contextual cue, ambiguity, and noise, thereby enabling fine-grained semantic modeling. A semantic dynamic routing interaction mechanism dynamically routes and aggregates semantic factors through a capsule-inspired interaction process to reconstruct modality representations with high-order compositionality. An uncertainty-aware semantic fusion strategy estimates the reliability of each semantic factor and adaptively integrates them across modalities for robust sentiment prediction under modality inconsistency.", "questions": "1. The method used for comparing experimental results lacks the latest findings, such as the experimental results from 2025.\nDlf: Disentangled-language focused multimodal sentiment analysis. In Proceedings of the AAAI Conference on Artificial Intelligence, pp. 21180–21188, 2025.\n2. The method proposed in this paper, an uncertainty-aware semantic fusion strategy, shares some similarities with the method “Uncertainty Score Disentanglement” which is presented in the paper [1]. A comparison between them is recommended. \n[1] Localization-assisted Uncertainty Score Disentanglement Network for Action Quality Assessment, ACM International Conference on Multimedia (ACMMM), pp. 8590–8597, 2023. \nAnd there remains a reference missing.\n[2] Cross-modality Representation Interactive Learning for Multimodal Sentiment Analysis. ACM MM, 2023. \n\n3. In Figure 1, how do elements of (c) and (d) correspond to network blocks in (b)? A detailed explanation is required.\n4. In section 4.4, the ablation study is neither clear and not sufficient. A Cross-validation across modules HSF, SFR, DRI can more effectively demonstrate the contribution of the main innovations to the paper's algorithm.", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "The paper presents a hierarchical semantic factorization module, enabling fine-grained semantic modeling. It introduces a semantic dynamic routing interaction mechanism, which dynamically routes and aggregates the semantic factors through a capsule-inspired interaction process to reconstruct modality representations with high-order compositionality. An uncertainty-aware semantic fusion strategy is presented to estimate the reliability of each semantic factor and integrate them across modalities for robust sentiment prediction under modality inconsistency. \nThe algorithm proposed in the paper demonstrates innovation, clear logic, and sufficient experimental verification, which effectively confirms the validity of the algorithm.", "weaknesses": "1. The method used for comparing experimental results lacks the latest findings, such as the experimental results from 2025.\nDlf: Disentangled-language focused multimodal sentiment analysis. In Proceedings of the AAAI Conference on Artificial Intelligence, pp. 21180–21188, 2025.\n2. The method proposed in this paper, an uncertainty-aware semantic fusion strategy, shares some similarities with the method “Uncertainty Score Disentanglement” which is presented in the paper [1]. A comparison between them is recommended. \n[1] Localization-assisted Uncertainty Score Disentanglement Network for Action Quality Assessment, ACM International Conference on Multimedia (ACMMM), pp. 8590–8597, 2023. \nAnd there remains a reference missing.\n[2] Cross-modality Representation Interactive Learning for Multimodal Sentiment Analysis. ACM MM, 2023.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "LzEZ6ACqqa", "reviewer": "Reviewer_aiK4", "summary": "This paper describes a method of multimodal sound separation.   This work differs from previous work in two ways.  First, it formulates sound separation as an inverse estimation problem and uses RL as an optimization technique to find the best separation masks.  Second, it utilizes a contrastively learned semantic network to assess the similarity between the reconstructed sound and the multimodal inputs. \n\nThe main argument presented in this paper is that sound separation should be treated similarly to the alignment of LLMs with human preferences.   In this way, the RL algorithm can be utilized as a meta-reasoning system to steer a base sound separation architecture (OmniSep).  The reward in this setting should be more semantic, closer to human preference, rather than raw sound waveforms.   A key insight is that the training multimodal separation masks could be used to infer this human preference.  \n\nHowever, the writing on this overall logic is unclear, instead focusing on the details of implementation.", "questions": "See above on weakness.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "The overall approach is creative.  This paper addressed a preference/measurement issue in multimodal sound separation: models optimized for signal-level metrics don't produce semantically meaningful results. \n\nIt argues that sound mask should not be used directly as a supervised training signal, but indirectly as a way to train a multimodal sound preference function that is semantically meaningful.  Once we have this preference measure, we can treat the sound separation as an inverse problem, and the RL can be used as an optimization tool.\n\nDiving into the details of RL.   The state space is the sound (mixed) spectrogram, and the query for a separation architecture called OmniSepbase.  The action is the predicted separation masks M.  The semantic similarity between the separated sound waveform and the multimodal query defines the reward function R.  The semantic similarity is fine-tuned from ImageBind, using a progressive refinement procedure on top of contrastive learning.   All these steps are reasonably designed.  The experimental results are good on both signal and semantic level measures.", "weaknesses": "The overall logic of the paper is not well presented.  The authors didn't directly outline the main chain of logic, but instead presented a linear sequence of computational and implementation steps.  The connection to LLMs-to-human preferences alignment is not apparent until the very end of the paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "l5kAuZxT44", "reviewer": "Reviewer_6A7c", "summary": "The paper introduces a reinforcement-learning framework for query-conditioned universal sound separation. Instead of regressing deterministic masks, a factorized Beta policy samples time–frequency masks. The training uses a trust-region clipped surrogate with entropy and KL penalties, sampling from a frozen old policy for stability. Multimodal rewards are embedded with a ImageBind encoder, and the target embeddings are fused via low-rank bilinear pooling, and cosine similarity with the separated audio provides the scalar reward. A three-stage progressive alignment procedure fine-tunes only projection heads to make the encoder’s rewards more discriminative before RL. Experiments on VGGSound-clean and MUSIC-clean report consistent gains across different modalities.", "questions": "1. How do MLBP and the fused-anchor strategy compare to (i) simple average of unimodal similarities, (ii) max pooling, (iii) learned weighted sums, especially under composed queries?\n\n2. What is the training time and inference latency vs. OmniSep? Does stochastic masking at test time improve robustness, or do you deploy the mean mask?\n\n3. Have you tried a time-domain separator or a phase-estimation front-end? Do the gains transfer?\n\n4. How does performance/compute scale with the number of target masks K?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. Using RL for separation is reasonable, as it refines pure mask prediction with a stochastic beta policy process. The old-policy sampling, clipped ratio, and entropy/KL yield a stable loop without a value network.\n\n2. The design of multimodal reward avoids modality dominance. Fusing target-side audio/text/vision to form a single semantic anchor addresses reward imbalance and supports composed queries. The motivation and construction are clearly explained.\n\n3. The three-stage curriculum is a pragmatic way to improve reward faithfulness while keeping encoders largely frozen.", "weaknesses": "1. The ablation study is not enough. The paper motivates several design choices, but some important hyperparameters are not ablated well. For example, the effect of k, \\lambda_H, \\lambda_KL, clip \\theta, etc. A systematic ablation suite isolating each factor would make the gains more convincing. \n\n2. The method is presented with a U-Net-style STFT separator using mixture phase at reconstruction. Would it work for a time-domain separator as well?\n\n3. Another issue with the paper is the lack of comparison methods. While the authors provide results of CLIPSEP-NIT, AudioSep, and OmniSep in table 1, there are more methods that can be compared. For example, for text query sound separation, LASS-Net[1], FlowSep[2], ZeroSep[3] are not included in the results. Especially FlowSep and ZeroSep that report CLAP score, which is one of the main claim from the paper that the proposed method achieve better semantic matching.\n\n[1] Separate What You Describe: Language-Queried Audio Source Separation\n[2] FlowSep: Language-Queried Sound Separation with Rectified Flow Matching\n[3] ZeroSep: Separate Anything in Audio with Zero Training\n \nSimilarly, for image/video query, only CLIPSEP-NIT is compared. In the visually guided separation domain, there are some missing references:\n[4] iQuery: Instruments As Queries for Audio-Visual Sound Separation\n[5] High-Quality Visually-Guided Sound Separation from Diverse Categories \n\nI believe these works are worth discussing in the paper and being compared. Particularly, for [5], the paper uses a generative objective to train rather than the traditional metric like SDR/SIR/SAR, which, a comparison, can make the paper's claim more convincing.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "mwWY061945", "reviewer": "Reviewer_bExK", "summary": "The paper proposes a query-conditioned universal sound separation method that reframes mask prediction as reinforcement learning over a factorized Beta-distributed mask policy. Rewards are multimodal, computed by comparing separated audio to fused audio/text/image query embeddings to directly incentivize semantic faithfulness. A three-stage progressive alignment fine-tunes projection heads of the multimodal encoder to improve cross-modal discrimination and reward stability. \n\nOn VGGSound-clean+ and MUSIC-clean+, MARS-Sep matches or surpasses baselines on SDR/SIR/SAR/SI-SDRi metrics and achieves higher CLAP scores.", "questions": "- Could the authors include more inference results on in-the-wild data?\n\n- As stated in the weakness part, could the author provide results of human user study to measure the semantic alignment?\n\n- Could the authors provide efficiency reporting, like provide training compute, inference RTF/latency on standard hardware, and memory use for different modalities.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- The method is novel. The paper reframes query-conditioned separation as RL with a factorized Beta-policy over masks and optimizes it via a PPO-style clipped surrogate.\n\n- The multimodal reward is well-designed. Rewards are computed by comparing separated audio with fused multi-modal embeddings, directly optimizing semantic faithfulness to the query.\n\n- On VGGSound-clean+ and MUSIC-clean+, MARS-Sep consistently surpasses CLIPSep-NIT and OmniSep for audio, image, and composed queries on SDR/SIR/SAR/CLAP.", "weaknesses": "- The evaluation is limited to only VGGSound/Music, also the qualitative results shown on the website are quite limited samples. As the paper claims for universal sound separation, the paper should demonstrate quantitative&qualitative results on in the wild data.\n\n- Limited discussion of efficiency. The paper details training knobs but lacks hardware/runtime, inference latency, or throughput measurements, especially for longer inputs.\n\n- Improvements lean on CLAP for semantic alignment, but CLAP may carry its own biases and may not fully reflect human perceptual quality. A human user study is needed for fully measure the semantic alignment.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)