Facing the Music: Tackling Singing Voice Separation in Cinematic Audio
  Source Separation
Authors: name: Karn N. Watcharasupat; name: Chih-Wei Wu; name: Iroro Orife
Published: 2024-08-07 07:04:29

Abstract: Cinematic audio source separation (CASS) is a fairly new subtask of audio
source separation. A typical setup of CASS is a three-stem problem, with the
aim of separating the mixture into the dialogue stem (DX), music stem (MX), and
effects stem (FX). In practice, however, several edge cases exist as some sound
sources do not fit neatly in either of these three stems, necessitating the use
of additional auxiliary stems in production. One very common edge case is the
singing voice in film audio, which may belong in either the DX or MX, depending
heavily on the cinematic context. In this work, we demonstrate a very
straightforward extension of the dedicated-decoder Bandit and query-based
single-decoder Banquet models to a four-stem problem, treating non-musical
dialogue, instrumental music, singing voice, and effects as separate stems.
Interestingly, the query-based Banquet model outperformed the dedicated-decoder
Bandit model. We hypothesized that this is due to a better feature alignment at
the bottleneck as enforced by the band-agnostic FiLM layer. Dataset and model
implementation will be made available at
https://github.com/kwatcharasupat/source-separation-landing.