Title: SongEcho: Cover Song Generation via Instance-Adaptive Element-wise Linear Modulation
Primary area: applications to computer vision, audio, language, and other modalities
Keywords: conditioning method, controllable song generation

Abstract: Cover songs constitute a vital aspect of musical culture, preserving the core melody of an original composition while reinterpreting it to infuse novel emotional depth and thematic emphasis. Although prior research has explored the reinterpretation of instrumental music through melody-conditioned text-to-music models, the task of resynthesizing full-length songs remains largely unaddressed. We formalize this challenge as Cover Song Generation, which requires preserving the source vocal melody while simultaneously synthesizing new vocals and accompaniment, posing higher demands for controllable music generation. To this end, we present SongEcho, which leverages Instance-Adaptive Element-wise Linear Modulation (IA-EiLM), a framework that incorporates controllable generation by improving both conditioning injection mechanism and conditional representation. To enhance the conditioning injection mechanism, we extend Feature-wise Linear Modulation (FiLM) to an Element-wise Linear Modulation (EiLM), to facilitate precise temporal alignment in melody control. For conditional representations, we propose Instance-Adaptive Condition Refinement (IACR), which refines conditioning features by interacting with the hidden states of the generative model, yielding instance-adaptive conditioning. Additionally, to address the scarcity of large-scale, open-source full-song datasets, we construct Suno70k, a high-quality AI song dataset enriched with comprehensive annotations.
Experimental results show that our approach yields superior cover songs, surpassing state-of-the-art approaches across all metrics on multiple datasets, while requiring fewer than 30% of the trainable parameters employed by other methods.

TL;DR: A cover song generation framework and a novel conditioning method