GETMusic: Generating Any Music Tracks with a Unified Representation and
  Diffusion Framework
Authors: name: Ang Lv; name: Xu Tan; name: Peiling Lu; name: Wei Ye; name: Shikun Zhang; name: Jiang Bian; name: Rui Yan
Published: 2023-05-18 09:53:23

Abstract: Symbolic music generation aims to create musical notes, which can help users
compose music, such as generating target instrumental tracks from scratch, or
based on user-provided source tracks. Considering the diverse and flexible
combination between source and target tracks, a unified model capable of
generating any arbitrary tracks is of crucial necessity. Previous works fail to
address this need due to inherent constraints in music representations and
model architectures. To address this need, we propose a unified representation
and diffusion framework named GETMusic (`GET' stands for GEnerate music
Tracks), which includes a novel music representation named GETScore, and a
diffusion model named GETDiff. GETScore represents notes as tokens and
organizes them in a 2D structure, with tracks stacked vertically and
progressing horizontally over time. During training, tracks are randomly
selected as either the target or source. In the forward process, target tracks
are corrupted by masking their tokens, while source tracks remain as ground
truth. In the denoising process, GETDiff learns to predict the masked target
tokens, conditioning on the source tracks. With separate tracks in GETScore and
the non-autoregressive behavior of the model, GETMusic can explicitly control
the generation of any target tracks from scratch or conditioning on source
tracks. We conduct experiments on music generation involving six instrumental
tracks, resulting in a total of 665 combinations. GETMusic provides
high-quality results across diverse combinations and surpasses prior works
proposed for some specific combinations.