Controllable Music Loops Generation with MIDI and Text via Multi-Stage Cross Attention and Instrument-Aware Reinforcement Learning

Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-04-07 06:16:49

Abstract: The burgeoning field of text-to-music generation models has shown great promise in their ability to generate high-quality music aligned with users' textual descriptions. These models effectively capture abstract/global musical features such as style and mood. However, they often inadequately produce the precise rendering of critical music loop attributes, including melody, rhythms, and instrumentation, which are essential for modern music loop production. To overcome this limitation, this paper proposed a Loops Transformer and a Multi-Stage Cross Attention mechanism that enable a cohesive integration of textual and MIDI input specifications. Additionally, a novel Instrument-Aware Reinforcement Learning technique was introduced to ensure the correct adoption of instrumentation. We demonstrated that the proposed model can generate music loops that simultaneously satisfy the conditions specified by both natural language texts and MIDI input, ensuring coherence between the two modalities. We also showed that our model outperformed the state-of-the-art baseline model, MusicGen, in both objective metrics (by lowering the FAD score by 1.3, indicating superior quality with lower scores, and by improving the Normalized Dynamic Time Warping Distance with given melodies by 12\%) and subjective metrics (by +2.56\% in OVL, +5.42\% in REL, and +7.74\% in Loop Consistency). These improvements highlight our model's capability to produce musically coherent loops that satisfy the complex requirements of contemporary music production, representing a notable advancement in the field. Generated music loop samples can be explored at: https://loopstransformer.netlify.app/ .

Main paper: 
 introduction: 
 Loops play a critical role in modern music production, enabling creators to construct complex and layered compositions by repeatedly using sound loops. These loops can include melodies, rhythms, or textures, and are particularly popular among DJs, electronic music producers, and contemporary musicians. In the process of music creation, loops often act as the foundation for musical accompaniment, with the final musical piece enriched through the layering of various elements. Essential to this process is the precise manipulation of melody, rhythm, BPM, key, and instrumentation within the loops.This study explores the generation of musical loops from text and MIDI inputs, which is particularly relevant during the music creation process. Creators commonly employ Digital Audio Workstations (DAWs) to translate their creative requirements-such as melody, rhythm, BPM, key, and velocity-into MIDI format. By integrating textual prompts with specific MIDI settings, it becomes possible to produce tailored loops. This approach not only accelerates the experimental and exploratory phases of music creation but also enhances its interactivity, offering composers a richer and more malleable creative landscape.The current text-to-music generation models, such as those based on Diffusion Models [26, 29] , including Noise2Music [12] , Audi-oLDM 2 [19] , JEN-1 [18] , and MusicLDM [2] ; as well as text-tomusic models based on Vector Quantization (VQ) [31] and Residual Vector Quantization (RVQ) [4, 38] , such as Jukebox [5] , MusicLM [1] , and MusicGen [3] , can generate music that aligns with the user's textual input. While these models effectively capture abstract/global musical features like style and mood, they often inadequately address the precise rendering of critical music loop attributes, including melody, rhythms, and instrumentation, which are essential for modern music loops production.To overcome the aforementioned challenges, we introduce Loops Transformer (model architecture shown in Figure 1 ), a controllable music loops generation model capable of generating high-quality loops given both textual descriptions and MIDI inputs. To better integrate text and MIDI, we propose a novel Multi-Stage Cross Attention mechanism that combines text embeddings (from the T5 model [24] and our Text-to-MIDI Transformer) and MIDI embeddings (from our MIDI Transformer).The model's training objective is to generate high-quality 32 kHz music loops by modeling multiple parallel streams of discrete audio tokens, which are obtained through the Encodec [4] using Residual Vector Quantization (RVQ). To enhance the quality and usability of the generated loops (making them more suitable for the characteristics of loopable music), we improve upon the Codebook Interleaving Pattern [1, 3] (illustrated in Figure 2 ). Based on the Delay Pattern [3] , we introduce 𝑠 and 𝑒 tokens in the Codebook Interleaving Pattern to enable the model to explicitly learn the start and end of a loop segment and generate results that can be seamlessly looped.Furthermore, during the pre-training stage, we introduce a Loop Shift mechanism for data augmentation, i.e., sampling the 𝑠 1 and 𝑠 2 as two randomly sampled integer values that determine the number of time steps to be added before the start token 𝑠 and after the end token 𝑒, respectively. The Loop Shift data augmentation mechanism is designed based on the characteristics of loops, as an ideal loop should have content generated after the start token 𝑠 that can form a continuous playback loop with the content before the end token 𝑒.Moreover, to improve the model's instrumentation capabilities and performance, i.e., ensuring that the instruments used in the generated loops match the textual prompts, we draw inspiration from the Reinforcement Learning from Human Feedback (RLHF) [22, 30] approach that has flourished in the development of large language models (LLMs). We propose a novel Instrument-Aware Reinforcement Learning training strategy, introducing a scoring model for instrumentation evaluation. We feed the loops generated by the Loops Transformer into this scoring model for multi-label instrument classification to obtain the reward to update the Loops Transformer's model parameters.We conduct the extensive evaluation, and the experimental results demonstrate that, compared to the state-of-the-art baseline model (MusicGen [3] ), our approach significantly improves melody matching by 12% in Normalized DTW Distance and shows notable gains in both objective and subjective evaluations. Notably, in the human evaluation conducted by evaluators with experience in using or creating loops, our model achieves significant improvements in subjective ratings, such as overall quality (OVL): +2.56%, relevance to the text input (REL): +5.42%, and Loop Consistency: +7.74%. This demonstrates our model's ability to produce musically consistent loops that meet the detailed requirements of contemporary music production.To summarize our contributions:(1) We introduce Loops Transformer, a novel controllable music loops generation model that integrates text and MIDI inputs using a Multi-Stage Cross Attention mechanism and an improved Codebook Interleaving Pattern with Loop Shift data augmentation for seamless and musically consistent loops generation.(2) We propose an Instrument-Aware Reinforcement Learning training strategy to enhance the model's ability to generate loops with instruments matching the textual prompts.(3) We demonstrate the effectiveness of our proposed methods through extensive experiments, significantly outperforming state-of-the-art baselines in generating high-quality, relevant, and consistent music loops, as validated by objective and subjective metrics. Furthermore, we conduct comprehensive ablation studies to analyze the impact of each key component and their individual contributions to the overall performance of our model. 
related work: 
 In this section, we review the relevant literature in the key areas of loop music generation. We discuss the various approaches and studies in relation to our proposed Loops Transformer model.The Transformer based models [32] has inspired various tokenization methods for effectively modeling the sequences of MIDI data. REMI [14] encodes notes as a combination of Pitch, Velocity, Duration, Bar, and Position tokens, enabling Transformers to better understand music structure. MIDI-Like [21] directly converts MIDI messages into tokens, a method employed by models like Music Transformer [11] and MT3 [9] . CPWord [10] builds upon REMI, reducing sequence length by combining embeddings.Our work focuses on diverse MIDI data, including single-track and multi-track compositions with various instruments. We leverage REMI+ [33] , an extension of REMI designed for general multitrack, multi-signature symbolic music sequences, introducing Program tokens for efficient representation of multiple instruments. We employ REMI+ as the tokenizer of our models.There are a few studies that have explored the topic of loop generation. [37] proposes an unconditional loop generation model based on StyleGAN2 [15] to create 2-second loops, while [39] presents a system that integrates ChatGPT with music generation models [3, 8] to enable interactive and iterative music creation through a multi-round dialogue interface.However, these works do not fully address the fundamental aspects of loops, which are essential building blocks in modern music production. Loops allow creators to construct intricate compositions by repeatedly using melodic, rhythmic, or textural elements, with precise control over melody, rhythm, and instrumentation being crucial to the creative process.Our work focuses on generating musical loops from text and MIDI inputs. By combining textual prompts with specific MIDI settings, our approach facilitates the creation of customized loops, expediting the experimentation and exploration phases of music production while providing composers with a more interactive and adaptable creative environment.Text-to-music generation models have made significant progress in recent years, with approaches based on Diffusion Models [26, 29] , such as Noise2Music [12] , AudioLDM 2 [19] , JEN-1 [18] , and MusicLDM [2] , as well as those utilizing Vector Quantization (VQ) [31] and Residual Vector Quantization (RVQ) [4, 38] , like Jukebox [5] , MusicLM [1] , and MusicGen [3] . While these models excel at capturing global musical features such as style and mood, they often struggle to precisely render critical attributes for music loop generation, including melody, rhythm, and instrumentation.Recent works have aimed to enhance the controllability of generated music by incorporating additional features. [3] explores melody conditioning using chromagrams, while [34] proposes Music Con-trolNet, a diffusion-based model with time-varying controls over melody, dynamics, and rhythm.Unlike previous works [3, 34] that rely on chromagrams for melody conditioning, our approach harnesses MIDI as a more comprehensive conditional input, encompassing pitch, velocity, duration, bar, position, multi-tracks, and multi-instruments information. We introduce Loops Transformer, a controllable music loops generation model that integrates text and MIDI using a novel Multi-Stage Cross Attention mechanism, improves upon the Codebook Interleaving Pattern [1, 3] for seamless looping, and employs a Loop Shift data augmentation mechanism and an Instrument-Aware Reinforcement Learning strategy to generate high-quality, relevant, and consistent music loops that match textual prompts and MIDI conditions. Extensive evaluations demonstrate our model's superiority compared to state-of-the-art baselines.  
methodology: 
 The overview of the Loops Transformer architecture is shown in Figure 1 . The following subsections will provide explanations for each component.In the proposed architecture, the conversion of text into an embedding representation marks the commencement of the process.We denote a text sequence by 𝑠, which is input into a transformerbased encoder, specifically T text . This encoder maps the input text into a high-dimensional vector space, yielding the primary text embeddings T 𝑠 = T text (𝑠).In this context, T 𝑠 materializes as a sequence of vectors in R 𝑙 ×𝑑 𝑡 , where 𝑙 characterizes the sequence length and 𝑑 𝑡 defines the dimensionality of the embedding space. The encoding procedure adheres to the approach by [3] , utilizing the T5 [24] encoder as T text .Advancing to integrate text with MIDI data, we introduce a secondary encoding step using the Text-to-MIDI Transformer encoder T t2m , which has been pre-trained on our MIDI Loops Dataset (with text and MIDI paired data). This step is designed to refine the text embeddings by incorporating musical context. Thus, the secondary text embeddings, denoted as T 𝑠 ′ = T t2m (𝑠), are generated.The secondary embeddings T 𝑠 ′ not only retain the encapsulated textual information but also incorporate a musical context, resulting in a sequence of vectors in R 𝑙 ×𝑑 𝑡 ′ . The dimension 𝑑 𝑡 ′ is representative of the embedding space.In this work, the Text-to-MIDI Transformer leverages the same encoder architecture and tokenizer as utilized by the T5-base model [24] . We employ REMI+ for MIDI tokenization, as detailed in [33] . To address the inherently longer sequence lengths of MIDI tokens relative to text, we adopt the Light Decoder architecture described by [36] , modifying the T5 decoder to accommodate these differences. This approach reduces the computational and memory consumption, thus enhancing our model's capability to process extended sequences.Concurrently, the MIDI input 𝑚, consisting of the sequence of tokens with their corresponding attributes such as pitch, velocity, and duration, are processed by a MIDI Transformer T MIDI with the REMI+ [33] tokenizer. This results in a series of MIDI embeddings M, which are represented as M = T MIDI (𝑚).with each M ∈ R 𝑙 𝑚 ×𝑑 𝑚 , where 𝑙 𝑚 is the length of MIDI representation sequence and 𝑑 𝑚 is the MIDI embedding size.The foundational principle of the proposed Multi-Stage Cross-Attention Mechanism involves leveraging the embeddings generated by the Text-to-MIDI Transformer as intermediaries. This mechanism encompasses two distinct stages of cross-attention [32] operations followed by a final integration stage:Textual Embeddings Cross-Attention: In the initial stage, cross-attention is applied between the primary text embeddings T 𝑠 and the secondary text embeddings T 𝑠 ′ , resulting in a new sequence of embeddings, denoted by C txt , which integrates the textual content with the encoded musical context as:C txt = CrossAttention(T 𝑠 , T 𝑠 ′ ).MIDI Embeddings Cross-Attention: Subsequently, a crossattention operation is performed between the MIDI embeddings M and the secondary text embeddings T 𝑠 ′ . This generates another new sequence of embeddings, C MIDI , that merges the musical information with the contextual data from the text as:C MIDI = CrossAttention(M, T 𝑠 ′ ).Final Cross-Attention Stage: The final stage involves a crossattention operation between the newly generated embeddings sequences C txt and C MIDI , producing the ultimate embedding sequence C final that encapsulates both textual and musical contexts comprehensively as:C final = CrossAttention(C txt , C MIDI ).The cross-attention operation is mathematically defined as:CrossAttention(Q, K, V) = Softmax QK 𝑇 √︁ 𝑑 𝑘 Vwhere Q represents the query matrix from one set of embeddings, K and V represent the key and value matrices from another set of embeddings, and 𝑑 𝑘 is the dimensionality of the key vectors, which is used for scaling.The Multi-Stage Cross-Attention Mechanism aims to seamlessly fuse textual and musical data through successive layers.The Loops Transformer serves as the core generative model of our architecture, leveraging the rich contextual embeddings C final to guide the autoregressive generation of discrete audio tokens. Following the approach of the MusicGen Transformer introduced by [3] , we employ the 32 kHz EnCodec [4] to convert audio into a discrete representation.The primary objective of the Loops Transformer is to model the conditional probability distribution over a sequence of discrete audio tokens A = (𝑎 1 , 𝑎 2 , . . . , 𝑎 𝑀 ), conditioned on the fused textual and musical representation C final :EQUATIONwhere 𝑎 𝑗 represents the audio token at index 𝑗, and 𝑎 < 𝑗 denotes the sequence of all preceding audio tokens. The Loops Transformer employs a transformer decoder architecture, tasked with predicting the subsequent audio token based on the previously generated tokens and the conditioning information embedded within C final .During the training phase, the objective is to maximize the loglikelihood of the audio token sequence, conditioned on the fused representation:EQUATIONThe Codebook Interleaving Pattern of Loops Transformer (shown in Figure 2 ) is a key component of the model, employed during both pre-training and fine-tuning stages. In the pre-training phase, the Codebook Interleaving Pattern with Loop Shift is used for data augmentation, where randomly sampled time steps (𝑠 1 and 𝑠 2 ) are added before the start token 𝑠 and after the end token 𝑒. This Loop Shift mechanism ensures that the content generated after the start token 𝑠 can seamlessly loop back to the content before the end token 𝑒, mimicking the characteristics of an ideal loop. During fine-tuning, the standard Codebook Interleaving Pattern is applied, incorporating the start token 𝑠 and end token 𝑒 for autoregressive modeling, which maintains the sequence's integrity while making predictions.To reconstruct the final generated musical audio, the discrete audio tokens produced by the Loops Transformer are decoded back into a continuous waveform representation using the decoder component of the EnCodec model. By leveraging the high-fidelity audio compression capabilities of EnCodec and the expressive power of the transformer architecture, the Loops Transformer enables the generation of high-quality and controllable music that aligns with the given textual and musical conditions.It is worth noting that, in order to enable the Loops Transformer to generate music not only based on the integrated text and MIDI conditions but also solely on the text description, we introduce a training strategy that randomly replaces C final with C txt during the training process. Specifically, with a probability of 1/3 (as set in our experiments), we substitute C final with C txt , allowing the model to learn how to generate music based only on the text condition, i.e., in the absence of MIDI information. This training strategy endows the Loops Transformer with greater flexibility,To further enhance the Loops Transformer's ability to generate music loops that align with the specified instruments in the textual prompts, we propose an Instrument-Aware Reinforcement Learning approach. This method leverages the Proximal Policy Optimization (PPO) algorithm [28] and a novel reward function based on an instrument classification model.Let 𝜋 𝜃 denote the policy of the Loops Transformer parameterized by 𝜃 , and let 𝑅(A) represent the reward function for a generated discrete audio sequence A. The objective of the reinforcement learning process is to maximize the expected reward:EQUATIONTo compute the reward, we introduce an instrument classification model Φ 𝜓 parameterized by 𝜓 , which is trained to predict the presence of various instruments in a given audio sequence. The model is trained using the same architecture as the Loops Transformer but with a multi-label classification objective. Let y denote the ground truth instrument labels for a discrete audio sequence, and let ŷ = Φ 𝜓 (A) represent the predicted instrument probabilities. We define the reward function as :EQUATION)where 𝑁 is the number of instrument classes, 𝑦 𝑖 is the ground truth label for the presence of instrument 𝑖, ŷ𝑖 is the predicted probability of instrument 𝑖 being present, and 𝛽 is a scaling factor. This reward function is based on the binary cross-entropy loss, which measures the discrepancy between the predicted probability distribution and the true label distribution. This reward function encourages the model to generate music loops with instrumentation that closely matches the given conditions.To optimize the policy using PPO, we define the advantage function Â𝑡 at time step 𝑡 as:EQUATION)where 𝛾 is the discount factor, 𝐾 is the number of steps in the rollout, and 𝑉 𝜙 is a value function parameterized by 𝜙, which estimates the expected cumulative reward from state s 𝑡 .The PPO objective function, L 𝑃𝑃𝑂 (𝜃 ), is constructed from two primary components: the probability ratio, 𝑟 𝑡 (𝜃 ), and the advantage estimate, Â𝑡 . Specifically, 𝑟 𝑡 (𝜃 ) quantifies the ratio of the new policy's probability of taking action a 𝑡 given state s 𝑡 to that of the old policy, mathematically represented as:EQUATIONWith these definitions, the PPO objective simplifies to:EQUATIONwhere the clip function limits 𝑟 𝑡 (𝜃 ) to the range [1-𝜖, 1+𝜖], with 𝜖 being a hyperparameter that controls the extent of clipping. This ensures modest updates to the policy, avoiding drastic deviations from the previous policy. The expectation Ê𝑡 averages over a batch of samples, calculating the expected value of the minimum between the unclipped and clipped objectives.Finally, we integrate the reinforcement learning objective with the original Loops Transformer training objective to obtain the final objective function:L final = L + 𝛽L 𝑃𝑃𝑂 (𝜃 ) (8)where 𝛽 is a hyperparameter that balances the importance of the reinforcement learning objective.By optimizing this combined objective, the Loops Transformer learns to generate music loops that not only adhere to the provided textual and musical conditions but also incorporate the desired instruments as specified in the prompts. 
experimental setup 4.1 model and hyperparameter settings: 
 For the text encoder T text , we employ the pretrained T5-base model [24] from the implementation 1 . The Text-to-MIDI Transformer encoder T t2m follows the same architecture as the T5-base model but is pre-trained on our MIDI Loops Dataset (with text and MIDI paired data). We modify the T5 decoder to accommodate the longer sequence lengths of MIDI inputs, adopting the light decoder architecture as ByT5-base [36] . The T t2m is trained for 1,000,000 steps with a batch size of 128 and a learning rate of 1e-4 using the AdamW optimizer [20] (𝛽 1 = 0.9, 𝛽 2 = 0.95, and a weight decay of 0.1).The MIDI Transformer T MIDI adopts MidiTok [7] 2 to implement the REMI+ Tokenization. It also follows the ByT5-base light decoder architecture [36] and is pretrained with our MIDI Loops Dataset (using MIDI data only) via an autoregressive manner, trained for 1,000,000 steps with a batch size of 128 and a learning rate of 1e-4 using the AdamW optimizer (𝛽 1 = 0.9, 𝛽 2 = 0.95, and a weight decay of 0.1).For the Loops Transformer, we follow the transformer decoder architecture of MusicGen-melody (1.5B) 3 , as presented in [3] , and utilize its pretrained weights. The Loops Transformer is first pretrained on our Audio Loops Dataset using the Codebook Interleaving Pattern with Loop Shift for data augmentation, with 1,000,000 training steps using the AdamW optimizer (𝛽 1 = 0.9, 𝛽 2 = 0.95, and a weight decay of 0.1) and a batch size of 128, employing a cosine learning rate schedule with a warmup of 2,500 steps. The fine-tuning process spans 500,000 steps with a batch size of 128 using the AdamW optimizer (𝛽 1 = 0.9, 𝛽 2 = 0.95, and a weight decay of 0.1). During fine-tuning, the standard Codebook Interleaving Pattern is applied. To enable the Loops Transformer to generate music based solely on text descriptions, we incorporate a training strategy that randomly replaces C final with C txt with a probability of 1/3. During the sampling process, we apply top-k sampling [6] , retaining the top 250 tokens at a temperature of 1.0, as suggested by [3] .In the Instrument-Aware Reinforcement Learning setup, the instrument classification model Φ 𝜓 follows the same architecture as the Loops Transformer. The model is trained using the binary cross-entropy loss with a learning rate of 1e-4 and a batch size of 64 for 200,000 steps, utilizing our Audio Loops Dataset for training. For the PPO algorithm, we set the discount factor 𝛾 to 0.99 and the clipping parameter 𝜖 to 0.2. The coefficient 𝛽 for balancing the reinforcement learning objective is set to 0.1. We train the Loops Transformer with the PPO objective for an additional 350,000 steps.All models are implemented using the PyTorch library [23] and trained on 8 NVIDIA A100 GPUs with 80GB memory each.We train, validate, and test our model using licensed music loops (including MIDI and wav files) created by professional musicians. These loops are accompanied by relevant musical information (e.g., BPM, key, instruments). We generate corresponding textual descriptions using ChatGPT4 and Claude5 , taking into account the descriptions of loop sample packs or loop bundles and the provided musical information. Human annotators verify and modify each description with its corresponding loop as necessary to ensure quality. The following is an example of a textual prompt of a loop: "A heartfelt piano loop in A minor at 140 BPM, weaving a narrative of delicate emotions and tender moments, ideal for storytelling and introspective journeys. ".We organize these loop data into three datasets for different stages of use. The MIDI Loops Dataset, consisting of 16,432 MIDI format loops (including single-track and multi-track data), is used to pre-train our MIDI Transformer and Text-to-MIDI Transformer. The Audio Loops Dataset, comprising 18,182 wav format loops (95.9 hours), is used for pre-training the Loops Transformer model.The MIDI-Audio Paired Loops Dataset contains 10,018 loops (Train: 8,129, Validation: 932, Test: 957), with MIDI and wav pair data for each loop, along with corresponding textual descriptions. We use this dataset to fine-tune the proposed model. To ensure the correctness and evaluability of the experiments, we carefully confirm that the test dataset comes from different sources (i.e., different packs, bundles, and producers) than the training and validation data of these three datasets (i.e. the MIDI, the Audio, and the MIDI-Audio Paired Loops Dataset), ensuring no artist overlap between the test set and the training and validation sets. This guarantees that the test data and the data used for pre-training and fine-tuning are from completely different sources, without any overlap.We use the test set of the MIDI-Audio Paired Loops Dataset for objective metric evaluation and randomly sample 150 test samples from the test set for subjective metric and the Instrumentation Score (IS) evaluation.Baseline Models. In this work, we primarily compare the proposed model with MusicGen [3]6 , a state-of-the-art text-to-music model capable of generating music conditioned on both text and melody. MusicGen serves as a strong baseline for our evaluation, as it represents the current best-performing model in the field of text-to-music generation. Since MusicGen can leverage the given audio for melody conditioning, we convert our MIDI input to audio using FluidSynth7 to provide the melody condition for MusicGen.Objective Evaluation Metrics. We evaluate the performance of the generated audio using three objective metrics: Fréchet Audio Distance (FAD) [16] , CLAP score [13, 35] , and Normalized Dynamic Time Warping Distance (Norm. DTW Distance) [27] .FAD measures the distance between the distributions of the generated audio and real audio in the feature space of a pre-trained VGGish model. We compute FAD using the official TensorFlow implementation 8 . A lower FAD score indicates that the generated audio is closer to real audio in terms of its distribution, suggesting better perceptual quality.The CLAP score, calculated using the official pre-trained CLAP model 9 , measures the semantic alignment between the generated audio and the corresponding text description. A higher CLAP score suggests that the generated audio effectively captures the semantic content of the input text.To quantify the similarity between the given MIDI melody and the generated audio, we introduce the Norm. DTW Distance. First, we convert the ground truth MIDI to audio using FluidSynth, employing the same process as we did for generating the melody condition for MusicGen. We then convert the ground truth audio and the generated audio into chromagrams. We apply an argmax operation to the chromagram, preserving only the most prominent pitch class in each frame, creating a frame-wise one-hot representation of the 12 semitones. We then compute the Normalized Dynamic Time Warping Distance between the two chromagram sequences. For each sample, we calculate the Norm. DTW Distance using a 10-second segment of the ground truth and generated audio, To further enhance our ablation studies, we have incorporated an additional objective metric: the Instrumentation Score (IS). This score is derived from multi-label instrument classification using F1 micro and F1 macro scores. The IS is crucial for assessing the models' capability to generate music that accurately represents the instruments specified in the textual descriptions. To obtain the IS, annotators, as described below, were tasked with listening to each sample generated by the model and annotating which instruments were audibly present in the loop.Subjective Evaluation Metrics. Following the experimental setup in [17] and [3] , we evaluate two subjective metrics: Overall Quality (OVL) and Relevance to Text Input (REL). Additionally, we introduce a new metric, Loop Consistency, to assess the seamlessness of the generated audio when played in a loop. Raters (annotators), recruited through Amazon Mechanical Turk, are required to have experience creating music with Digital Audio Workstations (DAWs) and experience in using or creating loops.OVL assesses the perceptual quality of the audio samples, while REL measures how well the generated audio matches the given text input. Loop Consistency evaluates the smoothness of transitions when the audio is played in a continuous loop. All metrics are rated on a scale from 1 to 100, with higher scores indicating better performance.We evaluate randomly sampled files, each assessed by a minimum of five raters to ensure reliable results. The CrowdMOS package is employed to identify and remove noisy annotations and outliers, following the guidelines provided in [25] and adopted by [3] . 
results: 
 We present the evaluation results of our proposed Loops Transformer model in comparison with the state-of-the-art baseline, Mu-sicGen [3] . Table 1 summarizes the performance of both models across various objective and subjective metrics. We consider three different settings for each model: (1) text-only conditioning, (2) text and full melody/MIDI conditioning, and (3) text and half length melody/MIDI conditioning.In terms of objective metrics, the Loops Transformer achieves lower FAD scores, indicating that the generated audio is closer to real audio in terms of its distribution. When conditioned on both text and MIDI, the Loops Transformer obtains the lowest Normalized DTW Distance of 0.11, demonstrating its ability to generate audio that closely matches the given MIDI melody. Interestingly, the Loops Transformer achieves the best FAD score of 2.5 when conditioned on text and half length MIDI, suggesting that partial MIDI information can be sufficient for generating high-quality loops audio.For the CLAP score, assessing the semantic alignment between generated audio and the corresponding text description, Music-Gen exhibits a marginal advantage over the Loops Transformer. This discrepancy can likely be ascribed to the training data each model was exposed to: while MusicGen was trained on a vast and diverse dataset encompassing a wide array of musical pieces, the Loops Transformer was specifically trained on loop-based data. This specialized focus on loops, although advantageous for generating cohesive and loopable segments, may not encompass the broader musical context to the same extent as MusicGen's training, influencing its performance on text-audio semantic alignment.In the subjective assessments, the Loops Transformer markedly outperforms MusicGen in all dimensions. Notably, with conditions involving both text and MIDI, it achieves top marks in OVL (88.25), REL (85.96), and Loop Consistency (88.79). These results highlight the Loops Transformer's capacity for generating audibly superior loops that more accurately embody the specified textual descriptions and musical conditions and demonstrate improved loop coherence compared to MusicGen.Interestingly, despite the Loops Transformer's lower performance on the CLAP score, indicative of text-audio semantic congruence, it achieves higher relevance ratings. This discrepancy may be attributed to textual prompts frequently specifying instruments (e.g., "A dynamic string loop set at 150 BPM in G minor... "), where accurate instrument representation may influence subjective relevance assessments. This contrasts with the CLAP metric potentially limited capacity to evaluate instrumentation accuracy, highlighting a discernible gap between algorithmic and human judgments regarding musical authenticity and textual congruity. Table 2 presents the evaluation results for the Instrument-Aware Reinforcement Learning (IARL) approach. We compare the instrumentation scores (IS) of the Loops Transformer with and without IARL, as well as MusicGen. The Loops Transformer with IARL achieves the highest f1 micro and f1 macro scores, indicating its superior performance in generating audio with instruments that match the textual prompts. This highlights the effectiveness of the proposed IARL training strategy in enhancing the model's instrumentation capabilities.To assess the impact of the Multi-Stage Cross Attention (MSCA) mechanism and the Text-to-MIDI Transformer, we evaluate different variations of the Loops Transformer using the FAD and CLAP metrics, as shown in Table 3 . The Loops Transformer with MSCA achieves the best FAD score of 2.6, demonstrating the effectiveness of the proposed attention mechanism in integrating text and MIDI conditions. Removing the Text-to-MIDI Transformer component results in a degradation in performance, emphasizing its importance in the overall architecture.Finally, Table 4 presents the performance of the Loops Transformer with and without the loop shift data augmentation technique on a reduced dataset (1/8 of the original pre-training data). The model trained with loop shift achieves better FAD and Normalized DTW Distance scores, highlighting the benefits of the proposed data augmentation strategy in improving the quality and consistency of the generated loops.Overall, our experimental results demonstrate the superiority of the proposed Loops Transformer model in generating high-quality, relevant, and consistent music loops compared to the state-of-theart baseline. The effectiveness of the key components, such as the Multi-Stage Cross Attention mechanism, Instrument-Aware Reinforcement Learning, and loop shift data augmentation, is validated through comprehensive ablation studies. 
conclusion and future work: 
 In this work, we introduced Loops Transformer, a novel controllable music loops generation model that integrates text and MIDI inputs to generate high-quality, relevant, and consistent music loops. We proposed a Multi-Stage Cross Attention mechanism to effectively combine textual and musical information, an improved Codebook Interleaving Pattern with Loop Shift data augmentation for seamless looping, and an Instrument-Aware Reinforcement Learning strategy to enhance the model's instrumentation capabilities.Extensive experiments demonstrated the superiority of the Loops Transformer compared to the state-of-the-art baseline, MusicGen, across various objective and subjective metrics. The proposed model achieved significant improvements in generating music loops that closely match the given text and MIDI conditions, exhibit better perceptual quality, and maintain loop consistency. Ablation studies validated the effectiveness of the key components in contributing to the overall performance of the Loops Transformer.Our work addresses the challenges in generating music loops that align with the detailed requirements of contemporary music production, offering a more interactive and adaptable creative framework for composers and music producers. The Loops Transformer empowers users to create customized loops by combining textual prompts with specific MIDI settings, accelerating the experimentation and exploration phases of music creation.Future research directions include incorporating audio input as a conditioning signal, expanding the model's capability to generate longer musical sequences, and developing an interactive refinement process for music loops generation. The Loops Transformer represents a significant step towards bridging the gap between the creative vision of composers and the technical requirements of modern music production, offering a powerful and adaptable solution for generating high-quality, relevant, and consistent music loops. 
 
 Appendix: https://github.com/facebookresearch/audiocraft/blob/main/docs/MUSICGEN.mdhttps://chat.openai.com/https://claude.ai/https://huggingface.co/facebook/musicgen-melodyhttps://www.fluidsynth.org/https://github.com/google-research/google-research/tree/master/frechet_audio_distancehttps://github.com/LAION-AI/CLAP 


Decision: The paper identifies a significant issue in current text-to-music generation systems: the lack of precise rendering of music loop features such as melody, rhythm, and instrumentation. To address this issue, the paper proposes a Multi-Stage Cross Attention mechanism and an Instrument-Aware Reinforcement Learning technique, which enhance controllability and ensure accurate instrumentation. Experimental results show the effectiveness of the proposed controllable music generation system. Despite above strengths, reviewers have also identified several weaknesses which include: 1) missing references for loop generation: 2) ill-defined terms such as Half melody/half MIDI; 3) the difference between repetition and loops in music should be clarified.
In consideration of all review comments including the authors' rebuttal, this is a borderline paper for MM. However, its merits slightly outweigh its weaknesses. I recommend an accept and strongly encourage the authors to take reviewers' critiques into consideration when preparing for the final submission, if this paper is accepted.

Reviews: review: paper_summary: paper_summary: This paper presents a Transformer-based system for generating music that can be controlled via text and MIDI inputs. The authors introduce a Multi-Stage Cross Attention mechanism to improve the controllability of both types of input. Additionally, they employ an Instrument-Aware Reinforcement Learning technique to ensure accurate instrument arrangement. The results indicate that the proposed system can produce music loops that align closely with both text and MIDI inputs, maintaining correct instrumentation throughout.
strength_weakness: strength_weakness: 1. The paper identifies a significant issue in current text-to-music generation systems: the lack of precise rendering of music loop features such as melody, rhythm, and instrumentation.
2. To address this, the authors propose a Multi-Stage Cross Attention mechanism and an Instrument-Aware Reinforcement Learning technique, which together enhance controllability and ensure accurate instrumentation.
3. The paper includes thorough and comprehensive experiments that demonstrate the effectiveness of the proposed controllable music generation system.
limitations: limitations: 1. The quality of the figures in this paper is poor; please provide higher-resolution versions.
2. The equations should be centered on the line and labeled (e.g., (1), (2), (3), etc.). Additionally, if the equation is not at the end of a paragraph, a comma should be added at the end.
3. Since the dataset is self-generated, I would prefer to see more detailed statistical information. For instance, provide statistics on the text prompts (descriptions) created by ChatGPT-4 and Claude.
4. Although this is a minor issue, I'm curious about the rationale for using a 1-100 scale for MOS, as it can be challenging for respondents to distinguish between closely spaced scores (e.g., 75 vs. 76).
5. In the demo page you provided, I noticed that in the section "Textual Description and Musical Conditions (with Explicit Instrument Specification)," there is no demonstration for cases where the text description contains multiple instruments.
paper_summary: paper_summary: This paper addresses the limitations of current text-to-music generation models in accurately rendering essential music loop attributes like melody, rhythms, and instrumentation. It introduces a new approach through the Loops Transformer and Multi-Stage Cross Attention mechanism that integrates textual and MIDI inputs effectively. An Instrument-Aware Reinforcement Learning technique is also developed to ensure accurate instrumentation. The model demonstrates superior performance over the state-of-the-art, improving both objective and subjective metrics, indicating its ability to produce musically coherent loops that meet the demands of modern music production.
strength_weakness: strength_weakness: 1. Importance of Research Topic: The research enhances multimedia by integrating text and music generation, advancing the interaction between textual descriptions and musical compositions. This integration enriches user experience and opens new avenues for creative expression.

2. Rigorous Technical Approach: The paper uses advanced techniques like Loops Transformer, Multi-Stage Cross Attention, and Instrument-Aware Reinforcement Learning. These methods improve the precision and coherence of music generation, aligning closely with textual and MIDI inputs.

3. Potential Application Scenarios: The model has broad implications in music industry. It can revolutionize music creation.
limitations: limitations: Questions Related to Figures 1 and 2:
1.1: What are the input and output of the model (MIDI, audio, text, or a combination)?
1.2: What is the main technical contribution of the work?
1.3: How do the figures relate to Section 3, especially the mathematical notations in the content and the figures?
1.4: What is the relationship between Figure 1 and Figure 2?

Questions Related to Datasets:
2.1: How do GPT-4 or Claude analyze and generate textual descriptions from MIDI or MP3, particularly for descriptive and subjective content?
2.2: How was the MIDI-Audio pair dataset collected, and is it open-sourced?

Questions in Experiment:
3.1: What does "half melody/half MIDI" mean?
3.2: What is the specific meaning of REL, and how is the relevance of the output to the text measured, considering detailed and subjective descriptions?
3.3: How do the authors prove in the experiments that the proposed model precisely renders melody, rhythm, and instrumentation, addressing the limitations of other models?

Overall Questions:
4.1: What is the difference between repetition and loops in music, and why are works discussing repetition structure not mentioned? What are the differences and similarities between these works and this work?
4.2: What is the difference between your work and other works such as MusicGen? How does your model generate results that better fit text descriptions?
4.3: Why is an instrument-aware mechanism needed to identify instruments when instrument information can be directly specified and modified if required?
4.4: What is the motivation for taking MIDI as input but outputting audio?
4.5: What is the specific controllability of the model, and how does it allow users to control the output? What scope of control do users have over the result?
4.6: In the demo, why are the generated results similar to the input with explicit instrument specifications, but different without explicit specifications, especially in examples 2, 3, and 4?


Overall, it is an interesting work with many insightful ideas, but due to the paper's structure and logic, it is challenging to identify and validate the main contributions of the work.
paper_summary: paper_summary: This paper introduces the Loops Transformer, a new framework for generating music loops by combining text and MIDI inputs. It employs a Multi-Stage Cross Attention mechanism to incorporate both textual and musical data, an enhanced Codebook Interleaving Pattern for seamless looping, and Instrument-Aware Reinforcement Learning to improve instrumentation in generated songs. Comparative experiments show that the Loops Transformer outperforms other state-of-the-art text-to-music generators by generating loops that align with input conditions, have higher perceptual quality, and maintain consistency. Ablation studies confirm the importance of its key components in achieving better performance.
strength_weakness: strength_weakness: * This work is well motivated - generating loops from texts is an understudied problem. This work can potentially serve as a useful supporting tool for expert composers.
* This paper is generally well-written, with some minor flaws & stylewise issues (see the limitations below) that sometimes may influence reading. 
* The proposed method is intuitive and elegantly resolves existing problems.
limitations: limitations: * Lines 751-753, more details of the three settings should be added (e.g. what does "half-length" refer to?)
* For the ablation studies (Tables 2, 3, 4), it seems the performance was not measured on human ratings. Do $FAD_{vgg}$ and $CLAP$ suffice to assess the quality of generated audio?
* For Table 4, why 1/8 of the pretraining data were used (why not 1/2, 1/4, 1/16, 1/32). Does reducing data always lead to performance drop? Is it possible that the selected 1/8 is of lower quality than others and therefore influences the performance?
* In the provided listenable samples, personally I feel that Loops Transformer (LT) tends to use simpler instrumentations and effects (e.g. fewer instruments, more plain instruments) compared to MusicGen (MG). This is noticeable especially under energetic music, for example, Example 1 under Textual Description only, where the result from LT seems less spirited than MG due to the instrumentation. In addition, in Example 2 under Textual Description Only, I feel that there's been a sudden change in LT's generation (approximately at 12s) that seems to belie the description "flow gently" compared to MG's generation. Therefore, I feel that while admittedly LT can provide finer-grained controls (e.g. keys, BPMs), the style-wise control and faithfulness to conditions do not exhibit strong improvements from MG. 
* Some minor nitpicks:
	* Line 230, has --> have
	* Section 3 seems to have some traits of AI writing, with many flowery wordings (e.g. marks the commencement, materializes, characterizes, advancing to, is representative of,)
	* In 4.2, are the three datasets (in bold) original? Any citations of the source of these datasets?
	* Using "interestingly" (Lines 777, 801, etc) does not sound formal and academic.
paper_summary: paper_summary: This paper proposes a Loops Transformer that can integrate text and MIDI inputs to generate music loops. Multi-Stage Cross Attention mechanism, Instrument-Aware Reinforcement Learning, and Codebook Interleaving Pattern with Loop Shift are used for effectively combining textual and musical information, enhancing the model’s instrumentation capabilities, and augmenting data, respectively.
strength_weakness: strength_weakness: 1. The authors collected the MIDI-Audio Paired Loops Dataset to achieve the goal of this paper, i.e., generating loops given both textual descriptions and MIDI inputs. The MIDI input facilitates the precise rendering of critical music loop attributes.
2. A Multi-Stage Cross Attention mechanism is devised to integrate text and MIDI inputs.
3. The proposed Loops Transformer outperforms the state-of-the-art baseline model MusicGen in terms of loops generation.
limitations: limitations: 1.	Missing references, the following studies are also relevant to loop generation:
* Symbolic Music Loop Generation with Neural Discrete Representations
* Context-aware Generation of Melodic MIDI Loops
* A Benchmarking Initiative for Audio-Domain Music Generation Using the Freesound Loop Dataset
2.	It's still not clear how $\mathrm{C_{txt}}$, $\mathrm{C_{MIDI}}$, and $\mathrm{C_{final}}$ are computed.
3.	Incomplete sentence:
* Left line 487-488: "This training strategy endows the Loops Transformer with greater flexibility, …"
4.	Lack of explanation for introducing reinforcement learning:
* Why not directly use instrument classification loss, i.e., binary cross-entropy loss, and instead introduce reinforcement learning and use the PPO loss? The reviewer is curious about the experimental results of using the instrument classification loss.
5.	Why do the subjective metrics include only Relevance to Text Input and not Relevance to MIDI/Audio Input?