# Enhancing Diffusion-Based Music Generation Performance with LoRA
## 5 Discussion  
This paper applies LoRA [10] to the AudioLDM [5] model to verify the possibility of maintaining or improving music generation performance without full fine-tuning. The experimental results revealed that CLAP [24] scores gradually increased with higher rank and alpha values, outperforming the baseline model. In contrast, the KAD [25] scores were generally lower for the proposed method compared with the baseline model, demonstrating improved performance in terms of statistical distribution similarity. In particular, the LoRA settings with rank = alpha = 1 and rank = alpha = 2 displayed the best KAD performance during the early training stages. These results suggest that lightweight fine-tuning using LoRA can be efficiently applied to music generation models. The LoRA method significantly reduces the number of trainable parameters while maintaining generation quality, making it a practical approach in resource-constrained environments. Furthermore, by applying genre-specific LoRA modules, various domain-specific music generation functions can be flexibly expanded on a single base model. This proposed method enables flexibility and scalability in music generation models. To further enhance the effectiveness and expand the applicability of LoRA in music generation, future studies could investigate its application to other components, such as cross-attention layers, variational autoencoders, or embedding adjustment layers related to the CLAP model. Additionally, systematic experiments involving various LoRA configurations (e.g., different rank values, dropout rates, and parallel setups) could help identify optimal architectures via quantitative and qualitative analyses. In particular, ablation studies involving dropout rates, initialization methods, and parallel configurations beyond rank \begin{table}
\begin{tabular}{l l c c} \hline \hline
**Prompt ID** & **Model** & **CLAP Score \(\uparrow\)** & **KAD Score \(\downarrow\)** \\ \hline \multirow{2}{*}{P1} & Baseline model & 0.5882 & 0.7253 \\  & Proposed method & 0.6394 & 0.4701 \\ \hline \multirow{2}{*}{P2} & Baseline model & 0.5785 & 0.5981 \\  & Proposed method & 0.6357 & 0.4637 \\ \hline \multirow{2}{*}{P3} & Baseline model & 0.6693 & 0.5315 \\  & Proposed method & 0.7082 & 0.4748 \\ \hline \hline \end{tabular}
\end{table} 
Table 10: Comparison of CLAP and KAD scores on non-Western prompts. \begin{table}
\begin{tabular}{l l c c c} \hline \hline
**Prompt ID** & **Model** & **Generation Time (s)** & **Music Length (s)** & **RTF** \\ \hline \multirow{2}{*}{P1} & Baseline model & 6.78 & 4.0 & 1.69\(\times\) \\  & Proposed method & 7.65 & 4.0 & 1.91\(\times\) \\ \hline \multirow{2}{*}{P2} & Baseline model & 6.67 & 4.0 & 1.67\(\times\) \\  & Proposed method & 7.62 & 4.0 & 1.90\(\times\) \\ \hline \multirow{2}{*}{P3} & Baseline model & 6.70 & 4.0 & 1.68\(\times\) \\  & Proposed method & 7.64 & 4.0 & 1.91\(\times\) \\ \hline \hline \end{tabular}
\end{table} 
Table 11: Generation latency comparison on non-Western prompts. and alpha could improve stability and mitigate overfitting, as suggested by the inverse relationship observed between CLAP and KAD scores. Visualizing loss curves across configurations would also enhance interpretability and training insight. Furthermore, due to its low computational overhead, LoRA is highly suitable for transfer learning across genres or themes. A promising research direction is to develop a transfer learning framework that reuses LoRA weights trained on base genres to adapt rapidly to subgenres or hybrid styles. Another promising direction is to extend LoRA integration to support multimodal inputs, such as combining textual prompts with audio references or visual cues (e.g., mood boards). This could enhance controllability for hybrid applications like film scoring, where prompt fusion across modalities plays a crucial role. Moreover, extending the controllable generation capabilities to support hybrid musical styles or emotion-based conditioning could enable more expressive and user-aligned outputs.