ManzhenWei
commited on
Commit
•
420c78e
1
Parent(s):
63bd2b1
Update README.md
Browse files
README.md
CHANGED
@@ -1,3 +1,38 @@
|
|
1 |
-
---
|
2 |
-
license: mit
|
3 |
-
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
---
|
2 |
+
license: mit
|
3 |
+
---
|
4 |
+
# Model Card
|
5 |
+
|
6 |
+
## Model Overview
|
7 |
+
The **Melody Guided Music Generation (MG²)** model is an innovative approach that uses melody to guide music generation, achieving impressive results despite its simplicity and minimal resource requirements. MG² aligns melody with audio waveforms and text descriptions via a multimodal alignment module and conditions its diffusion module on these learned melody representations. This enables MG² to create music that matches the style of given audio and reflects the content of text descriptions.
|
8 |
+
|
9 |
+
## Demo
|
10 |
+
Explore the capabilities of the MG² model through an online demo:
|
11 |
+
- **Demo Link**: [Model Demo](https://awesome-mmgen.github.io/)
|
12 |
+
- **Instructions**: Input a text description, then click "Generate" to see the music generated by the model.
|
13 |
+
|
14 |
+
## GitHub Repository
|
15 |
+
Access the code and additional resources for the MG² model:
|
16 |
+
- **GitHub Link**: [MG² on GitHub](https://github.com/shaopengw/Awesome-Music-Generation)
|
17 |
+
|
18 |
+
## Integration with Transformers and Hugging Face Hub
|
19 |
+
We are currently working on integrating MG² into the **Hugging Face Transformers** library and making it available on the **Hugging Face Hub** 🤗.
|
20 |
+
|
21 |
+
## Paper
|
22 |
+
> **Title**: "Melody Is All You Need For Music Generation"
|
23 |
+
> **Authors**: Shaopeng Wei, Manzhen Wei, Haoyu Wang, Yu Zhao, Gang Kou
|
24 |
+
> **Year**: 2024
|
25 |
+
> [arXiv Link](https://arxiv.org/abs/2409.20196)
|
26 |
+
|
27 |
+
|
28 |
+
## Citation
|
29 |
+
```bibtex
|
30 |
+
@article{wei2024melodyneedmusicgeneration,
|
31 |
+
title={Melody Is All You Need For Music Generation},
|
32 |
+
author={Shaopeng Wei and Manzhen Wei and Haoyu Wang and Yu Zhao and Gang Kou},
|
33 |
+
year={2024},
|
34 |
+
eprint={2409.20196},
|
35 |
+
archivePrefix={arXiv},
|
36 |
+
primaryClass={cs.SD},
|
37 |
+
url={https://arxiv.org/abs/2409.20196},
|
38 |
+
}
|