Update README.md

README.md

@@ -1,3 +1,30 @@
 ---
 license: mit
 ---
+
+# M<sup>2</sup>UGen Model with AudioLDM2-Music
+
+The M<sup>2</sup>UGen model is a Music Understanding and Generation model that is capable of Music Question Answering and also Music Generation 
+from texts, images, videos and audios, as well as Music Editing. The model utilizes encoders such as MERT for music understanding, ViT for image understanding 
+and ViViT for video understanding and the MusicGen/AudioLDM2 model as the music generation model (music decoder), coupled with adapters and the LLaMA 2 model 
+to make the model possible for multiple abilities. 
+
+M<sup>2</sup>UGen was published in [M<sup>2</sup>UGen: Multi-modal Music Understanding and Generation with the Power of Large Language Models](https://arxiv.org/abs/2311.11255) by *Atin Sakkeer Hussain, Shansong Liu, Chenshuo Sun and Ying Shan*.
+
+The code repository for the model is published in [crypto-code/M2UGen](https://github.com/crypto-code/M2UGen). Clone the repository, download the checkpoint and run the following for a model demo:
+```bash
+python gradio_app.py --model ./ckpts/M2UGen-AudioLDM2/checkpoint.pth --llama_dir ./ckpts/LLaMA-2 --music_decoder audioldm2 --music_decoder_path cvssp/audioldm2-music
+```
+
+## Citation
+
+If you find this model useful, please consider citing: 
+
+```bibtex
+@article{hussain2023m,
+  title={{M$^{2}$UGen: Multi-modal Music Understanding and Generation with the Power of Large Language Models}},
+  author={Hussain, Atin Sakkeer and Liu, Shansong and Sun, Chenshuo and Shan, Ying},
+  journal={arXiv preprint arXiv:2311.11255},
+  year={2023}
+}
+```