Update README.md

README.md

@@ -1,3 +1,56 @@
----
-license: mit
----
+---
+license: apache-2.0
+datasets:
+- openbmb/UltraFeedback
+- HuggingFaceH4/ultrafeedback_binarized
+---
+
+# Model Card for FuxiTranyu-8B-DPO
+
+## Model Summary
+
+FuxiTranyu-8B is a completely open source large language model trained from scratch, with a specific focus on the multilinguality.
+FuxiTranyu-8B was trained on 600B tokens with a much more smooth distribution across languages.
+
+We cover 43 natural languages: Arabic, Bengali, Bulgarian, Burmese, Catalan, Chinese, Czech, Dutch, English, Filipino, Finnish, French, German, Greek, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Kazakh, Khmer, Korean, Kurdish, Kyrgyz, Lao, Malay, Persian, Polish, Portuguese, Romanian, Russian, Spanish, Swedish, Tamil, Tajik, Thai, Turkish, Turkmen, Ukrainian, Urdu, Uzbek, and Vietnamese.
+And also cover 16 programming languages: Java, JavaScript, Python, PHP, C, C++, C#, TypeScript, Go, SQL, Rust, Ruby, Scala, Lua, Assembly, and Visual Basic.
+
+FuxiTranyu-8B-DPO is trained using DPO with the UltraFeedback dataset, based on the [FuxiTranyu-8B](https://huggingface.co/TJUNLP/FuxiTranyu-8B) model.
+
+More details can be found at our technical report.
+
+We also provide the instruction fine-tuned version at [here](https://huggingface.co/TJUNLP/FuxiTranyu-8B-SFT).
+## Usage
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+model_path = "TJUNLP/FuxiTranyu-8B-DPO"
+
+tokenizer = AutoTokenizer.from_pretrained(model_path, trust_remote_code=True)
+
+model = AutoModelForCausalLM.from_pretrained(model_path, device_map="auto", torch_dtype='auto', trust_remote_code=True)
+
+messages = [{"role": "user", "content": "This is an input text:"}]
+# format messages with the ChatML chat template
+input_ids = tokenizer.apply_chat_template(messages, tokenize=True, add_generation_prompt=True, return_tensors="pt").to(model.device)
+# <|im_start|>user\nThis is an input text:<|im_end|>\n<|im_start|>assistant\n
+
+output_ids = model.generate(input_ids, max_new_tokens=20)
+response = tokenizer.decode(output_ids[0], skip_special_tokens=True)
+
+print(response)
+
+```
+
+## Citation info
+
+```bibtex
+@misc{FuxiTranyu8B,
+      title={FuxiTranyu: A Multilingual Large Language Model Trained with Balanced Data}, 
+      author={Haoran Sun, Renren Jin, Shaoyang Xu, Leiyu Pan, Supryadi, Menglong Cui, Jiangcun Du, Yikun Lei, Lei Yang, Ling Shi, Juesi Xiao, Shaolin Zhu, and Deyi Xiong},
+      year={2024},
+      eprint={2408},
+      archivePrefix={arXiv},
+      primaryClass={cs.CL}
+}
+```