Quantization made by Richard Erkhov. [Github](https://github.com/RichardErkhov) [Discord](https://discord.gg/pvy7H8DZMG) [Request more models](https://github.com/RichardErkhov/quant_request) ALMA-13B-R - GGUF - Model creator: https://huggingface.co/haoranxu/ - Original model: https://huggingface.co/haoranxu/ALMA-13B-R/ | Name | Quant method | Size | | ---- | ---- | ---- | | [ALMA-13B-R.Q2_K.gguf](https://huggingface.co/RichardErkhov/haoranxu_-_ALMA-13B-R-gguf/blob/main/ALMA-13B-R.Q2_K.gguf) | Q2_K | 4.52GB | | [ALMA-13B-R.IQ3_XS.gguf](https://huggingface.co/RichardErkhov/haoranxu_-_ALMA-13B-R-gguf/blob/main/ALMA-13B-R.IQ3_XS.gguf) | IQ3_XS | 4.99GB | | [ALMA-13B-R.IQ3_S.gguf](https://huggingface.co/RichardErkhov/haoranxu_-_ALMA-13B-R-gguf/blob/main/ALMA-13B-R.IQ3_S.gguf) | IQ3_S | 5.27GB | | [ALMA-13B-R.Q3_K_S.gguf](https://huggingface.co/RichardErkhov/haoranxu_-_ALMA-13B-R-gguf/blob/main/ALMA-13B-R.Q3_K_S.gguf) | Q3_K_S | 5.27GB | | [ALMA-13B-R.IQ3_M.gguf](https://huggingface.co/RichardErkhov/haoranxu_-_ALMA-13B-R-gguf/blob/main/ALMA-13B-R.IQ3_M.gguf) | IQ3_M | 5.57GB | | [ALMA-13B-R.Q3_K.gguf](https://huggingface.co/RichardErkhov/haoranxu_-_ALMA-13B-R-gguf/blob/main/ALMA-13B-R.Q3_K.gguf) | Q3_K | 5.9GB | | [ALMA-13B-R.Q3_K_M.gguf](https://huggingface.co/RichardErkhov/haoranxu_-_ALMA-13B-R-gguf/blob/main/ALMA-13B-R.Q3_K_M.gguf) | Q3_K_M | 5.9GB | | [ALMA-13B-R.Q3_K_L.gguf](https://huggingface.co/RichardErkhov/haoranxu_-_ALMA-13B-R-gguf/blob/main/ALMA-13B-R.Q3_K_L.gguf) | Q3_K_L | 6.45GB | | [ALMA-13B-R.IQ4_XS.gguf](https://huggingface.co/RichardErkhov/haoranxu_-_ALMA-13B-R-gguf/blob/main/ALMA-13B-R.IQ4_XS.gguf) | IQ4_XS | 6.54GB | | [ALMA-13B-R.Q4_0.gguf](https://huggingface.co/RichardErkhov/haoranxu_-_ALMA-13B-R-gguf/blob/main/ALMA-13B-R.Q4_0.gguf) | Q4_0 | 6.86GB | | [ALMA-13B-R.IQ4_NL.gguf](https://huggingface.co/RichardErkhov/haoranxu_-_ALMA-13B-R-gguf/blob/main/ALMA-13B-R.IQ4_NL.gguf) | IQ4_NL | 6.9GB | | [ALMA-13B-R.Q4_K_S.gguf](https://huggingface.co/RichardErkhov/haoranxu_-_ALMA-13B-R-gguf/blob/main/ALMA-13B-R.Q4_K_S.gguf) | Q4_K_S | 6.91GB | | [ALMA-13B-R.Q4_K.gguf](https://huggingface.co/RichardErkhov/haoranxu_-_ALMA-13B-R-gguf/blob/main/ALMA-13B-R.Q4_K.gguf) | Q4_K | 7.33GB | | [ALMA-13B-R.Q4_K_M.gguf](https://huggingface.co/RichardErkhov/haoranxu_-_ALMA-13B-R-gguf/blob/main/ALMA-13B-R.Q4_K_M.gguf) | Q4_K_M | 7.33GB | | [ALMA-13B-R.Q4_1.gguf](https://huggingface.co/RichardErkhov/haoranxu_-_ALMA-13B-R-gguf/blob/main/ALMA-13B-R.Q4_1.gguf) | Q4_1 | 7.61GB | | [ALMA-13B-R.Q5_0.gguf](https://huggingface.co/RichardErkhov/haoranxu_-_ALMA-13B-R-gguf/blob/main/ALMA-13B-R.Q5_0.gguf) | Q5_0 | 8.36GB | | [ALMA-13B-R.Q5_K_S.gguf](https://huggingface.co/RichardErkhov/haoranxu_-_ALMA-13B-R-gguf/blob/main/ALMA-13B-R.Q5_K_S.gguf) | Q5_K_S | 8.36GB | | [ALMA-13B-R.Q5_K.gguf](https://huggingface.co/RichardErkhov/haoranxu_-_ALMA-13B-R-gguf/blob/main/ALMA-13B-R.Q5_K.gguf) | Q5_K | 8.6GB | | [ALMA-13B-R.Q5_K_M.gguf](https://huggingface.co/RichardErkhov/haoranxu_-_ALMA-13B-R-gguf/blob/main/ALMA-13B-R.Q5_K_M.gguf) | Q5_K_M | 8.6GB | | [ALMA-13B-R.Q5_1.gguf](https://huggingface.co/RichardErkhov/haoranxu_-_ALMA-13B-R-gguf/blob/main/ALMA-13B-R.Q5_1.gguf) | Q5_1 | 9.1GB | | [ALMA-13B-R.Q6_K.gguf](https://huggingface.co/RichardErkhov/haoranxu_-_ALMA-13B-R-gguf/blob/main/ALMA-13B-R.Q6_K.gguf) | Q6_K | 9.95GB | | [ALMA-13B-R.Q8_0.gguf](https://huggingface.co/RichardErkhov/haoranxu_-_ALMA-13B-R-gguf/blob/main/ALMA-13B-R.Q8_0.gguf) | Q8_0 | 12.88GB | Original model description: --- license: mit --- **[ALMA-R](https://arxiv.org/abs/2401.08417)** builds upon [ALMA models](https://arxiv.org/abs/2309.11674), with further LoRA fine-tuning with our proposed **Contrastive Preference Optimization (CPO)** as opposed to the Supervised Fine-tuning used in ALMA. CPO fine-tuning requires our [triplet preference data](https://huggingface.co/datasets/haoranxu/ALMA-R-Preference) for preference learning. ALMA-R now can matches or even exceeds GPT-4 or WMT winners! ``` @misc{xu2024contrastive, title={Contrastive Preference Optimization: Pushing the Boundaries of LLM Performance in Machine Translation}, author={Haoran Xu and Amr Sharaf and Yunmo Chen and Weiting Tan and Lingfeng Shen and Benjamin Van Durme and Kenton Murray and Young Jin Kim}, year={2024}, eprint={2401.08417}, archivePrefix={arXiv}, primaryClass={cs.CL} } ``` ``` @misc{xu2023paradigm, title={A Paradigm Shift in Machine Translation: Boosting Translation Performance of Large Language Models}, author={Haoran Xu and Young Jin Kim and Amr Sharaf and Hany Hassan Awadalla}, year={2023}, eprint={2309.11674}, archivePrefix={arXiv}, primaryClass={cs.CL} } ``` # Download ALMA(-R) Models and Dataset 🚀 We release six translation models presented in the paper: - ALMA-7B - ALMA-7B-LoRA - **ALMA-7B-R (NEW!)**: Further LoRA fine-tuning upon ALMA-7B-LoRA with contrastive preference optimization. - ALMA-13B - ALMA-13B-LoRA - **ALMA-13B-R (NEW!)**: Further LoRA fine-tuning upon ALMA-13B-LoRA with contrastive preference optimization (BEST MODEL!). Model checkpoints are released at huggingface: | Models | Base Model Link | LoRA Link | |:-------------:|:---------------:|:---------:| | ALMA-7B | [haoranxu/ALMA-7B](https://huggingface.co/haoranxu/ALMA-7B) | - | | ALMA-7B-LoRA | [haoranxu/ALMA-7B-Pretrain](https://huggingface.co/haoranxu/ALMA-7B-Pretrain) | [haoranxu/ALMA-7B-Pretrain-LoRA](https://huggingface.co/haoranxu/ALMA-7B-Pretrain-LoRA) | | **ALMA-7B-R (NEW!)** | [haoranxu/ALMA-7B-R (LoRA merged)](https://huggingface.co/haoranxu/ALMA-7B-R) | - | | ALMA-13B | [haoranxu/ALMA-13B](https://huggingface.co/haoranxu/ALMA-13B) | - | | ALMA-13B-LoRA | [haoranxu/ALMA-13B-Pretrain](https://huggingface.co/haoranxu/ALMA-13B-Pretrain) | [haoranxu/ALMA-13B-Pretrain-LoRA](https://huggingface.co/haoranxu/ALMA-13B-Pretrain-LoRA) | | **ALMA-13B-R (NEW!)** | [haoranxu/ALMA-13B-R (LoRA merged)](https://huggingface.co/haoranxu/ALMA-13B-R) | - | **Note that `ALMA-7B-Pretrain` and `ALMA-13B-Pretrain` are NOT translation models. They only experience stage 1 monolingual fine-tuning (20B tokens for the 7B model and 12B tokens for the 13B model), and should be utilized in conjunction with their LoRA models.** Datasets used by ALMA and ALMA-R are also released at huggingface now (NEW!) | Datasets | Train / Validation| Test | |:-------------:|:---------------:|:---------:| | Human-Written Parallel Data (ALMA) | [train and validation](https://huggingface.co/datasets/haoranxu/ALMA-Human-Parallel) | [WMT'22](https://huggingface.co/datasets/haoranxu/WMT22-Test) | | Triplet Preference Data | [train](https://huggingface.co/datasets/haoranxu/ALMA-R-Preference) | [WMT'22](https://huggingface.co/datasets/haoranxu/WMT22-Test) and [WMT'23](https://huggingface.co/datasets/haoranxu/WMT23-Test) | A quick start to use our best system (ALMA-13B-R) for translation. An example of translating "我爱机器翻译。" into English: ``` import torch from transformers import AutoModelForCausalLM from transformers import AutoTokenizer # Load base model and LoRA weights model = AutoModelForCausalLM.from_pretrained("haoranxu/ALMA-13B-R", torch_dtype=torch.float16, device_map="auto") tokenizer = AutoTokenizer.from_pretrained("haoranxu/ALMA-13B-R", padding_side='left') # Add the source sentence into the prompt template prompt="Translate this from Chinese to English:\nChinese: 我爱机器翻译。\nEnglish:" input_ids = tokenizer(prompt, return_tensors="pt", padding=True, max_length=40, truncation=True).input_ids.cuda() # Translation with torch.no_grad(): generated_ids = model.generate(input_ids=input_ids, num_beams=5, max_new_tokens=20, do_sample=True, temperature=0.6, top_p=0.9) outputs = tokenizer.batch_decode(generated_ids, skip_special_tokens=True) print(outputs) ``` Please find more details in our [GitHub repository](https://github.com/fe1ixxu/ALMA)