Quantization made by Richard Erkhov.
ALMA-13B-R - GGUF
- Model creator: https://huggingface.co/haoranxu/
- Original model: https://huggingface.co/haoranxu/ALMA-13B-R/
Name | Quant method | Size |
---|---|---|
ALMA-13B-R.Q2_K.gguf | Q2_K | 4.52GB |
ALMA-13B-R.IQ3_XS.gguf | IQ3_XS | 4.99GB |
ALMA-13B-R.IQ3_S.gguf | IQ3_S | 5.27GB |
ALMA-13B-R.Q3_K_S.gguf | Q3_K_S | 5.27GB |
ALMA-13B-R.IQ3_M.gguf | IQ3_M | 5.57GB |
ALMA-13B-R.Q3_K.gguf | Q3_K | 5.9GB |
ALMA-13B-R.Q3_K_M.gguf | Q3_K_M | 5.9GB |
ALMA-13B-R.Q3_K_L.gguf | Q3_K_L | 6.45GB |
ALMA-13B-R.IQ4_XS.gguf | IQ4_XS | 6.54GB |
ALMA-13B-R.Q4_0.gguf | Q4_0 | 6.86GB |
ALMA-13B-R.IQ4_NL.gguf | IQ4_NL | 6.9GB |
ALMA-13B-R.Q4_K_S.gguf | Q4_K_S | 6.91GB |
ALMA-13B-R.Q4_K.gguf | Q4_K | 7.33GB |
ALMA-13B-R.Q4_K_M.gguf | Q4_K_M | 7.33GB |
ALMA-13B-R.Q4_1.gguf | Q4_1 | 7.61GB |
ALMA-13B-R.Q5_0.gguf | Q5_0 | 8.36GB |
ALMA-13B-R.Q5_K_S.gguf | Q5_K_S | 8.36GB |
ALMA-13B-R.Q5_K.gguf | Q5_K | 8.6GB |
ALMA-13B-R.Q5_K_M.gguf | Q5_K_M | 8.6GB |
ALMA-13B-R.Q5_1.gguf | Q5_1 | 9.1GB |
ALMA-13B-R.Q6_K.gguf | Q6_K | 9.95GB |
ALMA-13B-R.Q8_0.gguf | Q8_0 | 12.88GB |
Original model description:
license: mit
ALMA-R builds upon ALMA models, with further LoRA fine-tuning with our proposed Contrastive Preference Optimization (CPO) as opposed to the Supervised Fine-tuning used in ALMA. CPO fine-tuning requires our triplet preference data for preference learning. ALMA-R now can matches or even exceeds GPT-4 or WMT winners!
@misc{xu2024contrastive,
title={Contrastive Preference Optimization: Pushing the Boundaries of LLM Performance in Machine Translation},
author={Haoran Xu and Amr Sharaf and Yunmo Chen and Weiting Tan and Lingfeng Shen and Benjamin Van Durme and Kenton Murray and Young Jin Kim},
year={2024},
eprint={2401.08417},
archivePrefix={arXiv},
primaryClass={cs.CL}
}
@misc{xu2023paradigm,
title={A Paradigm Shift in Machine Translation: Boosting Translation Performance of Large Language Models},
author={Haoran Xu and Young Jin Kim and Amr Sharaf and Hany Hassan Awadalla},
year={2023},
eprint={2309.11674},
archivePrefix={arXiv},
primaryClass={cs.CL}
}
Download ALMA(-R) Models and Dataset π
We release six translation models presented in the paper:
- ALMA-7B
- ALMA-7B-LoRA
- ALMA-7B-R (NEW!): Further LoRA fine-tuning upon ALMA-7B-LoRA with contrastive preference optimization.
- ALMA-13B
- ALMA-13B-LoRA
- ALMA-13B-R (NEW!): Further LoRA fine-tuning upon ALMA-13B-LoRA with contrastive preference optimization (BEST MODEL!).
Model checkpoints are released at huggingface:
Models | Base Model Link | LoRA Link |
---|---|---|
ALMA-7B | haoranxu/ALMA-7B | - |
ALMA-7B-LoRA | haoranxu/ALMA-7B-Pretrain | haoranxu/ALMA-7B-Pretrain-LoRA |
ALMA-7B-R (NEW!) | haoranxu/ALMA-7B-R (LoRA merged) | - |
ALMA-13B | haoranxu/ALMA-13B | - |
ALMA-13B-LoRA | haoranxu/ALMA-13B-Pretrain | haoranxu/ALMA-13B-Pretrain-LoRA |
ALMA-13B-R (NEW!) | haoranxu/ALMA-13B-R (LoRA merged) | - |
Note that ALMA-7B-Pretrain
and ALMA-13B-Pretrain
are NOT translation models. They only experience stage 1 monolingual fine-tuning (20B tokens for the 7B model and 12B tokens for the 13B model), and should be utilized in conjunction with their LoRA models.
Datasets used by ALMA and ALMA-R are also released at huggingface now (NEW!)
Datasets | Train / Validation | Test |
---|---|---|
Human-Written Parallel Data (ALMA) | train and validation | WMT'22 |
Triplet Preference Data | train | WMT'22 and WMT'23 |
A quick start to use our best system (ALMA-13B-R) for translation. An example of translating "ζη±ζΊε¨ηΏ»θ―γ" into English:
import torch
from transformers import AutoModelForCausalLM
from transformers import AutoTokenizer
# Load base model and LoRA weights
model = AutoModelForCausalLM.from_pretrained("haoranxu/ALMA-13B-R", torch_dtype=torch.float16, device_map="auto")
tokenizer = AutoTokenizer.from_pretrained("haoranxu/ALMA-13B-R", padding_side='left')
# Add the source sentence into the prompt template
prompt="Translate this from Chinese to English:\nChinese: ζη±ζΊε¨ηΏ»θ―γ\nEnglish:"
input_ids = tokenizer(prompt, return_tensors="pt", padding=True, max_length=40, truncation=True).input_ids.cuda()
# Translation
with torch.no_grad():
generated_ids = model.generate(input_ids=input_ids, num_beams=5, max_new_tokens=20, do_sample=True, temperature=0.6, top_p=0.9)
outputs = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)
print(outputs)
Please find more details in our GitHub repository
- Downloads last month
- 203