YAML Metadata
Warning:
empty or missing yaml metadata in repo card
(https://huggingface.co/docs/hub/model-cards#model-card-metadata)
Quantization made by Richard Erkhov.
gemma-2b-zephyr-sft - GGUF
- Model creator: https://huggingface.co/Columbia-NLP/
- Original model: https://huggingface.co/Columbia-NLP/gemma-2b-zephyr-sft/
Name | Quant method | Size |
---|---|---|
gemma-2b-zephyr-sft.Q2_K.gguf | Q2_K | 1.08GB |
gemma-2b-zephyr-sft.IQ3_XS.gguf | IQ3_XS | 1.16GB |
gemma-2b-zephyr-sft.IQ3_S.gguf | IQ3_S | 1.2GB |
gemma-2b-zephyr-sft.Q3_K_S.gguf | Q3_K_S | 1.2GB |
gemma-2b-zephyr-sft.IQ3_M.gguf | IQ3_M | 1.22GB |
gemma-2b-zephyr-sft.Q3_K.gguf | Q3_K | 1.29GB |
gemma-2b-zephyr-sft.Q3_K_M.gguf | Q3_K_M | 1.29GB |
gemma-2b-zephyr-sft.Q3_K_L.gguf | Q3_K_L | 1.36GB |
gemma-2b-zephyr-sft.IQ4_XS.gguf | IQ4_XS | 1.4GB |
gemma-2b-zephyr-sft.Q4_0.gguf | Q4_0 | 1.44GB |
gemma-2b-zephyr-sft.IQ4_NL.gguf | IQ4_NL | 1.45GB |
gemma-2b-zephyr-sft.Q4_K_S.gguf | Q4_K_S | 1.45GB |
gemma-2b-zephyr-sft.Q4_K.gguf | Q4_K | 1.52GB |
gemma-2b-zephyr-sft.Q4_K_M.gguf | Q4_K_M | 1.52GB |
gemma-2b-zephyr-sft.Q4_1.gguf | Q4_1 | 1.56GB |
gemma-2b-zephyr-sft.Q5_0.gguf | Q5_0 | 1.68GB |
gemma-2b-zephyr-sft.Q5_K_S.gguf | Q5_K_S | 1.68GB |
gemma-2b-zephyr-sft.Q5_K.gguf | Q5_K | 1.71GB |
gemma-2b-zephyr-sft.Q5_K_M.gguf | Q5_K_M | 1.71GB |
gemma-2b-zephyr-sft.Q5_1.gguf | Q5_1 | 1.79GB |
gemma-2b-zephyr-sft.Q6_K.gguf | Q6_K | 1.92GB |
gemma-2b-zephyr-sft.Q8_0.gguf | Q8_0 | 2.49GB |
Original model description:
license: other license_name: gemma-terms-of-use license_link: https://ai.google.dev/gemma/terms base_model: google/gemma-2b tags: - alignment-handbook - trl - sft - generated_from_trainer datasets: - HuggingFaceH4/deita-10k-v0-sft model-index: - name: gemma-2b-zephyr-sft results: - task: type: text-generation name: Text Generation dataset: name: AI2 Reasoning Challenge (25-Shot) type: ai2_arc config: ARC-Challenge split: test args: num_few_shot: 25 metrics: - type: acc_norm value: 51.88 name: normalized accuracy - task: type: text-generation name: Text Generation dataset: name: HellaSwag (10-Shot) type: hellaswag split: validation args: num_few_shot: 10 metrics: - type: acc_norm value: 72.63 name: normalized accuracy - task: type: text-generation name: Text Generation dataset: name: MMLU (5-Shot) type: cais/mmlu config: all split: test args: num_few_shot: 5 metrics: - type: acc value: 42.20 name: accuracy - task: type: text-generation name: Text Generation dataset: name: TruthfulQA (0-shot) type: truthful_qa config: multiple_choice split: validation args: num_few_shot: 0 metrics: - type: mc2 value: 41.96 - task: type: text-generation name: Text Generation dataset: name: Winogrande (5-shot) type: winogrande config: winogrande_xl split: validation args: num_few_shot: 5 metrics: - type: acc value: 63.85 name: accuracy - task: type: text-generation name: Text Generation dataset: name: GSM8k (5-shot) type: gsm8k config: main split: test args: num_few_shot: 5 metrics: - type: acc value: 20.09 name: accuracy
Model Card for Gemma 2B Zephyr SFT
We trained the google/gemma-2b with deita-10k-v0-sft. We carefully selected the hyper-parameters and masked the user tokens during training to achieve the best supervised fine-tuning performance.
Model description
- Model type: A 2.5B parameter GPT-like model fine-tuned on a mix of publicly available, synthetic datasets.
- Language(s) (NLP): Primarily English
- License: Gemma Terms of Use
- Finetuned from model: google/gemma-2b
License
This model has the same license as the original Gemma model collection
OpenLLM Leaderboard Performance
Models | Avg. | ARC | HellaSwag | MMLU | TruthfulQA | Winogrande | GSM8k |
---|---|---|---|---|---|---|---|
google/gemma-2b | 46.37 | 48.38 | 71.77 | 41.77 | 33.08 | 66.77 | 16.91 |
google/gemma-2b-it | 42.75 | 43.94 | 62.70 | 37.65 | 45.82 | 60.93 | 5.46 |
wandb/gemma-2b-zephyr-sft | 47.18 | 49.74 | 72.38 | 41.37 | 34.42 | 66.93 | 18.27 |
wandb/gemma-2b-zephyr-dpo | 46.92 | 49.66 | 72.23 | 41.13 | 34.47 | 66.54 | 17.51 |
Columbia-NLP/gemma-2b-zephyr-sft | 48.75 | 51.80 | 72.63 | 42.20 | 41.96 | 63.85 | 20.09 |
Columbia-NLP/gemma-2b-zephyr-dpo | 49.14 | 52.22 | 73.11 | 42.55 | 42.64 | 64.40 | 19.94 |
MT-Bench
GPT-4-0125-preview as Judge
Model | Total | Coding | Extraction | Humanities | Math | Reasoning | Roleplay | STEM | Writing |
---|---|---|---|---|---|---|---|---|---|
google/gemma-2b-it | 4.71 | 2.95 | 4.35 | 6.15 | 2.90 | 3.50 | 5.60 | 5.50 | 6.70 |
wandb/gemma-2b-zephyr-sft | 4.03 | 3.10 | 3.15 | 5.00 | 2.70 | 2.65 | 5.10 | 4.80 | 5.75 |
wandb/gemma-2b-zephyr-dpo | 4.06 | 2.80 | 2.90 | 5.55 | 2.65 | 2.70 | 5.20 | 4.80 | 5.85 |
Columbia-NLP/gemma-2b-zephyr-sft | 4.34 | 3.10 | 3.70 | 6.25 | 2.65 | 2.70 | 5.55 | 5.25 | 5.50 |
Columbia-NLP/gemma-2b-zephyr-dpo | 4.75 | 3.50 | 4.05 | 6.75 | 3.30 | 3.70 | 5.85 | 5.40 | 5.53 |
- Downloads last month
- 505