| license: apache-2.0 | |
| metrics: | |
| - accuracy | |
| pipeline_tag: text-generation | |
| ## Summary | |
| "Deer-3b," an instruction-following large language model based on "Bloom-3b," is fine-tuned using ±5k instructions. | |
| Deer will also be available in larger models size. | |
| ## Usage | |
| To use the model with the `transformers` library on a machine with GPUs. | |
| ```python | |
| import torch | |
| from transformers import pipeline | |
| generate_text = pipeline(model="PSanni/Deer-3b", torch_dtype=torch.bfloat16, trust_remote_code=True, device_map="auto") | |
| ``` | |
| You can then use the pipeline to answer instructions: | |
| ```python | |
| res = generate_text("Explain to me the difference between nuclear fission and fusion.") | |
| print(res[0]["generated_text"]) | |
| ``` | |
| ### Note: | |
| Kindly note that the model isn't attuned to human preferences and could generate unsuitable, unethical, biased, and toxic responses. | |
| # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard) | |
| Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_PSanni__Deer-3b) | |
| | Metric | Value | | |
| |-----------------------|---------------------------| | |
| | Avg. | 32.01 | | |
| | ARC (25-shot) | 38.48 | | |
| | HellaSwag (10-shot) | 57.41 | | |
| | MMLU (5-shot) | 25.64 | | |
| | TruthfulQA (0-shot) | 39.98 | | |
| | Winogrande (5-shot) | 57.46 | | |
| | GSM8K (5-shot) | 0.3 | | |
| | DROP (3-shot) | 4.83 | | |