Edit model card

OpenLlama-Platypus-3B

OpenLlama-Platypus-3B is an instruction fine-tuned model based on the OpenLLaMA-3B transformer architecture.

Model Details

  • Trained by: Robbe De Sutter
  • Model type: OpenLlama-Platypus-3B is an auto-regressive language model based on the OpenLLaMA-3B transformer architecture.
  • Language(s): English
  • License for base weights: Non-Commercial Creative Commons license (CC BY-NC-4.0)

Prompt Template

### Instruction:
<prompt> (without the <>)
### Response:

Training Dataset

RobbeD/OpenLlama-Platypus-3B trained using STEM and logic based dataset garage-bAInd/Open-Platypus.

Please see their paper and project webpage for additional information.

Training Procedure

RobbeD/OpenLlama-Platypus-3B was instruction fine-tuned using LoRA on 1 RX 6900 XT 16GB.

Citations

@article{platypus2023,
    title={Platypus: Quick, Cheap, and Powerful Refinement of LLMs}, 
    author={Ariel N. Lee and Cole J. Hunter and Nataniel Ruiz},
    booktitle={arXiv preprint arxiv:2308.07317},
    year={2023}
}
@software{openlm2023openllama,
  author = {Geng, Xinyang and Liu, Hao},
  title = {OpenLLaMA: An Open Reproduction of LLaMA},
  month = May,
  year = 2023,
  url = {https://github.com/openlm-research/open_llama}
}
@inproceedings{
    hu2022lora,
    title={Lo{RA}: Low-Rank Adaptation of Large Language Models},
    author={Edward J Hu and Yelong Shen and Phillip Wallis and Zeyuan Allen-Zhu and Yuanzhi Li and Shean Wang and Lu Wang and Weizhu Chen},
    booktitle={International Conference on Learning Representations},
    year={2022},
    url={https://openreview.net/forum?id=nZeVKeeFYf9}
}

Open LLM Leaderboard Evaluation Results

Detailed results can be found here

Metric Value
Avg. 36.76
ARC (25-shot) 41.21
HellaSwag (10-shot) 71.67
MMLU (5-shot) 29.86
TruthfulQA (0-shot) 36.45
Winogrande (5-shot) 65.98
GSM8K (5-shot) 1.14
DROP (3-shot) 11.01
Downloads last month
3,289
Safetensors
Model size
3.43B params
Tensor type
F32
Β·

Dataset used to train RobbeD/OpenLlama-Platypus-3B

Spaces using RobbeD/OpenLlama-Platypus-3B 11