OPT-19M-ChatSalad / README.md
leaderboard-pr-bot's picture
Adding Evaluation Results
17a3cf3
|
raw
history blame
1.49 kB
---
license: other
language:
- en
inference: false
widget:
- text: "How do I download this model?"
example_title: "Text Gen Example"
---
# OPT-19M-ChatSalad
This is an experimental OPT-based model with 19 million parameters trained entirely **from scratch** as a datasetting practice.
Thus, it should not be subject to the usual OPT license. You are free to use this model for any purpose.
The model is small enough (under 40mb) that it should run at very fast speeds even entirely on CPU.
It is recommend to use this model with the KoboldAI software, with the following parameters:
- Temperature: 0.9
- Repetition Penalty: 1.1
- Repetition Penalty Slope: 0.7
- Repetition Penalty Range: 1024
- Top-P Sampling: 0.9
- Disable all other samplers
All feedback and comments can be directed to Concedo on the KoboldAI discord.
# [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)
Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_concedo__OPT-19M-ChatSalad)
| Metric | Value |
|-----------------------|---------------------------|
| Avg. | 24.86 |
| ARC (25-shot) | 24.4 |
| HellaSwag (10-shot) | 25.15 |
| MMLU (5-shot) | 23.12 |
| TruthfulQA (0-shot) | 51.36 |
| Winogrande (5-shot) | 49.72 |
| GSM8K (5-shot) | 0.0 |
| DROP (3-shot) | 0.25 |