metadata
language:
- en
datasets:
- FinchResearch/AboveTheClouds
tags:
- llama
- llama2
Starlight (13B)
| Model | Average ⬆️ | ARC | HellaSwag | MMLU | TruthfulQA |
|---|---|---|---|---|---|
| NewstaR/Starlight-13B | 58.63 | 59.3 | 82.15 | 55.67 | 37.39 |
| NewstaR/Starlight-7B | 54.3 | 53.07 | 78.57 | 46.8 | 38.75 |
The model follows the Alpaca template:
### Instruction: {prompt} ### Response:
Example:
### Instruction: Summarize the key details of the Starlight model in a few sentences.
### Response: Starlight is a 13B parameter transformer model trained on the AverageData and Above the Clouds datasets for conversational text generation. It has strong language modeling capabilities but lacks true language understanding and may generate incorrect or biased text, so outputs should be monitored and safeguards implemented. The model is intended for use in chatbots and content creation applications.
Instructions for Safe Use
- Avoid exposing Starlight to offensive, unethical, dangerous or illegal prompts
- Monitor outputs for signs of bias, toxicity or factual incorrectness
- Do not rely on Starlight for high-stakes or safety critical applications
Limitations
- May hallucinate or generate incorrect information
- Large model size leads to high compute requirements
Open LLM Leaderboard Evaluation Results
Detailed results can be found here
| Metric | Value |
|---|---|
| Avg. | 46.87 |
| ARC (25-shot) | 59.3 |
| HellaSwag (10-shot) | 82.15 |
| MMLU (5-shot) | 55.67 |
| TruthfulQA (0-shot) | 37.39 |
| Winogrande (5-shot) | 76.64 |
| GSM8K (5-shot) | 10.84 |
| DROP (3-shot) | 6.08 |