OmegLLaMA-3B / README.md
leaderboard-pr-bot's picture
Adding Evaluation Results
04493a3
|
raw
history blame
2.43 kB
metadata
license: apache-2.0
datasets:
  - anon8231489123/Omegle_logs_dataset
language:
  - en
library_name: transformers
pipeline_tag: text-generation
tags:
  - not-for-all-audiences

Buy Me A Coffee

This is Xander Boyce's OmegLLaMA LoRA merged with OpenLLama 3B.

Prompt format:

Interests: {interests}
Conversation:
You: {prompt}
Stranger: 

For multiple interests, seperate them with space. Repeat You and Stranger for multi-turn conversations, which means Interests and Conversation are technically part of the system prompt.

q4_0 GGML and GGUF quants here.

This model is very good at NSFW ERP and sexting(For a 3B model). I recommend using this with Faraday.dev if you want ERP or sexting.

Open LLM Leaderboard Evaluation Results

Detailed results can be found here

Metric Value
Avg. 33.55
ARC (25-shot) 40.36
HellaSwag (10-shot) 66.13
MMLU (5-shot) 28.0
TruthfulQA (0-shot) 33.31
Winogrande (5-shot) 61.64
GSM8K (5-shot) 0.23
DROP (3-shot) 5.17

Open LLM Leaderboard Evaluation Results

Detailed results can be found here

Metric Value
Avg. 33.55
ARC (25-shot) 40.36
HellaSwag (10-shot) 66.13
MMLU (5-shot) 28.0
TruthfulQA (0-shot) 33.31
Winogrande (5-shot) 61.64
GSM8K (5-shot) 0.23
DROP (3-shot) 5.17