|
--- |
|
language: |
|
- en |
|
license: apache-2.0 |
|
tags: |
|
- mistral |
|
- not-for-all-audiences |
|
- merge |
|
pipeline_tag: text-generation |
|
inference: false |
|
model-index: |
|
- name: DarkSapling-7B-v2.0 |
|
results: |
|
- task: |
|
type: text-generation |
|
name: Text Generation |
|
dataset: |
|
name: AI2 Reasoning Challenge (25-Shot) |
|
type: ai2_arc |
|
config: ARC-Challenge |
|
split: test |
|
args: |
|
num_few_shot: 25 |
|
metrics: |
|
- type: acc_norm |
|
value: 64.16 |
|
name: normalized accuracy |
|
source: |
|
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=TeeZee/DarkSapling-7B-v2.0 |
|
name: Open LLM Leaderboard |
|
- task: |
|
type: text-generation |
|
name: Text Generation |
|
dataset: |
|
name: HellaSwag (10-Shot) |
|
type: hellaswag |
|
split: validation |
|
args: |
|
num_few_shot: 10 |
|
metrics: |
|
- type: acc_norm |
|
value: 85.1 |
|
name: normalized accuracy |
|
source: |
|
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=TeeZee/DarkSapling-7B-v2.0 |
|
name: Open LLM Leaderboard |
|
- task: |
|
type: text-generation |
|
name: Text Generation |
|
dataset: |
|
name: MMLU (5-Shot) |
|
type: cais/mmlu |
|
config: all |
|
split: test |
|
args: |
|
num_few_shot: 5 |
|
metrics: |
|
- type: acc |
|
value: 64.37 |
|
name: accuracy |
|
source: |
|
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=TeeZee/DarkSapling-7B-v2.0 |
|
name: Open LLM Leaderboard |
|
- task: |
|
type: text-generation |
|
name: Text Generation |
|
dataset: |
|
name: TruthfulQA (0-shot) |
|
type: truthful_qa |
|
config: multiple_choice |
|
split: validation |
|
args: |
|
num_few_shot: 0 |
|
metrics: |
|
- type: mc2 |
|
value: 52.21 |
|
source: |
|
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=TeeZee/DarkSapling-7B-v2.0 |
|
name: Open LLM Leaderboard |
|
- task: |
|
type: text-generation |
|
name: Text Generation |
|
dataset: |
|
name: Winogrande (5-shot) |
|
type: winogrande |
|
config: winogrande_xl |
|
split: validation |
|
args: |
|
num_few_shot: 5 |
|
metrics: |
|
- type: acc |
|
value: 78.61 |
|
name: accuracy |
|
source: |
|
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=TeeZee/DarkSapling-7B-v2.0 |
|
name: Open LLM Leaderboard |
|
- task: |
|
type: text-generation |
|
name: Text Generation |
|
dataset: |
|
name: GSM8k (5-shot) |
|
type: gsm8k |
|
config: main |
|
split: test |
|
args: |
|
num_few_shot: 5 |
|
metrics: |
|
- type: acc |
|
value: 45.41 |
|
name: accuracy |
|
source: |
|
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=TeeZee/DarkSapling-7B-v2.0 |
|
name: Open LLM Leaderboard |
|
--- |
|
# DarkSapling-7B-v2.0 |
|
|
|
![image/png](https://huggingface.co/TeeZee/DarkSapling-7B-v2.0/resolve/main/DarkSapling-7B-v2.0.jpg) |
|
|
|
|
|
## Model Details |
|
|
|
- A result of 4 models merge. DARE TIES method was used this time so resulting model better preserves characteristics of all included models than v1.x. |
|
- models used for merge: |
|
[cognitivecomputations/dolphin-2.6-mistral-7b-dpo-laser](https://huggingface.co/cognitivecomputations/dolphin-2.6-mistral-7b-dpo-laser) |
|
[KoboldAI/Mistral-7B-Holodeck-1](https://huggingface.co/KoboldAI/Mistral-7B-Holodeck-1) |
|
[KoboldAI/Mistral-7B-Erebus-v3](https://huggingface.co/KoboldAI/Mistral-7B-Erebus-v3) |
|
[cognitivecomputations/samantha-mistral-7b](https://huggingface.co/cognitivecomputations/samantha-mistral-7b) |
|
- See [mergekit-config.yml](https://huggingface.co/TeeZee/DarkSapling-7B-v2.0/resolve/main/mergekit-config.yml) for details on the merge method used. |
|
|
|
**Warning: This model can produce NSFW content!** |
|
|
|
## Results |
|
|
|
- a little different than version v1.0, more romantic and empathetic. |
|
- smarter than versions 1.0 and 1.1. |
|
- best for one-on-one ERP. |
|
- produces SFW nad NSFW content without issues, switches context seamlessly. |
|
- sticks to character card |
|
- pretty smart due to mistral, empathetic after Samantha and sometimes produces dark scenarions - Erebus. |
|
- storytelling is satisfactory due to Holodeck |
|
- good at following instructions |
|
|
|
|
|
All comments are greatly appreciated, download, test and if you appreciate my work, consider buying me my fuel: |
|
<a href="https://www.buymeacoffee.com/TeeZee" target="_blank"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me A Coffee" style="height: 60px !important;width: 217px !important;" ></a> |
|
|
|
# [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard) |
|
Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_TeeZee__DarkSapling-7B-v2.0) |
|
|
|
| Metric |Value| |
|
|---------------------------------|----:| |
|
|Avg. |64.98| |
|
|AI2 Reasoning Challenge (25-Shot)|64.16| |
|
|HellaSwag (10-Shot) |85.10| |
|
|MMLU (5-Shot) |64.37| |
|
|TruthfulQA (0-shot) |52.21| |
|
|Winogrande (5-shot) |78.61| |
|
|GSM8k (5-shot) |45.41| |
|
|
|
|