Merged Models
Collection
Using mergekit
•
10 items
•
Updated
•
2
It's experimental, but seems fine for me, I didn't run it deeply yet but should be good for Role-play 😈 considering the two merged models, feel free to leave a suggestion or feedback.
This is a merge of pre-trained language models created using mergekit(my experimental branch swapping here )
Merge Details
Merge Method
This model was merged using the task_swapping merge method using NeuralNovel/Senzu-7B-v0.1-DPO as a base.
Models Merged
The following models were included in the merge:
Configuration
The following YAML configuration was used to produce this model:
merge_method: task_swapping
base_model: NeuralNovel/Senzu-7B-v0.1-DPO
models:
- model: senseable/WestLake-7B-v2
parameters:
weight: 0.75
diagonal_offset: 2 #it doesn't do anything when you use random_mask
random_mask: 0.3333
random_mask_seed: 98557
dtype: bfloat16
Open LLM Leaderboard Evaluation Results
Detailed results can be found here
| Metric | Value |
|---|---|
| Avg. | 67.28 |
| AI2 Reasoning Challenge (25-Shot) | 68.34 |
| HellaSwag (10-Shot) | 85.70 |
| MMLU (5-Shot) | 64.14 |
| TruthfulQA (0-shot) | 50.43 |
| Winogrande (5-shot) | 82.48 |
| GSM8k (5-shot) | 52.62 |
- Downloads last month
- 398
This model does not have enough activity to be deployed to Inference API (serverless) yet.
Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated)
instead.
Merge of
Collection including Aryanne/WestSenzu-Swap-7B
Evaluation results
- normalized accuracy on AI2 Reasoning Challenge (25-Shot)test set Open LLM Leaderboard68.340
- normalized accuracy on HellaSwag (10-Shot)validation set Open LLM Leaderboard85.700
- accuracy on MMLU (5-Shot)test set Open LLM Leaderboard64.140
- mc2 on TruthfulQA (0-shot)validation set Open LLM Leaderboard50.430
- accuracy on Winogrande (5-shot)validation set Open LLM Leaderboard82.480
- accuracy on GSM8k (5-shot)test set Open LLM Leaderboard52.620