|
--- |
|
license: other |
|
language: |
|
- en |
|
inference: false |
|
--- |
|
|
|
# OPT-6.7B-Nerybus-Mix |
|
|
|
This is an experimental model containing a ***parameter-wise 50/50 blend (weighted average)*** of the weights of *NerysV2-6.7B* and *ErebusV1-6.7B* |
|
Preliminary testing produces pretty coherent outputs, however, it seems less impressive than the 2.7B variant of Nerybus, as both 6.7B source models appear more similar than their 2.7B counterparts. |
|
|
|
# License |
|
The two models used for this blend, *NerysV2-6.7B* and *ErebusV1-6.7B* are made by **Mr. Seeker**. |
|
- https://huggingface.co/KoboldAI/OPT-6.7B-Erebus |
|
- https://huggingface.co/KoboldAI/OPT-6B-nerys-v2 |
|
The base OPT-6.7B model is licensed under the OPT-175B license, Copyright (c) Meta Platforms, Inc. All Rights Reserved. |
|
|
|
# Evaluation Results |
|
No formal evaluation is available for this model at this time. This blend was created in FP16, due to available memory constraints. |
|
|
|
It is recommend to use this model with the KoboldAI software. All feedback and comments can be directed to Concedo on the KoboldAI discord. |
|
|
|
# [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard) |
|
Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_KoboldAI__OPT-6.7B-Nerybus-Mix) |
|
|
|
| Metric | Value | |
|
|-----------------------|---------------------------| |
|
| Avg. | 33.98 | |
|
| ARC (25-shot) | 39.16 | |
|
| HellaSwag (10-shot) | 68.63 | |
|
| MMLU (5-shot) | 24.47 | |
|
| TruthfulQA (0-shot) | 34.84 | |
|
| Winogrande (5-shot) | 65.11 | |
|
| GSM8K (5-shot) | 0.76 | |
|
| DROP (3-shot) | 4.85 | |
|
|