| license: llama2 | |
| language: | |
| - en | |
| ***ONLY UPLOADED FROM RUNPOD JUST TO TEST ON OWN SYSTEM. UNTESTED SO FAR. V2 SOON*** | |
| ***CURRENT CHANGES:*** | |
| <br>***INCREASED MODEL WEIGHTS AND DENSITIES IN TIES-MERGE FOR P1 & P2*** | |
| <br>***GRADIENT MERGE BETWEEN P1 & P2 CAN'T BE ILLUSTRATED, TENSORS EACH HAD UNIQUE RATIOS AND GRADIENTS APPLIED*** | |
| An experimental merging of Several Models using two various methods, [Ties-Merge](https://github.com/cg123/ties-merge) and [BlockMerge_Gradient](https://github.com/Gryphe/BlockMerge_Gradient) | |
| Stheno: | |
| <br>Gradient Merge of Stheno-P1 & Stheno-P2. | |
| Test Checklist: | |
| <br>Censorship - ____ | |
| <br>Writing - ____ | |
| <br>NSFW - ___ | |
| <br>IQ Level - ___ | |
| <br>Formatting - ____ | |
| Most formats could work, use Alpaca format and it works well. | |
| ``` | |
| ### Instruction: | |
| Your instruction or question here. | |
| For roleplay purposes, I suggest the following - Write <CHAR NAME>'s next reply in a chat between <YOUR NAME> and <CHAR NAME>. Write a single reply only. | |
| ### Response: | |
| ``` | |
| Gradient Merge Pictures Unavailable, Several Different Tensor Ratios applied. | |
| # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard) | |
| Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_Sao10K__Stheno-1.2-L2-13B) | |
| | Metric | Value | | |
| |-----------------------|---------------------------| | |
| | Avg. | 54.8 | | |
| | ARC (25-shot) | 60.75 | | |
| | HellaSwag (10-shot) | 83.67 | | |
| | MMLU (5-shot) | 56.27 | | |
| | TruthfulQA (0-shot) | 50.32 | | |
| | Winogrande (5-shot) | 74.98 | | |
| | GSM8K (5-shot) | 10.92 | | |
| | DROP (3-shot) | 46.72 | | |