File size: 2,564 Bytes
e2dd0f6 189b3d3 5c437cf e2dd0f6 189b3d3 e2dd0f6 189b3d3 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 |
---
base_model:
- InferenceIllusionist/Excalibur-7b
library_name: transformers
tags:
- finetune
- dpo
- chatml
- gguf
- iMat
license: apache-2.0
datasets:
- Intel/orca_dpo_pairs
---
# Excalibur-7b-DPO-iMat-GGUF
<img src="https://i.imgur.com/pbPbqq0.jpeg" width="550"/>
Quantized from fp32 with love.
iMatrix .dat file was calculated using groups_merged.txt.
<b>FP16 available [here](https://huggingface.co/InferenceIllusionist/Excalibur-7b-DPO)</b>
An initial foray into the world of fine-tuning. The goal of this release was to amplify the quality of the original model's responses, in particular for vision use cases*
## Notes & Methodology
* [Excalibur-7b](https://huggingface.co/InferenceIllusionist/Excalibur-7b) fine-tuned with Direct Preference Optimization (DPO) using Intel/orca_dpo_pairs
* This is a quick experiment to determine the impact of DPO finetuning on the original base model
* Ran for a little over an hour on a single A100
* Internal benchmarks showed improvement over base model, awaiting final results
* Precision: bfloat16
## Sample Question - Vision
<img src="https://i.imgur.com/7aRWtzU.jpeg" width="425"/>
*<b>Requires additional mmproj file. You have two options for vision functionality (available inside original repo or linked below):</b>
* [Quantized - Limited VRAM Option (197mb)](https://huggingface.co/InferenceIllusionist/Excalibur-7b-DPO-GGUF/resolve/main/mistral-7b-mmproj-v1.5-Q4_1.gguf?download=true)
* [Unquantized - Premium Option / Best Quality (596mb)](https://huggingface.co/InferenceIllusionist/Excalibur-7b-DPO-GGUF/resolve/main/mmproj-model-f16.gguf?download=true)
Select the gguf file of your choice in Kobold as usual, then make sure to choose the mmproj file above in the LLaVA mmproj field of the model submenu:
<img src="https://i.imgur.com/x8vqH29.png" width="425"/>
## Prompt Format
* For best results please use ChatML for the prompt format. Alpaca may also work.
# [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)
Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_InferenceIllusionist__Excalibur-7b-DPO)
| Metric |Value|
|---------------------------------|----:|
|Avg. |73.84|
|AI2 Reasoning Challenge (25-Shot)|70.90|
|HellaSwag (10-Shot) |87.93|
|MMLU (5-Shot) |65.46|
|TruthfulQA (0-shot) |70.82|
|Winogrande (5-shot) |82.48|
|GSM8k (5-shot) |65.43| |