|
--- |
|
license: cc-by-nc-4.0 |
|
--- |
|
**Calibration dataset**: VMware Open Instruct, 4096 |
|
<p align="center"><font size="7"> <b>Dumb assistant, 🔥 author.</b></font></p> |
|
<p align="center"><font size="5"> <b>4-bit Examples with Alpaca</b></font></p> |
|
<p align="center"><img src="https://i.postimg.cc/VNXpwDCY/Screenshot-2023-12-06-212944.png"/> |
|
<p align="center"><img src="https://i.ibb.co/TPDsD2z/Screenshot-2023-12-06-211216.png"/> |
|
<p align="center"><font size="5"><b><a href="https://i.ibb.co/4PhnXjK/Screenshot-2023-12-06-205831.png">!!NSFW!! - 🔥Erotica Writing Example🔥 - !!NSFW!!</font></a></b></p> |
|
|
|
Thanks to [Charles Goddard](https://huggingface.co/chargoddard) for the recipe. |
|
|
|
The idea here is to "move" Iambe from being based on vanilla L2 to being based on sequelbox/DynamicFactor instead. |
|
|
|
Because task_arithmetic uses the raw deltas, this should be similar to if the SFT had been done over DynamicFactor. |
|
|
|
## Recipe |
|
|
|
merge_method: task_arithmetic |
|
|
|
base_model: athirdpath/BigLlama-20b-v1.1 # Base model you want to "move out" from |
|
|
|
models: |
|
|
|
- model: athirdpath/Iambe-20b-DARE-v2 # SFTd model you want to transfer |
|
|
|
- model: athirdpath/DoubleFactor-20b # Base model you want to "move in" to |
|
|
|
parameters: |
|
|
|
- weight: 1.0 |
|
|
|
dtype: bfloat16 |
