SHRDFU-7b Δ
- Developed by: maldv
- License: cc-by-nc-4.0
- Finetuned from model: ammarali32/multi_verse_model
- Methodology: Peft to train; extending intelligence and problem solving w/ crabcanon
As I work on understanding how to layer information in to the model, this model used no conditioning and even with low LR's, had quite a sharp graph. It definitely inherited the style of the source.
I had been experimenting in this series with wrapping each paragraph or turn with <s></s> bos/eos tokens. This may be semi-compatible with instruct, but is incompatible with alpaca and chatml. Good to know.
- Downloads last month
- 1,740