|
--- |
|
datasets: |
|
- Aeala/ShareGPT_Vicuna_unfiltered |
|
--- |
|
|
|
## LoRA Info: |
|
Please note that this is a highly experimental LoRA model. It may do some good stuff, it might do some undesirable stuff. Training is basically done now. Feel free to try it!~ |
|
|
|
**Important Note**: While this is trained on a cleaned ShareGPT dataset like Vicuna used, this was trained in the *Alpaca* format, so prompting should be something like: |
|
|
|
``` |
|
### Instruction: |
|
|
|
<prompt> (without the <>) |
|
|
|
### Response: |
|
``` |
|
|
|
Current upload: Fully trained adapter model (3 epochs). |
|
|
|
Secondary upload: checkpoint of epoch 2.97 (of 3) |
|
|
|
Thanks to MetaIX for initial seemingly successful testing of the first uploaded checkpoint (epoch 0.8) as well as epoch 1. |
|
## Benchmarks |
|
**wikitext2:** 4.372413635253906 |
|
|
|
**ptb-new:** 24.69171714782715 |
|
|
|
**c4-new:** 6.469308853149414 |
|
|
|
Results generated with GPTQ evals (not quantized) thanks to [Neko-Institute-of-Science](https://huggingface.co/Neko-Institute-of-Science) |
