|
--- |
|
library_name: peft |
|
base_model: ai21labs/Jamba-v0.1 |
|
license: apache-2.0 |
|
datasets: |
|
- mhenrichsen/alpaca_2k_test |
|
tags: |
|
- axolotl |
|
--- |
|
|
|
# Jambalpaca-v0.1 |
|
|
|
This is a test run to fine-tune [ai21labs/Jamba-v0.1](https://huggingface.co/ai21labs/Jamba-v0.1) on an A100 80GB GPU using [Axolotl](https://github.com/OpenAccess-AI-Collective/axolotl) and a custom version of [LazyAxolotl](https://colab.research.google.com/drive/1TsDKNo2riwVmU55gjuBgB1AXVtRRfRHW). |
|
I used [mhenrichsen/alpaca_2k_test](https://huggingface.co/datasets/mhenrichsen/alpaca_2k_test) as a dataset. |
|
I had to quantize the base model in 8-bit precision due to how merging models with adapters work. Weirdly enough, I didn't have to do it in a notebook version I created. |
|
I also pushed the adapter so I or someone else could do a better merge. |
|
|
|
Let me know if you're interested, I can give you access to Jamba's version of LazyAxolotl. |
