Update README.md
Browse files
README.md
CHANGED
|
@@ -4,10 +4,12 @@ tags:
|
|
| 4 |
- trl
|
| 5 |
- sft
|
| 6 |
- generated_from_trainer
|
|
|
|
|
|
|
|
|
|
| 7 |
datasets:
|
| 8 |
-
- generator
|
| 9 |
- databricks/databricks-dolly-15k
|
| 10 |
-
base_model:
|
| 11 |
model-index:
|
| 12 |
- name: llama2-7-dolly-answer
|
| 13 |
results: []
|
|
@@ -26,7 +28,8 @@ Can be used in conjunction with [LukeOLuck/llama2-7-dolly-query](https://hugging
|
|
| 26 |
|
| 27 |
## Model description
|
| 28 |
|
| 29 |
-
A Fine-Tuned PEFT Adapter for the llama2 7b chat model
|
|
|
|
| 30 |
|
| 31 |
## Intended uses & limitations
|
| 32 |
|
|
|
|
| 4 |
- trl
|
| 5 |
- sft
|
| 6 |
- generated_from_trainer
|
| 7 |
+
- BitsAndBytes
|
| 8 |
+
- PEFT
|
| 9 |
+
- QLoRA
|
| 10 |
datasets:
|
|
|
|
| 11 |
- databricks/databricks-dolly-15k
|
| 12 |
+
base_model: Llama-2-7b-chat-hf
|
| 13 |
model-index:
|
| 14 |
- name: llama2-7-dolly-answer
|
| 15 |
results: []
|
|
|
|
| 28 |
|
| 29 |
## Model description
|
| 30 |
|
| 31 |
+
A Fine-Tuned PEFT Adapter for the llama2 7b chat hf model
|
| 32 |
+
Leverages [FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness](https://arxiv.org/abs/2205.14135), [QLoRA: Efficient Finetuning of Quantized LLMs](https://arxiv.org/abs/2305.14314), and [PEFT](https://huggingface.co/blog/peft)
|
| 33 |
|
| 34 |
## Intended uses & limitations
|
| 35 |
|