Update README.md
Browse files
README.md
CHANGED
|
@@ -48,7 +48,7 @@ Find more about the dataset here: [Code-74k-ShareGPT Dataset](https://huggingfac
|
|
| 48 |
|
| 49 |
- Adapter: qlora
|
| 50 |
- Learning Rate: 0.0002 with cosine lr scheduler
|
| 51 |
-
- Optimizer:
|
| 52 |
- Training Enhancements: bf16 training, gradient checkpointing, and flash attention
|
| 53 |
|
| 54 |
|
|
|
|
| 48 |
|
| 49 |
- Adapter: qlora
|
| 50 |
- Learning Rate: 0.0002 with cosine lr scheduler
|
| 51 |
+
- Optimizer: adamw_bnb_8bit
|
| 52 |
- Training Enhancements: bf16 training, gradient checkpointing, and flash attention
|
| 53 |
|
| 54 |
|