Update README.md
Browse files
README.md
CHANGED
|
@@ -154,6 +154,8 @@ If you *really* want to use `<|im_start|>` and `<|im_end|>`, just update your `t
|
|
| 154 |
|
| 155 |
*Note: I actually used my fork of [qlora](https://github.com/jondurbin/qlora)'s `train.py` for this, but I'm porting it to a minified version here, not tested yet!*
|
| 156 |
|
|
|
|
|
|
|
| 157 |
```bash
|
| 158 |
export BASE_DIR=/workspace
|
| 159 |
export WANDB_API_KEY=[redacted]
|
|
|
|
| 154 |
|
| 155 |
*Note: I actually used my fork of [qlora](https://github.com/jondurbin/qlora)'s `train.py` for this, but I'm porting it to a minified version here, not tested yet!*
|
| 156 |
|
| 157 |
+
*More notes: I stopped the fine-tune around 50% because of budget constraints - it's a lot of data...*
|
| 158 |
+
|
| 159 |
```bash
|
| 160 |
export BASE_DIR=/workspace
|
| 161 |
export WANDB_API_KEY=[redacted]
|