fix(doc): Add note on inference w sample packing (#712)
Browse files
README.md
CHANGED
@@ -876,6 +876,10 @@ Pass the appropriate flag to the train command:
|
|
876 |
--base_model="./completed-model" --prompter=None --load_in_8bit=True
|
877 |
```
|
878 |
|
|
|
|
|
|
|
|
|
879 |
### Merge LORA to base
|
880 |
|
881 |
Add below flag to train command above
|
|
|
876 |
--base_model="./completed-model" --prompter=None --load_in_8bit=True
|
877 |
```
|
878 |
|
879 |
+
Please use `--sample_packing False` if you have it on and receive the error similar to below:
|
880 |
+
|
881 |
+
> RuntimeError: stack expects each tensor to be equal size, but got [1, 32, 1, 128] at entry 0 and [1, 32, 8, 128] at entry 1
|
882 |
+
|
883 |
### Merge LORA to base
|
884 |
|
885 |
Add below flag to train command above
|