Nanobit commited on
Commit
11c48c5
1 Parent(s): 295b266

fix(doc): Add note on inference w sample packing (#712)

Browse files
Files changed (1) hide show
  1. README.md +4 -0
README.md CHANGED
@@ -876,6 +876,10 @@ Pass the appropriate flag to the train command:
876
  --base_model="./completed-model" --prompter=None --load_in_8bit=True
877
  ```
878
 
 
 
 
 
879
  ### Merge LORA to base
880
 
881
  Add below flag to train command above
 
876
  --base_model="./completed-model" --prompter=None --load_in_8bit=True
877
  ```
878
 
879
+ Please use `--sample_packing False` if you have it on and receive the error similar to below:
880
+
881
+ > RuntimeError: stack expects each tensor to be equal size, but got [1, 32, 1, 128] at entry 0 and [1, 32, 8, 128] at entry 1
882
+
883
  ### Merge LORA to base
884
 
885
  Add below flag to train command above