mrsteyk commited on
Commit
9c6d908
1 Parent(s): 7339d99

0iq moment - no batching was used

Browse files

I forgot to change 2 lines of code AAAAAAAA

Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -46,7 +46,7 @@ Heavy bias on IT.
46
 
47
  Input and output were straight up concatenated due to the nature of how ChatGPT works.
48
 
49
- This time dataset was batched into groups of 2048 tokens. Meaning i got 628/31 groups for training/eval. Maybe that's what made the difference. EOS was also being used after the final separator.
50
 
51
  ### Training hyperparameters
52
 
 
46
 
47
  Input and output were straight up concatenated due to the nature of how ChatGPT works.
48
 
49
+ EOS was being used after the final separator.
50
 
51
  ### Training hyperparameters
52