Blackroot
/

Llama-3-8B-Abomination-LORA

Model card Files Files and versions Community

Blackroot commited on May 28

Commit

4b12d3d

•

1 Parent(s): 16ba0f7

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -39,7 +39,7 @@ trainer = SFTTrainer(
     args = TrainingArguments(
         per_device_train_batch_size = 2,
         warmup_steps = 45,
-        num_train_epochs=2,
         fp16 = not torch.cuda.is_bf16_supported(),
         bf16 = torch.cuda.is_bf16_supported(),
         logging_steps = 15,

     args = TrainingArguments(
         per_device_train_batch_size = 2,
         warmup_steps = 45,
+        num_train_epochs=2, #1 for base-pretraining
         fp16 = not torch.cuda.is_bf16_supported(),
         bf16 = torch.cuda.is_bf16_supported(),
         logging_steps = 15,