andreaskoepf
/

pythia-6.9b-gpt4all-pretrain

Text Generation

Inference Endpoints

text-generation-inference

Model card Files Files and versions Community

andreaskoepf commited on Apr 9, 2023

Commit

78cc424

•

1 Parent(s): 38da841

Update README.md

Files changed (1) hide show

README.md +3 -1

README.md CHANGED Viewed

@@ -56,4 +56,6 @@ pythia-6.9b-pretrain:
   per_device_eval_batch_size: 8
   num_train_epochs: 1
   save_total_limit: 2
-```

   per_device_eval_batch_size: 8
   num_train_epochs: 1
   save_total_limit: 2
+```
+command: `deepspeed trainer_sft.py --configs defaults pretrain pythia-6.9b-pretrain --cache_dir .cache/ --output_dir .saved_models/pythia-6.9b-pre --residual_dropout 0.0 --deepspeed`