codeparrot
/

codeparrot

Text Generation

Inference Endpoints

text-generation-inference

Model card Files Files and versions Metrics Training metrics Community

lvwerra HF staff commited on Nov 11, 2021

Commit

c4c98c9

•

1 Parent(s): 9616188

Enable gradient_checkpointing

Files changed (1) hide show

codeparrot_training.py +1 -1

codeparrot_training.py CHANGED Viewed

@@ -133,7 +133,7 @@ config = {"train_batch_size": 1,
           "lr_scheduler_type": "cosine",
           "num_warmup_steps": 750,
           "gradient_accumulation_steps": 16,
-          "gradient_checkpointing": False,
           "max_train_steps": 50_000,
           "max_eval_steps": -1,
           "seq_length": 1024,

           "lr_scheduler_type": "cosine",
           "num_warmup_steps": 750,
           "gradient_accumulation_steps": 16,
+          "gradient_checkpointing": True,
           "max_train_steps": 50_000,
           "max_eval_steps": -1,
           "seq_length": 1024,