Mirelle commited on
Commit
bfb82c3
1 Parent(s): aa0bb09

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +9 -2
README.md CHANGED
@@ -15,9 +15,16 @@ pipeline_tag: text-generation
15
  ---
16
  # OPT-125M finetuned Portuguese
17
 
18
- Fine-tuning the [OPT-125M](https://huggingface.co/facebook/opt-125m) model on a reduced corpus of MC4-Portuguese with approximately 300M tokens.
19
 
20
- In this training a sequence length of 512 tokens was used, batch of 32 for 2 epochs.
 
 
 
 
 
 
 
21
 
22
  With an A100 with 40GB of RAM, the training took around 3 hours
23
 
 
15
  ---
16
  # OPT-125M finetuned Portuguese
17
 
18
+ Fine-tuning the [OPT-125M](https://huggingface.co/facebook/opt-125m) model on a reduced corpus of mc4-Portuguese with approximately 300M tokens.
19
 
20
+ ###### Hyper-parameters
21
+
22
+
23
+ - learning_rate = 5e-5
24
+ - batch_size = 32
25
+ - warmup = 500
26
+ - seq_length = 512
27
+ - num_train_epochs = 2.0
28
 
29
  With an A100 with 40GB of RAM, the training took around 3 hours
30