Update README.md

README.md

@@ -155,4 +155,20 @@ model_path = "5CD-AI/visobert-14gb-corpus"
 mask_filler = pipeline("fill-mask", model_path)
 
 mask_filler("ăn nói xà <mask>", top_k=10)
-```
+```
+
+## Fine-tune Configuration
+We fine-tune `5CD-AI/viso-twhin-bert-large` on 4 downstream tasks with `transformer` library with the following configuration:
+- seed: 42
+- gradient_accumulation_steps: 1
+- weight_decay: 0.01
+- optimizer: AdamW with betas=(0.9, 0.999) and epsilon=1e-08
+- training_epochs: 30
+- model_max_length: 128
+- learning_rate: 1e-5
+
+And different additional configurations for each task:
+
+| Emotion Recognition                                                               | Hate Speech Detection                                                             | Spam Reviews Detection                                                            | Hate Speech Spans Detection                                                       |
+| --------------------------------------------------------------------------------- | --------------------------------------------------------------------------------- | --------------------------------------------------------------------------------- | --------------------------------------------------------------------------------- |
+|\- train_batch_size: 64<br>\- lr_scheduler_type: linear | \- train_batch_size: 32<br>\- lr_scheduler_type: linear | \- train_batch_size: 32<br>\- lr_scheduler_type: cosine | \- train_batch_size: 32<br>\- lr_scheduler_type: cosine |