I帽igo L贸pez-Riob贸o Botana commited on
Commit
eeeaaf6
1 Parent(s): d1b62aa

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +14 -0
README.md CHANGED
@@ -66,6 +66,20 @@ for i in range(CHAT_TURNS):
66
 
67
  ---
68
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
69
  ## Fine-tuning in a different dataset or style
70
 
71
  If you want to fine-tune this model, we recommend you to start from the [DialoGPT model](https://huggingface.co/microsoft/DialoGPT-medium).
 
66
 
67
  ---
68
 
69
+ ## Fine-tuning hyper-parameters
70
+
71
+ | **Hyper-parameter** | **Value** |
72
+ |:----------------------------------------:|:---------------------------:|
73
+ | Validation partition (%) | 20% |
74
+ | Training batch size | 8 |
75
+ | Learning rate | 5e-4 |
76
+ | Max training epochs | 20 |
77
+ | Warmup training steps (%) | 6% |
78
+ | Weight decay | 0.01 |
79
+ | Optimiser (beta1, beta2, epsilon) | AdamW (0.9, 0.999, 1e-08) |
80
+ | Monitoring metric (delta, patience) | validation loss (0.1, 3) |
81
+
82
+
83
  ## Fine-tuning in a different dataset or style
84
 
85
  If you want to fine-tune this model, we recommend you to start from the [DialoGPT model](https://huggingface.co/microsoft/DialoGPT-medium).