I帽igo L贸pez-Riob贸o Botana
commited on
Commit
路
eeeaaf6
1
Parent(s):
d1b62aa
Update README.md
Browse files
README.md
CHANGED
@@ -66,6 +66,20 @@ for i in range(CHAT_TURNS):
|
|
66 |
|
67 |
---
|
68 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
69 |
## Fine-tuning in a different dataset or style
|
70 |
|
71 |
If you want to fine-tune this model, we recommend you to start from the [DialoGPT model](https://huggingface.co/microsoft/DialoGPT-medium).
|
|
|
66 |
|
67 |
---
|
68 |
|
69 |
+
## Fine-tuning hyper-parameters
|
70 |
+
|
71 |
+
| **Hyper-parameter** | **Value** |
|
72 |
+
|:----------------------------------------:|:---------------------------:|
|
73 |
+
| Validation partition (%) | 20% |
|
74 |
+
| Training batch size | 8 |
|
75 |
+
| Learning rate | 5e-4 |
|
76 |
+
| Max training epochs | 20 |
|
77 |
+
| Warmup training steps (%) | 6% |
|
78 |
+
| Weight decay | 0.01 |
|
79 |
+
| Optimiser (beta1, beta2, epsilon) | AdamW (0.9, 0.999, 1e-08) |
|
80 |
+
| Monitoring metric (delta, patience) | validation loss (0.1, 3) |
|
81 |
+
|
82 |
+
|
83 |
## Fine-tuning in a different dataset or style
|
84 |
|
85 |
If you want to fine-tune this model, we recommend you to start from the [DialoGPT model](https://huggingface.co/microsoft/DialoGPT-medium).
|