Model save

README.md

@@ -15,6 +15,8 @@ should probably proofread and complete it, then remove this comment. -->
 # sinhala_gpt2
 
 This model is a fine-tuned version of [gpt2](https://huggingface.co/gpt2) on the None dataset.
+It achieves the following results on the evaluation set:
+- Loss: 3.4181
 
 ## Model description
 
@@ -33,19 +35,34 @@ More information needed
 ### Training hyperparameters
 
 The following hyperparameters were used during training:
-- learning_rate: 5e-05
-- train_batch_size: 64
-- eval_batch_size: 64
+- learning_rate: 5e-06
+- train_batch_size: 16
+- eval_batch_size: 16
 - seed: 42
 - gradient_accumulation_steps: 4
-- total_train_batch_size: 256
+- total_train_batch_size: 64
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: cosine
-- lr_scheduler_warmup_steps: 500
-- num_epochs: 5
+- lr_scheduler_warmup_steps: 10
+- num_epochs: 1
 
 ### Training results
 
+| Training Loss | Epoch  | Step | Validation Loss |
+|:-------------:|:------:|:----:|:---------------:|
+| 12.5768       | 0.0737 | 20   | 11.7031         |
+| 10.6016       | 0.1475 | 40   | 10.1428         |
+| 9.5592        | 0.2212 | 60   | 8.4000          |
+| 7.7086        | 0.2949 | 80   | 6.1398          |
+| 6.1288        | 0.3687 | 100  | 5.1259          |
+| 5.2551        | 0.4424 | 120  | 4.4283          |
+| 4.7127        | 0.5161 | 140  | 4.0241          |
+| 4.3572        | 0.5899 | 160  | 3.7673          |
+| 4.1243        | 0.6636 | 180  | 3.6012          |
+| 3.9714        | 0.7373 | 200  | 3.5126          |
+| 3.8867        | 0.8111 | 220  | 3.4489          |
+| 3.8334        | 0.8848 | 240  | 3.4256          |
+| 3.8204        | 0.9585 | 260  | 3.4181          |
 
 
 ### Framework versions