ninagroot/GPT2-705Mtest

README.md

@@ -13,7 +13,7 @@ should probably proofread and complete it, then remove this comment. -->
 
 This model is a fine-tuned version of [](https://huggingface.co/) on an unknown dataset.
 It achieves the following results on the evaluation set:
-- Loss: 6.9571
+- Loss: 6.0919
 
 ## Model description
 
@@ -33,13 +33,14 @@ More information needed
 
 The following hyperparameters were used during training:
 - learning_rate: 0.00025
-- train_batch_size: 32
+- train_batch_size: 8
 - eval_batch_size: 8
 - seed: 42
 - gradient_accumulation_steps: 4
-- total_train_batch_size: 128
+- total_train_batch_size: 32
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: cosine
+- lr_scheduler_warmup_steps: 50
 - num_epochs: 20
 - mixed_precision_training: Native AMP
 
@@ -47,18 +48,26 @@ The following hyperparameters were used during training:
 
 | Training Loss | Epoch | Step | Validation Loss |
 |:-------------:|:-----:|:----:|:---------------:|
-| No log        | 0.57  | 1    | 10.8430         |
-| No log        | 1.71  | 3    | 9.6245          |
-| No log        | 2.86  | 5    | 8.3578          |
-| No log        | 4.0   | 7    | 7.6791          |
-| No log        | 4.57  | 8    | 7.6125          |
-| No log        | 5.71  | 10   | 7.1608          |
-| No log        | 6.86  | 12   | 7.0488          |
-| No log        | 8.0   | 14   | 7.0351          |
-| No log        | 8.57  | 15   | 7.0075          |
-| No log        | 9.71  | 17   | 6.9691          |
-| No log        | 10.86 | 19   | 6.9580          |
-| 7.4341        | 11.43 | 20   | 6.9571          |
+| No log        | 1.0   | 7    | 7.9651          |
+| No log        | 2.0   | 14   | 7.1619          |
+| 7.2683        | 3.0   | 21   | 6.5175          |
+| 7.2683        | 4.0   | 28   | 5.9957          |
+| 7.2683        | 5.0   | 35   | 5.5861          |
+| 4.9213        | 6.0   | 42   | 5.4206          |
+| 4.9213        | 7.0   | 49   | 5.3658          |
+| 4.9213        | 8.0   | 56   | 5.4741          |
+| 3.653         | 9.0   | 63   | 5.4698          |
+| 3.653         | 10.0  | 70   | 5.5100          |
+| 3.653         | 11.0  | 77   | 5.6280          |
+| 2.7835        | 12.0  | 84   | 5.7409          |
+| 2.7835        | 13.0  | 91   | 5.8478          |
+| 2.7835        | 14.0  | 98   | 5.9625          |
+| 2.1972        | 15.0  | 105  | 6.0096          |
+| 2.1972        | 16.0  | 112  | 6.0052          |
+| 2.1972        | 17.0  | 119  | 6.0647          |
+| 1.7144        | 18.0  | 126  | 6.0797          |
+| 1.7144        | 19.0  | 133  | 6.0888          |
+| 1.4404        | 20.0  | 140  | 6.0919          |
 
 
 ### Framework versions

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:70faebe0809ca7205d8d197f52a43a23f8eeaadee5a9ec0005fbe100adc8ee74
+oid sha256:f8f5286dca39854db17c3ddf3aaa69ed18a8f1e1e34651aa8f939d4ee13f5b2f
 size 2796386080

runs/Apr16_13-36-06_gcn62.local.snellius.surf.nl/events.out.tfevents.1713267373.gcn62.local.snellius.surf.nl.2830995.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9c580b5c9f61d812514187cdb2a61705218bdc27a0acaea90aaa9e0030a1a4e8
+size 11815

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:c9ba3507956ac8f0c49f71b7bb013008a63e70f634ebf0e9d9a6924f3f9df732
+oid sha256:31f627acde0acd2271fdd20aa65f4e7b4d840b45122d4ca5a25c12dcc9edb868
 size 4984