midi-causal-transformer

README.md

@@ -1,8 +1,8 @@
 ---
 license: apache-2.0
+base_model: distilbert/distilgpt2
 tags:
 - generated_from_trainer
-base_model: distilbert/distilgpt2
 datasets:
 - eli5_category
 model-index:
@@ -17,7 +17,7 @@ should probably proofread and complete it, then remove this comment. -->
 
 This model is a fine-tuned version of [distilbert/distilgpt2](https://huggingface.co/distilbert/distilgpt2) on the eli5_category dataset.
 It achieves the following results on the evaluation set:
-- Loss: 0.0479
+- Loss: 0.0316
 
 ## Model description
 
@@ -42,15 +42,17 @@ The following hyperparameters were used during training:
 - seed: 42
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: linear
-- num_epochs: 3.0
+- num_epochs: 5
 
 ### Training results
 
 | Training Loss | Epoch | Step | Validation Loss |
 |:-------------:|:-----:|:----:|:---------------:|
-| No log        | 1.0   | 35   | 0.0613          |
-| No log        | 2.0   | 70   | 0.0508          |
-| No log        | 3.0   | 105  | 0.0479          |
+| No log        | 1.0   | 35   | 0.0596          |
+| No log        | 2.0   | 70   | 0.0465          |
+| No log        | 3.0   | 105  | 0.0383          |
+| No log        | 4.0   | 140  | 0.0333          |
+| No log        | 5.0   | 175  | 0.0316          |
 
 
 ### Framework versions

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:12427d55d035f29aa8b55b8d04cd620369df2192a7564144a2b90a4925c0ecd1
+oid sha256:ca03580b07298b44aaa1b878aed08052e43eb0bf3d0a36d496c34388832cdc56
 size 327657928

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:4b097880e0646a31dc5f1c83ba3a94261fa2d25afba4555bd4ac898c92ed42ef
+oid sha256:45f0a0234d0ef5529172314894c9099569c6f476638cd37986e02f81a2ec105f
 size 5112