End of training

README.md

@@ -14,7 +14,7 @@ should probably proofread and complete it, then remove this comment. -->
 
 This model is a fine-tuned version of [roneneldan/TinyStories-33M](https://huggingface.co/roneneldan/TinyStories-33M) on the None dataset.
 It achieves the following results on the evaluation set:
-- Loss: 4.2416
+- Loss: 4.6514
 
 ## Model description
 
@@ -34,25 +34,26 @@ More information needed
 
 The following hyperparameters were used during training:
 - learning_rate: 2e-05
-- train_batch_size: 8
+- train_batch_size: 64
 - eval_batch_size: 8
 - seed: 42
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: linear
-- num_epochs: 3.0
+- num_epochs: 3
+- mixed_precision_training: Native AMP
 
 ### Training results
 
-| Training Loss | Epoch | Step  | Validation Loss |
-|:-------------:|:-----:|:-----:|:---------------:|
-| 4.4151        | 1.0   | 6142  | 4.3861          |
-| 3.8876        | 2.0   | 12284 | 4.2146          |
-| 3.5052        | 3.0   | 18426 | 4.2416          |
+| Training Loss | Epoch | Step | Validation Loss |
+|:-------------:|:-----:|:----:|:---------------:|
+| 5.4075        | 1.0   | 768  | 4.8289          |
+| 4.614         | 2.0   | 1536 | 4.6794          |
+| 4.4446        | 3.0   | 2304 | 4.6514          |
 
 
 ### Framework versions
 
-- Transformers 4.37.2
+- Transformers 4.38.0.dev0
 - Pytorch 2.1.0+cu121
 - Datasets 2.17.0
 - Tokenizers 0.15.2

generation_config.json

@@ -2,5 +2,5 @@
   "_from_model_config": true,
   "bos_token_id": 50256,
   "eos_token_id": 50256,
-  "transformers_version": "4.37.2"
+  "transformers_version": "4.38.0.dev0"
 }

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:ca1094f3228f82c0cef9e22ef201857237a99249fe7962621c28fdc3793c846c
+oid sha256:02355383ae69fb038910cd7d7143ee4454932939ec0ea5ffae148892e09056e3
 size 274062040

runs/Feb19_03-13-19_96172a72b114/events.out.tfevents.1708312400.96172a72b114.1925.0

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:c76179962beb4d50576133aca283e38b3b464c207ae42ac7e43767ab38b9ea0a
-size 6095
+oid sha256:bced8f78a6f39544e9841c8c0d4e23eee09cb917abff4d222cce4f041477e8f9
+size 6720