Model save

README.md

@@ -0,0 +1,79 @@
+---
+tags:
+- generated_from_trainer
+metrics:
+- accuracy
+model-index:
+- name: smolm-autoreg-bpe-counterfactual-babylm-random_removal-seed_1024-3e-4
+  results: []
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# smolm-autoreg-bpe-counterfactual-babylm-random_removal-seed_1024-3e-4
+
+This model was trained from scratch on an unknown dataset.
+It achieves the following results on the evaluation set:
+- Loss: 3.3909
+- Accuracy: 0.4102
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 0.0003
+- train_batch_size: 32
+- eval_batch_size: 64
+- seed: 1024
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: linear
+- lr_scheduler_warmup_steps: 32000
+- num_epochs: 20.0
+- mixed_precision_training: Native AMP
+
+### Training results
+
+| Training Loss | Epoch | Step   | Validation Loss | Accuracy |
+|:-------------:|:-----:|:------:|:---------------:|:--------:|
+| 3.7399        | 1.0   | 18586  | 3.9066          | 0.3474   |
+| 3.4368        | 2.0   | 37172  | 3.6279          | 0.3750   |
+| 3.294         | 3.0   | 55758  | 3.4854          | 0.3884   |
+| 3.2094        | 4.0   | 74344  | 3.4178          | 0.3968   |
+| 3.1515        | 5.0   | 92930  | 3.3861          | 0.4009   |
+| 3.1023        | 6.0   | 111516 | 3.3600          | 0.4041   |
+| 3.0643        | 7.0   | 130102 | 3.3565          | 0.4047   |
+| 3.0294        | 8.0   | 148688 | 3.3575          | 0.4059   |
+| 3.0007        | 9.0   | 167274 | 3.3660          | 0.4068   |
+| 2.9771        | 10.0  | 185860 | 3.3513          | 0.4075   |
+| 2.9526        | 11.0  | 204446 | 3.3433          | 0.4092   |
+| 2.9307        | 12.0  | 223032 | 3.3542          | 0.4094   |
+| 2.91          | 13.0  | 241618 | 3.3446          | 0.4095   |
+| 2.888         | 14.0  | 260204 | 3.3463          | 0.4100   |
+| 2.862         | 15.0  | 278790 | 3.3530          | 0.4103   |
+| 2.8465        | 16.0  | 297376 | 3.3666          | 0.4098   |
+| 2.8291        | 17.0  | 315962 | 3.3780          | 0.4099   |
+| 2.8072        | 18.0  | 334548 | 3.3858          | 0.4099   |
+| 2.786         | 19.0  | 353134 | 3.3847          | 0.4104   |
+| 2.773         | 20.0  | 371720 | 3.3909          | 0.4102   |
+
+
+### Framework versions
+
+- Transformers 4.37.2
+- Pytorch 2.1.0+cu121
+- Datasets 2.16.1
+- Tokenizers 0.15.1

generation_config.json

@@ -0,0 +1,7 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 1,
+  "eos_token_id": 2,
+  "pad_token_id": 0,
+  "transformers_version": "4.37.2"
+}

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:a1fed8ed4c2fad2181bd05bfe1c5f64affba8bc22e3411e81c8e9bc8a0f6f2f4
+oid sha256:4913f0637dc691719616f0dc21fedbdc372e72e4e05d0f155da625aa17639ae7
 size 391376736

runs/Feb08_23-12-20_phyl-ling-p01.la.utexas.edu/events.out.tfevents.1707455901.phyl-ling-p01.la.utexas.edu.3367925.0

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:03c2c3078f6d29de52f65d33d0f013a47ee1e7f34f6276ec1a3c0d8e49a4b460
-size 70182
+oid sha256:f2ac6f8685f03386d17d397851b982757c7da5894c885374527ae3f8229730cf
+size 71031