Model save

README.md

@@ -0,0 +1,79 @@
+---
+tags:
+- generated_from_trainer
+metrics:
+- accuracy
+model-index:
+- name: smolm-autoreg-bpe-counterfactual-babylm-only_other_det_removal-seed_211-1e-3
+  results: []
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# smolm-autoreg-bpe-counterfactual-babylm-only_other_det_removal-seed_211-1e-3
+
+This model was trained from scratch on an unknown dataset.
+It achieves the following results on the evaluation set:
+- Loss: 3.4143
+- Accuracy: 0.4110
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 0.001
+- train_batch_size: 32
+- eval_batch_size: 64
+- seed: 211
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: linear
+- lr_scheduler_warmup_steps: 32000
+- num_epochs: 20.0
+- mixed_precision_training: Native AMP
+
+### Training results
+
+| Training Loss | Epoch | Step   | Validation Loss | Accuracy |
+|:-------------:|:-----:|:------:|:---------------:|:--------:|
+| 3.6004        | 1.0   | 18597  | 3.8219          | 0.3575   |
+| 3.3852        | 2.0   | 37194  | 3.6092          | 0.3797   |
+| 3.2597        | 3.0   | 55791  | 3.4837          | 0.3910   |
+| 3.1758        | 4.0   | 74388  | 3.4364          | 0.3981   |
+| 3.1197        | 5.0   | 92985  | 3.4116          | 0.4017   |
+| 3.08          | 6.0   | 111582 | 3.3782          | 0.4040   |
+| 3.0418        | 7.0   | 130179 | 3.3885          | 0.4055   |
+| 3.0088        | 8.0   | 148776 | 3.3884          | 0.4062   |
+| 2.9856        | 9.0   | 167373 | 3.3548          | 0.4077   |
+| 2.9598        | 10.0  | 185970 | 3.3782          | 0.4090   |
+| 2.9364        | 11.0  | 204567 | 3.3851          | 0.4093   |
+| 2.9156        | 12.0  | 223164 | 3.3803          | 0.4097   |
+| 2.8949        | 13.0  | 241761 | 3.3869          | 0.4100   |
+| 2.8719        | 14.0  | 260358 | 3.3813          | 0.4104   |
+| 2.8526        | 15.0  | 278955 | 3.3859          | 0.4108   |
+| 2.8289        | 16.0  | 297552 | 3.3980          | 0.4103   |
+| 2.8104        | 17.0  | 316149 | 3.3981          | 0.4109   |
+| 2.7958        | 18.0  | 334746 | 3.4054          | 0.4110   |
+| 2.781         | 19.0  | 353343 | 3.4057          | 0.4110   |
+| 2.7571        | 20.0  | 371940 | 3.4143          | 0.4110   |
+
+
+### Framework versions
+
+- Transformers 4.37.2
+- Pytorch 2.1.0+cu121
+- Datasets 2.16.1
+- Tokenizers 0.15.1

generation_config.json

@@ -0,0 +1,7 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 1,
+  "eos_token_id": 2,
+  "pad_token_id": 0,
+  "transformers_version": "4.37.2"
+}

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:4a5d24e3b169f7f118f90871bdf75c045e2a2510d6da14abbe6559ede7753c10
+oid sha256:f4b90fdf80d01da546c277dd9e8995e6c9434ec754359d41571a10d52c79eb5c
 size 391376736

runs/Feb13_15-50-30_phyl-ling-p01.la.utexas.edu/events.out.tfevents.1707861415.phyl-ling-p01.la.utexas.edu.3643917.0

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:66925b4b2d8b5a31083ad4ca99d3302b742b3246089faeda30bdcf77d208ae1f
-size 70215
+oid sha256:b94aa72378ec53bb7a50b47909458af2b08524b1ceb6475e8d478238bb627bc8
+size 71064