End of training

README.md

@@ -0,0 +1,131 @@
+---
+license: apache-2.0
+base_model: distilgpt2
+tags:
+- generated_from_trainer
+model-index:
+- name: StatementOfWork_Generator_Omega_BS_1024_2
+  results: []
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# StatementOfWork_Generator_Omega_BS_1024_2
+
+This model is a fine-tuned version of [distilgpt2](https://huggingface.co/distilgpt2) on an unknown dataset.
+It achieves the following results on the evaluation set:
+- Loss: 1.3889
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 2e-05
+- train_batch_size: 10
+- eval_batch_size: 100
+- seed: 42
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: linear
+- num_epochs: 75
+
+### Training results
+
+| Training Loss | Epoch | Step | Validation Loss |
+|:-------------:|:-----:|:----:|:---------------:|
+| No log        | 1.0   | 10   | 3.3893          |
+| No log        | 2.0   | 20   | 3.1683          |
+| No log        | 3.0   | 30   | 2.9900          |
+| No log        | 4.0   | 40   | 2.8271          |
+| No log        | 5.0   | 50   | 2.6741          |
+| No log        | 6.0   | 60   | 2.5419          |
+| No log        | 7.0   | 70   | 2.4238          |
+| No log        | 8.0   | 80   | 2.3227          |
+| No log        | 9.0   | 90   | 2.2323          |
+| No log        | 10.0  | 100  | 2.1533          |
+| No log        | 11.0  | 110  | 2.0906          |
+| No log        | 12.0  | 120  | 2.0305          |
+| No log        | 13.0  | 130  | 1.9847          |
+| No log        | 14.0  | 140  | 1.9410          |
+| No log        | 15.0  | 150  | 1.9026          |
+| No log        | 16.0  | 160  | 1.8693          |
+| No log        | 17.0  | 170  | 1.8388          |
+| No log        | 18.0  | 180  | 1.8103          |
+| No log        | 19.0  | 190  | 1.7877          |
+| No log        | 20.0  | 200  | 1.7618          |
+| No log        | 21.0  | 210  | 1.7385          |
+| No log        | 22.0  | 220  | 1.7191          |
+| No log        | 23.0  | 230  | 1.6999          |
+| No log        | 24.0  | 240  | 1.6793          |
+| No log        | 25.0  | 250  | 1.6650          |
+| No log        | 26.0  | 260  | 1.6472          |
+| No log        | 27.0  | 270  | 1.6334          |
+| No log        | 28.0  | 280  | 1.6172          |
+| No log        | 29.0  | 290  | 1.6063          |
+| No log        | 30.0  | 300  | 1.5923          |
+| No log        | 31.0  | 310  | 1.5826          |
+| No log        | 32.0  | 320  | 1.5702          |
+| No log        | 33.0  | 330  | 1.5632          |
+| No log        | 34.0  | 340  | 1.5513          |
+| No log        | 35.0  | 350  | 1.5421          |
+| No log        | 36.0  | 360  | 1.5328          |
+| No log        | 37.0  | 370  | 1.5244          |
+| No log        | 38.0  | 380  | 1.5136          |
+| No log        | 39.0  | 390  | 1.5087          |
+| No log        | 40.0  | 400  | 1.4996          |
+| No log        | 41.0  | 410  | 1.4912          |
+| No log        | 42.0  | 420  | 1.4852          |
+| No log        | 43.0  | 430  | 1.4789          |
+| No log        | 44.0  | 440  | 1.4724          |
+| No log        | 45.0  | 450  | 1.4659          |
+| No log        | 46.0  | 460  | 1.4632          |
+| No log        | 47.0  | 470  | 1.4556          |
+| No log        | 48.0  | 480  | 1.4505          |
+| No log        | 49.0  | 490  | 1.4467          |
+| 1.8264        | 50.0  | 500  | 1.4397          |
+| 1.8264        | 51.0  | 510  | 1.4383          |
+| 1.8264        | 52.0  | 520  | 1.4330          |
+| 1.8264        | 53.0  | 530  | 1.4282          |
+| 1.8264        | 54.0  | 540  | 1.4263          |
+| 1.8264        | 55.0  | 550  | 1.4213          |
+| 1.8264        | 56.0  | 560  | 1.4186          |
+| 1.8264        | 57.0  | 570  | 1.4152          |
+| 1.8264        | 58.0  | 580  | 1.4129          |
+| 1.8264        | 59.0  | 590  | 1.4100          |
+| 1.8264        | 60.0  | 600  | 1.4079          |
+| 1.8264        | 61.0  | 610  | 1.4035          |
+| 1.8264        | 62.0  | 620  | 1.4022          |
+| 1.8264        | 63.0  | 630  | 1.3998          |
+| 1.8264        | 64.0  | 640  | 1.3979          |
+| 1.8264        | 65.0  | 650  | 1.3960          |
+| 1.8264        | 66.0  | 660  | 1.3944          |
+| 1.8264        | 67.0  | 670  | 1.3929          |
+| 1.8264        | 68.0  | 680  | 1.3926          |
+| 1.8264        | 69.0  | 690  | 1.3923          |
+| 1.8264        | 70.0  | 700  | 1.3907          |
+| 1.8264        | 71.0  | 710  | 1.3904          |
+| 1.8264        | 72.0  | 720  | 1.3897          |
+| 1.8264        | 73.0  | 730  | 1.3894          |
+| 1.8264        | 74.0  | 740  | 1.3890          |
+| 1.8264        | 75.0  | 750  | 1.3889          |
+
+
+### Framework versions
+
+- Transformers 4.38.2
+- Pytorch 2.2.1+cu121
+- Datasets 2.18.0
+- Tokenizers 0.15.2

config.json

@@ -0,0 +1,46 @@
+{
+  "_name_or_path": "distilgpt2",
+  "_num_labels": 1,
+  "activation_function": "gelu_new",
+  "architectures": [
+    "GPT2LMHeadModel"
+  ],
+  "attn_pdrop": 0.1,
+  "bos_token_id": 50256,
+  "embd_pdrop": 0.1,
+  "eos_token_id": 50256,
+  "id2label": {
+    "0": "LABEL_0"
+  },
+  "initializer_range": 0.02,
+  "label2id": {
+    "LABEL_0": 0
+  },
+  "layer_norm_epsilon": 1e-05,
+  "model_type": "gpt2",
+  "n_ctx": 1024,
+  "n_embd": 768,
+  "n_head": 12,
+  "n_inner": null,
+  "n_layer": 6,
+  "n_positions": 1024,
+  "reorder_and_upcast_attn": false,
+  "resid_pdrop": 0.1,
+  "scale_attn_by_inverse_layer_idx": false,
+  "scale_attn_weights": true,
+  "summary_activation": null,
+  "summary_first_dropout": 0.1,
+  "summary_proj_to_labels": true,
+  "summary_type": "cls_index",
+  "summary_use_proj": true,
+  "task_specific_params": {
+    "text-generation": {
+      "do_sample": true,
+      "max_length": 50
+    }
+  },
+  "torch_dtype": "float32",
+  "transformers_version": "4.38.2",
+  "use_cache": true,
+  "vocab_size": 50257
+}

generation_config.json

@@ -0,0 +1,6 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 50256,
+  "eos_token_id": 50256,
+  "transformers_version": "4.38.2"
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:aa153239fa10a2417e5e073b5013718c2e92c524d75532c9384ce1244002bbcf
+size 327657928

runs/Jul01_20-15-06_viridian/events.out.tfevents.1719864914.viridian.3786935.3

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e028bb67d941700066728546b9b86677e945fe46591002d326eea54119f8ee1e
+size 14974

runs/Jul01_20-19-12_viridian/events.out.tfevents.1719865157.viridian.3786935.4

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b52c0379b5b3cc8bc6bfbf48bc2869ea3e88d174975707bf1a2d36c3a3c2b53b
+size 5048

runs/Jul01_20-24-47_viridian/events.out.tfevents.1719865528.viridian.3874171.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5d01f9ba4cd5f422b975682fad4faab275621b305a17a3b60eb398ce75405113
+size 25879

special_tokens_map.json

@@ -0,0 +1,5 @@
+{
+  "bos_token": "<|endoftext|>",
+  "eos_token": "<|endoftext|>",
+  "unk_token": "<|endoftext|>"
+}

tokenizer_config.json

@@ -0,0 +1,19 @@
+{
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "50256": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<|endoftext|>",
+  "clean_up_tokenization_spaces": true,
+  "eos_token": "<|endoftext|>",
+  "model_max_length": 1024,
+  "tokenizer_class": "GPT2Tokenizer",
+  "unk_token": "<|endoftext|>"
+}

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9c4c95b1c49caaecefc9c892b87652f0f4ae1bed1e6c68c6f1d2bdc55f795c7e
+size 5048