End of training

README.md

@@ -0,0 +1,88 @@
+---
+license: mit
+library_name: peft
+tags:
+- generated_from_trainer
+base_model: openai-community/gpt2
+model-index:
+- name: GPT2_Pirate_2024_05_10_20_25_24_lora_weightTrue_loraR32_optim_adamw_torch_epoch2_lr3e-05
+  results: []
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# GPT2_Pirate_2024_05_10_20_25_24_lora_weightTrue_loraR32_optim_adamw_torch_epoch2_lr3e-05
+
+This model is a fine-tuned version of [openai-community/gpt2](https://huggingface.co/openai-community/gpt2) on the None dataset.
+It achieves the following results on the evaluation set:
+- Loss: 1.7723
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 3e-05
+- train_batch_size: 8
+- eval_batch_size: 8
+- seed: 42
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: linear
+- num_epochs: 2
+- mixed_precision_training: Native AMP
+
+### Training results
+
+| Training Loss | Epoch  | Step  | Validation Loss |
+|:-------------:|:------:|:-----:|:---------------:|
+| 2.3874        | 0.0673 | 1000  | 1.9785          |
+| 1.7367        | 0.1346 | 2000  | 1.8709          |
+| 1.6208        | 0.2020 | 3000  | 1.8520          |
+| 1.5521        | 0.2693 | 4000  | 1.8361          |
+| 1.5165        | 0.3366 | 5000  | 1.8298          |
+| 1.484         | 0.4039 | 6000  | 1.8227          |
+| 1.4262        | 0.4712 | 7000  | 1.8078          |
+| 1.4701        | 0.5385 | 8000  | 1.8000          |
+| 1.4188        | 0.6059 | 9000  | 1.7938          |
+| 1.403         | 0.6732 | 10000 | 1.7908          |
+| 1.4115        | 0.7405 | 11000 | 1.7940          |
+| 1.4153        | 0.8078 | 12000 | 1.7888          |
+| 1.3903        | 0.8751 | 13000 | 1.7844          |
+| 1.3918        | 0.9424 | 14000 | 1.7830          |
+| 1.4018        | 1.0098 | 15000 | 1.7843          |
+| 1.3579        | 1.0771 | 16000 | 1.7777          |
+| 1.3803        | 1.1444 | 17000 | 1.7776          |
+| 1.3545        | 1.2117 | 18000 | 1.7778          |
+| 1.3557        | 1.2790 | 19000 | 1.7742          |
+| 1.3739        | 1.3463 | 20000 | 1.7769          |
+| 1.3538        | 1.4137 | 21000 | 1.7778          |
+| 1.3761        | 1.4810 | 22000 | 1.7763          |
+| 1.347         | 1.5483 | 23000 | 1.7735          |
+| 1.3579        | 1.6156 | 24000 | 1.7729          |
+| 1.3581        | 1.6829 | 25000 | 1.7734          |
+| 1.3472        | 1.7503 | 26000 | 1.7726          |
+| 1.3377        | 1.8176 | 27000 | 1.7752          |
+| 1.3243        | 1.8849 | 28000 | 1.7732          |
+| 1.369         | 1.9522 | 29000 | 1.7723          |
+
+
+### Framework versions
+
+- PEFT 0.10.0
+- Transformers 4.40.2
+- Pytorch 2.3.0+cu121
+- Datasets 2.19.1
+- Tokenizers 0.19.1

adapter_config.json

@@ -0,0 +1,28 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "openai-community/gpt2",
+  "bias": "none",
+  "fan_in_fan_out": true,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 24,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 32,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "c_attn"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fb8fd7feeacb5ebfbf78c26ebc649d7c2c93b446389524f69b2fb7e449431824
+size 4721680

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b8e02097f802a16ab2eab68273b94221efad5efa678c67acbaee38726e862d94
+size 5112