Hugging Face's logo

LogischeIP
/
SentimentT2_GPT2

Text Classification
Transformers
TensorBoard
Safetensors
gpt2
Generated from Trainer
text-generation-inference
Inference Endpoints
Model card Files Metrics Community
LogischeIP commited on
Commit
fd30b00
•
1 Parent(s): d2ad407

End of training

Browse files
Files changed (11) hide show
  1. README.md +35 -18
  2. added_tokens.json +3 -0
  3. config.json +1 -1
  4. model.safetensors +2 -2
  5. runs/Jan24_20-51-53_7d2b1e2d4e1a/events.out.tfevents.1706129514.7d2b1e2d4e1a.26662.11 +3 -0
  6. runs/Jan24_20-52-17_7d2b1e2d4e1a/events.out.tfevents.1706129539.7d2b1e2d4e1a.26662.12 +3 -0
  7. runs/Jan24_20-55-38_7d2b1e2d4e1a/events.out.tfevents.1706129739.7d2b1e2d4e1a.48811.0 +3 -0
  8. special_tokens_map.json +0 -1
  9. tokenizer.json +9 -0
  10. tokenizer_config.json +8 -1
  11. training_args.bin +1 -1
README.md CHANGED
@@ -18,11 +18,11 @@ should probably proofread and complete it, then remove this comment. -->
18
 
19
  This model is a fine-tuned version of [gpt2](https://huggingface.co/gpt2) on the None dataset.
20
  It achieves the following results on the evaluation set:
21
- - Loss: 0.9470
22
  - Accuracy: 0.8644
23
- - F1: 0.8728
24
- - Auc Roc: 0.9185
25
- - Log Loss: 0.9470
26
 
27
  ## Model description
28
 
@@ -41,28 +41,45 @@ More information needed
41
  ### Training hyperparameters
42
 
43
  The following hyperparameters were used during training:
44
- - learning_rate: 1e-05
45
  - train_batch_size: 1
46
  - eval_batch_size: 1
47
  - seed: 42
48
- - gradient_accumulation_steps: 2
49
- - total_train_batch_size: 2
50
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
51
- - lr_scheduler_type: cosine
52
- - lr_scheduler_warmup_steps: 500
53
- - num_epochs: 6
54
  - mixed_precision_training: Native AMP
55
 
56
  ### Training results
57
 
58
- | Training Loss | Epoch | Step | Validation Loss | Accuracy | F1 | Auc Roc | Log Loss |
59
- |:-------------:|:-----:|:----:|:---------------:|:--------:|:------:|:-------:|:--------:|
60
- | 1.0534 | 1.0 | 1618 | 0.6479 | 0.8694 | 0.8696 | 0.9298 | 0.6479 |
61
- | 0.6971 | 2.0 | 3236 | 1.0859 | 0.8371 | 0.8581 | 0.9236 | 1.0859 |
62
- | 0.5832 | 3.0 | 4854 | 0.9261 | 0.8495 | 0.8672 | 0.9255 | 0.9261 |
63
- | 0.4402 | 4.0 | 6472 | 0.8507 | 0.8719 | 0.8804 | 0.9251 | 0.8507 |
64
- | 0.3475 | 5.0 | 8090 | 0.9284 | 0.8657 | 0.8735 | 0.9198 | 0.9283 |
65
- | 0.2985 | 6.0 | 9708 | 0.9470 | 0.8644 | 0.8728 | 0.9185 | 0.9470 |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
66
 
67
 
68
  ### Framework versions
 
18
 
19
  This model is a fine-tuned version of [gpt2](https://huggingface.co/gpt2) on the None dataset.
20
  It achieves the following results on the evaluation set:
21
+ - Loss: 1.0308
22
  - Accuracy: 0.8644
23
+ - F1: 0.8685
24
+ - Auc Roc: 0.9297
25
+ - Log Loss: 1.0307
26
 
27
  ## Model description
28
 
 
41
  ### Training hyperparameters
42
 
43
  The following hyperparameters were used during training:
44
+ - learning_rate: 2e-05
45
  - train_batch_size: 1
46
  - eval_batch_size: 1
47
  - seed: 42
 
 
48
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
49
+ - lr_scheduler_type: linear
50
+ - lr_scheduler_warmup_steps: 100
51
+ - num_epochs: 4
52
  - mixed_precision_training: Native AMP
53
 
54
  ### Training results
55
 
56
+ | Training Loss | Epoch | Step | Validation Loss | Accuracy | F1 | Auc Roc | Log Loss |
57
+ |:-------------:|:-----:|:-----:|:---------------:|:--------:|:------:|:-------:|:--------:|
58
+ | 1.1785 | 0.15 | 500 | 0.7334 | 0.8346 | 0.8400 | 0.9144 | 0.7334 |
59
+ | 1.1409 | 0.31 | 1000 | 0.8797 | 0.8520 | 0.8649 | 0.9269 | 0.8796 |
60
+ | 1.0906 | 0.46 | 1500 | 0.7869 | 0.8744 | 0.8805 | 0.9394 | 0.7869 |
61
+ | 1.0163 | 0.62 | 2000 | 0.8381 | 0.8706 | 0.8771 | 0.9366 | 0.8381 |
62
+ | 1.0602 | 0.77 | 2500 | 0.9904 | 0.8458 | 0.8616 | 0.9253 | 0.9904 |
63
+ | 1.1456 | 0.93 | 3000 | 0.8833 | 0.8483 | 0.8452 | 0.9275 | 0.8832 |
64
+ | 0.9662 | 1.08 | 3500 | 0.9737 | 0.8507 | 0.8618 | 0.9354 | 0.9737 |
65
+ | 0.8496 | 1.24 | 4000 | 0.9361 | 0.8619 | 0.8680 | 0.9351 | 0.9361 |
66
+ | 0.8571 | 1.39 | 4500 | 0.8660 | 0.8619 | 0.8702 | 0.9346 | 0.8660 |
67
+ | 0.7506 | 1.55 | 5000 | 0.9359 | 0.8507 | 0.8558 | 0.9316 | 0.9359 |
68
+ | 0.8236 | 1.7 | 5500 | 1.1721 | 0.8184 | 0.8433 | 0.9229 | 1.1721 |
69
+ | 0.6897 | 1.85 | 6000 | 0.9876 | 0.8532 | 0.8547 | 0.9318 | 0.9876 |
70
+ | 0.6699 | 2.01 | 6500 | 0.8947 | 0.8570 | 0.8671 | 0.9323 | 0.8946 |
71
+ | 0.6137 | 2.16 | 7000 | 0.9318 | 0.8557 | 0.8661 | 0.9344 | 0.9318 |
72
+ | 0.4646 | 2.32 | 7500 | 0.9943 | 0.8595 | 0.8660 | 0.9312 | 0.9944 |
73
+ | 0.7042 | 2.47 | 8000 | 0.9150 | 0.8657 | 0.8714 | 0.9345 | 0.9150 |
74
+ | 0.4079 | 2.63 | 8500 | 1.0215 | 0.8657 | 0.8750 | 0.9312 | 1.0214 |
75
+ | 0.4646 | 2.78 | 9000 | 0.9809 | 0.8619 | 0.8714 | 0.9310 | 0.9809 |
76
+ | 0.4707 | 2.94 | 9500 | 1.0151 | 0.8644 | 0.8719 | 0.9279 | 1.0150 |
77
+ | 0.5005 | 3.09 | 10000 | 1.0748 | 0.8607 | 0.8651 | 0.9289 | 1.0747 |
78
+ | 0.3817 | 3.24 | 10500 | 0.8819 | 0.8781 | 0.8858 | 0.9299 | 0.8818 |
79
+ | 0.279 | 3.4 | 11000 | 1.0542 | 0.8607 | 0.8627 | 0.9302 | 1.0541 |
80
+ | 0.3527 | 3.55 | 11500 | 1.0148 | 0.8607 | 0.8637 | 0.9312 | 1.0147 |
81
+ | 0.3873 | 3.71 | 12000 | 1.0421 | 0.8619 | 0.8648 | 0.9294 | 1.0420 |
82
+ | 0.3552 | 3.86 | 12500 | 1.0308 | 0.8644 | 0.8685 | 0.9297 | 1.0307 |
83
 
84
 
85
  ### Framework versions
added_tokens.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ {
2
+ "[PAD]": 50257
3
+ }
config.json CHANGED
@@ -36,5 +36,5 @@
36
  "torch_dtype": "float32",
37
  "transformers_version": "4.35.2",
38
  "use_cache": true,
39
- "vocab_size": 50257
40
  }
 
36
  "torch_dtype": "float32",
37
  "transformers_version": "4.35.2",
38
  "use_cache": true,
39
+ "vocab_size": 50258
40
  }
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:bba396cdd3954fa6449e56fd26d07b8e71a29415aa7f0202a2844805faf46396
3
- size 497780432
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e26ef8998b3af60784b5e86969675377cd913d59fe21c1c652c68304befc30a0
3
+ size 497783504
runs/Jan24_20-51-53_7d2b1e2d4e1a/events.out.tfevents.1706129514.7d2b1e2d4e1a.26662.11 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:077fdc8d9f2ab5d8e550b4247b0266f233fdff2a7c0d86334fce442e4141cab2
3
+ size 88
runs/Jan24_20-52-17_7d2b1e2d4e1a/events.out.tfevents.1706129539.7d2b1e2d4e1a.26662.12 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:41fafd4e599519920a300571bbfb2d2f6ac302642ca3cf24b0847cc283aa899e
3
+ size 88
runs/Jan24_20-55-38_7d2b1e2d4e1a/events.out.tfevents.1706129739.7d2b1e2d4e1a.48811.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e1dc9c8ac040e111f9e5d942a2b146e52b5ce26a5ad850ff06c2d37d6a782b61
3
+ size 20621
special_tokens_map.json CHANGED
@@ -1,6 +1,5 @@
1
  {
2
  "bos_token": "<|endoftext|>",
3
  "eos_token": "<|endoftext|>",
4
- "pad_token": "<|endoftext|>",
5
  "unk_token": "<|endoftext|>"
6
  }
 
1
  {
2
  "bos_token": "<|endoftext|>",
3
  "eos_token": "<|endoftext|>",
 
4
  "unk_token": "<|endoftext|>"
5
  }
tokenizer.json CHANGED
@@ -11,6 +11,15 @@
11
  "rstrip": false,
12
  "normalized": true,
13
  "special": true
 
 
 
 
 
 
 
 
 
14
  }
15
  ],
16
  "normalizer": null,
 
11
  "rstrip": false,
12
  "normalized": true,
13
  "special": true
14
+ },
15
+ {
16
+ "id": 50257,
17
+ "content": "[PAD]",
18
+ "single_word": false,
19
+ "lstrip": false,
20
+ "rstrip": false,
21
+ "normalized": true,
22
+ "special": false
23
  }
24
  ],
25
  "normalizer": null,
tokenizer_config.json CHANGED
@@ -8,13 +8,20 @@
8
  "rstrip": false,
9
  "single_word": false,
10
  "special": true
 
 
 
 
 
 
 
 
11
  }
12
  },
13
  "bos_token": "<|endoftext|>",
14
  "clean_up_tokenization_spaces": true,
15
  "eos_token": "<|endoftext|>",
16
  "model_max_length": 1024,
17
- "pad_token": "<|endoftext|>",
18
  "tokenizer_class": "GPT2Tokenizer",
19
  "unk_token": "<|endoftext|>"
20
  }
 
8
  "rstrip": false,
9
  "single_word": false,
10
  "special": true
11
+ },
12
+ "50257": {
13
+ "content": "[PAD]",
14
+ "lstrip": false,
15
+ "normalized": true,
16
+ "rstrip": false,
17
+ "single_word": false,
18
+ "special": false
19
  }
20
  },
21
  "bos_token": "<|endoftext|>",
22
  "clean_up_tokenization_spaces": true,
23
  "eos_token": "<|endoftext|>",
24
  "model_max_length": 1024,
 
25
  "tokenizer_class": "GPT2Tokenizer",
26
  "unk_token": "<|endoftext|>"
27
  }
training_args.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:4687dd4a4162f59df791f1cccd877165e6c3d736f89908c5329d32a5eaea2f11
3
  size 4664
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:97232c0749c09a31249963f38d7824f92beb3a268a74e8688220162579edc0a0
3
  size 4664