Hugging Face's logo

jchwenger
/
gpt2.shak

Text Generation
Transformers
TensorBoard
Safetensors
gpt2
generated_from_trainer
Inference Endpoints
text-generation-inference
Model card Files Metrics Community
jchwenger commited on
Commit
7a69728
•
1 Parent(s): 16606b2

gpt2.shak

Browse files
Files changed (21) hide show
  1. README.md +63 -0
  2. config.json +39 -0
  3. generation_config.json +6 -0
  4. model.safetensors +3 -0
  5. runs/Nov15_10-26-58_6ecb3bb64e18/events.out.tfevents.1700044086.6ecb3bb64e18.4193.0 +3 -0
  6. runs/Nov15_10-38-43_6ecb3bb64e18/events.out.tfevents.1700044728.6ecb3bb64e18.7577.0 +3 -0
  7. runs/Nov15_10-39-24_6ecb3bb64e18/events.out.tfevents.1700044767.6ecb3bb64e18.7577.1 +3 -0
  8. runs/Nov15_10-39-41_6ecb3bb64e18/events.out.tfevents.1700044784.6ecb3bb64e18.7577.2 +3 -0
  9. runs/Nov15_10-39-52_6ecb3bb64e18/events.out.tfevents.1700044793.6ecb3bb64e18.7577.3 +3 -0
  10. runs/Nov15_10-40-15_6ecb3bb64e18/events.out.tfevents.1700044816.6ecb3bb64e18.7577.4 +3 -0
  11. runs/Nov15_10-40-36_6ecb3bb64e18/events.out.tfevents.1700044837.6ecb3bb64e18.7577.5 +3 -0
  12. runs/Nov15_10-41-48_6ecb3bb64e18/events.out.tfevents.1700044908.6ecb3bb64e18.7577.6 +3 -0
  13. runs/Nov15_10-41-57_6ecb3bb64e18/events.out.tfevents.1700044918.6ecb3bb64e18.7577.7 +3 -0
  14. runs/Nov15_10-42-03_6ecb3bb64e18/events.out.tfevents.1700044924.6ecb3bb64e18.7577.8 +3 -0
  15. runs/Nov15_11-36-31_6ecb3bb64e18/events.out.tfevents.1700048193.6ecb3bb64e18.22425.0 +3 -0
  16. runs/Nov15_11-40-08_6ecb3bb64e18/events.out.tfevents.1700048409.6ecb3bb64e18.22425.1 +3 -0
  17. runs/Nov15_11-40-16_6ecb3bb64e18/events.out.tfevents.1700048417.6ecb3bb64e18.22425.2 +3 -0
  18. runs/Nov15_11-40-24_6ecb3bb64e18/events.out.tfevents.1700048424.6ecb3bb64e18.22425.3 +3 -0
  19. runs/Nov15_11-40-31_6ecb3bb64e18/events.out.tfevents.1700048432.6ecb3bb64e18.22425.4 +3 -0
  20. runs/Nov15_11-40-39_6ecb3bb64e18/events.out.tfevents.1700048440.6ecb3bb64e18.22425.5 +3 -0
  21. training_args.bin +3 -0
README.md ADDED
@@ -0,0 +1,63 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: mit
3
+ base_model: gpt2
4
+ tags:
5
+ - generated_from_trainer
6
+ datasets:
7
+ - tiny_shakespeare
8
+ model-index:
9
+ - name: gpt2.shak
10
+ results: []
11
+ ---
12
+
13
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
14
+ should probably proofread and complete it, then remove this comment. -->
15
+
16
+ # gpt2.shak
17
+
18
+ This model is a fine-tuned version of [gpt2](https://huggingface.co/gpt2) on the tiny_shakespeare dataset.
19
+ It achieves the following results on the evaluation set:
20
+ - Loss: 3.1852
21
+
22
+ ## Model description
23
+
24
+ More information needed
25
+
26
+ ## Intended uses & limitations
27
+
28
+ More information needed
29
+
30
+ ## Training and evaluation data
31
+
32
+ More information needed
33
+
34
+ ## Training procedure
35
+
36
+ ### Training hyperparameters
37
+
38
+ The following hyperparameters were used during training:
39
+ - learning_rate: 3e-05
40
+ - train_batch_size: 3
41
+ - eval_batch_size: 8
42
+ - seed: 42
43
+ - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
44
+ - lr_scheduler_type: linear
45
+ - num_epochs: 5
46
+
47
+ ### Training results
48
+
49
+ | Training Loss | Epoch | Step | Validation Loss |
50
+ |:-------------:|:-----:|:----:|:---------------:|
51
+ | 3.3259 | 1.0 | 98 | 3.2194 |
52
+ | 3.3527 | 2.0 | 196 | 3.2017 |
53
+ | 3.1783 | 3.0 | 294 | 3.1900 |
54
+ | 3.2653 | 4.0 | 392 | 3.1863 |
55
+ | 3.2787 | 5.0 | 490 | 3.1852 |
56
+
57
+
58
+ ### Framework versions
59
+
60
+ - Transformers 4.35.1
61
+ - Pytorch 2.1.0+cu118
62
+ - Datasets 2.14.7
63
+ - Tokenizers 0.14.1
config.json ADDED
@@ -0,0 +1,39 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "gpt2",
3
+ "activation_function": "gelu_new",
4
+ "architectures": [
5
+ "GPT2LMHeadModel"
6
+ ],
7
+ "attn_pdrop": 0.1,
8
+ "bos_token_id": 50256,
9
+ "embd_pdrop": 0.1,
10
+ "eos_token_id": 50256,
11
+ "initializer_range": 0.02,
12
+ "layer_norm_epsilon": 1e-05,
13
+ "model_type": "gpt2",
14
+ "n_ctx": 1024,
15
+ "n_embd": 768,
16
+ "n_head": 12,
17
+ "n_inner": null,
18
+ "n_layer": 12,
19
+ "n_positions": 1024,
20
+ "reorder_and_upcast_attn": false,
21
+ "resid_pdrop": 0.1,
22
+ "scale_attn_by_inverse_layer_idx": false,
23
+ "scale_attn_weights": true,
24
+ "summary_activation": null,
25
+ "summary_first_dropout": 0.1,
26
+ "summary_proj_to_labels": true,
27
+ "summary_type": "cls_index",
28
+ "summary_use_proj": true,
29
+ "task_specific_params": {
30
+ "text-generation": {
31
+ "do_sample": true,
32
+ "max_length": 50
33
+ }
34
+ },
35
+ "torch_dtype": "float32",
36
+ "transformers_version": "4.35.1",
37
+ "use_cache": true,
38
+ "vocab_size": 50257
39
+ }
generation_config.json ADDED
@@ -0,0 +1,6 @@
 
 
 
 
 
 
 
1
+ {
2
+ "_from_model_config": true,
3
+ "bos_token_id": 50256,
4
+ "eos_token_id": 50256,
5
+ "transformers_version": "4.35.1"
6
+ }
model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:554415c11ea55319e7f31a95a32c95c5b9940992e39ad4e05a39b24d2bc9bdfc
3
+ size 497774208
runs/Nov15_10-26-58_6ecb3bb64e18/events.out.tfevents.1700044086.6ecb3bb64e18.4193.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4da8f24e50c81e96e53d5056bc5c327917617338289662328002328191343a58
3
+ size 10912
runs/Nov15_10-38-43_6ecb3bb64e18/events.out.tfevents.1700044728.6ecb3bb64e18.7577.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d6c71a1bb5d7e0a9c9828337040930be2a29e84184a87b77cd19a21dd212f838
3
+ size 8756
runs/Nov15_10-39-24_6ecb3bb64e18/events.out.tfevents.1700044767.6ecb3bb64e18.7577.1 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ceca09c7c0508ef849719348f5117b0a2d74057f1796a8f1e0b8d7b1fe9ee977
3
+ size 5192
runs/Nov15_10-39-41_6ecb3bb64e18/events.out.tfevents.1700044784.6ecb3bb64e18.7577.2 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6ebc4882dc3246793f61dfb3110ec3dcaad0bff235495b2ef7aec933f90a9516
3
+ size 4576
runs/Nov15_10-39-52_6ecb3bb64e18/events.out.tfevents.1700044793.6ecb3bb64e18.7577.3 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c950c7f3add2032336619ae687daec2b0a35d2681b668d42b9cffa5520b5e142
3
+ size 4730
runs/Nov15_10-40-15_6ecb3bb64e18/events.out.tfevents.1700044816.6ecb3bb64e18.7577.4 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d72a0e57700b21685c85a9ce10691fcfb1f86dc24f8be3f95913845f32a0e02e
3
+ size 4884
runs/Nov15_10-40-36_6ecb3bb64e18/events.out.tfevents.1700044837.6ecb3bb64e18.7577.5 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1957fd334f2fde4947b6f4cb53f8308711a6dcdeb291226a5e0552058227f46a
3
+ size 4422
runs/Nov15_10-41-48_6ecb3bb64e18/events.out.tfevents.1700044908.6ecb3bb64e18.7577.6 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ade364a0395b7b7ee19007b38ad1faad1f7b788f9c05907f65e7c77b640b4ce1
3
+ size 4422
runs/Nov15_10-41-57_6ecb3bb64e18/events.out.tfevents.1700044918.6ecb3bb64e18.7577.7 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8f6145f6031d2f496ea35e5c1eeb9de5d8d9fad2d344f7361f97c38b6dea132d
3
+ size 4422
runs/Nov15_10-42-03_6ecb3bb64e18/events.out.tfevents.1700044924.6ecb3bb64e18.7577.8 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0c71f5af45a8f682a9a207264f7cd73d9e465216b360c4e6c95e5412d70e0026
3
+ size 21437
runs/Nov15_11-36-31_6ecb3bb64e18/events.out.tfevents.1700048193.6ecb3bb64e18.22425.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:09d2b716e376db7fd285ede7791d2e565891856d73db27ce201d4cd5405ba23b
3
+ size 10579
runs/Nov15_11-40-08_6ecb3bb64e18/events.out.tfevents.1700048409.6ecb3bb64e18.22425.1 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:bc4a085dbffe045ae6cd8d727634e30cc2b2045e313f5a36398674309b222249
3
+ size 4422
runs/Nov15_11-40-16_6ecb3bb64e18/events.out.tfevents.1700048417.6ecb3bb64e18.22425.2 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4288099159446eac3c897c4c9ca06e4b0cd565097cdfce99a9b0c443f73c1a0d
3
+ size 4576
runs/Nov15_11-40-24_6ecb3bb64e18/events.out.tfevents.1700048424.6ecb3bb64e18.22425.3 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4f38c82582623d57569b9289f81f42fe25f871ec176899562f91d1e40aeea554
3
+ size 4422
runs/Nov15_11-40-31_6ecb3bb64e18/events.out.tfevents.1700048432.6ecb3bb64e18.22425.4 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e96fff56815677f8cfee52575539893827ac6caa7c4f071ac9e4743096b07d08
3
+ size 4422
runs/Nov15_11-40-39_6ecb3bb64e18/events.out.tfevents.1700048440.6ecb3bb64e18.22425.5 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:aad41a97b7a8dd15d118e1eed88d1f63e03d90959e34da27d23f7f077fd9a2ca
3
+ size 21437
training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d1f61b89f9f38c0cffd11091182814f1f324e6ee808835e7c948a8617e67a623
3
+ size 4536