Hugging Face's logo

davidt123
/
gpt2-elon

Text Generation
Transformers
PyTorch
gpt2
text-generation-inference
Inference Endpoints
Model card Files Community
davidt123 commited on
Commit
de10b94
•
1 Parent(s): f5831e2

Upload 8 files

Browse files
Files changed (8) hide show
  1. added_tokens.json +5 -0
  2. config.json +39 -0
  3. merges.txt +0 -0
  4. pytorch_model.bin +3 -0
  5. special_tokens_map.json +30 -0
  6. tokenizer_config.json +41 -0
  7. train_model.ipynb +2450 -0
  8. vocab.json +0 -0
added_tokens.json ADDED
@@ -0,0 +1,5 @@
 
 
 
 
 
 
1
+ {
2
+ "<|eos|>": 50258,
3
+ "<|pad|>": 50259,
4
+ "<|sos|>": 50257
5
+ }
config.json ADDED
@@ -0,0 +1,39 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "gpt2",
3
+ "activation_function": "gelu_new",
4
+ "architectures": [
5
+ "GPT2LMHeadModel"
6
+ ],
7
+ "attn_pdrop": 0.1,
8
+ "bos_token_id": 50256,
9
+ "embd_pdrop": 0.1,
10
+ "eos_token_id": 50256,
11
+ "initializer_range": 0.02,
12
+ "layer_norm_epsilon": 1e-05,
13
+ "model_type": "gpt2",
14
+ "n_ctx": 1024,
15
+ "n_embd": 768,
16
+ "n_head": 12,
17
+ "n_inner": null,
18
+ "n_layer": 12,
19
+ "n_positions": 1024,
20
+ "reorder_and_upcast_attn": false,
21
+ "resid_pdrop": 0.1,
22
+ "scale_attn_by_inverse_layer_idx": false,
23
+ "scale_attn_weights": true,
24
+ "summary_activation": null,
25
+ "summary_first_dropout": 0.1,
26
+ "summary_proj_to_labels": true,
27
+ "summary_type": "cls_index",
28
+ "summary_use_proj": true,
29
+ "task_specific_params": {
30
+ "text-generation": {
31
+ "do_sample": true,
32
+ "max_length": 50
33
+ }
34
+ },
35
+ "torch_dtype": "float32",
36
+ "transformers_version": "4.20.1",
37
+ "use_cache": true,
38
+ "vocab_size": 50260
39
+ }
merges.txt ADDED
The diff for this file is too large to render. See raw diff
 
pytorch_model.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:70e979bea5a6421d15180c640aaaa3b8d250a8491ee436f62fdabb0979e81a91
3
+ size 510405737
special_tokens_map.json ADDED
@@ -0,0 +1,30 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "bos_token": {
3
+ "content": "<|sos|>",
4
+ "lstrip": false,
5
+ "normalized": true,
6
+ "rstrip": false,
7
+ "single_word": false
8
+ },
9
+ "eos_token": {
10
+ "content": "<|eos|>",
11
+ "lstrip": false,
12
+ "normalized": true,
13
+ "rstrip": false,
14
+ "single_word": false
15
+ },
16
+ "pad_token": {
17
+ "content": "<|pad|>",
18
+ "lstrip": false,
19
+ "normalized": true,
20
+ "rstrip": false,
21
+ "single_word": false
22
+ },
23
+ "unk_token": {
24
+ "content": "<|endoftext|>",
25
+ "lstrip": false,
26
+ "normalized": true,
27
+ "rstrip": false,
28
+ "single_word": false
29
+ }
30
+ }
tokenizer_config.json ADDED
@@ -0,0 +1,41 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "add_bos_token": false,
3
+ "add_prefix_space": false,
4
+ "bos_token": {
5
+ "__type": "AddedToken",
6
+ "content": "<|sos|>",
7
+ "lstrip": false,
8
+ "normalized": true,
9
+ "rstrip": false,
10
+ "single_word": false
11
+ },
12
+ "eos_token": {
13
+ "__type": "AddedToken",
14
+ "content": "<|eos|>",
15
+ "lstrip": false,
16
+ "normalized": true,
17
+ "rstrip": false,
18
+ "single_word": false
19
+ },
20
+ "errors": "replace",
21
+ "model_max_length": 1024,
22
+ "name_or_path": "gpt2",
23
+ "pad_token": {
24
+ "__type": "AddedToken",
25
+ "content": "<|pad|>",
26
+ "lstrip": false,
27
+ "normalized": true,
28
+ "rstrip": false,
29
+ "single_word": false
30
+ },
31
+ "special_tokens_map_file": null,
32
+ "tokenizer_class": "GPT2Tokenizer",
33
+ "unk_token": {
34
+ "__type": "AddedToken",
35
+ "content": "<|endoftext|>",
36
+ "lstrip": false,
37
+ "normalized": true,
38
+ "rstrip": false,
39
+ "single_word": false
40
+ }
41
+ }
train_model.ipynb ADDED
@@ -0,0 +1,2450 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "nbformat": 4,
3
+ "nbformat_minor": 0,
4
+ "metadata": {
5
+ "colab": {
6
+ "provenance": [],
7
+ "machine_shape": "hm"
8
+ },
9
+ "kernelspec": {
10
+ "name": "python3",
11
+ "display_name": "Python 3"
12
+ },
13
+ "language_info": {
14
+ "name": "python"
15
+ },
16
+ "accelerator": "GPU",
17
+ "gpuClass": "standard",
18
+ "widgets": {
19
+ "application/vnd.jupyter.widget-state+json": {
20
+ "9ba818960e884a82a265d9455c3f9846": {
21
+ "model_module": "@jupyter-widgets/controls",
22
+ "model_name": "HBoxModel",
23
+ "model_module_version": "1.5.0",
24
+ "state": {
25
+ "_dom_classes": [],
26
+ "_model_module": "@jupyter-widgets/controls",
27
+ "_model_module_version": "1.5.0",
28
+ "_model_name": "HBoxModel",
29
+ "_view_count": null,
30
+ "_view_module": "@jupyter-widgets/controls",
31
+ "_view_module_version": "1.5.0",
32
+ "_view_name": "HBoxView",
33
+ "box_style": "",
34
+ "children": [
35
+ "IPY_MODEL_bd89b9c900024a799bfaf8f6dfa26bbb",
36
+ "IPY_MODEL_237748491b834ff3a774a84703bd5ca6",
37
+ "IPY_MODEL_866bf549da8d4cb0ac03c1b108f4f5c6"
38
+ ],
39
+ "layout": "IPY_MODEL_4578787847a8468b899e9e42840b250e"
40
+ }
41
+ },
42
+ "bd89b9c900024a799bfaf8f6dfa26bbb": {
43
+ "model_module": "@jupyter-widgets/controls",
44
+ "model_name": "HTMLModel",
45
+ "model_module_version": "1.5.0",
46
+ "state": {
47
+ "_dom_classes": [],
48
+ "_model_module": "@jupyter-widgets/controls",
49
+ "_model_module_version": "1.5.0",
50
+ "_model_name": "HTMLModel",
51
+ "_view_count": null,
52
+ "_view_module": "@jupyter-widgets/controls",
53
+ "_view_module_version": "1.5.0",
54
+ "_view_name": "HTMLView",
55
+ "description": "",
56
+ "description_tooltip": null,
57
+ "layout": "IPY_MODEL_4dc4e8f371664dbaa5ddef539d7cece3",
58
+ "placeholder": "​",
59
+ "style": "IPY_MODEL_a2752dd9fccd41f2a2f5d1c6aa59fec3",
60
+ "value": "Downloading: 100%"
61
+ }
62
+ },
63
+ "237748491b834ff3a774a84703bd5ca6": {
64
+ "model_module": "@jupyter-widgets/controls",
65
+ "model_name": "FloatProgressModel",
66
+ "model_module_version": "1.5.0",
67
+ "state": {
68
+ "_dom_classes": [],
69
+ "_model_module": "@jupyter-widgets/controls",
70
+ "_model_module_version": "1.5.0",
71
+ "_model_name": "FloatProgressModel",
72
+ "_view_count": null,
73
+ "_view_module": "@jupyter-widgets/controls",
74
+ "_view_module_version": "1.5.0",
75
+ "_view_name": "ProgressView",
76
+ "bar_style": "success",
77
+ "description": "",
78
+ "description_tooltip": null,
79
+ "layout": "IPY_MODEL_5cc6738e683e474c9a8f1912ee2b7f4d",
80
+ "max": 1042301,
81
+ "min": 0,
82
+ "orientation": "horizontal",
83
+ "style": "IPY_MODEL_fe4b9ccec34d440486f08263444f0606",
84
+ "value": 1042301
85
+ }
86
+ },
87
+ "866bf549da8d4cb0ac03c1b108f4f5c6": {
88
+ "model_module": "@jupyter-widgets/controls",
89
+ "model_name": "HTMLModel",
90
+ "model_module_version": "1.5.0",
91
+ "state": {
92
+ "_dom_classes": [],
93
+ "_model_module": "@jupyter-widgets/controls",
94
+ "_model_module_version": "1.5.0",
95
+ "_model_name": "HTMLModel",
96
+ "_view_count": null,
97
+ "_view_module": "@jupyter-widgets/controls",
98
+ "_view_module_version": "1.5.0",
99
+ "_view_name": "HTMLView",
100
+ "description": "",
101
+ "description_tooltip": null,
102
+ "layout": "IPY_MODEL_5769167aae2149e89087f473b18b799c",
103
+ "placeholder": "​",
104
+ "style": "IPY_MODEL_2a42ff88334748e9aa18c0e7d61d0d02",
105
+ "value": " 1.04M/1.04M [00:01&lt;00:00, 1.13MB/s]"
106
+ }
107
+ },
108
+ "4578787847a8468b899e9e42840b250e": {
109
+ "model_module": "@jupyter-widgets/base",
110
+ "model_name": "LayoutModel",
111
+ "model_module_version": "1.2.0",
112
+ "state": {
113
+ "_model_module": "@jupyter-widgets/base",
114
+ "_model_module_version": "1.2.0",
115
+ "_model_name": "LayoutModel",
116
+ "_view_count": null,
117
+ "_view_module": "@jupyter-widgets/base",
118
+ "_view_module_version": "1.2.0",
119
+ "_view_name": "LayoutView",
120
+ "align_content": null,
121
+ "align_items": null,
122
+ "align_self": null,
123
+ "border": null,
124
+ "bottom": null,
125
+ "display": null,
126
+ "flex": null,
127
+ "flex_flow": null,
128
+ "grid_area": null,
129
+ "grid_auto_columns": null,
130
+ "grid_auto_flow": null,
131
+ "grid_auto_rows": null,
132
+ "grid_column": null,
133
+ "grid_gap": null,
134
+ "grid_row": null,
135
+ "grid_template_areas": null,
136
+ "grid_template_columns": null,
137
+ "grid_template_rows": null,
138
+ "height": null,
139
+ "justify_content": null,
140
+ "justify_items": null,
141
+ "left": null,
142
+ "margin": null,
143
+ "max_height": null,
144
+ "max_width": null,
145
+ "min_height": null,
146
+ "min_width": null,
147
+ "object_fit": null,
148
+ "object_position": null,
149
+ "order": null,
150
+ "overflow": null,
151
+ "overflow_x": null,
152
+ "overflow_y": null,
153
+ "padding": null,
154
+ "right": null,
155
+ "top": null,
156
+ "visibility": null,
157
+ "width": null
158
+ }
159
+ },
160
+ "4dc4e8f371664dbaa5ddef539d7cece3": {
161
+ "model_module": "@jupyter-widgets/base",
162
+ "model_name": "LayoutModel",
163
+ "model_module_version": "1.2.0",
164
+ "state": {
165
+ "_model_module": "@jupyter-widgets/base",
166
+ "_model_module_version": "1.2.0",
167
+ "_model_name": "LayoutModel",
168
+ "_view_count": null,
169
+ "_view_module": "@jupyter-widgets/base",
170
+ "_view_module_version": "1.2.0",
171
+ "_view_name": "LayoutView",
172
+ "align_content": null,
173
+ "align_items": null,
174
+ "align_self": null,
175
+ "border": null,
176
+ "bottom": null,
177
+ "display": null,
178
+ "flex": null,
179
+ "flex_flow": null,
180
+ "grid_area": null,
181
+ "grid_auto_columns": null,
182
+ "grid_auto_flow": null,
183
+ "grid_auto_rows": null,
184
+ "grid_column": null,
185
+ "grid_gap": null,
186
+ "grid_row": null,
187
+ "grid_template_areas": null,
188
+ "grid_template_columns": null,
189
+ "grid_template_rows": null,
190
+ "height": null,
191
+ "justify_content": null,
192
+ "justify_items": null,
193
+ "left": null,
194
+ "margin": null,
195
+ "max_height": null,
196
+ "max_width": null,
197
+ "min_height": null,
198
+ "min_width": null,
199
+ "object_fit": null,
200
+ "object_position": null,
201
+ "order": null,
202
+ "overflow": null,
203
+ "overflow_x": null,
204
+ "overflow_y": null,
205
+ "padding": null,
206
+ "right": null,
207
+ "top": null,
208
+ "visibility": null,
209
+ "width": null
210
+ }
211
+ },
212
+ "a2752dd9fccd41f2a2f5d1c6aa59fec3": {
213
+ "model_module": "@jupyter-widgets/controls",
214
+ "model_name": "DescriptionStyleModel",
215
+ "model_module_version": "1.5.0",
216
+ "state": {
217
+ "_model_module": "@jupyter-widgets/controls",
218
+ "_model_module_version": "1.5.0",
219
+ "_model_name": "DescriptionStyleModel",
220
+ "_view_count": null,
221
+ "_view_module": "@jupyter-widgets/base",
222
+ "_view_module_version": "1.2.0",
223
+ "_view_name": "StyleView",
224
+ "description_width": ""
225
+ }
226
+ },
227
+ "5cc6738e683e474c9a8f1912ee2b7f4d": {
228
+ "model_module": "@jupyter-widgets/base",
229
+ "model_name": "LayoutModel",
230
+ "model_module_version": "1.2.0",
231
+ "state": {
232
+ "_model_module": "@jupyter-widgets/base",
233
+ "_model_module_version": "1.2.0",
234
+ "_model_name": "LayoutModel",
235
+ "_view_count": null,
236
+ "_view_module": "@jupyter-widgets/base",
237
+ "_view_module_version": "1.2.0",
238
+ "_view_name": "LayoutView",
239
+ "align_content": null,
240
+ "align_items": null,
241
+ "align_self": null,
242
+ "border": null,
243
+ "bottom": null,
244
+ "display": null,
245
+ "flex": null,
246
+ "flex_flow": null,
247
+ "grid_area": null,
248
+ "grid_auto_columns": null,
249
+ "grid_auto_flow": null,
250
+ "grid_auto_rows": null,
251
+ "grid_column": null,
252
+ "grid_gap": null,
253
+ "grid_row": null,
254
+ "grid_template_areas": null,
255
+ "grid_template_columns": null,
256
+ "grid_template_rows": null,
257
+ "height": null,
258
+ "justify_content": null,
259
+ "justify_items": null,
260
+ "left": null,
261
+ "margin": null,
262
+ "max_height": null,
263
+ "max_width": null,
264
+ "min_height": null,
265
+ "min_width": null,
266
+ "object_fit": null,
267
+ "object_position": null,
268
+ "order": null,
269
+ "overflow": null,
270
+ "overflow_x": null,
271
+ "overflow_y": null,
272
+ "padding": null,
273
+ "right": null,
274
+ "top": null,
275
+ "visibility": null,
276
+ "width": null
277
+ }
278
+ },
279
+ "fe4b9ccec34d440486f08263444f0606": {
280
+ "model_module": "@jupyter-widgets/controls",
281
+ "model_name": "ProgressStyleModel",
282
+ "model_module_version": "1.5.0",
283
+ "state": {
284
+ "_model_module": "@jupyter-widgets/controls",
285
+ "_model_module_version": "1.5.0",
286
+ "_model_name": "ProgressStyleModel",
287
+ "_view_count": null,
288
+ "_view_module": "@jupyter-widgets/base",
289
+ "_view_module_version": "1.2.0",
290
+ "_view_name": "StyleView",
291
+ "bar_color": null,
292
+ "description_width": ""
293
+ }
294
+ },
295
+ "5769167aae2149e89087f473b18b799c": {
296
+ "model_module": "@jupyter-widgets/base",
297
+ "model_name": "LayoutModel",
298
+ "model_module_version": "1.2.0",
299
+ "state": {
300
+ "_model_module": "@jupyter-widgets/base",
301
+ "_model_module_version": "1.2.0",
302
+ "_model_name": "LayoutModel",
303
+ "_view_count": null,
304
+ "_view_module": "@jupyter-widgets/base",
305
+ "_view_module_version": "1.2.0",
306
+ "_view_name": "LayoutView",
307
+ "align_content": null,
308
+ "align_items": null,
309
+ "align_self": null,
310
+ "border": null,
311
+ "bottom": null,
312
+ "display": null,
313
+ "flex": null,
314
+ "flex_flow": null,
315
+ "grid_area": null,
316
+ "grid_auto_columns": null,
317
+ "grid_auto_flow": null,
318
+ "grid_auto_rows": null,
319
+ "grid_column": null,
320
+ "grid_gap": null,
321
+ "grid_row": null,
322
+ "grid_template_areas": null,
323
+ "grid_template_columns": null,
324
+ "grid_template_rows": null,
325
+ "height": null,
326
+ "justify_content": null,
327
+ "justify_items": null,
328
+ "left": null,
329
+ "margin": null,
330
+ "max_height": null,
331
+ "max_width": null,
332
+ "min_height": null,
333
+ "min_width": null,
334
+ "object_fit": null,
335
+ "object_position": null,
336
+ "order": null,
337
+ "overflow": null,
338
+ "overflow_x": null,
339
+ "overflow_y": null,
340
+ "padding": null,
341
+ "right": null,
342
+ "top": null,
343
+ "visibility": null,
344
+ "width": null
345
+ }
346
+ },
347
+ "2a42ff88334748e9aa18c0e7d61d0d02": {
348
+ "model_module": "@jupyter-widgets/controls",
349
+ "model_name": "DescriptionStyleModel",
350
+ "model_module_version": "1.5.0",
351
+ "state": {
352
+ "_model_module": "@jupyter-widgets/controls",
353
+ "_model_module_version": "1.5.0",
354
+ "_model_name": "DescriptionStyleModel",
355
+ "_view_count": null,
356
+ "_view_module": "@jupyter-widgets/base",
357
+ "_view_module_version": "1.2.0",
358
+ "_view_name": "StyleView",
359
+ "description_width": ""
360
+ }
361
+ },
362
+ "055803bb341f412d95386eb37b367681": {
363
+ "model_module": "@jupyter-widgets/controls",
364
+ "model_name": "HBoxModel",
365
+ "model_module_version": "1.5.0",
366
+ "state": {
367
+ "_dom_classes": [],
368
+ "_model_module": "@jupyter-widgets/controls",
369
+ "_model_module_version": "1.5.0",
370
+ "_model_name": "HBoxModel",
371
+ "_view_count": null,
372
+ "_view_module": "@jupyter-widgets/controls",
373
+ "_view_module_version": "1.5.0",
374
+ "_view_name": "HBoxView",
375
+ "box_style": "",
376
+ "children": [
377
+ "IPY_MODEL_0036c108105d4b3c86155a15e048ff6b",
378
+ "IPY_MODEL_58851499d1f0416590cd7ca275c1ddef",
379
+ "IPY_MODEL_417ce7fdf3f049d8b9d9b49913b56c45"
380
+ ],
381
+ "layout": "IPY_MODEL_942710acc5e54bbba743632472cc1399"
382
+ }
383
+ },
384
+ "0036c108105d4b3c86155a15e048ff6b": {
385
+ "model_module": "@jupyter-widgets/controls",
386
+ "model_name": "HTMLModel",
387
+ "model_module_version": "1.5.0",
388
+ "state": {
389
+ "_dom_classes": [],
390
+ "_model_module": "@jupyter-widgets/controls",
391
+ "_model_module_version": "1.5.0",
392
+ "_model_name": "HTMLModel",
393
+ "_view_count": null,
394
+ "_view_module": "@jupyter-widgets/controls",
395
+ "_view_module_version": "1.5.0",
396
+ "_view_name": "HTMLView",
397
+ "description": "",
398
+ "description_tooltip": null,
399
+ "layout": "IPY_MODEL_d462118b7f114fcb8c3589996f5ab536",
400
+ "placeholder": "​",
401
+ "style": "IPY_MODEL_3b25cfad56434cb1a44e5a9853b97573",
402
+ "value": "Downloading: 100%"
403
+ }
404
+ },
405
+ "58851499d1f0416590cd7ca275c1ddef": {
406
+ "model_module": "@jupyter-widgets/controls",
407
+ "model_name": "FloatProgressModel",
408
+ "model_module_version": "1.5.0",
409
+ "state": {
410
+ "_dom_classes": [],
411
+ "_model_module": "@jupyter-widgets/controls",
412
+ "_model_module_version": "1.5.0",
413
+ "_model_name": "FloatProgressModel",
414
+ "_view_count": null,
415
+ "_view_module": "@jupyter-widgets/controls",
416
+ "_view_module_version": "1.5.0",
417
+ "_view_name": "ProgressView",
418
+ "bar_style": "success",
419
+ "description": "",
420
+ "description_tooltip": null,
421
+ "layout": "IPY_MODEL_a2bc95389bc64e089b3b950327e4302a",
422
+ "max": 456318,
423
+ "min": 0,
424
+ "orientation": "horizontal",
425
+ "style": "IPY_MODEL_051c868590b641ae9f3adf386beabdee",
426
+ "value": 456318
427
+ }
428
+ },
429
+ "417ce7fdf3f049d8b9d9b49913b56c45": {
430
+ "model_module": "@jupyter-widgets/controls",
431
+ "model_name": "HTMLModel",
432
+ "model_module_version": "1.5.0",
433
+ "state": {
434
+ "_dom_classes": [],
435
+ "_model_module": "@jupyter-widgets/controls",
436
+ "_model_module_version": "1.5.0",
437
+ "_model_name": "HTMLModel",
438
+ "_view_count": null,
439
+ "_view_module": "@jupyter-widgets/controls",
440
+ "_view_module_version": "1.5.0",
441
+ "_view_name": "HTMLView",
442
+ "description": "",
443
+ "description_tooltip": null,
444
+ "layout": "IPY_MODEL_24b5d163f535447d8237ec44021cf537",
445
+ "placeholder": "​",
446
+ "style": "IPY_MODEL_98e7e8229a5c45678bb8fc50cb56995f",
447
+ "value": " 456k/456k [00:00&lt;00:00, 642kB/s]"
448
+ }
449
+ },
450
+ "942710acc5e54bbba743632472cc1399": {
451
+ "model_module": "@jupyter-widgets/base",
452
+ "model_name": "LayoutModel",
453
+ "model_module_version": "1.2.0",
454
+ "state": {
455
+ "_model_module": "@jupyter-widgets/base",
456
+ "_model_module_version": "1.2.0",
457
+ "_model_name": "LayoutModel",
458
+ "_view_count": null,
459
+ "_view_module": "@jupyter-widgets/base",
460
+ "_view_module_version": "1.2.0",
461
+ "_view_name": "LayoutView",
462
+ "align_content": null,
463
+ "align_items": null,
464
+ "align_self": null,
465
+ "border": null,
466
+ "bottom": null,
467
+ "display": null,
468
+ "flex": null,
469
+ "flex_flow": null,
470
+ "grid_area": null,
471
+ "grid_auto_columns": null,
472
+ "grid_auto_flow": null,
473
+ "grid_auto_rows": null,
474
+ "grid_column": null,
475
+ "grid_gap": null,
476
+ "grid_row": null,
477
+ "grid_template_areas": null,
478
+ "grid_template_columns": null,
479
+ "grid_template_rows": null,
480
+ "height": null,
481
+ "justify_content": null,
482
+ "justify_items": null,
483
+ "left": null,
484
+ "margin": null,
485
+ "max_height": null,
486
+ "max_width": null,
487
+ "min_height": null,
488
+ "min_width": null,
489
+ "object_fit": null,
490
+ "object_position": null,
491
+ "order": null,
492
+ "overflow": null,
493
+ "overflow_x": null,
494
+ "overflow_y": null,
495
+ "padding": null,
496
+ "right": null,
497
+ "top": null,
498
+ "visibility": null,
499
+ "width": null
500
+ }
501
+ },
502
+ "d462118b7f114fcb8c3589996f5ab536": {
503
+ "model_module": "@jupyter-widgets/base",
504
+ "model_name": "LayoutModel",
505
+ "model_module_version": "1.2.0",
506
+ "state": {
507
+ "_model_module": "@jupyter-widgets/base",
508
+ "_model_module_version": "1.2.0",
509
+ "_model_name": "LayoutModel",
510
+ "_view_count": null,
511
+ "_view_module": "@jupyter-widgets/base",
512
+ "_view_module_version": "1.2.0",
513
+ "_view_name": "LayoutView",
514
+ "align_content": null,
515
+ "align_items": null,
516
+ "align_self": null,
517
+ "border": null,
518
+ "bottom": null,
519
+ "display": null,
520
+ "flex": null,
521
+ "flex_flow": null,
522
+ "grid_area": null,
523
+ "grid_auto_columns": null,
524
+ "grid_auto_flow": null,
525
+ "grid_auto_rows": null,
526
+ "grid_column": null,
527
+ "grid_gap": null,
528
+ "grid_row": null,
529
+ "grid_template_areas": null,
530
+ "grid_template_columns": null,
531
+ "grid_template_rows": null,
532
+ "height": null,
533
+ "justify_content": null,
534
+ "justify_items": null,
535
+ "left": null,
536
+ "margin": null,
537
+ "max_height": null,
538
+ "max_width": null,
539
+ "min_height": null,
540
+ "min_width": null,
541
+ "object_fit": null,
542
+ "object_position": null,
543
+ "order": null,
544
+ "overflow": null,
545
+ "overflow_x": null,
546
+ "overflow_y": null,
547
+ "padding": null,
548
+ "right": null,
549
+ "top": null,
550
+ "visibility": null,
551
+ "width": null
552
+ }
553
+ },
554
+ "3b25cfad56434cb1a44e5a9853b97573": {
555
+ "model_module": "@jupyter-widgets/controls",
556
+ "model_name": "DescriptionStyleModel",
557
+ "model_module_version": "1.5.0",
558
+ "state": {
559
+ "_model_module": "@jupyter-widgets/controls",
560
+ "_model_module_version": "1.5.0",
561
+ "_model_name": "DescriptionStyleModel",
562
+ "_view_count": null,
563
+ "_view_module": "@jupyter-widgets/base",
564
+ "_view_module_version": "1.2.0",
565
+ "_view_name": "StyleView",
566
+ "description_width": ""
567
+ }
568
+ },
569
+ "a2bc95389bc64e089b3b950327e4302a": {
570
+ "model_module": "@jupyter-widgets/base",
571
+ "model_name": "LayoutModel",
572
+ "model_module_version": "1.2.0",
573
+ "state": {
574
+ "_model_module": "@jupyter-widgets/base",
575
+ "_model_module_version": "1.2.0",
576
+ "_model_name": "LayoutModel",
577
+ "_view_count": null,
578
+ "_view_module": "@jupyter-widgets/base",
579
+ "_view_module_version": "1.2.0",
580
+ "_view_name": "LayoutView",
581
+ "align_content": null,
582
+ "align_items": null,
583
+ "align_self": null,
584
+ "border": null,
585
+ "bottom": null,
586
+ "display": null,
587
+ "flex": null,
588
+ "flex_flow": null,
589
+ "grid_area": null,
590
+ "grid_auto_columns": null,
591
+ "grid_auto_flow": null,
592
+ "grid_auto_rows": null,
593
+ "grid_column": null,
594
+ "grid_gap": null,
595
+ "grid_row": null,
596
+ "grid_template_areas": null,
597
+ "grid_template_columns": null,
598
+ "grid_template_rows": null,
599
+ "height": null,
600
+ "justify_content": null,
601
+ "justify_items": null,
602
+ "left": null,
603
+ "margin": null,
604
+ "max_height": null,
605
+ "max_width": null,
606
+ "min_height": null,
607
+ "min_width": null,
608
+ "object_fit": null,
609
+ "object_position": null,
610
+ "order": null,
611
+ "overflow": null,
612
+ "overflow_x": null,
613
+ "overflow_y": null,
614
+ "padding": null,
615
+ "right": null,
616
+ "top": null,
617
+ "visibility": null,
618
+ "width": null
619
+ }
620
+ },
621
+ "051c868590b641ae9f3adf386beabdee": {
622
+ "model_module": "@jupyter-widgets/controls",
623
+ "model_name": "ProgressStyleModel",
624
+ "model_module_version": "1.5.0",
625
+ "state": {
626
+ "_model_module": "@jupyter-widgets/controls",
627
+ "_model_module_version": "1.5.0",
628
+ "_model_name": "ProgressStyleModel",
629
+ "_view_count": null,
630
+ "_view_module": "@jupyter-widgets/base",
631
+ "_view_module_version": "1.2.0",
632
+ "_view_name": "StyleView",
633
+ "bar_color": null,
634
+ "description_width": ""
635
+ }
636
+ },
637
+ "24b5d163f535447d8237ec44021cf537": {
638
+ "model_module": "@jupyter-widgets/base",
639
+ "model_name": "LayoutModel",
640
+ "model_module_version": "1.2.0",
641
+ "state": {
642
+ "_model_module": "@jupyter-widgets/base",
643
+ "_model_module_version": "1.2.0",
644
+ "_model_name": "LayoutModel",
645
+ "_view_count": null,
646
+ "_view_module": "@jupyter-widgets/base",
647
+ "_view_module_version": "1.2.0",
648
+ "_view_name": "LayoutView",
649
+ "align_content": null,
650
+ "align_items": null,
651
+ "align_self": null,
652
+ "border": null,
653
+ "bottom": null,
654
+ "display": null,
655
+ "flex": null,
656
+ "flex_flow": null,
657
+ "grid_area": null,
658
+ "grid_auto_columns": null,
659
+ "grid_auto_flow": null,
660
+ "grid_auto_rows": null,
661
+ "grid_column": null,
662
+ "grid_gap": null,
663
+ "grid_row": null,
664
+ "grid_template_areas": null,
665
+ "grid_template_columns": null,
666
+ "grid_template_rows": null,
667
+ "height": null,
668
+ "justify_content": null,
669
+ "justify_items": null,
670
+ "left": null,
671
+ "margin": null,
672
+ "max_height": null,
673
+ "max_width": null,
674
+ "min_height": null,
675
+ "min_width": null,
676
+ "object_fit": null,
677
+ "object_position": null,
678
+ "order": null,
679
+ "overflow": null,
680
+ "overflow_x": null,
681
+ "overflow_y": null,
682
+ "padding": null,
683
+ "right": null,
684
+ "top": null,
685
+ "visibility": null,
686
+ "width": null
687
+ }
688
+ },
689
+ "98e7e8229a5c45678bb8fc50cb56995f": {
690
+ "model_module": "@jupyter-widgets/controls",
691
+ "model_name": "DescriptionStyleModel",
692
+ "model_module_version": "1.5.0",
693
+ "state": {
694
+ "_model_module": "@jupyter-widgets/controls",
695
+ "_model_module_version": "1.5.0",
696
+ "_model_name": "DescriptionStyleModel",
697
+ "_view_count": null,
698
+ "_view_module": "@jupyter-widgets/base",
699
+ "_view_module_version": "1.2.0",
700
+ "_view_name": "StyleView",
701
+ "description_width": ""
702
+ }
703
+ },
704
+ "3f34f1aea3674271a03837ac87a54620": {
705
+ "model_module": "@jupyter-widgets/controls",
706
+ "model_name": "HBoxModel",
707
+ "model_module_version": "1.5.0",
708
+ "state": {
709
+ "_dom_classes": [],
710
+ "_model_module": "@jupyter-widgets/controls",
711
+ "_model_module_version": "1.5.0",
712
+ "_model_name": "HBoxModel",
713
+ "_view_count": null,
714
+ "_view_module": "@jupyter-widgets/controls",
715
+ "_view_module_version": "1.5.0",
716
+ "_view_name": "HBoxView",
717
+ "box_style": "",
718
+ "children": [
719
+ "IPY_MODEL_3e06e5ca1ee04650bf0d442487e76872",
720
+ "IPY_MODEL_3c6ab591598649b7818409ac15f52c1e",
721
+ "IPY_MODEL_5f988b61bd3a49e0be57556c6f678fcc"
722
+ ],
723
+ "layout": "IPY_MODEL_288be48cc6644e738369b0f1dbbd3365"
724
+ }
725
+ },
726
+ "3e06e5ca1ee04650bf0d442487e76872": {
727
+ "model_module": "@jupyter-widgets/controls",
728
+ "model_name": "HTMLModel",
729
+ "model_module_version": "1.5.0",
730
+ "state": {
731
+ "_dom_classes": [],
732
+ "_model_module": "@jupyter-widgets/controls",
733
+ "_model_module_version": "1.5.0",
734
+ "_model_name": "HTMLModel",
735
+ "_view_count": null,
736
+ "_view_module": "@jupyter-widgets/controls",
737
+ "_view_module_version": "1.5.0",
738
+ "_view_name": "HTMLView",
739
+ "description": "",
740
+ "description_tooltip": null,
741
+ "layout": "IPY_MODEL_88025aee2bd34c5abcc7e1d697fe5729",
742
+ "placeholder": "​",
743
+ "style": "IPY_MODEL_1f505ffcfd3b4d4aa4f7f36f7f96edc4",
744
+ "value": "Downloading: 100%"
745
+ }
746
+ },
747
+ "3c6ab591598649b7818409ac15f52c1e": {
748
+ "model_module": "@jupyter-widgets/controls",
749
+ "model_name": "FloatProgressModel",
750
+ "model_module_version": "1.5.0",
751
+ "state": {
752
+ "_dom_classes": [],
753
+ "_model_module": "@jupyter-widgets/controls",
754
+ "_model_module_version": "1.5.0",
755
+ "_model_name": "FloatProgressModel",
756
+ "_view_count": null,
757
+ "_view_module": "@jupyter-widgets/controls",
758
+ "_view_module_version": "1.5.0",
759
+ "_view_name": "ProgressView",
760
+ "bar_style": "success",
761
+ "description": "",
762
+ "description_tooltip": null,
763
+ "layout": "IPY_MODEL_d60db7a441764f4191f8ea84fb36fffb",
764
+ "max": 718,
765
+ "min": 0,
766
+ "orientation": "horizontal",
767
+ "style": "IPY_MODEL_f49eb751667f4e0aab6f06472d5422e2",
768
+ "value": 718
769
+ }
770
+ },
771
+ "5f988b61bd3a49e0be57556c6f678fcc": {
772
+ "model_module": "@jupyter-widgets/controls",
773
+ "model_name": "HTMLModel",
774
+ "model_module_version": "1.5.0",
775
+ "state": {
776
+ "_dom_classes": [],
777
+ "_model_module": "@jupyter-widgets/controls",
778
+ "_model_module_version": "1.5.0",
779
+ "_model_name": "HTMLModel",
780
+ "_view_count": null,
781
+ "_view_module": "@jupyter-widgets/controls",
782
+ "_view_module_version": "1.5.0",
783
+ "_view_name": "HTMLView",
784
+ "description": "",
785
+ "description_tooltip": null,
786
+ "layout": "IPY_MODEL_29f9630999554f0a8a319e722cf16ccd",
787
+ "placeholder": "​",
788
+ "style": "IPY_MODEL_0b56cf47dca143efb2f3f946e681bb52",
789
+ "value": " 718/718 [00:00&lt;00:00, 26.8kB/s]"
790
+ }
791
+ },
792
+ "288be48cc6644e738369b0f1dbbd3365": {
793
+ "model_module": "@jupyter-widgets/base",
794
+ "model_name": "LayoutModel",
795
+ "model_module_version": "1.2.0",
796
+ "state": {
797
+ "_model_module": "@jupyter-widgets/base",
798
+ "_model_module_version": "1.2.0",
799
+ "_model_name": "LayoutModel",
800
+ "_view_count": null,
801
+ "_view_module": "@jupyter-widgets/base",
802
+ "_view_module_version": "1.2.0",
803
+ "_view_name": "LayoutView",
804
+ "align_content": null,
805
+ "align_items": null,
806
+ "align_self": null,
807
+ "border": null,
808
+ "bottom": null,
809
+ "display": null,
810
+ "flex": null,
811
+ "flex_flow": null,
812
+ "grid_area": null,
813
+ "grid_auto_columns": null,
814
+ "grid_auto_flow": null,
815
+ "grid_auto_rows": null,
816
+ "grid_column": null,
817
+ "grid_gap": null,
818
+ "grid_row": null,
819
+ "grid_template_areas": null,
820
+ "grid_template_columns": null,
821
+ "grid_template_rows": null,
822
+ "height": null,
823
+ "justify_content": null,
824
+ "justify_items": null,
825
+ "left": null,
826
+ "margin": null,
827
+ "max_height": null,
828
+ "max_width": null,
829
+ "min_height": null,
830
+ "min_width": null,
831
+ "object_fit": null,
832
+ "object_position": null,
833
+ "order": null,
834
+ "overflow": null,
835
+ "overflow_x": null,
836
+ "overflow_y": null,
837
+ "padding": null,
838
+ "right": null,
839
+ "top": null,
840
+ "visibility": null,
841
+ "width": null
842
+ }
843
+ },
844
+ "88025aee2bd34c5abcc7e1d697fe5729": {
845
+ "model_module": "@jupyter-widgets/base",
846
+ "model_name": "LayoutModel",
847
+ "model_module_version": "1.2.0",
848
+ "state": {
849
+ "_model_module": "@jupyter-widgets/base",
850
+ "_model_module_version": "1.2.0",
851
+ "_model_name": "LayoutModel",
852
+ "_view_count": null,
853
+ "_view_module": "@jupyter-widgets/base",
854
+ "_view_module_version": "1.2.0",
855
+ "_view_name": "LayoutView",
856
+ "align_content": null,
857
+ "align_items": null,
858
+ "align_self": null,
859
+ "border": null,
860
+ "bottom": null,
861
+ "display": null,
862
+ "flex": null,
863
+ "flex_flow": null,
864
+ "grid_area": null,
865
+ "grid_auto_columns": null,
866
+ "grid_auto_flow": null,
867
+ "grid_auto_rows": null,
868
+ "grid_column": null,
869
+ "grid_gap": null,
870
+ "grid_row": null,
871
+ "grid_template_areas": null,
872
+ "grid_template_columns": null,
873
+ "grid_template_rows": null,
874
+ "height": null,
875
+ "justify_content": null,
876
+ "justify_items": null,
877
+ "left": null,
878
+ "margin": null,
879
+ "max_height": null,
880
+ "max_width": null,
881
+ "min_height": null,
882
+ "min_width": null,
883
+ "object_fit": null,
884
+ "object_position": null,
885
+ "order": null,
886
+ "overflow": null,
887
+ "overflow_x": null,
888
+ "overflow_y": null,
889
+ "padding": null,
890
+ "right": null,
891
+ "top": null,
892
+ "visibility": null,
893
+ "width": null
894
+ }
895
+ },
896
+ "1f505ffcfd3b4d4aa4f7f36f7f96edc4": {
897
+ "model_module": "@jupyter-widgets/controls",
898
+ "model_name": "DescriptionStyleModel",
899
+ "model_module_version": "1.5.0",
900
+ "state": {
901
+ "_model_module": "@jupyter-widgets/controls",
902
+ "_model_module_version": "1.5.0",
903
+ "_model_name": "DescriptionStyleModel",
904
+ "_view_count": null,
905
+ "_view_module": "@jupyter-widgets/base",
906
+ "_view_module_version": "1.2.0",
907
+ "_view_name": "StyleView",
908
+ "description_width": ""
909
+ }
910
+ },
911
+ "d60db7a441764f4191f8ea84fb36fffb": {
912
+ "model_module": "@jupyter-widgets/base",
913
+ "model_name": "LayoutModel",
914
+ "model_module_version": "1.2.0",
915
+ "state": {
916
+ "_model_module": "@jupyter-widgets/base",
917
+ "_model_module_version": "1.2.0",
918
+ "_model_name": "LayoutModel",
919
+ "_view_count": null,
920
+ "_view_module": "@jupyter-widgets/base",
921
+ "_view_module_version": "1.2.0",
922
+ "_view_name": "LayoutView",
923
+ "align_content": null,
924
+ "align_items": null,
925
+ "align_self": null,
926
+ "border": null,
927
+ "bottom": null,
928
+ "display": null,
929
+ "flex": null,
930
+ "flex_flow": null,
931
+ "grid_area": null,
932
+ "grid_auto_columns": null,
933
+ "grid_auto_flow": null,
934
+ "grid_auto_rows": null,
935
+ "grid_column": null,
936
+ "grid_gap": null,
937
+ "grid_row": null,
938
+ "grid_template_areas": null,
939
+ "grid_template_columns": null,
940
+ "grid_template_rows": null,
941
+ "height": null,
942
+ "justify_content": null,
943
+ "justify_items": null,
944
+ "left": null,
945
+ "margin": null,
946
+ "max_height": null,
947
+ "max_width": null,
948
+ "min_height": null,
949
+ "min_width": null,
950
+ "object_fit": null,
951
+ "object_position": null,
952
+ "order": null,
953
+ "overflow": null,
954
+ "overflow_x": null,
955
+ "overflow_y": null,
956
+ "padding": null,
957
+ "right": null,
958
+ "top": null,
959
+ "visibility": null,
960
+ "width": null
961
+ }
962
+ },
963
+ "f49eb751667f4e0aab6f06472d5422e2": {
964
+ "model_module": "@jupyter-widgets/controls",
965
+ "model_name": "ProgressStyleModel",
966
+ "model_module_version": "1.5.0",
967
+ "state": {
968
+ "_model_module": "@jupyter-widgets/controls",
969
+ "_model_module_version": "1.5.0",
970
+ "_model_name": "ProgressStyleModel",
971
+ "_view_count": null,
972
+ "_view_module": "@jupyter-widgets/base",
973
+ "_view_module_version": "1.2.0",
974
+ "_view_name": "StyleView",
975
+ "bar_color": null,
976
+ "description_width": ""
977
+ }
978
+ },
979
+ "29f9630999554f0a8a319e722cf16ccd": {
980
+ "model_module": "@jupyter-widgets/base",
981
+ "model_name": "LayoutModel",
982
+ "model_module_version": "1.2.0",
983
+ "state": {
984
+ "_model_module": "@jupyter-widgets/base",
985
+ "_model_module_version": "1.2.0",
986
+ "_model_name": "LayoutModel",
987
+ "_view_count": null,
988
+ "_view_module": "@jupyter-widgets/base",
989
+ "_view_module_version": "1.2.0",
990
+ "_view_name": "LayoutView",
991
+ "align_content": null,
992
+ "align_items": null,
993
+ "align_self": null,
994
+ "border": null,
995
+ "bottom": null,
996
+ "display": null,
997
+ "flex": null,
998
+ "flex_flow": null,
999
+ "grid_area": null,
1000
+ "grid_auto_columns": null,
1001
+ "grid_auto_flow": null,
1002
+ "grid_auto_rows": null,
1003
+ "grid_column": null,
1004
+ "grid_gap": null,
1005
+ "grid_row": null,
1006
+ "grid_template_areas": null,
1007
+ "grid_template_columns": null,
1008
+ "grid_template_rows": null,
1009
+ "height": null,
1010
+ "justify_content": null,
1011
+ "justify_items": null,
1012
+ "left": null,
1013
+ "margin": null,
1014
+ "max_height": null,
1015
+ "max_width": null,
1016
+ "min_height": null,
1017
+ "min_width": null,
1018
+ "object_fit": null,
1019
+ "object_position": null,
1020
+ "order": null,
1021
+ "overflow": null,
1022
+ "overflow_x": null,
1023
+ "overflow_y": null,
1024
+ "padding": null,
1025
+ "right": null,
1026
+ "top": null,
1027
+ "visibility": null,
1028
+ "width": null
1029
+ }
1030
+ },
1031
+ "0b56cf47dca143efb2f3f946e681bb52": {
1032
+ "model_module": "@jupyter-widgets/controls",
1033
+ "model_name": "DescriptionStyleModel",
1034
+ "model_module_version": "1.5.0",
1035
+ "state": {
1036
+ "_model_module": "@jupyter-widgets/controls",
1037
+ "_model_module_version": "1.5.0",
1038
+ "_model_name": "DescriptionStyleModel",
1039
+ "_view_count": null,
1040
+ "_view_module": "@jupyter-widgets/base",
1041
+ "_view_module_version": "1.2.0",
1042
+ "_view_name": "StyleView",
1043
+ "description_width": ""
1044
+ }
1045
+ },
1046
+ "d0da26df89d943a48cf782d37c29c994": {
1047
+ "model_module": "@jupyter-widgets/controls",
1048
+ "model_name": "HBoxModel",
1049
+ "model_module_version": "1.5.0",
1050
+ "state": {
1051
+ "_dom_classes": [],
1052
+ "_model_module": "@jupyter-widgets/controls",
1053
+ "_model_module_version": "1.5.0",
1054
+ "_model_name": "HBoxModel",
1055
+ "_view_count": null,
1056
+ "_view_module": "@jupyter-widgets/controls",
1057
+ "_view_module_version": "1.5.0",
1058
+ "_view_name": "HBoxView",
1059
+ "box_style": "",
1060
+ "children": [
1061
+ "IPY_MODEL_9a31634d798c480bb8e294dff5de7795",
1062
+ "IPY_MODEL_41ab5d5f96b74fe0aad1b5f7a0d6c211",
1063
+ "IPY_MODEL_c049b6f4c8d548fb88a4af78994e5aa3"
1064
+ ],
1065
+ "layout": "IPY_MODEL_cb7026bb90d044b98c722b6a7b90d5dd"
1066
+ }
1067
+ },
1068
+ "9a31634d798c480bb8e294dff5de7795": {
1069
+ "model_module": "@jupyter-widgets/controls",
1070
+ "model_name": "HTMLModel",
1071
+ "model_module_version": "1.5.0",
1072
+ "state": {
1073
+ "_dom_classes": [],
1074
+ "_model_module": "@jupyter-widgets/controls",
1075
+ "_model_module_version": "1.5.0",
1076
+ "_model_name": "HTMLModel",
1077
+ "_view_count": null,
1078
+ "_view_module": "@jupyter-widgets/controls",
1079
+ "_view_module_version": "1.5.0",
1080
+ "_view_name": "HTMLView",
1081
+ "description": "",
1082
+ "description_tooltip": null,
1083
+ "layout": "IPY_MODEL_b4e3c39c1d564fda8776b8b0bb6a2797",
1084
+ "placeholder": "​",
1085
+ "style": "IPY_MODEL_2d1b7605a31a422a8030ac16e7b0aab3",
1086
+ "value": "Downloading: 100%"
1087
+ }
1088
+ },
1089
+ "41ab5d5f96b74fe0aad1b5f7a0d6c211": {
1090
+ "model_module": "@jupyter-widgets/controls",
1091
+ "model_name": "FloatProgressModel",
1092
+ "model_module_version": "1.5.0",
1093
+ "state": {
1094
+ "_dom_classes": [],
1095
+ "_model_module": "@jupyter-widgets/controls",
1096
+ "_model_module_version": "1.5.0",
1097
+ "_model_name": "FloatProgressModel",
1098
+ "_view_count": null,
1099
+ "_view_module": "@jupyter-widgets/controls",
1100
+ "_view_module_version": "1.5.0",
1101
+ "_view_name": "ProgressView",
1102
+ "bar_style": "success",
1103
+ "description": "",
1104
+ "description_tooltip": null,
1105
+ "layout": "IPY_MODEL_cf65384d80f849bfbb7e0b3e54bae664",
1106
+ "max": 665,
1107
+ "min": 0,
1108
+ "orientation": "horizontal",
1109
+ "style": "IPY_MODEL_fdd9de7b00d0417d8fe8fcfc59119739",
1110
+ "value": 665
1111
+ }
1112
+ },
1113
+ "c049b6f4c8d548fb88a4af78994e5aa3": {
1114
+ "model_module": "@jupyter-widgets/controls",
1115
+ "model_name": "HTMLModel",
1116
+ "model_module_version": "1.5.0",
1117
+ "state": {
1118
+ "_dom_classes": [],
1119
+ "_model_module": "@jupyter-widgets/controls",
1120
+ "_model_module_version": "1.5.0",
1121
+ "_model_name": "HTMLModel",
1122
+ "_view_count": null,
1123
+ "_view_module": "@jupyter-widgets/controls",
1124
+ "_view_module_version": "1.5.0",
1125
+ "_view_name": "HTMLView",
1126
+ "description": "",
1127
+ "description_tooltip": null,
1128
+ "layout": "IPY_MODEL_e83bc96f3c4e4084b65ea22a893edea2",
1129
+ "placeholder": "​",
1130
+ "style": "IPY_MODEL_b55968f802eb4df9b15829ec04e14071",
1131
+ "value": " 665/665 [00:00&lt;00:00, 45.4kB/s]"
1132
+ }
1133
+ },
1134
+ "cb7026bb90d044b98c722b6a7b90d5dd": {
1135
+ "model_module": "@jupyter-widgets/base",
1136
+ "model_name": "LayoutModel",
1137
+ "model_module_version": "1.2.0",
1138
+ "state": {
1139
+ "_model_module": "@jupyter-widgets/base",
1140
+ "_model_module_version": "1.2.0",
1141
+ "_model_name": "LayoutModel",
1142
+ "_view_count": null,
1143
+ "_view_module": "@jupyter-widgets/base",
1144
+ "_view_module_version": "1.2.0",
1145
+ "_view_name": "LayoutView",
1146
+ "align_content": null,
1147
+ "align_items": null,
1148
+ "align_self": null,
1149
+ "border": null,
1150
+ "bottom": null,
1151
+ "display": null,
1152
+ "flex": null,
1153
+ "flex_flow": null,
1154
+ "grid_area": null,
1155
+ "grid_auto_columns": null,
1156
+ "grid_auto_flow": null,
1157
+ "grid_auto_rows": null,
1158
+ "grid_column": null,
1159
+ "grid_gap": null,
1160
+ "grid_row": null,
1161
+ "grid_template_areas": null,
1162
+ "grid_template_columns": null,
1163
+ "grid_template_rows": null,
1164
+ "height": null,
1165
+ "justify_content": null,
1166
+ "justify_items": null,
1167
+ "left": null,
1168
+ "margin": null,
1169
+ "max_height": null,
1170
+ "max_width": null,
1171
+ "min_height": null,
1172
+ "min_width": null,
1173
+ "object_fit": null,
1174
+ "object_position": null,
1175
+ "order": null,
1176
+ "overflow": null,
1177
+ "overflow_x": null,
1178
+ "overflow_y": null,
1179
+ "padding": null,
1180
+ "right": null,
1181
+ "top": null,
1182
+ "visibility": null,
1183
+ "width": null
1184
+ }
1185
+ },
1186
+ "b4e3c39c1d564fda8776b8b0bb6a2797": {
1187
+ "model_module": "@jupyter-widgets/base",
1188
+ "model_name": "LayoutModel",
1189
+ "model_module_version": "1.2.0",
1190
+ "state": {
1191
+ "_model_module": "@jupyter-widgets/base",
1192
+ "_model_module_version": "1.2.0",
1193
+ "_model_name": "LayoutModel",
1194
+ "_view_count": null,
1195
+ "_view_module": "@jupyter-widgets/base",
1196
+ "_view_module_version": "1.2.0",
1197
+ "_view_name": "LayoutView",
1198
+ "align_content": null,
1199
+ "align_items": null,
1200
+ "align_self": null,
1201
+ "border": null,
1202
+ "bottom": null,
1203
+ "display": null,
1204
+ "flex": null,
1205
+ "flex_flow": null,
1206
+ "grid_area": null,
1207
+ "grid_auto_columns": null,
1208
+ "grid_auto_flow": null,
1209
+ "grid_auto_rows": null,
1210
+ "grid_column": null,
1211
+ "grid_gap": null,
1212
+ "grid_row": null,
1213
+ "grid_template_areas": null,
1214
+ "grid_template_columns": null,
1215
+ "grid_template_rows": null,
1216
+ "height": null,
1217
+ "justify_content": null,
1218
+ "justify_items": null,
1219
+ "left": null,
1220
+ "margin": null,
1221
+ "max_height": null,
1222
+ "max_width": null,
1223
+ "min_height": null,
1224
+ "min_width": null,
1225
+ "object_fit": null,
1226
+ "object_position": null,
1227
+ "order": null,
1228
+ "overflow": null,
1229
+ "overflow_x": null,
1230
+ "overflow_y": null,
1231
+ "padding": null,
1232
+ "right": null,
1233
+ "top": null,
1234
+ "visibility": null,
1235
+ "width": null
1236
+ }
1237
+ },
1238
+ "2d1b7605a31a422a8030ac16e7b0aab3": {
1239
+ "model_module": "@jupyter-widgets/controls",
1240
+ "model_name": "DescriptionStyleModel",
1241
+ "model_module_version": "1.5.0",
1242
+ "state": {
1243
+ "_model_module": "@jupyter-widgets/controls",
1244
+ "_model_module_version": "1.5.0",
1245
+ "_model_name": "DescriptionStyleModel",
1246
+ "_view_count": null,
1247
+ "_view_module": "@jupyter-widgets/base",
1248
+ "_view_module_version": "1.2.0",
1249
+ "_view_name": "StyleView",
1250
+ "description_width": ""
1251
+ }
1252
+ },
1253
+ "cf65384d80f849bfbb7e0b3e54bae664": {
1254
+ "model_module": "@jupyter-widgets/base",
1255
+ "model_name": "LayoutModel",
1256
+ "model_module_version": "1.2.0",
1257
+ "state": {
1258
+ "_model_module": "@jupyter-widgets/base",
1259
+ "_model_module_version": "1.2.0",
1260
+ "_model_name": "LayoutModel",
1261
+ "_view_count": null,
1262
+ "_view_module": "@jupyter-widgets/base",
1263
+ "_view_module_version": "1.2.0",
1264
+ "_view_name": "LayoutView",
1265
+ "align_content": null,
1266
+ "align_items": null,
1267
+ "align_self": null,
1268
+ "border": null,
1269
+ "bottom": null,
1270
+ "display": null,
1271
+ "flex": null,
1272
+ "flex_flow": null,
1273
+ "grid_area": null,
1274
+ "grid_auto_columns": null,
1275
+ "grid_auto_flow": null,
1276
+ "grid_auto_rows": null,
1277
+ "grid_column": null,
1278
+ "grid_gap": null,
1279
+ "grid_row": null,
1280
+ "grid_template_areas": null,
1281
+ "grid_template_columns": null,
1282
+ "grid_template_rows": null,
1283
+ "height": null,
1284
+ "justify_content": null,
1285
+ "justify_items": null,
1286
+ "left": null,
1287
+ "margin": null,
1288
+ "max_height": null,
1289
+ "max_width": null,
1290
+ "min_height": null,
1291
+ "min_width": null,
1292
+ "object_fit": null,
1293
+ "object_position": null,
1294
+ "order": null,
1295
+ "overflow": null,
1296
+ "overflow_x": null,
1297
+ "overflow_y": null,
1298
+ "padding": null,
1299
+ "right": null,
1300
+ "top": null,
1301
+ "visibility": null,
1302
+ "width": null
1303
+ }
1304
+ },
1305
+ "fdd9de7b00d0417d8fe8fcfc59119739": {
1306
+ "model_module": "@jupyter-widgets/controls",
1307
+ "model_name": "ProgressStyleModel",
1308
+ "model_module_version": "1.5.0",
1309
+ "state": {
1310
+ "_model_module": "@jupyter-widgets/controls",
1311
+ "_model_module_version": "1.5.0",
1312
+ "_model_name": "ProgressStyleModel",
1313
+ "_view_count": null,
1314
+ "_view_module": "@jupyter-widgets/base",
1315
+ "_view_module_version": "1.2.0",
1316
+ "_view_name": "StyleView",
1317
+ "bar_color": null,
1318
+ "description_width": ""
1319
+ }
1320
+ },
1321
+ "e83bc96f3c4e4084b65ea22a893edea2": {
1322
+ "model_module": "@jupyter-widgets/base",
1323
+ "model_name": "LayoutModel",
1324
+ "model_module_version": "1.2.0",
1325
+ "state": {
1326
+ "_model_module": "@jupyter-widgets/base",
1327
+ "_model_module_version": "1.2.0",
1328
+ "_model_name": "LayoutModel",
1329
+ "_view_count": null,
1330
+ "_view_module": "@jupyter-widgets/base",
1331
+ "_view_module_version": "1.2.0",
1332
+ "_view_name": "LayoutView",
1333
+ "align_content": null,
1334
+ "align_items": null,
1335
+ "align_self": null,
1336
+ "border": null,
1337
+ "bottom": null,
1338
+ "display": null,
1339
+ "flex": null,
1340
+ "flex_flow": null,
1341
+ "grid_area": null,
1342
+ "grid_auto_columns": null,
1343
+ "grid_auto_flow": null,
1344
+ "grid_auto_rows": null,
1345
+ "grid_column": null,
1346
+ "grid_gap": null,
1347
+ "grid_row": null,
1348
+ "grid_template_areas": null,
1349
+ "grid_template_columns": null,
1350
+ "grid_template_rows": null,
1351
+ "height": null,
1352
+ "justify_content": null,
1353
+ "justify_items": null,
1354
+ "left": null,
1355
+ "margin": null,
1356
+ "max_height": null,
1357
+ "max_width": null,
1358
+ "min_height": null,
1359
+ "min_width": null,
1360
+ "object_fit": null,
1361
+ "object_position": null,
1362
+ "order": null,
1363
+ "overflow": null,
1364
+ "overflow_x": null,
1365
+ "overflow_y": null,
1366
+ "padding": null,
1367
+ "right": null,
1368
+ "top": null,
1369
+ "visibility": null,
1370
+ "width": null
1371
+ }
1372
+ },
1373
+ "b55968f802eb4df9b15829ec04e14071": {
1374
+ "model_module": "@jupyter-widgets/controls",
1375
+ "model_name": "DescriptionStyleModel",
1376
+ "model_module_version": "1.5.0",
1377
+ "state": {
1378
+ "_model_module": "@jupyter-widgets/controls",
1379
+ "_model_module_version": "1.5.0",
1380
+ "_model_name": "DescriptionStyleModel",
1381
+ "_view_count": null,
1382
+ "_view_module": "@jupyter-widgets/base",
1383
+ "_view_module_version": "1.2.0",
1384
+ "_view_name": "StyleView",
1385
+ "description_width": ""
1386
+ }
1387
+ },
1388
+ "c1ee8f5c98e74b36b4df462e42ea3e4c": {
1389
+ "model_module": "@jupyter-widgets/controls",
1390
+ "model_name": "HBoxModel",
1391
+ "model_module_version": "1.5.0",
1392
+ "state": {
1393
+ "_dom_classes": [],
1394
+ "_model_module": "@jupyter-widgets/controls",
1395
+ "_model_module_version": "1.5.0",
1396
+ "_model_name": "HBoxModel",
1397
+ "_view_count": null,
1398
+ "_view_module": "@jupyter-widgets/controls",
1399
+ "_view_module_version": "1.5.0",
1400
+ "_view_name": "HBoxView",
1401
+ "box_style": "",
1402
+ "children": [
1403
+ "IPY_MODEL_857bfb8cffe74ba982039164473e85ff",
1404
+ "IPY_MODEL_b191d213711445c0a42cb30efc2e3dce",
1405
+ "IPY_MODEL_b7dbcc4ce0904337a5c6a90502e4778b"
1406
+ ],
1407
+ "layout": "IPY_MODEL_7b45355ace8b402486b25ca6db01e97f"
1408
+ }
1409
+ },
1410
+ "857bfb8cffe74ba982039164473e85ff": {
1411
+ "model_module": "@jupyter-widgets/controls",
1412
+ "model_name": "HTMLModel",
1413
+ "model_module_version": "1.5.0",
1414
+ "state": {
1415
+ "_dom_classes": [],
1416
+ "_model_module": "@jupyter-widgets/controls",
1417
+ "_model_module_version": "1.5.0",
1418
+ "_model_name": "HTMLModel",
1419
+ "_view_count": null,
1420
+ "_view_module": "@jupyter-widgets/controls",
1421
+ "_view_module_version": "1.5.0",
1422
+ "_view_name": "HTMLView",
1423
+ "description": "",
1424
+ "description_tooltip": null,
1425
+ "layout": "IPY_MODEL_af5e9c87620c4f34b568af4ce81743b8",
1426
+ "placeholder": "​",
1427
+ "style": "IPY_MODEL_7d89810046434801a5b7f6c524e37a97",
1428
+ "value": "Downloading: 100%"
1429
+ }
1430
+ },
1431
+ "b191d213711445c0a42cb30efc2e3dce": {
1432
+ "model_module": "@jupyter-widgets/controls",
1433
+ "model_name": "FloatProgressModel",
1434
+ "model_module_version": "1.5.0",
1435
+ "state": {
1436
+ "_dom_classes": [],
1437
+ "_model_module": "@jupyter-widgets/controls",
1438
+ "_model_module_version": "1.5.0",
1439
+ "_model_name": "FloatProgressModel",
1440
+ "_view_count": null,
1441
+ "_view_module": "@jupyter-widgets/controls",
1442
+ "_view_module_version": "1.5.0",
1443
+ "_view_name": "ProgressView",
1444
+ "bar_style": "success",
1445
+ "description": "",
1446
+ "description_tooltip": null,
1447
+ "layout": "IPY_MODEL_570c86720e2941f28a123c4159082a8e",
1448
+ "max": 548118077,
1449
+ "min": 0,
1450
+ "orientation": "horizontal",
1451
+ "style": "IPY_MODEL_db9bac2505e94c658eb695e7d6ccb801",
1452
+ "value": 548118077
1453
+ }
1454
+ },
1455
+ "b7dbcc4ce0904337a5c6a90502e4778b": {
1456
+ "model_module": "@jupyter-widgets/controls",
1457
+ "model_name": "HTMLModel",
1458
+ "model_module_version": "1.5.0",
1459
+ "state": {
1460
+ "_dom_classes": [],
1461
+ "_model_module": "@jupyter-widgets/controls",
1462
+ "_model_module_version": "1.5.0",
1463
+ "_model_name": "HTMLModel",
1464
+ "_view_count": null,
1465
+ "_view_module": "@jupyter-widgets/controls",
1466
+ "_view_module_version": "1.5.0",
1467
+ "_view_name": "HTMLView",
1468
+ "description": "",
1469
+ "description_tooltip": null,
1470
+ "layout": "IPY_MODEL_7ff86423f64c45b8917cca7bdd97473f",
1471
+ "placeholder": "​",
1472
+ "style": "IPY_MODEL_54222b6fb925408bb096372409fb9829",
1473
+ "value": " 548M/548M [00:08&lt;00:00, 58.8MB/s]"
1474
+ }
1475
+ },
1476
+ "7b45355ace8b402486b25ca6db01e97f": {
1477
+ "model_module": "@jupyter-widgets/base",
1478
+ "model_name": "LayoutModel",
1479
+ "model_module_version": "1.2.0",
1480
+ "state": {
1481
+ "_model_module": "@jupyter-widgets/base",
1482
+ "_model_module_version": "1.2.0",
1483
+ "_model_name": "LayoutModel",
1484
+ "_view_count": null,
1485
+ "_view_module": "@jupyter-widgets/base",
1486
+ "_view_module_version": "1.2.0",
1487
+ "_view_name": "LayoutView",
1488
+ "align_content": null,
1489
+ "align_items": null,
1490
+ "align_self": null,
1491
+ "border": null,
1492
+ "bottom": null,
1493
+ "display": null,
1494
+ "flex": null,
1495
+ "flex_flow": null,
1496
+ "grid_area": null,
1497
+ "grid_auto_columns": null,
1498
+ "grid_auto_flow": null,
1499
+ "grid_auto_rows": null,
1500
+ "grid_column": null,
1501
+ "grid_gap": null,
1502
+ "grid_row": null,
1503
+ "grid_template_areas": null,
1504
+ "grid_template_columns": null,
1505
+ "grid_template_rows": null,
1506
+ "height": null,
1507
+ "justify_content": null,
1508
+ "justify_items": null,
1509
+ "left": null,
1510
+ "margin": null,
1511
+ "max_height": null,
1512
+ "max_width": null,
1513
+ "min_height": null,
1514
+ "min_width": null,
1515
+ "object_fit": null,
1516
+ "object_position": null,
1517
+ "order": null,
1518
+ "overflow": null,
1519
+ "overflow_x": null,
1520
+ "overflow_y": null,
1521
+ "padding": null,
1522
+ "right": null,
1523
+ "top": null,
1524
+ "visibility": null,
1525
+ "width": null
1526
+ }
1527
+ },
1528
+ "af5e9c87620c4f34b568af4ce81743b8": {
1529
+ "model_module": "@jupyter-widgets/base",
1530
+ "model_name": "LayoutModel",
1531
+ "model_module_version": "1.2.0",
1532
+ "state": {
1533
+ "_model_module": "@jupyter-widgets/base",
1534
+ "_model_module_version": "1.2.0",
1535
+ "_model_name": "LayoutModel",
1536
+ "_view_count": null,
1537
+ "_view_module": "@jupyter-widgets/base",
1538
+ "_view_module_version": "1.2.0",
1539
+ "_view_name": "LayoutView",
1540
+ "align_content": null,
1541
+ "align_items": null,
1542
+ "align_self": null,
1543
+ "border": null,
1544
+ "bottom": null,
1545
+ "display": null,
1546
+ "flex": null,
1547
+ "flex_flow": null,
1548
+ "grid_area": null,
1549
+ "grid_auto_columns": null,
1550
+ "grid_auto_flow": null,
1551
+ "grid_auto_rows": null,
1552
+ "grid_column": null,
1553
+ "grid_gap": null,
1554
+ "grid_row": null,
1555
+ "grid_template_areas": null,
1556
+ "grid_template_columns": null,
1557
+ "grid_template_rows": null,
1558
+ "height": null,
1559
+ "justify_content": null,
1560
+ "justify_items": null,
1561
+ "left": null,
1562
+ "margin": null,
1563
+ "max_height": null,
1564
+ "max_width": null,
1565
+ "min_height": null,
1566
+ "min_width": null,
1567
+ "object_fit": null,
1568
+ "object_position": null,
1569
+ "order": null,
1570
+ "overflow": null,
1571
+ "overflow_x": null,
1572
+ "overflow_y": null,
1573
+ "padding": null,
1574
+ "right": null,
1575
+ "top": null,
1576
+ "visibility": null,
1577
+ "width": null
1578
+ }
1579
+ },
1580
+ "7d89810046434801a5b7f6c524e37a97": {
1581
+ "model_module": "@jupyter-widgets/controls",
1582
+ "model_name": "DescriptionStyleModel",
1583
+ "model_module_version": "1.5.0",
1584
+ "state": {
1585
+ "_model_module": "@jupyter-widgets/controls",
1586
+ "_model_module_version": "1.5.0",
1587
+ "_model_name": "DescriptionStyleModel",
1588
+ "_view_count": null,
1589
+ "_view_module": "@jupyter-widgets/base",
1590
+ "_view_module_version": "1.2.0",
1591
+ "_view_name": "StyleView",
1592
+ "description_width": ""
1593
+ }
1594
+ },
1595
+ "570c86720e2941f28a123c4159082a8e": {
1596
+ "model_module": "@jupyter-widgets/base",
1597
+ "model_name": "LayoutModel",
1598
+ "model_module_version": "1.2.0",
1599
+ "state": {
1600
+ "_model_module": "@jupyter-widgets/base",
1601
+ "_model_module_version": "1.2.0",
1602
+ "_model_name": "LayoutModel",
1603
+ "_view_count": null,
1604
+ "_view_module": "@jupyter-widgets/base",
1605
+ "_view_module_version": "1.2.0",
1606
+ "_view_name": "LayoutView",
1607
+ "align_content": null,
1608
+ "align_items": null,
1609
+ "align_self": null,
1610
+ "border": null,
1611
+ "bottom": null,
1612
+ "display": null,
1613
+ "flex": null,
1614
+ "flex_flow": null,
1615
+ "grid_area": null,
1616
+ "grid_auto_columns": null,
1617
+ "grid_auto_flow": null,
1618
+ "grid_auto_rows": null,
1619
+ "grid_column": null,
1620
+ "grid_gap": null,
1621
+ "grid_row": null,
1622
+ "grid_template_areas": null,
1623
+ "grid_template_columns": null,
1624
+ "grid_template_rows": null,
1625
+ "height": null,
1626
+ "justify_content": null,
1627
+ "justify_items": null,
1628
+ "left": null,
1629
+ "margin": null,
1630
+ "max_height": null,
1631
+ "max_width": null,
1632
+ "min_height": null,
1633
+ "min_width": null,
1634
+ "object_fit": null,
1635
+ "object_position": null,
1636
+ "order": null,
1637
+ "overflow": null,
1638
+ "overflow_x": null,
1639
+ "overflow_y": null,
1640
+ "padding": null,
1641
+ "right": null,
1642
+ "top": null,
1643
+ "visibility": null,
1644
+ "width": null
1645
+ }
1646
+ },
1647
+ "db9bac2505e94c658eb695e7d6ccb801": {
1648
+ "model_module": "@jupyter-widgets/controls",
1649
+ "model_name": "ProgressStyleModel",
1650
+ "model_module_version": "1.5.0",
1651
+ "state": {
1652
+ "_model_module": "@jupyter-widgets/controls",
1653
+ "_model_module_version": "1.5.0",
1654
+ "_model_name": "ProgressStyleModel",
1655
+ "_view_count": null,
1656
+ "_view_module": "@jupyter-widgets/base",
1657
+ "_view_module_version": "1.2.0",
1658
+ "_view_name": "StyleView",
1659
+ "bar_color": null,
1660
+ "description_width": ""
1661
+ }
1662
+ },
1663
+ "7ff86423f64c45b8917cca7bdd97473f": {
1664
+ "model_module": "@jupyter-widgets/base",
1665
+ "model_name": "LayoutModel",
1666
+ "model_module_version": "1.2.0",
1667
+ "state": {
1668
+ "_model_module": "@jupyter-widgets/base",
1669
+ "_model_module_version": "1.2.0",
1670
+ "_model_name": "LayoutModel",
1671
+ "_view_count": null,
1672
+ "_view_module": "@jupyter-widgets/base",
1673
+ "_view_module_version": "1.2.0",
1674
+ "_view_name": "LayoutView",
1675
+ "align_content": null,
1676
+ "align_items": null,
1677
+ "align_self": null,
1678
+ "border": null,
1679
+ "bottom": null,
1680
+ "display": null,
1681
+ "flex": null,
1682
+ "flex_flow": null,
1683
+ "grid_area": null,
1684
+ "grid_auto_columns": null,
1685
+ "grid_auto_flow": null,
1686
+ "grid_auto_rows": null,
1687
+ "grid_column": null,
1688
+ "grid_gap": null,
1689
+ "grid_row": null,
1690
+ "grid_template_areas": null,
1691
+ "grid_template_columns": null,
1692
+ "grid_template_rows": null,
1693
+ "height": null,
1694
+ "justify_content": null,
1695
+ "justify_items": null,
1696
+ "left": null,
1697
+ "margin": null,
1698
+ "max_height": null,
1699
+ "max_width": null,
1700
+ "min_height": null,
1701
+ "min_width": null,
1702
+ "object_fit": null,
1703
+ "object_position": null,
1704
+ "order": null,
1705
+ "overflow": null,
1706
+ "overflow_x": null,
1707
+ "overflow_y": null,
1708
+ "padding": null,
1709
+ "right": null,
1710
+ "top": null,
1711
+ "visibility": null,
1712
+ "width": null
1713
+ }
1714
+ },
1715
+ "54222b6fb925408bb096372409fb9829": {
1716
+ "model_module": "@jupyter-widgets/controls",
1717
+ "model_name": "DescriptionStyleModel",
1718
+ "model_module_version": "1.5.0",
1719
+ "state": {
1720
+ "_model_module": "@jupyter-widgets/controls",
1721
+ "_model_module_version": "1.5.0",
1722
+ "_model_name": "DescriptionStyleModel",
1723
+ "_view_count": null,
1724
+ "_view_module": "@jupyter-widgets/base",
1725
+ "_view_module_version": "1.2.0",
1726
+ "_view_name": "StyleView",
1727
+ "description_width": ""
1728
+ }
1729
+ }
1730
+ }
1731
+ }
1732
+ },
1733
+ "cells": [
1734
+ {
1735
+ "cell_type": "code",
1736
+ "source": [
1737
+ "!pip install transformers"
1738
+ ],
1739
+ "metadata": {
1740
+ "id": "SyzeJaeuxguB",
1741
+ "colab": {
1742
+ "base_uri": "https://localhost:8080/"
1743
+ },
1744
+ "outputId": "6d0ee6ac-fbf6-41a7-e4ae-d79a520c3f39"
1745
+ },
1746
+ "execution_count": 1,
1747
+ "outputs": [
1748
+ {
1749
+ "output_type": "stream",
1750
+ "name": "stdout",
1751
+ "text": [
1752
+ "Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n",
1753
+ "Collecting transformers\n",
1754
+ " Downloading transformers-4.25.1-py3-none-any.whl (5.8 MB)\n",
1755
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m5.8/5.8 MB\u001b[0m \u001b[31m46.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
1756
+ "\u001b[?25hRequirement already satisfied: filelock in /usr/local/lib/python3.8/dist-packages (from transformers) (3.8.2)\n",
1757
+ "Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.8/dist-packages (from transformers) (6.0)\n",
1758
+ "Collecting tokenizers!=0.11.3,<0.14,>=0.11.1\n",
1759
+ " Downloading tokenizers-0.13.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (7.6 MB)\n",
1760
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m7.6/7.6 MB\u001b[0m \u001b[31m110.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
1761
+ "\u001b[?25hRequirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.8/dist-packages (from transformers) (2022.6.2)\n",
1762
+ "Requirement already satisfied: tqdm>=4.27 in /usr/local/lib/python3.8/dist-packages (from transformers) (4.64.1)\n",
1763
+ "Requirement already satisfied: requests in /usr/local/lib/python3.8/dist-packages (from transformers) (2.25.1)\n",
1764
+ "Requirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.8/dist-packages (from transformers) (1.21.6)\n",
1765
+ "Collecting huggingface-hub<1.0,>=0.10.0\n",
1766
+ " Downloading huggingface_hub-0.11.1-py3-none-any.whl (182 kB)\n",
1767
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━��━━━━━━━━━\u001b[0m \u001b[32m182.4/182.4 KB\u001b[0m \u001b[31m22.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
1768
+ "\u001b[?25hRequirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.8/dist-packages (from transformers) (21.3)\n",
1769
+ "Requirement already satisfied: typing-extensions>=3.7.4.3 in /usr/local/lib/python3.8/dist-packages (from huggingface-hub<1.0,>=0.10.0->transformers) (4.4.0)\n",
1770
+ "Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.8/dist-packages (from packaging>=20.0->transformers) (3.0.9)\n",
1771
+ "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.8/dist-packages (from requests->transformers) (2022.12.7)\n",
1772
+ "Requirement already satisfied: chardet<5,>=3.0.2 in /usr/local/lib/python3.8/dist-packages (from requests->transformers) (4.0.0)\n",
1773
+ "Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.8/dist-packages (from requests->transformers) (2.10)\n",
1774
+ "Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.8/dist-packages (from requests->transformers) (1.24.3)\n",
1775
+ "Installing collected packages: tokenizers, huggingface-hub, transformers\n",
1776
+ "Successfully installed huggingface-hub-0.11.1 tokenizers-0.13.2 transformers-4.25.1\n"
1777
+ ]
1778
+ }
1779
+ ]
1780
+ },
1781
+ {
1782
+ "cell_type": "code",
1783
+ "source": [
1784
+ "import pandas as pd\n",
1785
+ "import numpy as np"
1786
+ ],
1787
+ "metadata": {
1788
+ "id": "IpBTHx_hxLOT"
1789
+ },
1790
+ "execution_count": 2,
1791
+ "outputs": []
1792
+ },
1793
+ {
1794
+ "cell_type": "code",
1795
+ "source": [
1796
+ "full_df = pd.read_csv('cleanedTweetData.csv')"
1797
+ ],
1798
+ "metadata": {
1799
+ "id": "C3DGzGRCxOgN"
1800
+ },
1801
+ "execution_count": 3,
1802
+ "outputs": []
1803
+ },
1804
+ {
1805
+ "cell_type": "markdown",
1806
+ "source": [
1807
+ "To fine tune a GPT-2 model, we only need the tweets"
1808
+ ],
1809
+ "metadata": {
1810
+ "id": "UBnuHPgGxYiQ"
1811
+ }
1812
+ },
1813
+ {
1814
+ "cell_type": "code",
1815
+ "source": [
1816
+ "tweets = full_df['tweet']"
1817
+ ],
1818
+ "metadata": {
1819
+ "id": "j9PYWkqtxbUa"
1820
+ },
1821
+ "execution_count": 4,
1822
+ "outputs": []
1823
+ },
1824
+ {
1825
+ "cell_type": "markdown",
1826
+ "source": [
1827
+ "# Pre-processing the text\n",
1828
+ "\n",
1829
+ "For the model to be able to handle raw data, we need to first prepocess it. We need to keep in mind that we need to prepocess it the same way as the original data was prepocessed when tranining the model.\n",
1830
+ "\n",
1831
+ "To make sure we get the correct tokenizer, we can use the transformers library and import the model and its corresponding tokenizer\n",
1832
+ "\n",
1833
+ "\n",
1834
+ "Add three new tokens in the pre-trained GPT2 tokenizer: \\ <|sos|> : start of sentence \\ <|eos|> : end of sentence \\ <|pad|> : padding token"
1835
+ ],
1836
+ "metadata": {
1837
+ "id": "P70XJeWixo8E"
1838
+ }
1839
+ },
1840
+ {
1841
+ "cell_type": "code",
1842
+ "source": [
1843
+ "from transformers import GPT2Tokenizer\n",
1844
+ "\n",
1845
+ "tokenizer = GPT2Tokenizer.from_pretrained('gpt2-medium',\n",
1846
+ " bos_token='<|sos|>',\n",
1847
+ " eos_token='<|eos|>',\n",
1848
+ " pad_token='<|pad|>')\n",
1849
+ "\n",
1850
+ "#Example of tokenized sample text\n",
1851
+ "tokenizer.encode('Hello World!')"
1852
+ ],
1853
+ "metadata": {
1854
+ "colab": {
1855
+ "base_uri": "https://localhost:8080/",
1856
+ "height": 168,
1857
+ "referenced_widgets": [
1858
+ "9ba818960e884a82a265d9455c3f9846",
1859
+ "bd89b9c900024a799bfaf8f6dfa26bbb",
1860
+ "237748491b834ff3a774a84703bd5ca6",
1861
+ "866bf549da8d4cb0ac03c1b108f4f5c6",
1862
+ "4578787847a8468b899e9e42840b250e",
1863
+ "4dc4e8f371664dbaa5ddef539d7cece3",
1864
+ "a2752dd9fccd41f2a2f5d1c6aa59fec3",
1865
+ "5cc6738e683e474c9a8f1912ee2b7f4d",
1866
+ "fe4b9ccec34d440486f08263444f0606",
1867
+ "5769167aae2149e89087f473b18b799c",
1868
+ "2a42ff88334748e9aa18c0e7d61d0d02",
1869
+ "055803bb341f412d95386eb37b367681",
1870
+ "0036c108105d4b3c86155a15e048ff6b",
1871
+ "58851499d1f0416590cd7ca275c1ddef",
1872
+ "417ce7fdf3f049d8b9d9b49913b56c45",
1873
+ "942710acc5e54bbba743632472cc1399",
1874
+ "d462118b7f114fcb8c3589996f5ab536",
1875
+ "3b25cfad56434cb1a44e5a9853b97573",
1876
+ "a2bc95389bc64e089b3b950327e4302a",
1877
+ "051c868590b641ae9f3adf386beabdee",
1878
+ "24b5d163f535447d8237ec44021cf537",
1879
+ "98e7e8229a5c45678bb8fc50cb56995f",
1880
+ "3f34f1aea3674271a03837ac87a54620",
1881
+ "3e06e5ca1ee04650bf0d442487e76872",
1882
+ "3c6ab591598649b7818409ac15f52c1e",
1883
+ "5f988b61bd3a49e0be57556c6f678fcc",
1884
+ "288be48cc6644e738369b0f1dbbd3365",
1885
+ "88025aee2bd34c5abcc7e1d697fe5729",
1886
+ "1f505ffcfd3b4d4aa4f7f36f7f96edc4",
1887
+ "d60db7a441764f4191f8ea84fb36fffb",
1888
+ "f49eb751667f4e0aab6f06472d5422e2",
1889
+ "29f9630999554f0a8a319e722cf16ccd",
1890
+ "0b56cf47dca143efb2f3f946e681bb52"
1891
+ ]
1892
+ },
1893
+ "id": "JzleK0mRyGMa",
1894
+ "outputId": "cb21f654-9123-404d-ab9f-2272ff30d001"
1895
+ },
1896
+ "execution_count": 5,
1897
+ "outputs": [
1898
+ {
1899
+ "output_type": "display_data",
1900
+ "data": {
1901
+ "text/plain": [
1902
+ "Downloading: 0%| | 0.00/1.04M [00:00<?, ?B/s]"
1903
+ ],
1904
+ "application/vnd.jupyter.widget-view+json": {
1905
+ "version_major": 2,
1906
+ "version_minor": 0,
1907
+ "model_id": "9ba818960e884a82a265d9455c3f9846"
1908
+ }
1909
+ },
1910
+ "metadata": {}
1911
+ },
1912
+ {
1913
+ "output_type": "display_data",
1914
+ "data": {
1915
+ "text/plain": [
1916
+ "Downloading: 0%| | 0.00/456k [00:00<?, ?B/s]"
1917
+ ],
1918
+ "application/vnd.jupyter.widget-view+json": {
1919
+ "version_major": 2,
1920
+ "version_minor": 0,
1921
+ "model_id": "055803bb341f412d95386eb37b367681"
1922
+ }
1923
+ },
1924
+ "metadata": {}
1925
+ },
1926
+ {
1927
+ "output_type": "display_data",
1928
+ "data": {
1929
+ "text/plain": [
1930
+ "Downloading: 0%| | 0.00/718 [00:00<?, ?B/s]"
1931
+ ],
1932
+ "application/vnd.jupyter.widget-view+json": {
1933
+ "version_major": 2,
1934
+ "version_minor": 0,
1935
+ "model_id": "3f34f1aea3674271a03837ac87a54620"
1936
+ }
1937
+ },
1938
+ "metadata": {}
1939
+ },
1940
+ {
1941
+ "output_type": "stream",
1942
+ "name": "stderr",
1943
+ "text": [
1944
+ "Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.\n"
1945
+ ]
1946
+ },
1947
+ {
1948
+ "output_type": "execute_result",
1949
+ "data": {
1950
+ "text/plain": [
1951
+ "[15496, 2159, 0]"
1952
+ ]
1953
+ },
1954
+ "metadata": {},
1955
+ "execution_count": 5
1956
+ }
1957
+ ]
1958
+ },
1959
+ {
1960
+ "cell_type": "markdown",
1961
+ "source": [
1962
+ "We want to find out our longest tweet to know how to know how to set our maximum token length"
1963
+ ],
1964
+ "metadata": {
1965
+ "id": "9qc3q8gByqqo"
1966
+ }
1967
+ },
1968
+ {
1969
+ "cell_type": "code",
1970
+ "source": [
1971
+ "max_tweet = max([len(tokenizer.encode(tweet)) for tweet in tweets])\n",
1972
+ "\n",
1973
+ "print(f'The longest tweet is {max_tweet} tokens long.')"
1974
+ ],
1975
+ "metadata": {
1976
+ "colab": {
1977
+ "base_uri": "https://localhost:8080/"
1978
+ },
1979
+ "id": "ERv9n0ugyqN8",
1980
+ "outputId": "c5233db3-f86b-45c3-971f-030fbc8a32a7"
1981
+ },
1982
+ "execution_count": 6,
1983
+ "outputs": [
1984
+ {
1985
+ "output_type": "stream",
1986
+ "name": "stdout",
1987
+ "text": [
1988
+ "The longest tweet is 158 tokens long.\n"
1989
+ ]
1990
+ }
1991
+ ]
1992
+ },
1993
+ {
1994
+ "cell_type": "markdown",
1995
+ "source": [
1996
+ "# Training GPT-2"
1997
+ ],
1998
+ "metadata": {
1999
+ "id": "djpoLppcy2TJ"
2000
+ }
2001
+ },
2002
+ {
2003
+ "cell_type": "markdown",
2004
+ "source": [
2005
+ "**Training**\n",
2006
+ "\n",
2007
+ "**What is GPT-2 and why do we use it?**\n",
2008
+ "\n",
2009
+ "GPT-2 Is a transformer-based architecture NN that was trained on a massive amount of unlabeled raw text data in a self-supervised fashion in order to predict the next word in a given sentence, and the attempts at using it in a transfer-learning manner have been very successful so far.\n",
2010
+ "\n",
2011
+ "You can use it yourself to create models that do anything from answering questions, generating stories, to mimicing someone on Twitter- which we're going to do here.\n",
2012
+ "Next we create a custom dataloader for our tweets using torch Dataset. \\ Each entry in the dataset will be two tensors, one which is the encoding for the string and one which is the attention mask"
2013
+ ],
2014
+ "metadata": {
2015
+ "id": "zICteBdwy9hB"
2016
+ }
2017
+ },
2018
+ {
2019
+ "cell_type": "code",
2020
+ "source": [
2021
+ "batch_size = 32\n",
2022
+ "\n",
2023
+ "import torch\n",
2024
+ "from torch.utils.data import Dataset\n",
2025
+ "\n",
2026
+ "class TweetDataset(Dataset):\n",
2027
+ " def __init__(self,tweets,tokenizer,gpt2_type=\"gpt2-medium\",max_length=max_tweet):\n",
2028
+ " self.tokenizer = tokenizer\n",
2029
+ " self.input_ids = []\n",
2030
+ " self.attention_masks = []\n",
2031
+ " \n",
2032
+ " for tweet in tweets:\n",
2033
+ " encoding_dict = tokenizer('<|sos|>'+ tweet +'<|eos|>',truncation=True,\n",
2034
+ " max_length=max_length,\n",
2035
+ " padding='max_length')\n",
2036
+ " \n",
2037
+ " self.input_ids.append(torch.tensor(encoding_dict['input_ids']))\n",
2038
+ " self.attention_masks.append(torch.tensor(encoding_dict['attention_mask']))\n",
2039
+ " \n",
2040
+ " def __len__(self):\n",
2041
+ " return len(self.input_ids)\n",
2042
+ " \n",
2043
+ " def __getitem__(self,idx):\n",
2044
+ " return self.input_ids[idx], self.attention_masks[idx]"
2045
+ ],
2046
+ "metadata": {
2047
+ "id": "IMZ03wrbzK7R"
2048
+ },
2049
+ "execution_count": 7,
2050
+ "outputs": []
2051
+ },
2052
+ {
2053
+ "cell_type": "code",
2054
+ "source": [
2055
+ "from torch.utils.data import random_split\n",
2056
+ "\n",
2057
+ "#Note that we set max_length to max_tweet (we created this variable a few cells ago)\n",
2058
+ "dataset = TweetDataset(tweets,tokenizer,max_length=max_tweet)"
2059
+ ],
2060
+ "metadata": {
2061
+ "id": "JjiMWIr7zOk1"
2062
+ },
2063
+ "execution_count": 8,
2064
+ "outputs": []
2065
+ },
2066
+ {
2067
+ "cell_type": "code",
2068
+ "source": [
2069
+ "#Split our data into training and validation set\n",
2070
+ "\n",
2071
+ "train_size = int(0.9 * len(dataset)) #90% train, 10% validation\n",
2072
+ "val_size = len(dataset)-train_size\n",
2073
+ "\n",
2074
+ "train,val = random_split(dataset,[train_size,val_size])\n",
2075
+ "print(f'No of train samples = {train_size} and Number of validation samples = {val_size}')"
2076
+ ],
2077
+ "metadata": {
2078
+ "colab": {
2079
+ "base_uri": "https://localhost:8080/"
2080
+ },
2081
+ "id": "qWT6yt45zZBv",
2082
+ "outputId": "82d23b66-5845-4d88-fc88-55d608fbc78b"
2083
+ },
2084
+ "execution_count": 9,
2085
+ "outputs": [
2086
+ {
2087
+ "output_type": "stream",
2088
+ "name": "stdout",
2089
+ "text": [
2090
+ "No of train samples = 14274 and Number of validation samples = 1586\n"
2091
+ ]
2092
+ }
2093
+ ]
2094
+ },
2095
+ {
2096
+ "cell_type": "code",
2097
+ "source": [
2098
+ "from torch.utils.data import DataLoader, RandomSampler, SequentialSampler\n",
2099
+ "#Use our dataloader to prepare our data\n",
2100
+ "train_dataloader = DataLoader(train,sampler = RandomSampler(train),\n",
2101
+ " batch_size = batch_size)\n",
2102
+ "\n",
2103
+ "val_dataloader = DataLoader(val,sampler = SequentialSampler(val),\n",
2104
+ " batch_size = batch_size)"
2105
+ ],
2106
+ "metadata": {
2107
+ "id": "EYlcKXxPz2K3"
2108
+ },
2109
+ "execution_count": 10,
2110
+ "outputs": []
2111
+ },
2112
+ {
2113
+ "cell_type": "markdown",
2114
+ "source": [
2115
+ "Now we that our data is preprocessed, we load our model from transformers library"
2116
+ ],
2117
+ "metadata": {
2118
+ "id": "Dbx9C5lAz_Of"
2119
+ }
2120
+ },
2121
+ {
2122
+ "cell_type": "code",
2123
+ "source": [
2124
+ "import random\n",
2125
+ "from transformers import GPT2LMHeadModel, GPT2Config"
2126
+ ],
2127
+ "metadata": {
2128
+ "id": "PNcevH-l0DHf"
2129
+ },
2130
+ "execution_count": 11,
2131
+ "outputs": []
2132
+ },
2133
+ {
2134
+ "cell_type": "code",
2135
+ "source": [
2136
+ "configuration = GPT2Config.from_pretrained('gpt2', output_hidden_states=False)\n",
2137
+ "model = GPT2LMHeadModel.from_pretrained(\"gpt2\", config=configuration)\n",
2138
+ "model.resize_token_embeddings(len(tokenizer))\n",
2139
+ "\n",
2140
+ "device = torch.device(\"cuda\")\n",
2141
+ "model.cuda()\n",
2142
+ "\n",
2143
+ "seed_val = 42\n",
2144
+ "\n",
2145
+ "random.seed(seed_val)\n",
2146
+ "np.random.seed(seed_val)\n",
2147
+ "torch.manual_seed(seed_val)\n",
2148
+ "torch.cuda.manual_seed_all(seed_val)"
2149
+ ],
2150
+ "metadata": {
2151
+ "colab": {
2152
+ "base_uri": "https://localhost:8080/",
2153
+ "height": 81,
2154
+ "referenced_widgets": [
2155
+ "d0da26df89d943a48cf782d37c29c994",
2156
+ "9a31634d798c480bb8e294dff5de7795",
2157
+ "41ab5d5f96b74fe0aad1b5f7a0d6c211",
2158
+ "c049b6f4c8d548fb88a4af78994e5aa3",
2159
+ "cb7026bb90d044b98c722b6a7b90d5dd",
2160
+ "b4e3c39c1d564fda8776b8b0bb6a2797",
2161
+ "2d1b7605a31a422a8030ac16e7b0aab3",
2162
+ "cf65384d80f849bfbb7e0b3e54bae664",
2163
+ "fdd9de7b00d0417d8fe8fcfc59119739",
2164
+ "e83bc96f3c4e4084b65ea22a893edea2",
2165
+ "b55968f802eb4df9b15829ec04e14071",
2166
+ "c1ee8f5c98e74b36b4df462e42ea3e4c",
2167
+ "857bfb8cffe74ba982039164473e85ff",
2168
+ "b191d213711445c0a42cb30efc2e3dce",
2169
+ "b7dbcc4ce0904337a5c6a90502e4778b",
2170
+ "7b45355ace8b402486b25ca6db01e97f",
2171
+ "af5e9c87620c4f34b568af4ce81743b8",
2172
+ "7d89810046434801a5b7f6c524e37a97",
2173
+ "570c86720e2941f28a123c4159082a8e",
2174
+ "db9bac2505e94c658eb695e7d6ccb801",
2175
+ "7ff86423f64c45b8917cca7bdd97473f",
2176
+ "54222b6fb925408bb096372409fb9829"
2177
+ ]
2178
+ },
2179
+ "id": "jb-nFiwi0EwL",
2180
+ "outputId": "c9ffeada-1179-4d31-8252-554588c76619"
2181
+ },
2182
+ "execution_count": 12,
2183
+ "outputs": [
2184
+ {
2185
+ "output_type": "display_data",
2186
+ "data": {
2187
+ "text/plain": [
2188
+ "Downloading: 0%| | 0.00/665 [00:00<?, ?B/s]"
2189
+ ],
2190
+ "application/vnd.jupyter.widget-view+json": {
2191
+ "version_major": 2,
2192
+ "version_minor": 0,
2193
+ "model_id": "d0da26df89d943a48cf782d37c29c994"
2194
+ }
2195
+ },
2196
+ "metadata": {}
2197
+ },
2198
+ {
2199
+ "output_type": "display_data",
2200
+ "data": {
2201
+ "text/plain": [
2202
+ "Downloading: 0%| | 0.00/548M [00:00<?, ?B/s]"
2203
+ ],
2204
+ "application/vnd.jupyter.widget-view+json": {
2205
+ "version_major": 2,
2206
+ "version_minor": 0,
2207
+ "model_id": "c1ee8f5c98e74b36b4df462e42ea3e4c"
2208
+ }
2209
+ },
2210
+ "metadata": {}
2211
+ }
2212
+ ]
2213
+ },
2214
+ {
2215
+ "cell_type": "markdown",
2216
+ "source": [
2217
+ "The time for training will depend on the number of samples divided by the batch size, then multiplied by the epochs, I encourage people to expirement with hyperparameters such as batch size, epochs, learning rate, LR scheduling and optimizers, and compare results.\n",
2218
+ "\n",
2219
+ "I recommend doing at least 5-6 if you want good results."
2220
+ ],
2221
+ "metadata": {
2222
+ "id": "tJ2ngVrl0MkM"
2223
+ }
2224
+ },
2225
+ {
2226
+ "cell_type": "code",
2227
+ "source": [
2228
+ "# the warmup steps are steps at the start of training that are ignored\n",
2229
+ "# every x steps we will sample the model to test the output\n",
2230
+ "\n",
2231
+ "epochs = 5\n",
2232
+ "warmup_steps = 1e2\n",
2233
+ "sample_every = 100"
2234
+ ],
2235
+ "metadata": {
2236
+ "id": "Y0cj_0L50V8s"
2237
+ },
2238
+ "execution_count": 13,
2239
+ "outputs": []
2240
+ },
2241
+ {
2242
+ "cell_type": "code",
2243
+ "source": [
2244
+ "from transformers import AdamW\n",
2245
+ "\n",
2246
+ "optimizer = AdamW(model.parameters(),\n",
2247
+ " lr = 5e-4,\n",
2248
+ " eps = 1e-8\n",
2249
+ " )"
2250
+ ],
2251
+ "metadata": {
2252
+ "colab": {
2253
+ "base_uri": "https://localhost:8080/"
2254
+ },
2255
+ "id": "SopBnrZr0b11",
2256
+ "outputId": "7f38502b-acef-4273-c37c-ba51c9ed6d50"
2257
+ },
2258
+ "execution_count": 14,
2259
+ "outputs": [
2260
+ {
2261
+ "output_type": "stream",
2262
+ "name": "stderr",
2263
+ "text": [
2264
+ "/usr/local/lib/python3.8/dist-packages/transformers/optimization.py:306: FutureWarning: This implementation of AdamW is deprecated and will be removed in a future version. Use the PyTorch implementation torch.optim.AdamW instead, or set `no_deprecation_warning=True` to disable this warning\n",
2265
+ " warnings.warn(\n"
2266
+ ]
2267
+ }
2268
+ ]
2269
+ },
2270
+ {
2271
+ "cell_type": "code",
2272
+ "source": [
2273
+ "from transformers import get_linear_schedule_with_warmup\n",
2274
+ "\n",
2275
+ "total_steps = len(train_dataloader) * epochs\n",
2276
+ "scheduler = get_linear_schedule_with_warmup(optimizer, \n",
2277
+ " num_warmup_steps = warmup_steps, \n",
2278
+ " num_training_steps = total_steps)"
2279
+ ],
2280
+ "metadata": {
2281
+ "id": "dr8HPNxo0eBW"
2282
+ },
2283
+ "execution_count": 15,
2284
+ "outputs": []
2285
+ },
2286
+ {
2287
+ "cell_type": "code",
2288
+ "source": [
2289
+ "import random\n",
2290
+ "import time\n",
2291
+ "import datetime\n",
2292
+ "\n",
2293
+ "def format_time(elapsed):\n",
2294
+ " return str(datetime.timedelta(seconds=int(round((elapsed)))))\n",
2295
+ "\n",
2296
+ "total_t0 = time.time()\n",
2297
+ "\n",
2298
+ "training_stats = []\n",
2299
+ "\n",
2300
+ "model = model.to(device)\n",
2301
+ "\n",
2302
+ "for epoch_i in range(0, epochs):\n",
2303
+ "\n",
2304
+ " print(f'Beginning epoch {epoch_i + 1} of {epochs}')\n",
2305
+ "\n",
2306
+ " t0 = time.time()\n",
2307
+ "\n",
2308
+ " total_train_loss = 0\n",
2309
+ "\n",
2310
+ " model.train()\n",
2311
+ "\n",
2312
+ " for step, batch in enumerate(train_dataloader):\n",
2313
+ "\n",
2314
+ " b_input_ids = batch[0].to(device)\n",
2315
+ " b_labels = batch[0].to(device)\n",
2316
+ " b_masks = batch[1].to(device)\n",
2317
+ "\n",
2318
+ " model.zero_grad() \n",
2319
+ "\n",
2320
+ " outputs = model( b_input_ids,\n",
2321
+ " labels=b_labels, \n",
2322
+ " attention_mask = b_masks,\n",
2323
+ " token_type_ids=None\n",
2324
+ " )\n",
2325
+ "\n",
2326
+ " loss = outputs[0] \n",
2327
+ "\n",
2328
+ " batch_loss = loss.item()\n",
2329
+ " total_train_loss += batch_loss\n",
2330
+ "\n",
2331
+ " # Get sample every 100 batches.\n",
2332
+ " if step % sample_every == 0 and not step == 0:\n",
2333
+ "\n",
2334
+ " elapsed = format_time(time.time() - t0)\n",
2335
+ " print(f'Batch {step} of {len(train_dataloader)}. Loss:{batch_loss}. Time:{elapsed}')\n",
2336
+ "\n",
2337
+ " model.eval()\n",
2338
+ "\n",
2339
+ " sample_outputs = model.generate(\n",
2340
+ " bos_token_id=random.randint(1,30000),\n",
2341
+ " do_sample=True, \n",
2342
+ " top_k=50, \n",
2343
+ " max_length = 200,\n",
2344
+ " top_p=0.95, \n",
2345
+ " num_return_sequences=1\n",
2346
+ " )\n",
2347
+ " for i, sample_output in enumerate(sample_outputs):\n",
2348
+ " print(f'Example output: {tokenizer.decode(sample_output, skip_special_tokens=True)}')\n",
2349
+ " \n",
2350
+ " model.train()\n",
2351
+ "\n",
2352
+ " loss.backward()\n",
2353
+ "\n",
2354
+ " optimizer.step()\n",
2355
+ "\n",
2356
+ " scheduler.step()\n",
2357
+ "\n",
2358
+ " # Calculate the average loss over all of the batches.\n",
2359
+ " avg_train_loss = total_train_loss / len(train_dataloader) \n",
2360
+ " \n",
2361
+ " # Measure how long this epoch took.\n",
2362
+ " training_time = format_time(time.time() - t0)\n",
2363
+ "\n",
2364
+ " print(f'Average Training Loss: {avg_train_loss}. Epoch time: {training_time}')\n",
2365
+ "\n",
2366
+ " t0 = time.time()\n",
2367
+ "\n",
2368
+ " model.eval()\n",
2369
+ "\n",
2370
+ " total_eval_loss = 0\n",
2371
+ " nb_eval_steps = 0\n",
2372
+ "\n",
2373
+ " # Evaluate data for one epoch\n",
2374
+ " for batch in val_dataloader:\n",
2375
+ " \n",
2376
+ " b_input_ids = batch[0].to(device)\n",
2377
+ " b_labels = batch[0].to(device)\n",
2378
+ " b_masks = batch[1].to(device)\n",
2379
+ " \n",
2380
+ " with torch.no_grad(): \n",
2381
+ "\n",
2382
+ " outputs = model(b_input_ids, \n",
2383
+ " attention_mask = b_masks,\n",
2384
+ " labels=b_labels)\n",
2385
+ " \n",
2386
+ " loss = outputs[0] \n",
2387
+ " \n",
2388
+ " batch_loss = loss.item()\n",
2389
+ " total_eval_loss += batch_loss \n",
2390
+ "\n",
2391
+ " avg_val_loss = total_eval_loss / len(val_dataloader)\n",
2392
+ " \n",
2393
+ " validation_time = format_time(time.time() - t0) \n",
2394
+ "\n",
2395
+ " print(f'Validation loss: {avg_val_loss}. Validation Time: {validation_time}')\n",
2396
+ "\n",
2397
+ " # Record all statistics from this epoch.\n",
2398
+ " training_stats.append(\n",
2399
+ " {\n",
2400
+ " 'epoch': epoch_i + 1,\n",
2401
+ " 'Training Loss': avg_train_loss,\n",
2402
+ " 'Valid. Loss': avg_val_loss,\n",
2403
+ " 'Training Time': training_time,\n",
2404
+ " 'Validation Time': validation_time\n",
2405
+ " }\n",
2406
+ " )\n",
2407
+ "\n",
2408
+ "print(f'Total training took {format_time(time.time()-total_t0)}')"
2409
+ ],
2410
+ "metadata": {
2411
+ "colab": {
2412
+ "base_uri": "https://localhost:8080/",
2413
+ "height": 519
2414
+ },
2415
+ "id": "zq2tCT260grV",
2416
+ "outputId": "3c4f0bfc-3382-414f-f0bd-fa9757a47f63"
2417
+ },
2418
+ "execution_count": 16,
2419
+ "outputs": [
2420
+ {
2421
+ "output_type": "stream",
2422
+ "name": "stdout",
2423
+ "text": [
2424
+ "Beginning epoch 1 of 5\n"
2425
+ ]
2426
+ },
2427
+ {
2428
+ "output_type": "error",
2429
+ "ename": "OutOfMemoryError",
2430
+ "evalue": "ignored",
2431
+ "traceback": [
2432
+ "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
2433
+ "\u001b[0;31mOutOfMemoryError\u001b[0m Traceback (most recent call last)",
2434
+ "\u001b[0;32m<ipython-input-16-824a5b9b7350>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m 62\u001b[0m \u001b[0mmodel\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtrain\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 63\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 64\u001b[0;31m \u001b[0mloss\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mbackward\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 65\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 66\u001b[0m \u001b[0moptimizer\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mstep\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
2435
+ "\u001b[0;32m/usr/local/lib/python3.8/dist-packages/torch/_tensor.py\u001b[0m in \u001b[0;36mbackward\u001b[0;34m(self, gradient, retain_graph, create_graph, inputs)\u001b[0m\n\u001b[1;32m 485\u001b[0m \u001b[0minputs\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0minputs\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 486\u001b[0m )\n\u001b[0;32m--> 487\u001b[0;31m torch.autograd.backward(\n\u001b[0m\u001b[1;32m 488\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mgradient\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mretain_graph\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcreate_graph\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0minputs\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0minputs\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 489\u001b[0m )\n",
2436
+ "\u001b[0;32m/usr/local/lib/python3.8/dist-packages/torch/autograd/__init__.py\u001b[0m in \u001b[0;36mbackward\u001b[0;34m(tensors, grad_tensors, retain_graph, create_graph, grad_variables, inputs)\u001b[0m\n\u001b[1;32m 195\u001b[0m \u001b[0;31m# some Python versions print out the first line of a multi-line function\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 196\u001b[0m \u001b[0;31m# calls in the traceback and some print out the last line\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 197\u001b[0;31m Variable._execution_engine.run_backward( # Calls into the C++ engine to run the backward pass\n\u001b[0m\u001b[1;32m 198\u001b[0m \u001b[0mtensors\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mgrad_tensors_\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mretain_graph\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcreate_graph\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0minputs\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 199\u001b[0m allow_unreachable=True, accumulate_grad=True) # Calls into the C++ engine to run the backward pass\n",
2437
+ "\u001b[0;31mOutOfMemoryError\u001b[0m: CUDA out of memory. Tried to allocate 964.00 MiB (GPU 0; 14.76 GiB total capacity; 12.69 GiB already allocated; 151.75 MiB free; 13.82 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF"
2438
+ ]
2439
+ }
2440
+ ]
2441
+ },
2442
+ {
2443
+ "cell_type": "markdown",
2444
+ "source": [],
2445
+ "metadata": {
2446
+ "id": "KlmSdI7U17Wk"
2447
+ }
2448
+ }
2449
+ ]
2450
+ }
vocab.json ADDED
The diff for this file is too large to render. See raw diff