boris commited on
Commit
7d01111
1 Parent(s): d03b45b

New model from https://wandb.ai/wandb/huggingtweets/runs/2sprqk7j

Browse files
README.md CHANGED
@@ -1,18 +1,27 @@
1
  ---
2
  language: en
3
- thumbnail: https://www.huggingtweets.com/realdonaldtrump/1609765003661/predictions.png
4
  tags:
5
  - huggingtweets
6
  widget:
7
  - text: "My dream is"
8
  ---
9
 
10
-
11
- <div>
12
- <div style="width: 132px; height:132px; border-radius: 50%; background-size: cover; background-image: url('https://pbs.twimg.com/profile_images/874276197357596672/kUuht00m_400x400.jpg')">
13
- </div>
14
- <div style="margin-top: 8px; font-size: 19px; font-weight: 800">Donald J. Trump 🤖 AI Bot </div>
15
- <div style="font-size: 15px">@realdonaldtrump bot</div>
 
 
 
 
 
 
 
 
 
16
  </div>
17
 
18
  I was made with [huggingtweets](https://github.com/borisdayma/huggingtweets).
@@ -25,28 +34,28 @@ The model uses the following pipeline.
25
 
26
  ![pipeline](https://github.com/borisdayma/huggingtweets/blob/master/img/pipeline.png?raw=true)
27
 
28
- To understand how the model was developed, check the [W&B report](https://app.wandb.ai/wandb/huggingtweets/reports/HuggingTweets-Train-a-model-to-generate-tweets--VmlldzoxMTY5MjI).
29
 
30
  ## Training data
31
 
32
- The model was trained on [@realdonaldtrump's tweets](https://twitter.com/realdonaldtrump).
33
 
34
- | Data | Quantity |
35
- | ---- | -------- |
36
- | Tweets downloaded | 2195 |
37
- | Retweets | 716 |
38
- | Short tweets | 364 |
39
- | Tweets kept | 1115 |
40
 
41
- [Explore the data](https://wandb.ai/wandb/huggingtweets/runs/1zmvkb6j/artifacts), which is tracked with [W&B artifacts](https://docs.wandb.com/artifacts) at every step of the pipeline.
42
 
43
  ## Training procedure
44
 
45
  The model is based on a pre-trained [GPT-2](https://huggingface.co/gpt2) which is fine-tuned on @realdonaldtrump's tweets.
46
 
47
- Hyperparameters and metrics are recorded in the [W&B training run](https://wandb.ai/wandb/huggingtweets/runs/1lh3uc7o) for full transparency and reproducibility.
48
 
49
- At the end of training, [the final model](https://wandb.ai/wandb/huggingtweets/runs/1lh3uc7o/artifacts) is logged and versioned.
50
 
51
  ## How to use
52
 
@@ -59,7 +68,6 @@ generator = pipeline('text-generation',
59
  generator("My dream is", num_return_sequences=5)
60
  ```
61
 
62
-
63
  ## Limitations and bias
64
 
65
  The model suffers from [the same limitations and bias as GPT-2](https://huggingface.co/gpt2#limitations-and-bias).
 
1
  ---
2
  language: en
3
+ thumbnail: https://github.com/borisdayma/huggingtweets/blob/master/img/logo.png?raw=true
4
  tags:
5
  - huggingtweets
6
  widget:
7
  - text: "My dream is"
8
  ---
9
 
10
+ <div class="inline-flex flex-col" style="line-height: 1.5;">
11
+ <div class="flex">
12
+ <div
13
+ style="display:inherit; margin-left: 4px; margin-right: 4px; width: 92px; height:92px; border-radius: 50%; background-size: cover; background-image: url(&#39;https://pbs.twimg.com/profile_images/874276197357596672/kUuht00m_400x400.jpg&#39;)">
14
+ </div>
15
+ <div
16
+ style="display:none; margin-left: 4px; margin-right: 4px; width: 92px; height:92px; border-radius: 50%; background-size: cover; background-image: url(&#39;&#39;)">
17
+ </div>
18
+ <div
19
+ style="display:none; margin-left: 4px; margin-right: 4px; width: 92px; height:92px; border-radius: 50%; background-size: cover; background-image: url(&#39;&#39;)">
20
+ </div>
21
+ </div>
22
+ <div style="text-align: center; margin-top: 3px; font-size: 16px; font-weight: 800">🤖 AI BOT 🤖</div>
23
+ <div style="text-align: center; font-size: 16px; font-weight: 800">Donald J. Trump</div>
24
+ <div style="text-align: center; font-size: 14px;">@realdonaldtrump</div>
25
  </div>
26
 
27
  I was made with [huggingtweets](https://github.com/borisdayma/huggingtweets).
 
34
 
35
  ![pipeline](https://github.com/borisdayma/huggingtweets/blob/master/img/pipeline.png?raw=true)
36
 
37
+ To understand how the model was developed, check the [W&B report](https://wandb.ai/wandb/huggingtweets/reports/HuggingTweets-Train-a-Model-to-Generate-Tweets--VmlldzoxMTY5MjI).
38
 
39
  ## Training data
40
 
41
+ The model was trained on tweets from Donald J. Trump.
42
 
43
+ | Data | Donald J. Trump |
44
+ | --- | --- |
45
+ | Tweets downloaded | 3050 |
46
+ | Retweets | 958 |
47
+ | Short tweets | 518 |
48
+ | Tweets kept | 1574 |
49
 
50
+ [Explore the data](https://wandb.ai/wandb/huggingtweets/runs/1o4eds23/artifacts), which is tracked with [W&B artifacts](https://docs.wandb.com/artifacts) at every step of the pipeline.
51
 
52
  ## Training procedure
53
 
54
  The model is based on a pre-trained [GPT-2](https://huggingface.co/gpt2) which is fine-tuned on @realdonaldtrump's tweets.
55
 
56
+ Hyperparameters and metrics are recorded in the [W&B training run](https://wandb.ai/wandb/huggingtweets/runs/2sprqk7j) for full transparency and reproducibility.
57
 
58
+ At the end of training, [the final model](https://wandb.ai/wandb/huggingtweets/runs/2sprqk7j/artifacts) is logged and versioned.
59
 
60
  ## How to use
61
 
 
68
  generator("My dream is", num_return_sequences=5)
69
  ```
70
 
 
71
  ## Limitations and bias
72
 
73
  The model suffers from [the same limitations and bias as GPT-2](https://huggingface.co/gpt2#limitations-and-bias).
config.json CHANGED
@@ -8,7 +8,6 @@
8
  "bos_token_id": 50256,
9
  "embd_pdrop": 0.1,
10
  "eos_token_id": 50256,
11
- "gradient_checkpointing": false,
12
  "initializer_range": 0.02,
13
  "layer_norm_epsilon": 1e-05,
14
  "model_type": "gpt2",
@@ -18,7 +17,10 @@
18
  "n_inner": null,
19
  "n_layer": 12,
20
  "n_positions": 1024,
 
21
  "resid_pdrop": 0.1,
 
 
22
  "summary_activation": null,
23
  "summary_first_dropout": 0.1,
24
  "summary_proj_to_labels": true,
@@ -34,6 +36,8 @@
34
  "top_p": 0.95
35
  }
36
  },
 
 
37
  "use_cache": true,
38
  "vocab_size": 50257
39
  }
 
8
  "bos_token_id": 50256,
9
  "embd_pdrop": 0.1,
10
  "eos_token_id": 50256,
 
11
  "initializer_range": 0.02,
12
  "layer_norm_epsilon": 1e-05,
13
  "model_type": "gpt2",
 
17
  "n_inner": null,
18
  "n_layer": 12,
19
  "n_positions": 1024,
20
+ "reorder_and_upcast_attn": false,
21
  "resid_pdrop": 0.1,
22
+ "scale_attn_by_inverse_layer_idx": false,
23
+ "scale_attn_weights": true,
24
  "summary_activation": null,
25
  "summary_first_dropout": 0.1,
26
  "summary_proj_to_labels": true,
 
36
  "top_p": 0.95
37
  }
38
  },
39
+ "torch_dtype": "float32",
40
+ "transformers_version": "4.25.1",
41
  "use_cache": true,
42
  "vocab_size": 50257
43
  }
flax_model.msgpack DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:c729969b575a502683854b6dd38958eebd11e44eb07bda22c14f21b8716964c7
3
- size 497764120
 
 
 
 
pytorch_model.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:292d778797128f03c7d3deb3e17ae78fad5c1f5b3736a6b62a3c4d07b9695949
3
- size 510406559
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:192068bd810eedba8d5ac98b0ce98f6df3ec11f544ded06b67a1bbbdbbbaa54f
3
+ size 510396521
special_tokens_map.json CHANGED
@@ -1 +1,5 @@
1
- {"bos_token": "<|endoftext|>", "eos_token": "<|endoftext|>", "unk_token": "<|endoftext|>"}
 
 
 
 
 
1
+ {
2
+ "bos_token": "<|endoftext|>",
3
+ "eos_token": "<|endoftext|>",
4
+ "unk_token": "<|endoftext|>"
5
+ }
tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
tokenizer_config.json CHANGED
@@ -1 +1,10 @@
1
- {"unk_token": "<|endoftext|>", "bos_token": "<|endoftext|>", "eos_token": "<|endoftext|>", "add_prefix_space": false, "model_max_length": 1024, "name_or_path": "gpt2"}
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "add_prefix_space": false,
3
+ "bos_token": "<|endoftext|>",
4
+ "eos_token": "<|endoftext|>",
5
+ "model_max_length": 1024,
6
+ "name_or_path": "gpt2",
7
+ "special_tokens_map_file": null,
8
+ "tokenizer_class": "GPT2Tokenizer",
9
+ "unk_token": "<|endoftext|>"
10
+ }
training_args.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:de9b5fca9d63f6778840f0dadf867ee473b6101210722c371d46aca947f3398a
3
- size 1903
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a783d6f779c0d34a0141100740d46bd072e34b1348d326efbcdc375c0ed5e993
3
+ size 3439