DarwinAnim8or commited on
Commit
07029f8
1 Parent(s): cd2865e

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +60 -0
README.md CHANGED
@@ -1,3 +1,63 @@
1
  ---
2
  license: mit
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: mit
3
+ datasets:
4
+ - DarwinAnim8or/greentext
5
+ language:
6
+ - en
7
+ tags:
8
+ - fun
9
+ - greentext
10
+ widget:
11
+ - text: ">be me"
12
+ example_title: "be me"
13
+ - text: ">be zoo keeper"
14
+ co2_eq_emissions:
15
+ emissions: 30
16
+ source: "https://mlco2.github.io/impact/#compute"
17
+ training_type: "fine-tuning"
18
+ geographical_location: "Oregon, USA"
19
+ hardware_used: "1x T4, Google Colab"
20
  ---
21
+
22
+ # GPT-Greentext-1.5b
23
+ A finetuned version of [GPT2-XL](https://huggingface.co/gpt2-xl) on the 'greentext' dataset.
24
+ A demo is available [here](https://huggingface.co/spaces/DarwinAnim8or/GPT-Greentext-Playground)
25
+ The demo playground is recommended over the inference box on the right.
26
+
27
+ This is the largest release of the "GPT-Greentext" model series. The other models can be found here:
28
+ * [355m size model](https://huggingface.co/DarwinAnim8or/GPT-Greentext-355m)
29
+ * [125m size model](https://huggingface.co/DarwinAnim8or/GPT-Greentext-125m)
30
+
31
+ # Training Procedure
32
+ This was trained on the 'greentext' dataset, on Google Colab.
33
+ This model was trained for 1 epoch with learning rate 1e-2.
34
+ Notably this uses the "prompt" and "completion" style jsonl file, rather than the plain text file found in the greentext dataset.
35
+ This nets somewhat better, mostly more consistent results.
36
+
37
+ # Biases & Limitations
38
+ This likely contains the same biases and limitations as the original GPT2 that it is based on, and additionally heavy biases from the greentext dataset.
39
+ It should be noted that offensive or not PG-output is definitely possible and likely will happen.
40
+
41
+ # Intended Use
42
+ This model is meant for fun, nothing else.
43
+
44
+ # Noteworthy differences between this model and the others
45
+ This model tends to like no_repeat_ngram_size values of 1 or 2; whereas the other models in this series tend to prefer 3.
46
+
47
+ # Sample Use
48
+ ```python
49
+ #Import model:
50
+ from happytransformer import HappyGeneration
51
+ happy_gen = HappyGeneration("GPT2", "DarwinAnim8or/GPT-Greentext-1.5b")
52
+
53
+ #Set generation settings:
54
+ from happytransformer import GENSettings
55
+ args_top_k = GENSettingsGENSettings(no_repeat_ngram_size=1, do_sample=True, top_k=80, temperature=0.8, max_length=150, early_stopping=False)
56
+
57
+ #Generate a response:
58
+ result = happy_gen.generate_text(""">be me
59
+ >""", args=args_top_k)
60
+
61
+ print(result)
62
+ print(result.text)
63
+ ```