julien-c HF staff commited on
Commit
7294375
1 Parent(s): 7e58921

Migrate model card from transformers-repo

Browse files

Read announcement at https://discuss.huggingface.co/t/announcement-all-model-cards-will-be-migrated-to-hf-co-model-repos/2755
Original file history: https://github.com/huggingface/transformers/commits/master/model_cards/stevhliu/astroGPT/README.md

Files changed (1) hide show
  1. README.md +51 -0
README.md ADDED
@@ -0,0 +1,51 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language: "en"
3
+ thumbnail: "https://raw.githubusercontent.com/stevhliu/satsuma/master/images/astroGPT-thumbnail.png"
4
+ widget:
5
+ - text: "Jan 18, 2020"
6
+ - text: "Feb 14, 2020"
7
+ - text: "Jul 04, 2020"
8
+ ---
9
+
10
+ # astroGPT 🪐
11
+
12
+ ## Model description
13
+
14
+ This is a GPT-2 model fine-tuned on Western zodiac signs. For more information about GPT-2, take a look at 🤗 Hugging Face's GPT-2 [model card](https://huggingface.co/gpt2). You can use astroGPT to generate a daily horoscope by entering the current date.
15
+
16
+ ## How to use
17
+
18
+ To use this model, simply enter the current date like so `Mon DD, YEAR`:
19
+
20
+ ```python
21
+ from transformers import AutoTokenizer, AutoModelWithLMHead
22
+
23
+ tokenizer = AutoTokenizer.from_pretrained("stevhliu/astroGPT")
24
+ model = AutoModelWithLMHead.from_pretrained("stevhliu/astroGPT")
25
+
26
+ input_ids = tokenizer.encode('Sep 03, 2020', return_tensors='pt').to('cuda')
27
+
28
+ sample_output = model.generate(input_ids,
29
+ do_sample=True,
30
+ max_length=75,
31
+ top_k=20,
32
+ top_p=0.97)
33
+
34
+ print(sample_output)
35
+ ```
36
+
37
+ ## Limitations and bias
38
+
39
+ astroGPT inherits the same biases that affect GPT-2 as a result of training on a lot of non-neutral content on the internet. The model does not currently support zodiac sign-specific generation and only returns a general horoscope. While the generated text may occasionally mention a specific zodiac sign, this is due to how the horoscopes were originally written by it's human authors.
40
+
41
+ ## Data
42
+
43
+ The data was scraped from [Horoscope.com](https://www.horoscope.com/us/index.aspx) and trained on 4.7MB of text. The text was collected from four categories (daily, love, wellness, career) and span from 09/01/19 to 08/01/2020. The archives only store horoscopes dating a year back from the current date.
44
+
45
+ ## Training and results
46
+
47
+ The text was tokenized using the fast GPT-2 BPE [tokenizer](https://huggingface.co/transformers/model_doc/gpt2.html#gpt2tokenizerfast). It has a vocabulary size of 50,257 and sequence length of 1024 tokens. The model was trained with on one of Google Colaboratory's GPU's for approximately 2.5 hrs with [fastai's](https://docs.fast.ai/) learning rate finder, discriminative learning rates and 1cycle policy. See table below for a quick summary of the training procedure and results.
48
+
49
+ | dataset size | epochs | lr | training time | train_loss | valid_loss | perplexity |
50
+ |:-------------:|:------:|:-----------------:|:-------------:|:----------:|:----------:|:----------:|
51
+ | 5.9MB |32 | slice(1e-7,1e-5) | 2.5 hrs | 2.657170 | 2.642387 | 14.046692 |