ShawnLJW commited on
Commit
d234444
·
1 Parent(s): b2718b8

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +6 -6
README.md CHANGED
@@ -4,14 +4,14 @@ language:
4
  pipeline_tag: text-generation
5
  ---
6
 
7
- # harry-GPoTter
8
 
9
- harry-GPoTter is a transformer text generation model implemented in PyTorch. It has been trained on text from all 7 books from from all 7 books of the Harry Potter series. In only 10 minutes of training with the free tier of [Google Colaboratory](https://colab.research.google.com/), the model learnt to generate coherent and grammatically correct sentences.
10
 
11
- - Code and more information in the [GitHub Repository](https://github.com/ShawnLJW/harry-GPoTter)
12
- - Download the [weights](https://huggingface.co/ShawnLJW/harry-GPoTter/resolve/main/checkpoint.pt)
13
 
14
- ## Text Generation with harry-GPoTter
15
  > “Ah,” said Mrs. Weasley, hiscolored lips looking unpleasant. “He wasn’t talking about her, he has tried to think he was saying he had looked up. The bleers were flooding.”
16
  >
17
  > “My master died?” whispered Voldemort, but the wasnoddenbling until he are, making to be seeing him.
@@ -24,7 +24,7 @@ harry-GPoTter is a transformer text generation model implemented in PyTorch. It
24
 
25
  ## Model Details
26
 
27
- harry-GPoTter is a relatively small language model with 56M parameters (less than 1/2x of smallest gpt-2). It contains 8 layers of 8 headed attention with a hidden size of 384. It supports a maximum sequence length of 128. For tokenization, we use the same tokenizer as text-davinci-003, which has a vocabulary of 50,280 in total.
28
 
29
  The model was trained for 2000 epochs in about 10 minutes with the free tier of Google Colab GPU Runtime. It achieves a cross-entropy loss of 3.1189.
30
 
 
4
  pipeline_tag: text-generation
5
  ---
6
 
7
+ # harry-GPTter
8
 
9
+ harry-GPTter is a transformer text generation model implemented in PyTorch. It has been trained on text from all 7 books from from all 7 books of the Harry Potter series. In only 10 minutes of training with the free tier of [Google Colaboratory](https://colab.research.google.com/), the model learnt to generate coherent and grammatically correct sentences.
10
 
11
+ - Code and more information in the [GitHub Repository](https://github.com/ShawnLJW/harry-GPTter)
12
+ - Download the [weights](https://huggingface.co/ShawnLJW/harry-GPTter/resolve/main/checkpoint.pt)
13
 
14
+ ## Text Generation with harry-GPTter
15
  > “Ah,” said Mrs. Weasley, hiscolored lips looking unpleasant. “He wasn’t talking about her, he has tried to think he was saying he had looked up. The bleers were flooding.”
16
  >
17
  > “My master died?” whispered Voldemort, but the wasnoddenbling until he are, making to be seeing him.
 
24
 
25
  ## Model Details
26
 
27
+ harry-GPTter is a relatively small language model with 56M parameters (less than 1/2x of smallest gpt-2). It contains 8 layers of 8 headed attention with a hidden size of 384. It supports a maximum sequence length of 128. For tokenization, we use the same tokenizer as text-davinci-003, which has a vocabulary of 50,280 in total.
28
 
29
  The model was trained for 2000 epochs in about 10 minutes with the free tier of Google Colab GPU Runtime. It achieves a cross-entropy loss of 3.1189.
30