KennethTM commited on
Commit
2649b38
1 Parent(s): eebe77d

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +25 -3
README.md CHANGED
@@ -9,17 +9,39 @@ widget:
9
 
10
  # What is this?
11
 
12
- GPT-2 model (small version, 124 M parameters) for Danish text generation.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
13
 
14
  # Model training
15
 
16
  The model is trained using the Danish part of the [oscar dataset](https://huggingface.co/datasets/oscar) ('unshuffled_deduplicated_da') and a context length of 1024 tokens.
17
 
18
- The model is initilized from the English [GPT-2 small model](https://huggingface.co/gpt2) with new word token embeddings created for Danish using [WECHSEL](https://github.com/CPJKU/wechsel).
19
 
20
  Initially, only the word token embeddings are trained using 50.000 samples. Finally, the whole model is trained using 1.000.000 samples.
21
 
22
- Model training is carried out on a 8 GB GPU.
23
 
24
  # Notes
25
 
 
9
 
10
  # What is this?
11
 
12
+ A GPT-2 model (small version, 124 M parameters) for Danish text generation. The model was not pre-trained from scratch but adapted from the English version.
13
+
14
+ # How to use
15
+
16
+ Test the model using the pipeline from the [🤗 Transformers](https://github.com/huggingface/transformers) library:
17
+
18
+ ```python
19
+ from transformers import pipeline
20
+
21
+ generator = pipeline("text-generation", model="gpt2-small-danish")
22
+ text = generator("Manden arbejdede som")
23
+
24
+ print(text[0]["generated_text"])
25
+ ```
26
+
27
+ Or load it using the Auto* classes:
28
+
29
+ ```python
30
+ from transformers import AutoTokenizer, AutoModelForCausalLM
31
+
32
+ tokenizer = AutoTokenizer.from_pretrained("KennethTM/gpt2-small-danish")
33
+ model = AutoModelForCausalLM.from_pretrained("KennethTM/gpt2-small-danish")
34
+ ```
35
 
36
  # Model training
37
 
38
  The model is trained using the Danish part of the [oscar dataset](https://huggingface.co/datasets/oscar) ('unshuffled_deduplicated_da') and a context length of 1024 tokens.
39
 
40
+ The model is initialized from the English [GPT-2 small model](https://huggingface.co/gpt2) with new word token embeddings created for Danish using [WECHSEL](https://github.com/CPJKU/wechsel).
41
 
42
  Initially, only the word token embeddings are trained using 50.000 samples. Finally, the whole model is trained using 1.000.000 samples.
43
 
44
+ Model training is carried out on an 8 GB GPU.
45
 
46
  # Notes
47