BenasSabalys
commited on
Commit
•
ba2084d
1
Parent(s):
b0e2b86
Update README.md
Browse files
README.md
CHANGED
@@ -11,20 +11,17 @@ probably proofread and complete it, then remove this comment. -->
|
|
11 |
|
12 |
## Model description
|
13 |
|
14 |
-
This is a gpt2 model trained on 142 612 different Lithuanian Wikipedia articles
|
15 |
|
16 |
## Intended uses & limitations
|
17 |
|
18 |
-
|
19 |
-
|
20 |
-
## Training and evaluation data
|
21 |
-
|
22 |
-
Will be updated
|
23 |
|
24 |
## Training procedure
|
25 |
Will be updated
|
|
|
26 |
### Training hyperparameters
|
27 |
-
|
28 |
The following hyperparameters were used during training:
|
29 |
|
30 |
- optimizer: None
|
@@ -32,7 +29,7 @@ The following hyperparameters were used during training:
|
|
32 |
|
33 |
### Training results
|
34 |
|
35 |
-
|
36 |
|
37 |
### Framework versions
|
38 |
|
@@ -40,3 +37,28 @@ Will be updated
|
|
40 |
TensorFlow 2.4.1
|
41 |
Tokenizers 0.12.1
|
42 |
Torch 1.4.0
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
11 |
|
12 |
## Model description
|
13 |
|
14 |
+
This is a gpt2 model trained on 142 612 different Lithuanian Wikipedia articles + 11 405 articles taken from delfi.lt, ve.lt and www.respublika.lt portals.
|
15 |
|
16 |
## Intended uses & limitations
|
17 |
|
18 |
+
This is a model I trained when writing my bachelors. You can use it anywhere you want.
|
|
|
|
|
|
|
|
|
19 |
|
20 |
## Training procedure
|
21 |
Will be updated
|
22 |
+
|
23 |
### Training hyperparameters
|
24 |
+
|
25 |
The following hyperparameters were used during training:
|
26 |
|
27 |
- optimizer: None
|
|
|
29 |
|
30 |
### Training results
|
31 |
|
32 |
+
Model reached 36.83% accuracy with training data and 37.02% with validation data
|
33 |
|
34 |
### Framework versions
|
35 |
|
|
|
37 |
TensorFlow 2.4.1
|
38 |
Tokenizers 0.12.1
|
39 |
Torch 1.4.0
|
40 |
+
|
41 |
+
How to use it:
|
42 |
+
|
43 |
+
import tensorflow as tf
|
44 |
+
from transformers import WEIGHTS_NAME, CONFIG_NAME
|
45 |
+
from transformers import GPT2Config, TFGPT2LMHeadModel, GPT2Tokenizer
|
46 |
+
import os
|
47 |
+
output_dir = '...' #local file or link to this page
|
48 |
+
tokenizer = GPT2Tokenizer.from_pretrained(output_dir)
|
49 |
+
model = TFGPT2LMHeadModel.from_pretrained(output_dir)
|
50 |
+
|
51 |
+
text = "Siekdamas"
|
52 |
+
# encoding the input text
|
53 |
+
input_ids = tokenizer.encode(text, return_tensors='tf')
|
54 |
+
# getting out output
|
55 |
+
beam_outputs = model.generate(
|
56 |
+
input_ids,
|
57 |
+
max_length = 150,
|
58 |
+
num_beams = 5,
|
59 |
+
temperature = 0.7,
|
60 |
+
no_repeat_ngram_size=2,
|
61 |
+
num_return_sequences=5
|
62 |
+
)
|
63 |
+
|
64 |
+
print(tokenizer.decode(beam_outputs[0]))
|