kkirchheim
commited on
Commit
•
3238a54
1
Parent(s):
f8b7775
Update README.md
Browse files
README.md
CHANGED
@@ -6,14 +6,8 @@ library_name: transformers
|
|
6 |
---
|
7 |
# German GPT-2 Medium (355M Parameters)
|
8 |
|
9 |
-
This model is a German variant of GPT-2 with approximately 355 million parameters and
|
10 |
-
It is pre-trained on 300 GB of German text data and is intended as a base model for text generation tasks in
|
11 |
-
|
12 |
-
- **Model Architecture:** GPT-2 medium architecture adapted for the German language with an extended context window.
|
13 |
-
- **Languages Supported:** German
|
14 |
-
- **Intended Use Cases:** The model is designed for downstream tasks involving text generation in German, such as language modeling and text completion
|
15 |
-
|
16 |
-
## Model Details
|
17 |
|
18 |
- **Version:** 1.0 (Initial and likely final release)
|
19 |
- **Model Type:** Pre-trained language model
|
@@ -47,8 +41,6 @@ The model is trained on a large-scale German text corpus derived from Common Cra
|
|
47 |
|
48 |
## Ethical Considerations and Bias
|
49 |
|
50 |
-
**Disclaimer:**
|
51 |
-
|
52 |
The presented and trained language model is for **research purposes only**. The GC4 corpus—used for training—contains crawled texts from the internet. Thus, this GPT-2 model can be considered as highly biased, potentially encoding stereotypical associations along gender, race, ethnicity, and disability status. Before using and working with the released checkpoints, it is highly recommended to read:
|
53 |
|
54 |
- **"On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?"** by Emily M. Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell.
|
@@ -71,9 +63,8 @@ The presented and trained language model is for **research purposes only**. The
|
|
71 |
```python
|
72 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
73 |
|
74 |
-
model_name = "your-model-identifier" # Replace with your model's identifier
|
75 |
tokenizer = AutoTokenizer.from_pretrained("kkirchheim/german-gpt2-medium")
|
76 |
-
model = AutoModelForCausalLM.from_pretrained(
|
77 |
```
|
78 |
|
79 |
**Code Example:**
|
@@ -85,7 +76,7 @@ model_name = "kkirchheim/german-gpt2-medium" # Replace with your model's identi
|
|
85 |
|
86 |
generator = pipeline('text-generation', model=model_name, tokenizer=tokenizer)
|
87 |
|
88 |
-
prompt = "
|
89 |
outputs = generator(prompt, max_length=50, num_return_sequences=1)
|
90 |
|
91 |
print(outputs[0]['generated_text'])
|
@@ -98,10 +89,11 @@ print(outputs[0]['generated_text'])
|
|
98 |
- **Third-Party Resources:**
|
99 |
- Tokenizer and initial model architecture from [stefan-it/german-gpt2-larger](https://huggingface.co/stefan-it/german-gpt2-larger).
|
100 |
|
101 |
-
|
102 |
---
|
103 |
|
104 |
-
**Disclaimer:** This model is provided for **research purposes only** and comes with no warranties.
|
|
|
|
|
105 |
|
106 |
|
107 |
**Changelog:**
|
|
|
6 |
---
|
7 |
# German GPT-2 Medium (355M Parameters)
|
8 |
|
9 |
+
This model is a German-only variant of GPT-2 with approximately 355 million parameters and an extended context size of 2048 tokens.
|
10 |
+
It is pre-trained on 300 GB of German text data and is intended as a base model for text generation tasks in German.
|
|
|
|
|
|
|
|
|
|
|
|
|
11 |
|
12 |
- **Version:** 1.0 (Initial and likely final release)
|
13 |
- **Model Type:** Pre-trained language model
|
|
|
41 |
|
42 |
## Ethical Considerations and Bias
|
43 |
|
|
|
|
|
44 |
The presented and trained language model is for **research purposes only**. The GC4 corpus—used for training—contains crawled texts from the internet. Thus, this GPT-2 model can be considered as highly biased, potentially encoding stereotypical associations along gender, race, ethnicity, and disability status. Before using and working with the released checkpoints, it is highly recommended to read:
|
45 |
|
46 |
- **"On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?"** by Emily M. Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell.
|
|
|
63 |
```python
|
64 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
65 |
|
|
|
66 |
tokenizer = AutoTokenizer.from_pretrained("kkirchheim/german-gpt2-medium")
|
67 |
+
model = AutoModelForCausalLM.from_pretrained("kkirchheim/german-gpt2-medium")
|
68 |
```
|
69 |
|
70 |
**Code Example:**
|
|
|
76 |
|
77 |
generator = pipeline('text-generation', model=model_name, tokenizer=tokenizer)
|
78 |
|
79 |
+
prompt = "Der Sinn des Lebens ist"
|
80 |
outputs = generator(prompt, max_length=50, num_return_sequences=1)
|
81 |
|
82 |
print(outputs[0]['generated_text'])
|
|
|
89 |
- **Third-Party Resources:**
|
90 |
- Tokenizer and initial model architecture from [stefan-it/german-gpt2-larger](https://huggingface.co/stefan-it/german-gpt2-larger).
|
91 |
|
|
|
92 |
---
|
93 |
|
94 |
+
**Disclaimer:** This model is provided for **research purposes only** and comes with no warranties.
|
95 |
+
The authors are not responsible for any output generated by the model.
|
96 |
+
Users should exercise caution and are responsible for compliance with applicable laws and regulations.
|
97 |
|
98 |
|
99 |
**Changelog:**
|