readme: add more references

README.md

@@ -6,7 +6,7 @@ license: mit
 ---
 
 # German GPT-2 model
-In this repository we release (yet another) GPT-2 model, that was trained on ~90 GB from the ["German colossal, clean Common Crawl corpus" ](https://german-nlp-group.github.io/projects/gc4-corpus.html).
+In this repository we release (yet another) GPT-2 model, that was trained on ~90 GB from the ["German colossal, clean Common Crawl corpus" (GC4)](https://german-nlp-group.github.io/projects/gc4-corpus.html).
 
 The model is meant to be an entry point for fine-tuning on other texts, and it is definitely not as good or "dangerous" as the English GPT-3 model. We do not plan extensive PR or staged releases for this model 😉
 
@@ -77,6 +77,9 @@ The following archives are used for training the (first version) of this GPT-2 m
 * `de_head_0007_2019-47.tar.gz`
 * `de_head_0007_2020-10.tar.gz`
 
+Details and URLs can be found on the [GC4](https://german-nlp-group.github.io/projects/gc4-corpus.html)
+page.
+
 Archives are then extracted and NLTK (`german` model) is used to sentence split the corpus.
 This results in a total training corpus size of 90GB.
 
@@ -88,15 +91,7 @@ Thus, the tokenizer and vocab is the same as used in the `dbmdz/german-gpt2` mod
 The model was trained on a v3-8 TPU, with the following parameters:
 
 ```bash
-python ./run_clm_flax.py --output_dir=/mnt/datasets/german-gpt2-larger/ \
---name_or_path dbmdz/german-gpt2 --do_train --do_eval --block_size=512 \
---per_device_train_batch_size=16 --per_device_eval_batch_size=16 \
---learning_rate=5e-3 --warmup_steps=1000 --adam_beta1=0.9 --adam_beta2=0.98 \
---weight_decay=0.01 --overwrite_output_dir --num_train_epochs=20 \
---logging_steps=500 --save_steps=2500 --eval_steps=2500 \
---train_file /mnt/datasets/gc4/train.txt \
---validation_file /mnt/datasets/gc4/validation.txt \
---preprocessing_num_workers 16
+python ./run_clm_flax.py --output_dir=/mnt/datasets/german-gpt2-larger/ \\n--name_or_path dbmdz/german-gpt2 --do_train --do_eval --block_size=512 \\n--per_device_train_batch_size=16 --per_device_eval_batch_size=16 \\n--learning_rate=5e-3 --warmup_steps=1000 --adam_beta1=0.9 --adam_beta2=0.98 \\n--weight_decay=0.01 --overwrite_output_dir --num_train_epochs=20 \\n--logging_steps=500 --save_steps=2500 --eval_steps=2500 \\n--train_file /mnt/datasets/gc4/train.txt \\n--validation_file /mnt/datasets/gc4/validation.txt \\n--preprocessing_num_workers 16
 ```
 
 Training took around 17 days for 20 epochs.