Create README.md

README.md

@@ -0,0 +1,27 @@
+---
+language:
+- ja
+license: cc-by-sa-4.0
+datasets:
+- wikipedia
+- cc100
+widget:
+- text: "早稲田 大学 で 自然 言語 処理 を"
+---
+
+# nlp-waseda/gpt2-xl-japanese
+
+This model is Japanese GPT-2 pretrained on Japanese Wikipedia and CC-100.
+The parameters of the model are based on [Radford+ 2019](https://paperswithcode.com/paper/language-models-are-unsupervised-multitask).
+
+## Intended uses & limitations
+
+You can use the raw model for text generation or fine-tune it to a downstream task.
+
+Note that the texts should be segmented into words using Juman++ in advance.
+
+### Preprocessing
+
+The texts are normalized using zenhan, segmented into words using Juman++, and tokenized using SentencePiece. Juman++ 2.0.0-rc3 was used for pretraining.
+
+The model was trained on 8 NVIDIA A100 GPUs.