update readme

README.md

@@ -19,8 +19,17 @@ inference: false
 
 ![rinna-icon](./rinna.png)
 
+# Overview
 This repository provides a Japanese GPT-NeoX model of 3.6 billion parameters. The model was trained using code based on [EleutherAI/gpt-neox](https://github.com/EleutherAI/gpt-neox).
 
+## Model architecture
+A 36-layer, 2816-hidden-size transformer-based language model.
+
+# Pre-training
+The model was trained on around **312.5B** tokens from [Japanese CC-100](http://data.statmt.org/cc-100/ja.txt.xz), [Japanese C4](https://huggingface.co/datasets/mc4), and [Japanese Wikipedia](https://dumps.wikimedia.org/other/cirrussearch) to optimize a traditional language modelling objective. 
+
+A final validation perplexity of **8.68** has been reached.
+
 # How to use the model
 
 ~~~~python
@@ -53,14 +62,6 @@ print(output)
 """西田幾多郎は、この「絶対矛盾的自己同一」を「世界の自己同一」と置きかえ、さらに西田哲学を出発点として「絶対無」を「世界の成立」に変え、世界と自己を一つの統一物とみなす哲学として展開する。この世界と自己は絶対矛盾的自己同一として同一の性質を有し、同じ働きをする。西田哲学においては、この世界と自己は矛盾しあうのではなく、同一の性質をもっている。この世界と自己は同一である。絶対"""
 ~~~~
 
-# Model architecture
-A 36-layer, 2816-hidden-size transformer-based language model.
-
-# Training
-The model was trained on around **312.5B** tokens from [Japanese CC-100](http://data.statmt.org/cc-100/ja.txt.xz), [Japanese C4](https://huggingface.co/datasets/mc4), and [Japanese Wikipedia](https://dumps.wikimedia.org/other/cirrussearch) to optimize a traditional language modelling objective. 
-
-A final validation perplexity of **8.68** has been reached.
-
 # Tokenization
 The model uses a [sentencepiece](https://github.com/google/sentencepiece)-based tokenizer.
 * The tokenizer has a vocabulary size of 32,000.
@@ -88,5 +89,9 @@ The model uses a [sentencepiece](https://github.com/google/sentencepiece)-based
     # 'გამარ[UNK]ობა 吾輩は 猫である </s>'
     ~~~
 
+# Authors
+* [Tianyu Zhao](https://huggingface.co/tianyuz)
+* [Kei Sawada](https://huggingface.co/keisawada)
+
 # Licenese
 [The MIT license](https://opensource.org/licenses/MIT)