Add tokenizer.json

README.md

@@ -2,6 +2,8 @@
 language: ja
 thumbnail: https://github.com/rinnakk/japanese-pretrained-models/blob/master/rinna.png
 tags:
+- ja
+- japanese
 - gpt-neox
 - text-generation
 - lm
@@ -60,25 +62,5 @@ Here are a few samples generated with and without the toy prefix weights, respec
 # Inference with FasterTransformer
 After version 5.1, [NVIDIA FasterTransformer](https://github.com/NVIDIA/FasterTransformer) now supports both inference for GPT-NeoX and a variety of soft prompts (including prefix-tuning). The released pretrained model and prefix weights in this repo have been verified to work with FasterTransformer 5.1.
 
-# How to cite
-```bibtex
-@misc{rinna-japanese-gpt-neox-small,
-    title = {rinna/japanese-gpt-neox-small},
-    author = {Zhao, Tianyu and Sawada, Kei},
-    url = {https://huggingface.co/rinna/japanese-gpt-neox-small}
-}
-
-@inproceedings{sawada2024release,
-    title = {Release of Pre-Trained Models for the {J}apanese Language},
-    author = {Sawada, Kei and Zhao, Tianyu and Shing, Makoto and Mitsui, Kentaro and Kaga, Akio and Hono, Yukiya and Wakatsuki, Toshiaki and Mitsuda, Koh},
-    booktitle = {Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)},
-    month = {5},
-    year = {2024},
-    pages = {13898--13905},
-    url = {https://aclanthology.org/2024.lrec-main.1213},
-    note = {\url{https://arxiv.org/abs/2404.01657}}
-}
-```
-
 # Licenese
 [The MIT license](https://opensource.org/licenses/MIT)