add readme

README.md

@@ -1,3 +1,165 @@
 ---
+thumbnail: https://github.com/rinnakk/japanese-pretrained-models/blob/master/rinna.png
+language: ja
+datasets:
+  - reazon-research/reazonspeech
+tags:
+  - automatic-speech-recognition
+  - speech
+  - audio
+  - hubert
+  - gpt_neox
+  - asr
+  - nlp
 license: apache-2.0
 ---
+
+# `rinna/nue-asr`
+
+![rinna-icon](./rinna.png)
+
+# Overview
+[[Paper]](https://arxiv.org/abs/2312.03668)
+[[GitHub]](https://github.com/rinnakk/nue-asr)
+
+We propose a novel end-to-end speech recognition model, `Nue ASR`, which integrates pre-trained speech and language models.
+
+The name `Nue` comes from the Japanese word ([`鵺/ぬえ/Nue`](https://en.wikipedia.org/wiki/Nue)), one of the Japanese legendary creatures ([`妖怪/ようかい/Yōkai`](https://en.wikipedia.org/wiki/Y%C5%8Dkai)).
+
+This model is capable of performing highly accurate Japanese speech recognition.
+By utilizing a GPU, it can recognize speech at speeds exceeding real-time.
+
+Benchmark score including our models can be seen at https://rinnakk.github.io/research/benchmarks/asr/
+
+* **Model architecture**
+
+    This model consists of three main components: HuBERT audio encoder, bridge network, and GPT-NeoX decoder.
+    The weights of HuBERT and GPT-NeoX were initialized with the pre-trained weights of HuBERT and GPT-NeoX, respectively.
+    - [japanese-hubert-base](https://huggingface.co/rinna/japanese-hubert-base)
+    - [japanese-gpt-neox-3.6b](https://huggingface.co/rinna/japanese-gpt-neox-3.6b)
+
+* **Training**
+
+    The model was trained on approximately 19,000 hours of following Japanese speech corpus.
+    - [ReazonSpeech](https://huggingface.co/datasets/reazon-research/reazonspeech)
+
+
+* **Authors**
+
+    - [Yukiya Hono](https://huggingface.co/yky-h)
+    - [Koh Mitsuda](https://huggingface.co/mitsu-koh)
+    - [Tianyu Zhao](https://huggingface.co/tianyuz)
+    - [Kentaro Mitsui](https://huggingface.co/Kentaro321)
+    - [Toshiaki Wakatsuki](https://huggingface.co/t-w)
+    - [Kei Sawada](https://huggingface.co/keisawada)
+
+---
+
+# How to use the model
+
+First, install the code for inference this model.
+
+```bash
+pip install git+https://github.com/rinnakk/nue-asr.git
+```
+
+Command-line interface and python interface are available.
+
+## Command-line usage
+The following command will transcribe the audio file via the command line interface.
+Audio files will be automatically downsampled to 16kHz.
+```bash
+nue-asr audio1.wav
+```
+You can specify multiple audio files.
+```bash
+nue-asr audio1.wav audio2.flac audio3.mp3
+```
+
+We can use DeepSpeed-Inference to accelerate the inference speed of GPT-NeoX module.
+If you use DeepSpeed-Inference, you need to install DeepSpeed.
+```bash
+pip install deepspeed
+```
+
+Then, you can use DeepSpeed-Inference as follows:
+```bash
+nue-asr --use-deepspeed audio1.wav
+```
+
+Run `nue-asr --help` for more information.
+
+## Python usage
+The example of python interface is as follows:
+```python
+import nue_asr
+
+model = nue_asr.load_model("rinna/nue-asr")
+tokenizer = nue_asr.load_tokenizer("rinna/nue-asr")
+
+result = nue_asr.transcribe(model, tokenizer, "path_to_audio.wav")
+print(result.text)
+```
+`nue_asr.transcribe` function can accept audio data as either a `numpy.array` or a `torch.Tensor`, in addition to traditional audio waveform file paths.
+
+Accelerating the inference speed of models using DeepSpeed-Inference is also available through the python interface.
+```python
+import nue_asr
+
+model = nue_asr.load_model("rinna/nue-asr", use_deepspeed=True)
+tokenizer = nue_asr.load_tokenizer("rinna/nue-asr")
+
+result = nue_asr.transcribe(model, tokenizer, "path_to_audio.wav")
+print(result.text)
+```
+
+---
+
+# Tokenization
+The model uses the same sentencepiece-based tokenizer as [japanese-gpt-neox-3.6b](https://huggingface.co/rinna/japanese-gpt-neox-3.6b).
+
+---
+
+# How to cite
+```bibtex
+@article{hono2023integration,
+    title={An Integration of Pre-Trained Speech and Language Models for End-to-End Speech Recognition},
+    author={Hono, Yukiya and Mitsuda, Koh and Zhao, Tianyu and Mitsui, Kentaro and Wakatsuki, Toshiaki and Sawada, Kei},
+    journal={arXiv preprint arXiv:2312.03668},
+    year={2023}
+}
+
+@misc{rinna-nue-asr,
+    title={rinna/nue-asr},
+    author={Hono, Yukiya and Mitsuda, Koh and Zhao, Tianyu and Mitsui, Kentaro and Wakatsuki, Toshiaki and Sawada, Kei},
+    url={https://huggingface.co/rinna/nue-asr}
+}
+```
+---
+
+# Citations
+```bibtex
+@article{hsu2021hubert,
+    title={HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units},
+    author={Hsu, Wei-Ning and Bolte, Benjamin and Tsai, Yao-Hung Hubert and Lakhotia, Kushal and Salakhutdinov, Ruslan and Mohamed, Abdelrahman},
+    journal={IEEE/ACM Transactions on Audio, Speech, and Language Processing},
+    year={2021},
+    volume={29},
+    pages={3451-3460},
+    doi={10.1109/TASLP.2021.3122291}
+}
+
+@software{andoniangpt2021gpt,
+    title={{GPT-NeoX: Large Scale Autoregressive Language Modeling in PyTorch}},
+    author={Andonian, Alex and Anthony, Quentin and Biderman, Stella and Black, Sid and Gali, Preetham and Gao, Leo and Hallahan, Eric and Levy-Kramer, Josh and Leahy, Connor and Nestler, Lucas and Parker, Kip and Pieler, Michael and Purohit, Shivanshu and Songz, Tri and Phil, Wang and Weinbach, Samuel},
+    url={https://www.github.com/eleutherai/gpt-neox},
+    doi={10.5281/zenodo.5879544},
+    month={8},
+    year={2021},
+    version={0.0.1},
+}
+```
+---
+
+# License
+[The Apache 2.0 license](https://www.apache.org/licenses/LICENSE-2.0)