File size: 1,959 Bytes
8dc4420 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 |
---
license: cc-by-sa-4.0
language:
- ja
tags:
- japanese input
- kana kanji conversion
---
# zenz-v1 Checkpoints
<!-- Provide a quick summary of what the model is/does. -->
[zenz-v1](https://huggingface.co/Miwa-Keita/zenz-v1) is a language model specialized for kana-kanji conversion tasks based on the GPT-2 architecture. It is intended for use in the neural kana-kanji conversion system "Zenzai."
This repository publishes the checkpoints for zenz-v1.
* 90M parameters
* Character-level + byte-level BPE tokenizer
* High performance in kana-kanji conversion tasks using greedy decoding
## Model Details
### Model Description
<!-- Provide a longer summary of what this model is. -->
The base model used is [ku-nlp/gpt2-small-japanese-char](https://huggingface.co/ku-nlp/gpt2-small-japanese-char) provided under [CC-BY-SA 4.0](https://creativecommons.org/licenses/by-sa/4.0/deed.ja).
This model is provided under [CC-BY-SA 4.0](https://creativecommons.org/licenses/by-sa/4.0/deed.ja).
- **Developed by:** Keita Miwa ([𝕏](https://twitter.com/miwa_ensan))
- **Model type:** GPT-2
- **Language(s) (NLP):** Japanese
- **License:** CC-BY-SA 4.0
- **Finetuned from model:** [ku-nlp/gpt2-small-japanese-char](https://huggingface.co/ku-nlp/gpt2-small-japanese-char)
### Model Sources
<!-- Provide the basic links for the model. -->
This model is intended for use with Zenzai (AzooKeyKanaKanjiConverter).
- **Repository:** https://github.com/ensan-hcl/AzooKeyKanaKanjiConverter
## Acknowledgements
The following libraries, tools, and language resources were utilized in constructing this model.
* MeCab (https://taku910.github.io/mecab/)
* ipadic-NEologd (https://github.com/neologd/mecab-ipadic-neologd)
* torch (https://pypi.org/project/torch/)
* transformers (https://pypi.org/project/transformers/)
* datasets (https://pypi.org/project/datasets/)
* jaconv (https://pypi.org/project/jaconv/)
* llama.cpp (https://github.com/ggerganov/llama.cpp) |