Update README.md
Browse files
README.md
CHANGED
@@ -1,3 +1,55 @@
|
|
1 |
-
---
|
2 |
-
license: cc-by-sa-4.0
|
3 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
---
|
2 |
+
license: cc-by-sa-4.0
|
3 |
+
language:
|
4 |
+
- ja
|
5 |
+
tags:
|
6 |
+
- japanese input
|
7 |
+
- kana kanji conversion
|
8 |
+
---
|
9 |
+
# zenz-v1 Checkpoints
|
10 |
+
|
11 |
+
<!-- Provide a quick summary of what the model is/does. -->
|
12 |
+
|
13 |
+
[zenz-v1](https://huggingface.co/Miwa-Keita/zenz-v1) is a language model specialized for kana-kanji conversion tasks based on the GPT-2 architecture. It is intended for use in the neural kana-kanji conversion system "Zenzai."
|
14 |
+
|
15 |
+
This repository publishes the checkpoints for zenz-v1.
|
16 |
+
|
17 |
+
* 90M parameters
|
18 |
+
* Character-level + byte-level BPE tokenizer
|
19 |
+
* High performance in kana-kanji conversion tasks using greedy decoding
|
20 |
+
|
21 |
+
## Model Details
|
22 |
+
|
23 |
+
### Model Description
|
24 |
+
|
25 |
+
<!-- Provide a longer summary of what this model is. -->
|
26 |
+
|
27 |
+
The base model used is [ku-nlp/gpt2-small-japanese-char](https://huggingface.co/ku-nlp/gpt2-small-japanese-char) provided under [CC-BY-SA 4.0](https://creativecommons.org/licenses/by-sa/4.0/deed.ja).
|
28 |
+
|
29 |
+
This model is provided under [CC-BY-SA 4.0](https://creativecommons.org/licenses/by-sa/4.0/deed.ja).
|
30 |
+
|
31 |
+
- **Developed by:** Keita Miwa ([𝕏](https://twitter.com/miwa_ensan))
|
32 |
+
- **Model type:** GPT-2
|
33 |
+
- **Language(s) (NLP):** Japanese
|
34 |
+
- **License:** CC-BY-SA 4.0
|
35 |
+
- **Finetuned from model:** [ku-nlp/gpt2-small-japanese-char](https://huggingface.co/ku-nlp/gpt2-small-japanese-char)
|
36 |
+
|
37 |
+
### Model Sources
|
38 |
+
|
39 |
+
<!-- Provide the basic links for the model. -->
|
40 |
+
|
41 |
+
This model is intended for use with Zenzai (AzooKeyKanaKanjiConverter).
|
42 |
+
|
43 |
+
- **Repository:** https://github.com/ensan-hcl/AzooKeyKanaKanjiConverter
|
44 |
+
|
45 |
+
## Acknowledgements
|
46 |
+
|
47 |
+
The following libraries, tools, and language resources were utilized in constructing this model.
|
48 |
+
|
49 |
+
* MeCab (https://taku910.github.io/mecab/)
|
50 |
+
* ipadic-NEologd (https://github.com/neologd/mecab-ipadic-neologd)
|
51 |
+
* torch (https://pypi.org/project/torch/)
|
52 |
+
* transformers (https://pypi.org/project/transformers/)
|
53 |
+
* datasets (https://pypi.org/project/datasets/)
|
54 |
+
* jaconv (https://pypi.org/project/jaconv/)
|
55 |
+
* llama.cpp (https://github.com/ggerganov/llama.cpp)
|