nobu-g commited on
Commit
855548c
1 Parent(s): 8298794

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -61,7 +61,7 @@ Also note that Japanese Wikipedia was duplicated 10 times to make the total size
61
  ## Training procedure
62
 
63
  We first segmented texts in the corpora into words using [Juman++ 2.0.0-rc3](https://github.com/ku-nlp/jumanpp/releases/tag/v2.0.0-rc3) for whole word masking.
64
- Then, we built a sentencepiece model with 32000 tokens including all characters that appear in the training corpus.
65
 
66
  We tokenized raw corpora into character-level subwords using the sentencepiece model and trained the Japanese DeBERTa model using [transformers](https://github.com/huggingface/transformers) library.
67
  The training took 20 days using 8 NVIDIA A100-SXM4-40GB GPUs.
61
  ## Training procedure
62
 
63
  We first segmented texts in the corpora into words using [Juman++ 2.0.0-rc3](https://github.com/ku-nlp/jumanpp/releases/tag/v2.0.0-rc3) for whole word masking.
64
+ Then, we built a sentencepiece model with 22,012 tokens including all characters that appear in the training corpus.
65
 
66
  We tokenized raw corpora into character-level subwords using the sentencepiece model and trained the Japanese DeBERTa model using [transformers](https://github.com/huggingface/transformers) library.
67
  The training took 20 days using 8 NVIDIA A100-SXM4-40GB GPUs.