megagonlabs
/

t5-base-japanese-web

Text2Text Generation

text-generation-inference

Inference Endpoints

Model card Files Files and versions Community

t5-base-japanese-web / README.md

Yuta Hayashibe

Update README.md

c501405 almost 3 years ago

|

No virus

1.41 kB

metadata

language: ja
tags:
  - t5
  - text2text-generation
  - seq2seq
license: apache-2.0
datasets:
  - mc4
  - wiki40b

t5-base-japanese-web (with Byte-fallback)

Description

megagonlabs/t5-base-japanese-web is a T5 (Text-to-Text Transfer Transformer) model pre-trained on Japanese web texts.
Training codes are available on GitHub.

Corpus

Japanese in mC4/3.0.1
Japanese in wiki40b/1.3.0

Tokenizer

SentencePiece trained on Japanese Wikipedia

Vocabulary size: 32,000
Byte-fallback: Enabled

Parameters

T5 model: models/t5.1.1.base.gin
Training steps: 1,000,000

It took about 126 hours with TPU v3-8

Related models

License

Apache License 2.0