cointegrated
/

rut5-small

Text2Text Generation Transformers PyTorch JAX Safetensors Russian mt5 paraphrasing russian Inference Endpoints text-generation-inference

Model card Files Files and versions Community

cointegrated commited on Apr 28, 2021

Commit

3ebdb48

•

1 Parent(s): 013152b

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -12,7 +12,7 @@ It has rather poor paraphrasing performance, but can be fine tuned for this or o
 This model was created by taking the [alenusch/mt5small-ruparaphraser](https://huggingface.co/alenusch/mt5small-ruparaphraser) model and stripping 96% of its vocabulary which is unrelated to the Russian language or infrequent.
 * The original model has 300M parameters, with 256M of them being input and output embeddings.
-* After shrinking the `sentencepiece` vocabulary from 250K to 20K the number of model parameters reduced from 1.1GB to 246MB.
    * The first 5K tokens in the new vocabulary are taken from the original `mt5-small`.
    * The next 15K tokens are the most frequent tokens obtained by tokenizing a Russian web corpus from the [Leipzig corpora collection](https://wortschatz.uni-leipzig.de/en/download/Russian).

 This model was created by taking the [alenusch/mt5small-ruparaphraser](https://huggingface.co/alenusch/mt5small-ruparaphraser) model and stripping 96% of its vocabulary which is unrelated to the Russian language or infrequent.
 * The original model has 300M parameters, with 256M of them being input and output embeddings.
+* After shrinking the `sentencepiece` vocabulary from 250K to 20K the number of model parameters reduced to 65M parameters, and model size reduced from 1.1GB to 246MB.
    * The first 5K tokens in the new vocabulary are taken from the original `mt5-small`.
    * The next 15K tokens are the most frequent tokens obtained by tokenizing a Russian web corpus from the [Leipzig corpora collection](https://wortschatz.uni-leipzig.de/en/download/Russian).