README.md

metadata

language: ru
tags:
  - paraphrasing
  - russian
license: mit

This is a small Russian paraphraser based on the google/mt5-small model.

It was obtained by taking the alenusch/mt5small-ruparaphraser model and stripping 96% of its vocabulary which is unrelated to the Russian language or infrequent.

• The original model has 300M parameters, with 256M of them being input and output embeddings.
• After shrinking the sentencepiece vocabulary from 250K to 20K the number of model parameters reduced from 1.1GB to 246MB.
  • The first 5K tokens in the new vocabulary are taken from the original mt5-small.
  • The next 15K tokens are the most frequent tokens obtained by tokenizing a Russian web corpus from the Leipzig corpora collection.