Hugging Face model: Shu-vi/Russian_Unigram_Tokenizer_16k
Author: Shu-vi
Created: 2025-10-10 00:49:34
Last modified: 2025-10-10 13:34:29
Library: tokenizers
Likes: 0 | Downloads: 0
Tags: tokenizers, tokenizer, Unigram, subword, russian, custom, ru, dataset:ria.ru, dataset:lenta.ru, dataset:meduza.io, dataset:kommersant.ru, license:mit, region:us
Card metadata:
license: mit
language: ru
library_name: tokenizers
tags: tokenizer, Unigram, subword, russian, custom
datasets: ria.ru, lenta.ru, meduza.io, kommersant.ru
metrics: oov_rate, reconstruction_accuracy, compression_ratio