Hugging Face model: Shu-vi/Russian_wordpiece_tokenizer_32k
Author: Shu-vi
Created: 2025-10-10 00:34:00
Last modified: 2025-10-10 13:31:23
Library: tokenizers
Likes: 0 | Downloads: 0
Tags: tokenizers, tokenizer, wordpiece, subword, russian, custom, ru, dataset:ria.ru, dataset:lenta.ru, dataset:meduza.io, dataset:kommersant.ru, license:mit, region:us
Card metadata:
license: mit
language: ru
library_name: tokenizers
tags: tokenizer, wordpiece, subword, russian, custom
datasets: ria.ru, lenta.ru, meduza.io, kommersant.ru
metrics: oov_rate, reconstruction_accuracy, compression_ratio