Hugging Face model: Shu-vi/Russian_BPE_Tokenizer_16k
Author: Shu-vi
Created: 2025-10-09 22:44:47
Last modified: 2025-10-10 13:33:50
Library: tokenizers
Likes: 0 | Downloads: 0
Tags: tokenizers, tokenizer, BPE, subword, russian, custom, ru, dataset:ria.ru, dataset:lenta.ru, dataset:meduza.io, dataset:kommersant.ru, license:mit, region:us
Card metadata:
license: mit
language: ru
library_name: tokenizers
tags: tokenizer, BPE, subword, russian, custom
datasets: ria.ru, lenta.ru, meduza.io, kommersant.ru
metrics: oov_rate, reconstruction_accuracy, compression_ratio