Hugging Face model: wikilangs/tok
Author: wikilangs
Created: 2026-01-11 01:32:14
Last modified: 2026-01-11 01:32:30
Pipeline: text-generation
Library: wikilangs
Likes: 0 | Downloads: 0
Tags: wikilangs, nlp, tokenizer, embeddings, n-gram, markov, wikipedia, feature-extraction, sentence-similarity, tokenization, n-grams, markov-chain, text-mining, fasttext, babelvec, vocabulous, vocabulary, monolingual, family-constructed_auxlang, text-generation, tok, dataset:omarkamali/wikipedia-monthly, license:mit, region:us
Card metadata:
language: tok
language_name: Toki Pona
language_family: constructed_auxlang
tags: wikilangs, nlp, tokenizer, embeddings, n-gram, markov, wikipedia, feature-extraction, sentence-similarity, tokenization, n-grams, markov-chain, text-mining, fasttext, babelvec, vocabulous, vocabulary, monolingual, family-constructed_auxlang
license: mit
library_name: wikilangs
pipeline_tag: text-generation
datasets: omarkamali/wikipedia-monthly
dataset_info: name: wikipedia-monthly
description: Monthly snapshots of Wikipedia articles across 300+ languages
metrics: name: best_compression_ratio
type: compression
value: 3.877, name: best_isotropy
type: isotropy
value: 0.6399, name: vocabulary_size
type: vocab
value: 0
generated: 2026-01-11T00:00:00.000Z