NeonTech Tokenizers

Custom multimodal tokenizer family for the NeonTech model series.

Available Versions

v1 — 131,072 vocab, DeepSeek V4 base + Igbo/Japanese/Arabic extensions, 80 special tokens
See the v1 branch for full details on special tokens and architecture.

from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained(“Neon-tech/Tokenizers”, revision=“v1”)

Downloads last month: -; Downloads are not tracked for this model. How to track

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support