Hugging Face Space: Jhayase/superbpe-tokenizer-playground
Author: Jhayase
SDK: static
title: SuperBPE Playground
short_description: Experiment with and compare different tokenizers
models: Xenova/gpt-4o
Xenova/llama-3-tokenizer
UW/OLMo2-8B-SuperBPE-t180k
UW/OLMo2-11B-SuperBPE-t180k
UW/OLMo2-8B-SuperBPE-t160k
UW/OLMo2-8B-SuperBPE-t80k
UW/OLMo2-8B-BPE
alisawuffles/superbpe-tokenizer-128k