Space: xu-song/tokenizer-arena
title: Tokenizer Arena
sdk: gradio
datasets: cc100
tags: tokenizer
short_description: Compare different tokenizers in char-level and byte-level.
Tags: gradio, tokenizer, region:us