---
{}
---
This is a very small uncased tokenizer for the [non-ascii version of TinyStories](https://huggingface.co/datasets/tdooms/TinyStories), based on the [original TinyStories dataset](https://huggingface.co/datasets/roneneldan/TinyStories). I use a WordPiece tokenizer with a vocabulary of 2048.

The tokenizer is strictly fitted to the mentioned dataset and probably won't work well in any context outside of children's stories.