Hugging Face Space: saish-shetty/hindi-bpe-tokenizer
Author: saish-shetty
SDK: gradio
title: Hindi BPE Tokenizer
license: apache-2.0
short_description: Advanced Tokenizer of Hindi language using Regex Splitting