Hugging Face Space: AravindKumarRajendran/TamilBPE-10k-Tokenizer
Author: AravindKumarRajendran
SDK: gradio
title: TamilBPE 10k Tokenizer
license: apache-2.0
short_description: TamilBPE-10K - Byte-pair encoding based tokenizer for Tamil