Space: sagargurujula/telugu-tokenizer
title: Telugu Tokenizer App
sdk: docker
description: A tokenizer app for tokenizing Telugu text. It uses BPE (Byte Pair Encoding) to tokenize Telugu text. 5k is the vocab size.
tags: telugu, tokenizer, NLP, transformers
license: apache-2.0
model: telugu-tokenizer-model
datasets: telugu-dataset
isPrivate: no
Tags: docker, telugu, tokenizer, NLP, transformers, region:us