Evaluating Tokenizer Performance of Large Language Models Across Official Indian Languages
Paper
•
2411.12240
•
Published
•
6
•
2
SAGAR TAMANG
tamang0000
·
AI & ML interests
Neural Machine Translation (NMT)
Recent Activity
authored
a paper
about 2 months ago
Enhancing Assamese NLP Capabilities: Introducing a Centralized Dataset
Repository
authored
a paper
about 2 months ago
Evaluating Tokenizer Performance of Large Language Models Across
Official Indian Languages
commented
a paper
about 2 months ago
Evaluating Tokenizer Performance of Large Language Models Across
Official Indian Languages
Organizations
None yet
tamang0000's activity
commented
a
paper
about 2 months ago