---
license: mit
datasets:
- BiniyamAjaw/amharic_dataset_v2
language:
- am
---
# Amharic Tokenizer

<!-- The model is trained on a vast amharic data to tokenize unseen data into tokens. -->


## Model Details
- **Vocabulary Size:** 100,000
- **Tokenizer Type:** Byte-Pair Encoder

### Model Description

<!-- Tokenizer that uses BPE -->



- **Developed by:** Biniyam Ajaw
- **Language(s) (NLP):** Amharic and Amharic-Driven Languages
- **License:** MIT

### Model Sources [optional]

- **Repository:** https://github.com/biniyam69/Amharic-LLM-Finetuning/


## Uses

Model can be called by the autotokenizer module from the transformers package and can be used to tokenize any amharic text perfectly