---
language: dv
---

# byt5-base-dv

Pretrained from scratch on Dhivei (language of the Maldives)
with ByT5, Google's new byte-level tokenizer strategy.

**Use byt5-dv for now; this is less accurate**

Corpus: Sofwath's Dhivehi corpus https://github.com/Sofwath/DhivehiDatasets

Pretraining Notebook: 
https://colab.research.google.com/drive/1ERIZ1PyHn-yN_jo7dTQeODn22vrt-d1d?usp=sharing

## Fine-tuning Demo

On Dhivehi news classification task

https://colab.research.google.com/drive/11u5SafR4bKICmArgDl6KQ9vqfYtDpyWp?usp=sharing

## Issues

There was an issue with the vocabulary size, final layer, and/or accuracy on fine-tuning.