File size: 990 Bytes
4a98580 b4f829a 1ac8773 ddbf8d6 9906e8d 4a98580 4c62be5 4a98580 4c62be5 4a98580 4c62be5 4a98580 4c62be5 4a98580 4c62be5 4a98580 b4f829a 4a98580 b4f829a 4a98580 b4f829a 4a98580 b4f829a 1ac8773 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 |
---
model-index:
- name: bert-base-dv
results: []
datasets:
- alakxender/haveeru-articles
language:
- dv
pipeline_tag: fill-mask
library_name: transformers
---
# BERT base for Dhivehi
Pretrained model on Dhivehi language using masked language modeling (MLM).
## Tokenizer
The *WordPiece* tokenizer uses several components:
* **Normalization**: lowercase and then NFKD unicode normalization.
* **Pretokenization**: splits by whitespace and punctuation.
* **Postprocessing**: single sentences are output in format `[CLS] sentence A [SEP]` and pair sentences in format `[CLS] sentence A [SEP] sentence B [SEP]`.
### Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 32
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 2
### Framework versions
- Transformers 4.41.2
- Pytorch 2.3.1+cu121
- Datasets 2.19.2
- Tokenizers 0.19.1 |