adding Swahili BERT
Browse files- README.md +60 -0
- config.json +30 -0
- pytorch_model.bin +3 -0
- special_tokens_map.json +1 -0
- tokenizer_config.json +1 -0
- training_args.bin +3 -0
- vocab.txt +0 -0
README.md
ADDED
@@ -0,0 +1,60 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
Hugging Face's logo
|
2 |
+
---
|
3 |
+
language: ha
|
4 |
+
datasets:
|
5 |
+
|
6 |
+
---
|
7 |
+
# bert-base-multilingual-cased-finetuned-hausa
|
8 |
+
## Model description
|
9 |
+
**bert-base-multilingual-cased-finetuned-hausa** is a **Hausa BERT** model obtained by fine-tuning **bert-base-multilingual-cased** model on Hausa language texts. It provides **better performance** than the multilingual BERT on text classification and named entity recognition datasets.
|
10 |
+
|
11 |
+
Specifically, this model is a *bert-base-multilingual-cased* model that was fine-tuned on Hausa corpus.
|
12 |
+
## Intended uses & limitations
|
13 |
+
#### How to use
|
14 |
+
You can use this model with Transformers *pipeline* for masked token prediction.
|
15 |
+
```python
|
16 |
+
>>> from transformers import pipeline
|
17 |
+
>>> unmasker = pipeline('fill-mask', model='Davlan/bert-base-multilingual-cased-finetuned-hausa')
|
18 |
+
>>> unmasker("Shugaban [MASK] Muhammadu Buhari ya amince da shawarar da ma’aikatar sufuri karkashin jagoranci")
|
19 |
+
|
20 |
+
[{'sequence':
|
21 |
+
'[CLS] Shugaban Nigeria Muhammadu Buhari ya amince da shawarar da ma [UNK] aikatar sufuri karkashin jagoranci [SEP]',
|
22 |
+
'score': 0.9762618541717529,
|
23 |
+
'token': 22045,
|
24 |
+
'token_str': 'Nigeria'},
|
25 |
+
{'sequence': '[CLS] Shugaban Ka Muhammadu Buhari ya amince da shawarar da ma [UNK] aikatar sufuri karkashin jagoranci [SEP]', 'score': 0.007239189930260181,
|
26 |
+
'token': 25444,
|
27 |
+
'token_str': 'Ka'},
|
28 |
+
{'sequence': '[CLS] Shugaban, Muhammadu Buhari ya amince da shawarar da ma [UNK] aikatar sufuri karkashin jagoranci [SEP]', 'score': 0.001990817254409194,
|
29 |
+
'token': 117,
|
30 |
+
'token_str': ','},
|
31 |
+
{'sequence': '[CLS] Shugaban Ghana Muhammadu Buhari ya amince da shawarar da ma [UNK] aikatar sufuri karkashin jagoranci [SEP]', 'score': 0.001566368737258017,
|
32 |
+
'token': 28682,
|
33 |
+
'token_str': 'Ghana'},
|
34 |
+
{'sequence': '[CLS] Shugabanmu Muhammadu Buhari ya amince da shawarar da ma [UNK] aikatar sufuri karkashin jagoranci [SEP]', 'score': 0.0009375187801197171,
|
35 |
+
'token': 11717,
|
36 |
+
'token_str': '##mu'}]
|
37 |
+
|
38 |
+
```
|
39 |
+
#### Limitations and bias
|
40 |
+
This model is limited by its training dataset of entity-annotated news articles from a specific span of time. This may not generalize well for all use cases in different domains.
|
41 |
+
## Training data
|
42 |
+
This model was fine-tuned on [Hausa CC-100](http://data.statmt.org/cc-100/)
|
43 |
+
|
44 |
+
## Training procedure
|
45 |
+
This model was trained on a single NVIDIA V100 GPU
|
46 |
+
|
47 |
+
## Eval results on Test set (F-score, average over 5 runs)
|
48 |
+
Dataset| mBERT F1 | ha_bert F1
|
49 |
+
-|-|-
|
50 |
+
[Hausa VOA NER](https://huggingface.co/datasets/hausa_voa_ner) | |
|
51 |
+
[MasakhaNER](https://github.com/masakhane-io/masakhane-ner) | 86.65 |
|
52 |
+
[VOA Hausa Textclass](https://huggingface.co/datasets/hausa_voa_topics) | 84.76 | 90.98
|
53 |
+
|
54 |
+
### BibTeX entry and citation info
|
55 |
+
By David Adelani
|
56 |
+
```
|
57 |
+
|
58 |
+
```
|
59 |
+
|
60 |
+
|
config.json
ADDED
@@ -0,0 +1,30 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
{
|
2 |
+
"_name_or_path": "bert-base-multilingual-cased",
|
3 |
+
"architectures": [
|
4 |
+
"BertForMaskedLM"
|
5 |
+
],
|
6 |
+
"attention_probs_dropout_prob": 0.1,
|
7 |
+
"directionality": "bidi",
|
8 |
+
"gradient_checkpointing": false,
|
9 |
+
"hidden_act": "gelu",
|
10 |
+
"hidden_dropout_prob": 0.1,
|
11 |
+
"hidden_size": 768,
|
12 |
+
"initializer_range": 0.02,
|
13 |
+
"intermediate_size": 3072,
|
14 |
+
"layer_norm_eps": 1e-12,
|
15 |
+
"max_position_embeddings": 512,
|
16 |
+
"model_type": "bert",
|
17 |
+
"num_attention_heads": 12,
|
18 |
+
"num_hidden_layers": 12,
|
19 |
+
"pad_token_id": 0,
|
20 |
+
"pooler_fc_size": 768,
|
21 |
+
"pooler_num_attention_heads": 12,
|
22 |
+
"pooler_num_fc_layers": 3,
|
23 |
+
"pooler_size_per_head": 128,
|
24 |
+
"pooler_type": "first_token_transform",
|
25 |
+
"position_embedding_type": "absolute",
|
26 |
+
"transformers_version": "4.4.2",
|
27 |
+
"type_vocab_size": 2,
|
28 |
+
"use_cache": true,
|
29 |
+
"vocab_size": 119547
|
30 |
+
}
|
pytorch_model.bin
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:5b633faf8a8eade25966b9ef2e1bdffe5c08dccf78cffd1c89cbc6757e4b860d
|
3 |
+
size 711988242
|
special_tokens_map.json
ADDED
@@ -0,0 +1 @@
|
|
|
|
|
1 |
+
{"unk_token": "[UNK]", "sep_token": "[SEP]", "pad_token": "[PAD]", "cls_token": "[CLS]", "mask_token": "[MASK]"}
|
tokenizer_config.json
ADDED
@@ -0,0 +1 @@
|
|
|
|
|
1 |
+
{"do_lower_case": false, "unk_token": "[UNK]", "sep_token": "[SEP]", "pad_token": "[PAD]", "cls_token": "[CLS]", "mask_token": "[MASK]", "tokenize_chinese_chars": true, "strip_accents": null, "model_max_length": 512, "special_tokens_map_file": null, "name_or_path": "bert-base-multilingual-cased"}
|
training_args.bin
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:ebed21c7d316110025a61ba6b4ee97aadc323b4f52e450b6a53ac13993ee5f25
|
3 |
+
size 2287
|
vocab.txt
ADDED
The diff for this file is too large to render.
See raw diff
|
|