Ifyokoh commited on
Commit
7af5526
1 Parent(s): 4e13d07

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +69 -0
README.md ADDED
@@ -0,0 +1,69 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language:
3
+ - ig
4
+ metrics:
5
+ - bleu
6
+ - ter
7
+ base_model:
8
+ - masakhane/m2m100_418M_ibo_en_rel_news
9
+ pipeline_tag: translation
10
+ ---
11
+
12
+
13
+ ### Model Description
14
+
15
+ This is a machine translation model from dialectal Igbo to English with the ability to handle regional language variations.
16
+
17
+
18
+ **Paper:** https://huggingface.co/papers/2405.00997
19
+
20
+ ## How to use
21
+
22
+
23
+ ```python
24
+ from transformers import AutoModelForSeq2SeqLM
25
+ from transformers import AutoTokenizer
26
+
27
+
28
+ model = AutoModelForSeq2SeqLM.from_pretrained('Ifyokoh/m2m100_dialect_ibo_en_mt')
29
+ tokenizer = AutoTokenizer.from_pretrained('Ifyokoh/m2m100_dialect_ibo_en_mt')
30
+ ```
31
+
32
+
33
+ ## Citation
34
+
35
+ ```cite
36
+ @inproceedings{emezue-etal-2024-igboapi,
37
+ title = "The {I}gbo{API} Dataset: Empowering {I}gbo Language Technologies through Multi-dialectal Enrichment",
38
+ author = "Emezue, Chris Chinenye and
39
+ Okoh, Ifeoma and
40
+ Mbonu, Chinedu Emmanuel and
41
+ Chukwuneke, Chiamaka and
42
+ Lal, Daisy Monika and
43
+ Ezeani, Ignatius and
44
+ Rayson, Paul and
45
+ Onwuzulike, Ijemma and
46
+ Okeke, Chukwuma Onyebuchi and
47
+ Nweya, Gerald Okey and
48
+ Ogbonna, Bright Ikechukwu and
49
+ Oraegbunam, Chukwuebuka Uchenna and
50
+ Awo-Ndubuisi, Esther Chidinma and
51
+ Osuagwu, Akudo Amarachukwu",
52
+ editor = "Calzolari, Nicoletta and
53
+ Kan, Min-Yen and
54
+ Hoste, Veronique and
55
+ Lenci, Alessandro and
56
+ Sakti, Sakriani and
57
+ Xue, Nianwen",
58
+ booktitle = "Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)",
59
+ month = may,
60
+ year = "2024",
61
+ address = "Torino, Italia",
62
+ publisher = "ELRA and ICCL",
63
+ url = "https://aclanthology.org/2024.lrec-main.1384",
64
+ pages = "15932--15941",
65
+ abstract = "The Igbo language is facing a risk of becoming endangered, as indicated by a 2025 UNESCO study. This highlights the need to develop language technologies for Igbo to foster communication, learning and preservation. To create robust, impactful, and widely adopted language technologies for Igbo, it is essential to incorporate the multi-dialectal nature of the language. The primary obstacle in achieving dialectal-aware language technologies is the lack of comprehensive dialectal datasets. In response, we present the IgboAPI dataset, a multi-dialectal Igbo-English dictionary dataset, developed with the aim of enhancing the representation of Igbo dialects. Furthermore, we illustrate the practicality of the IgboAPI dataset through two distinct studies: one focusing on Igbo semantic lexicon and the other on machine translation. In the semantic lexicon project, we successfully establish an initial Igbo semantic lexicon for the Igbo semantic tagger, while in the machine translation study, we demonstrate that by finetuning existing machine translation systems using the IgboAPI dataset, we significantly improve their ability to handle dialectal variations in sentences.",
66
+ }
67
+ ```
68
+
69
+