Edit model card

afro-xlmr-large-61L

AfroXLMR-large was created by MLM adaptation of XLM-R-large model on 61 languages widely spoken in Africa including 4 high-resource languages.

Pre-training corpus

A mix of mC4, Wikipedia and OPUS data

Languages

There are 61 languages available :

  • English (eng)
  • Amharic (amh)
  • Arabic (ara)
  • Somali (som)
  • Kiswahili (swa)
  • Portuguese (por)
  • Afrikaans (afr)
  • French (fra)
  • isiZulu (zul)
  • Malagasy (mlg)
  • Hausa (hau)
  • chiShona (sna)
  • Egyptian Arabic (arz)
  • Chichewa (nya)
  • Igbo (ibo)
  • isiXhosa (xho)
  • Yorùbá (yor)
  • Sesotho (sot)
  • Kinyarwanda (kin)
  • Tigrinya (tir)
  • Tsonga (tso)
  • Oromo (orm)
  • Rundi (run)
  • Northern Sotho (nso)
  • Ewe (ewe)
  • Lingala (lin)
  • Twi (twi)
  • Nigerian Pidgin (pcm)
  • Ga (gaa)
  • Lozi (loz)
  • Luganda (lug)
  • Gun (guw)
  • Bemba (bem)
  • Efik (efi)
  • Luvale (lue)
  • Luba-Lulua (lua)
  • Tonga (toi)
  • Tshivenḓa (ven)
  • Tumbuka (tum)
  • Tetela (tll)
  • Isoko (iso)
  • Kaonde (kqn)
  • Zande (zne)
  • Umbundu (umb)
  • Mossi (mos)
  • Tiv (tiv)
  • Luba-Katanga (lub)
  • Fula (fuv)
  • San Salvador Kongo (kwy)
  • Baoulé (bci)
  • Ruund (rnd)
  • Luo (luo)
  • Wolaitta (wal)
  • Swazi (ssw)
  • Lunda (lun)
  • Wolof (wol)
  • Nyaneka (nyk)
  • Kwanyama (kua)
  • Kikuyu (kik)
  • Fon (fon)

Acknowledgment

We would like to thank Google Cloud for providing us access to TPU v3-8 through the free cloud credits. Model trained using flax, before converted to pytorch.

BibTeX entry and citation info.

@misc{adelani2023sib200,
      title={SIB-200: A Simple, Inclusive, and Big Evaluation Dataset for Topic Classification in 200+ Languages and Dialects}, 
      author={David Ifeoluwa Adelani and Hannah Liu and Xiaoyu Shen and Nikita Vassilyev and Jesujoba O. Alabi and Yanke Mao and Haonan Gao and Annie En-Shiun Lee},
      year={2023},
      eprint={2309.07445},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}
Downloads last month
233
Safetensors
Model size
560M params
Tensor type
I64
·
F32
·