Update README.md

README.md

@@ -14,28 +14,32 @@ tags:
 - masked-language-model
 - south africa
 ---
-# Zabantu - Explorting Multilingual Language Model training for South African Bantu Languages
-
-Zabantu( "Za" for South Africa, "bantu" for Bantu languages) is a collection of masked language models that have been trained from scratch using a compact dataset comprising various subsets of Bantu languages spoken in South Africa. These models are inspired by the work done on AfriBERTa, which demonstrated the effectiveness of training on XLM-R architecture using a smaller dataset. The focus of this work was to use LLMs to advance NLP applications in Tshivenda and also to serve as a benchmark for future works covering Bantu languages.
 
-## Model Overview
+# Zabantu - Explorting Multilingual Language Model training for South African Bantu Languages
 
-This model card provides an overview of the multilingual language models developed for South African languages, with a specific focus on advancing Tshivenda natural language processing (NLP) coverage. Zabantu-XLMR refers to a fleet of models trained on different combinations of South African Bantu languages. These include:
+> Zabantu( "Za" for South Africa, "bantu" for Bantu languages) is a collection of masked language models that have been trained from scratch using a compact dataset comprising various subsets of Bantu languages spoken in South Africa. These models are inspired by the work done on AfriBERTa, which demonstrated the effectiveness of training on XLM-R architecture using a smaller dataset. The focus of this work was to use LLMs to advance NLP applications in Tshivenda and also to serve as a benchmark for future works covering Bantu languages.
 
-- Zabantu-VEN: A monolingual language model trained on 73k raw sentences in Tshivenda
-- Zabantu-NSO: A monolingual language model trained on 179k raw sentences in Sepedi
-- Zabantu-NSO+VEN: A bilingual language model trained on 179k raw sentences in Sepedi and 73k sentences in Tshivenda
-- Zabantu-SOT+VEN: A multilingual language model trained on 479k raw sentences from Sesotho, Sepedi, Setswana, and Tshivenda
-- Zabantu-BANTU: A multilingual language model trained on 1.4M raw sentences from 9 South African Bantu languages
 
-## Model Details
+# Model Details
 
-- **Model Name:** Zabantu-XLMR
-- **Model Version:** 1.0.0
+- **Model Name:** Zabantu-XLM-Roberta
+- **Model Version:** 0.0.1
 - **Model Architecture:** [XLM-RoBERTa architecture](https://arxiv.org/abs/1911.02116)
 - **Model Size:** 80 - 250 million parameters
 - **Language Support:** Tshivenda, Nguni languages (Zulu, Xhosa, Swati), Sotho languages (Northern Sotho, Southern Sotho, Setswana), and Xitsonga.
 
+
+## Model Varients
+
+This model card provides an overview of the multilingual language models developed for South African languages, with a specific focus on advancing Tshivenda natural language processing (NLP) coverage. Zabantu-XLMR refers to a fleet of models trained on different combinations of South African Bantu languages. These include:
+
+- [Zabantu-VEN](https://huggingface.co/dsfsi/zabantu-ven-120m): A monolingual language model trained on 73k raw sentences in Tshivenda
+- [Zabantu-NSO](https://huggingface.co/dsfsi/zabantu-nso-80m): A monolingual language model trained on 179k raw sentences in Sepedi
+- [Zabantu-NSO+VEN](https://huggingface.co/dsfsi/zabantu-nso-ven-170m): A bilingual language model trained on 179k raw sentences in Sepedi and 73k sentences in Tshivenda
+- [Zabantu-SOT+VEN](https://huggingface.co/dsfsi/zabantu-sot-ven-170m): A multilingual language model trained on 479k raw sentences from Sesotho, Sepedi, Setswana, and Tshivenda
+- [Zabantu-BANTU](https://huggingface.co/dsfsi/zabantu-bantu-250m): A multilingual language model trained on 1.4M raw sentences from 9 South African Bantu languages
+
+
 ## Intended Use
 
 The Zabantu models are intended to be used for various NLP tasks involving Tshivenda and related South African languages. In addition, the model can be fine-tuned on a variety of downstream tasks, such as:
@@ -70,6 +74,12 @@ We also acknowledge the potential to further improve the model by training it on
 
 As with any language model, the generated output should be carefully reviewed and post-processed to ensure accuracy and cultural sensitivity.
 
-## Training Data
+# Training Data
+
+The models have been trained on a large corpus of text data collected from various sources, including [SADiLaR](https://repo.sadilar.org/handle/20.500.12185/7), [Leipnets](https://wortschatz.uni-leipzig.de/en/download/Venda#ven_community_2017), [Flores](https://github.com/facebookresearch/flores), [CC-100](https://data.statmt.org/cc-100/), [Opus](https://opus.nlpl.eu/opus-100.php) and various South African government websites. The training data covers a wide range of topics and domains, notably religion, politics, academics and health (mostly Covid-19).
+
+<hr/>
+
+# Closing Remarks
 
-The models have been trained on a large corpus of text data collected from various sources, including [SADiLaR](https://repo.sadilar.org/handle/20.500.12185/7), [Leipnets](https://wortschatz.uni-leipzig.de/en/download/Venda#ven_community_2017), [Flores](https://github.com/facebookresearch/flores), [CC-100](https://data.statmt.org/cc-100/), [Opus](https://opus.nlpl.eu/opus-100.php) and various South African government websites. The training data
+The Zabantu models provide a valuable resource for advancing Tshivenda NLP coverage and promoting cross-lingual learning techniques for South African languages. They have the potential to enhance various NLP applications, foster linguistic diversity, and contribute to the development of language technologies in the South African context.