Update README.md

README.md

@@ -11,21 +11,17 @@
 ### Model Description
 
 <!-- Provide a longer summary of what this model is. -->
-Varta BERT is a model pre-trained on full training set from Varta on English and 14 Indic languages (Assamese, Bhojpuri, Bengali, Gujarati, Hindi, Kannada, Malayalam, Marathi, Nepali, Oriya, Punjabi, Tamil, Telugu, and Urdu) using a masked language modeling (MLM) objective from scratch. 
-Varta is a large-scale headline-generation dataset for Indic languages, including 41.8 million news articles in 14 different Indic languages (and English), which come from a variety of high-quality sources. 
+Varta-BERT is a model pre-trained on the `full` training set of Varta in 14 Indic languages (Assamese, Bhojpuri, Bengali, Gujarati, Hindi, Kannada, Malayalam, Marathi, Nepali, Oriya, Punjabi, Tamil, Telugu, and Urdu) and English, using a masked language modeling (MLM) objective. 
 
-
-The dataset and the model were introduced in [this paper](https://arxiv.org/pdf/2305.05858.pdf). The code was released in [this repository](https://github.com/rahular/varta). The data was released in [this bucket](https://console.cloud.google.com/storage/browser/varta-eu/data-release).
+Varta is a large-scale news corpus for Indic languages, including 41.8 million news articles in 14 different Indic languages (and English), which come from a variety of high-quality sources. 
+The dataset and the model are introduced in [this paper](https://arxiv.org/abs/2305.05858). The code is released in [this repository](https://github.com/rahular/varta). The data is released in [this bucket](https://console.cloud.google.com/storage/browser/varta-eu/data-release).
 
 
 ## Uses
-
 <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+You can use the raw model for masked language modelling, but it is mostly intended to be fine-tuned on a downstream task.
 
-You can use the raw model for masked language modelling, but it's mostly intended to be fine-tuned on a downstream task. <br>
-
-
-Note that this model is primarily aimed at being fine-tuned on tasks that use the whole sentence (potentially masked) to make decisions, such as sequence classification, token classification or question answering. For tasks such as text generation, you should look at our Varta T5 model.
+Note that this model is primarily aimed at being fine-tuned on tasks that use the whole sentence (potentially masked) to make decisions, such as sequence classification, token classification or question answering. For tasks such as text generation, you should look at our [Varta-T5](https://huggingface.co/rahular/varta-t5) model.
 
 ## Bias, Risks, and Limitations
 
@@ -55,22 +51,22 @@ model = AutoModelForMaskedLM.from_pretrained("rahular/varta-bert")
 
 ### Training Data
 Varta contains 41.8 million high-quality news articles in 14 Indic languages and English. 
-With 34.5 million non-English article-headline pairs, it is the largest headline-generation dataset of its kind. 
+With 34.5 million non-English article-headline pairs, it is the largest document-level dataset of its kind. 
 
 ### Pretraining
-We pretrain the Varta-BERT model using the standard BERT-Base architecture with 12 encoder layers. 
-We train with a maximum sequence length of 512 tokens with an embedding dimension of 768. 
-We use 12 attention heads with feed-forward width of 3072. 
-To support all the 15 languages in dataset we use a wordpiece vocabulary of size 128K. 
-In total, the model has 184M parameters. The model is trained with AdamW optimizer with alpha=0.9 and beta=0.98.
-We use an initial learning rate of 1e-4 with a warm-up of 10K steps and linearly decay the learning rate till the end of training. 
-We train the model for a total of 1M steps which takes 10 days to finish. 
-We use an effective batch size of 4096 and train the model on TPU v3-128 chips.
+- We pretrain the Varta-BERT model using the standard BERT-Base architecture with 12 encoder layers. 
+- We train with a maximum sequence length of 512 tokens with an embedding dimension of 768. 
+- We use 12 attention heads with feed-forward width of 3072. 
+- To support all the 15 languages in dataset we use a wordpiece vocabulary of size 128K. 
+- In total, the model has 184M parameters. The model is trained with AdamW optimizer with alpha=0.9 and beta=0.98.
+- We use an initial learning rate of 1e-4 with a warm-up of 10K steps and linearly decay the learning rate till the end of training. 
+- We train the model for a total of 1M steps which takes 10 days to finish. 
+- We use an effective batch size of 4096 and train the model on TPU v3-128 chips.
 
 <!-- This should link to a Data Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
 
 ### Evaluation Results
-To come.
+Please see [the paper](https://arxiv.org/pdf/2305.05858.pdf).
 
 ## Citation