Update README.md

README.md

@@ -10,15 +10,17 @@ Model and tokenizer files for the InCaseLawBERT model.
 
 ### Training Data
 For building the pre-training corpus of Indian legal text, we collected a large corpus of case documents from the Indian Supreme Court and many High Courts of India.
-These documents were collected from diverse publicly available sources on the Web, such as official websites of these courts (e.g., [the website of the Indian Supreme Court](https://main.sci.gov.in/)), the erstwhile website of the Legal Information Institute of India, 
-the popular legal repository [IndianKanoon](https://www.indiankanoon.org), and so on.
 The court cases in our dataset range from 1950 to 2019, and belong to all legal domains, such as Civil, Criminal, Constitutional, and so on.
-Additionally, we collected 1,113 Central Government Acts, which are the documents codifying the laws of the country. Each Act is a collection of related laws, called Sections. These 1,113 Acts contain a total of 32,021 Sections. 
 In total, our dataset contains around 5.4 million Indian legal documents (all in the English language). 
 The raw text corpus size is around 27 GB.
 
 ### Training Objective
 This model is initialized with the [Legal-BERT model](https://huggingface.co/zlucia/legalbert) from the paper [When does pretraining help?: assessing self-supervised learning for law and the CaseHOLD dataset of 53,000+ legal holdings](https://dl.acm.org/doi/abs/10.1145/3462757.3466088). In our work, we refer to this model as CaseLawBERT, and our re-trained model as InCaseLawBERT.
+We further train this model on our data for 300K steps on the Masked Language Modeling (MLM) and Next Sentence Prediction (NSP) tasks.
+
+### Model Overview
+This model has the same configuration as the [bert-base-uncased model](https://huggingface.co/bert-base-uncased):
+12 hidden layers, 768 hidden dimensionality, 12 attention heads, ~110M parameters.
 
 ### Usage
 Using the tokenizer (same as [CaseLawBERT](https://huggingface.co/zlucia/legalbert))
@@ -31,10 +33,26 @@ Using the model to get embeddings/representations for a sentence
 from transformers import AutoModel
 model = AutoModel.from_pretrained("law-ai/InCaseLawBERT")
 ```
-Using the model for further pre-training with MLM and NSP
-```python
-from transformers import BertForPreTraining
-model_with_pretraining_heads = BertForPreTraining.from_pretrained("law-ai/InCaseLawBERT")
-```
 
-### Citation
+### Fine-tuning Results
+
+### Citation
+```
+@inproceedings{paul-2022-ptinlegal,
+    title = "Pre-training Transformers on Indian Legal Text",
+    author = "Paul, Shounak  and
+      Mandal, Arpan  and
+      Goyal, Pawan  and
+      Ghosh, Saptarshi",
+    eprinttype = {arXiv}
+}
+```
+### About Us
+We are a group of researchers from the Department of Computer Science and Technology, Indian Insitute of Technology, Kharagpur. 
+Our research interests are primarily ML and NLP applications for the legal domain, with a special focus on the challenges and oppurtunites for the Indian legal scenario. 
+We have, and are currently working on several legal tasks such as:
+* named entity recognition, summarization of legal documents
+* semantic segmentation of legal documents
+* legal statute identification from facts, court judgment prediction
+* legal document matching
+You can find our publicly available codes and datasets [here](https://github.com/Law-AI)