Update README.md

README.md

@@ -6,7 +6,8 @@ tags:
 license: mit
 ---
 ###  InCaseLawBERT
-Model and tokenizer files for the InCaseLawBERT model.
+Model and tokenizer files for the InCaseLawBERT model from the paper [Pre-training Transformers on Indian Legal Text](https://arxiv.org/abs/2209.06049).
+
 
 ### Training Data
 For building the pre-training corpus of Indian legal text, we collected a large corpus of case documents from the Indian Supreme Court and many High Courts of India.
@@ -19,34 +20,43 @@ This model is initialized with the [Legal-BERT model](https://huggingface.co/zlu
 We further train this model on our data for 300K steps on the Masked Language Modeling (MLM) and Next Sentence Prediction (NSP) tasks.
 
 ### Model Overview
+This model uses the same tokenizer as [CaseLawBERT](https://huggingface.co/zlucia/legalbert).
 This model has the same configuration as the [bert-base-uncased model](https://huggingface.co/bert-base-uncased):
 12 hidden layers, 768 hidden dimensionality, 12 attention heads, ~110M parameters.
 
 ### Usage
-Using the tokenizer (same as [CaseLawBERT](https://huggingface.co/zlucia/legalbert))
+Using the model to get embeddings/representations for a piece of text
 ```python
-from transformers import AutoTokenizer
+from transformers import AutoTokenizer, AutoModel
 tokenizer = AutoTokenizer.from_pretrained("law-ai/InCaseLawBERT")
-```
-Using the model to get embeddings/representations for a sentence
-```python
-from transformers import AutoModel
+text = "Replace this string with yours"
+encoded_input = tokenizer(text, return_tensors="pt")
 model = AutoModel.from_pretrained("law-ai/InCaseLawBERT")
+output = model(**encoded_input)
+last_hidden_state = output.last_hidden_state
 ```
 
 ### Fine-tuning Results
+We have fine-tuned all pre-trained models on 3 legal tasks with Indian datasets:
+* Legal Statute Identification ([ILSI Dataset](https://arxiv.org/abs/2112.14731))[Multi-label Text Classification]: Identifying relevant statutes (law articles) based on the facts of a court case
+* Semantic Segmentation ([ISS Dataset](https://arxiv.org/abs/1911.05405))[Sentence Tagging]: Segmenting the document into 7 functional parts (semantic segments) such as Facts, Arguments, etc.
+* Court Judgment Prediction ([ILDC Dataset](https://arxiv.org/abs/2105.13562))[Binary Text Classification]: Predicting whether the claims/petitions of a court case will be accepted/rejected
+
+InCaseLawBERT performs close to CaseLawBERT across the three tasks, but not as good as InLegalBERT. For details, see our [paper](https://arxiv.org/abs/2209.06049).
 
 ### Citation
 ```
-@inproceedings{paul-2022-ptinlegal,
-    title = "Pre-training Transformers on Indian Legal Text",
-    author = "Paul, Shounak  and
-      Mandal, Arpan  and
-      Goyal, Pawan  and
-      Ghosh, Saptarshi",
-    eprinttype = {arXiv}
+@article{paul-2022-pretraining,
+  doi = {10.48550/ARXIV.2209.06049},
+  url = {https://arxiv.org/abs/2209.06049},
+  author = {Paul, Shounak and Mandal, Arpan and Goyal, Pawan and Ghosh, Saptarshi},
+  title = {Pre-training Transformers on Indian Legal Text},
+  publisher = {arXiv}, 
+  year = {2022},
+  copyright = {Creative Commons Attribution 4.0 International}
 }
 ```
+
 ### About Us
 We are a group of researchers from the Department of Computer Science and Technology, Indian Insitute of Technology, Kharagpur. 
 Our research interests are primarily ML and NLP applications for the legal domain, with a special focus on the challenges and oppurtunites for the Indian legal scenario. 
@@ -55,4 +65,4 @@ We have, and are currently working on several legal tasks such as:
 * semantic segmentation of legal documents
 * legal statute identification from facts, court judgment prediction
 * legal document matching
-You can find our publicly available codes and datasets [here](https://github.com/Law-AI)
+You can find our publicly available codes and datasets [here](https://github.com/Law-AI).