Update README.md

README.md

@@ -1,3 +1,24 @@
+---
+language: en
+tags:
+- longformer
+- longformer-scico
+license: apache-2.0
+datasets:
+- allenai/scico
+---
+
+# Longformer for SciCo
+
+This model is the `unified` model discussed in the paper [SciCo: Hierarchical Cross-Document Coreference for Scientific Concepts (AKBC 2021)](https://openreview.net/forum?id=OFLbgUP04nC) that formulates the task of hierarchical cross-document coreference resolution (H-CDCR) as a multiclass problem. The model takes as input two mentions `m1` and `m2` with their corresponding context and outputs 4 scores: 
+
+* 0: not related
+* 1: `m1` and `m2` corefer
+* 2: `m1` is a parent of `m2`
+* 3: `m1` is a child of `m2`.
+
+We provide the following code as an example to set the global attention on the special tokens: `<s>`, `<m>` and `</m>`.
+
 ```python
 from transformers import AutoTokenizer, AutoModelForSequenceClassification
 import torch
@@ -30,5 +51,20 @@ with torch.no_grad():
     output = model(tokens['input_ids'], tokens['attention_mask'], global_attention_mask)
     
 scores = torch.softmax(output.logits, dim=-1)
+# tensor([[0.0818, 0.0023, 0.0019, 0.9139]]) -- m1 is a child of m2
+```
+
+**Note:** There is a slight difference between this model and the original model presented in the [paper](https://openreview.net/forum?id=OFLbgUP04nC). The original model includes a single linear layer on top of the `<s>` token (equivalent to `[CLS]`) while this model includes a two-layers MLP to be in line with `LongformerForSequenceClassification`. The original repository can be found [here](https://github.com/ariecattan/scico). 
 
-```
+# Citation
+
+```python
+@inproceedings{
+    cattan2021scico,
+    title={SciCo: Hierarchical Cross-Document Coreference for Scientific Concepts},
+    author={Arie Cattan and Sophie Johnson and Daniel S Weld and Ido Dagan and Iz Beltagy and Doug Downey and Tom Hope},
+    booktitle={3rd Conference on Automated Knowledge Base Construction},
+    year={2021},
+    url={https://openreview.net/forum?id=OFLbgUP04nC}
+}
+```