arazd commited on
Commit
159c184
1 Parent(s): cee7259

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +14 -3
README.md CHANGED
@@ -13,8 +13,8 @@ widget:
13
  - text: "Tissue-based diagnostics and research is incessantly evolving with the development of new molecular tools. It has long been realized that immunohistochemistry can add an important new level of information on top of morphology and that protein expression patterns in a cancer may yield crucial diagnostic and prognostic information. We have generated an immunohistochemistry-based map of protein expression profiles in normal tissues, cancer and cell lines."
14
  example_title: "Journal prediction"
15
  ---
16
- This is the finetuned model presented in **MIReAD: a simple method for learning high-quality representations from
17
- scientific documents (ACL 2023)**.
18
 
19
  We trained MIReAD on >500,000 PubMed and arXiv abstracts across over 2,000 journal classes. MIReAD was initialized with SciBERT weights and finetuned to predict journal class based on the abstract and title of the paper. MIReAD uses SciBERT's tokenizer.
20
 
@@ -52,7 +52,18 @@ logits = out.logits
52
  out = model.bert(**inputs)
53
  # IMPORTANT: use [CLS] token representation as document-level representation (hence, 0th idx)
54
  feature = out.last_hidden_state[:, 0, :]
55
-
56
  ```
57
 
58
  You can access our PubMed and arXiv abstracts and journal labels data here: [https://huggingface.co/datasets/brainchalov/pubmed_arxiv_abstracts_data](https://huggingface.co/datasets/brainchalov/pubmed_arxiv_abstracts_data).
 
 
 
 
 
 
 
 
 
 
 
 
 
13
  - text: "Tissue-based diagnostics and research is incessantly evolving with the development of new molecular tools. It has long been realized that immunohistochemistry can add an important new level of information on top of morphology and that protein expression patterns in a cancer may yield crucial diagnostic and prognostic information. We have generated an immunohistochemistry-based map of protein expression profiles in normal tissues, cancer and cell lines."
14
  example_title: "Journal prediction"
15
  ---
16
+ This is the finetuned model presented in **[https://arxiv.org/abs/2305.04177](MIReAD: simple method for learning high-quality representations from
17
+ scientific documents (ACL 2023))**.
18
 
19
  We trained MIReAD on >500,000 PubMed and arXiv abstracts across over 2,000 journal classes. MIReAD was initialized with SciBERT weights and finetuned to predict journal class based on the abstract and title of the paper. MIReAD uses SciBERT's tokenizer.
20
 
 
52
  out = model.bert(**inputs)
53
  # IMPORTANT: use [CLS] token representation as document-level representation (hence, 0th idx)
54
  feature = out.last_hidden_state[:, 0, :]
 
55
  ```
56
 
57
  You can access our PubMed and arXiv abstracts and journal labels data here: [https://huggingface.co/datasets/brainchalov/pubmed_arxiv_abstracts_data](https://huggingface.co/datasets/brainchalov/pubmed_arxiv_abstracts_data).
58
+
59
+ Check our GitHub repo here: [https://github.com/arazd/miread](https://github.com/arazd/miread)
60
+
61
+ To cite this work:
62
+ ```bibtex
63
+ @inproceedings{razdaibiedina2023miread,
64
+ title={MIReAD: simple method for learning high-quality representations from scientific documents},
65
+ author={Razdaibiedina, Anastasia and Brechalov, Alexander},
66
+ booktitle={Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics},
67
+ year={2023}
68
+ }
69
+ ```