Create README.md

README.md

@@ -0,0 +1,19 @@
+**SpaceSciBERT**
+
+This is one of the 3 further pre-trained models from the SpaceTransformers family presented in [SpaceTransformers: Language Modeling for Space Systems](https://ieeexplore.ieee.org/document/9548078). The original Git repo is [strath-ace/smart-nlp](https://github.com/strath-ace/smart-nlp).
+
+The further pre-training corpus includes publications abstracts, books, and Wikipedia pages related to space systems. Corpus size is 14.3 GB. SpaceSciBERT was further pre-trained on this domain-specific corpus from [SciBERT-SciVocab (uncased)](https://huggingface.co/allenai/scibert_scivocab_uncased). In our paper, it is then fine-tuned for a Concept Recognition task.
+
+If using this model, please cite the following paper:
+
+@ARTICLE{
+9548078,  
+author={Berquand, Audrey and Darm, Paul and Riccardi, Annalisa},  
+journal={IEEE Access},   
+title={SpaceTransformers: Language Modeling for Space Systems},   
+year={2021},  
+volume={9},  
+number={},  
+pages={133111-133122},  
+doi={10.1109/ACCESS.2021.3115659}
+}