File size: 2,060 Bytes
8e7c856 5c3e361 419bbb3 5c3e361 419bbb3 8e7c856 5c3e361 419bbb3 5c3e361 419bbb3 5c3e361 419bbb3 5c3e361 419bbb3 55a1ca5 a1d08f2 419bbb3 5c3e361 419bbb3 5c3e361 419bbb3 5c3e361 419bbb3 5c3e361 419bbb3 5c3e361 419bbb3 5c3e361 f01d42f a381c3f 5c3e361 419bbb3 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 |
---
license: apache-2.0
language:
- en
library_name: transformers
pipeline_tag: fill-mask
tags:
- climate
- biology
---
# Model Card for nasa-smd-ibm-v0.1
nasa-smd-ibm-v0.1 is a RoBERTa-based, Encoder-only transformer model, domain-adapted for NASA Science Mission Directorate (SMD) applications. It's fine-tuned on scientific journals and articles relevant to NASA SMD, aiming to enhance natural language technologies like information retrieval and intelligent search.
## Model Details
- **Base Model**: RoBERTa
- **Tokenizer**: Custom
- **Parameters**: 125M
- **Pretraining Strategy**: Masked Language Modeling (MLM)
## Training Data
- Wikipedia English (Feb 1, 2020)
- AGU Publications
- AMS Publications
- Scientific papers from Astrophysics Data Systems
- PubMed abstracts
- PMC (commercial license subset)
## Training Procedure
- **Framework**: fairseq 0.12.1 with PyTorch 1.9.1
- **Transformer Version**: 4.2.0
- **Strategy**: Masked Language Modeling (MLM)
## Evaluation
- BLURB Benchmark
- Pruned SQuAD2.0 (SQ2) Benchmark
- NASA SMD Experts Benchmark (WIP)
## Uses
- Named Entity Recognition (NER)
- Information Retrieval
- Sentence Transformers
## Citation
If you find this work useful, please cite using the following bibtex citation:
@misc {nasa-impact_2023,
author = { {NASA-IMPACT} },
title = { nasa-smd-ibm-v0.1 (Revision f01d42f) },
year = 2023,
url = { https://huggingface.co/nasa-impact/nasa-smd-ibm-v0.1 },
doi = { 10.57967/hf/1429 },
publisher = { Hugging Face }
}
## Contacts
- Bishwaranjan Bhattacharjee, IBM Research
- Muthukumaran Ramasubramanian, NASA-IMPACT (mr0051@uah.edu)
|