Fill-Mask
Transformers
PyTorch
English
roberta
earth science
climate
biology
Inference Endpoints
Muthukumaran commited on
Commit
89f349f
·
verified ·
1 Parent(s): 556e635

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +12 -10
README.md CHANGED
@@ -16,7 +16,7 @@ datasets:
16
 
17
  # Model Card for nasa-smd-ibm-v0.1 (Indus)
18
 
19
- nasa-smd-ibm-v0.1 (Currently named as Indus) is a RoBERTa-based, Encoder-only transformer model, domain-adapted for NASA Science Mission Directorate (SMD) applications. It's fine-tuned on scientific journals and articles relevant to NASA SMD, aiming to enhance natural language technologies like information retrieval and intelligent search.
20
 
21
  ## Model Details
22
  - **Base Model**: RoBERTa
@@ -41,19 +41,21 @@ nasa-smd-ibm-v0.1 (Currently named as Indus) is a RoBERTa-based, Encoder-only tr
41
  - **Strategy**: Masked Language Modeling (MLM)
42
 
43
  ## Evaluation
44
- - BLURB Benchmark
45
- - Pruned SQuAD2.0 (SQ2) Benchmark (Amazon Rainforest, Oxygen, Geology and NASA ES QAs)
46
- - NASA SMD Expert QA Benchmark (WIP)
47
 
 
48
 
49
- ![image/png](https://cdn-uploads.huggingface.co/production/uploads/61099e5d86580d4580767226/EtCC3U_tMCv3bfLqQdqQm.png)
 
50
 
51
- ![Pruned SQ2 Benchmark](https://cdn-uploads.huggingface.co/production/uploads/61099e5d86580d4580767226/ruh6-IyiNlUiK21Ej4lDM.png)
 
 
 
52
 
53
  Please refer to the following dataset cards for further benchmarks and evaluation
54
- - NASA IR Benchmark - https://huggingface.co/datasets/nasa-impact/nasa-smd-IR-benchmark
55
- - NASA SMD Expert QA Benchmark - https://huggingface.co/datasets/nasa-impact/nasa-smd-qa-benchmark
56
- - Climate CHange Benchmark - https://huggingface.co/datasets/ibm/Climate-Change-NER
57
 
58
  ## Uses
59
  - Named Entity Recognition (NER)
@@ -65,7 +67,7 @@ For NASA SMD related, scientific usecases.
65
 
66
  ## Note
67
 
68
- Accompanying paper can be found here: https://arxiv.org/abs/2405.10725
69
 
70
 
71
  ## Citation
 
16
 
17
  # Model Card for nasa-smd-ibm-v0.1 (Indus)
18
 
19
+ nasa-smd-ibm-v0.1 (Indus) is a RoBERTa-based, Encoder-only transformer model, domain-adapted for NASA Science Mission Directorate (SMD) applications. It's fine-tuned on scientific journals and articles relevant to NASA SMD, aiming to enhance natural language technologies like information retrieval and intelligent search.
20
 
21
  ## Model Details
22
  - **Base Model**: RoBERTa
 
41
  - **Strategy**: Masked Language Modeling (MLM)
42
 
43
  ## Evaluation
 
 
 
44
 
45
+ ### BLURB benchmark
46
 
47
+ ![image/png](https://cdn-uploads.huggingface.co/production/uploads/61099e5d86580d4580767226/Ipw5xR99qUORGkn5Z8JkA.png)
48
+ (Standard deviation across 10 random seeds in parenthesis. Macro avg. reported across datasets and micro avg. computed by averaging scores on each task then averaging across task averages.)
49
 
50
+ ### Climate Change NER, and NASA-QA benchmark
51
+
52
+ ![image/png](https://cdn-uploads.huggingface.co/production/uploads/61099e5d86580d4580767226/ncTupw9tNVf6pSBD5VnOh.png)
53
+ (Climate Change NER and NASA-QA benchmark results. Standard Deviation over multiple runs given in parantheses)
54
 
55
  Please refer to the following dataset cards for further benchmarks and evaluation
56
+ - NASA-IR Benchmark - https://huggingface.co/datasets/nasa-impact/nasa-smd-IR-benchmark
57
+ - NASA-QA Benchmark - https://huggingface.co/datasets/nasa-impact/nasa-smd-qa-benchmark
58
+ - Climate Change NER Benchmark - https://huggingface.co/datasets/ibm/Climate-Change-NER
59
 
60
  ## Uses
61
  - Named Entity Recognition (NER)
 
67
 
68
  ## Note
69
 
70
+ Accompanying preprint paper can be found here: https://arxiv.org/abs/2405.10725.
71
 
72
 
73
  ## Citation