Muthukumaran
commited on
Update README.md
Browse files
README.md
CHANGED
@@ -16,7 +16,7 @@ datasets:
|
|
16 |
|
17 |
# Model Card for nasa-smd-ibm-v0.1 (Indus)
|
18 |
|
19 |
-
nasa-smd-ibm-v0.1 (
|
20 |
|
21 |
## Model Details
|
22 |
- **Base Model**: RoBERTa
|
@@ -41,19 +41,21 @@ nasa-smd-ibm-v0.1 (Currently named as Indus) is a RoBERTa-based, Encoder-only tr
|
|
41 |
- **Strategy**: Masked Language Modeling (MLM)
|
42 |
|
43 |
## Evaluation
|
44 |
-
- BLURB Benchmark
|
45 |
-
- Pruned SQuAD2.0 (SQ2) Benchmark (Amazon Rainforest, Oxygen, Geology and NASA ES QAs)
|
46 |
-
- NASA SMD Expert QA Benchmark (WIP)
|
47 |
|
|
|
48 |
|
49 |
-
![image/png](https://cdn-uploads.huggingface.co/production/uploads/61099e5d86580d4580767226/
|
|
|
50 |
|
51 |
-
|
|
|
|
|
|
|
52 |
|
53 |
Please refer to the following dataset cards for further benchmarks and evaluation
|
54 |
-
- NASA
|
55 |
-
- NASA
|
56 |
-
- Climate
|
57 |
|
58 |
## Uses
|
59 |
- Named Entity Recognition (NER)
|
@@ -65,7 +67,7 @@ For NASA SMD related, scientific usecases.
|
|
65 |
|
66 |
## Note
|
67 |
|
68 |
-
Accompanying paper can be found here: https://arxiv.org/abs/2405.10725
|
69 |
|
70 |
|
71 |
## Citation
|
|
|
16 |
|
17 |
# Model Card for nasa-smd-ibm-v0.1 (Indus)
|
18 |
|
19 |
+
nasa-smd-ibm-v0.1 (Indus) is a RoBERTa-based, Encoder-only transformer model, domain-adapted for NASA Science Mission Directorate (SMD) applications. It's fine-tuned on scientific journals and articles relevant to NASA SMD, aiming to enhance natural language technologies like information retrieval and intelligent search.
|
20 |
|
21 |
## Model Details
|
22 |
- **Base Model**: RoBERTa
|
|
|
41 |
- **Strategy**: Masked Language Modeling (MLM)
|
42 |
|
43 |
## Evaluation
|
|
|
|
|
|
|
44 |
|
45 |
+
### BLURB benchmark
|
46 |
|
47 |
+
![image/png](https://cdn-uploads.huggingface.co/production/uploads/61099e5d86580d4580767226/Ipw5xR99qUORGkn5Z8JkA.png)
|
48 |
+
(Standard deviation across 10 random seeds in parenthesis. Macro avg. reported across datasets and micro avg. computed by averaging scores on each task then averaging across task averages.)
|
49 |
|
50 |
+
### Climate Change NER, and NASA-QA benchmark
|
51 |
+
|
52 |
+
![image/png](https://cdn-uploads.huggingface.co/production/uploads/61099e5d86580d4580767226/ncTupw9tNVf6pSBD5VnOh.png)
|
53 |
+
(Climate Change NER and NASA-QA benchmark results. Standard Deviation over multiple runs given in parantheses)
|
54 |
|
55 |
Please refer to the following dataset cards for further benchmarks and evaluation
|
56 |
+
- NASA-IR Benchmark - https://huggingface.co/datasets/nasa-impact/nasa-smd-IR-benchmark
|
57 |
+
- NASA-QA Benchmark - https://huggingface.co/datasets/nasa-impact/nasa-smd-qa-benchmark
|
58 |
+
- Climate Change NER Benchmark - https://huggingface.co/datasets/ibm/Climate-Change-NER
|
59 |
|
60 |
## Uses
|
61 |
- Named Entity Recognition (NER)
|
|
|
67 |
|
68 |
## Note
|
69 |
|
70 |
+
Accompanying preprint paper can be found here: https://arxiv.org/abs/2405.10725.
|
71 |
|
72 |
|
73 |
## Citation
|