|
--- |
|
license: mit |
|
widget: |
|
- text: "The closest planet to earth is <mask>." |
|
- text: "Electrical power is stored on a spacecraft with <mask>." |
|
--- |
|
|
|
### CosmicRoBERTa |
|
|
|
This model is a further pre-trained version of RoBERTa for space science on a domain-specific corpus, which includes abstracts from the NTRS library, abstracts from SCOPUS, ECSS requirements, and other sources from this domain. |
|
This totals to a pre-training corpus of around 75 mio words. |
|
|
|
The model performs slightly better on a subset (0.6 of total data set) of the CR task presented in our paper [SpaceTransformers: Language Modeling for Space Systems](https://ieeexplore.ieee.org/document/9548078). |
|
|
|
| | RoBERTa | CosmiRoBERTa | SpaceRoBERTa | |
|
|-----------------------------------------------|----------------|---------------------|---------------------| |
|
| Parameter | 0.475 | 0.515 | 0.485 | |
|
| GN&C | 0.488 | 0.609 | 0.602 | |
|
| System engineering | 0.523 | 0.559 | 0.555 | |
|
| Propulsion | 0.403 | 0.521 | 0.465 | |
|
| Project Scope | 0.493 | 0.541 | 0.497 | |
|
| OBDH | 0.717 | 0.789 | 0.794 | |
|
| Thermal | 0.432 | 0.509 | 0.491 | |
|
| Quality control | 0.686 | 0.704 | 0.678 | |
|
| Telecom. | 0.360 | 0.614 | 0.557 | |
|
| Measurement | 0.833 | 0.849 | 0.858 | |
|
| Structure & Mechanism | 0.489 | 0.581 | 0.566 | |
|
| Space Environment | 0.543 | 0.681 | 0.605 | |
|
| Cleanliness | 0.616 | 0.621 | 0.651 | |
|
| Project Organisation / Documentation | 0.355 | 0.427 | 0.429 | |
|
| Power | 0.638 | 0.735 | 0.661 | |
|
| Safety / Risk (Control) | 0.647 | 0.727 | 0.676 | |
|
| Materials / EEEs | 0.585 | 0.642 | 0.639 | |
|
| Nonconformity | 0.365 | 0.333 | 0.419 | |
|
| weighted | 0.584 | 0.652(+7%) | 0.633(+5%) | |
|
| Valid. Loss | 0.605 | 0.505 | 0.542 | |
|
|
|
|
|
### BibTeX entry and citation info |
|
|
|
``` |
|
@ARTICLE{ |
|
9548078, |
|
author={Berquand, Audrey and Darm, Paul and Riccardi, Annalisa}, |
|
journal={IEEE Access}, |
|
title={SpaceTransformers: Language Modeling for Space Systems}, |
|
year={2021}, |
|
volume={9}, |
|
number={}, |
|
pages={133111-133122}, |
|
doi={10.1109/ACCESS.2021.3115659} |
|
} |
|
``` |
