|
--- |
|
license: apache-2.0 |
|
datasets: |
|
- allenai/SciRIFF-train-mix |
|
- allenai/tulu-v2-sft-mixture |
|
language: |
|
- en |
|
--- |
|
# Model Card for SciTulu 70B |
|
|
|
SciTulu is a collection of instruction-following language models targeting scientific literature understanding use cases. Starting from the [Tulu v2 70B](https://huggingface.co/allenai/tulu-2-70b) model, SciTulu is trained on a mix of science-specific demonstrations from the [SciRIFF dataset](https://huggingface.co/datasets/allenai/SciRIFF-train-mix), together with general-domain instructions from the [Tulu v2 SFT mix](https://huggingface.co/datasets/allenai/tulu-v2-sft-mixture). SciTulu 70B achives a 6.5% average improvement over Tulu v2 70B on nine held-out scientific literature understanding tasks. More information can be found in our preprint: [SciRIFF: A Resource to Enhance Language Model Instruction-Following over Scientific Literature](https://arxiv.org/abs/2406.07835). |
|
|
|
Training and evaluation code for SciTulu is available in our GitHub repository: https://github.com/allenai/SciRIFF. |
|
|
|
See the [Tulu model card](https://huggingface.co/allenai/tulu-2-70b) for more information on potential risks, biases, and limitations. |
|
|
