File size: 468 Bytes
8708928 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 |
---
license: mit
tags:
- nlp
- math
language:
- en
pipeline_tag: text-generation
---
## Rho-1: Not All Tokens Are What You Need
The Rho-1 series are pretrained language models that utilize Selective Language Modeling (SLM) objectives.
In math reasoning pretraining, SLM improves average few-shot accuracy on GSM8k and MATH by over 16%, achieving the baseline performance 5-10x faster.
For more details please check our [github](https://github.com/microsoft/rho).
|