flax-community
/

roberta-base-als

Inference Endpoints

Model card Files Files and versions Metrics Training metrics Community

roberta-base-als / README.md

patrickvonplaten's picture

patrickvonplaten

Create README.md

fb1e3e0 almost 3 years ago

|

raw history blame contribute delete

No virus

248 Bytes

This project pretrains a roberta-base on the Alemannic (als) data subset of the OSCAR corpus in JAX/Flax.

We will be using the masked-language modeling loss for pretraining.