flax-community
/

roberta-base-als

Inference Endpoints

Model card Files Files and versions Metrics Training metrics Community

roberta-base-als / README.md

patrickvonplaten's picture

patrickvonplaten

Create README.md

fb1e3e0 almost 3 years ago

|

raw history blame contribute delete

No virus

248 Bytes

	This project pretrains a [`roberta-base`](https://huggingface.co/roberta-base) on the Alemannic (`als`) data subset of the [OSCAR](https://oscar-corpus.com/) corpus in JAX/Flax.

	We will be using the masked-language modeling loss for pretraining.