Merge branch 'main' of https://huggingface.co/bertin-project/bertin-base-gaussian-exp-512seqlen into main

README.md

@@ -0,0 +1,29 @@
+---
+language: es
+license: CC-BY 4.0
+tags:
+- spanish
+- roberta
+pipeline_tag: fill-mask
+widget:
+- text: "Fui a la librería a comprar un <mask>."
+---
+
+This is a **RoBERTa-base** model trained from scratch in Spanish.
+
+The training dataset is [mc4](https://huggingface.co/datasets/bertin-project/mc4-es-sampled ) subsampling documents to a total of about 50 million examples. Sampling is biased towards average perplexity values (using a Gaussian function), discarding more often documents with very large values (poor quality) of very small values (short, repetitive texts).
+
+This model takes the one using [sequence length 128](https://huggingface.co/bertin-project/bertin-base-gaussian) and trains during 25.000 steps using sequence length 512.
+
+This is part of the
+[Flax/Jax Community Week](https://discuss.huggingface.co/t/open-to-the-community-community-week-using-jax-flax-for-nlp-cv/7104), organised by [HuggingFace](https://huggingface.co/) and TPU usage sponsored by Google.
+
+
+## Team members
+
+- Eduardo González ([edugp](https://huggingface.co/edugp))
+- Javier de la Rosa ([versae](https://huggingface.co/versae))
+- Manu Romero ([mrm8488](https://huggingface.co/))
+- María Grandury ([mariagrandury](https://huggingface.co/))
+- Pablo González de Prado ([Pablogps](https://huggingface.co/Pablogps))
+- Paulo Villegas ([paulo](https://huggingface.co/paulo))