---
datasets:
- c4
language:
- en
metrics:
- accuracy
pipeline_tag: fill-mask
---

A small version of `DeBERTa` trained on the clean version of google C4 dataset. For more info about the size of the model, see `config.json`.

The model has been trained for **100K** steps with a batch size of **2048** and a sequence length of **512**, for a total of **104B** tokens.

The vocabulary and the tokenizer are the same as `microsoft/deberta-base`.