Edit model card


gaBERT is a BERT-base model trained on 7.9M Irish sentences. For more details, including the hyperparameters and pretraining corpora used please refer to our paper.

Model description

Encoder-based Transformer to be used to obtain features for finetuning for downstream tasks in Irish.

Intended uses & limitations

Some data used to pretrain gaBERT was scraped from the web which potentially contains ethically problematic text (bias, hate, adult content, etc.). Consequently, downstream tasks/applications using gaBERT should be thoroughly tested with respect to ethical considerations.

Training hyperparameters

The following hyperparameters were used during training:

  • optimizer: None
  • training_precision: float32

Framework versions

  • Transformers 4.20.1
  • TensorFlow 2.9.1
  • Datasets 2.3.2
  • Tokenizers 0.12.1
Downloads last month
Hosted inference API
Mask token: [MASK]
This model can be loaded on the Inference API on-demand.