---
tags:
- generated_from_keras_callback
model-index:
- name: bert-base-irish-cased-v1
  results: []
widget:
- text: "Ceoltóir [MASK] ab ea Johnny Cash."
---


# bert-base-irish-cased-v1

[gaBERT](https://arxiv.org/abs/2107.12930) is a BERT-base model trained on 7.9M Irish sentences. For more details, including the hyperparameters and pretraining corpora used please refer to our paper.

## Model description

Encoder-based Transformer to be used to obtain features for finetuning for downstream tasks in Irish.

## Intended uses & limitations

Some data used to pretrain gaBERT was scraped from the web which potentially contains ethically problematic text (bias, hate, adult content, etc.). Consequently, downstream tasks/applications using gaBERT should be thoroughly tested with respect to ethical considerations.


### Training hyperparameters

The following hyperparameters were used during training:
- optimizer: None
- training_precision: float32



### Framework versions

- Transformers 4.20.1
- TensorFlow 2.9.1
- Datasets 2.3.2
- Tokenizers 0.12.1