metadata
tags:
- generated_from_keras_callback
model-index:
- name: bert-base-irish-cased-v1
results: []
widget:
- text: Ceolt贸ir [MASK] ab ea Johnny Cash.
bert-base-irish-cased-v1
gaBERT is a BERT-base model trained on 7.9M Irish sentences. For more details, including the hyperparameters and pretraining corpora used please refer to our paper.
Model description
Encoder-based Transformer to be used to obtain features for finetuning for downstream tasks in Irish.
Intended uses & limitations
Some data used to pretrain gaBERT was scraped from the web which potentially contains ethically problematic text (bias, hate, adult content, etc.). Consequently, downstream tasks/applications using gaBERT should be thoroughly tested with respect to ethical considerations.
Training hyperparameters
The following hyperparameters were used during training:
- optimizer: None
- training_precision: float32
Framework versions
- Transformers 4.20.1
- TensorFlow 2.9.1
- Datasets 2.3.2
- Tokenizers 0.12.1