|
--- |
|
tags: |
|
- generated_from_keras_callback |
|
model-index: |
|
- name: bert-base-irish-cased-v1 |
|
results: [] |
|
widget: |
|
- text: "Ceolt贸ir [MASK] ab ea Johnny Cash." |
|
--- |
|
|
|
|
|
# bert-base-irish-cased-v1 |
|
|
|
[gaBERT](https://arxiv.org/abs/2107.12930) is a BERT-base model trained on 7.9M Irish sentences. For more details, including the hyperparameters and pretraining corpora used please refer to our paper. |
|
|
|
## Model description |
|
|
|
Encoder-based Transformer to be used to obtain features for finetuning for downstream tasks in Irish. |
|
|
|
## Intended uses & limitations |
|
|
|
Some data used to pretrain gaBERT was scraped from the web which potentially contains ethically problematic text (bias, hate, adult content, etc.). Consequently, downstream tasks/applications using gaBERT should be thoroughly tested with respect to ethical considerations. |
|
|
|
|
|
### Training hyperparameters |
|
|
|
The following hyperparameters were used during training: |
|
- optimizer: None |
|
- training_precision: float32 |
|
|
|
|
|
|
|
### Framework versions |
|
|
|
- Transformers 4.20.1 |
|
- TensorFlow 2.9.1 |
|
- Datasets 2.3.2 |
|
- Tokenizers 0.12.1 |
|
|
