DCU-NLP's picture
Update README.md
63f70ee
|
raw
history blame
1.06 kB
metadata
tags:
  - generated_from_keras_callback
model-index:
  - name: bert-base-irish-cased-v1
    results: []
widget:
  - text: Ceolt贸ir [MASK] ab ea Johnny Cash.

bert-base-irish-cased-v1

gaBERT is a BERT-base model trained on 7.9M Irish sentences. For more details, including the hyperparameters and pretraining corpora used please refer to our paper.

Model description

Encoder-based Transformer to be used to obtain features for finetuning for downstream tasks in Irish.

Intended uses & limitations

Some data used to pretrain gaBERT was scraped from the web which potentially contains ethically problematic text (bias, hate, adult content, etc.). Consequently, downstream tasks/applications using gaBERT should be thoroughly tested with respect to ethical considerations.

Training hyperparameters

The following hyperparameters were used during training:

  • optimizer: None
  • training_precision: float32

Framework versions

  • Transformers 4.20.1
  • TensorFlow 2.9.1
  • Datasets 2.3.2
  • Tokenizers 0.12.1