DCU-NLP commited on
Commit
741f560
1 Parent(s): a7fbe12

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +19 -20
README.md CHANGED
@@ -10,7 +10,7 @@ widget:
10
  ---
11
 
12
  # gaELECTRA
13
- [gaELECTRA](https://arxiv.org/abs/2107.12930) is an ELECTRA model trained on 7.9M Irish sentences. For more details, including the hyperparameters and pretraining corpora used please refer to our paper. For fine-tuning this model on a token classification task, e.g. Named Entity Recognition, use the discriminator model.
14
 
15
  ### Limitations and bias
16
  Some data used to pretrain gaBERT was scraped from the web which potentially contains ethically problematic text (bias, hate, adult content, etc.). Consequently, downstream tasks/applications using gaBERT should be thoroughly tested with respect to ethical considerations.
@@ -20,24 +20,23 @@ Some data used to pretrain gaBERT was scraped from the web which potentially con
20
  If you use this model in your research, please consider citing our paper:
21
 
22
  ```
23
- @article{DBLP:journals/corr/abs-2107-12930,
24
- author = {James Barry and
25
- Joachim Wagner and
26
- Lauren Cassidy and
27
- Alan Cowap and
28
- Teresa Lynn and
29
- Abigail Walsh and
30
- M{\'{\i}}che{\'{a}}l J. {\'{O}} Meachair and
31
- Jennifer Foster},
32
- title = {gaBERT - an Irish Language Model},
33
- journal = {CoRR},
34
- volume = {abs/2107.12930},
35
- year = {2021},
36
- url = {https://arxiv.org/abs/2107.12930},
37
- archivePrefix = {arXiv},
38
- eprint = {2107.12930},
39
- timestamp = {Fri, 30 Jul 2021 13:03:06 +0200},
40
- biburl = {https://dblp.org/rec/journals/corr/abs-2107-12930.bib},
41
- bibsource = {dblp computer science bibliography, https://dblp.org}
42
  }
43
  ```
 
10
  ---
11
 
12
  # gaELECTRA
13
+ [gaELECTRA](https://aclanthology.org/2022.lrec-1.511/) is an ELECTRA model trained on 7.9M Irish sentences. For more details, including the hyperparameters and pretraining corpora used please refer to our paper. For fine-tuning this model on a token classification task, e.g. Named Entity Recognition, use the discriminator model.
14
 
15
  ### Limitations and bias
16
  Some data used to pretrain gaBERT was scraped from the web which potentially contains ethically problematic text (bias, hate, adult content, etc.). Consequently, downstream tasks/applications using gaBERT should be thoroughly tested with respect to ethical considerations.
 
20
  If you use this model in your research, please consider citing our paper:
21
 
22
  ```
23
+ @inproceedings{barry-etal-2022-gabert,
24
+ title = "ga{BERT} {---} an {I}rish Language Model",
25
+ author = "Barry, James and
26
+ Wagner, Joachim and
27
+ Cassidy, Lauren and
28
+ Cowap, Alan and
29
+ Lynn, Teresa and
30
+ Walsh, Abigail and
31
+ {\'O} Meachair, M{\'\i}che{\'a}l J. and
32
+ Foster, Jennifer",
33
+ booktitle = "Proceedings of the Thirteenth Language Resources and Evaluation Conference",
34
+ month = jun,
35
+ year = "2022",
36
+ address = "Marseille, France",
37
+ publisher = "European Language Resources Association",
38
+ url = "https://aclanthology.org/2022.lrec-1.511",
39
+ pages = "4774--4788",
40
+ abstract = "The BERT family of neural language models have become highly popular due to their ability to provide sequences of text with rich context-sensitive token encodings which are able to generalise well to many NLP tasks. We introduce gaBERT, a monolingual BERT model for the Irish language. We compare our gaBERT model to multilingual BERT and the monolingual Irish WikiBERT, and we show that gaBERT provides better representations for a downstream parsing task. We also show how different filtering criteria, vocabulary size and the choice of subword tokenisation model affect downstream performance. We compare the results of fine-tuning a gaBERT model with an mBERT model for the task of identifying verbal multiword expressions, and show that the fine-tuned gaBERT model also performs better at this task. We release gaBERT and related code to the community.",
 
41
  }
42
  ```