|
--- |
|
language: en |
|
license: apache-2.0 |
|
datasets: |
|
- wikipedia |
|
--- |
|
|
|
# ALBERT Large (CDA) - Counterfactual Data Augmentation |
|
|
|
Pretrained model on English language using a masked language modeling (MLM) objective. It was introduced |
|
in [this paper](https://arxiv.org/abs/1810.04805) and first released |
|
in [this repository](https://github.com/google-research-datasets/Zari). The model is pre-trained from scratch over |
|
Wikipedia. Word substitutions for data augmentation are determined using the word lists provided |
|
at [corefBias](https://github.com/uclanlp/corefBias) ([Zhao et al. (2018)](https://arxiv.org/abs/1804.06876)). |
|
|
|
Disclaimer: The team releasing ALBERT did not write a model card for this model so this model card has been written by |
|
the FairNLP team. |
|
|
|
|
|
### BibTeX entry and citation info |
|
|
|
``` |
|
@misc{zari, |
|
title={Measuring and Reducing Gendered Correlations in Pre-trained Models}, |
|
author={Kellie Webster and Xuezhi Wang and Ian Tenney and Alex Beutel and Emily Pitler and Ellie Pavlick and Jilin Chen and Slav Petrov}, |
|
year={2020}, |
|
eprint={2010.06032}, |
|
archivePrefix={arXiv}, |
|
primaryClass={cs.CL} |
|
} |
|
``` |
