|
--- |
|
tags: |
|
- bert |
|
license: cc-by-4.0 |
|
--- |
|
## bert-ascii-medium |
|
is a medium size BERT Language Model pre-trained by predicting the summation of the **ASCII** code values of the characters in a masked token as a pre-training objective. For more details about the pre-training objective and the pre-training hyperparameters, please refer to [How does the pre-training objective affect what large language models learn about linguistic properties?](https://aclanthology.org/2022.acl-short.16/) |
|
|
|
## License |
|
CC BY 4.0 |
|
|
|
## Citation |
|
If you use this model, please cite the following paper: |
|
``` |
|
@inproceedings{alajrami2022does, |
|
title={How does the pre-training objective affect what large language models learn about linguistic properties?}, |
|
author={Alajrami, Ahmed and Aletras, Nikolaos}, |
|
booktitle={Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)}, |
|
pages={131--147}, |
|
year={2022} |
|
} |
|
``` |
