|
--- |
|
license: mit |
|
language: |
|
- en |
|
--- |
|
# MacBERTh |
|
|
|
This model is a Historical Language Model for English coming from the [MacBERTh project](https://macberth.netlify.app/). |
|
|
|
The architecture is based on BERT base uncased from the original BERT pre-training codebase. |
|
The training material comes from different sources including: |
|
|
|
- EEBO |
|
- ECCO |
|
- COHA |
|
- CLMET3.1 |
|
- EVANS |
|
- Hansard Corpus |
|
|
|
with a total word count of approximately 3.9B tokens. |
|
|
|
Details and evaluation can be found in the accompanying publications: |
|
- [MacBERTh: Development and Evaluation of a Historically Pre-trained Language Model for English (1450-1950)](https://aclanthology.org/2021.nlp4dh-1.4/) |
|
- [Adapting vs. Pre-training Language Models for Historical Languages](https://doi.org/10.46298/jdmdh.9152) |
