File size: 2,344 Bytes
c7750ee |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 |
---
language:
- multilingual
- grc
- en
- la
license: apache-2.0
inference: false
---
# PhilBerta
The paper [Exploring Language Models for Classical Philology](https://todo.com) is the first effort to systematically provide state-of-the-art language models for Classical Philology. PhilBerta is a RoBerta-base sized, multilingual, encoder-only variant.
This model was trained using data from the [Open Greek & Latin Project](https://opengreekandlatin.org/), the CLARIN corpus [Greek Medieval Texts](https://inventory.clarin.gr/corpus/890), the [Patrologia Graeca](https://patristica.net/graeca/), the [Corpus Corporum](https://mlat.uzh.ch/), and [Project Gutenberg](https://www.gutenberg.org/).
Further information can be found in our paper or in our [GitHub repository](https://github.com/Heidelberg-NLP/ancient-language-models).
## Usage
```python
from transformers import AutoTokenizer, AutoModelForMaskedLM
tokenizer = AutoTokenizer.from_pretrained('bowphs/PhilBerta')
model = AutoModelForMaskedLM.from_pretrained('bowphs/PhilBerta')
```
Please check out the awesome Hugging Face tutorials on how to fine-tune our models.
## Evaluation Results
When fine-tuned on data from [Universal Dependencies 2.10](https://universaldependencies.org/), PhilBerta achieves the following results on the Ancient Greek Perseus dataset:
| Task | XPoS | UPoS | UAS | LAS |
|:--:|:--:|:--:|:--:|:--:|
| |95.60|90.41|86.99|82.69|
When fine-tuned on PoS data from [EvaLatin 2022](https://universaldependencies.org/), it achieves the following results:
| Task | Classical | Cross-genre | Cross-time |
|:--:|:--:|:--:|:--:|
| |98.23|96.59|93.25|
## Contact
If you have any questions or problems, feel free to [reach out](mailto:riemenschneider@cl.uni-heidelberg.de).
## Citation
```bibtex
@incollection{riemenschneiderfrank:2023,
address = "Toronto, Canada",
author = "Riemenschneider, Frederick and Frank, Anette",
booktitle = "Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (ACL’23)",
note = "to appear",
pubType = "incollection",
publisher = "Association for Computational Linguistics",
title = "Exploring Large Language Models for Classical Philology",
url = "https://arxiv.org/abs/2305.13698",
year = "2023",
key = "riemenschneiderfrank:2023"
}
```
|