Update README.md
Browse files
README.md
CHANGED
@@ -18,13 +18,16 @@ multilinguality:
|
|
18 |
license: cc-by-sa-4.0
|
19 |
inference: false
|
20 |
---
|
|
|
21 |
|
22 |
This learned metric is for evaluating models trained on the TaTA dataset. It was trained as per instructions in [TaTA: A Multilingual Table-to-Text Dataset for African Languages](https://aclanthology.org/2023.findings-emnlp.118/) (StATA-QE variant).
|
23 |
|
24 |
-
It achieves an RMSE loss of 0.41 on the dev split, and a Pearson correlation of 0.23 with human evaluations on the test split ("attributable" column of [this dataset](https://huggingface.co/datasets/adenhaus/stata)).
|
25 |
-
|
26 |
The original file can be found [here](https://github.com/google-research/url-nlp/tree/main/tata).
|
27 |
|
|
|
|
|
|
|
|
|
28 |
[Here](https://huggingface.co/adenhaus/mt5-large-stata/blob/main/README.md) is a version trained with mT5-Large instead of mT5-Small, which achieves a correlation of 0.59.
|
29 |
|
30 |
# Example use
|
|
|
18 |
license: cc-by-sa-4.0
|
19 |
inference: false
|
20 |
---
|
21 |
+
# Background
|
22 |
|
23 |
This learned metric is for evaluating models trained on the TaTA dataset. It was trained as per instructions in [TaTA: A Multilingual Table-to-Text Dataset for African Languages](https://aclanthology.org/2023.findings-emnlp.118/) (StATA-QE variant).
|
24 |
|
|
|
|
|
25 |
The original file can be found [here](https://github.com/google-research/url-nlp/tree/main/tata).
|
26 |
|
27 |
+
# Performance
|
28 |
+
|
29 |
+
It achieves an RMSE loss of 0.41 on the dev split, and a Pearson correlation of 0.23 with human evaluations on the test split ("attributable" column of [this dataset](https://huggingface.co/datasets/adenhaus/stata)).
|
30 |
+
|
31 |
[Here](https://huggingface.co/adenhaus/mt5-large-stata/blob/main/README.md) is a version trained with mT5-Large instead of mT5-Small, which achieves a correlation of 0.59.
|
32 |
|
33 |
# Example use
|