File size: 5,453 Bytes
b036bee 52ecdfe b036bee 751b93b 0e3b0ed 7e2ded8 0e3b0ed e7a525b 0e3b0ed 751b93b |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 |
---
language:
- en
- cs
license: cc-by-4.0
metrics:
- bleurt
- bleu
- bertscore
pipeline_tag: text-classification
---
# AlignScoreCS
MultiTask multilingual model for assessing facticity in various NLU tasks in Czech and English language. We followed the initial paper AlignScore https://arxiv.org/abs/2305.16739.
We trained a model using a shared architecture of checkpoint xlm-roberta-large https://huggingface.co/FacebookAI/xlm-roberta-large with three linear layers for regression,
binary classification and ternary classification.
# Usage
```python
# Assuming you copied the attached Files_and_versions/AlignScore.py file for ease of use in transformers.
from AlignScoreCS import AlignScoreCS
alignScoreCS = AlignScoreCS.from_pretrained("krotima1/AlignScoreCS")
# put the model to cuda to accelerate
print(alignScoreCS.score(context="This is context", claim="This is claim"))
```
# Results
# Training datasets
The following table shows datasets that has been utilized for training the model. We translated these english datasets to Czech using seamLessM4t.
| NLP Task | Dataset | Training Task | Context (n words) | Claim (n words) | Sample Count |
|-----------------------|-------------------|---------------|-------------------|-----------------|--------------|
| NLI | SNLI | 3-way | 10 | 13 | Cs: 500k |
| | | | | | En: 550k |
| | MultiNLI | 3-way | 16 | 20 | Cs: 393k |
| | | | | | En: 393k |
| | Adversarial NLI | 3-way | 48 | 54 | Cs: 163k |
| | | | | | En: 163k |
| | DocNLI | 2-way | 97 | 285 | Cs: 200k |
| | | | | | En: 942k |
| Fact Verification | NLI-style FEVER | 3-way | 48 | 50 | Cs: 208k |
| | | | | | En: 208k |
| | Vitamin C | 3-way | 23 | 25 | Cs: 371k |
| | | | | | En: 371k |
| Paraphrase | QQP | 2-way | 9 | 11 | Cs: 162k |
| | | | | | En: 364k |
| | PAWS | 2-way | - | 18 | Cs: - |
| | | | | | En: 707k |
| | PAWS labeled | 2-way | 18 | - | Cs: 49k |
| | | | | | En: - |
| | PAWS unlabeled | 2-way | 18 | - | Cs: 487k |
| | | | | | En: - |
| STS | SICK | reg | - | 10 | Cs: - |
| | | | | | En: 4k |
| | STS Benchmark | reg | - | 10 | Cs: - |
| | | | | | En: 6k |
| | Free-N1 | reg | 18 | - | Cs: 20k |
| | | | | | En: - |
| QA | SQuAD v2 | 2-way | 105 | 119 | Cs: 130k |
| | | | | | En: 130k |
| | RACE | 2-way | 266 | 273 | Cs: 200k |
| | | | | | En: 351k |
| Information Retrieval| MS MARCO | 2-way | 49 | 56 | Cs: 200k |
| | | | | | En: 5M |
| Summarization | WikiHow | 2-way | 434 | 508 | Cs: 157k |
| | | | | | En: 157k |
| | SumAug | 2-way | - | - | Cs: - |
| | | | | | En: - |
|