File size: 3,386 Bytes
53c0c2a fa86202 e0600a7 53c0c2a 15390e9 53c0c2a 99fd96c 53c0c2a 62d918e 53c0c2a 2b053b6 53c0c2a b86f54f 53c0c2a b86f54f 53c0c2a 7869c42 b86f54f 53c0c2a db490b1 53c0c2a |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 |
---
license: mit
datasets:
- web_nlg
language:
- en
widget:
- text: "Bourg-la-Reine is located in France and I love this town. I'm from People's Republic of China. [SEP] A Chinese, Loves, Bourg-la-Reine"
- text: "Bucharest is a city in Romania. [SEP] Romania | is located in | Bucharest"
---
# Model card for Inria-CEDAR/FactSpotter-DeBERTaV3-Small
## Model description
This model is related to the paper **"FactSpotter: Evaluating the Factual Faithfulness of Graph-to-Text Generation"**.
Given a triple of format "subject, predicate, object" and a text, the model determines if the triple is present in the text.
The delimiter can be ", " or " | ".
Different from the paper using ELECTRA, this model is finetuned on DeBERTaV3.
We also provide Base and Large versions of this model:
https://huggingface.co/Inria-CEDAR/FactSpotter-DeBERTaV3-Base
https://huggingface.co/Inria-CEDAR/FactSpotter-DeBERTaV3-Large
## How to use the model
```python
from transformers import AutoTokenizer, AutoModelForSequenceClassification
def sentence_cls_score(input_strings, predicate_cls_model, predicate_cls_tokenizer):
tokenized_cls_input = predicate_cls_tokenizer(input_strings, truncation=True, padding=True,
return_token_type_ids=True)
input_ids = torch.Tensor(tokenized_cls_input['input_ids']).long().to(torch.device("cuda"))
token_type_ids = torch.Tensor(tokenized_cls_input['token_type_ids']).long().to(torch.device("cuda"))
attention_mask = torch.Tensor(tokenized_cls_input['attention_mask']).long().to(torch.device("cuda"))
prev_cls_output = predicate_cls_model(input_ids, attention_mask=attention_mask, token_type_ids=token_type_ids)
softmax_cls_output = torch.softmax(prev_cls_output.logits, dim=1, )
return softmax_cls_output
tokenizer = AutoTokenizer.from_pretrained("Inria-CEDAR/FactSpotter-DeBERTaV3-Small")
model = AutoModelForSequenceClassification.from_pretrained("Inria-CEDAR/FactSpotter-DeBERTaV3-Small")
model.to(torch.device("cuda"))
# pairs of texts (as premises) and triples (as hypotheses)
cls_texts = [("the aarhus is the airport of aarhus, denmark", "aarhus airport, city served, aarhus, denmark"),
("aarhus airport is 25.0 metres above the sea level", "aarhus airport, elevation above the sea level, 1174")]
cls_scores = sentence_cls_score(cls_texts, model, tokenizer)
# Dimensions: 0-entailment, 1-neutral, 2-contradiction
label_names = ["entailment", "neutral", "contradiction"]
```
## Citation
If the model is useful to you, please cite the paper
```
@inproceedings{zhang:hal-04257838,
TITLE = {{FactSpotter: Evaluating the Factual Faithfulness of Graph-to-Text Generation}},
AUTHOR = {Zhang, Kun and Balalau, Oana and Manolescu, Ioana},
URL = {https://hal.science/hal-04257838},
BOOKTITLE = {{Findings of EMNLP 2023 - Conference on Empirical Methods in Natural Language Processing}},
ADDRESS = {Singapore, Singapore},
YEAR = {2023},
MONTH = Dec,
KEYWORDS = {Graph-to-Text Generation ; Factual Faithfulness ; Constrained Text Generation},
PDF = {https://hal.science/hal-04257838/file/_EMNLP_2023__Evaluating_the_Factual_Faithfulness_of_Graph_to_Text_Generation_Camera.pdf},
HAL_ID = {hal-04257838},
HAL_VERSION = {v1},
}
```
## Questions
If you have some questions, please contact through my email zhangkun@ieee.org or kun.zhang@inria.fr
|