Edit model card
YAML Metadata Error: "model-index[0].results[0].metrics" is required

t5-weighter_cnndm-en

Model description

This model is a Classifier model based on T5-small, that predicts if a answer / question couple is considered as important fact or not (Is this answer enough relevant to appear in a plausible summary?). It is actually a component of QuestEval metric but can be used independently as it is.

How to use

from transformers import T5Tokenizer, T5ForConditionalGeneration

tokenizer = T5Tokenizer.from_pretrained("ThomasNLG/t5-weighter_cnndm-en")

model = T5ForConditionalGeneration.from_pretrained("ThomasNLG/t5-weighter_cnndm-en")

You can play with the model using the inference API, the text input format should follow this template (accordingly to the training stage of the model):

text_input = "{ANSWER} </s> {QUESTION} </s> {CONTEXT}"

Training data

The model was trained on synthetic data as described in Questeval: Summarization asks for fact-based evaluation.

Citation info

@article{scialom2021questeval,
  title={Questeval: Summarization asks for fact-based evaluation},
  author={Scialom, Thomas and Dray, Paul-Alexis and Gallinari, Patrick and Lamprier, Sylvain and Piwowarski, Benjamin and Staiano, Jacopo and Wang, Alex},
  journal={arXiv preprint arXiv:2103.12693},
  year={2021}
}
Downloads last month
37
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Dataset used to train ThomasNLG/t5-weighter_cnndm-en

Evaluation results

Model card error

This model's model-index metadata is invalid: Schema validation error. "model-index[0].results[0].metrics" is required