nightdessert commited on
Commit
e817f0f
1 Parent(s): d15d6ed

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +32 -3
README.md CHANGED
@@ -1,12 +1,41 @@
1
- #Factual Consistency Evaluator/Metric in ACL 2023 paper
2
 
3
  *[WeCheck: Strong Factual Consistency Checker via Weakly Supervised Learning
4
  ](https://arxiv.org/abs/2212.10057)*
5
 
6
- #Introduction
7
- WeCheck
8
 
 
 
 
 
 
 
 
 
9
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
10
  license: openrail
11
  pipeline_tag: text-classification
12
  tags:
 
1
+ # Factual Consistency Evaluator/Metric in ACL 2023 paper
2
 
3
  *[WeCheck: Strong Factual Consistency Checker via Weakly Supervised Learning
4
  ](https://arxiv.org/abs/2212.10057)*
5
 
6
+ ## Model description
7
+ WeCheck is a factual consistency metric trained from weakly annotated samples.
8
 
9
+ This WeCheck checkpoint can be used to check the following three generation tasks:
10
+
11
+ **Text Summarization/Knowlege grounded dialogue Generation/Paraphrase**
12
+
13
+ This WeCheck checkpoint is trained based on the following three weak labler:
14
+
15
+ *[QAFactEval
16
+ ](https://github.com/salesforce/QAFactEval)* / *[Summarc](https://github.com/tingofurro/summac)* / *[NLI warmup](https://huggingface.co/MoritzLaurer/DeBERTa-v3-large-mnli-fever-anli-ling-wanli)*
17
  ---
18
+
19
+ ### How to use the model
20
+ ```python
21
+ from transformers import AutoTokenizer, AutoModelForSequenceClassification
22
+ import torch
23
+
24
+ device = torch.device("cuda") if torch.cuda.is_available() else torch.device("cpu")
25
+ model_name = "nightdessert
26
+ /
27
+ WeCheck "
28
+ tokenizer = AutoTokenizer.from_pretrained(model_name)
29
+ model = AutoModelForSequenceClassification.from_pretrained(model_name)
30
+ premise = "I first thought that I liked the movie, but upon second thought it was actually disappointing."
31
+ hypothesis = "The movie was not good."
32
+ input = tokenizer(premise, hypothesis, truncation=True, return_tensors="pt")
33
+ output = model(input["input_ids"].to(device)) # device = "cuda:0" or "cpu"
34
+ prediction = torch.softmax(output["logits"][0], -1).tolist()
35
+ label_names = ["entailment", "neutral", "contradiction"]
36
+ prediction = {name: round(float(pred) * 100, 1) for pred, name in zip(prediction, label_names)}
37
+ print(prediction)
38
+
39
  license: openrail
40
  pipeline_tag: text-classification
41
  tags: