simonhughes22 commited on
Commit
9afe510
1 Parent(s): 1b467f3

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +39 -0
README.md CHANGED
@@ -1,3 +1,42 @@
1
  ---
2
  license: apache-2.0
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: apache-2.0
3
  ---
4
+ # Cross-Encoder for Hallucination Detection
5
+ This model was trained using [SentenceTransformers](https://sbert.net) [Cross-Encoder](https://www.sbert.net/examples/applications/cross-encoder/README.html) class. This model is based on [microsoft/deberta-v3-base](https://huggingface.co/microsoft/deberta-v3-base).
6
+
7
+ ## Training Data
8
+ The model was trained on the NLI data and a variety of datasets evaluating summarization accuracy for factual consistency, including [FEVER](https://huggingface.co/datasets/fever), [Vitamin C](https://huggingface.co/datasets/tals/vitaminc) and [PAWS](https://huggingface.co/datasets/paws).
9
+
10
+ ## Performance
11
+ TODO
12
+ ## Usage
13
+
14
+ Pre-trained models can be used like this:
15
+ ```python
16
+ from sentence_transformers import CrossEncoder
17
+ model = CrossEncoder('cross-encoder/nli-deberta-v3-large')
18
+ scores = model.predict([('A man is eating pizza', 'A man eats something'), ('A black race car starts up in front of a crowd of people.', 'A man is driving down a lonely road.')])
19
+
20
+ #Convert scores to labels
21
+ label_mapping = ['contradiction', 'entailment', 'neutral']
22
+ labels = [label_mapping[score_max] for score_max in scores.argmax(axis=1)]
23
+ ```
24
+
25
+ ## Usage with Transformers AutoModel
26
+ You can use the model also directly with Transformers library (without SentenceTransformers library):
27
+ ```python
28
+ from transformers import AutoTokenizer, AutoModelForSequenceClassification
29
+ import torch
30
+
31
+ model = AutoModelForSequenceClassification.from_pretrained('cross-encoder/nli-deberta-v3-large')
32
+ tokenizer = AutoTokenizer.from_pretrained('cross-encoder/nli-deberta-v3-large')
33
+
34
+ features = tokenizer(['A man is eating pizza', 'A black race car starts up in front of a crowd of people.'], ['A man eats something', 'A man is driving down a lonely road.'], padding=True, truncation=True, return_tensors="pt")
35
+
36
+ model.eval()
37
+ with torch.no_grad():
38
+ scores = model(**features).logits
39
+ label_mapping = ['contradiction', 'entailment', 'neutral']
40
+ labels = [label_mapping[score_max] for score_max in scores.argmax(dim=1)]
41
+ print(labels)
42
+ ```