mjwong commited on
Commit
0668a17
1 Parent(s): aafea2d

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +99 -0
README.md CHANGED
@@ -1,3 +1,102 @@
1
  ---
 
 
 
 
 
 
 
 
 
2
  license: mit
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ datasets:
3
+ - glue
4
+ - anli
5
+ model-index:
6
+ - name: e5-base-v2-mnli-anli
7
+ results: []
8
+ pipeline_tag: zero-shot-classification
9
+ language:
10
+ - en
11
  license: mit
12
  ---
13
+
14
+ # e5-base-v2-mnli-anli
15
+
16
+ This model is a fine-tuned version of [intfloat/e5-base-v2](https://huggingface.co/intfloat/e5-base-v2) on the glue (mnli) and anli dataset.
17
+
18
+ ## Model description
19
+
20
+ [Text Embeddings by Weakly-Supervised Contrastive Pre-training](https://arxiv.org/pdf/2212.03533.pdf).
21
+ Liang Wang, Nan Yang, Xiaolong Huang, Binxing Jiao, Linjun Yang, Daxin Jiang, Rangan Majumder, Furu Wei, arXiv 2022
22
+
23
+ ## How to use the model
24
+
25
+ ### With the zero-shot classification pipeline
26
+
27
+ The model can be loaded with the `zero-shot-classification` pipeline like so:
28
+
29
+ ```python
30
+ from transformers import pipeline
31
+ classifier = pipeline("zero-shot-classification",
32
+ model="mjwong/e5-base-v2-mnli-anli")
33
+ ```
34
+
35
+ You can then use this pipeline to classify sequences into any of the class names you specify.
36
+
37
+ ```python
38
+ sequence_to_classify = "one day I will see the world"
39
+ candidate_labels = ['travel', 'cooking', 'dancing']
40
+ classifier(sequence_to_classify, candidate_labels)
41
+ ```
42
+
43
+ If more than one candidate label can be correct, pass `multi_class=True` to calculate each class independently:
44
+
45
+ ```python
46
+ candidate_labels = ['travel', 'cooking', 'dancing', 'exploration']
47
+ classifier(sequence_to_classify, candidate_labels, multi_class=True)
48
+ ```
49
+ ### With manual PyTorch
50
+
51
+ The model can also be applied on NLI tasks like so:
52
+
53
+ ```python
54
+ import torch
55
+ from transformers import AutoTokenizer, AutoModelForSequenceClassification
56
+
57
+ # device = "cuda:0" or "cpu"
58
+ device = torch.device("cuda") if torch.cuda.is_available() else torch.device("cpu")
59
+
60
+ model_name = "mjwong/e5-base-v2-mnli-anli"
61
+ tokenizer = AutoTokenizer.from_pretrained(model_name)
62
+ model = AutoModelForSequenceClassification.from_pretrained(model_name)
63
+
64
+ premise = "But I thought you'd sworn off coffee."
65
+ hypothesis = "I thought that you vowed to drink more coffee."
66
+
67
+ input = tokenizer(premise, hypothesis, truncation=True, return_tensors="pt")
68
+ output = model(input["input_ids"].to(device))
69
+ prediction = torch.softmax(output["logits"][0], -1).tolist()
70
+ label_names = ["entailment", "neutral", "contradiction"]
71
+ prediction = {name: round(float(pred) * 100, 2) for pred, name in zip(prediction, label_names)}
72
+ print(prediction)
73
+ ```
74
+
75
+ ### Eval results
76
+ The model was evaluated using the dev sets for MultiNLI and test sets for ANLI. The metric used is accuracy.
77
+
78
+ |Datasets|mnli_dev_m|mnli_dev_mm|anli_test_r1|anli_test_r2|anli_test_r3|
79
+ | :---: | :---: | :---: | :---: | :---: | :---: |
80
+ |[e5-base-v2-mnli-anli](https://huggingface.co/mjwong/e5-base-v2-mnli-anli)|0.812|0.809|0.557|0.460|0.448|
81
+ |[e5-large-mnli](https://huggingface.co/mjwong/e5-large-mnli)|0.868|0.869|0.301|0.296|0.294|
82
+ |[e5-large-mnli-anli](https://huggingface.co/mjwong/e5-large-mnli-anli)|0.843|0.848|0.646|0.484|0.458|
83
+ |[e5-large-v2-mnli](https://huggingface.co/mjwong/e5-large-v2-mnli)|0.875|0.876|0.354|0.298|0.313|
84
+ |[e5-large-v2-mnli-anli](https://huggingface.co/mjwong/e5-large-v2-mnli-anli)|0.846|0.848|0.638|0.474|0.479|
85
+
86
+ ### Training hyperparameters
87
+
88
+ The following hyperparameters were used during training:
89
+
90
+ - learning_rate: 2e-05
91
+ - train_batch_size: 16
92
+ - eval_batch_size: 16
93
+ - seed: 42
94
+ - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
95
+ - lr_scheduler_type: linear
96
+ - lr_scheduler_warmup_ratio: 0.1
97
+
98
+ ### Framework versions
99
+ - Transformers 4.28.1
100
+ - Pytorch 1.12.1+cu116
101
+ - Datasets 2.11.0
102
+ - Tokenizers 0.12.1