Update README.md

README.md

@@ -1,3 +1,95 @@
 ---
+datasets:
+- glue
+- anli
+model-index:
+- name: bge-large-en-mnli-anli
+  results: []
+pipeline_tag: zero-shot-classification
+language:
+- en
 license: mit
 ---
+
+# bge-large-en-mnli-anli
+
+This model is a fine-tuned version of [BAAI/bge-large-en](https://huggingface.co/BAAI/bge-large-en) on the glue and ANLI dataset.
+
+## Model description
+
+[RetroMAE: Pre-Training Retrieval-oriented Language Models Via Masked Auto-Encoder](https://arxiv.org/abs/2205.12035).
+Shitao Xiao, Zheng Liu, Yingxia Shao, Zhao Cao, arXiv 2022
+
+## How to use the model
+
+### With the zero-shot classification pipeline
+
+The model can be loaded with the `zero-shot-classification` pipeline like so:
+
+```python
+from transformers import pipeline
+classifier = pipeline("zero-shot-classification",
+                      model="mjwong/bge-large-en-mnli-anli")
+```
+
+You can then use this pipeline to classify sequences into any of the class names you specify.
+
+```python
+sequence_to_classify = "one day I will see the world"
+candidate_labels = ['travel', 'cooking', 'dancing']
+classifier(sequence_to_classify, candidate_labels)
+```
+
+If more than one candidate label can be correct, pass `multi_class=True` to calculate each class independently:
+
+```python
+candidate_labels = ['travel', 'cooking', 'dancing', 'exploration']
+classifier(sequence_to_classify, candidate_labels, multi_class=True)
+```
+
+### With manual PyTorch
+
+The model can also be applied on NLI tasks like so:
+
+```python
+import torch
+from transformers import AutoTokenizer, AutoModelForSequenceClassification
+# device = "cuda:0" or "cpu"
+device = torch.device("cuda") if torch.cuda.is_available() else torch.device("cpu")
+model_name = "mjwong/bge-large-en-mnli-anli"
+tokenizer = AutoTokenizer.from_pretrained(model_name)
+model = AutoModelForSequenceClassification.from_pretrained(model_name)
+premise = "But I thought you'd sworn off coffee."
+hypothesis = "I thought that you vowed to drink more coffee."
+input = tokenizer(premise, hypothesis, truncation=True, return_tensors="pt")
+output = model(input["input_ids"].to(device))
+prediction = torch.softmax(output["logits"][0], -1).tolist()
+label_names = ["entailment", "neutral", "contradiction"]
+prediction = {name: round(float(pred) * 100, 2) for pred, name in zip(prediction, label_names)}
+print(prediction)
+```
+
+### Eval results
+The model was also evaluated using the dev sets for MultiNLI and test sets for ANLI. The metric used is accuracy.
+
+|Datasets|mnli_dev_m|mnli_dev_mm|anli_test_r1|anli_test_r2|anli_test_r3|
+| :---: | :---: | :---: | :---: | :---: | :---: |
+|[bge-large-en-mnli-anli](https://huggingface.co/mjwong/bge-large-en-mnli-anli)|0.846|0.842|0.602|0.451|0.452|
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+
+- learning_rate: 2e-05
+- train_batch_size: 16
+- eval_batch_size: 16
+- seed: 42
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: linear
+- lr_scheduler_warmup_ratio: 0.1
+
+### Framework versions
+- Transformers 4.28.1
+- Pytorch 2.0.1+cu118
+- Datasets 2.11.0
+- Tokenizers 0.13.3