Upload README.md

README.md

@@ -1,43 +1,85 @@
 ---
+license: apache-2.0
+pipeline_tag: text-generation
+tags:
+- finetuned
 inference: false
 ---
-# Vicuna Model Card
 
-## Model Details
+# Model Card for Mistral-7B-Instruct-v0.2
 
-Vicuna is a chat assistant trained by fine-tuning LLaMA on user-shared conversations collected from ShareGPT.
+The Mistral-7B-Instruct-v0.2 Large Language Model (LLM) is an improved instruct fine-tuned version of [Mistral-7B-Instruct-v0.1](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.1).
 
-- **Developed by:** [LMSYS](https://lmsys.org/)
-- **Model type:** An auto-regressive language model based on the transformer architecture.
-- **License:** Non-commercial license
-- **Finetuned from model:** [LLaMA](https://arxiv.org/abs/2302.13971).
+For full details of this model please read our [paper](https://arxiv.org/abs/2310.06825) and [release blog post](https://mistral.ai/news/la-plateforme/).
 
-### Model Sources
+## Instruction format
 
-- **Repository:** https://github.com/lm-sys/FastChat
-- **Blog:** https://lmsys.org/blog/2023-03-30-vicuna/
-- **Paper:** https://arxiv.org/abs/2306.05685
-- **Demo:** https://chat.lmsys.org/
+In order to leverage instruction fine-tuning, your prompt should be surrounded by `[INST]` and `[/INST]` tokens. The very first instruction should begin with a begin of sentence id. The next instructions should not. The assistant generation will be ended by the end-of-sentence token id.
 
-## Uses
+E.g.
+```
+text = "<s>[INST] What is your favourite condiment? [/INST]"
+"Well, I'm quite partial to a good squeeze of fresh lemon juice. It adds just the right amount of zesty flavour to whatever I'm cooking up in the kitchen!</s> "
+"[INST] Do you have mayonnaise recipes? [/INST]"
+```
 
-The primary use of Vicuna is research on large language models and chatbots.
-The primary intended users of the model are researchers and hobbyists in natural language processing, machine learning, and artificial intelligence.
+This format is available as a [chat template](https://huggingface.co/docs/transformers/main/chat_templating) via the `apply_chat_template()` method:
 
-## How to Get Started with the Model
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
 
-- Command line interface: https://github.com/lm-sys/FastChat#vicuna-weights.  
-- APIs (OpenAI API, Huggingface API): https://github.com/lm-sys/FastChat/tree/main#api.  
+device = "cuda" # the device to load the model onto
 
-## Training Details
+model = AutoModelForCausalLM.from_pretrained("mistralai/Mistral-7B-Instruct-v0.2")
+tokenizer = AutoTokenizer.from_pretrained("mistralai/Mistral-7B-Instruct-v0.2")
 
-Vicuna v1.3 is fine-tuned from LLaMA with supervised instruction fine-tuning.
-The training data is around 125K conversations collected from ShareGPT.com.
-See more details in the "Training Details of Vicuna Models" section in the appendix of this [paper](https://arxiv.org/pdf/2306.05685.pdf).
+messages = [
+    {"role": "user", "content": "What is your favourite condiment?"},
+    {"role": "assistant", "content": "Well, I'm quite partial to a good squeeze of fresh lemon juice. It adds just the right amount of zesty flavour to whatever I'm cooking up in the kitchen!"},
+    {"role": "user", "content": "Do you have mayonnaise recipes?"}
+]
 
-## Evaluation
+encodeds = tokenizer.apply_chat_template(messages, return_tensors="pt")
 
-Vicuna is evaluated with standard benchmarks, human preference, and LLM-as-a-judge. See more details in this [paper](https://arxiv.org/pdf/2306.05685.pdf) and [leaderboard](https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboard).
+model_inputs = encodeds.to(device)
+model.to(device)
 
-## Difference between different versions of Vicuna
-See [vicuna_weights_version.md](https://github.com/lm-sys/FastChat/blob/main/docs/vicuna_weights_version.md)
+generated_ids = model.generate(model_inputs, max_new_tokens=1000, do_sample=True)
+decoded = tokenizer.batch_decode(generated_ids)
+print(decoded[0])
+```
+
+## Model Architecture
+This instruction model is based on Mistral-7B-v0.1, a transformer model with the following architecture choices:
+- Grouped-Query Attention
+- Sliding-Window Attention
+- Byte-fallback BPE tokenizer
+
+## Troubleshooting
+- If you see the following error:
+```
+Traceback (most recent call last):
+File "", line 1, in
+File "/transformers/models/auto/auto_factory.py", line 482, in from_pretrained
+config, kwargs = AutoConfig.from_pretrained(
+File "/transformers/models/auto/configuration_auto.py", line 1022, in from_pretrained
+config_class = CONFIG_MAPPING[config_dict["model_type"]]
+File "/transformers/models/auto/configuration_auto.py", line 723, in getitem
+raise KeyError(key)
+KeyError: 'mistral'
+```
+
+Installing transformers from source should solve the issue
+pip install git+https://github.com/huggingface/transformers
+
+This should not be required after transformers-v4.33.4.
+
+## Limitations
+
+The Mistral 7B Instruct model is a quick demonstration that the base model can be easily fine-tuned to achieve compelling performance. 
+It does not have any moderation mechanisms. We're looking forward to engaging with the community on ways to
+make the model finely respect guardrails, allowing for deployment in environments requiring moderated outputs.
+
+## The Mistral AI Team
+
+Albert Jiang, Alexandre Sablayrolles, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, Gianna Lengyel, Guillaume Bour, Guillaume Lample, Lélio Renard Lavaud, Louis Ternon, Lucile Saulnier, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Théophile Gervet, Thibaut Lavril, Thomas Wang, Timothée Lacroix, William El Sayed.