Update README.md

README.md

@@ -1,3 +1,98 @@
 ---
 license: apache-2.0
+library_name: transformers
+tags:
+- trl
+- sft
+datasets:
+- pubmed
+- bigbio/czi_drsm
+- bigbio/bc5cdr
+- bigbio/distemist
+- pubmed_qa
+- medmcqa
 ---
+
+# Model Card for med_mistral
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+Model Mistral-7B-Instruct-v0.2 finetuned with QLoRA on multiple medical datasets.
+
+4-bit version: [med_mistral_4bit](https://huggingface.co/adriata/med_mistral_4bit)
+
+- **License:** apache-2.0
+- **Finetuned from model :** mistralai/Mistral-7B-Instruct-v0.2
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** https://github.com/atadria/med_llm
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+The model is finetuned on medical data and is intended only for research. It should not be used as a substitute for professional medical advice, diagnosis, or treatment.
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+The model's predictions are based on the information available in the finetuned medical dataset. It may not generalize well to all medical conditions or diverse patient populations.
+
+Sensitivity to variations in input data and potential biases present in the training data may impact the model's performance.
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+```python
+# !pip install -q transformers accelerate bitsandbytes
+
+from transformers import AutoTokenizer, AutoModelForCausalLM
+
+tokenizer = AutoTokenizer.from_pretrained("adriata/med_mistral")
+model = AutoModelForCausalLM.from_pretrained("adriata/med_mistral")
+
+prompt_template = """<s>[INST] {prompt} [/INST]"""
+
+prompt = "What is influenza?"
+
+model_inputs = tokenizer.encode(prompt_template.format(prompt=prompt),
+                                return_tensors="pt").to("cuda")
+
+generated_ids = model.generate(model_inputs, max_new_tokens=512, do_sample=True)
+decoded = tokenizer.batch_decode(generated_ids)
+print(decoded[0])
+```
+
+## Training Details
+~13h - 20k examples x 1 epoch 
+
+GPU: OVH - 1 × NVIDIA TESLA V100S (32 GiB RAM) 
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+Training data included 20k examples randomly selected from datasets:
+- pubmed
+- bigbio/czi_drsm
+- bigbio/bc5cdr
+- bigbio/distemist
+- pubmed_qa
+- medmcqa