|
--- |
|
license: apache-2.0 |
|
library_name: transformers |
|
tags: |
|
- trl |
|
- sft |
|
datasets: |
|
- pubmed |
|
- bigbio/czi_drsm |
|
- bigbio/bc5cdr |
|
- bigbio/distemist |
|
- pubmed_qa |
|
- medmcqa |
|
--- |
|
|
|
# Model Card for med_mistral_4bit |
|
|
|
<!-- Provide a quick summary of what the model is/does. --> |
|
|
|
|
|
|
|
## Model Details |
|
|
|
### Model Description |
|
|
|
<!-- Provide a longer summary of what this model is. --> |
|
|
|
Model 4-bit Mistral-7B-Instruct-v0.2 finetuned with QLoRA on multiple medical datasets. |
|
|
|
16-bit version: [med_mistral](https://huggingface.co/adriata/med_mistral) |
|
|
|
- **License:** apache-2.0 |
|
- **Finetuned from model :** mistralai/Mistral-7B-Instruct-v0.2 |
|
|
|
### Model Sources [optional] |
|
|
|
<!-- Provide the basic links for the model. --> |
|
|
|
- **Repository:** https://github.com/atadria/med_llm |
|
|
|
## Uses |
|
|
|
<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. --> |
|
|
|
The model is finetuned on medical data and is intended only for research. It should not be used as a substitute for professional medical advice, diagnosis, or treatment. |
|
|
|
## Bias, Risks, and Limitations |
|
|
|
<!-- This section is meant to convey both technical and sociotechnical limitations. --> |
|
The model's predictions are based on the information available in the finetuned medical dataset. It may not generalize well to all medical conditions or diverse patient populations. |
|
|
|
Sensitivity to variations in input data and potential biases present in the training data may impact the model's performance. |
|
|
|
### Recommendations |
|
|
|
<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. --> |
|
|
|
Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations. |
|
|
|
## How to Get Started with the Model |
|
|
|
Use the code below to get started with the model. |
|
|
|
```python |
|
# !pip install -q transformers accelerate bitsandbytes |
|
|
|
from transformers import AutoTokenizer, AutoModelForCausalLM |
|
|
|
tokenizer = AutoTokenizer.from_pretrained("adriata/med_mistral") |
|
model = AutoModelForCausalLM.from_pretrained("adriata/med_mistral") |
|
|
|
prompt_template = """<s>[INST] {prompt} [/INST]""" |
|
|
|
prompt = "What is influenza?" |
|
|
|
model_inputs = tokenizer.encode(prompt_template.format(prompt=prompt), |
|
return_tensors="pt").to("cuda") |
|
|
|
generated_ids = model.generate(model_inputs, max_new_tokens=512, do_sample=True) |
|
decoded = tokenizer.batch_decode(generated_ids) |
|
print(decoded[0]) |
|
``` |
|
|
|
## Training Details |
|
~13h - 20k examples x 1 epoch |
|
|
|
GPU: OVH - 1 × NVIDIA TESLA V100S (32 GiB RAM) |
|
|
|
### Training Data |
|
|
|
<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. --> |
|
Training data included 20k examples randomly selected from datasets: |
|
- pubmed |
|
- bigbio/czi_drsm |
|
- bigbio/bc5cdr |
|
- bigbio/distemist |
|
- pubmed_qa |
|
- medmcqa |
|
|
