metadata
license: other
library_name: peft
tags:
- llama-factory
- lora
- generated_from_trainer
base_model: mistralai/Mistral-7B-v0.1
model-index:
- name: mistral-7b-medwiki-v1
results: []
mistral-7b-medwiki-v1
This model is a fine-tuned version of mistralai/Mistral-7B-v0.1 on the medical_meadow_wikidoc dataset.
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.0005
- train_batch_size: 4
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 4
- total_train_batch_size: 16
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- num_epochs: 1.0
Framework versions
- PEFT 0.8.2
- Transformers 4.37.2
- Pytorch 2.1.1+cu121
- Datasets 2.16.1
- Tokenizers 0.15.1
Perfromance
hf (pretrained=mistralai/Mistral-7B-v0.1,parallelize=True,load_in_4bit=True,peft=chenhugging/mistral-7b-medwiki-v1), limit: 100.0, num_fewshot: None
| Tasks | Version | Filter | n-shot | Metric | Value | Stderr | |
|---|---|---|---|---|---|---|---|
| pubmedqa | 1 | none | 0 | acc | 0.99 | ± | 0.0100 |
| professional_medicine | 0 | none | 0 | acc | 0.57 | ± | 0.0498 |
| college_medicine | 0 | none | 0 | acc | 0.59 | ± | 0.0494 |
| clinical_knowledge | 0 | none | 0 | acc | 0.58 | ± | 0.0496 |
| medmcqa | Yaml | none | 0 | acc | 0.40 | ± | 0.0492 |
| none | 0 | acc_norm | 0.40 | ± | 0.0492 |