metadata
license: other
library_name: peft
tags:
- llama-factory
- lora
- generated_from_trainer
base_model: mistralai/Mistral-7B-v0.1
model-index:
- name: mistral-7b-ocn-v2
results: []
mistral-7b-ocn-v2
This model is a fine-tuned version of mistralai/Mistral-7B-v0.1 on the oncc_medqa_instruct dataset.
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.0005
- train_batch_size: 4
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 4
- total_train_batch_size: 16
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- lr_scheduler_warmup_steps: 10
- num_epochs: 1.0
- mixed_precision_training: Native AMP
Framework versions
- PEFT 0.8.2
- Transformers 4.37.2
- Pytorch 2.1.1+cu121
- Datasets 2.16.1
- Tokenizers 0.15.1
Performance
hf (pretrained=mistralai/Mistral-7B-v0.1,parallelize=True,load_in_4bit=True,peft=chenhugging/mistral-7b-ocn-v2), gen_kwargs: (None), limit: 100.0, num_fewshot: None, batch_size: 1
Tasks | Version | Filter | n-shot | Metric | Value | Stderr | |
---|---|---|---|---|---|---|---|
pubmedqa | 1 | none | 0 | acc | 0.98 | ± | 0.0141 |
medmcqa | Yaml | none | 0 | acc | 0.40 | ± | 0.0492 |
professional_medicine | 0 | none | 0 | acc | 0.69 | ± | 0.0465 |
college_medicine | 0 | none | 0 | acc | 0.53 | ± | 0.0502 |
clinical_knowledge | 0 | none | 0 | acc | 0.59 | ± | 0.0494 |
ocn | Yaml | none | 0 | acc | 0.80 | ± | 0.0402 |
aocnp | Yaml | none | 0 | acc | 0.63 | ± | 0.0485 |