xiong3/modicalSFT_ckpt15800

---

library_name: peft

license: other

base_model: /home/ma-user/modelarts/lss/models/qwen2.5_7b

tags:

• base_model:adapter:/home/ma-user/modelarts/lss/models/qwen2.5_7b

• llama-factory

• lora

• transformers

pipeline_tag: text-generation

model-index:

• name: output_med_7b_8x

  results: []

---

output_med_7b_8x

This model is a fine-tuned version of /home/ma-user/modelarts/lss/models/qwen2.5_7b on the med_train_zh and the med_train_en datasets. It achieves the following results on the evaluation set:

• Loss: 1.8047

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• distributed_type: multi-GPU
• num_devices: 8
• gradient_accumulation_steps: 2
• total_train_batch_size: 128
• total_eval_batch_size: 64
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.05
• num_epochs: 1.0

Training results

Training Loss	Epoch	Step	Validation Loss
2.2011	0.0124	200	2.0800
2.1184	0.0248	400	2.0568
2.1089	0.0372	600	2.0438
2.145	0.0495	800	2.0309
2.0716	0.0619	1000	2.0190
2.1246	0.0743	1200	2.0093
2.0569	0.0867	1400	2.0018
2.0999	0.0991	1600	1.9916
2.1314	0.1115	1800	1.9830
2.1368	0.1239	2000	1.9759
2.0638	0.1363	2200	1.9679
2.063	0.1486	2400	1.9606
1.9771	0.1610	2600	1.9556
2.0441	0.1734	2800	1.9479
2.0637	0.1858	3000	1.9430
2.06	0.1982	3200	1.9377
2.0226	0.2106	3400	1.9309
2.0371	0.2230	3600	1.9264
1.9605	0.2354	3800	1.9230
2.0309	0.2477	4000	1.9190
1.9907	0.2601	4200	1.9149
1.9698	0.2725	4400	1.9097
1.9952	0.2849	4600	1.9044
1.9257	0.2973	4800	1.9024
2.0119	0.3097	5000	1.8972
1.9703	0.3221	5200	1.8923
1.9605	0.3345	5400	1.8886
1.9412	0.3468	5600	1.8868
1.9087	0.3592	5800	1.8825
1.9649	0.3716	6000	1.8801
2.0017	0.3840	6200	1.8757
1.9899	0.3964	6400	1.8715
1.9111	0.4088	6600	1.8675
1.9524	0.4212	6800	1.8656
1.9696	0.4336	7000	1.8625
2.0064	0.4459	7200	1.8580
1.9452	0.4583	7400	1.8556
1.9636	0.4707	7600	1.8518
1.8893	0.4831	7800	1.8491
1.8808	0.4955	8000	1.8464
1.929	0.5079	8200	1.8450
1.8784	0.5203	8400	1.8431
1.8657	0.5327	8600	1.8409
1.9059	0.5450	8800	1.8381
1.8583	0.5574	9000	1.8358
1.8116	0.5698	9200	1.8348
1.8755	0.5822	9400	1.8322
1.803	0.5946	9600	1.8307
1.8891	0.6070	9800	1.8286
1.8604	0.6194	10000	1.8258
1.9637	0.6318	10200	1.8249
1.8665	0.6441	10400	1.8231
1.8784	0.6565	10600	1.8212
1.9064	0.6689	10800	1.8200
1.9137	0.6813	11000	1.8191
1.9271	0.6937	11200	1.8175
1.887	0.7061	11400	1.8154
1.798	0.7185	11600	1.8142
1.9178	0.7309	11800	1.8132
1.8639	0.7432	12000	1.8120
1.8801	0.7556	12200	1.8117
1.8652	0.7680	12400	1.8104
1.8397	0.7804	12600	1.8094
1.8515	0.7928	12800	1.8089
1.884	0.8052	13000	1.8084
1.9381	0.8176	13200	1.8077
1.9644	0.8300	13400	1.8073
1.8815	0.8423	13600	1.8065
1.8915	0.8547	13800	1.8063
1.906	0.8671	14000	1.8060
1.8984	0.8795	14200	1.8056
1.7885	0.8919	14400	1.8054
1.9127	0.9043	14600	1.8050
1.8702	0.9167	14800	1.8050
1.8911	0.9291	15000	1.8051
1.8821	0.9414	15200	1.8049
1.8598	0.9538	15400	1.8048
1.8818	0.9662	15600	1.8048
1.8787	0.9786	15800	1.8047
1.8594	0.9910	16000	1.8050

Framework versions

• PEFT 0.17.1
• Transformers 4.57.1
• Pytorch 2.9.1+cu128
• Datasets 4.0.0
• Tokenizers 0.22.1