IMIT-PMCL/rad-impression-llm

Overcoming Dataset Scarcity for Robust Radiology Impression Generation with Expert-Specific LLMs

This repository provides the pretrained model weights for our paper Overcoming Dataset Scarcity for Robust Radiology Impression Generation with Expert-Specific LLMs.

Model Nomenclature

Model group	Model name	Model description
Benchmark SFT models	bloom_1b1_3	Bloomz-1B benchmark SFT model trained for 3 epochs on the 360,001-report SFT cohort.
	bloom_1b1_16	Bloomz-1B benchmark SFT model trained for 16 epochs on the 360,001-report SFT cohort.
	bloom_3b_3	Bloomz-3B benchmark SFT model trained for 3 epochs on the 360,001-report SFT cohort.
	bloom_3b_16	Bloomz-3B benchmark SFT model trained for 16 epochs on the 360,001-report SFT cohort.
RLHF checkpoints	rlhf_checkpoint-40	RLHF model initialized from bloom_3b_3 and trained after 40 PPO steps.
	rlhf_checkpoint-80	RLHF model initialized from bloom_3b_3 and trained after 80 PPO steps.
	rlhf_checkpoint-120	RLHF model initialized from bloom_3b_3 and trained after 120 PPO steps.
	rlhf_checkpoint-160	RLHF model initialized from bloom_3b_3 and trained after 160 PPO steps.
	rlhf_checkpoint-200	RLHF model initialized from bloom_3b_3 and trained after 200 PPO steps.
	rlhf_checkpoint-240	RLHF model initialized from bloom_3b_3 and trained after 240 PPO steps.
	rlhf_checkpoint-280	RLHF model initialized from bloom_3b_3 and trained after 280 PPO steps.
Expert-specific models	3b_radiologist1	Bloomz-3B expert-specific SFT model trained for 3 epochs on expert 1's cohort of 5,606 reports.
	3b_radiologist2	Bloomz-3B expert-specific SFT model trained for 3 epochs on expert 2's cohort of 9,016 reports.
	3b_radiologist3	Bloomz-3B expert-specific SFT model trained for 3 epochs on expert 3's cohort of 4,375 reports.
	3b_radiologist4	Bloomz-3B expert-specific SFT model trained for 3 epochs on expert 4's cohort of 5,916 reports.
	3b_radiologist5	Bloomz-3B expert-specific SFT model trained for 3 epochs on expert 5's cohort of 2,175 reports.
	7b_radiologist1	Bloomz-7B expert-specific SFT model trained for 3 epochs on expert 1's cohort of 5,606 reports.
	7b_radiologist2	Bloomz-7B expert-specific SFT model trained for 3 epochs on expert 2's cohort of 9,016 reports.
	7b_radiologist3	Bloomz-7B expert-specific SFT model trained for 3 epochs on expert 3's cohort of 4,375 reports.
	7b_radiologist4	Bloomz-7B expert-specific SFT model trained for 3 epochs on expert 4's cohort of 5,916 reports.
	7b_radiologist5	Bloomz-7B expert-specific SFT model trained for 3 epochs on expert 5's cohort of 2,175 reports.
General-purpose public LLMs	llm_gpt35	gpt-3.5-turbo-0125 evaluated using fixed 3-shot prompting.
	llm_gpt4	gpt-4-0125-preview evaluated using fixed 3-shot prompting.
	llm_baidu	Qianfan-Chinese-Llama-2-13B evaluated using fixed 3-shot prompting.