Title: Is Model Size a Barrier to Quality? Evaluating Small Language Models for  Clinical Text Summarization and Report Generation
Primary area: foundation or frontier models, including LLMs
Keywords: Small language models, Small vision–language models, clinical text summarization, report generation, parameter efficient finetuning

Abstract: Several recent efforts have produced small language models (SLMs) and small vision-language models (SVLMs) alongside their large, domain-adapted counterparts for medical applications. In clinical text summarization and report generation, it is often asserted that continuing to pre-train large language models (LLMs) on biomedical corpora yields clear advantages. To test this claim, we evaluated three publicly available small LMs and two small VLMs against their medically adapted LLM and VLM equivalents. Our analysis considers both the off-the-shelf base SLMs and versions fine-tuned via parameter-efficient techniques. We employ a battery of quantitative measures spanning semantic fidelity, syntactic quality, and conceptual coherence to assess each model’s outputs under zero-shot and few-shot prompting. Contrary to common assumptions, we find that nearly all small models match or exceed the summarization and generation effectiveness of larger, medically adapted models, with several fine-tuned SLMs outperforming their LLM counterparts. Conversely, on the report generation task, small VLMs while showing measurable improvement after fine-tuning, still fall short of matching the clinical accuracy and details produced by larger, medically specialized VLMs. These results, grounded in direct head-to-head comparisons and rigorous evaluation of parameter-efficient fine-tuning, indicate that small models can deliver high-quality clinical summaries while demanding substantially less computational resources.