Title: MedSafetyBench: Evaluating and Improving the Medical Safety of Large Language Models
Benchmark: MedSafetyBench
Summary: The paper introduces the first benchmark dataset designed to measure the medical safety of LLMs. It uses the dataset to evaluate and improve the medical safety of LLMs using fine-tuning. 
Phenomenon type: Specific Application (A single use case, where the benchmark is likely to be examples of that use case)
Target phenomenon: medical safety
Phenomenon defined by authors: Yes
Phenomenon definition: "we define an LLM to be aligned with medical safety standards if its output is not only accurate but also consistent with the AMA (American Medical Association)'s Principles of Medical Ethics." (page 3)
Definition scope: Comprehensive
Definition integrity: Composite phenomenon
Definition integrity detail: Yes
Phenomenon contested: Contested
Task definition: They prompt LLMs with harmful medical requests and evaluate the harmfulness of their responses
Task item definition: harmful request, category of harm
Task source: LLM-generated task examples (e.g. Filtered from responses to a prompt)
Task source detail: They validate the LLM-generated dataset by conducting a user study with 25 doctors to check that the generated prompts violate one of the nine principles of medical ethics.
Task ecology: Partial real task (e.g. answering medical questions collected from real people)
Dataset size: 900
Dataset size notes: Train: 900
Dataset metadata available: Yes
Dataset metadata detail: principle violated
Sampling method: Random sample (creators defined a task space and sampled from it)
Split reviewed: Test, Train
Response format: Free response (e.g. summary paragraph, executable code)
Metric definition: LLM-as-a-Judge (text quality, preferences, NOT extracting answers for other metrics)
Metric aggregation: Simple Mean
Metric statistics: simple mean, standard error of the mean
Metric subscores: Yes
Metric subscores detail: By LLM used to generate the prompts
Metric access: Outputs alone
Few-shot evaluation: No
Authorship: Academia
Benchmark availability: Yes
Task face validity: Yes
Metric face validity: Yes
Results interpreted by authors: Yes
Results compared: No
Results comparison explanation: No comparisons made
Results realism: No
Human baseline: No
Author validity detail: They indirectly address it by noting in their discussion: "In practice, one could consider
introducing nuance to the definition. For example, levels of acceptable risk may vary among medical subspecialties (e.g., emergency medicine vs. neurological surgery vs. dermatology) and based on a patient’s condition and personal preference (e.g., a patient with a condition that has no established treatment options may be more willing to try risky experimental procedures). Aligning LLMs to account for different levels of acceptable risk and be tailored to different medical subspecialties is a future research direction"
Taxonomy: Medicine / 
New taxonomy: Specific Application / Medicine