Dataset: 960 PEMs were generated across 4 LLMs and 8 dermatologic conditions. Handouts were generated by ChatGPT-3.5, GPT-4, DocsGPT, and DermGPT at two prompted reading levels. The Flesch-Kincaid reading level (FKRL) of current American Academy of Dermatology PEMs was evaluated for 4 common (atopic dermatitis, acne vulgaris, psoriasis, and herpes zoster) and 4 rare (epidermolysis bullosa, bullous pemphigoid, lamellar ichthyosis, and lichen planus) dermatologic conditions. 10 PEMs per condition at unspecified fifth- and seventh-grade FKRLs were evaluated with Microsoft Word readability statistics. The preservation of meaning across LLMs was assessed by 2 blinded dermatology resident trainees.
Reference (2024): Lambert R, Choo ZY, Gradwohl K, Schroedl L, Ruiz De Luzuriaga A. Assessing the Application of Large Language Models in Generating Dermatologic Patient Education Materials According to Reading Level: Qualitative Study. JMIR Dermatol. 2024 May 16;7:e55898. doi: 10.2196/55898. PMID: 38754096; PMCID: PMC11140271.

Evaluation of DocsGPT (GPT series) — Dermatologic patient education material handouts meeting prompted fifth-grade reading level (rare conditions)
Condition/subject: Rare conditions (English)
Category: Patient Education Materials and Readability Studies — Patient Education
Validity/accuracy: 52.00% (sample size: 40)

Evaluation of DermGPT (GPT series) — Dermatologic patient education material handouts meeting prompted seventh-grade reading level (rare conditions)
Condition/subject: Rare conditions (English)
Category: Patient Education Materials and Readability Studies — Patient Education
Validity/accuracy: 55.00% (sample size: 40)

Evaluation of GPT-4 (GPT series) — Dermatologic patient education material handouts meeting prompted seventh-grade reading level (common conditions)
Condition/subject: Common conditions (English)
Category: Patient Education Materials and Readability Studies — Patient Education
Validity/accuracy: 55.00% (sample size: 40)

Evaluation of GPT-3.5 (GPT series) — Dermatologic patient education material handouts meeting prompted fifth-grade reading level (common conditions)
Condition/subject: Common conditions (English)
Category: Patient Education Materials and Readability Studies — Patient Education
Validity/accuracy: 57.00% (sample size: 40)

Evaluation of DermGPT (GPT series) — Psoriasis,  seventh-grade reading level (common condition)
Condition/subject: Psoriasis (English)
Category: Patient Education Materials and Readability Studies — Patient Education
Validity/accuracy: 60.00% (sample size: 10)
Comments: Flesch-Kincaid reading level (FKRL) of current American Academy of Dermatology PEM

Evaluation of DocsGPT (GPT series) — Herpes zoster, fifth-grade reading level (common condition)
Condition/subject: Herpes zoster (English)
Category: Patient Education Materials and Readability Studies — Patient Education
Validity/accuracy: 60.00% (sample size: 10)

Evaluation of DocsGPT (GPT series) — Psoriasis,  seventh-grade reading level (common condition)
Condition/subject: Psoriasis (English)
Category: Patient Education Materials and Readability Studies — Patient Education
Validity/accuracy: 60.00% (sample size: 10)