Aurametrix/dermatology_longitudinal huggingface dataset:
Dataset: 960 PEMs were generated across 4 LLMs and 8 dermatologic conditions. Handouts were generated by ChatGPT-3.5, GPT-4, DocsGPT, and DermGPT at two prompted reading levels. The Flesch-Kincaid reading level (FKRL) of current American Academy of Dermatology PEMs was evaluated for 4 common (atopic dermatitis, acne vulgaris, psoriasis, and herpes zoster) and 4 rare (epidermolysis bullosa, bullous pemphigoid, lamellar ichthyosis, and lichen planus) dermatologic conditions. 10 PEMs per condition at unspecified fifth- and seventh-grade FKRLs were evaluated with Microsoft Word readability statistics. The preservation of meaning across LLMs was assessed by 2 blinded dermatology resident trainees.
Reference (2024): Lambert R, Choo ZY, Gradwohl K, Schroedl L, Ruiz De Luzuriaga A. Assessing the Application of Large Language Models in Generating Dermatologic Patient Education Materials According to Reading Level: Qualitative Study. JMIR Dermatol. 2024 May 16;7:e55898. doi: 10.2196/55898. PMID: 38754096; PMCID: PMC11140271.

Evaluation of DermGPT (GPT series) — Dermatologic patient education material handouts meeting prompted fifth-grade reading level (rare conditions)
Condition/subject: Rare conditions (English)
Category: Patient Education Materials and Readability Studies — Patient Education
Validity/accuracy: 10.00% (sample size: 40)

Evaluation of DocsGPT (GPT series) — Atopic dermatitis,  fifth-grade reading level (common condition)
Condition/subject: Atopic dermatitis (English)
Category: Patient Education Materials and Readability Studies — Patient Education
Validity/accuracy: 10.00% (sample size: 10)

Evaluation of GPT-3.5 (GPT series) — Epidermolysis bullosa, fifth-grade reading level (rare condition)
Condition/subject: Epidermolysis bullosa (English)
Category: Patient Education Materials and Readability Studies — Patient Education
Validity/accuracy: 10.00% (sample size: 10)

Evaluation of GPT-3.5 (GPT series) — Epidermolysis bullosa, seventh-grade reading level (rare condition)
Condition/subject: Epidermolysis bullosa (English)
Category: Patient Education Materials and Readability Studies — Patient Education
Validity/accuracy: 10.00% (sample size: 10)

Evaluation of GPT-3.5 (GPT series) — Herpes zoster, fifth-grade reading level (common condition)
Condition/subject: Herpes zoster (English)
Category: Patient Education Materials and Readability Studies — Patient Education
Validity/accuracy: 10.00% (sample size: 10)

Evaluation of GPT-4 (GPT series) — Psoriasis,  seventh-grade reading level (common condition)
Condition/subject: Psoriasis (English)
Category: Patient Education Materials and Readability Studies — Patient Education
Validity/accuracy: 10.00% (sample size: 10)

Evaluation of DermGPT (GPT series) — Bullous pemphigoid, seventh-grade reading level (rare condition)
Condition/subject: Bullous pemphigoid (English)
Category: Patient Education Materials and Readability Studies — Patient Education
Validity/accuracy: 20.00% (sample size: 10)
Dataset: 960 PEMs were generated across 4 LLMs and 8 dermatologic conditions. Handouts were generated by ChatGPT-3.5, GPT-4, DocsGPT, and DermGPT at two prompted reading levels. The Flesch-Kincaid reading level (FKRL) of current American Academy of Dermatology PEMs was evaluated for 4 common (atopic dermatitis, acne vulgaris, psoriasis, and herpes zoster) and 4 rare (epidermolysis bullosa, bullous pemphigoid, lamellar ichthyosis, and lichen planus) dermatologic conditions. 10 PEMs per condition at unspecified fifth- and seventh-grade FKRLs were evaluated with Microsoft Word readability statistics. The preservation of meaning across LLMs was assessed by 2 blinded dermatology resident trainees.
Reference (2024): Lambert R, Choo ZY, Gradwohl K, Schroedl L, Ruiz De Luzuriaga A. Assessing the Application of Large Language Models in Generating Dermatologic Patient Education Materials According to Reading Level: Qualitative Study. JMIR Dermatol. 2024 May 16;7:e55898. doi: 10.2196/55898. PMID: 38754096; PMCID: PMC11140271.

Evaluation of GPT-3.5 (GPT series) — Lichen planus, fifth-grade reading level (rare condition)
Condition/subject: Lichen planus (English)
Category: Patient Education Materials and Readability Studies — Patient Education
Validity/accuracy: 20.00% (sample size: 10)

Evaluation of GPT-4, ChatGPT (GPT series) — Bullous pemphigoid, seventh-grade reading level (rare condition)
Condition/subject: Bullous pemphigoid (English)
Category: Patient Education Materials and Readability Studies — Patient Education
Validity/accuracy: 20.00% (sample size: 10)

Evaluation of DermGPT (GPT series) — Acne vulgaris, seventh-grade reading level (common condition)
Condition/subject: Acne vulgaris (English)
Category: Patient Education Materials and Readability Studies — Patient Education
Validity/accuracy: 30.00% (sample size: 10)

Evaluation of DocsGPT (GPT series) — Atopic dermatitis,  seventh-grade reading level (common condition)
Condition/subject: Atopic dermatitis (English)
Category: Patient Education Materials and Readability Studies — Patient Education
Validity/accuracy: 30.00% (sample size: 10)

Evaluation of GPT-4 (GPT series) — Epidermolysis bullosa, fifth-grade reading level (rare condition)
Condition/subject: Epidermolysis bullosa (English)
Category: Patient Education Materials and Readability Studies — Patient Education
Validity/accuracy: 30.00% (sample size: 10)

Evaluation of GPT-4 (GPT series) — Lichen planus, seventh-grade reading level (rare condition)
Condition/subject: Lichen planus (English)
Category: Patient Education Materials and Readability Studies — Patient Education
Validity/accuracy: 30.00% (sample size: 10)

Evaluation of GPT-4 (GPT series) — Dermatologic patient education material handouts meeting prompted seventh-grade reading level (rare conditions)
Condition/subject: Rare conditions (English)
Category: Patient Education Materials and Readability Studies — Patient Education
Validity/accuracy: 35.00% (sample size: 40)
Dataset: 960 PEMs were generated across 4 LLMs and 8 dermatologic conditions. Handouts were generated by ChatGPT-3.5, GPT-4, DocsGPT, and DermGPT at two prompted reading levels. The Flesch-Kincaid reading level (FKRL) of current American Academy of Dermatology PEMs was evaluated for 4 common (atopic dermatitis, acne vulgaris, psoriasis, and herpes zoster) and 4 rare (epidermolysis bullosa, bullous pemphigoid, lamellar ichthyosis, and lichen planus) dermatologic conditions. 10 PEMs per condition at unspecified fifth- and seventh-grade FKRLs were evaluated with Microsoft Word readability statistics. The preservation of meaning across LLMs was assessed by 2 blinded dermatology resident trainees.
Reference (2024): Lambert R, Choo ZY, Gradwohl K, Schroedl L, Ruiz De Luzuriaga A. Assessing the Application of Large Language Models in Generating Dermatologic Patient Education Materials According to Reading Level: Qualitative Study. JMIR Dermatol. 2024 May 16;7:e55898. doi: 10.2196/55898. PMID: 38754096; PMCID: PMC11140271.

Evaluation of DermGPT (GPT series) — Lichen planus, fifth-grade reading level (rare condition)
Condition/subject: Lichen planus (English)
Category: Patient Education Materials and Readability Studies — Patient Education
Validity/accuracy: 40.00% (sample size: 10)

Evaluation of DocsGPT (GPT series) — Lamellar ichthyosis, fifth-grade reading level (rare condition)
Condition/subject: Lamellar ichthyosis (English)
Category: Patient Education Materials and Readability Studies — Patient Education
Validity/accuracy: 40.00% (sample size: 10)

Evaluation of DocsGPT (GPT series) — Dermatologic patient education material handouts meeting prompted fifth-grade reading level (common conditions)
Condition/subject: Common conditions (English)
Category: Patient Education Materials and Readability Studies — Patient Education
Validity/accuracy: 42.00% (sample size: 40)

Evaluation of DermGPT (GPT series) — Dermatologic patient education material handouts meeting prompted seventh-grade reading level (common conditions)
Condition/subject: Common conditions (English)
Category: Patient Education Materials and Readability Studies — Patient Education
Validity/accuracy: 47.00% (sample size: 40)

Evaluation of DocsGPT (GPT series) — Bullous pemphigoid, seventh-grade reading level (rare condition)
Condition/subject: Bullous pemphigoid (English)
Category: Patient Education Materials and Readability Studies — Patient Education
Validity/accuracy: 50.00% (sample size: 10)

Evaluation of GPT-3.5 (GPT series) — Dermatologic patient education material handouts meeting prompted fifth-grade reading level (rare conditions)
Condition/subject: Rare conditions (English)
Category: Patient Education Materials and Readability Studies — Patient Education
Validity/accuracy: 50.00% (sample size: 40)

Evaluation of GPT-4 (GPT series) — Atopic dermatitis,  seventh-grade reading level (common condition)
Condition/subject: Atopic dermatitis (English)
Category: Patient Education Materials and Readability Studies — Patient Education
Validity/accuracy: 50.00% (sample size: 10)
Dataset: 960 PEMs were generated across 4 LLMs and 8 dermatologic conditions. Handouts were generated by ChatGPT-3.5, GPT-4, DocsGPT, and DermGPT at two prompted reading levels. The Flesch-Kincaid reading level (FKRL) of current American Academy of Dermatology PEMs was evaluated for 4 common (atopic dermatitis, acne vulgaris, psoriasis, and herpes zoster) and 4 rare (epidermolysis bullosa, bullous pemphigoid, lamellar ichthyosis, and lichen planus) dermatologic conditions. 10 PEMs per condition at unspecified fifth- and seventh-grade FKRLs were evaluated with Microsoft Word readability statistics. The preservation of meaning across LLMs was assessed by 2 blinded dermatology resident trainees.
Reference (2024): Lambert R, Choo ZY, Gradwohl K, Schroedl L, Ruiz De Luzuriaga A. Assessing the Application of Large Language Models in Generating Dermatologic Patient Education Materials According to Reading Level: Qualitative Study. JMIR Dermatol. 2024 May 16;7:e55898. doi: 10.2196/55898. PMID: 38754096; PMCID: PMC11140271.

Evaluation of GPT-3.5 (GPT series) — Acne vulgaris, fifth-grade reading level (common condition)
Condition/subject: Acne vulgaris (English)
Category: Patient Education Materials and Readability Studies — Patient Education
Validity/accuracy: 60.00% (sample size: 10)

Evaluation of GPT-3.5 (GPT series) — Dermatologic patient education material handouts meeting prompted seventh-grade reading level (common conditions)
Condition/subject: Common conditions (English)
Category: Patient Education Materials and Readability Studies — Patient Education
Validity/accuracy: 60.00% (sample size: 40)

Evaluation of GPT-4 (GPT series) — Psoriasis,  fifth-grade reading level (common condition)
Condition/subject: Psoriasis (English)
Category: Patient Education Materials and Readability Studies — Patient Education
Validity/accuracy: 60.00% (sample size: 10)

Evaluation of DocsGPT (GPT series) — Lamellar ichthyosis, seventh-grade reading level (rare condition)
Condition/subject: Lamellar ichthyosis (English)
Category: Patient Education Materials and Readability Studies — Patient Education
Validity/accuracy: 70.00% (sample size: 10)

Evaluation of GPT-3.5 (GPT series) — Atopic dermatitis,  fifth-grade reading level (common condition)
Condition/subject: Atopic dermatitis (English)
Category: Patient Education Materials and Readability Studies — Patient Education
Validity/accuracy: 70.00% (sample size: 10)

Evaluation of GPT-3.5 (GPT series) — Psoriasis,  seventh-grade reading level (common condition)
Condition/subject: Psoriasis (English)
Category: Patient Education Materials and Readability Studies — Patient Education
Validity/accuracy: 70.00% (sample size: 10)

Evaluation of GPT-3.5 (GPT series) — Dermatologic patient education material handouts meeting prompted seventh-grade reading level (rare conditions)
Condition/subject: Rare conditions (English)
Category: Patient Education Materials and Readability Studies — Patient Education
Validity/accuracy: 70.00% (sample size: 40)
Dataset: 960 PEMs were generated across 4 LLMs and 8 dermatologic conditions. Handouts were generated by ChatGPT-3.5, GPT-4, DocsGPT, and DermGPT at two prompted reading levels. The Flesch-Kincaid reading level (FKRL) of current American Academy of Dermatology PEMs was evaluated for 4 common (atopic dermatitis, acne vulgaris, psoriasis, and herpes zoster) and 4 rare (epidermolysis bullosa, bullous pemphigoid, lamellar ichthyosis, and lichen planus) dermatologic conditions. 10 PEMs per condition at unspecified fifth- and seventh-grade FKRLs were evaluated with Microsoft Word readability statistics. The preservation of meaning across LLMs was assessed by 2 blinded dermatology resident trainees.
Reference (2024): Lambert R, Choo ZY, Gradwohl K, Schroedl L, Ruiz De Luzuriaga A. Assessing the Application of Large Language Models in Generating Dermatologic Patient Education Materials According to Reading Level: Qualitative Study. JMIR Dermatol. 2024 May 16;7:e55898. doi: 10.2196/55898. PMID: 38754096; PMCID: PMC11140271.

Evaluation of GPT-4 (GPT series) — Herpes zoster,  seventh-grade reading level (common condition)
Condition/subject: Herpes zoster (English)
Category: Patient Education Materials and Readability Studies — Patient Education
Validity/accuracy: 70.00% (sample size: 10)

Evaluation of DocsGPT (GPT series) — Dermatologic patient education material handouts meeting prompted seventh-grade reading level (common conditions)
Condition/subject: Common conditions (English)
Category: Patient Education Materials and Readability Studies — Patient Education
Validity/accuracy: 72.00% (sample size: 40)

Evaluation of DocsGPT (GPT series) — Dermatologic patient education material handouts meeting prompted seventh-grade reading level (rare conditions)
Condition/subject: Rare conditions (English)
Category: Patient Education Materials and Readability Studies — Patient Education
Validity/accuracy: 77.00% (sample size: 40)

Evaluation of DocsGPT (GPT series) — Lichen planus, fifth-grade reading level (rare condition)
Condition/subject: Lichen planus (English)
Category: Patient Education Materials and Readability Studies — Patient Education
Validity/accuracy: 80.00% (sample size: 10)

Evaluation of GPT-3.5 (GPT series) — Herpes zoster,  seventh-grade reading level (common condition)
Condition/subject: Herpes zoster (English)
Category: Patient Education Materials and Readability Studies — Patient Education
Validity/accuracy: 80.00% (sample size: 10)

Evaluation of GPT-3.5 (GPT series) — Lamellar ichthyosis, fifth-grade reading level (rare condition)
Condition/subject: Lamellar ichthyosis (English)
Category: Patient Education Materials and Readability Studies — Patient Education
Validity/accuracy: 80.00% (sample size: 10)

Evaluation of GPT-4 (GPT series) — Dermatologic patient education material handouts meeting prompted fifth-grade reading level (rare conditions)
Condition/subject: Rare conditions (English)
Category: Patient Education Materials and Readability Studies — Patient Education
Validity/accuracy: 82.00% (sample size: 40)
Dataset: 960 PEMs were generated across 4 LLMs and 8 dermatologic conditions. Handouts were generated by ChatGPT-3.5, GPT-4, DocsGPT, and DermGPT at two prompted reading levels. The Flesch-Kincaid reading level (FKRL) of current American Academy of Dermatology PEMs was evaluated for 4 common (atopic dermatitis, acne vulgaris, psoriasis, and herpes zoster) and 4 rare (epidermolysis bullosa, bullous pemphigoid, lamellar ichthyosis, and lichen planus) dermatologic conditions. 10 PEMs per condition at unspecified fifth- and seventh-grade FKRLs were evaluated with Microsoft Word readability statistics. The preservation of meaning across LLMs was assessed by 2 blinded dermatology resident trainees.
Reference (2024): Lambert R, Choo ZY, Gradwohl K, Schroedl L, Ruiz De Luzuriaga A. Assessing the Application of Large Language Models in Generating Dermatologic Patient Education Materials According to Reading Level: Qualitative Study. JMIR Dermatol. 2024 May 16;7:e55898. doi: 10.2196/55898. PMID: 38754096; PMCID: PMC11140271.

Evaluation of DocsGPT (GPT series) — Epidermolysis bullosa, fifth-grade reading level (rare condition)
Condition/subject: Epidermolysis bullosa (English)
Category: Patient Education Materials and Readability Studies — Patient Education
Validity/accuracy: 90.00% (sample size: 10)

Evaluation of DocsGPT (GPT series) — Lichen planus, seventh-grade reading level (rare condition)
Condition/subject: Lichen planus (English)
Category: Patient Education Materials and Readability Studies — Patient Education
Validity/accuracy: 90.00% (sample size: 10)

Evaluation of GPT-3.5 (GPT series) — Acne vulgaris, seventh-grade reading level (common condition)
Condition/subject: Acne vulgaris (English)
Category: Patient Education Materials and Readability Studies — Patient Education
Validity/accuracy: 90.00% (sample size: 10)

Evaluation of GPT-3.5 (GPT series) — Bullous pemphigoid, fifth-grade reading level (rare condition)
Condition/subject: Bullous pemphigoid (English)
Category: Patient Education Materials and Readability Studies — Patient Education
Validity/accuracy: 90.00% (sample size: 10)

Evaluation of GPT-3.5 (GPT series) — Bullous pemphigoid, seventh-grade reading level (rare condition)
Condition/subject: Bullous pemphigoid (English)
Category: Patient Education Materials and Readability Studies — Patient Education
Validity/accuracy: 90.00% (sample size: 10)

Evaluation of GPT-3.5 (GPT series) — Lamellar ichthyosis, seventh-grade reading level (rare condition)
Condition/subject: Lamellar ichthyosis (English)
Category: Patient Education Materials and Readability Studies — Patient Education
Validity/accuracy: 90.00% (sample size: 10)

Evaluation of GPT-3.5 (GPT series) — Lichen planus, seventh-grade reading level (rare condition)
Condition/subject: Lichen planus (English)
Category: Patient Education Materials and Readability Studies — Patient Education
Validity/accuracy: 90.00% (sample size: 10)
Dataset: 960 PEMs were generated across 4 LLMs and 8 dermatologic conditions. Handouts were generated by ChatGPT-3.5, GPT-4, DocsGPT, and DermGPT at two prompted reading levels. The Flesch-Kincaid reading level (FKRL) of current American Academy of Dermatology PEMs was evaluated for 4 common (atopic dermatitis, acne vulgaris, psoriasis, and herpes zoster) and 4 rare (epidermolysis bullosa, bullous pemphigoid, lamellar ichthyosis, and lichen planus) dermatologic conditions. 10 PEMs per condition at unspecified fifth- and seventh-grade FKRLs were evaluated with Microsoft Word readability statistics. The preservation of meaning across LLMs was assessed by 2 blinded dermatology resident trainees.
Reference (2024): Lambert R, Choo ZY, Gradwohl K, Schroedl L, Ruiz De Luzuriaga A. Assessing the Application of Large Language Models in Generating Dermatologic Patient Education Materials According to Reading Level: Qualitative Study. JMIR Dermatol. 2024 May 16;7:e55898. doi: 10.2196/55898. PMID: 38754096; PMCID: PMC11140271.

Evaluation of GPT-3.5 (GPT series) — Psoriasis,  fifth-grade reading level (common condition)
Condition/subject: Psoriasis (English)
Category: Patient Education Materials and Readability Studies — Patient Education
Validity/accuracy: 90.00% (sample size: 10)

Evaluation of GPT-4 (GPT series) — Acne vulgaris, seventh-grade reading level (common condition)
Condition/subject: Acne vulgaris (English)
Category: Patient Education Materials and Readability Studies — Patient Education
Validity/accuracy: 90.00% (sample size: 10)

Evaluation of GPT-4 (GPT series) — Lamellar ichthyosis, seventh-grade reading level (rare condition)
Condition/subject: Lamellar ichthyosis (English)
Category: Patient Education Materials and Readability Studies — Patient Education
Validity/accuracy: 90.00% (sample size: 10)

Evaluation of GPT-4 (GPT series) — Dermatologic patient education material handouts meeting prompted fifth-grade reading level (common conditions)
Condition/subject: Common conditions (English)
Category: Patient Education Materials and Readability Studies — Patient Education
Validity/accuracy: 90.00% (sample size: 40)

Evaluation of DermGPT (GPT series) — Atopic dermatitis,  seventh-grade reading level (common condition)
Condition/subject: Atopic dermatitis (English)
Category: Patient Education Materials and Readability Studies — Patient Education
Validity/accuracy: 100.00% (sample size: 10)

Evaluation of DermGPT (GPT series) — Lamellar ichthyosis, seventh-grade reading level (rare condition)
Condition/subject: Lamellar ichthyosis (English)
Category: Patient Education Materials and Readability Studies — Patient Education
Validity/accuracy: 100.00% (sample size: 10)

Evaluation of DermGPT (GPT series) — Lichen planus, seventh-grade reading level (rare condition)
Condition/subject: Lichen planus (English)
Category: Patient Education Materials and Readability Studies — Patient Education
Validity/accuracy: 100.00% (sample size: 10)
Dataset: 960 PEMs were generated across 4 LLMs and 8 dermatologic conditions. Handouts were generated by ChatGPT-3.5, GPT-4, DocsGPT, and DermGPT at two prompted reading levels. The Flesch-Kincaid reading level (FKRL) of current American Academy of Dermatology PEMs was evaluated for 4 common (atopic dermatitis, acne vulgaris, psoriasis, and herpes zoster) and 4 rare (epidermolysis bullosa, bullous pemphigoid, lamellar ichthyosis, and lichen planus) dermatologic conditions. 10 PEMs per condition at unspecified fifth- and seventh-grade FKRLs were evaluated with Microsoft Word readability statistics. The preservation of meaning across LLMs was assessed by 2 blinded dermatology resident trainees.
Reference (2024): Lambert R, Choo ZY, Gradwohl K, Schroedl L, Ruiz De Luzuriaga A. Assessing the Application of Large Language Models in Generating Dermatologic Patient Education Materials According to Reading Level: Qualitative Study. JMIR Dermatol. 2024 May 16;7:e55898. doi: 10.2196/55898. PMID: 38754096; PMCID: PMC11140271.

Evaluation of DocsGPT (GPT series) — Acne vulgaris, fifth-grade reading level (common condition)
Condition/subject: Acne vulgaris (English)
Category: Patient Education Materials and Readability Studies — Patient Education
Validity/accuracy: 100.00% (sample size: 10)

Evaluation of DocsGPT (GPT series) — Acne vulgaris, seventh-grade reading level (common condition)
Condition/subject: Acne vulgaris (English)
Category: Patient Education Materials and Readability Studies — Patient Education
Validity/accuracy: 100.00% (sample size: 10)

Evaluation of DocsGPT (GPT series) — Epidermolysis bullosa, seventh-grade reading level (rare condition)
Condition/subject: Epidermolysis bullosa (English)
Category: Patient Education Materials and Readability Studies — Patient Education
Validity/accuracy: 100.00% (sample size: 10)

Evaluation of DocsGPT (GPT series) — Herpes zoster,  seventh-grade reading level (common condition)
Condition/subject: Herpes zoster (English)
Category: Patient Education Materials and Readability Studies — Patient Education
Validity/accuracy: 100.00% (sample size: 10)

Evaluation of GPT-4 (GPT series) — Acne vulgaris, fifth-grade reading level (common condition)
Condition/subject: Acne vulgaris (English)
Category: Patient Education Materials and Readability Studies — Patient Education
Validity/accuracy: 100.00% (sample size: 10)

Evaluation of GPT-4 (GPT series) — Atopic dermatitis,  fifth-grade reading level (common condition)
Condition/subject: Atopic dermatitis (English)
Category: Patient Education Materials and Readability Studies — Patient Education
Validity/accuracy: 100.00% (sample size: 10)

Evaluation of GPT-4 (GPT series) — Bullous pemphigoid, fifth-grade reading level (rare condition)
Condition/subject: Bullous pemphigoid (English)
Category: Patient Education Materials and Readability Studies — Patient Education
Validity/accuracy: 100.00% (sample size: 10)
Dataset: 960 PEMs were generated across 4 LLMs and 8 dermatologic conditions. Handouts were generated by ChatGPT-3.5, GPT-4, DocsGPT, and DermGPT at two prompted reading levels. The Flesch-Kincaid reading level (FKRL) of current American Academy of Dermatology PEMs was evaluated for 4 common (atopic dermatitis, acne vulgaris, psoriasis, and herpes zoster) and 4 rare (epidermolysis bullosa, bullous pemphigoid, lamellar ichthyosis, and lichen planus) dermatologic conditions. 10 PEMs per condition at unspecified fifth- and seventh-grade FKRLs were evaluated with Microsoft Word readability statistics. The preservation of meaning across LLMs was assessed by 2 blinded dermatology resident trainees.
Reference (2024): Lambert R, Choo ZY, Gradwohl K, Schroedl L, Ruiz De Luzuriaga A. Assessing the Application of Large Language Models in Generating Dermatologic Patient Education Materials According to Reading Level: Qualitative Study. JMIR Dermatol. 2024 May 16;7:e55898. doi: 10.2196/55898. PMID: 38754096; PMCID: PMC11140271.

Evaluation of GPT-4 (GPT series) — Herpes zoster, fifth-grade reading level (common condition)
Condition/subject: Herpes zoster (English)
Category: Patient Education Materials and Readability Studies — Patient Education
Validity/accuracy: 100.00% (sample size: 10)

Evaluation of GPT-4 (GPT series) — Lamellar ichthyosis, fifth-grade reading level (rare condition)
Condition/subject: Lamellar ichthyosis (English)
Category: Patient Education Materials and Readability Studies — Patient Education
Validity/accuracy: 100.00% (sample size: 10)

Evaluation of GPT-4 (GPT series) — Lichen planus, fifth-grade reading level (rare condition)
Condition/subject: Lichen planus (English)
Category: Patient Education Materials and Readability Studies — Patient Education
Validity/accuracy: 100.00% (sample size: 10)
Dataset: 162000 sentences describing tobacco smoking behavior in records of 29823 patients (medical narrative comboined with ICD codes, histology, cancer treatment records and death certificated) diagnosed with cancer in 2009-2018 in Southwest Finland
Reference (2021): Karlsson A, Ellonen A, Irjala H, V�liaho V, Mattila K, Nissi L, Kyt� E, Kurki S, Ristam�ki R, Vihinen P, Laitinen T, �lgars A, Jyrkki� S, Minn H, Heerv� E. Impact of deep learning-determined smoking status on mortality of cancer patients: never too late to quit. ESMO Open. 2021 Jun;6(3):100175. doi: 10.1016/j.esmoop.2021.100175. Epub 2021 Jun 3. PMID: 34091262; PMCID: PMC8182259.

Evaluation of ULMFit (ULMFit) — Skin Melanoma, smoking status and mortality in patients' records
Condition/subject: Melanoma (English)
Category: Medical Records and Diagnostic Processes — Clinical Practice
Validity/accuracy: 93.30% (sample size: 661)
Comments: Specificity and Sensitivity for Never Smokers  (11,338 patients/600 with cutaneous melanoma), Former Smokers (5,789 patients/203), Persistent Smokers (5,904/62): 96%/96%, 98%/68%, 88%/99%

Evaluation of BERT (Google's Family of LLMs) — Skin Melanoma, smoking status and mortality in patients' records
Condition/subject: Melanoma (English)
Category: Medical Records and Diagnostic Processes — Clinical Practice
Validity/accuracy: 93.50% (sample size: 661)
Comments: Specificity and Sensitivity for Never Smokers  (11,338 patients/600 with cutaneous melanoma), Former Smokers (5,789 patients/203), Persistent Smokers (5,904/62): 96%/96%, 98%/68%, 88%/99%
Dataset: 50 melanoma-specific questions from Dutch dermatologists
Reference (2024): Kamminga NC, Kievits JE, Plaisier PW, Burgers JS, van der Veldt AM, van den Brand JAGJ, Mulder M, Wakkee M, Lugtenberg M, Nijsten T. Do Large Language Model Chatbots perform better than established patient information resources in answering patient questions? A comparative study on melanoma. Br J Dermatol. 2024 Oct 4:ljae377. doi: 10.1093/bjd/ljae377. Epub ahead of print. PMID: 39365602.

Evaluation of Gemini (Google's Family of LLMs) — Melanoma-specific questions
Condition/subject: Melanoma (English)
Category: Patient Education Materials and Readability Studies — Patient Education
Validity/accuracy: 36.00% (sample size: 50)

Evaluation of GPT-4.0, ChatGPT (GPT series) — Melanoma-specific questions
Condition/subject: Melanoma (English)
Category: Patient Education Materials and Readability Studies — Patient Education
Validity/accuracy: 44.00% (sample size: 50)

Evaluation of GPT-3.5, ChatGPT (GPT series) — Melanoma-specific questions
Condition/subject: Melanoma (English)
Category: Patient Education Materials and Readability Studies — Patient Education
Validity/accuracy: 46.00% (sample size: 50)
Dataset: 477 psoriasis-affected body areas inferred from EMR data from 94 patients treated at the Dermatology Department and Psoriasis Outpatient Clinic of Sheba Medical Center between 2008 and 2022. The data were processed using the ChatGPT-4 interface to identify and report the body areas affected by psoriasis. These identified areas were then categorized, and the accuracy of ChatGPT-4�s analysis was compared with that of a senior dermatologist.
Reference (2024): Jonathan Shapiro, Sharon Baum, Felix Pavlotzky, Yaron Ben Mordehai, Aviv Barzilai, Tamar Freud, Rotem Gershon, Application of a natural language processing artificial intelligence tool in psoriasis: A cross-sectional comparative study on identifying affected areas in patients� data, Clinics in Dermatology, Volume 42, Issue 5, 2024, Pages 480-486, ISSN 0738-081X, https://doi.org/10.1016/j.clindermatol.2024.06.018. https://www.sciencedirect.com/science/article/pii/S0738081X24001020

Evaluation of GPT-4V (GPT series) — All Psoriasis affected body areas, including nails and joints, recognized in any given patient's case, from EMR records
Condition/subject: Psoriasis (English)
Category: Dermatological Conditions and Management — Professional Education
Validity/accuracy: 57.40% (sample size: 94)
Comments: From 94 cases, complete accuracy was achieved in 54 cases (57.4%), although inaccuracies were observed in 40 cases (42.6%).

Evaluation of GPT-4V (GPT series) — Nail involvement in psoriasis
Condition/subject: Psoriasis (English)
Category: Dermatological Conditions and Management — Professional Education
Validity/accuracy: 90.63% (sample size: 94)
Comments: Nail involvement was detected in 32 cases (34.0% of all cases), with ChatGPT-4 correctly identifying 29 cases

Evaluation of GPT-4V (GPT series) — Psoriasis affected body areas, including nails and joints, recognized from EMR records
Condition/subject: Psoriasis (English)
Category: Dermatological Conditions and Management — Professional Education
Validity/accuracy: 92.80% (sample size: 477)
Comments: While senior dermatologist identified 477 psoriasis-affected body areas, ChatGPT-4 accurately recognized 443 (92.8%) of these areas, missed 34, and incorrectly identified 30 areas as affected.

Evaluation of GPT-4V (GPT series) — Joint involvement in psoriasis
Condition/subject: Psoriasis (English)
Category: Dermatological Conditions and Management — Professional Education
Validity/accuracy: 96.00% (sample size: 25)
Comments: Joint involvement was noted in 25 cases (26.6% of all cases), with 24 correctly identified using ChatGPT-4.
Dataset: Practice dermatology board certification examination.
Reference (2023): Joly-Chevrier M, Nguyen AX, Lesko-Krleza M, Lefran�ois P. Performance of ChatGPT on a practice dermatology board certification examination. Journal of cutaneous medicine and surgery. 2023 Jul;27(4):407-9.

Evaluation of GPT-3.5 (GPT series) — Benign and Malignant Neoplasms
Condition/subject: Mohs surgery (English)
Category: Dermatology Examinations and Practice Questions — Professional Education
Validity/accuracy: 30.00% (sample size: 10)

Evaluation of GPT-3.5 (GPT series) — Dermatology Practice Exam
Condition/subject: Certification (English)
Category: Dermatology Examinations and Practice Questions — Professional Education
Validity/accuracy: 59.30% (sample size: 241)
Comments: No Association Between Question Length and Accuracy. Wilcoxon Rank-Sum (Mann-Whitney) Test p value = 0.13

Evaluation of GPT-3.5 (GPT series) — Pediatric Dermatology
Condition/subject: Mohs surgery (English)
Category: Dermatology Examinations and Practice Questions — Professional Education
Validity/accuracy: 80.00% (sample size: 5)

Evaluation of GPT-3.5 (GPT series) — Basic Science and Structure of the Skin
Condition/subject: Mohs surgery (English)
Category: Dermatology Examinations and Practice Questions — Professional Education
Validity/accuracy: 80.00% (sample size: 10)
Dataset: 200 text-based questions designed based on the test blueprint of Dermatology SCE in Korea. In Korea the dermatology specialty certification examination (SCE) comprises 200 best-of-five multiple-choice questions (MCQs), half of which consists of clinical photographs and histological images and another half consists of text-based MCQs.
Reference (2024): Joh HC, Kim MH, Ko JY, Kim JS, Jue MS. Evaluating the Performance of ChatGPT in Dermatology Specialty Certificate Examination-style Questions: A Comparative Analysis between English and Korean Language Settings. Indian J Dermatol. 2024 Jul-Aug;69(4):338-341. doi: 10.4103/ijd.ijd_1050_23. Epub 2024 Aug 19. PMID: 39296696; PMCID: PMC11407566.

Evaluation of GPT-3.5, ChatGPT (GPT series) — Erythema, vascular disorders, urticaria (topics tested in Dermatology certificate), English
Condition/subject: Erythema (English)
Category: Dermatology Examinations and Practice Questions — Professional Education
Validity/accuracy: 27.30% (sample size: 11)

Evaluation of GPT-3.5, ChatGPT (GPT series) — Pigmentation, metabolic, genetic, and endocrine disorders (topics tested in Dermatology certificate), Korean
Condition/subject: Pigmentary disorders, Genetics, Endocrinology, Metabolic disorders (Korean)
Category: Dermatology Examinations and Practice Questions — Professional Education
Validity/accuracy: 33.30% (sample size: 18)

Evaluation of GPT-3.5, ChatGPT (GPT series) — Papulosquamous disorders (topics tested in Dermatology certificate), Korean
Condition/subject: Papulosquamous disorders (Korean)
Category: Dermatology Examinations and Practice Questions — Professional Education
Validity/accuracy: 42.10% (sample size: 19)

Evaluation of GPT-3.5, ChatGPT (GPT series) — Symptom, Diagnosis (Evaluation functions tested in Dermatology certificate), Korean
Condition/subject: Diagnostics (Korean)
Category: Dermatology Examinations and Practice Questions — Professional Education
Validity/accuracy: 42.30% (sample size: 59)

Evaluation of GPT-3.5, ChatGPT (GPT series) — Judgement (Cognitive functions tested in Dermatology certificate), Korean
Condition/subject: Judgement (Korean)
Category: Dermatology Examinations and Practice Questions — Professional Education
Validity/accuracy: 46.30% (sample size: 54)

Evaluation of GPT-3.5, ChatGPT (GPT series) — Pigmentation, metabolic, genetic, and endocrine disorders (topics tested in Dermatology certificate), English
Condition/subject: Pigmentary disorders, Genetics, Endocrinology, Metabolic disorders (English)
Category: Dermatology Examinations and Practice Questions — Professional Education
Validity/accuracy: 50.00% (sample size: 18)

Evaluation of GPT-3.5, ChatGPT (GPT series) — Eczematous disorders (topics tested in Dermatology certificate), Korean
Condition/subject: Eczema (Korean)
Category: Dermatology Examinations and Practice Questions — Professional Education
Validity/accuracy: 50.00% (sample size: 22)
Dataset: 200 text-based questions designed based on the test blueprint of Dermatology SCE in Korea. In Korea the dermatology specialty certification examination (SCE) comprises 200 best-of-five multiple-choice questions (MCQs), half of which consists of clinical photographs and histological images and another half consists of text-based MCQs.
Reference (2024): Joh HC, Kim MH, Ko JY, Kim JS, Jue MS. Evaluating the Performance of ChatGPT in Dermatology Specialty Certificate Examination-style Questions: A Comparative Analysis between English and Korean Language Settings. Indian J Dermatol. 2024 Jul-Aug;69(4):338-341. doi: 10.4103/ijd.ijd_1050_23. Epub 2024 Aug 19. PMID: 39296696; PMCID: PMC11407566.

Evaluation of GPT-3.5, ChatGPT (GPT series) — Problem Solving (Cognitive functions tested in Dermatology certificate), Korean
Condition/subject: Problem Solving (Korean)
Category: Dermatology Examinations and Practice Questions — Professional Education
Validity/accuracy: 51.60% (sample size: 64)

Evaluation of GPT-3.5, ChatGPT (GPT series) — Erythema, vascular disorders, urticaria (topics tested in Dermatology certificate), Korean
Condition/subject: Erythema, vascular disorders, urticaria (Korean)
Category: Dermatology Examinations and Practice Questions — Professional Education
Validity/accuracy: 54.50% (sample size: 11)

Evaluation of GPT-3.5, ChatGPT (GPT series) — Autoimmune and vesiculobullous disorders (topics tested in Dermatology certificate), English
Condition/subject: Autoimmune Disorders, Vesiculobullous disorders (English)
Category: Dermatology Examinations and Practice Questions — Professional Education
Validity/accuracy: 55.60% (sample size: 18)

Evaluation of GPT-3.5, ChatGPT (GPT series) — Disorders of skin Appendages and mucous membranes (topics tested in Dermatology certificate), Korean
Condition/subject: Disorders of skin, Certification (Korean)
Category: Dermatology Examinations and Practice Questions — Professional Education
Validity/accuracy: 55.60% (sample size: 18)

Evaluation of GPT-3.5, ChatGPT (GPT series) — Benign and malignant neoplasms (topics tested in Dermatology certificate), Korean
Condition/subject: Benign and malignant neoplasms (Korean)
Category: Dermatology Examinations and Practice Questions — Professional Education
Validity/accuracy: 55.60% (sample size: 18)

Evaluation of GPT-3.5, ChatGPT (GPT series) — Physical stimuli, fat atrophy, cosmetic dermatology, and others (topics tested in Dermatology certificate), English
Condition/subject: Cosmetic Dermatology (English)
Category: Dermatology Examinations and Practice Questions — Professional Education
Validity/accuracy: 57.00% (sample size: 29)

Evaluation of GPT-3.5, ChatGPT (GPT series) — Dermatology Specialty Certificate Examination (Korean)
Condition/subject: Certification (Korean)
Category: Dermatology Examinations and Practice Questions — Professional Education
Validity/accuracy: 57.00% (sample size: 200)
Dataset: 200 text-based questions designed based on the test blueprint of Dermatology SCE in Korea. In Korea the dermatology specialty certification examination (SCE) comprises 200 best-of-five multiple-choice questions (MCQs), half of which consists of clinical photographs and histological images and another half consists of text-based MCQs.
Reference (2024): Joh HC, Kim MH, Ko JY, Kim JS, Jue MS. Evaluating the Performance of ChatGPT in Dermatology Specialty Certificate Examination-style Questions: A Comparative Analysis between English and Korean Language Settings. Indian J Dermatol. 2024 Jul-Aug;69(4):338-341. doi: 10.4103/ijd.ijd_1050_23. Epub 2024 Aug 19. PMID: 39296696; PMCID: PMC11407566.

Evaluation of GPT-3.5, ChatGPT (GPT series) — Treatment, Prognosis (Evaluation functions tested in Dermatology certificate), Korean
Condition/subject: Therapy (Korean)
Category: Dermatology Examinations and Practice Questions — Professional Education
Validity/accuracy: 60.00% (sample size: 70)

Evaluation of GPT-3.5, ChatGPT (GPT series) — Autoimmune and vesiculobullous disorders (topics tested in Dermatology certificate), Korean
Condition/subject: Autoimmune Disorders, Vesiculobullous disorders (Korean)
Category: Dermatology Examinations and Practice Questions — Professional Education
Validity/accuracy: 61.10% (sample size: 18)

Evaluation of GPT-3.5, ChatGPT (GPT series) — Infectious disorders (topics tested in Dermatology certificate), Korean
Condition/subject: Infectious disorders (Korean)
Category: Dermatology Examinations and Practice Questions — Professional Education
Validity/accuracy: 62.10% (sample size: 29)

Evaluation of GPT-3.5, ChatGPT (GPT series) — Symptom, Diagnosis (Evaluation functions tested in Dermatology certificate), English
Condition/subject: Diagnostics (English)
Category: Dermatology Examinations and Practice Questions — Professional Education
Validity/accuracy: 62.70% (sample size: 59)

Evaluation of GPT-3.5, ChatGPT (GPT series) — Papulosquamous disorders (topics tested in Dermatology certificate), English
Condition/subject: Papulosquamous (English)
Category: Dermatology Examinations and Practice Questions — Professional Education
Validity/accuracy: 63.20% (sample size: 19)

Evaluation of GPT-3.5, ChatGPT (GPT series) — Problem Solving (Cognitive functions tested in Dermatology certificate), English
Condition/subject: Problem Solving (English)
Category: Dermatology Examinations and Practice Questions — Professional Education
Validity/accuracy: 64.00% (sample size: 64)

Evaluation of GPT-3.5, ChatGPT (GPT series) — Judgement (Cognitive functions tested in Dermatology certificate), English
Condition/subject: Judgement (English)
Category: Dermatology Examinations and Practice Questions — Professional Education
Validity/accuracy: 64.80% (sample size: 54)

Evaluation of GPT-3.5, ChatGPT (GPT series) — Definition, Cause, Mechanism (Evaluation functions tested in Dermatology certificate), Korean
Condition/subject: Diagnostic Foundations (Korean)
Category: Dermatology Examinations and Practice Questions — Professional Education
Validity/accuracy: 66.20% (sample size: 71)
Dataset: 200 text-based questions designed based on the test blueprint of Dermatology SCE in Korea. In Korea the dermatology specialty certification examination (SCE) comprises 200 best-of-five multiple-choice questions (MCQs), half of which consists of clinical photographs and histological images and another half consists of text-based MCQs.
Reference (2024): Joh HC, Kim MH, Ko JY, Kim JS, Jue MS. Evaluating the Performance of ChatGPT in Dermatology Specialty Certificate Examination-style Questions: A Comparative Analysis between English and Korean Language Settings. Indian J Dermatol. 2024 Jul-Aug;69(4):338-341. doi: 10.4103/ijd.ijd_1050_23. Epub 2024 Aug 19. PMID: 39296696; PMCID: PMC11407566.

Evaluation of GPT-3.5, ChatGPT (GPT series) — Memorization (Cognitive functions tested in Dermatology certificate), Korean
Condition/subject: Memorization (Korean)
Category: Dermatology Examinations and Practice Questions — Professional Education
Validity/accuracy: 68.30% (sample size: 82)

Evaluation of GPT-3.5, ChatGPT (GPT series) — Physical stimuli, fat atrophy, cosmetic dermatology, and others (topics tested in Dermatology certificate), Korean
Condition/subject: Cosmetic Dermatology (Korean)
Category: Dermatology Examinations and Practice Questions — Professional Education
Validity/accuracy: 69.00% (sample size: 29)

Evaluation of GPT-3.5, ChatGPT (GPT series) — Dermatology Specialty Certificate Examination (English)
Condition/subject: Certification (English)
Category: Dermatology Examinations and Practice Questions — Professional Education
Validity/accuracy: 69.00% (sample size: 200)

Evaluation of GPT-3.5, ChatGPT (GPT series) — Treatment, Prognosis (Evaluation functions tested in Dermatology certificate), English
Condition/subject: Therapy (English)
Category: Dermatology Examinations and Practice Questions — Professional Education
Validity/accuracy: 70.00% (sample size: 70)

Evaluation of GPT-3.5, ChatGPT (GPT series) — Eczematous disorders (topics tested in Dermatology certificate), English
Condition/subject: Eczema (English)
Category: Dermatology Examinations and Practice Questions — Professional Education
Validity/accuracy: 72.70% (sample size: 22)

Evaluation of GPT-3.5, ChatGPT (GPT series) — Definition, Cause, Mechanism (Evaluation functions tested in Dermatology certificate), English
Condition/subject: Diagnostic Foundations (English)
Category: Dermatology Examinations and Practice Questions — Professional Education
Validity/accuracy: 73.20% (sample size: 71)

Evaluation of GPT-3.5, ChatGPT (GPT series) — Memorization (Cognitive functions tested in Dermatology certificate), English
Condition/subject: Memorization (English)
Category: Dermatology Examinations and Practice Questions — Professional Education
Validity/accuracy: 75.60% (sample size: 82)

Evaluation of GPT-3.5, ChatGPT (GPT series) — Basic Dermatology (topics tested in Dermatology certificate), Korean
Condition/subject: Basic dermatology (Korean)
Category: Dermatology Examinations and Practice Questions — Professional Education
Validity/accuracy: 77.80% (sample size: 18)
Dataset: 200 text-based questions designed based on the test blueprint of Dermatology SCE in Korea. In Korea the dermatology specialty certification examination (SCE) comprises 200 best-of-five multiple-choice questions (MCQs), half of which consists of clinical photographs and histological images and another half consists of text-based MCQs.
Reference (2024): Joh HC, Kim MH, Ko JY, Kim JS, Jue MS. Evaluating the Performance of ChatGPT in Dermatology Specialty Certificate Examination-style Questions: A Comparative Analysis between English and Korean Language Settings. Indian J Dermatol. 2024 Jul-Aug;69(4):338-341. doi: 10.4103/ijd.ijd_1050_23. Epub 2024 Aug 19. PMID: 39296696; PMCID: PMC11407566.

Evaluation of GPT-3.5, ChatGPT (GPT series) — Infectious disorders (topics tested in Dermatology certificate), English
Condition/subject: Infectious disorders (English)
Category: Dermatology Examinations and Practice Questions — Professional Education
Validity/accuracy: 82.80% (sample size: 29)

Evaluation of GPT-3.5, ChatGPT (GPT series) — Basic Dermatology (topics tested in Dermatology certificate), English
Condition/subject: Basic dermatology (English)
Category: Dermatology Examinations and Practice Questions — Professional Education
Validity/accuracy: 83.30% (sample size: 18)

Evaluation of GPT-3.5, ChatGPT (GPT series) — Benign and malignant neoplasms (topics tested in Dermatology certificate), English
Condition/subject: Benign and malignant neoplasms (English)
Category: Dermatology Examinations and Practice Questions — Professional Education
Validity/accuracy: 83.30% (sample size: 18)

Evaluation of GPT-3.5, ChatGPT (GPT series) — Disorders of skin Appendages and mucous membranes (topics tested in Dermatology certificate), English
Condition/subject: Disorders of skin, Certification (English)
Category: Dermatology Examinations and Practice Questions — Professional Education
Validity/accuracy: 88.90% (sample size: 18)