Aurametrix/dermatology_longitudinal huggingface dataset:
Dataset: 60 real case scenarios, with half being neoplastic conditions and the other half non-neoplastic, selected by a pathologist from a hospital�s medical database. The cases involved patients who had undergone biopsy and histopathological examination for skin conditions.
Reference (2025): Talar Sabir Ahmed, Rawa M. Ali, Ari M. Abdullah, Hadeel A. Yasseen, Ronak S. Ahmed, Ameer M. Salih, et al. Exploring Large Language Models Integration in the Histopathologic Diagnosis of Skin Diseases: A Comparative Study. Barw Medical Journal. 2025 Apr. 30;3(3):6-12. doi: https://doi.org/10.58742/bmj.v3i3.180

Evaluation of Gemini (Google's Family of LLMs) — Diagnostics from histopathologic report texts, Agreement with external pathologist
Condition/subject: Pustular psoriasis (English)
Category: Medical Records and Diagnostic Processes — Clinical Practice
Validity/accuracy: 0.00% (sample size: 1)
Comments: Agreement: None agreement. Description: A 53-year-old female patient presented with a maculopapular and pustular rash over the upper back bi...

Evaluation of ChatGPT-3.5 (GPT series) — Diagnostics from histopathologic report texts, Agreement with external pathologist
Condition/subject: Actinic keratosis, Bowenoid type (English)
Category: Medical Records and Diagnostic Processes — Clinical Practice
Validity/accuracy: 100.00% (sample size: 1)
Comments: Agreement: Complete agreement. Description: A 29-year-old male patient with albinism has numerous skin lesions. Excisional biopsy of one lesion ...

Evaluation of Gemini (Google's Family of LLMs) — Diagnostics from histopathologic report texts, Agreement with external pathologist
Condition/subject: Actinic keratosis, Bowenoid type (English)
Category: Medical Records and Diagnostic Processes — Clinical Practice
Validity/accuracy: 0.00% (sample size: 1)
Comments: Agreement: None agreement. Description: A 29-year-old male patient with albinism has numerous skin lesions. Excisional biopsy of one lesion ...

Evaluation of ChatGPT-3.5 (GPT series) — Diagnostics from histopathologic report texts, Agreement with external pathologist
Condition/subject: Lichen planopilaris (English)
Category: Medical Records and Diagnostic Processes — Clinical Practice
Validity/accuracy: 100.00% (sample size: 1)
Comments: Agreement: Complete agreement. Description: A 17-year-old male patient presented with erythematous, scaly plaques with central scarring over the...

Evaluation of Gemini (Google's Family of LLMs) — Diagnostics from histopathologic report texts, Agreement with external pathologist
Condition/subject: Lichen planopilaris (English)
Category: Medical Records and Diagnostic Processes — Clinical Practice
Validity/accuracy: 0.00% (sample size: 1)
Comments: Agreement: None agreement. Description: A 17-year-old male patient presented with erythematous, scaly plaques with central scarring over the...
Dataset: 60 real case scenarios, with half being neoplastic conditions and the other half non-neoplastic, selected by a pathologist from a hospital�s medical database. The cases involved patients who had undergone biopsy and histopathological examination for skin conditions.
Reference (2025): Talar Sabir Ahmed, Rawa M. Ali, Ari M. Abdullah, Hadeel A. Yasseen, Ronak S. Ahmed, Ameer M. Salih, et al. Exploring Large Language Models Integration in the Histopathologic Diagnosis of Skin Diseases: A Comparative Study. Barw Medical Journal. 2025 Apr. 30;3(3):6-12. doi: https://doi.org/10.58742/bmj.v3i3.180

Evaluation of ChatGPT-3.5 (GPT series) — Diagnostics from histopathologic report texts, Agreement with external pathologist
Condition/subject: Nevus sebaceus with basaloid follicular hamartoma (English)
Category: Medical Records and Diagnostic Processes — Clinical Practice
Validity/accuracy: 50.00% (sample size: 1)
Comments: Agreement: Partial agreement. Description: A 25-year-old male patient underwent wide local excision of a skin lesion. Histology shows acanthosi...

Evaluation of Gemini (Google's Family of LLMs) — Diagnostics from histopathologic report texts, Agreement with external pathologist
Condition/subject: Nevus sebaceus with basaloid follicular hamartoma (English)
Category: Medical Records and Diagnostic Processes — Clinical Practice
Validity/accuracy: 0.00% (sample size: 1)
Comments: Agreement: None agreement. Description: A 25-year-old male patient underwent wide local excision of a skin lesion. Histology shows acanthosi...

Evaluation of ChatGPT-3.5 (GPT series) — Diagnostics from histopathologic report texts, Agreement with external pathologist
Condition/subject: Dermatofibrosarcoma protuberans (English)
Category: Medical Records and Diagnostic Processes — Clinical Practice
Validity/accuracy: 0.00% (sample size: 1)
Comments: Agreement: None agreement. Description: A 23-year-old female patient has a skin lesion excised. Histology shows an ill-defined proliferation...

Evaluation of Gemini (Google's Family of LLMs) — Diagnostics from histopathologic report texts, Agreement with external pathologist
Condition/subject: Dermatofibrosarcoma protuberans (English)
Category: Medical Records and Diagnostic Processes — Clinical Practice
Validity/accuracy: 0.00% (sample size: 1)
Comments: Agreement: None agreement. Description: A 23-year-old female patient has a skin lesion excised. Histology shows an ill-defined proliferation...

Evaluation of ChatGPT-3.5 (GPT series) — Diagnostics from histopathologic report texts, Agreement with external pathologist
Condition/subject: Morphea (English)
Category: Medical Records and Diagnostic Processes — Clinical Practice
Validity/accuracy: 100.00% (sample size: 1)
Comments: Agreement: Complete agreement. Description: A 45-year-old male patient presented with gradual hardening of the skin of both legs and thighs for ...
Dataset: 60 real case scenarios, with half being neoplastic conditions and the other half non-neoplastic, selected by a pathologist from a hospital�s medical database. The cases involved patients who had undergone biopsy and histopathological examination for skin conditions.
Reference (2025): Talar Sabir Ahmed, Rawa M. Ali, Ari M. Abdullah, Hadeel A. Yasseen, Ronak S. Ahmed, Ameer M. Salih, et al. Exploring Large Language Models Integration in the Histopathologic Diagnosis of Skin Diseases: A Comparative Study. Barw Medical Journal. 2025 Apr. 30;3(3):6-12. doi: https://doi.org/10.58742/bmj.v3i3.180

Evaluation of Gemini (Google's Family of LLMs) — Diagnostics from histopathologic report texts, Agreement with external pathologist
Condition/subject: Morphea (English)
Category: Medical Records and Diagnostic Processes — Clinical Practice
Validity/accuracy: 0.00% (sample size: 1)
Comments: Agreement: None agreement. Description: A 45-year-old male patient presented with gradual hardening of the skin of both legs and thighs for ...

Evaluation of ChatGPT-3.5 (GPT series) — Diagnostics from histopathologic report texts, Agreement with external pathologist
Condition/subject: Seborrheic keratosis, irritated type (English)
Category: Medical Records and Diagnostic Processes — Clinical Practice
Validity/accuracy: 50.00% (sample size: 1)
Comments: Agreement: Partial agreement. Description: A 57-year-old male patient presented with a forehead skin lesion that was excised. Histology shows s...

Evaluation of Gemini (Google's Family of LLMs) — Diagnostics from histopathologic report texts, Agreement with external pathologist
Condition/subject: Seborrheic keratosis, irritated type (English)
Category: Medical Records and Diagnostic Processes — Clinical Practice
Validity/accuracy: 50.00% (sample size: 1)
Comments: Agreement: Partial agreement. Description: A 57-year-old male patient presented with a forehead skin lesion that was excised. Histology shows s...

Evaluation of ChatGPT-3.5 (GPT series) — Diagnostics from histopathologic report texts, Agreement with external pathologist
Condition/subject: Focal acantholytic dyskeratosis; differentials include Darier disease and others (English)
Category: Medical Records and Diagnostic Processes — Clinical Practice
Validity/accuracy: 100.00% (sample size: 1)
Comments: Agreement: Complete agreement. Description: A 68-year-old male patient presented with a right inguinal skin lesion. Excision biopsy shows focal ...

Evaluation of Gemini (Google's Family of LLMs) — Diagnostics from histopathologic report texts, Agreement with external pathologist
Condition/subject: Focal acantholytic dyskeratosis; differentials include Darier disease and others (English)
Category: Medical Records and Diagnostic Processes — Clinical Practice
Validity/accuracy: 0.00% (sample size: 1)
Comments: Agreement: None agreement. Description: A 68-year-old male patient presented with a right inguinal skin lesion. Excision biopsy shows focal ...
Dataset: 60 real case scenarios, with half being neoplastic conditions and the other half non-neoplastic, selected by a pathologist from a hospital�s medical database. The cases involved patients who had undergone biopsy and histopathological examination for skin conditions.
Reference (2025): Talar Sabir Ahmed, Rawa M. Ali, Ari M. Abdullah, Hadeel A. Yasseen, Ronak S. Ahmed, Ameer M. Salih, et al. Exploring Large Language Models Integration in the Histopathologic Diagnosis of Skin Diseases: A Comparative Study. Barw Medical Journal. 2025 Apr. 30;3(3):6-12. doi: https://doi.org/10.58742/bmj.v3i3.180

Evaluation of ChatGPT-3.5 (GPT series) — Diagnostics from histopathologic report texts, Agreement with external pathologist
Condition/subject: Cutaneous xanthogranuloma (English)
Category: Medical Records and Diagnostic Processes — Clinical Practice
Validity/accuracy: 0.00% (sample size: 1)
Comments: Agreement: None agreement. Description: A 31-year-old male patient presented with an asymptomatic papule on the right temple for three month...

Evaluation of Gemini (Google's Family of LLMs) — Diagnostics from histopathologic report texts, Agreement with external pathologist
Condition/subject: Cutaneous xanthogranuloma (English)
Category: Medical Records and Diagnostic Processes — Clinical Practice
Validity/accuracy: 0.00% (sample size: 1)
Comments: Agreement: None agreement. Description: A 31-year-old male patient presented with an asymptomatic papule on the right temple for three month...

Evaluation of ChatGPT-3.5 (GPT series) — Diagnostics from histopathologic report texts, Agreement with external pathologist
Condition/subject: Rosacea (English)
Category: Medical Records and Diagnostic Processes — Clinical Practice
Validity/accuracy: 50.00% (sample size: 1)
Comments: Agreement: Partial agreement. Description: A 75-year-old male patient presented with a right cheek skin lesion. Excisional biopsy shows multipl...

Evaluation of Gemini (Google's Family of LLMs) — Diagnostics from histopathologic report texts, Agreement with external pathologist
Condition/subject: Rosacea (English)
Category: Medical Records and Diagnostic Processes — Clinical Practice
Validity/accuracy: 0.00% (sample size: 1)
Comments: Agreement: None agreement. Description: A 75-year-old male patient presented with a right cheek skin lesion. Excisional biopsy shows multipl...

Evaluation of ChatGPT-3.5 (GPT series) — Diagnostics from histopathologic report texts, Agreement with external pathologist
Condition/subject: Prurigo nodularis (English)
Category: Medical Records and Diagnostic Processes — Clinical Practice
Validity/accuracy: 0.00% (sample size: 1)
Comments: Agreement: None agreement. Description: A 37-year-old female patient presented with a recurrent painless lesion under the chin. Clinical dif...
Dataset: 60 real case scenarios, with half being neoplastic conditions and the other half non-neoplastic, selected by a pathologist from a hospital�s medical database. The cases involved patients who had undergone biopsy and histopathological examination for skin conditions.
Reference (2025): Talar Sabir Ahmed, Rawa M. Ali, Ari M. Abdullah, Hadeel A. Yasseen, Ronak S. Ahmed, Ameer M. Salih, et al. Exploring Large Language Models Integration in the Histopathologic Diagnosis of Skin Diseases: A Comparative Study. Barw Medical Journal. 2025 Apr. 30;3(3):6-12. doi: https://doi.org/10.58742/bmj.v3i3.180

Evaluation of Gemini (Google's Family of LLMs) — Diagnostics from histopathologic report texts, Agreement with external pathologist
Condition/subject: Prurigo nodularis (English)
Category: Medical Records and Diagnostic Processes — Clinical Practice
Validity/accuracy: 0.00% (sample size: 1)
Comments: Agreement: None agreement. Description: A 37-year-old female patient presented with a recurrent painless lesion under the chin. Clinical dif...

Evaluation of ChatGPT-3.5 (GPT series) — Diagnostics from histopathologic report texts, Agreement with external pathologist
Condition/subject: Prurigo nodularis (English)
Category: Medical Records and Diagnostic Processes — Clinical Practice
Validity/accuracy: 0.00% (sample size: 1)
Comments: Agreement: None agreement. Description: A 47-year-old male patient who had a history of diabetes mellitus for 7 years and was on regular tre...

Evaluation of Gemini (Google's Family of LLMs) — Diagnostics from histopathologic report texts, Agreement with external pathologist
Condition/subject: Prurigo nodularis (English)
Category: Medical Records and Diagnostic Processes — Clinical Practice
Validity/accuracy: 0.00% (sample size: 1)
Comments: Agreement: None agreement. Description: A 47-year-old male patient who had a history of diabetes mellitus for 7 years and was on regular tre...

Evaluation of ChatGPT-3.5 (GPT series) — Diagnostics from histopathologic report texts, Agreement with external pathologist
Condition/subject: Kaposi sarcoma, nodular stage (English)
Category: Medical Records and Diagnostic Processes — Clinical Practice
Validity/accuracy: 100.00% (sample size: 1)
Comments: Agreement: Complete agreement. Description: A 42-year-old male patient presented with a skin lesion on the ventral surface of the glans penis. A...

Evaluation of Gemini (Google's Family of LLMs) — Diagnostics from histopathologic report texts, Agreement with external pathologist
Condition/subject: Kaposi sarcoma, nodular stage (English)
Category: Medical Records and Diagnostic Processes — Clinical Practice
Validity/accuracy: 0.00% (sample size: 1)
Comments: Agreement: None agreement. Description: A 42-year-old male patient presented with a skin lesion on the ventral surface of the glans penis. A...
Dataset: 60 real case scenarios, with half being neoplastic conditions and the other half non-neoplastic, selected by a pathologist from a hospital�s medical database. The cases involved patients who had undergone biopsy and histopathological examination for skin conditions.
Reference (2025): Talar Sabir Ahmed, Rawa M. Ali, Ari M. Abdullah, Hadeel A. Yasseen, Ronak S. Ahmed, Ameer M. Salih, et al. Exploring Large Language Models Integration in the Histopathologic Diagnosis of Skin Diseases: A Comparative Study. Barw Medical Journal. 2025 Apr. 30;3(3):6-12. doi: https://doi.org/10.58742/bmj.v3i3.180

Evaluation of ChatGPT-3.5 (GPT series) — Diagnostics from histopathologic report texts, Agreement with external pathologist
Condition/subject: Late-stage follicular lichen planus (lichen planopilaris) (English)
Category: Medical Records and Diagnostic Processes — Clinical Practice
Validity/accuracy: 100.00% (sample size: 1)
Comments: Agreement: Complete agreement. Description: A 67-year-old male patient presented with a red, itchy, atrophic patch on the scalp with adherent sc...

Evaluation of Gemini (Google's Family of LLMs) — Diagnostics from histopathologic report texts, Agreement with external pathologist
Condition/subject: Late-stage follicular lichen planus (lichen planopilaris) (English)
Category: Medical Records and Diagnostic Processes — Clinical Practice
Validity/accuracy: 0.00% (sample size: 1)
Comments: Agreement: None agreement. Description: A 67-year-old male patient presented with a red, itchy, atrophic patch on the scalp with adherent sc...

Evaluation of ChatGPT-3.5 (GPT series) — Diagnostics from histopathologic report texts, Agreement with external pathologist
Condition/subject: Pityriasis rosea (English)
Category: Medical Records and Diagnostic Processes — Clinical Practice
Validity/accuracy: 0.00% (sample size: 1)
Comments: Agreement: None agreement. Description: A 71-year-old male patient presented with diffuse erythematous and eczematous patches. An incisional...

Evaluation of Gemini (Google's Family of LLMs) — Diagnostics from histopathologic report texts, Agreement with external pathologist
Condition/subject: Pityriasis rosea (English)
Category: Medical Records and Diagnostic Processes — Clinical Practice
Validity/accuracy: 0.00% (sample size: 1)
Comments: Agreement: None agreement. Description: A 71-year-old male patient presented with diffuse erythematous and eczematous patches. An incisional...

Evaluation of ChatGPT-3.5 (GPT series) — Diagnostics from histopathologic report texts, Agreement with external pathologist
Condition/subject: Lichen planus (English)
Category: Medical Records and Diagnostic Processes — Clinical Practice
Validity/accuracy: 100.00% (sample size: 1)
Comments: Agreement: Complete agreement. Description: A 34-year-old female patient presented with multiple very itchy papules that developed 8 months ago ...
Dataset: 60 real case scenarios, with half being neoplastic conditions and the other half non-neoplastic, selected by a pathologist from a hospital�s medical database. The cases involved patients who had undergone biopsy and histopathological examination for skin conditions.
Reference (2025): Talar Sabir Ahmed, Rawa M. Ali, Ari M. Abdullah, Hadeel A. Yasseen, Ronak S. Ahmed, Ameer M. Salih, et al. Exploring Large Language Models Integration in the Histopathologic Diagnosis of Skin Diseases: A Comparative Study. Barw Medical Journal. 2025 Apr. 30;3(3):6-12. doi: https://doi.org/10.58742/bmj.v3i3.180

Evaluation of Gemini (Google's Family of LLMs) — Diagnostics from histopathologic report texts, Agreement with external pathologist
Condition/subject: Lichen planus (English)
Category: Medical Records and Diagnostic Processes — Clinical Practice
Validity/accuracy: 100.00% (sample size: 1)
Comments: Agreement: Complete agreement. Description: A 34-year-old female patient presented with multiple very itchy papules that developed 8 months ago ...

Evaluation of ChatGPT-3.5 (GPT series) — Diagnostics from histopathologic report texts, Agreement with external pathologist
Condition/subject: Necrobiosis lipoidica (English)
Category: Medical Records and Diagnostic Processes — Clinical Practice
Validity/accuracy: 100.00% (sample size: 1)
Comments: Agreement: Complete agreement. Description: A 13-year-old female patient with type 1 diabetes mellitus presented with an itchy lesion on the lef...

Evaluation of Gemini (Google's Family of LLMs) — Diagnostics from histopathologic report texts, Agreement with external pathologist
Condition/subject: Necrobiosis lipoidica (English)
Category: Medical Records and Diagnostic Processes — Clinical Practice
Validity/accuracy: 100.00% (sample size: 1)
Comments: Agreement: Complete agreement. Description: A 13-year-old female patient with type 1 diabetes mellitus presented with an itchy lesion on the lef...
Reference (2025): Needle, Carli et al. 66: Assessing Accuracy, Completeness, and Readability of ChatGPT-Generated Handouts on Pediatric Dermatology Topics Journal of Investigative Dermatology, 2025, Volume 145, Issue 3, e17

Evaluation of ChatGPT-3.5 (GPT series) — Accuracy of LLM-generated handouts
Condition/subject: Eczema (English)
Category: Patient Education Materials and Readability Studies — Patient Education
Validity/accuracy: 81.33% (sample size: 6)
Comments: LLM-generated Handouts were evaluated by pediatric dermatologists using 3-point scale. Five pediatric dermatology conditions were selected based on prevalence and complexity. 6 pediatric dermatologists reviewed handouts for Eczema and 4 for Juvenile Xanthogranuloma (JXG). For Eczema, ChatGPT 3.5 scored 2.44 (accuracy) and 2.56 (completeness), while ChatGPT 4 scored 2.47 (accuracy) and 2.57 (completeness). For JXG, ChatGPT 3.5 scored 2.56 (accuracy) and 2.69 (completeness), while ChatGPT 4 scored 2.81 (accuracy) and 3.00 (completeness). Validity scores are based on a 3-point scale, converted to percentages: Complete (3/3) = 100% Partial (2/3) ? 66.67%  Minimal (1/3) ? 33.33%

Evaluation of ChatGPT-3.5 (GPT series) — Completeness of LLM-generated handouts
Condition/subject: Eczema (English)
Category: Patient Education Materials and Readability Studies — Patient Education
Validity/accuracy: 85.33% (sample size: 6)
Comments: LLM-generated Handouts were evaluated by pediatric dermatologists using 3-point scale. Five pediatric dermatology conditions were selected based on prevalence and complexity. 6 pediatric dermatologists reviewed handouts for Eczema and 4 for Juvenile Xanthogranuloma (JXG). For Eczema, ChatGPT 3.5 scored 2.44 (accuracy) and 2.56 (completeness), while ChatGPT 4 scored 2.47 (accuracy) and 2.57 (completeness). For JXG, ChatGPT 3.5 scored 2.56 (accuracy) and 2.69 (completeness), while ChatGPT 4 scored 2.81 (accuracy) and 3.00 (completeness). Validity scores are based on a 3-point scale, converted to percentages: Complete (3/3) = 100% Partial (2/3) ? 66.67%  Minimal (1/3) ? 33.33%

Evaluation of ChatGPT-4 (GPT series) — Accuracy of LLM-generated handouts
Condition/subject: Eczema (English)
Category: Patient Education Materials and Readability Studies — Patient Education
Validity/accuracy: 82.33% (sample size: 6)
Comments: LLM-generated Handouts were evaluated by pediatric dermatologists using 3-point scale. Five pediatric dermatology conditions were selected based on prevalence and complexity. 6 pediatric dermatologists reviewed handouts for Eczema and 4 for Juvenile Xanthogranuloma (JXG). For Eczema, ChatGPT 3.5 scored 2.44 (accuracy) and 2.56 (completeness), while ChatGPT 4 scored 2.47 (accuracy) and 2.57 (completeness). For JXG, ChatGPT 3.5 scored 2.56 (accuracy) and 2.69 (completeness), while ChatGPT 4 scored 2.81 (accuracy) and 3.00 (completeness). Validity scores are based on a 3-point scale, converted to percentages: Complete (3/3) = 100% Partial (2/3) ? 66.67%  Minimal (1/3) ? 33.33%
Reference (2025): Needle, Carli et al. 66: Assessing Accuracy, Completeness, and Readability of ChatGPT-Generated Handouts on Pediatric Dermatology Topics Journal of Investigative Dermatology, 2025, Volume 145, Issue 3, e17

Evaluation of ChatGPT-4 (GPT series) — Completeness of LLM-generated handouts
Condition/subject: Eczema (English)
Category: Patient Education Materials and Readability Studies — Patient Education
Validity/accuracy: 85.67% (sample size: 6)
Comments: LLM-generated Handouts were evaluated by pediatric dermatologists using 3-point scale. Five pediatric dermatology conditions were selected based on prevalence and complexity. 6 pediatric dermatologists reviewed handouts for Eczema and 4 for Juvenile Xanthogranuloma (JXG). For Eczema, ChatGPT 3.5 scored 2.44 (accuracy) and 2.56 (completeness), while ChatGPT 4 scored 2.47 (accuracy) and 2.57 (completeness). For JXG, ChatGPT 3.5 scored 2.56 (accuracy) and 2.69 (completeness), while ChatGPT 4 scored 2.81 (accuracy) and 3.00 (completeness). Validity scores are based on a 3-point scale, converted to percentages: Complete (3/3) = 100% Partial (2/3) ? 66.67%  Minimal (1/3) ? 33.33%

Evaluation of ChatGPT-3.5 (GPT series) — Accuracy of LLM-generated handouts
Condition/subject: Juvenile Xanthogranuloma (JXG) (English)
Category: Patient Education Materials and Readability Studies — Patient Education
Validity/accuracy: 85.33% (sample size: 4)
Comments: LLM-generated Handouts were evaluated by pediatric dermatologists using 3-point scale. Five pediatric dermatology conditions were selected based on prevalence and complexity. 6 pediatric dermatologists reviewed handouts for Eczema and 4 for Juvenile Xanthogranuloma (JXG). For Eczema, ChatGPT 3.5 scored 2.44 (accuracy) and 2.56 (completeness), while ChatGPT 4 scored 2.47 (accuracy) and 2.57 (completeness). For JXG, ChatGPT 3.5 scored 2.56 (accuracy) and 2.69 (completeness), while ChatGPT 4 scored 2.81 (accuracy) and 3.00 (completeness). Validity scores are based on a 3-point scale, converted to percentages: Complete (3/3) = 100% Partial (2/3) ? 66.67%  Minimal (1/3) ? 33.33%

Evaluation of ChatGPT-3.5 (GPT series) — Completeness of LLM-generated handouts
Condition/subject: Juvenile Xanthogranuloma (JXG) (English)
Category: Patient Education Materials and Readability Studies — Patient Education
Validity/accuracy: 89.67% (sample size: 4)
Comments: LLM-generated Handouts were evaluated by pediatric dermatologists using 3-point scale. Five pediatric dermatology conditions were selected based on prevalence and complexity. 6 pediatric dermatologists reviewed handouts for Eczema and 4 for Juvenile Xanthogranuloma (JXG). For Eczema, ChatGPT 3.5 scored 2.44 (accuracy) and 2.56 (completeness), while ChatGPT 4 scored 2.47 (accuracy) and 2.57 (completeness). For JXG, ChatGPT 3.5 scored 2.56 (accuracy) and 2.69 (completeness), while ChatGPT 4 scored 2.81 (accuracy) and 3.00 (completeness). Validity scores are based on a 3-point scale, converted to percentages: Complete (3/3) = 100% Partial (2/3) ? 66.67%  Minimal (1/3) ? 33.33%
Reference (2025): Needle, Carli et al. 66: Assessing Accuracy, Completeness, and Readability of ChatGPT-Generated Handouts on Pediatric Dermatology Topics Journal of Investigative Dermatology, 2025, Volume 145, Issue 3, e17

Evaluation of ChatGPT-4 (GPT series) — Accuracy of LLM-generated handouts
Condition/subject: Juvenile Xanthogranuloma (JXG) (English)
Category: Patient Education Materials and Readability Studies — Patient Education
Validity/accuracy: 93.67% (sample size: 4)
Comments: LLM-generated Handouts were evaluated by pediatric dermatologists using 3-point scale. Five pediatric dermatology conditions were selected based on prevalence and complexity. 6 pediatric dermatologists reviewed handouts for Eczema and 4 for Juvenile Xanthogranuloma (JXG). For Eczema, ChatGPT 3.5 scored 2.44 (accuracy) and 2.56 (completeness), while ChatGPT 4 scored 2.47 (accuracy) and 2.57 (completeness). For JXG, ChatGPT 3.5 scored 2.56 (accuracy) and 2.69 (completeness), while ChatGPT 4 scored 2.81 (accuracy) and 3.00 (completeness). Validity scores are based on a 3-point scale, converted to percentages: Complete (3/3) = 100% Partial (2/3) ? 66.67%  Minimal (1/3) ? 33.33%

Evaluation of ChatGPT-4 (GPT series) — Completeness of LLM-generated handouts
Condition/subject: Juvenile Xanthogranuloma (JXG) (English)
Category: Patient Education Materials and Readability Studies — Patient Education
Validity/accuracy: 100.00% (sample size: 4)
Comments: LLM-generated Handouts were evaluated by pediatric dermatologists using 3-point scale. Five pediatric dermatology conditions were selected based on prevalence and complexity. 6 pediatric dermatologists reviewed handouts for Eczema and 4 for Juvenile Xanthogranuloma (JXG). For Eczema, ChatGPT 3.5 scored 2.44 (accuracy) and 2.56 (completeness), while ChatGPT 4 scored 2.47 (accuracy) and 2.57 (completeness). For JXG, ChatGPT 3.5 scored 2.56 (accuracy) and 2.69 (completeness), while ChatGPT 4 scored 2.81 (accuracy) and 3.00 (completeness). Validity scores are based on a 3-point scale, converted to percentages: Complete (3/3) = 100% Partial (2/3) ? 66.67%  Minimal (1/3) ? 33.33%
Dataset: 542 Medical domain-specific questions (MCQs) including 144 dermatology questions
Reference (2025): Panagoulias, D.P. et al. (2025). COGnitive Network Evaluation Toolkit for Medical Domains: Evaluating Large Language Model Performance in Medical Diagnostics�An Assessment Framework and Dataset for Healthcare AI. In: Nagar, A., Jat, D.S., Mishra, D., Joshi, A. (eds) Intelligent Sustainable Systems. Worlds4 2024. Lecture Notes in Networks and Systems, vol 1178. Springer, Singapore. https://doi.org/10.1007/978-981-97-9559-8_38  https://huggingface.co/datasets/DimitriosPanagoulias/COGNET-MD/viewer/default/train?q=dermatology&views%5B%5D=train

Evaluation of ChatGPT 4 (GPT series) — CognetMD benchmark questions
Condition/subject: General Dermatology (English)
Category: Dermatology Examinations and Practice Questions — Professional Education
Validity/accuracy: 55.00% (sample size: 542)
Comments: We found that GPT-4 performed well regarding questions with only one correct answer, as it achieved full credit for 64/75 questions in Psychiatry, 26/36 in Pulmonology and 53/76 in Dentistry. However, when penalized for erroneous choices, GPT-4 scored averagely, achieving only a 55% accuracy

Evaluation of ChatGPT o3 5-28-2025 (GPT series) — On the Following Dermatology Multiple choice Quiz choose correct response or responses. This is an example question: Which condition may include generalised pruritus as a symptom? A. Hodgkins disease B. Pityriasis rosea C. Diabetes mellitus D. Haemolytic jaundice E. Polycythaemia rubra vera 
Condition/subject: Itchy skin (pruritus) (English)
Category: Dermatology Examinations and Practice Questions — Professional Education
Validity/accuracy: 100.00% (sample size: 1)
Comments: Correct responses: A, B, E.  A: Hodgkin�s disease Generalised (sometimes intense) pruritus is a well-recognised paraneoplastic symptom and may precede other signs of the lymphoma;  B Pityriasis rosea Around half of patients report troublesome itch; because the secondary eruption is widespread over the trunk and limbs, the pruritus is often perceived as generalised;  E Polycythaemia rubra vera Aquagenic or spontaneous generalised itching affects up to 70 % of patients with this myeloproliferative disorder.

Evaluation of ChatGPT o3 4-16-2025 (GPT series) — On the Following Dermatology Multiple choice Quiz choose correct response or responses. This is an example question: Which condition may include generalised pruritus as a symptom? A. Hodgkins disease B. Pityriasis rosea C. Diabetes mellitus D. Haemolytic jaundice E. Polycythaemia rubra vera 
Condition/subject: Itchy skin (pruritus) (English)
Category: Dermatology Examinations and Practice Questions — Professional Education
Validity/accuracy: 83.33% (sample size: 1)
Comments: Correct answers: A. Hodgkin�s disease E. Polycythaemia rubra vera. B: Pityriasis rosea may be mildly itchy within the lesions but isn�t a systemic cause of generalised pruritus.
Dataset: 542 Medical domain-specific questions (MCQs) including 144 dermatology questions
Reference (2025): Panagoulias, D.P. et al. (2025). COGnitive Network Evaluation Toolkit for Medical Domains: Evaluating Large Language Model Performance in Medical Diagnostics�An Assessment Framework and Dataset for Healthcare AI. In: Nagar, A., Jat, D.S., Mishra, D., Joshi, A. (eds) Intelligent Sustainable Systems. Worlds4 2024. Lecture Notes in Networks and Systems, vol 1178. Springer, Singapore. https://doi.org/10.1007/978-981-97-9559-8_38  https://huggingface.co/datasets/DimitriosPanagoulias/COGNET-MD/viewer/default/train?q=dermatology&views%5B%5D=train

Evaluation of command-a-03-2025 (Cohere series) — On the Following Dermatology Multiple choice Quiz choose correct response or responses. This is an example question: Which condition may include generalised pruritus as a symptom? A. Hodgkins disease B. Pityriasis rosea C. Diabetes mellitus D. Haemolytic jaundice E. Polycythaemia rubra vera 
Condition/subject: Itchy skin (pruritus) (English)
Category: Dermatology Examinations and Practice Questions — Professional Education
Validity/accuracy: 66.67% (sample size: 1)
Comments: The correct responses that may include **generalised pruritus** as a symptom are: **A. Hodgkins disease** , **D. Haemolytic jaundice** , **E. Polycythaemia rubra vera**- **Hodgkins disease (A):** Generalised pruritus is a well-known symptom in Hodgkins lymphoma, often associated with the release of cytokines and other inflammatory mediators.  - **Haemolytic jaundice (D):** Pruritus is a common symptom in haemolytic conditions due to the accumulation of bile salts in the skin.  - **Polycythaemia rubra vera (E):** Pruritus, especially after warm water exposure, is a classic symptom of polycythaemia rubra vera, likely due to increased histamine release.  **B. Pityriasis rosea (B)** and **C. Diabetes mellitus (C)** are not typically associated with generalised pruritus as a primary symptom. Pityriasis rosea may cause localized itching, and diabetes can cause pruritus in specific areas (e.g., due to dry skin or fungal infections), but not generalized pruritus as a hallmark symptom.

Evaluation of Gemini 2.5 Flash (Google's family of models) — On the Following Dermatology Multiple choice Quiz choose correct response or responses. This is an example question: Which condition may include generalised pruritus as a symptom? A. Hodgkins disease B. Pityriasis rosea C. Diabetes mellitus D. Haemolytic jaundice E. Polycythaemia rubra vera 
Condition/subject: Itchy skin (pruritus) (English)
Category: Dermatology Examinations and Practice Questions — Professional Education
Validity/accuracy: 33.33% (sample size: 1)
Comments: All of the listed conditions (A. Hodgkin's disease, B. Pityriasis rosea, C. Diabetes mellitus, D. Haemolytic jaundice, E. Polycythaemia rubra vera) can include generalized pruritus as a symptom.
Dataset: 542 Medical domain-specific questions (MCQs) including 144 dermatology questions
Reference (2025): Panagoulias, D.P. et al. (2025). COGnitive Network Evaluation Toolkit for Medical Domains: Evaluating Large Language Model Performance in Medical Diagnostics�An Assessment Framework and Dataset for Healthcare AI. In: Nagar, A., Jat, D.S., Mishra, D., Joshi, A. (eds) Intelligent Sustainable Systems. Worlds4 2024. Lecture Notes in Networks and Systems, vol 1178. Springer, Singapore. https://doi.org/10.1007/978-981-97-9559-8_38  https://huggingface.co/datasets/DimitriosPanagoulias/COGNET-MD/viewer/default/train?q=dermatology&views%5B%5D=train

Evaluation of Gemini 2.5 Flash (Google's family of models) — Which among the following options might prove beneficial for an old man suffering from a gravitational ulcer with adjacent eczema? A. The usage of a supportive elastic bandage B. Daily application of Betnovate-C ointment C. Povidine-iodine ointment on the ulcer D. Diuretic therapy for oedema reduction E. Lassar�s paste application around the ulcer
Condition/subject:  (English)
Category: Dermatology Examinations and Practice Questions — Professional Education
Validity/accuracy: 33.33% (sample size: 1)
Comments: the dataset suggests A-C-E as correct choices. All LLMs picked A as the only answer. Gemini's comments about C and E: C: While it can kill a broad range of microorganisms, its routine use on chronic ulcers is controversial. E:  While it might be beneficial for the surrounding eczematous skin by providing a barrier and soothing properties, it's generally applied around the ulcer and not directly into the wound itself. It can be a useful adjunct for managing the eczema.

Evaluation of Gemini-2.5-flash-preview-05-20 (Google's family of models) — Which among the following options might prove beneficial for an old man suffering from a gravitational ulcer with adjacent eczema? A. The usage of a supportive elastic bandage B. Daily application of Betnovate-C ointment C. Povidine-iodine ointment on the ulcer D. Diuretic therapy for oedema reduction E. Lassar�s paste application around the ulcer
Condition/subject:  (English)
Category: Dermatology Examinations and Practice Questions — Professional Education
Validity/accuracy: 33.33% (sample size: 1)
Comments: the dataset suggests A-C-E as correct choices. All LLMs picked A as the only answer. Gemini's (5/20/2025) comments about C and E: C: While it kills bacteria, it can be cytotoxic to healthy granulation tissue and may hinder the healing process of chronic wounds. Modern wound care generally avoids routine use of strong antiseptics on clean ulcers. E: Lassar's paste (zinc oxide paste) is a protective and mildly astringent paste often used for irritated skin or mild eczema. It could be helpful for the eczema around the ulcer by providing a barrier and soothing effect, but it does not treat the ulcer itself or the underlying cause of venous insufficiency.
Dataset: 542 Medical domain-specific questions (MCQs) including 144 dermatology questions
Reference (2025): Panagoulias, D.P. et al. (2025). COGnitive Network Evaluation Toolkit for Medical Domains: Evaluating Large Language Model Performance in Medical Diagnostics�An Assessment Framework and Dataset for Healthcare AI. In: Nagar, A., Jat, D.S., Mishra, D., Joshi, A. (eds) Intelligent Sustainable Systems. Worlds4 2024. Lecture Notes in Networks and Systems, vol 1178. Springer, Singapore. https://doi.org/10.1007/978-981-97-9559-8_38  https://huggingface.co/datasets/DimitriosPanagoulias/COGNET-MD/viewer/default/train?q=dermatology&views%5B%5D=train

Evaluation of gpt-4.1-mini-2025-04-14 (GPT series) — Which among the following options might prove beneficial for an old man suffering from a gravitational ulcer with adjacent eczema? A. The usage of a supportive elastic bandage B. Daily application of Betnovate-C ointment C. Povidine-iodine ointment on the ulcer D. Diuretic therapy for oedema reduction E. Lassar�s paste application around the ulcer
Condition/subject:  (English)
Category: Dermatology Examinations and Practice Questions — Professional Education
Validity/accuracy: 33.33% (sample size: 1)
Comments: the dataset suggests A-C-E as correct choices. All LLMs picked A as the only answer. ChatGPT 4.1's comments about C and E: C: Not the first choice for venous ulcers.  E:  Lassar�s paste (zinc oxide and starch) is used for mild eczema and irritant dermatitis but is not a primary treatment for venous ulcers or venous eczema.

Evaluation of llama-3.3-70b-instruct (LLaMA Series by Meta) — Which disease can insects be vectors for? A. Cholera B. AIDS C. Bubonic Plague D. Leishmaniasis E. Lyme Disease
Condition/subject: C-D-E (English)
Category: Dermatology Examinations and Practice Questions — Professional Education
Validity/accuracy: 100.00% (sample size: 1)
Comments: Note that ofifcially given correct answer is C,D and E even though ticks are arachnids, not insects.Model's answer: Insects can be vectors for several diseases. The correct options from the given list are:* C. Bubonic Plague (fleas are the primary vectors) * D. Leishmaniasis (sandflies are the vectors) * E. Lyme Disease (ticks are the vectors) Insects or arthropods are not typically vectors for: * A. Cholera (usually spread through contaminated water or food) * B. AIDS (spread through bodily fluids, primarily through sexual contact or sharing needles). So, the diseases that insects can be vectors for from the given options are Bubonic Plague, Leishmaniasis, and Lyme Disease.
Dataset: 542 Medical domain-specific questions (MCQs) including 144 dermatology questions
Reference (2025): Panagoulias, D.P. et al. (2025). COGnitive Network Evaluation Toolkit for Medical Domains: Evaluating Large Language Model Performance in Medical Diagnostics�An Assessment Framework and Dataset for Healthcare AI. In: Nagar, A., Jat, D.S., Mishra, D., Joshi, A. (eds) Intelligent Sustainable Systems. Worlds4 2024. Lecture Notes in Networks and Systems, vol 1178. Springer, Singapore. https://doi.org/10.1007/978-981-97-9559-8_38  https://huggingface.co/datasets/DimitriosPanagoulias/COGNET-MD/viewer/default/train?q=dermatology&views%5B%5D=train

Evaluation of llama-4-maverick-17b-128e-instruct (LLaMA Series by Meta) — Which disease can insects be vectors for? A. Cholera B. AIDS C. Bubonic Plague D. Leishmaniasis E. Lyme Disease
Condition/subject:  (English)
Category: Dermatology Examinations and Practice Questions — Professional Education
Validity/accuracy: 70.00% (sample size: 1)
Comments: Note that ofifcially given correct answer is C,D and E even though ticks are arachnids, not insects. Model's answer: Given that the question is looking for a disease that insects can be vectors for and since both Bubonic Plague C and Leishmaniasis D are valid examples, we need to consider if the question implies a single answer or if it allows for multiple interpretations. Since "D. Leishmaniasis" is a correct example and directly answers the question without ambiguity regarding the vector being an insect, it is a straightforward choice. The best answer is D.

Evaluation of ChatGPT o3 (GPT series) — Which disease can insects be vectors for? A. Cholera B. AIDS C. Bubonic Plague D. Leishmaniasis E. Lyme Disease
Condition/subject:  (English)
Category: Dermatology Examinations and Practice Questions — Professional Education
Validity/accuracy: 100.00% (sample size: 1)
Comments: Note that ofifcially given correct answer is C,D and E even though ticks are arachnids, not insects. Model's answer: Insects act as biological vectors for:* **Bubonic plague** � transmitted by the bites of infected rat fleas (*Xenopsylla cheopis* and relatives). * **Leishmaniasis** � transmitted by infected female sand flies (*Phlebotomus* or *Lutzomyia* spp.). The other choices are not spread by insects: * **Cholera** � mainly water/food borne. * **AIDS (HIV)** � spread through blood, sexual contact, etc., not by insects. * **Lyme disease** � transmitted by *Ixodes* ticks, which are arachnids, not insects. So the correct answers are **C. Bubonic Plague** and **D. Leishmaniasis**.

Evaluation of command-a-03-2025 (Cohere series) — Which disease can insects be vectors for? A. Cholera B. AIDS C. Bubonic Plague D. Leishmaniasis E. Lyme Disease
Condition/subject:  (English)
Category: Dermatology Examinations and Practice Questions — Professional Education
Validity/accuracy: 100.00% (sample size: 1)
Comments: The correct answers are C, D, and E.
Dataset: 542 Medical domain-specific questions (MCQs) including 144 dermatology questions
Reference (2025): Panagoulias, D.P. et al. (2025). COGnitive Network Evaluation Toolkit for Medical Domains: Evaluating Large Language Model Performance in Medical Diagnostics�An Assessment Framework and Dataset for Healthcare AI. In: Nagar, A., Jat, D.S., Mishra, D., Joshi, A. (eds) Intelligent Sustainable Systems. Worlds4 2024. Lecture Notes in Networks and Systems, vol 1178. Springer, Singapore. https://doi.org/10.1007/978-981-97-9559-8_38  https://huggingface.co/datasets/DimitriosPanagoulias/COGNET-MD/viewer/default/train?q=dermatology&views%5B%5D=train

Evaluation of ChatGPT 4o (GPT series) — Which of the following conditions is UVB an appropriate treatment for? A. acne vulgaris B. psoriasis C. atopic eczema D. pityriasis versicolor E. vitiligo
Condition/subject: A-B (English)
Category: Dermatology Examinations and Practice Questions — Professional Education
Validity/accuracy: 50.00% (sample size: 1)
Comments: B (A-B was defined as correct answer in the resource)

Evaluation of ChatGPT 4.1 (GPT series) — Which of the following conditions is UVB an appropriate treatment for? A. acne vulgaris B. psoriasis C. atopic eczema D. pityriasis versicolor E. vitiligo
Condition/subject:  (English)
Category: Dermatology Examinations and Practice Questions — Professional Education
Validity/accuracy: 50.00% (sample size: 1)
Comments: The correct answers are: B. Psoriasis C. Atopic eczema (also known as atopic dermatitis) E. Vitiligo.  Psoriasis is effectively managed with UVB phototherapy, particularly narrowband UVB, due to its ability to reduce inflammation and slow down skin cell turnover.  Atopic eczema (atopic dermatitis) also responds well to UVB treatment because it reduces skin inflammation and itchiness. Vitiligo responds to narrowband UVB, which stimulates melanocyte repigmentation in affected skin areas.  Conditions like acne vulgaris and pityriasis versicolor typically do not benefit significantly from UVB phototherapy and have alternative preferred treatments.

Evaluation of gemini-2.0-flash-001 (Google's family of models) — Which of the following conditions is UVB an appropriate treatment for? A. acne vulgaris B. psoriasis C. atopic eczema D. pityriasis versicolor E. vitiligo
Condition/subject:  (English)
Category: Dermatology Examinations and Practice Questions — Professional Education
Validity/accuracy: 50.00% (sample size: 1)
Comments: B (B and E if asked for second best answer)

Evaluation of command-a-03-2025 (Cohere series) — Which of the following conditions is UVB an appropriate treatment for? A. acne vulgaris B. psoriasis C. atopic eczema D. pityriasis versicolor E. vitiligo
Condition/subject:  (English)
Category: Dermatology Examinations and Practice Questions — Professional Education
Validity/accuracy: 50.00% (sample size: 1)
Comments: B, C, and E. While some types of light therapy are used for acne, UVB is not typically the first-line treatment.
Dataset: 542 Medical domain-specific questions (MCQs) including 144 dermatology questions
Reference (2025): Panagoulias, D.P. et al. (2025). COGnitive Network Evaluation Toolkit for Medical Domains: Evaluating Large Language Model Performance in Medical Diagnostics�An Assessment Framework and Dataset for Healthcare AI. In: Nagar, A., Jat, D.S., Mishra, D., Joshi, A. (eds) Intelligent Sustainable Systems. Worlds4 2024. Lecture Notes in Networks and Systems, vol 1178. Springer, Singapore. https://doi.org/10.1007/978-981-97-9559-8_38  https://huggingface.co/datasets/DimitriosPanagoulias/COGNET-MD/viewer/default/train?q=dermatology&views%5B%5D=train

Evaluation of ChatGPT 4o (GPT series) — Which of the following conditions is UVB an appropriate treatment for? A. acne vulgaris B. psoriasis C. atopic eczema D. pityriasis versicolor E. vitiligo
Condition/subject:  (English)
Category: Dermatology Examinations and Practice Questions — Professional Education
Validity/accuracy: 50.00% (sample size: 1)
Comments: B, C, and E. While some light - based therapies (e.g., blue light) may be used in specific cases, UVB is not a primary or well - established option for acne.

Evaluation of ChatGPT 4.5 (GPT series) — Which of the following conditions is UVB an appropriate treatment for? A. acne vulgaris B. psoriasis C. atopic eczema D. pityriasis versicolor E. vitiligo
Condition/subject:  (English)
Category: Dermatology Examinations and Practice Questions — Professional Education
Validity/accuracy: 50.00% (sample size: 1)
Comments: B, C, and E. While some light - based therapies (e.g., blue light) may be used in specific cases, UVB is not a primary or well - established option for acne.

Evaluation of ChatGPT o4-mini (GPT series) — Which of the following conditions is UVB an appropriate treatment for? A. acne vulgaris B. psoriasis C. atopic eczema D. pityriasis versicolor E. vitiligo
Condition/subject:  (English)
Category: Dermatology Examinations and Practice Questions — Professional Education
Validity/accuracy: 50.00% (sample size: 1)
Comments: UVB phototherapy is indicated in a number of inflammatory and pigmentary dermatoses. Of the options listed, it�s appropriate for: B. Psoriasis � NB-UVB is a first-line phototherapy for chronic plaque psoriasis. C. Atopic eczema � Narrowband UVB can be used for moderate-to-severe atopic dermatitis. E. Vitiligo � NB-UVB is the treatment of choice for widespread vitiligo. It is not used for acne vulgaris (A), which relies on topical/systemic retinoids, antibiotics, etc.; nor for pityriasis versicolor (D), which is managed with antifungals.

Evaluation of gemini-2.5-flash (Google's family of models) — Which of the following conditions is UVB an appropriate treatment for? A. acne vulgaris B. psoriasis C. atopic eczema D. pityriasis versicolor E. vitiligo
Condition/subject:  (English)
Category: Dermatology Examinations and Practice Questions — Professional Education
Validity/accuracy: 50.00% (sample size: 1)
Comments: The most appropriate responses where UVB is a well-established and common treatment are B. Psoriasis, C. Atopic Eczema, and E. Vitiligo