Aurametrix/dermatology_longitudinal huggingface dataset:
Dataset: 100 histopathology?validated dermoscopic images
Reference (2024): Wang J, Yu TC, Kolodney MS, Perrotta PL, Hu G. Adapting ChatGPT for Color Blindness in Medical Education. Annals of Biomedical Engineering. 2024 Nov 27:1-4.  https://link.springer.com/article/10.1007/s10439-024-03656-0, doi: 10.1007/s10439-024-03656-0

Evaluation of GPT?4V (GPT series) — Original (non?simulated) color, mean accuracy: Image-based diagnostics; dermoscopic images including Melanoma (skin cancer) and Benign Nevus (Moles)
Condition/subject: Images, Diagnostics, Melanoma, Benign Moles (English)
Validity/accuracy: 77.70% (sample size: 100)
Comments: GPT?4V can still reliably detect melanoma even if reds, greens, or blues look different. Despite altered color perception, ChatGPT?4V maintained comparable diagnostic accuracy across color vision deficiency (CVD) simulations. ChatGPT?4V can �see� images as someone with a given color vision deficiency might, retaining strong diagnostic performance and providing explanations that reflect the altered color perception. This points to the potential use of GPT?4V as an assistive tool in visually demanding medical fields�especially for learners and practitioners with CVD. Conditions tested were Original (non?simulated) color; Protanopia?simulated (red?deficient); Deuteranopia?simulated (green?deficient); Tritanopia?simulated (blue?deficient). By converting both the �query� images and the �reference� images to the same simulated color space, GPT?4V can maintain consistent color cues and classify lesions more accurately.

Evaluation of GPT?4V (GPT series) — Protanopia?simulated (red?deficient), mean accuracy: Image-based diagnostics; dermoscopic images including Melanoma (skin cancer) and Benign Nevus (Moles)
Condition/subject: Images, Diagnostics, Melanoma, Benign Moles (English)
Validity/accuracy: 76.00% (sample size: 100)
Comments: GPT?4V can still reliably detect melanoma even if reds, greens, or blues look different. Despite altered color perception, ChatGPT?4V maintained comparable diagnostic accuracy across color vision deficiency (CVD) simulations. ChatGPT?4V can �see� images as someone with a given color vision deficiency might, retaining strong diagnostic performance and providing explanations that reflect the altered color perception. This points to the potential use of GPT?4V as an assistive tool in visually demanding medical fields�especially for learners and practitioners with CVD. Conditions tested were Original (non?simulated) color; Protanopia?simulated (red?deficient); Deuteranopia?simulated (green?deficient); Tritanopia?simulated (blue?deficient). By converting both the �query� images and the �reference� images to the same simulated color space, GPT?4V can maintain consistent color cues and classify lesions more accurately.
Dataset: 100 histopathology?validated dermoscopic images
Reference (2024): Wang J, Yu TC, Kolodney MS, Perrotta PL, Hu G. Adapting ChatGPT for Color Blindness in Medical Education. Annals of Biomedical Engineering. 2024 Nov 27:1-4.  https://link.springer.com/article/10.1007/s10439-024-03656-0, doi: 10.1007/s10439-024-03656-0

Evaluation of GPT?4V (GPT series) — Deuteranopia?simulated (green?deficient), mean accuracy: Image-based diagnostics; dermoscopic images including Melanoma (skin cancer) and Benign Nevus (Moles)
Condition/subject: Images, Diagnostics, Melanoma, Benign Moles (English)
Validity/accuracy: 73.30% (sample size: 100)
Comments: GPT?4V can still reliably detect melanoma even if reds, greens, or blues look different. Despite altered color perception, ChatGPT?4V maintained comparable diagnostic accuracy across color vision deficiency (CVD) simulations. ChatGPT?4V can �see� images as someone with a given color vision deficiency might, retaining strong diagnostic performance and providing explanations that reflect the altered color perception. This points to the potential use of GPT?4V as an assistive tool in visually demanding medical fields�especially for learners and practitioners with CVD. Conditions tested were Original (non?simulated) color; Protanopia?simulated (red?deficient); Deuteranopia?simulated (green?deficient); Tritanopia?simulated (blue?deficient). By converting both the �query� images and the �reference� images to the same simulated color space, GPT?4V can maintain consistent color cues and classify lesions more accurately.

Evaluation of GPT?4V (GPT series) — Tritanopia?simulated (blue?deficient), mean accuracy: Image-based diagnostics; dermoscopic images including Melanoma (skin cancer) and Benign Nevus (Moles)
Condition/subject: Images, Diagnostics, Melanoma, Benign Moles (English)
Validity/accuracy: 72.10% (sample size: 100)
Comments: GPT?4V can still reliably detect melanoma even if reds, greens, or blues look different. Despite altered color perception, ChatGPT?4V maintained comparable diagnostic accuracy across color vision deficiency (CVD) simulations. ChatGPT?4V can �see� images as someone with a given color vision deficiency might, retaining strong diagnostic performance and providing explanations that reflect the altered color perception. This points to the potential use of GPT?4V as an assistive tool in visually demanding medical fields�especially for learners and practitioners with CVD. Conditions tested were Original (non?simulated) color; Protanopia?simulated (red?deficient); Deuteranopia?simulated (green?deficient); Tritanopia?simulated (blue?deficient). By converting both the �query� images and the �reference� images to the same simulated color space, GPT?4V can maintain consistent color cues and classify lesions more accurately.
Dataset: 100 histopathology?validated dermoscopic images
Reference (2024): Wang J, Yu TC, Kolodney MS, Perrotta PL, Hu G. Adapting ChatGPT for Color Blindness in Medical Education. Annals of Biomedical Engineering. 2024 Nov 27:1-4.  https://link.springer.com/article/10.1007/s10439-024-03656-0, doi: 10.1007/s10439-024-03656-0

Evaluation of GPT?4V (GPT series) — Original (non?simulated) color, accuracy after voting across ten repeats: Image-based diagnostics; dermoscopic images including Melanoma (skin cancer) and Benign Nevus (Moles)
Condition/subject: Images, Diagnostics, Melanoma, Benign Moles (English)
Validity/accuracy: 83.80% (sample size: 100)
Comments: Using a consensus strategy (voting across ten repeats), accuracy increased further. GPT?4V can still reliably detect melanoma even if reds, greens, or blues look different. Despite altered color perception, ChatGPT?4V maintained comparable diagnostic accuracy across color vision deficiency (CVD) simulations. ChatGPT?4V can �see� images as someone with a given color vision deficiency might, retaining strong diagnostic performance and providing explanations that reflect the altered color perception. This points to the potential use of GPT?4V as an assistive tool in visually demanding medical fields�especially for learners and practitioners with CVD. Conditions tested were Original (non?simulated) color; Protanopia?simulated (red?deficient); Deuteranopia?simulated (green?deficient); Tritanopia?simulated (blue?deficient). By converting both the �query� images and the �reference� images to the same simulated color space, GPT?4V can maintain consistent color cues and classify lesions more accurately.

Evaluation of GPT?4V (GPT series) — Protanopia?simulated (red?deficient), accuracy after voting across ten repeats: Image-based diagnostics; dermoscopic images including Melanoma (skin cancer) and Benign Nevus (Moles)
Condition/subject: Images, Diagnostics, Melanoma, Benign Moles (English)
Validity/accuracy: 79.80% (sample size: 100)
Comments: Using a consensus strategy (voting across ten repeats), accuracy increased further. GPT?4V can still reliably detect melanoma even if reds, greens, or blues look different. Despite altered color perception, ChatGPT?4V maintained comparable diagnostic accuracy across color vision deficiency (CVD) simulations. ChatGPT?4V can �see� images as someone with a given color vision deficiency might, retaining strong diagnostic performance and providing explanations that reflect the altered color perception. This points to the potential use of GPT?4V as an assistive tool in visually demanding medical fields�especially for learners and practitioners with CVD. Conditions tested were Original (non?simulated) color; Protanopia?simulated (red?deficient); Deuteranopia?simulated (green?deficient); Tritanopia?simulated (blue?deficient). By converting both the �query� images and the �reference� images to the same simulated color space, GPT?4V can maintain consistent color cues and classify lesions more accurately.
Dataset: 100 histopathology?validated dermoscopic images
Reference (2024): Wang J, Yu TC, Kolodney MS, Perrotta PL, Hu G. Adapting ChatGPT for Color Blindness in Medical Education. Annals of Biomedical Engineering. 2024 Nov 27:1-4.  https://link.springer.com/article/10.1007/s10439-024-03656-0, doi: 10.1007/s10439-024-03656-0

Evaluation of GPT?4V (GPT series) — Deuteranopia?simulated (green?deficient), accuracy after voting across ten repeats: Image-based diagnostics; dermoscopic images including Melanoma (skin cancer) and Benign Nevus (Moles)
Condition/subject: Images, Diagnostics, Melanoma, Benign Moles (English)
Validity/accuracy: 76.80% (sample size: 100)
Comments: Using a consensus strategy (voting across ten repeats), accuracy increased further. GPT?4V can still reliably detect melanoma even if reds, greens, or blues look different. Despite altered color perception, ChatGPT?4V maintained comparable diagnostic accuracy across color vision deficiency (CVD) simulations. ChatGPT?4V can �see� images as someone with a given color vision deficiency might, retaining strong diagnostic performance and providing explanations that reflect the altered color perception. This points to the potential use of GPT?4V as an assistive tool in visually demanding medical fields�especially for learners and practitioners with CVD. Conditions tested were Original (non?simulated) color; Protanopia?simulated (red?deficient); Deuteranopia?simulated (green?deficient); Tritanopia?simulated (blue?deficient). By converting both the �query� images and the �reference� images to the same simulated color space, GPT?4V can maintain consistent color cues and classify lesions more accurately.

Evaluation of GPT?4V (GPT series) — Tritanopia?simulated (blue?deficient), accuracy after voting across ten repeats: Image-based diagnostics; dermoscopic images including Melanoma (skin cancer) and Benign Nevus (Moles)
Condition/subject: Images, Diagnostics, Melanoma, Benign Moles (English)
Validity/accuracy: 80.80% (sample size: 100)
Comments: Using a consensus strategy (voting across ten repeats), accuracy increased further. GPT?4V can still reliably detect melanoma even if reds, greens, or blues look different. Despite altered color perception, ChatGPT?4V maintained comparable diagnostic accuracy across color vision deficiency (CVD) simulations. ChatGPT?4V can �see� images as someone with a given color vision deficiency might, retaining strong diagnostic performance and providing explanations that reflect the altered color perception. This points to the potential use of GPT?4V as an assistive tool in visually demanding medical fields�especially for learners and practitioners with CVD. Conditions tested were Original (non?simulated) color; Protanopia?simulated (red?deficient); Deuteranopia?simulated (green?deficient); Tritanopia?simulated (blue?deficient). By converting both the �query� images and the �reference� images to the same simulated color space, GPT?4V can maintain consistent color cues and classify lesions more accurately.
Dataset: 10 questions used to generate Patient education materials (PEMs)
Reference (2024): Duran S, Gonzalez AM, Nguyen K, Nguyen J, Zinn Z. Enhancing Spanish Patient Education Materials: Comparing the Readability of Artificial Intelligence-Generated Spanish Patient Education Materials to the Society of Pediatric Dermatology Spanish Patient Brochures. Pediatr Dermatol. 2024 Nov 12. doi: 10.1111/pde.15805. Epub ahead of print. PMID: 39533849.

Evaluation of GPT-4 (GPT series) — Fern�ndez Huerta Index: Readability level of Patient education materials
Condition/subject: Patient education, Cosmetic dermatology, keloids, scar management and wound healing (Spanish)
Validity/accuracy: 93.00% (sample size: 10)
Comments: Readability of Spanish PEMs from the Society of Pediatric Dermatology (SPD) was compared with those generated by Open-AI's ChatGPT 4.0 and Google Gemini. AI-generated PEMs could better meet readability standards and potentially improve patient outcomes.

Evaluation of Gemini 1.0 (Google's Family of LLMs) — Fern�ndez Huerta Index: Readability level of Patient education materials
Condition/subject: Patient education, Cosmetic dermatology, keloids, scar management and wound healing (Spanish)
Validity/accuracy: 90.00% (sample size: 10)
Comments: Readability of Spanish PEMs from the Society of Pediatric Dermatology (SPD) was compared with those generated by Open-AI's ChatGPT 4.0 and Google Gemini. AI-generated PEMs could better meet readability standards and potentially improve patient outcomes.

Evaluation of GPT-4 (GPT series) — Inflesz Scale: Readability level of Patient education materials
Condition/subject: Patient education, Cosmetic dermatology, keloids, scar management and wound healing (Spanish)
Validity/accuracy: 93.00% (sample size: 10)
Comments: Readability of Spanish PEMs from the Society of Pediatric Dermatology (SPD) was compared with those generated by Open-AI's ChatGPT 4.0 and Google Gemini. AI-generated PEMs could better meet readability standards and potentially improve patient outcomes.

Evaluation of Gemini 1.0 (Google's Family of LLMs) — Inflesz Scale: Readability level of Patient education materials
Condition/subject: Patient education, Cosmetic dermatology, keloids, scar management and wound healing (Spanish)
Validity/accuracy: 90.00% (sample size: 10)
Comments: Readability of Spanish PEMs from the Society of Pediatric Dermatology (SPD) was compared with those generated by Open-AI's ChatGPT 4.0 and Google Gemini. AI-generated PEMs could better meet readability standards and potentially improve patient outcomes.
Dataset: ChatGPT-4 generated 402 questions, with 208 (51.7%) deemed acceptable by at least 1 reviewer. However, only 72 questions (18%) were accepted by both reviewers. After consensus discussions, 53 of the 136 initially disputed questions were approved, resulting in a total of 125 questions deemed suitable for the exam. The suitable questions were classified as 51 (40.8%) easy, 45 (36%) medium-difficulty, and 29 (23.2%) hard. The main issues with unsuitable questions included questions that contained errors or improperly structured or with potential for an appeal (118 questions, 27.8%) and excessive simplicity (113 questions, 28.1%). By subject area: Biopsy techniques and B-cell lymphoma had the highest rates of suitable questions (63�65%). In addition, 37 questions were 2-stage complicated questions. Of those 7 were determined as appropriate (18.9%). 
Reference (2025): Shapiro J, Lyakhovitsky A, Freud T, Pavlotsky F, Khamaysi Z, Valdman-Grinshpoun Y, Dodiuk-Gad R, Goldberg I, Ingber A, Kaplan B, Avitan-Hersh E. Assessing ChatGPT-4's Capabilities in Generating Dermatology Board Examination Content: An Explorational Study. Acta Derm Venereol. 2025 Jan 3;105:adv41208. doi: 10.2340/actadv.v105.41208. PMID: 39749389; PMCID: PMC11697136.

Evaluation of ChatGPT-4 (GPT series) — Efficiency of AI in Dermatology Board Exam question generation
Condition/subject: Professional education (English)
Category: Dermatology Examinations and Practice Questions — Professional Education
Validity/accuracy: 31.10% (sample size: 402)
Comments: Validity Score  is based  on the Suitablity measure. It will be slightly lower if the Overall Validity Score could be also computed as the unweighted average of five HealthBench-style dimensions for each subject. Completness can be assessed as the �suitability� reflecting whether the answer was seen as fully addressing the topic (percentage of suitable questions). Accuracy combines the two reviewers� independent judgments of answer correctness: (R1 % + R2 %) / 2; Context awareness = 100 � Dispute % High disagreement implies the model lacks clarity/context ? deductive inverse. Context awareness could be also derived based on rejection reasons like "too easy," "too hard," or "errored."; Communication quality = 100 � Dispute % Again, high dispute suggests unclear or ambiguous answers; Instruction following = (Suitable % + R2 %) / 2 Reflects a blend of general suitability and alignment with a reviewer�s expectations.
Dataset: ChatGPT-4 generated 402 questions, with 208 (51.7%) deemed acceptable by at least 1 reviewer. However, only 72 questions (18%) were accepted by both reviewers. After consensus discussions, 53 of the 136 initially disputed questions were approved, resulting in a total of 125 questions deemed suitable for the exam. The suitable questions were classified as 51 (40.8%) easy, 45 (36%) medium-difficulty, and 29 (23.2%) hard. The main issues with unsuitable questions included questions that contained errors or improperly structured or with potential for an appeal (118 questions, 27.8%) and excessive simplicity (113 questions, 28.1%). By subject area: Biopsy techniques and B-cell lymphoma had the highest rates of suitable questions (63�65%). In addition, 37 questions were 2-stage complicated questions. Of those 7 were determined as appropriate (18.9%). 
Reference (2025): Shapiro J, Lyakhovitsky A, Freud T, Pavlotsky F, Khamaysi Z, Valdman-Grinshpoun Y, Dodiuk-Gad R, Goldberg I, Ingber A, Kaplan B, Avitan-Hersh E. Assessing ChatGPT-4's Capabilities in Generating Dermatology Board Examination Content: An Explorational Study. Acta Derm Venereol. 2025 Jan 3;105:adv41208. doi: 10.2340/actadv.v105.41208. PMID: 39749389; PMCID: PMC11697136.

Evaluation of ChatGPT-4 (GPT series) — Biopsy techniques in dermatology
Condition/subject: Professional education (English)
Category: Dermatology Examinations and Practice Questions — Professional Education
Validity/accuracy: 65.00% (sample size: 20)
Comments: Based on suitability and rejection reasons, tends to be "too easy" or "Errored" (lower scores for complexity). Validity Score  is based  on the Suitablity measure. It will be slightly lower if computed as the unweighted average of five HealthBench-style dimensions for each subject or average of Accuracy,�Context awareness, and�Instruction following, most appropriate for assessment.. Completness can be assessed as the �suitability� reflecting whether the answer was seen as fully addressing the topic (percentage of suitable questions). Accuracy combines the two reviewers� independent judgments of answer correctness: (R1 % + R2 %) / 2; Context awareness = 100 � Dispute % High disagreement implies the model lacks clarity/context ? deductive inverse. Context awareness could be also derived based on rejection reasons like "too easy," "too hard," or "errored."; Communication quality = 100 � Dispute % Again, high dispute suggests unclear or ambiguous answers; Instruction following = (Suitable % + R2 %) / 2 Reflects a blend of general suitability and alignment with a reviewer�s expectations.
Dataset: ChatGPT-4 generated 402 questions, with 208 (51.7%) deemed acceptable by at least 1 reviewer. However, only 72 questions (18%) were accepted by both reviewers. After consensus discussions, 53 of the 136 initially disputed questions were approved, resulting in a total of 125 questions deemed suitable for the exam. The suitable questions were classified as 51 (40.8%) easy, 45 (36%) medium-difficulty, and 29 (23.2%) hard. The main issues with unsuitable questions included questions that contained errors or improperly structured or with potential for an appeal (118 questions, 27.8%) and excessive simplicity (113 questions, 28.1%). By subject area: Biopsy techniques and B-cell lymphoma had the highest rates of suitable questions (63�65%). In addition, 37 questions were 2-stage complicated questions. Of those 7 were determined as appropriate (18.9%). 
Reference (2025): Shapiro J, Lyakhovitsky A, Freud T, Pavlotsky F, Khamaysi Z, Valdman-Grinshpoun Y, Dodiuk-Gad R, Goldberg I, Ingber A, Kaplan B, Avitan-Hersh E. Assessing ChatGPT-4's Capabilities in Generating Dermatology Board Examination Content: An Explorational Study. Acta Derm Venereol. 2025 Jan 3;105:adv41208. doi: 10.2340/actadv.v105.41208. PMID: 39749389; PMCID: PMC11697136.

Evaluation of ChatGPT-4 (GPT series) — CBCL
Condition/subject: Professional education (English)
Category: Dermatology Examinations and Practice Questions — Professional Education
Validity/accuracy: 63.30% (sample size: 30)
Comments: Dispute reasons and suitability indicate moderate scores. Validity Score  is based  on the Suitablity measure. It will be slightly lower if computed as the unweighted average of five HealthBench-style dimensions for each subject or average of Accuracy,�Context awareness, and�Instruction following, most appropriate for assessment.. Completness can be assessed as the �suitability� reflecting whether the answer was seen as fully addressing the topic (percentage of suitable questions). Accuracy combines the two reviewers� independent judgments of answer correctness: (R1 % + R2 %) / 2; Context awareness = 100 � Dispute % High disagreement implies the model lacks clarity/context ? deductive inverse. Context awareness could be also derived based on rejection reasons like "too easy," "too hard," or "errored."; Communication quality = 100 � Dispute % Again, high dispute suggests unclear or ambiguous answers; Instruction following = (Suitable % + R2 %) / 2 Reflects a blend of general suitability and alignment with a reviewer�s expectations.
Dataset: ChatGPT-4 generated 402 questions, with 208 (51.7%) deemed acceptable by at least 1 reviewer. However, only 72 questions (18%) were accepted by both reviewers. After consensus discussions, 53 of the 136 initially disputed questions were approved, resulting in a total of 125 questions deemed suitable for the exam. The suitable questions were classified as 51 (40.8%) easy, 45 (36%) medium-difficulty, and 29 (23.2%) hard. The main issues with unsuitable questions included questions that contained errors or improperly structured or with potential for an appeal (118 questions, 27.8%) and excessive simplicity (113 questions, 28.1%). By subject area: Biopsy techniques and B-cell lymphoma had the highest rates of suitable questions (63�65%). In addition, 37 questions were 2-stage complicated questions. Of those 7 were determined as appropriate (18.9%). 
Reference (2025): Shapiro J, Lyakhovitsky A, Freud T, Pavlotsky F, Khamaysi Z, Valdman-Grinshpoun Y, Dodiuk-Gad R, Goldberg I, Ingber A, Kaplan B, Avitan-Hersh E. Assessing ChatGPT-4's Capabilities in Generating Dermatology Board Examination Content: An Explorational Study. Acta Derm Venereol. 2025 Jan 3;105:adv41208. doi: 10.2340/actadv.v105.41208. PMID: 39749389; PMCID: PMC11697136.

Evaluation of ChatGPT-4 (GPT series) — CTCL
Condition/subject: Professional education (English)
Category: Dermatology Examinations and Practice Questions — Professional Education
Validity/accuracy: 53.30% (sample size: 30)
Comments: Similar to CBCL, moderate scores. Validity Score  is based  on the Suitablity measure. It will be slightly lower if computed as the unweighted average of five HealthBench-style dimensions for each subject or average of Accuracy,�Context awareness, and�Instruction following, most appropriate for assessment.. Completness can be assessed as the �suitability� reflecting whether the answer was seen as fully addressing the topic (percentage of suitable questions). Accuracy combines the two reviewers� independent judgments of answer correctness: (R1 % + R2 %) / 2; Context awareness = 100 � Dispute % High disagreement implies the model lacks clarity/context ? deductive inverse. Context awareness could be also derived based on rejection reasons like "too easy," "too hard," or "errored."; Communication quality = 100 � Dispute % Again, high dispute suggests unclear or ambiguous answers; Instruction following = (Suitable % + R2 %) / 2 Reflects a blend of general suitability and alignment with a reviewer�s expectations.
Dataset: ChatGPT-4 generated 402 questions, with 208 (51.7%) deemed acceptable by at least 1 reviewer. However, only 72 questions (18%) were accepted by both reviewers. After consensus discussions, 53 of the 136 initially disputed questions were approved, resulting in a total of 125 questions deemed suitable for the exam. The suitable questions were classified as 51 (40.8%) easy, 45 (36%) medium-difficulty, and 29 (23.2%) hard. The main issues with unsuitable questions included questions that contained errors or improperly structured or with potential for an appeal (118 questions, 27.8%) and excessive simplicity (113 questions, 28.1%). By subject area: Biopsy techniques and B-cell lymphoma had the highest rates of suitable questions (63�65%). In addition, 37 questions were 2-stage complicated questions. Of those 7 were determined as appropriate (18.9%). 
Reference (2025): Shapiro J, Lyakhovitsky A, Freud T, Pavlotsky F, Khamaysi Z, Valdman-Grinshpoun Y, Dodiuk-Gad R, Goldberg I, Ingber A, Kaplan B, Avitan-Hersh E. Assessing ChatGPT-4's Capabilities in Generating Dermatology Board Examination Content: An Explorational Study. Acta Derm Venereol. 2025 Jan 3;105:adv41208. doi: 10.2340/actadv.v105.41208. PMID: 39749389; PMCID: PMC11697136.

Evaluation of ChatGPT-4 (GPT series) — HPV
Condition/subject: Professional education (English)
Category: Dermatology Examinations and Practice Questions — Professional Education
Validity/accuracy: 40.60% (sample size: 32)
Comments: Slightly lower scores, "Too hard" questions tend to lower accuracy & completeness. Validity Score  is based  on the Suitablity measure. It will be slightly lower if computed as the unweighted average of five HealthBench-style dimensions for each subject or average of Accuracy,�Context awareness, and�Instruction following, most appropriate for assessment.. Completness can be assessed as the �suitability� reflecting whether the answer was seen as fully addressing the topic (percentage of suitable questions). Accuracy combines the two reviewers� independent judgments of answer correctness: (R1 % + R2 %) / 2; Context awareness = 100 � Dispute % High disagreement implies the model lacks clarity/context ? deductive inverse. Context awareness could be also derived based on rejection reasons like "too easy," "too hard," or "errored."; Communication quality = 100 � Dispute % Again, high dispute suggests unclear or ambiguous answers; Instruction following = (Suitable % + R2 %) / 2 Reflects a blend of general suitability and alignment with a reviewer�s expectations.
Dataset: ChatGPT-4 generated 402 questions, with 208 (51.7%) deemed acceptable by at least 1 reviewer. However, only 72 questions (18%) were accepted by both reviewers. After consensus discussions, 53 of the 136 initially disputed questions were approved, resulting in a total of 125 questions deemed suitable for the exam. The suitable questions were classified as 51 (40.8%) easy, 45 (36%) medium-difficulty, and 29 (23.2%) hard. The main issues with unsuitable questions included questions that contained errors or improperly structured or with potential for an appeal (118 questions, 27.8%) and excessive simplicity (113 questions, 28.1%). By subject area: Biopsy techniques and B-cell lymphoma had the highest rates of suitable questions (63�65%). In addition, 37 questions were 2-stage complicated questions. Of those 7 were determined as appropriate (18.9%). 
Reference (2025): Shapiro J, Lyakhovitsky A, Freud T, Pavlotsky F, Khamaysi Z, Valdman-Grinshpoun Y, Dodiuk-Gad R, Goldberg I, Ingber A, Kaplan B, Avitan-Hersh E. Assessing ChatGPT-4's Capabilities in Generating Dermatology Board Examination Content: An Explorational Study. Acta Derm Venereol. 2025 Jan 3;105:adv41208. doi: 10.2340/actadv.v105.41208. PMID: 39749389; PMCID: PMC11697136.

Evaluation of ChatGPT-4 (GPT series) — Alopecia
Condition/subject: Professional education (English)
Category: Dermatology Examinations and Practice Questions — Professional Education
Validity/accuracy: 40.00% (sample size: 40)
Comments: Mixed, with lower context awareness. Validity Score  is based  on the Suitablity measure. It will be slightly lower if computed as the unweighted average of five HealthBench-style dimensions for each subject or average of Accuracy,�Context awareness, and�Instruction following, most appropriate for assessment.. Completness can be assessed as the �suitability� reflecting whether the answer was seen as fully addressing the topic (percentage of suitable questions). Accuracy combines the two reviewers� independent judgments of answer correctness: (R1 % + R2 %) / 2; Context awareness = 100 � Dispute % High disagreement implies the model lacks clarity/context ? deductive inverse. Context awareness could be also derived based on rejection reasons like "too easy," "too hard," or "errored."; Communication quality = 100 � Dispute % Again, high dispute suggests unclear or ambiguous answers; Instruction following = (Suitable % + R2 %) / 2 Reflects a blend of general suitability and alignment with a reviewer�s expectations.
Dataset: ChatGPT-4 generated 402 questions, with 208 (51.7%) deemed acceptable by at least 1 reviewer. However, only 72 questions (18%) were accepted by both reviewers. After consensus discussions, 53 of the 136 initially disputed questions were approved, resulting in a total of 125 questions deemed suitable for the exam. The suitable questions were classified as 51 (40.8%) easy, 45 (36%) medium-difficulty, and 29 (23.2%) hard. The main issues with unsuitable questions included questions that contained errors or improperly structured or with potential for an appeal (118 questions, 27.8%) and excessive simplicity (113 questions, 28.1%). By subject area: Biopsy techniques and B-cell lymphoma had the highest rates of suitable questions (63�65%). In addition, 37 questions were 2-stage complicated questions. Of those 7 were determined as appropriate (18.9%). 
Reference (2025): Shapiro J, Lyakhovitsky A, Freud T, Pavlotsky F, Khamaysi Z, Valdman-Grinshpoun Y, Dodiuk-Gad R, Goldberg I, Ingber A, Kaplan B, Avitan-Hersh E. Assessing ChatGPT-4's Capabilities in Generating Dermatology Board Examination Content: An Explorational Study. Acta Derm Venereol. 2025 Jan 3;105:adv41208. doi: 10.2340/actadv.v105.41208. PMID: 39749389; PMCID: PMC11697136.

Evaluation of ChatGPT-4 (GPT series) — Systemic disease
Condition/subject: Professional education (English)
Category: Dermatology Examinations and Practice Questions — Professional Education
Validity/accuracy: 33.30% (sample size: 30)
Comments: Slightly lower in completeness & accuracy. Validity Score  is based  on the Suitablity measure. It will be slightly lower if computed as the unweighted average of five HealthBench-style dimensions for each subject or average of Accuracy,�Context awareness, and�Instruction following, most appropriate for assessment.. Completness can be assessed as the �suitability� reflecting whether the answer was seen as fully addressing the topic (percentage of suitable questions). Accuracy combines the two reviewers� independent judgments of answer correctness: (R1 % + R2 %) / 2; Context awareness = 100 � Dispute % High disagreement implies the model lacks clarity/context ? deductive inverse. Context awareness could be also derived based on rejection reasons like "too easy," "too hard," or "errored."; Communication quality = 100 � Dispute % Again, high dispute suggests unclear or ambiguous answers; Instruction following = (Suitable % + R2 %) / 2 Reflects a blend of general suitability and alignment with a reviewer�s expectations.
Dataset: ChatGPT-4 generated 402 questions, with 208 (51.7%) deemed acceptable by at least 1 reviewer. However, only 72 questions (18%) were accepted by both reviewers. After consensus discussions, 53 of the 136 initially disputed questions were approved, resulting in a total of 125 questions deemed suitable for the exam. The suitable questions were classified as 51 (40.8%) easy, 45 (36%) medium-difficulty, and 29 (23.2%) hard. The main issues with unsuitable questions included questions that contained errors or improperly structured or with potential for an appeal (118 questions, 27.8%) and excessive simplicity (113 questions, 28.1%). By subject area: Biopsy techniques and B-cell lymphoma had the highest rates of suitable questions (63�65%). In addition, 37 questions were 2-stage complicated questions. Of those 7 were determined as appropriate (18.9%). 
Reference (2025): Shapiro J, Lyakhovitsky A, Freud T, Pavlotsky F, Khamaysi Z, Valdman-Grinshpoun Y, Dodiuk-Gad R, Goldberg I, Ingber A, Kaplan B, Avitan-Hersh E. Assessing ChatGPT-4's Capabilities in Generating Dermatology Board Examination Content: An Explorational Study. Acta Derm Venereol. 2025 Jan 3;105:adv41208. doi: 10.2340/actadv.v105.41208. PMID: 39749389; PMCID: PMC11697136.

Evaluation of ChatGPT-4 (GPT series) — Mycobacteria
Condition/subject: Professional education (English)
Category: Dermatology Examinations and Practice Questions — Professional Education
Validity/accuracy: 26.70% (sample size: 30)
Comments: Lower in completeness and communication. Validity Score  is based  on the Suitablity measure. It will be slightly lower if computed as the unweighted average of five HealthBench-style dimensions for each subject or average of Accuracy,�Context awareness, and�Instruction following, most appropriate for assessment.. Completness can be assessed as the �suitability� reflecting whether the answer was seen as fully addressing the topic (percentage of suitable questions). Accuracy combines the two reviewers� independent judgments of answer correctness: (R1 % + R2 %) / 2; Context awareness = 100 � Dispute % High disagreement implies the model lacks clarity/context ? deductive inverse. Context awareness could be also derived based on rejection reasons like "too easy," "too hard," or "errored."; Communication quality = 100 � Dispute % Again, high dispute suggests unclear or ambiguous answers; Instruction following = (Suitable % + R2 %) / 2 Reflects a blend of general suitability and alignment with a reviewer�s expectations.
Dataset: ChatGPT-4 generated 402 questions, with 208 (51.7%) deemed acceptable by at least 1 reviewer. However, only 72 questions (18%) were accepted by both reviewers. After consensus discussions, 53 of the 136 initially disputed questions were approved, resulting in a total of 125 questions deemed suitable for the exam. The suitable questions were classified as 51 (40.8%) easy, 45 (36%) medium-difficulty, and 29 (23.2%) hard. The main issues with unsuitable questions included questions that contained errors or improperly structured or with potential for an appeal (118 questions, 27.8%) and excessive simplicity (113 questions, 28.1%). By subject area: Biopsy techniques and B-cell lymphoma had the highest rates of suitable questions (63�65%). In addition, 37 questions were 2-stage complicated questions. Of those 7 were determined as appropriate (18.9%). 
Reference (2025): Shapiro J, Lyakhovitsky A, Freud T, Pavlotsky F, Khamaysi Z, Valdman-Grinshpoun Y, Dodiuk-Gad R, Goldberg I, Ingber A, Kaplan B, Avitan-Hersh E. Assessing ChatGPT-4's Capabilities in Generating Dermatology Board Examination Content: An Explorational Study. Acta Derm Venereol. 2025 Jan 3;105:adv41208. doi: 10.2340/actadv.v105.41208. PMID: 39749389; PMCID: PMC11697136.

Evaluation of ChatGPT-4 (GPT series) — Darier disease
Condition/subject: Professional education (English)
Category: Dermatology Examinations and Practice Questions — Professional Education
Validity/accuracy: 25.00% (sample size: 20)
Comments: Slightly higher in communication quality. Validity Score  is based  on the Suitablity measure. It will be slightly lower if computed as the unweighted average of five HealthBench-style dimensions for each subject or average of Accuracy,�Context awareness, and�Instruction following, most appropriate for assessment.. Completness can be assessed as the �suitability� reflecting whether the answer was seen as fully addressing the topic (percentage of suitable questions). Accuracy combines the two reviewers� independent judgments of answer correctness: (R1 % + R2 %) / 2; Context awareness = 100 � Dispute % High disagreement implies the model lacks clarity/context ? deductive inverse. Context awareness could be also derived based on rejection reasons like "too easy," "too hard," or "errored."; Communication quality = 100 � Dispute % Again, high dispute suggests unclear or ambiguous answers; Instruction following = (Suitable % + R2 %) / 2 Reflects a blend of general suitability and alignment with a reviewer�s expectations.
Dataset: ChatGPT-4 generated 402 questions, with 208 (51.7%) deemed acceptable by at least 1 reviewer. However, only 72 questions (18%) were accepted by both reviewers. After consensus discussions, 53 of the 136 initially disputed questions were approved, resulting in a total of 125 questions deemed suitable for the exam. The suitable questions were classified as 51 (40.8%) easy, 45 (36%) medium-difficulty, and 29 (23.2%) hard. The main issues with unsuitable questions included questions that contained errors or improperly structured or with potential for an appeal (118 questions, 27.8%) and excessive simplicity (113 questions, 28.1%). By subject area: Biopsy techniques and B-cell lymphoma had the highest rates of suitable questions (63�65%). In addition, 37 questions were 2-stage complicated questions. Of those 7 were determined as appropriate (18.9%). 
Reference (2025): Shapiro J, Lyakhovitsky A, Freud T, Pavlotsky F, Khamaysi Z, Valdman-Grinshpoun Y, Dodiuk-Gad R, Goldberg I, Ingber A, Kaplan B, Avitan-Hersh E. Assessing ChatGPT-4's Capabilities in Generating Dermatology Board Examination Content: An Explorational Study. Acta Derm Venereol. 2025 Jan 3;105:adv41208. doi: 10.2340/actadv.v105.41208. PMID: 39749389; PMCID: PMC11697136.

Evaluation of ChatGPT-4 (GPT series) — Ichthyoses
Condition/subject: Professional education (English)
Category: Dermatology Examinations and Practice Questions — Professional Education
Validity/accuracy: 15.00% (sample size: 40)
Comments: Very low scores, "Errored" questions. Validity Score  is based  on the Suitablity measure. It will be slightly lower if computed as the unweighted average of five HealthBench-style dimensions for each subject or average of Accuracy,�Context awareness, and�Instruction following, most appropriate for assessment.. Completness can be assessed as the �suitability� reflecting whether the answer was seen as fully addressing the topic (percentage of suitable questions). Accuracy combines the two reviewers� independent judgments of answer correctness: (R1 % + R2 %) / 2; Context awareness = 100 � Dispute % High disagreement implies the model lacks clarity/context ? deductive inverse. Context awareness could be also derived based on rejection reasons like "too easy," "too hard," or "errored."; Communication quality = 100 � Dispute % Again, high dispute suggests unclear or ambiguous answers; Instruction following = (Suitable % + R2 %) / 2 Reflects a blend of general suitability and alignment with a reviewer�s expectations.
Dataset: ChatGPT-4 generated 402 questions, with 208 (51.7%) deemed acceptable by at least 1 reviewer. However, only 72 questions (18%) were accepted by both reviewers. After consensus discussions, 53 of the 136 initially disputed questions were approved, resulting in a total of 125 questions deemed suitable for the exam. The suitable questions were classified as 51 (40.8%) easy, 45 (36%) medium-difficulty, and 29 (23.2%) hard. The main issues with unsuitable questions included questions that contained errors or improperly structured or with potential for an appeal (118 questions, 27.8%) and excessive simplicity (113 questions, 28.1%). By subject area: Biopsy techniques and B-cell lymphoma had the highest rates of suitable questions (63�65%). In addition, 37 questions were 2-stage complicated questions. Of those 7 were determined as appropriate (18.9%). 
Reference (2025): Shapiro J, Lyakhovitsky A, Freud T, Pavlotsky F, Khamaysi Z, Valdman-Grinshpoun Y, Dodiuk-Gad R, Goldberg I, Ingber A, Kaplan B, Avitan-Hersh E. Assessing ChatGPT-4's Capabilities in Generating Dermatology Board Examination Content: An Explorational Study. Acta Derm Venereol. 2025 Jan 3;105:adv41208. doi: 10.2340/actadv.v105.41208. PMID: 39749389; PMCID: PMC11697136.

Evaluation of ChatGPT-4 (GPT series) — Acne
Condition/subject: Professional education (English)
Category: Dermatology Examinations and Practice Questions — Professional Education
Validity/accuracy: 15.00% (sample size: 60)
Comments: Moderate scores, "Too easy" questions. Validity Score  is based  on the Suitablity measure. It will be slightly lower if computed as the unweighted average of five HealthBench-style dimensions for each subject or average of Accuracy,�Context awareness, and�Instruction following, most appropriate for assessment.. Completness can be assessed as the �suitability� reflecting whether the answer was seen as fully addressing the topic (percentage of suitable questions). Accuracy combines the two reviewers� independent judgments of answer correctness: (R1 % + R2 %) / 2; Context awareness = 100 � Dispute % High disagreement implies the model lacks clarity/context ? deductive inverse. Context awareness could be also derived based on rejection reasons like "too easy," "too hard," or "errored."; Communication quality = 100 � Dispute % Again, high dispute suggests unclear or ambiguous answers; Instruction following = (Suitable % + R2 %) / 2 Reflects a blend of general suitability and alignment with a reviewer�s expectations.
Dataset: ChatGPT-4 generated 402 questions, with 208 (51.7%) deemed acceptable by at least 1 reviewer. However, only 72 questions (18%) were accepted by both reviewers. After consensus discussions, 53 of the 136 initially disputed questions were approved, resulting in a total of 125 questions deemed suitable for the exam. The suitable questions were classified as 51 (40.8%) easy, 45 (36%) medium-difficulty, and 29 (23.2%) hard. The main issues with unsuitable questions included questions that contained errors or improperly structured or with potential for an appeal (118 questions, 27.8%) and excessive simplicity (113 questions, 28.1%). By subject area: Biopsy techniques and B-cell lymphoma had the highest rates of suitable questions (63�65%). In addition, 37 questions were 2-stage complicated questions. Of those 7 were determined as appropriate (18.9%). 
Reference (2025): Shapiro J, Lyakhovitsky A, Freud T, Pavlotsky F, Khamaysi Z, Valdman-Grinshpoun Y, Dodiuk-Gad R, Goldberg I, Ingber A, Kaplan B, Avitan-Hersh E. Assessing ChatGPT-4's Capabilities in Generating Dermatology Board Examination Content: An Explorational Study. Acta Derm Venereol. 2025 Jan 3;105:adv41208. doi: 10.2340/actadv.v105.41208. PMID: 39749389; PMCID: PMC11697136.

Evaluation of ChatGPT-4 (GPT series) — Rosacea
Condition/subject: Professional education (English)
Category: Dermatology Examinations and Practice Questions — Professional Education
Validity/accuracy: 15.00% (sample size: 20)
Comments: Slightly higher in context & communication. Validity Score  is based  on the Suitablity measure. It will be slightly lower if computed as the unweighted average of five HealthBench-style dimensions for each subject or average of Accuracy,�Context awareness, and�Instruction following, most appropriate for assessment.. Completness can be assessed as the �suitability� reflecting whether the answer was seen as fully addressing the topic (percentage of suitable questions). Accuracy combines the two reviewers� independent judgments of answer correctness: (R1 % + R2 %) / 2; Context awareness = 100 � Dispute % High disagreement implies the model lacks clarity/context ? deductive inverse. Context awareness could be also derived based on rejection reasons like "too easy," "too hard," or "errored."; Communication quality = 100 � Dispute % Again, high dispute suggests unclear or ambiguous answers; Instruction following = (Suitable % + R2 %) / 2 Reflects a blend of general suitability and alignment with a reviewer�s expectations.
Dataset: ChatGPT-4 generated 402 questions, with 208 (51.7%) deemed acceptable by at least 1 reviewer. However, only 72 questions (18%) were accepted by both reviewers. After consensus discussions, 53 of the 136 initially disputed questions were approved, resulting in a total of 125 questions deemed suitable for the exam. The suitable questions were classified as 51 (40.8%) easy, 45 (36%) medium-difficulty, and 29 (23.2%) hard. The main issues with unsuitable questions included questions that contained errors or improperly structured or with potential for an appeal (118 questions, 27.8%) and excessive simplicity (113 questions, 28.1%). By subject area: Biopsy techniques and B-cell lymphoma had the highest rates of suitable questions (63�65%). In addition, 37 questions were 2-stage complicated questions. Of those 7 were determined as appropriate (18.9%). 
Reference (2025): Shapiro J, Lyakhovitsky A, Freud T, Pavlotsky F, Khamaysi Z, Valdman-Grinshpoun Y, Dodiuk-Gad R, Goldberg I, Ingber A, Kaplan B, Avitan-Hersh E. Assessing ChatGPT-4's Capabilities in Generating Dermatology Board Examination Content: An Explorational Study. Acta Derm Venereol. 2025 Jan 3;105:adv41208. doi: 10.2340/actadv.v105.41208. PMID: 39749389; PMCID: PMC11697136.

Evaluation of ChatGPT-4 (GPT series) — Vasculitis
Condition/subject: Professional education (English)
Category: Dermatology Examinations and Practice Questions — Professional Education
Validity/accuracy: 14.00% (sample size: 50)
Comments: Very low scores, "too easy" but low in accuracy & communication. Validity Score  is based  on the Suitablity measure. It will be slightly lower if computed as the unweighted average of five HealthBench-style dimensions for each subject or average of Accuracy,�Context awareness, and�Instruction following, most appropriate for assessment.. Completness can be assessed as the �suitability� reflecting whether the answer was seen as fully addressing the topic (percentage of suitable questions). Accuracy combines the two reviewers� independent judgments of answer correctness: (R1 % + R2 %) / 2; Context awareness = 100 � Dispute % High disagreement implies the model lacks clarity/context ? deductive inverse. Context awareness could be also derived based on rejection reasons like "too easy," "too hard," or "errored."; Communication quality = 100 � Dispute % Again, high dispute suggests unclear or ambiguous answers; Instruction following = (Suitable % + R2 %) / 2 Reflects a blend of general suitability and alignment with a reviewer�s expectations.
Dataset: Clinical notes synthetically generated using two distinct multimodal large language models (LLMs), corresponding to each patient�s skin cancer images, encapsulating both visual and textual data representative of real-world scenarios.: https://www.kaggle.com/competitions/siim-isic-melanoma-classification The generated clinical notes are validated through a dual-method approach involving cross-evaluation and consensus scoring, utilizing metrics such as the BLEU score, ROUGE score, Overlap Coefficient, and Jaccard index. the full Kaggle competition package has ? 44 k images in all; most work focuses on the 33 126-image training set because it comes with diagnostic labels. 584 (?1.8 %) are histopathology-confirmed melanomas; the remainder are benign lesions.
Reference (2025): Chakkarapani, V., Poornapushpakala, S. & Suresh, S. Enhancing Skin Cancer Detection with Multimodal Data Integration: A Combined Approach Using Images and Clinical Notes. SN COMPUT. SCI. 6, 72 (2025). https://doi.org/10.1007/s42979-024-03601-x; https://www.kaggle.com/competitions/siim-isic-melanoma-classification

Evaluation of ChatGPT-4 (GPT series) — Melanoma, Clinical note generation
Condition/subject: Clinical dermatology (English)
Validity/accuracy: 97.03% (sample size: 33126)
Comments: The experimental results demonstrates a significant improvement in all classification metrics compared to single-modality models, achieving an accuracy of 99.51%, precision of 96.19%, recall of 97.95% and f1-score of 97.03% with the multimodal ALBEF model that uses GPT-4-turbo for clinical notes generation