Aurametrix/dermatology_longitudinal huggingface dataset:
Dataset: 100 images of people with psoriasis generated by two standard AI models (Dall-E and Midjourney). Additionally, a custom model was developed which incorporated a prompt injection aimed at �forcing� the AI (Dall-E 3) to reflect the skin tone distribution of the US population according to the 2012 American National Election Survey. This custom model generated another set of 100 images. The skin tones in these images were assessed by three researchers using the New Immigrant Survey skin tone scale, with the median value representing each image. 
Reference (2024): O'Malley A, Veenhuizen M, Ahmed A. Ensuring Appropriate Representation in Artificial Intelligence-Generated Medical Imagery: Protocol for a Methodological Approach to Address Skin Tone Bias. JMIR AI. 2024 Nov 27;3:e58275. doi: 10.2196/58275. PMID: 39602221; PMCID: PMC11635324. // was   Ensuring appropriate representation in AI-generated medical imagery: A Methodological Approach to Address Skin Tone Bias (Preprint) DOI: 10.2196/58275 URL: https://preprints.jmir.org/preprint/58275

Evaluation of Custom GPT: O�Malley, A., Representative Skin Tones. 2024. (GPT series) — Skin tone 10 (NIS scale) images -  approximately corresponds to  FST VI (very dark, deeply pigmented skin)
Condition/subject: Dark skin, pigmented skin (English)
Category: Medical Records and Diagnostic Processes — Clinical Practice
Validity/accuracy: 0.00% (sample size: 100)
Comments: The range of skin tones present in the images generated by both Dall-E 3 and Midjourney are significantly different to what would be expected in the US population.  CustomGPT shows a better result. Custom GPT Model with prompt injection closely aligns with the expected demographic distribution, evidenced by a lower Chi-Square statistic (17.35) and a non-significant p-value (P = .0435), indicating minimal deviation from the target distribution. The New Immigrant Survey (NIS) skin tone scale is a standardized scale used to classify and assess skin tone, primarily for research on social and demographic factors. The scale ranges from 1 to 10, where each score represents a gradation of skin tone, with 1 being the lightest skin tone and 10 being the darkest. Validity is computed as Goodness of Fit Percentage estimated based on Standardized Residuals
Dataset: 100 images of people with psoriasis generated by two standard AI models (Dall-E and Midjourney). Additionally, a custom model was developed which incorporated a prompt injection aimed at �forcing� the AI (Dall-E 3) to reflect the skin tone distribution of the US population according to the 2012 American National Election Survey. This custom model generated another set of 100 images. The skin tones in these images were assessed by three researchers using the New Immigrant Survey skin tone scale, with the median value representing each image. 
Reference (2024): O'Malley A, Veenhuizen M, Ahmed A. Ensuring Appropriate Representation in Artificial Intelligence-Generated Medical Imagery: Protocol for a Methodological Approach to Address Skin Tone Bias. JMIR AI. 2024 Nov 27;3:e58275. doi: 10.2196/58275. PMID: 39602221; PMCID: PMC11635324. // was   Ensuring appropriate representation in AI-generated medical imagery: A Methodological Approach to Address Skin Tone Bias (Preprint) DOI: 10.2196/58275 URL: https://preprints.jmir.org/preprint/58275

Evaluation of Dall-E (GPT series) — Skin tone 5 (NIS scale) images -  approximately corresponds to FST IV (olive or medium-dark skin tones).
Condition/subject: Medium Skin (English)
Category: Medical Records and Diagnostic Processes — Clinical Practice
Validity/accuracy: 0.00% (sample size: 96)
Comments: The range of skin tones present in the images generated by both Dall-E 3 and Midjourney are significantly different to what would be expected in the US population.  CustomGPT shows a better result. Custom GPT Model with prompt injection closely aligns with the expected demographic distribution, evidenced by a lower Chi-Square statistic (17.35) and a non-significant p-value (P = .0435), indicating minimal deviation from the target distribution. The New Immigrant Survey (NIS) skin tone scale is a standardized scale used to classify and assess skin tone, primarily for research on social and demographic factors. The scale ranges from 1 to 10, where each score represents a gradation of skin tone, with 1 being the lightest skin tone and 10 being the darkest. Validity is computed as Goodness of Fit Percentage estimated based on Standardized Residuals
Dataset: 100 images of people with psoriasis generated by two standard AI models (Dall-E and Midjourney). Additionally, a custom model was developed which incorporated a prompt injection aimed at �forcing� the AI (Dall-E 3) to reflect the skin tone distribution of the US population according to the 2012 American National Election Survey. This custom model generated another set of 100 images. The skin tones in these images were assessed by three researchers using the New Immigrant Survey skin tone scale, with the median value representing each image. 
Reference (2024): O'Malley A, Veenhuizen M, Ahmed A. Ensuring Appropriate Representation in Artificial Intelligence-Generated Medical Imagery: Protocol for a Methodological Approach to Address Skin Tone Bias. JMIR AI. 2024 Nov 27;3:e58275. doi: 10.2196/58275. PMID: 39602221; PMCID: PMC11635324. // was   Ensuring appropriate representation in AI-generated medical imagery: A Methodological Approach to Address Skin Tone Bias (Preprint) DOI: 10.2196/58275 URL: https://preprints.jmir.org/preprint/58275

Evaluation of Dall-E (GPT series) — Skin tone 6 (NIS scale) images -  approximately corresponds to FST IV (olive or medium-dark skin tones).
Condition/subject: Olive skin (English)
Category: Medical Records and Diagnostic Processes — Clinical Practice
Validity/accuracy: 0.00% (sample size: 96)
Comments: The range of skin tones present in the images generated by both Dall-E 3 and Midjourney are significantly different to what would be expected in the US population.  CustomGPT shows a better result. Custom GPT Model with prompt injection closely aligns with the expected demographic distribution, evidenced by a lower Chi-Square statistic (17.35) and a non-significant p-value (P = .0435), indicating minimal deviation from the target distribution. The New Immigrant Survey (NIS) skin tone scale is a standardized scale used to classify and assess skin tone, primarily for research on social and demographic factors. The scale ranges from 1 to 10, where each score represents a gradation of skin tone, with 1 being the lightest skin tone and 10 being the darkest. Validity is computed as Goodness of Fit Percentage estimated based on Standardized Residuals
Dataset: 100 images of people with psoriasis generated by two standard AI models (Dall-E and Midjourney). Additionally, a custom model was developed which incorporated a prompt injection aimed at �forcing� the AI (Dall-E 3) to reflect the skin tone distribution of the US population according to the 2012 American National Election Survey. This custom model generated another set of 100 images. The skin tones in these images were assessed by three researchers using the New Immigrant Survey skin tone scale, with the median value representing each image. 
Reference (2024): O'Malley A, Veenhuizen M, Ahmed A. Ensuring Appropriate Representation in Artificial Intelligence-Generated Medical Imagery: Protocol for a Methodological Approach to Address Skin Tone Bias. JMIR AI. 2024 Nov 27;3:e58275. doi: 10.2196/58275. PMID: 39602221; PMCID: PMC11635324. // was   Ensuring appropriate representation in AI-generated medical imagery: A Methodological Approach to Address Skin Tone Bias (Preprint) DOI: 10.2196/58275 URL: https://preprints.jmir.org/preprint/58275

Evaluation of Dall-E (GPT series) — Skin tone 7 (NIS scale) images -  approximately corresponds to  FST V-VI
Condition/subject: Dark skin (English)
Category: Medical Records and Diagnostic Processes — Clinical Practice
Validity/accuracy: 0.00% (sample size: 96)
Comments: The range of skin tones present in the images generated by both Dall-E 3 and Midjourney are significantly different to what would be expected in the US population.  CustomGPT shows a better result. Custom GPT Model with prompt injection closely aligns with the expected demographic distribution, evidenced by a lower Chi-Square statistic (17.35) and a non-significant p-value (P = .0435), indicating minimal deviation from the target distribution. The New Immigrant Survey (NIS) skin tone scale is a standardized scale used to classify and assess skin tone, primarily for research on social and demographic factors. The scale ranges from 1 to 10, where each score represents a gradation of skin tone, with 1 being the lightest skin tone and 10 being the darkest. Validity is computed as Goodness of Fit Percentage estimated based on Standardized Residuals
Dataset: 100 images of people with psoriasis generated by two standard AI models (Dall-E and Midjourney). Additionally, a custom model was developed which incorporated a prompt injection aimed at �forcing� the AI (Dall-E 3) to reflect the skin tone distribution of the US population according to the 2012 American National Election Survey. This custom model generated another set of 100 images. The skin tones in these images were assessed by three researchers using the New Immigrant Survey skin tone scale, with the median value representing each image. 
Reference (2024): O'Malley A, Veenhuizen M, Ahmed A. Ensuring Appropriate Representation in Artificial Intelligence-Generated Medical Imagery: Protocol for a Methodological Approach to Address Skin Tone Bias. JMIR AI. 2024 Nov 27;3:e58275. doi: 10.2196/58275. PMID: 39602221; PMCID: PMC11635324. // was   Ensuring appropriate representation in AI-generated medical imagery: A Methodological Approach to Address Skin Tone Bias (Preprint) DOI: 10.2196/58275 URL: https://preprints.jmir.org/preprint/58275

Evaluation of Dall-E (GPT series) — Skin tone 9 (NIS scale) images -  approximately corresponds to  FST V-VI (dark brown skin with warm, neutral, or occasionally cooler undertones)
Condition/subject: Dark skin (English)
Category: Medical Records and Diagnostic Processes — Clinical Practice
Validity/accuracy: 0.00% (sample size: 96)
Comments: The range of skin tones present in the images generated by both Dall-E 3 and Midjourney are significantly different to what would be expected in the US population.  CustomGPT shows a better result. Custom GPT Model with prompt injection closely aligns with the expected demographic distribution, evidenced by a lower Chi-Square statistic (17.35) and a non-significant p-value (P = .0435), indicating minimal deviation from the target distribution. The New Immigrant Survey (NIS) skin tone scale is a standardized scale used to classify and assess skin tone, primarily for research on social and demographic factors. The scale ranges from 1 to 10, where each score represents a gradation of skin tone, with 1 being the lightest skin tone and 10 being the darkest. Validity is computed as Goodness of Fit Percentage estimated based on Standardized Residuals
Dataset: 100 images of people with psoriasis generated by two standard AI models (Dall-E and Midjourney). Additionally, a custom model was developed which incorporated a prompt injection aimed at �forcing� the AI (Dall-E 3) to reflect the skin tone distribution of the US population according to the 2012 American National Election Survey. This custom model generated another set of 100 images. The skin tones in these images were assessed by three researchers using the New Immigrant Survey skin tone scale, with the median value representing each image. 
Reference (2024): O'Malley A, Veenhuizen M, Ahmed A. Ensuring Appropriate Representation in Artificial Intelligence-Generated Medical Imagery: Protocol for a Methodological Approach to Address Skin Tone Bias. JMIR AI. 2024 Nov 27;3:e58275. doi: 10.2196/58275. PMID: 39602221; PMCID: PMC11635324. // was   Ensuring appropriate representation in AI-generated medical imagery: A Methodological Approach to Address Skin Tone Bias (Preprint) DOI: 10.2196/58275 URL: https://preprints.jmir.org/preprint/58275

Evaluation of Dall-E (GPT series) — Skin tone 10 (NIS scale) images -  approximately corresponds to  FST VI (very dark, deeply pigmented skin)
Condition/subject: Dark skin, pigmented skin (English)
Category: Medical Records and Diagnostic Processes — Clinical Practice
Validity/accuracy: 0.00% (sample size: 96)
Comments: The range of skin tones present in the images generated by both Dall-E 3 and Midjourney are significantly different to what would be expected in the US population.  CustomGPT shows a better result. Custom GPT Model with prompt injection closely aligns with the expected demographic distribution, evidenced by a lower Chi-Square statistic (17.35) and a non-significant p-value (P = .0435), indicating minimal deviation from the target distribution. The New Immigrant Survey (NIS) skin tone scale is a standardized scale used to classify and assess skin tone, primarily for research on social and demographic factors. The scale ranges from 1 to 10, where each score represents a gradation of skin tone, with 1 being the lightest skin tone and 10 being the darkest. Validity is computed as Goodness of Fit Percentage estimated based on Standardized Residuals
Dataset: 100 images of people with psoriasis generated by two standard AI models (Dall-E and Midjourney). Additionally, a custom model was developed which incorporated a prompt injection aimed at �forcing� the AI (Dall-E 3) to reflect the skin tone distribution of the US population according to the 2012 American National Election Survey. This custom model generated another set of 100 images. The skin tones in these images were assessed by three researchers using the New Immigrant Survey skin tone scale, with the median value representing each image. 
Reference (2024): O'Malley A, Veenhuizen M, Ahmed A. Ensuring Appropriate Representation in Artificial Intelligence-Generated Medical Imagery: Protocol for a Methodological Approach to Address Skin Tone Bias. JMIR AI. 2024 Nov 27;3:e58275. doi: 10.2196/58275. PMID: 39602221; PMCID: PMC11635324. // was   Ensuring appropriate representation in AI-generated medical imagery: A Methodological Approach to Address Skin Tone Bias (Preprint) DOI: 10.2196/58275 URL: https://preprints.jmir.org/preprint/58275

Evaluation of Midjourney (Midjourney) — Skin tone 4 (NIS scale) images -  approximately corresponds to FST III (medium skin tones).
Condition/subject: Medium Skin (English)
Category: Medical Records and Diagnostic Processes — Clinical Practice
Validity/accuracy: 0.00% (sample size: 99)
Comments: The range of skin tones present in the images generated by both Dall-E 3 and Midjourney are significantly different to what would be expected in the US population.  CustomGPT shows a better result. Custom GPT Model with prompt injection closely aligns with the expected demographic distribution, evidenced by a lower Chi-Square statistic (17.35) and a non-significant p-value (P = .0435), indicating minimal deviation from the target distribution. The New Immigrant Survey (NIS) skin tone scale is a standardized scale used to classify and assess skin tone, primarily for research on social and demographic factors. The scale ranges from 1 to 10, where each score represents a gradation of skin tone, with 1 being the lightest skin tone and 10 being the darkest. Validity is computed as Goodness of Fit Percentage estimated based on Standardized Residuals
Dataset: 100 images of people with psoriasis generated by two standard AI models (Dall-E and Midjourney). Additionally, a custom model was developed which incorporated a prompt injection aimed at �forcing� the AI (Dall-E 3) to reflect the skin tone distribution of the US population according to the 2012 American National Election Survey. This custom model generated another set of 100 images. The skin tones in these images were assessed by three researchers using the New Immigrant Survey skin tone scale, with the median value representing each image. 
Reference (2024): O'Malley A, Veenhuizen M, Ahmed A. Ensuring Appropriate Representation in Artificial Intelligence-Generated Medical Imagery: Protocol for a Methodological Approach to Address Skin Tone Bias. JMIR AI. 2024 Nov 27;3:e58275. doi: 10.2196/58275. PMID: 39602221; PMCID: PMC11635324. // was   Ensuring appropriate representation in AI-generated medical imagery: A Methodological Approach to Address Skin Tone Bias (Preprint) DOI: 10.2196/58275 URL: https://preprints.jmir.org/preprint/58275

Evaluation of Midjourney (Midjourney) — Skin tone 5 (NIS scale) images -  approximately corresponds to FST IV (olive or medium-dark skin tones).
Condition/subject: Medium Skin (English)
Category: Medical Records and Diagnostic Processes — Clinical Practice
Validity/accuracy: 0.00% (sample size: 99)
Comments: The range of skin tones present in the images generated by both Dall-E 3 and Midjourney are significantly different to what would be expected in the US population.  CustomGPT shows a better result. Custom GPT Model with prompt injection closely aligns with the expected demographic distribution, evidenced by a lower Chi-Square statistic (17.35) and a non-significant p-value (P = .0435), indicating minimal deviation from the target distribution. The New Immigrant Survey (NIS) skin tone scale is a standardized scale used to classify and assess skin tone, primarily for research on social and demographic factors. The scale ranges from 1 to 10, where each score represents a gradation of skin tone, with 1 being the lightest skin tone and 10 being the darkest. Validity is computed as Goodness of Fit Percentage estimated based on Standardized Residuals
Dataset: 100 images of people with psoriasis generated by two standard AI models (Dall-E and Midjourney). Additionally, a custom model was developed which incorporated a prompt injection aimed at �forcing� the AI (Dall-E 3) to reflect the skin tone distribution of the US population according to the 2012 American National Election Survey. This custom model generated another set of 100 images. The skin tones in these images were assessed by three researchers using the New Immigrant Survey skin tone scale, with the median value representing each image. 
Reference (2024): O'Malley A, Veenhuizen M, Ahmed A. Ensuring Appropriate Representation in Artificial Intelligence-Generated Medical Imagery: Protocol for a Methodological Approach to Address Skin Tone Bias. JMIR AI. 2024 Nov 27;3:e58275. doi: 10.2196/58275. PMID: 39602221; PMCID: PMC11635324. // was   Ensuring appropriate representation in AI-generated medical imagery: A Methodological Approach to Address Skin Tone Bias (Preprint) DOI: 10.2196/58275 URL: https://preprints.jmir.org/preprint/58275

Evaluation of Midjourney (Midjourney) — Skin tone 6 (NIS scale) images -  approximately corresponds to FST IV (olive or medium-dark skin tones).
Condition/subject: Olive skin (English)
Category: Medical Records and Diagnostic Processes — Clinical Practice
Validity/accuracy: 0.00% (sample size: 99)
Comments: The range of skin tones present in the images generated by both Dall-E 3 and Midjourney are significantly different to what would be expected in the US population.  CustomGPT shows a better result. Custom GPT Model with prompt injection closely aligns with the expected demographic distribution, evidenced by a lower Chi-Square statistic (17.35) and a non-significant p-value (P = .0435), indicating minimal deviation from the target distribution. The New Immigrant Survey (NIS) skin tone scale is a standardized scale used to classify and assess skin tone, primarily for research on social and demographic factors. The scale ranges from 1 to 10, where each score represents a gradation of skin tone, with 1 being the lightest skin tone and 10 being the darkest. Validity is computed as Goodness of Fit Percentage estimated based on Standardized Residuals
Dataset: 100 images of people with psoriasis generated by two standard AI models (Dall-E and Midjourney). Additionally, a custom model was developed which incorporated a prompt injection aimed at �forcing� the AI (Dall-E 3) to reflect the skin tone distribution of the US population according to the 2012 American National Election Survey. This custom model generated another set of 100 images. The skin tones in these images were assessed by three researchers using the New Immigrant Survey skin tone scale, with the median value representing each image. 
Reference (2024): O'Malley A, Veenhuizen M, Ahmed A. Ensuring Appropriate Representation in Artificial Intelligence-Generated Medical Imagery: Protocol for a Methodological Approach to Address Skin Tone Bias. JMIR AI. 2024 Nov 27;3:e58275. doi: 10.2196/58275. PMID: 39602221; PMCID: PMC11635324. // was   Ensuring appropriate representation in AI-generated medical imagery: A Methodological Approach to Address Skin Tone Bias (Preprint) DOI: 10.2196/58275 URL: https://preprints.jmir.org/preprint/58275

Evaluation of Midjourney (Midjourney) — Skin tone 7 (NIS scale) images -  approximately corresponds to  FST V-VI
Condition/subject: Dark skin (English)
Category: Medical Records and Diagnostic Processes — Clinical Practice
Validity/accuracy: 0.00% (sample size: 99)
Comments: The range of skin tones present in the images generated by both Dall-E 3 and Midjourney are significantly different to what would be expected in the US population.  CustomGPT shows a better result. Custom GPT Model with prompt injection closely aligns with the expected demographic distribution, evidenced by a lower Chi-Square statistic (17.35) and a non-significant p-value (P = .0435), indicating minimal deviation from the target distribution. The New Immigrant Survey (NIS) skin tone scale is a standardized scale used to classify and assess skin tone, primarily for research on social and demographic factors. The scale ranges from 1 to 10, where each score represents a gradation of skin tone, with 1 being the lightest skin tone and 10 being the darkest. Validity is computed as Goodness of Fit Percentage estimated based on Standardized Residuals
Dataset: 100 images of people with psoriasis generated by two standard AI models (Dall-E and Midjourney). Additionally, a custom model was developed which incorporated a prompt injection aimed at �forcing� the AI (Dall-E 3) to reflect the skin tone distribution of the US population according to the 2012 American National Election Survey. This custom model generated another set of 100 images. The skin tones in these images were assessed by three researchers using the New Immigrant Survey skin tone scale, with the median value representing each image. 
Reference (2024): O'Malley A, Veenhuizen M, Ahmed A. Ensuring Appropriate Representation in Artificial Intelligence-Generated Medical Imagery: Protocol for a Methodological Approach to Address Skin Tone Bias. JMIR AI. 2024 Nov 27;3:e58275. doi: 10.2196/58275. PMID: 39602221; PMCID: PMC11635324. // was   Ensuring appropriate representation in AI-generated medical imagery: A Methodological Approach to Address Skin Tone Bias (Preprint) DOI: 10.2196/58275 URL: https://preprints.jmir.org/preprint/58275

Evaluation of Midjourney (Midjourney) — Skin tone 8 (NIS scale) images -  approximately corresponds to  FST V (dark brown)
Condition/subject: Dark skin (English)
Category: Medical Records and Diagnostic Processes — Clinical Practice
Validity/accuracy: 0.00% (sample size: 99)
Comments: The range of skin tones present in the images generated by both Dall-E 3 and Midjourney are significantly different to what would be expected in the US population.  CustomGPT shows a better result. Custom GPT Model with prompt injection closely aligns with the expected demographic distribution, evidenced by a lower Chi-Square statistic (17.35) and a non-significant p-value (P = .0435), indicating minimal deviation from the target distribution. The New Immigrant Survey (NIS) skin tone scale is a standardized scale used to classify and assess skin tone, primarily for research on social and demographic factors. The scale ranges from 1 to 10, where each score represents a gradation of skin tone, with 1 being the lightest skin tone and 10 being the darkest. Validity is computed as Goodness of Fit Percentage estimated based on Standardized Residuals
Dataset: 100 images of people with psoriasis generated by two standard AI models (Dall-E and Midjourney). Additionally, a custom model was developed which incorporated a prompt injection aimed at �forcing� the AI (Dall-E 3) to reflect the skin tone distribution of the US population according to the 2012 American National Election Survey. This custom model generated another set of 100 images. The skin tones in these images were assessed by three researchers using the New Immigrant Survey skin tone scale, with the median value representing each image. 
Reference (2024): O'Malley A, Veenhuizen M, Ahmed A. Ensuring Appropriate Representation in Artificial Intelligence-Generated Medical Imagery: Protocol for a Methodological Approach to Address Skin Tone Bias. JMIR AI. 2024 Nov 27;3:e58275. doi: 10.2196/58275. PMID: 39602221; PMCID: PMC11635324. // was   Ensuring appropriate representation in AI-generated medical imagery: A Methodological Approach to Address Skin Tone Bias (Preprint) DOI: 10.2196/58275 URL: https://preprints.jmir.org/preprint/58275

Evaluation of Midjourney (Midjourney) — Skin tone 9 (NIS scale) images -  approximately corresponds to  FST V-VI (dark brown skin with warm, neutral, or occasionally cooler undertones)
Condition/subject: Dark skin (English)
Category: Medical Records and Diagnostic Processes — Clinical Practice
Validity/accuracy: 0.00% (sample size: 99)
Comments: The range of skin tones present in the images generated by both Dall-E 3 and Midjourney are significantly different to what would be expected in the US population.  CustomGPT shows a better result. Custom GPT Model with prompt injection closely aligns with the expected demographic distribution, evidenced by a lower Chi-Square statistic (17.35) and a non-significant p-value (P = .0435), indicating minimal deviation from the target distribution. The New Immigrant Survey (NIS) skin tone scale is a standardized scale used to classify and assess skin tone, primarily for research on social and demographic factors. The scale ranges from 1 to 10, where each score represents a gradation of skin tone, with 1 being the lightest skin tone and 10 being the darkest. Validity is computed as Goodness of Fit Percentage estimated based on Standardized Residuals
Dataset: 100 images of people with psoriasis generated by two standard AI models (Dall-E and Midjourney). Additionally, a custom model was developed which incorporated a prompt injection aimed at �forcing� the AI (Dall-E 3) to reflect the skin tone distribution of the US population according to the 2012 American National Election Survey. This custom model generated another set of 100 images. The skin tones in these images were assessed by three researchers using the New Immigrant Survey skin tone scale, with the median value representing each image. 
Reference (2024): O'Malley A, Veenhuizen M, Ahmed A. Ensuring Appropriate Representation in Artificial Intelligence-Generated Medical Imagery: Protocol for a Methodological Approach to Address Skin Tone Bias. JMIR AI. 2024 Nov 27;3:e58275. doi: 10.2196/58275. PMID: 39602221; PMCID: PMC11635324. // was   Ensuring appropriate representation in AI-generated medical imagery: A Methodological Approach to Address Skin Tone Bias (Preprint) DOI: 10.2196/58275 URL: https://preprints.jmir.org/preprint/58275

Evaluation of Midjourney (Midjourney) — Skin tone 10 (NIS scale) images -  approximately corresponds to  FST VI (very dark, deeply pigmented skin)
Condition/subject: Dark skin, pigmented skin (English)
Category: Medical Records and Diagnostic Processes — Clinical Practice
Validity/accuracy: 0.00% (sample size: 99)
Comments: The range of skin tones present in the images generated by both Dall-E 3 and Midjourney are significantly different to what would be expected in the US population.  CustomGPT shows a better result. Custom GPT Model with prompt injection closely aligns with the expected demographic distribution, evidenced by a lower Chi-Square statistic (17.35) and a non-significant p-value (P = .0435), indicating minimal deviation from the target distribution. The New Immigrant Survey (NIS) skin tone scale is a standardized scale used to classify and assess skin tone, primarily for research on social and demographic factors. The scale ranges from 1 to 10, where each score represents a gradation of skin tone, with 1 being the lightest skin tone and 10 being the darkest. Validity is computed as Goodness of Fit Percentage estimated based on Standardized Residuals
Dataset: 100 images of people with psoriasis generated by two standard AI models (Dall-E and Midjourney). Additionally, a custom model was developed which incorporated a prompt injection aimed at �forcing� the AI (Dall-E 3) to reflect the skin tone distribution of the US population according to the 2012 American National Election Survey. This custom model generated another set of 100 images. The skin tones in these images were assessed by three researchers using the New Immigrant Survey skin tone scale, with the median value representing each image. 
Reference (2024): O'Malley A, Veenhuizen M, Ahmed A. Ensuring Appropriate Representation in Artificial Intelligence-Generated Medical Imagery: Protocol for a Methodological Approach to Address Skin Tone Bias. JMIR AI. 2024 Nov 27;3:e58275. doi: 10.2196/58275. PMID: 39602221; PMCID: PMC11635324. // was   Ensuring appropriate representation in AI-generated medical imagery: A Methodological Approach to Address Skin Tone Bias (Preprint) DOI: 10.2196/58275 URL: https://preprints.jmir.org/preprint/58275

Evaluation of Dall-E (GPT series) — Skin tone 1 (NIS scale) images -  approximately corresponds to FST I-II (lighter skin tones).
Condition/subject: Light Skin (English)
Category: Medical Records and Diagnostic Processes — Clinical Practice
Validity/accuracy: 0.95% (sample size: 96)
Comments: The range of skin tones present in the images generated by both Dall-E 3 and Midjourney are significantly different to what would be expected in the US population.  CustomGPT shows a better result. Custom GPT Model with prompt injection closely aligns with the expected demographic distribution, evidenced by a lower Chi-Square statistic (17.35) and a non-significant p-value (P = .0435), indicating minimal deviation from the target distribution. The New Immigrant Survey (NIS) skin tone scale is a standardized scale used to classify and assess skin tone, primarily for research on social and demographic factors. The scale ranges from 1 to 10, where each score represents a gradation of skin tone, with 1 being the lightest skin tone and 10 being the darkest. Validity is computed as Goodness of Fit Percentage estimated based on Standardized Residuals
Dataset: 100 images of people with psoriasis generated by two standard AI models (Dall-E and Midjourney). Additionally, a custom model was developed which incorporated a prompt injection aimed at �forcing� the AI (Dall-E 3) to reflect the skin tone distribution of the US population according to the 2012 American National Election Survey. This custom model generated another set of 100 images. The skin tones in these images were assessed by three researchers using the New Immigrant Survey skin tone scale, with the median value representing each image. 
Reference (2024): O'Malley A, Veenhuizen M, Ahmed A. Ensuring Appropriate Representation in Artificial Intelligence-Generated Medical Imagery: Protocol for a Methodological Approach to Address Skin Tone Bias. JMIR AI. 2024 Nov 27;3:e58275. doi: 10.2196/58275. PMID: 39602221; PMCID: PMC11635324. // was   Ensuring appropriate representation in AI-generated medical imagery: A Methodological Approach to Address Skin Tone Bias (Preprint) DOI: 10.2196/58275 URL: https://preprints.jmir.org/preprint/58275

Evaluation of Midjourney (Midjourney) — Skin tone 3 (NIS scale) images -  approximately corresponds to FST III (medium skin tones).
Condition/subject: Fitzpatrick III skin type (English)
Category: Medical Records and Diagnostic Processes — Clinical Practice
Validity/accuracy: 6.87% (sample size: 99)
Comments: The range of skin tones present in the images generated by both Dall-E 3 and Midjourney are significantly different to what would be expected in the US population.  CustomGPT shows a better result. Custom GPT Model with prompt injection closely aligns with the expected demographic distribution, evidenced by a lower Chi-Square statistic (17.35) and a non-significant p-value (P = .0435), indicating minimal deviation from the target distribution. The New Immigrant Survey (NIS) skin tone scale is a standardized scale used to classify and assess skin tone, primarily for research on social and demographic factors. The scale ranges from 1 to 10, where each score represents a gradation of skin tone, with 1 being the lightest skin tone and 10 being the darkest. Validity is computed as Goodness of Fit Percentage estimated based on Standardized Residuals
Dataset: 100 images of people with psoriasis generated by two standard AI models (Dall-E and Midjourney). Additionally, a custom model was developed which incorporated a prompt injection aimed at �forcing� the AI (Dall-E 3) to reflect the skin tone distribution of the US population according to the 2012 American National Election Survey. This custom model generated another set of 100 images. The skin tones in these images were assessed by three researchers using the New Immigrant Survey skin tone scale, with the median value representing each image. 
Reference (2024): O'Malley A, Veenhuizen M, Ahmed A. Ensuring Appropriate Representation in Artificial Intelligence-Generated Medical Imagery: Protocol for a Methodological Approach to Address Skin Tone Bias. JMIR AI. 2024 Nov 27;3:e58275. doi: 10.2196/58275. PMID: 39602221; PMCID: PMC11635324. // was   Ensuring appropriate representation in AI-generated medical imagery: A Methodological Approach to Address Skin Tone Bias (Preprint) DOI: 10.2196/58275 URL: https://preprints.jmir.org/preprint/58275

Evaluation of Custom GPT: O�Malley, A., Representative Skin Tones. 2024. (GPT series) — Skin tone 9 (NIS scale) images -  approximately corresponds to  FST V-VI (dark brown skin with warm, neutral, or occasionally cooler undertones)
Condition/subject: Skin of color (English)
Category: Medical Records and Diagnostic Processes — Clinical Practice
Validity/accuracy: 19.28% (sample size: 100)
Comments: The range of skin tones present in the images generated by both Dall-E 3 and Midjourney are significantly different to what would be expected in the US population.  CustomGPT shows a better result. Custom GPT Model with prompt injection closely aligns with the expected demographic distribution, evidenced by a lower Chi-Square statistic (17.35) and a non-significant p-value (P = .0435), indicating minimal deviation from the target distribution. The New Immigrant Survey (NIS) skin tone scale is a standardized scale used to classify and assess skin tone, primarily for research on social and demographic factors. The scale ranges from 1 to 10, where each score represents a gradation of skin tone, with 1 being the lightest skin tone and 10 being the darkest. Validity is computed as Goodness of Fit Percentage estimated based on Standardized Residuals
Dataset: 100 images of people with psoriasis generated by two standard AI models (Dall-E and Midjourney). Additionally, a custom model was developed which incorporated a prompt injection aimed at �forcing� the AI (Dall-E 3) to reflect the skin tone distribution of the US population according to the 2012 American National Election Survey. This custom model generated another set of 100 images. The skin tones in these images were assessed by three researchers using the New Immigrant Survey skin tone scale, with the median value representing each image. 
Reference (2024): O'Malley A, Veenhuizen M, Ahmed A. Ensuring Appropriate Representation in Artificial Intelligence-Generated Medical Imagery: Protocol for a Methodological Approach to Address Skin Tone Bias. JMIR AI. 2024 Nov 27;3:e58275. doi: 10.2196/58275. PMID: 39602221; PMCID: PMC11635324. // was   Ensuring appropriate representation in AI-generated medical imagery: A Methodological Approach to Address Skin Tone Bias (Preprint) DOI: 10.2196/58275 URL: https://preprints.jmir.org/preprint/58275

Evaluation of Custom GPT: O�Malley, A., Representative Skin Tones. 2024. (GPT series) — Skin tone 8 (NIS scale) images -  approximately corresponds to  FST V (dark brown)
Condition/subject: Skin of color (English)
Category: Medical Records and Diagnostic Processes — Clinical Practice
Validity/accuracy: 19.34% (sample size: 100)
Comments: The range of skin tones present in the images generated by both Dall-E 3 and Midjourney are significantly different to what would be expected in the US population.  CustomGPT shows a better result. Custom GPT Model with prompt injection closely aligns with the expected demographic distribution, evidenced by a lower Chi-Square statistic (17.35) and a non-significant p-value (P = .0435), indicating minimal deviation from the target distribution. The New Immigrant Survey (NIS) skin tone scale is a standardized scale used to classify and assess skin tone, primarily for research on social and demographic factors. The scale ranges from 1 to 10, where each score represents a gradation of skin tone, with 1 being the lightest skin tone and 10 being the darkest. Validity is computed as Goodness of Fit Percentage estimated based on Standardized Residuals
Dataset: 100 images of people with psoriasis generated by two standard AI models (Dall-E and Midjourney). Additionally, a custom model was developed which incorporated a prompt injection aimed at �forcing� the AI (Dall-E 3) to reflect the skin tone distribution of the US population according to the 2012 American National Election Survey. This custom model generated another set of 100 images. The skin tones in these images were assessed by three researchers using the New Immigrant Survey skin tone scale, with the median value representing each image. 
Reference (2024): O'Malley A, Veenhuizen M, Ahmed A. Ensuring Appropriate Representation in Artificial Intelligence-Generated Medical Imagery: Protocol for a Methodological Approach to Address Skin Tone Bias. JMIR AI. 2024 Nov 27;3:e58275. doi: 10.2196/58275. PMID: 39602221; PMCID: PMC11635324. // was   Ensuring appropriate representation in AI-generated medical imagery: A Methodological Approach to Address Skin Tone Bias (Preprint) DOI: 10.2196/58275 URL: https://preprints.jmir.org/preprint/58275

Evaluation of Dall-E (GPT series) — Skin tone 8 (NIS scale) images -  approximately corresponds to  FST V (dark brown)
Condition/subject: Skin of color (English)
Category: Medical Records and Diagnostic Processes — Clinical Practice
Validity/accuracy: 20.16% (sample size: 96)
Comments: The range of skin tones present in the images generated by both Dall-E 3 and Midjourney are significantly different to what would be expected in the US population.  CustomGPT shows a better result. Custom GPT Model with prompt injection closely aligns with the expected demographic distribution, evidenced by a lower Chi-Square statistic (17.35) and a non-significant p-value (P = .0435), indicating minimal deviation from the target distribution. The New Immigrant Survey (NIS) skin tone scale is a standardized scale used to classify and assess skin tone, primarily for research on social and demographic factors. The scale ranges from 1 to 10, where each score represents a gradation of skin tone, with 1 being the lightest skin tone and 10 being the darkest. Validity is computed as Goodness of Fit Percentage estimated based on Standardized Residuals
Dataset: 100 images of people with psoriasis generated by two standard AI models (Dall-E and Midjourney). Additionally, a custom model was developed which incorporated a prompt injection aimed at �forcing� the AI (Dall-E 3) to reflect the skin tone distribution of the US population according to the 2012 American National Election Survey. This custom model generated another set of 100 images. The skin tones in these images were assessed by three researchers using the New Immigrant Survey skin tone scale, with the median value representing each image. 
Reference (2024): O'Malley A, Veenhuizen M, Ahmed A. Ensuring Appropriate Representation in Artificial Intelligence-Generated Medical Imagery: Protocol for a Methodological Approach to Address Skin Tone Bias. JMIR AI. 2024 Nov 27;3:e58275. doi: 10.2196/58275. PMID: 39602221; PMCID: PMC11635324. // was   Ensuring appropriate representation in AI-generated medical imagery: A Methodological Approach to Address Skin Tone Bias (Preprint) DOI: 10.2196/58275 URL: https://preprints.jmir.org/preprint/58275

Evaluation of Midjourney (Midjourney) — Skin tone 2 (NIS scale) images -  approximately corresponds to FST I-II (lighter skin tones).
Condition/subject: Skin of color (English)
Category: Medical Records and Diagnostic Processes — Clinical Practice
Validity/accuracy: 20.16% (sample size: 99)
Comments: The range of skin tones present in the images generated by both Dall-E 3 and Midjourney are significantly different to what would be expected in the US population.  CustomGPT shows a better result. Custom GPT Model with prompt injection closely aligns with the expected demographic distribution, evidenced by a lower Chi-Square statistic (17.35) and a non-significant p-value (P = .0435), indicating minimal deviation from the target distribution. The New Immigrant Survey (NIS) skin tone scale is a standardized scale used to classify and assess skin tone, primarily for research on social and demographic factors. The scale ranges from 1 to 10, where each score represents a gradation of skin tone, with 1 being the lightest skin tone and 10 being the darkest. Validity is computed as Goodness of Fit Percentage estimated based on Standardized Residuals
Dataset: 100 images of people with psoriasis generated by two standard AI models (Dall-E and Midjourney). Additionally, a custom model was developed which incorporated a prompt injection aimed at �forcing� the AI (Dall-E 3) to reflect the skin tone distribution of the US population according to the 2012 American National Election Survey. This custom model generated another set of 100 images. The skin tones in these images were assessed by three researchers using the New Immigrant Survey skin tone scale, with the median value representing each image. 
Reference (2024): O'Malley A, Veenhuizen M, Ahmed A. Ensuring Appropriate Representation in Artificial Intelligence-Generated Medical Imagery: Protocol for a Methodological Approach to Address Skin Tone Bias. JMIR AI. 2024 Nov 27;3:e58275. doi: 10.2196/58275. PMID: 39602221; PMCID: PMC11635324. // was   Ensuring appropriate representation in AI-generated medical imagery: A Methodological Approach to Address Skin Tone Bias (Preprint) DOI: 10.2196/58275 URL: https://preprints.jmir.org/preprint/58275

Evaluation of Dall-E (GPT series) — Skin tone 2 (NIS scale) images -  approximately corresponds to FST I-II (lighter skin tones).
Condition/subject: Skin of color (English)
Category: Medical Records and Diagnostic Processes — Clinical Practice
Validity/accuracy: 23.39% (sample size: 96)
Comments: The range of skin tones present in the images generated by both Dall-E 3 and Midjourney are significantly different to what would be expected in the US population.  CustomGPT shows a better result. Custom GPT Model with prompt injection closely aligns with the expected demographic distribution, evidenced by a lower Chi-Square statistic (17.35) and a non-significant p-value (P = .0435), indicating minimal deviation from the target distribution. The New Immigrant Survey (NIS) skin tone scale is a standardized scale used to classify and assess skin tone, primarily for research on social and demographic factors. The scale ranges from 1 to 10, where each score represents a gradation of skin tone, with 1 being the lightest skin tone and 10 being the darkest. Validity is computed as Goodness of Fit Percentage estimated based on Standardized Residuals
Dataset: 100 images of people with psoriasis generated by two standard AI models (Dall-E and Midjourney). Additionally, a custom model was developed which incorporated a prompt injection aimed at �forcing� the AI (Dall-E 3) to reflect the skin tone distribution of the US population according to the 2012 American National Election Survey. This custom model generated another set of 100 images. The skin tones in these images were assessed by three researchers using the New Immigrant Survey skin tone scale, with the median value representing each image. 
Reference (2024): O'Malley A, Veenhuizen M, Ahmed A. Ensuring Appropriate Representation in Artificial Intelligence-Generated Medical Imagery: Protocol for a Methodological Approach to Address Skin Tone Bias. JMIR AI. 2024 Nov 27;3:e58275. doi: 10.2196/58275. PMID: 39602221; PMCID: PMC11635324. // was   Ensuring appropriate representation in AI-generated medical imagery: A Methodological Approach to Address Skin Tone Bias (Preprint) DOI: 10.2196/58275 URL: https://preprints.jmir.org/preprint/58275

Evaluation of Dall-E (GPT series) — Skin tone 4 (NIS scale) images -  approximately corresponds to FST III (medium skin tones).
Condition/subject: Skin of color (English)
Category: Medical Records and Diagnostic Processes — Clinical Practice
Validity/accuracy: 27.94% (sample size: 96)
Comments: The range of skin tones present in the images generated by both Dall-E 3 and Midjourney are significantly different to what would be expected in the US population.  CustomGPT shows a better result. Custom GPT Model with prompt injection closely aligns with the expected demographic distribution, evidenced by a lower Chi-Square statistic (17.35) and a non-significant p-value (P = .0435), indicating minimal deviation from the target distribution. The New Immigrant Survey (NIS) skin tone scale is a standardized scale used to classify and assess skin tone, primarily for research on social and demographic factors. The scale ranges from 1 to 10, where each score represents a gradation of skin tone, with 1 being the lightest skin tone and 10 being the darkest. Validity is computed as Goodness of Fit Percentage estimated based on Standardized Residuals