CanoeBoat/epivascan-skin

EpivaScan skin-condition classifier

Given one photo of a skin concern, this model returns the conditions the photo most closely matches, each with a percentage. It is the image classifier behind EpivaScan.

This is a screening aid in development. It is not a medical device, it does not give a diagnosis, and it has not been validated in a clinic or in the field. A percentage is how strongly the photo resembles a condition, not the chance that a person has it.

Conditions covered

Condition	Cases: train	val	test
Eczema (dermatitis)	313	71	72
Contact dermatitis	328	69	72
Hives (urticaria)	144	36	33
Insect bite	134	26	28
Folliculitis	95	21	18
Psoriasis	81	16	15
Tinea (ringworm)	57	15	15
Cold sore (herpes simplex)	48	12	12
Shingles (herpes zoster)	56	10	9
Acne	37	9	8
Impetigo	61	7	8
Pityriasis rosea	41	6	9
Drug rash	34	9	6
Other / unclear	704	150	153

Other / unclear is everything else. The model can only name the conditions above, so a condition it was not trained on lands here or on a look-alike. In the training data it is mostly: Pigmented purpuric eruption, Leukocytoclastic Vasculitis, Hypersensitivity, O/E - ecchymoses present, Tinea versicolor, Lichen Simplex Chronicus, Lichen planus/lichenoid eruption, Keratosis pilaris, Rosacea, Molluscum contagiosum, Viral Exanthem, Stasis Dermatitis.

Training data

• Source: google/scin, the Skin Condition Image Network: photos contributed with consent by internet users in the United States, labelled by dermatologists. Released under the SCIN Data Use License.
• Used here: 3,048 cases, 6,494 photos. Cases the dermatologists could not grade are left out.
• Labels: each case has a weighted differential, several conditions with weights that sum to 1. The model is trained on the whole distribution.
• Split: 70/15/15 by case, so photos of one person never sit on both sides, stratified by condition and skin-tone group.

Results on the held-out test set

458 cases, 1,002 photos, never used for training or tuning. A match counts as right when it is the dermatologists' leading condition, or one of them when two are tied. Brackets are 95% intervals.

Measure	Result
Top match is right	47% (43-51)
Right condition in the top 3	85% (82-88)
Average over conditions (balanced accuracy)	51%
Calibration gap (ECE)	6.4 points

Accuracy for each condition

Condition	Test cases	Top match right	In top 3	When named, how often right
Eczema (dermatitis)	72	30% (22-39)	82% (74-89)	52%
Contact dermatitis	72	62% (51-72)	92% (87-97)	42%
Hives (urticaria)	33	57% (43-71)	71% (55-85)	74%
Insect bite	28	46% (29-66)	69% (52-85)	38%
Folliculitis	18	43% (25-61)	76% (59-90)	31%
Psoriasis	15	36% (19-53)	78% (56-95)	29%
Tinea (ringworm)	15	41% (21-66)	69% (44-93)	37%
Cold sore (herpes simplex)	12	50% (28-65)	96% (83-100)	44%
Shingles (herpes zoster)	9	78% (56-100)	89% (74-100)	38%
Acne	8	47% (24-75)	82% (62-100)	22%
Impetigo	8	59% (27-88)	86% (72-96)	14%
Pityriasis rosea	9	67% (36-91)	83% (60-100)	47%
Drug rash	6	50% (30-69)	93% (79-100)	23%
Other / unclear	153	44% (37-51)	89% (85-93)	68%

Performance by skin tone

Group	Test cases	Top match right	In top 3
Fitzpatrick I-II	197	39% (33-44)	80% (75-85)
Fitzpatrick III-IV	206	53% (47-58)	88% (84-92)
Fitzpatrick V-VI	41	59% (46-72)	89% (79-97)
Monk 1-3	326	47% (42-52)	85% (81-88)
Monk 4-6	124	45% (38-53)	84% (79-89)
Monk 7-10	8	68% (41-92)	95% (86-100)

Fitzpatrick type is the dermatologists' estimate from the photos (self-reported where there is none); Monk tone is the mean of two trained raters.

Do the percentages mean what they say?

After training, one temperature (0.94) was fitted on the validation set and folded into the final layer, so the model's own output is calibrated.

Known limitations

• It can only name the 13 conditions listed. Anything else is reported as Other / unclear or mistaken for a look-alike.
• Darker skin is under-represented: the test set has 41 Fitzpatrick V-VI cases, too few to show the model works as well there.
• These conditions have fewer than 30 test cases, so their numbers are rough: Insect bite, Folliculitis, Psoriasis, Tinea (ringworm), Cold sore (herpes simplex), Shingles (herpes zoster), Acne, Impetigo, Pityriasis rosea, Drug rash.
• All contributors were adults. It is untested on children.
• The photos came from internet users in the United States. Conditions common elsewhere, and photos taken in other settings, may be handled poorly.
• It sees one photo. It cannot feel texture, take a history or look at the rest of the skin.
• The reference labels are dermatologists' opinions from photos, not confirmed diagnoses.

Use

from transformers import pipeline

scan = pipeline("image-classification", model="CanoeBoat/epivascan-skin")
scan("photo.jpg", top_k=3)
# [{'label': '...', 'score': 0.64}, ...]

Training details

• Base model: facebook/dinov2-base, fine-tuned end to end
• Input: 448 px, shorter side resized then centre-cropped
• 30 epochs at most, batch 32, AdamW, learning rate 2e-05 (head 0.001), layer decay 0.75, cosine schedule, mixup 0.2, seed 17
• Checkpoint chosen by validation balanced accuracy: 50% (top match 43%, top 3 81%)

Licence

Released under the SCIN Data Use License, the licence of the photos and labels it was trained on. Anyone sharing this model, or something built from it, keeps this notice and the link to SCIN, and must not try to re-identify the people in the data. The base model, facebook/dinov2-base, has its own licence (Apache-2.0 for DINOv2).

Citation

Ward A, Li J, Wang J, et al. Creating an Empirical Dermatology Dataset Through Crowdsourcing With Web Search Advertisements. JAMA Network Open. 2024;7(11):e2446615.