DermaLens — Explainable Skin Condition Detector

Working software implementation of the research proposal "Explainable Deep Learning-Based Skin Condition Detection Using Smartphone Images" (S. Faththima Simra).

It detects common skin conditions — insect bites and rashes (plus a normal-skin baseline) — from ordinary smartphone photos using a CNN trained with transfer learning, and explains every prediction with a Grad-CAM heatmap showing exactly which skin regions drove the result.

⚠️ For early awareness only — not a medical diagnosis. This is a research prototype and cannot replace examination by a qualified clinician.

---

What's implemented (mapped to the proposal)

Proposal section	Implementation
6.1 Dataset	ImageFolder layout under sample_data/ for SCIN / Fitzpatrick17k / custom images
6.2 Preprocessing	Resize, normalise, augmentation (rotation, flip, scale, jitter) — train.py
6.3 Model development	MobileNetV2 / ResNet18 transfer learning + Softmax head — model.py, train.py
6.4 Explainability	Grad-CAM from scratch (forward/backward hooks) — model.py
6.5 System implementation	Web app: upload → predict → confidence + Grad-CAM — app.py, frontend/
4.2 Deliverables	Trained model, web app, confidence scores, evaluation report (evaluate.py)

Architecture

Smartphone photo
      │
      ▼
[ Frontend  (frontend/index.html) ]      drag-drop upload, results UI
      │  POST /api/predict (multipart)
      ▼
[ Backend   (FastAPI, backend/app.py) ]
      │
      ├─ Preprocess (resize 224², normalise)        inference.py
      ├─ CNN forward pass  → Softmax confidences     model.py  (MobileNetV2)
      └─ Grad-CAM backward pass → heatmap overlay    model.py  (GradCAM)
      │
      ▼
JSON: prediction, per-class confidence, Grad-CAM image

Backend note: the proposal lists Spring Boot. Because the model runs in Python (PyTorch), the API is hosted in the same Python process with FastAPI to avoid a second runtime and an extra network hop. A Spring Boot layer can be added later simply by proxying to these endpoints.

Evaluation results

Trained with transfer learning (MobileNetV2) on the SCIN smartphone-image dataset, mapped to the three project classes. Training used partial fine-tuning, a class-balanced sampler, an LR scheduler, and early stopping. Evaluated on a held-out validation split (cases split by case_id, so no image leakage).

Per-class metrics (validation)

Class	Precision	Recall	F1	Support
insect_bite	0.165	0.360	0.226	100
rash	0.910	0.793	0.847	1468
normal_skin	0.062	0.120	0.081	75
Accuracy			0.736	1643
Macro avg	0.379	0.424	0.385	1643
Weighted avg	0.826	0.736	0.775	1643

Interpretation

• Headline accuracy (0.736) is inflated by class imbalance — the validation set is ~89% rash, so accuracy rewards a rash-leaning model. The fair metric for this imbalanced problem is macro-F1 = 0.385.
• rash is detected well (F1 0.85) — it has thousands of training images.
• insect_bite (F1 0.23) and normal_skin (F1 0.08) underperform because SCIN provides only a few hundred labelled images for each, and they overlap visually with the highly heterogeneous rash class.
• This directly illustrates the proposal's core thesis: the clinical-to- consumer domain gap and the data-scarcity / class-imbalance challenge of screening real smartphone photos. Stronger minority-class performance would require additional labelled insect_bite / normal_skin images, which are scarce in public datasets.

Project layout

skin-condition-detector/
├── README.md
├── requirements.txt
├── backend/
│   ├── config.py        # classes, paths, image settings (edit classes here)
│   ├── model.py         # transfer-learning CNN + Grad-CAM
│   ├── inference.py     # preprocess → predict → Grad-CAM overlay
│   ├── train.py         # train/fine-tune on your dataset
│   ├── evaluate.py      # accuracy, confusion matrix, per-class metrics
│   ├── demo.py          # CLI smoke test (no server needed)
│   └── app.py           # FastAPI server + API
├── frontend/
│   └── index.html       # single-page web UI
├── models/              # trained weights + evaluation report land here
└── sample_data/         # your train/ and val/ image folders

Quick start

1. Install

cd skin-condition-detector
python -m venv venv && source venv/bin/activate      # optional
pip install -r requirements.txt

2. Run the web app (works immediately in demo mode)

cd backend
uvicorn app:app --reload --port 8000

Open http://localhost:8000, upload a photo, and view the prediction + Grad-CAM. Until you train a model, it runs in demo mode (an untrained classifier head, so predictions are not yet meaningful — the UI says so).

3. Train on real data

Populate sample_data/train/ and sample_data/val/ (see sample_data/README.md), then:

cd backend
python train.py --data ../sample_data --epochs 15
# faster option when you have limited data:
python train.py --data ../sample_data --epochs 10 --freeze-backbone

Weights are saved to models/skin_model.pt. Restart the server and it automatically serves the trained model.

4. Evaluate (produces the evaluation-report deliverable)

cd backend
python evaluate.py --data ../sample_data --split val
# writes models/evaluation_report.txt (accuracy, confusion matrix, P/R/F1)

CLI smoke test (no browser)

cd backend
python demo.py path/to/photo.jpg      # or no argument for a synthetic test image

Configuration

Everything adjustable lives in backend/config.py:

• CLASS_NAMES — the conditions to classify (must match your data folders).
• BACKBONE — "mobilenet_v2" (light) or "resnet18" (heavier).
• IMAGE_SIZE, normalisation, low-confidence threshold.

Notes & limitations

• The CNN backbone downloads ImageNet-pretrained weights on first run (needs internet). Offline, it falls back to random initialisation automatically.
• Real accuracy requires training on a properly labelled dataset; the included pipeline is what produces meaningful results.
• This tool is for screening / awareness only and is not a medical device.