0xAgamy/unsloth_Qwen3vl_finetune

Qwen3-VL-2B-Instruct Skin Lesion Extractor

This model is a specialized vision-language model fine-tuned from Qwen3-VL-2B-Instruct using Unsloth for efficient training. It is designed to analyze dermatological images (skin lesions) and extract structured, clinical features in a strict JSON format.

Model Description

This model takes an image of a skin lesion and a textual prompt as input, and outputs a structured JSON object containing key dermatological features. It was fine-tuned on a curated medical dataset of skin extracts to recognize and describe:

• Lesion Type: The clinical classification of the lesion (e.g., melanoma, nevus, basal cell carcinoma).
• Color: Predominant colors observed (e.g., brown, black, red, white, blue).
• Symmetry: Assessment of the lesion's symmetry (e.g., symmetric, asymmetric).
• Borders: Description of the lesion's edges (e.g., regular, irregular, scalloped).
• Texture: Surface characteristics (e.g., smooth, rough, scaly, ulcerated).
• Summary: A concise, professional clinical summary of the findings.

Training Details

• Base Model: Qwen/Qwen3-VL-2B-Instruct
• Training Framework: Unsloth (enabling up to 2x faster training and 50% less memory usage via optimized LoRA/QLoRA).
• Dataset: Skin_Extract
• Optimization: 4-bit quantization (QLoRA) with rank r=16, alpha 32, and dropout 0.05 (adjust these values to match your actual Unsloth config).

Evaluation Metrics

The model's extraction accuracy was evaluated using BERTScore to measure the semantic similarity between the model's generated JSON fields and the clinical ground truth annotations.

Below are the detailed BERTScore (F1) statistics for each extracted feature:

Feature Field	Mean Score	Min Score	Max Score	Std Dev
Texture	0.9449	0.8409	1.0000	0.0670
Lesion Type	0.9361	0.8490	1.0000	0.0378
Borders	0.9108	0.8433	1.0000	0.0377
Symmetry	0.9092	0.8397	1.0000	0.0417
Summary	0.9087	0.8498	0.9672	0.0227
Color	0.8972	0.8315	0.9647	0.0245
Overall Average	~0.9178	-	-	-

Note: The high mean scores (>0.89 across all fields) and low standard deviations indicate that the model consistently generates descriptions that are semantically highly aligned with the clinical ground truth, with minimal variance.

Limitations and Bias

• Not a Diagnostic Tool: The model can hallucinate or misclassify rare conditions. Human oversight is strictly required.
• Dataset Bias: Performance may vary depending on skin tones, image quality, lighting conditions, and the specific demographics represented in the training dataset.
• Scope: The model is strictly tuned for cutaneous/dermatological images and will perform poorly or unpredictably on non-skin medical imagery (e.g., X-rays, MRIs).

---

Developed by: 0xAgamy