DOI: 10.5281/zenodo.14201692
title: IMA++: ISIC Archive Multi-Annotator Dermoscopic Skin Lesion Segmentation Dataset

description: Overview

The IMA++ dataset is the largest publicly available multi-annotator skin lesion segmentation (SLS) dataset, collected from the ISIC Archive to facilitate skin lesion image segmentation research. It contains 17,684 segmentation masks spanning 14,967 dermoscopic images, where 2,394 dermoscopic images have 2-5 segmentations per image from the ISIC Archive, annotated by 16 distinct annotators, with at least one annotation per image. The dataset captures a wide range of segmentation styles influenced by annotator expertise, tools used, and manual review processes, making it a valuable resource for developing and evaluating SLS models.

Files



segs.zip: A ZIP archive of the 22,472 segmentation masks. This includes 17,684 segmentation masks and the output of two consensus algorithms, bringing the number of segmentations up to 22, 472.

seg_metadata.csv: A CSV file containing the metadata for all the 22,472 segmentation masks.

img_metadata.csv: A CSV file containing the metadata for all the 14,967 skin lesion images.

seg_metadata_multiannotator_subset.csv: A CSV file containing the metadata for only the multi-annotator subset of IMA++ (i.e., 2,394 images with multiple segmentations per image).

iaa_metrics_pairwise.csv: A CSV file with the IAA metrics calculated for all mask pairs for all the images in the multi-annotator subset of IMA++.

iaa_metrics_image.csv: A CSV file with the pairwise IAA metrics averaged per image.

splits/{train,val,test}.csv: Standardized {training, validation, testing} partitions for the multi-annotator subset of IMA++.


Note that the 14,967 skin lesion images from the ISIC Archive corresponding to these segmentation masks are not included in this repository, and they can be obtained from the ISIC Archive. You can use the ISIC API to download images by their ISIC ID using the code in this repository.

Key Features





Inter-Annotator Variability: The dataset captures a wide range of segmentation styles, reflecting differences in annotator expertise, tools used (T1, T2, T3), and manual review processes (expert or novice).




Realistic multi-annotator scenario: Unlike most medical image segmentation datasets where every image is segmented by every annotator (i.e., a complete bipartite graph between annotators and images), IMA++ features an incomplete bipartite graph (i.e., every image is segmented by at least one annotator, but not all images are segmented by all annotators). This setup simulates real-world annotation scenarios where multiple annotators contribute to a subset of images.




Large-Scale Multi-Annotator Data: With 17,684 image-mask pairs from 16 annotators, IMA++ is the largest publicly available multi-annotator skin lesion segmentation dataset, enabling robust analysis of annotator preferences and segmentation styles.


Tool-Specific Segmentation Styles: The dataset allows for the exploration of tool-specific segmentation styles, as demonstrated in the segmentation style discovery paper, where differences among the three annotation tools (T1, T2, T3) were learned and captured by the segmentation model.


Citation

If you use the IMA++ dataset in your research, please cite the following papers:






Abhishek, K., Kawahara, J., Hamarneh, G. (2025). Segmentation Style Discovery: Application to Skin Lesion Images. In: Medical Image Computing and Computer-Assisted Intervention (MICCAI) ISIC Skin Image Analysis Workshop (MICCAI ISIC). MICCAI 2024. Lecture Notes in Computer Science, vol 15274, pages 24–34. Springer, Cham. https://doi.org/10.1007/978-3-031-77610-6_3

Abhishek, K., Kawahara, J., Hamarneh, G. (2025). What Can We Learn from Inter-Annotator Variability in Skin Lesion Segmentation?. In: Medical Image Computing and Computer-Assisted Intervention (MICCAI) ISIC Skin Image Analysis Workshop (MICCAI ISIC). MICCAI 2025. Lecture Notes in Computer Science, vol 16149, pages 23–33. Springer, Cham. https://doi.org/10.1007/978-3-032-05825-6_3





 

The BibTeX entries for these papers are:

@InProceedings{abhishek2024segmentation,
    author = {Abhishek, Kumar and Kawahara, Jeremy and Hamarneh, Ghassan},
    title = {Segmentation Style Discovery: Application to Skin Lesion Images},
    booktitle = {Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI) ISIC Skin Image Analysis Workshop},
    pages = {24--34}
    year = {2025},
    doi = {https://doi.org/10.1007/978-3-031-77610-6_3},
    url = {https://link.springer.com/chapter/10.1007/978-3-031-77610-6_3},
    publisher = {Springer Nature Switzerland},
    address = {Cham},
    isbn = {9783031776106}
}@InProceedings{abhishek2024segmentation,
    author = {Abhishek, Kumar and Kawahara, Jeremy and Hamarneh, Ghassan},
    title = {What Can We Learn from Inter-Annotator Variability in Skin Lesion Segmentation?},
    booktitle = {Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI) ISIC Skin Image Analysis Workshop},
    pages = {23--33}
    year = {2025},
    doi = {https://doi.org/10.1007/978-3-032-05825-6_3},
    url = {https://link.springer.com/chapter/10.1007/978-3-032-05825-6_3},
    publisher = {Springer Nature Switzerland},
    address = {Cham},
    isbn = {9783032058256}
}

License

The IMA++ dataset is made available under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0) license. Additionally, since these annotations are collected from the ISIC Archive, by using these annotations, you acknowledge and confirm your compliance with their Terms of Use.