Venue: NeurIPS 2022 Track Datasets and Benchmarks

Reviews:

final_score: rating: 7: Good paper, accept
review_content: summary and contributions: The paper introduces SKINCON, a set of expert annotations on images in existing dermatology datasets (Fitzpatrick 17k (Groh 2021) and Diverse Dermatology Images (DDI)) (Daneshjou 2022)). Motivated by concept analysis work that aim to analyze whether a model encodes a meta-label, i.e., an annotation attached to data, but for which the model is not trained for, the dataset allows analysis of explanation of model mistakes. As one of the first meta-learning applications in medical imaging and dermatology, the dataset would be extremely valuable both to the CV/ML and the medical image community. 
strengths: - Innovative dataset for meta-label analysis in dermatology
- Thorough analysis and discussion of applications
- Paper is well-written and easy to understand

weaknesses: - Table 1: what is the skin tone for III-IV (see lines 94-107)

- A single annotator (a board-certified dermatologist) labelled images from two existing datasets. In medical data annotation, it is often the standard to have multiple annotators. It is also not clear how much experience this dermatologist has. 

- Analysis os conducted using a single model (DeepDerm with InceptionV3 backbone). 

- Only a single concept analysis method (Concept Activation Vectors (CAV)) is studied.

- Lack of visual explanations. There is no discussion of methods such as CAM or Grad-CAM, and not analysis of concept heat maps. In Figure 9 of CCE (Abid 2021), it seems like it is possible to generate heat maps and it is unclear why the authors do not discuss/show them. 

- Limited analysis of embeddings. In Section 3.2, embeddings are analyzed using a linear predictor. Given that the number of concepts is small, visuals such as colored cluster maps would be helpful to understand whether embeddings are disentangled or not

- In Section 3.2, analysis of concepts by skin tone is discussed. However, it is difficult to understand the conclusions. It would help to include quantitative results the describe concept weight change or other statistics by the Fitzpatrick skin tone and/or number of images.

correctness: The contribution of the paper is a mainly a dataset. Given the limited number of methods studied, it is unclear if any conclusions can be drawn. 

In Line 115: “we release 3230 clinical images”. This is an overstatement. These clinical images are available in a public datasets and the paper provides additional labels to them. 

clarity: Paper is well-written and easy to understand. 

relation to prior work: Paper describes prior work in densely annotated general and medical datasets fairly well. Certain concept explanation works are also described, although a dedicated analysis of meta-concept analysis is missing. It would be good to give a broad overview. Also, the following reference is missing:

Jose Oramas, Kaili Wang, Tinne Tuytelaars. Visual Explanation by Interpretation: Improving Visual Feedback Capabilities of Deep Neural Networks. ICLR 2019

additional feedback: The motivation and contributions of the paper are excellent. As described in weaknesses, the main issue is the lack of discussion to visual explanation methods and limited evaluation.
review_title: Innovative meta-label benchmark for dermatology with limited evaluation

final_score: rating: 7: Good paper, accept
review_content: summary and contributions: The authors proposed dataset consists of dermoscopic images with skin diseases, densely annotated by domain experts for developing interpretability/explainability methods and fine-grained error analysis. The images are taken from two prior publicly available skin lesion datasets, Fitzpatrick 17k and Diverse Dermatology Images (DDI).
strengths: 1. The paper is well-written, and the authors presented a clear conceptualisation associated with the paper.
2. Provided a fine clinical annotation, which is lacking in previous data settings for two existing datasets, Fitzpatrick 17k and Diverse Dermatology Images (DDI) 
3. Considering the factors for skin lesions based on their texture, shape, and colour, which doctors use for determining the type of skin lesion is used and is  an important factor in medical image analysis 
4. Datasets incorporated the concept of skin tone with the samples to reduce the effect of data bias

weaknesses: 1. Data imbalance is present across the clinical labels, which can affect the model efficacy
2. The number of dermatologists used to annotate is missing in the paper. Also, how the annotations were balanced among them? Any coefficient such as Cohen's Kappa etc. is, used?
3. How can we adhere to the ‘concept-based’ decision and say it's responsible for the particular decision? For instance, predicting a lesion as benign or malignant if it's not present in the sample, as all the mentioned  ‘concepts’ can be present in a single sample, particularly in applications 1, 2 and 3. (section 4).

correctness: The claims described in the paper appear sound.
clarity: Yes
relation to prior work: Yes.
additional feedback: Writing mistakes:
Line 136: I think positive for Pi is missing.
review_title: proposed dataset consists of dermoscopic images with skin diseases, densely annotated by domain experts for developing interpretability/explainability methods and fine-grained error analysis. 

final_score: rating: 7: Good paper, accept
review_content: summary and contributions: - the paper and dataset represents the work of manually labeling 3,866 images (from two separate datasets) for 48 concepts (> 180k total feature annotations), together with potential applications of informing users of conventional (binary) classification as to whether or not specific features, if detected, would alter the classification (predicted label)
- this is an interesting and useful contribution to the literature that helps readers become aware of the kinds of mistakes (dermatological) AI models can make, and how to inquire into those mistakes

strengths: - the authors clearly demonstrate the benefit of collecting and using "within-image" (meta/feature) labels to further examine and inspect AI model mis-classification cases
- exhaustively labeling this number of images for 48 potential features must have been an incredibly time consuming effort, and I commend the authors for this undertaking!
- providing the dermatology community with insights into how and why models may make mistakes is of critical importance to reduce misdiagnoses, and the authors' work makes in important contribution in that direction

weaknesses: Most importantly, the annotations are incredibly skewed and the distribution fairly dissimilar across the two datasets, warranting some caution as to how anything "learned" from these specific labels translates to a population of images used in a different setting/context

Minor points:
- section 2.4/tables: it would be great if the authors could provide a table of the 48 labels with the frequency (absolute or fraction) of images in which the label was annotated, to give readers a sense of the density (and likely reliability) of each of the 48 labels; this would also reveal a discrepancy between the two datasets (e.g., papule is labeled in 36.2% of images in one, but 64.5% of images in the second dataset), putting into question the influence of baserates (distributional properties) during training
- the paper is missing an evaluation of how clinicians would utilize the information presented by such methods -- e.g., given an AI prediction class label and the additional information of, say, a PCBM output, how does this affect clinician reliance on/trust in the AI predicted label?
- table 2 suggests a whopping improvement in performance specifically for FST III and IV; I would like to see some explanation as to why that might be the case (possibly the concepts with the densest annotations occur most frequently in those skin types?); this might also lead to a limitation of the presented dataset as primarily benefiting AI aided diagnoses in certain populations (FST III and IV)?

correctness: The overall dataset construction (two CSV files in the main) seems sound and easily usable across softwares (e.g. python/pandas, etc.).

Minor points (please correct):
- line 64 and table 1 describe 656 DDI images, whereas the downloaded CSV file (from https://skincon-dataset.github.io/index.html only contains 636 entries)
- line 65 describes 28 concepts with 50+ images; combining the two CSV files, my count is 25 concepts with 50+ images

clarity: Overall, the paper is well written, with some minor points here:

- line 63: why was the threshold of 50 chosen (as opposed to, say, 20; at which 13 additional concepts would have qualified)?
- lines 140-143: while I appreciate the difficulty summarizing the referenced work (Abid et al., 2021), I believe it would be beneficial for the reader to appreciate how the SVM output (of samples on which the classifier was not trained) is used to derive the weights; if, for instance, a sample image's embeddings (latent variables) produce a projection into one of the two clusters (P for positive and N for negative training samples), with a distance relating to c (SVM parameter for separability of the hyperplane), how does this value then relate to the weight given a specific concept?
- as much as the "counterfactual hypothetical" (if one were to add a concept to an image, the label would be correct) is useful, it does ultimately not explain why the model *missed* the concept
- instead of phrasing as "adding the concept to the image would help fix the model mistake", I would suggest "would increase the likelihood of the model assigning the opposite label" (in practice, the ground truth is often not known, and it is important for clinicians to appreciate the aspect of a model "missing" a concept in an image, and that if the model had detected the concept, it *might* have assigned the opposite label)
- what then would be important to know is maybe some indicator of *how likely* (for each missed label) it would be that the model would alter the predicted label (does a weight of 1, for instance, mean a shift by 1 unit log-odds?)
- lines 152 following: while the term "post-hoc concept bottleneck models" appears (as a whole) in the caption of table 2, it is (in the full text) never linked directly to the PCBM acronym; it might be useful to do so in either line 160 or 154 -- by adding the word "model" to the sentence, and then adding (PCBM) in parentheses.

relation to prior work: The article discusses relevant prior work. No concerns here.
additional feedback: - lines 110-113: does that mean that the core concept annotations (for the whole dataset) are from a single dermatologist expert? while the validation (of 10% of images) reached a very high agreement (>95%), I wonder to what extent this might also be due to the validating expert having received similar/identical training to the original annotating expert, and how a group of experts might reveal higher (or similar) concordance?
- there are a few typos throughout the manuscript (in particular, section 5.2 contains several superfluous words)
review_title: Dense (exhaustive) 48-feature clinical dermatology annotations in two image datasets, important contribution to the literature

final_score: rating: 8: Top 50% of accepted papers, clear accept
review_content: summary and contributions: Authors present dense annotations for two different skin image datasets, aiming to increase model interpretability of AI models. The dataset is composed of skin images taken from the well-known Fitzpatrick 17K dataset (3230 images) and Diverse Dermatology Images (DDI) dataset (656 images). The photos are densely annotated for 48 different (predefined) concepts (an image may take on more than one concept) defined by two dermatologists. 
The authors also illustrate potential applications that can utilize such dense annotations. They used the dataset to understand better the mistakes of an existing dermatology image classification model and interpret the same classification model via post-hoc concept bottlenecks.
strengths: The interest in adopting dermatology AI models has increased in the last few years. Unfortunately, most of the developed models are black box models, where understanding the fail cases and solving them is very hard. SKINCON dataset can help  users/developers/clinicians in understanding the behaviour of these black box models as well as improve their performance. 

For efficient training of the AI models using the concept labels, the authors used methods developed for another applications by the co-authors. This shows that the authors have full control over the data. 

The images in the dataset represent different skin colors in a balanced manner. 

Beyond the dataset, the provided applications to test the dataset are well choosen

weaknesses: It is not clear how are the concepts decided. The authors mentioned that the concepts are based on clinical descriptor terms used to describe skin lesions. However, do the authors think, the opinion of two dermatologists is enough for consensus. 

The labeling is done by a reader and then finetuned (corrected) by an expert dermatologist. Again, do the authors think labeling by two readers is reliable enough? 

How do the authors ensure that the proposed concepts are generalizable to a larger cohort of dermatologists?

Concept labels are not available neither at the github site or the supplementary zip file (or at least they were not easily available so the reviewer missed them)
correctness: Besides the clarification needs mentioned in the weakness section, the dataset is constructed in a sound way 
The presented benchmark and evaluation methods sound correct too
clarity: The presentation of the paper is good. 
relation to prior work: Yes
additional feedback: NA
review_title: A densely annotated skin image dataset to achieve model interpretability

final_score: rating: 6: Marginally above acceptance threshold
review_content: summary and contributions: This paper introduces SKINCON, a densely annotated dermatological dataset which includes annotations of 48 clinical concepts performed by one dermatologist for 3886 images. Another dermatologist reviewed less than 10% of these annotations, with high levels of agreement between the two raters. The authors demonstrate SKINCON’s value by using it on two tasks: model mistake explanation and model interpretability. Four more possible applications are proposed.
strengths: - The dataset contains interesting, densely annotated data that will increase the explainability and debugging capabilities of skin cancer diagnosis models.
- The images in the dataset cover a variety of skin tones.
- A list of use-cases for this dataset is demonstrated and prototyped, acting as guidelines for future research.

weaknesses: - The methodology offers little insight into the levels of noise present in the data:
    - No intra-rater analysis has been performed, thus making it difficult to ascertain the level of noise present in this dataset.
    - While the authors have performed a very limited validation of the label correctness, the review process cannot be interpreted as inter-rater agreement. A correct inter-rater agreement analysis would require the raters to be blinded to each other’s evaluations in order to avoid bias. The current validation setup is likely due to overstate the level of agreement between the two raters.
- More information about the labelling protocol should be provided to allow for reproducing this work and to better understand the labels:
    - No information is given about the level of education and expertise of the original labeller and of the reviewer.
    - How were the 48 clinical concepts chosen?
    - The caption in Figure 1 states that the labellers were asked to consider 48 concepts, but Figure 1a) shows only 11 concepts. Could you explain this discrepancy?
    - Figure 1a) shows that labellers had an option to discard images by selecting “Do not consider this image”. How often did this happen? Why were images being discarded?
- In the abstract, the authors claim that this is the first medical dataset annotated by domain experts. However, in Section 5.2 they present several other medical datasets with additional or dense annotations, including in dermatology. Moreover, two similarly densely annotated dermatological datasets are not included in the related work: please refer to [DermX](https://openreview.net/forum?id=wBOBL5b-aa9&noteId=EeXtzTjYNBi) and [DermIS](https://www.dermis.net/dermisroot/en/home/index.htm) for more details.
- The explanation mechanism suggested in Section 3.1 and the interpretability method presented in Section 3.2 require that the models perform reasonably well at detecting the clinical concepts present in the image. Could you report this performance to prove that this dataset can indeed be used in the proposed manner? Without a high detection performance, the explanations and the interpretations may not be valid from a domain perspective.
- No information is given about how the validation set was selected.
- While the Fitzpatrick skin tone distribution is discussed, more demographics metadata (e.g. age, sex) is needed to understand whether the dataset is balanced over the demographics.

correctness: The dataset construction and the use-cases seem mostly correct, although more information about the labelling protocol is needed.
clarity: The paper is well written, with a few typos and undefined abbreviations.
relation to prior work: While the authors describe the differences between their work and densely annotated datasets from other medical fields, two densely annotated dermatological datasets with similar annotations are not mentioned or discussed.
additional feedback: - For ease of reading, I would recommend moving Table 1 (or an abridged version of Table 1) to the main body of the paper.
- The caption for Figure 1b) may be slightly confusing: I suggest changing “we let them mark the ones they disagree with” to “we let them mark the ones they agree with”, so that the caption matches the phrasing depicted in the image.
- Figure 2b) shows the distribution only for the Fitzpatrick17k subset. Could you add a similar figure for the DDI data?
- Several abbreviations are used without having been introduced a priori:
    - L77: FST
    - L160: pCBM
    - L195: tCAV
    - L199: CBMs
- Typos:
    - L11-12: the double usage of “to provide” is redundant
    - L128: redundant “of the”, and an uncapitalized “deepderm”.
    - L211: capitalize the x in “x-rays”
    - L246: the link is missing a /
    - L259: "moddeels"
    - L283: “their terms of use anc collection”
- Could you mention how many images was the PCBM linear predictor trained on (L160-161)? 
- Could you mention the exact number of concepts that were labelled in tCAV (L195)?
review_title: Interesting dataset with too little information about the labelling protocol and the levels of noise

Meta-review: metareview: This paper provides an augmented dataset with additional annotations, intended to enhance interpretability and meta analysis of images for skin disease classification. 

Pros:
i) The work is well motivated, justified and executed

ii) The dense annotation will significantly help test ML models against this dataset.

Cons:
i) Variability of annotations themselves can determine generalizability of the learned models and subsequent interpretations

ii) Currently it appears that few models have been test for analysis on the dataset.

Nonetheless there seems to be a consensus among reviewers that this dataset is indeed valuable. Hence I am recommending an accept.

confidence: 3: The area chair is somewhat confident
Decision: Accept