Venue: NeurIPS 2019 Workshop Neuro_AI

Reviews:

final_score: rating: null
review_content: importance_comment: The authors raise a very important question – how well do brain parcellations map onto functional units? How we choose to discretize data has incredibly important ramifications for the conclusions we draw, and the authors recognize this importance. However, while the authors clearly evidence the importance of their question, it was unclear how important their specifics results were. There was no clear take-away from the article, thus making it difficult to gauge the overall importance.


rigor_comment: The authors did not provide much detail on their results. I appreciated their outline of the ways in which they empirically investigated atlas utility, but I could not follow their results and thus could not adequately interpret the rigor. The figures were not described or interpreted, and I wasn't sure what the axes corresponded to. I am also not familiar with all of these functional atlases, and thus not sure how they differ beyond the extraction method. 

clarity_comment: The introduction described the general premise quite well. However, this section took up a majority of the paper. The results section made sense in terms of grammar, but the lack of detail made it slightly difficult to follow, and made the article as a whole unclear to me in terms of take-home message and scientific impact. 

There is a typo on line 17: yet it is also strongly associated.
intersection_comment: While some ML techniques were used in the extraction and decoding, it seemed relatively light on the AI front. 
comment: Overall, the general question that the authors investigate is very interesting and important. However, the specific scientific contribution of this paper is unclear to me.
review_title: Interesting question, but execution needs work

final_score: rating: null
review_content: importance_comment: The goal of the study is important and interesting: to assign quantitative criteria to the quality of a given brain parcellation. Interesting and clear discussion of the issues involved.  Went a little long and left little room for results. One major conclusion is that AAL does not perform as well as other parcellations methods but does not make quantitative comparison.


rigor_comment: Nice choice of quantitative criteria for parcellation and the carrying out of comparisons is convincing.

No apparent statistical tests of differences between the performance of methods. Unclear what error bars are (standard error, 95% CI, quantiles) so it is unclear whether differences large by eye.

In figure 1 c it is interesting to see for the same, I assume number regions, there is difference in performance. Really gets at the nice point that some parcellations may be averaging over unrelated regions. Some accounting for degrees of freedom through a held out data set would have been nice or if there was some clear mention of it would have been good.
clarity_comment: Introduction was clear and nicely discussed issues of parcellation philosophical and quantitative.
Wasn’t clear what x axis in Figure 1 referred to. I assumed: dimension=number brain regions?
Wasn’t clear what individual points were on Figure 1 subplots (individual subjects?).
What error bars corresponded to was not mentioned (SE, 95% CI).
intersection_comment: Could not find explicit discussion of relation of brain parcellation to AI or vice-versa. The extent of relation as far as I can tell is that machine learning techniques are used in the parcellation. Some discussion to point out why the connection is deeper would have been good.
comment: Would have been nice if there was more focus on results and firm conclusions then introduction to the topic, though the introduction was nice. Interesection to AI was never made explicit or discussed.
review_title: Interesting topic and discussion, clarity could be improved and results emphasized.

final_score: rating: null
review_content: importance_comment: Validating the choice of the parcellations that we use for obtaining and interpreting our results in neuroimaging data analysis is an important problem. There are many recent and ongoing efforts in this direction. The analyses presented here do not further clarify the matter, apart from "it's complicated", and more importantly do not answer the (perhaps over-ambitious) question which makes the title.
rigor_comment: The results appear to be robust. What is less clear is their motivation and interpretation.
clarity_comment: The paper is generally readable, even though a discontinuity is felt going from the very general and ambitious objectives, to the very specific analyses without a particular comment, to the again general conclusions.

There is a typo at the end: "functionnal"
intersection_comment: The application is to neuroimaging data, and some of the indicators (decoding performance and classification) are related to AI research.
comment: Maybe this work could be better presented in a more extended study, explicitly focused on some technical aspects, clearly referencing previous studies and novel results.
review_title: This paper does not elucidate what's in a functional brain parcellation

Decision: Accept (Poster)