Paper:
Optimal Ridge Detection using Coverage Risk
We introduce the concept of coverage risk as an error measure for density ridge estimation. The coverage risk generalizes the mean integrated square error to set estimation. We propose two risk estimators for the coverage risk and we show that we can select tuning parameters by minimizing the estimated risk. We study the rate of convergence for coverage risk and prove consistency of the risk estimators. We apply our method to three simulated datasets and to cosmology data. In all the examples, the proposed method successfully recover the underlying density structure.

Review:
The paper focuses on density ridge estimation. The authors propose the use of coverage risk which is a generalization of the mean integrated expected error. The authors propose two estimators (based on either smoothed bootstrap or data splitting), prove their consistency, and study convergence rates. Particularly nice is that, unlike some of the cited works, the proposed method does not suffer from the problems related to the notorious estimation of the smoothing parameter.Experiments are done on simulated data and Cosmic Web data.   Quality: The paper is technically sound, and its overall quality is high. The problem attacked is interesting and of importance.Clarity: The paper is well written and clear.Originality: As far as I am aware of, the idea presented is new. Significance: I believe this work will have a nice impact.  A remark on the experiments: In both Figures 3 and 4, the authors show under-smoothing and over-smoothing results next to theirs. That's fine, but what is really missing here is a comparison with what ones gets when one uses simple cross validation for the optimal smoothing parameter for standard Kernel Density Estimation (e.g., using a Gaussian kernel) and then does the ridge extraction after the fact. Presumably, according to the authors' claims this would be inferior to their method which finds the optimal smoothing parameter wrt the task at hand (ridge detection) as opposed to the optimal parameter for general-purpose KDE. Such an experimental result, whose absence is disappointing, would have strengthened the authors' main claim about the utility of the method and would have also convinced me to give their paper a higher grade.  A remark regarding principal curves:The authors may want to also cite Hastie's work from the 80's.  ----------------------------- ----------------------------- Having read the rebuttal, my impression remained positive. I keep my score as it is. That said, I am sightly concerned about the wording used in the rebuttal regarding my remark on cross validation: while they committed themselves in the rebuttal to add the results obtained using ordinary CV, they avoided stating whether this comparison indeed makes their method look favorable. I hope this is just a poor choice of wording and nothing more. It will be very disappointing to find out that the results are actually the same.