Hooge et al. ( 2018 ) asked the question: "Is human classification by experienced untrained observers a gold standard in fixation detection?" They conclude the answer is no. I have had a close look at their data and their report, and I find that both the data itself and the method of presentation of the data to the human raters  are problematic. I think that data used to address this important question should be very high quality, and the method of presentation of the data should be optimized. The Hooge et al . (2018) study does not meet this standard. A number of fixations have clear saccades within them. No expert human rater would intentionally include such saccades in fixations. Something has gone wrong . Also, the treatment of some inter-fixation intervals as saccades that might have been identified by human experts is completely unjustified. The results regarding these saccades are without merit or meaning.
Hooge et al. 2018 asked the question: "Is human classification by experienced untrained observers a gold standard in fixation detection?" They conclude the answer is no. I have had a close look at their data and their report, and I find that both the data itself , the method of presentation of the data to the human raters , and the analysis of agreement statistics are problematic. I think that data used to address this important question should be very high quality, and the method of presentation of the data should be optimized. Alternatively, the title should be changed to: "Is human classification by experienced untrained observers a gold standard in fixation detection when data quality is very poor?" Numerous examples of various types of low quality data are presented herein. In addition, it appears that, by including missing data in the data submitted for assessment of agreement statistics, Hooge et al (2018) have made a serious mistake . Also, the treatment of some inter-fixation intervals as saccades that might have been identified by human experts is completely unjustified. The results regarding these saccades are without merit or meaning.
D
(
D
)
R
and
,
A
, and the analysis of agreement statistics
R
The
Alternatively, the title should be changed to: "Is human classification by experienced untrained observers a gold standard in fixation detection when data quality is very poor?" Numerous examples of various types of low quality data are presented herein. In addition, it appears that, by including missing data in the data submitted for assessment of agreement statistics,
D
.
R
study does not meet this standard. A number of fixations have clear saccades within them. No expert human rater would intentionally include such saccades in fixations. Something has gone wrong
have made a serious mistake