Before revision:  We verify thataccuracy scores may be inflated up to 41\% , even if a well-designed 10x5 iterated cross-validation DAP is applied, unless all images from the same subject are kept together either in the internal training or validation splits. Results are replicated for 4 classification tasks in digital pathology on 3  datasets, for a total of 373 subjects, and 543 total slides (around 27,000 tiles ).
After revision:  We verify thataccuracy scores may be inflated up to 41\% , even if a well-designed 10x5 iterated cross-validation DAP is applied, unless all images from the same subject are used both in training and validation sets by deep learning models. We replicate the experiments for 4 classification tasks in digital pathology on 3 datasets, for a total of 373 subjects, and 543 total slides (around 27,000 tiles ).