Do Image Classifiers Generalize Across Time?

1 INTRODUCTION . Convolutional neural networks ( CNNs ) still exhibit many troubling failure modes . At one extreme , ` p-adversarial examples cause large drops in accuracy for state-of-the-art models while relying only on visually imperceptible changes to the input image ( Goodfellow et al. , 2014 ; Biggio and Roli , 2018 ) . However , this failure mode usually does not pose a problem outside a fully adversarial context because carefully crafted ` p-perturbations are unlikely to occur naturally in the real world . To study more realistic failure modes , researchers have investigated benign image perturbations such as rotations & translations , colorspace changes , and various image corruptions ( Fawzi and Frossard , 2015 ; Engstrom et al. , 2017 ; Fawzi and Frossard , 2015 ; Hendrycks and Dietterich , 2019 ) . However , it is still unclear whether these perturbations reflect the robustness challenges arising in real data since the perturbations also rely on synthetic image modifications . Recent work has therefore turned to videos as a source of naturally occurring perturbations of images ( Zheng et al. , 2016 ; Azulay and Weiss , 2018 ; Gu et al. , 2019 ) . In contrast to other failure modes , the perturbed images are taken from existing image data without further modifications that make the task more difficult . As a result , robustness to such perturbations directly corresponds to performance improvements on real data . However , it is currently unclear to what extent such video perturbations pose a significant robustness challenge . Azulay and Weiss ( 2018 ) and Zheng et al . ( 2016 ) only provide anecdotal evidence from a small number of videos . Gu et al . ( 2019 ) go beyond individual videos and utilize a large video dataset ( Real et al. , 2017 ) in order to measure the effect of video perturbations more quantitatively . In their evaluation , the best image classifiers lose about 3 % accuracy for video frames up to 0.3 seconds away . However , the authors did not employ humans to review the frames in their videos . Hence the accuracy drop could also be caused by significant changes in the video frames ( e.g. , due to fast camera or object motion ) . Since the 3 % accuracy drop is small to begin with , it remains unclear whether video perturbations are a robustness challenge for current image classifiers . We address these issues by conducting a thorough evaluation of robustness to natural perturbations arising in videos . As a cornerstone of our investigation , we introduce two test sets for evaluating model robustness : ImageNet-Vid-Robust and YTBB-Robust , carefully curated from the ImageNet-Vid and Youtube-BB datasets , respectively ( Russakovsky et al. , 2015 ; Real et al. , 2017 ) . All images in the two datasets were screened by a set of expert labelers to ensure high annotation quality and minimize selection biases that arise when filtering a dataset with CNNs . To the best of our knowledge these are the first datasets of their kind , containing tens of thousands of images that are human reviewed and grouped into thousands of perceptually similar sets . In total , our datasets contain 3,139 sets of temporally adjacent and visually similar images ( 57,897 images total ) . We then utilize these datasets to measure the accuracy of current CNNs to small , naturally occurring perturbations . Our testbed contains over 45 different models , varying both architecture and training methodology ( adversarial training , data augmentation , etc. ) . To better understand the drop in accuracy due to natural perturbations , we also introduce a robustness metric that is more stringent than those employed in prior work . Under this metric , we find that natural perturbations from ImageNet-Vid-Robust and YTBB-Robust induce a median accuracy drop of 16 % and 10 % respectively for classification tasks and a median 14 point drop in mAP for detection tasks.1 Even for the best-performing classification models , we observe an accuracy drop of 14 % for ImageNet-Vid-Robust and 8 % for YTBB-Robust . Our results show that robustness to natural perturbations in videos is indeed a significant challenge for current CNNs . As these models are increasingly deployed in safety-critical environments that require both high accuracy and low latency ( e.g. , autonomous vehicles ) , ensuring reliable predictions on every frame of a video is an important direction for future work . 2 CONSTRUCTING A TEST SET FOR ROBUSTNESS . ImageNet-Vid-Robust and YTBB-Robust are sourced from videos in the ImageNet-Vid and Youtube-BB datasets ( Russakovsky et al. , 2015 ; Real et al. , 2017 ) . All object classes in ImageNet-Vid and Youtube-BB are from the WordNet hierarchy ( Miller , 1995 ) and direct ancestors of ILSVRC-2012 classes . Using the WordNet hierarchy , we construct a canonical mapping from ILSVRC-2012 classes to ImageNet-Vid and Youtube-BB classes , which allows us to evaluate off-the-shelf ILSVRC-2012 models on ImageNet-Vid-Robust and YTBB-Robust . We provide more background on the source datsets in Appendix A . 2.1 CONSTRUCTING IMAGENET-VID-ROBUST AND YTBB-ROBUST Next , we describe how we extracted sets of naturally perturbed frames from ImageNet-Vid and Youtube-BB to create ImageNet-Vid-Robust and YTBB-Robust . A straightforward approach would be to select a set of anchor frames and use temporally adjacent frames in the video with the assumption that such frames contain only small perturbations from the anchor . However , as Fig . 2 illustrates , this assumption is frequently violated , especially due to fast camera or object motion . Instead , we first collect preliminary datasets of natural perturbations following the same approach , and then manually review each of the frame sets . For each video , we randomly sample an anchor frame and take k = 10 frames before and after the anchor frame as candidate perturbation images.2 This results in two datasets containing one anchor frame each from 3,139 videos , with approximately 20 candidate perturbation per anchor frame.3 1We only evaluated detection on ImageNet-Vid-Robust as bounding-box annotations in Youtube-BB were only at 1 frame-per-second and not dense enough for our evaluation . 2For YTBB-Robust we use a subset of the anchor frames used by Gu et al . ( 2019 ) . 3Anchor frames near the start or end of the video may have less than 20 candidate frames . Anchor frame Discarded frame Anchor frame Anchor frame Discarded frameDiscarded frame Next , we curate the dataset with the help of four expert human annotators . The goal of the curation step is to ensure that each anchor frame and its nearby frames are correctly labeled with the same ground truth class , and that the anchor frame and the nearby frames are visually similar . Denser labels for Youtube-BB . As Youtube-BB contains only a single category label per frame at 1 frame per second , annotators first viewed each anchor frame individually and marked any missing labels . In total , annotators corrected the labels for 834 frames , adding an average of 0.5 labels per anchor frame . These labels are then propagated to nearby , unlabeled frames at the native frame rate and verified in the next step . ImageNet-Vid densely labels all classes per frame , so we skip this step . Frame pairs review . Next , for each pair of anchor and candidate perturbation frames , a human annotates ( i ) whether the pair is correctly labeled in the dataset , and ( ii ) whether the pair is similar . We took several steps to mitigate the subjectivity of this task and ensure high annotation quality . First , we trained reviewers to mark frames as dissimilar if the scene undergoes any of the following transformations : significant motion , significant background change , or significant blur change . We asked reviewers to mark each dissimilar frame with one of these transformations , or “ other ” , and to mark a pair of images as dissimilar if a distinctive feature of the object is only visible in one of the two frames ( such as the face of a dog ) . If an annotator was unsure about the correct label , she could mark the pair as “ unsure ” . Second , we present only a single pair of frames at a time to reviewers because presenting videos or groups of frames could cause them to miss large changes due to the phenomenon of change blindness ( Pashler , 1988 ) . Verification . In the previous stage , all annotators were given identical labeling instructions and individually reviewed a total of 71,660 images pairs . To increase consistency in annotation , annotators jointly reviewed all frames marked as dissimilar , incorrectly labeled , or “ unsure ” . A frame was only considered similar to its anchor if a strict majority of the annotators marked the pair as such . After the reviewing was complete , we discarded all anchor frames and candidate perturbations that annotators marked as dissimilar or incorrectly labeled . The final datasets contain a combined total of 3,139 anchor frames with a median of 20 similar frames each . 2.2 THE PM-K EVALUATION METRIC Given the datasets introduced above , we propose a metric to measure a model ’ s robustness to natural perturbations . In particular , let A = { a1 , ... , an } be the set of valid anchor frames in our dataset . Let Y = { y1 , ... , yn } be the set of labels for A . We let Nk ( ai ) be the set of frames marked as similar to anchor frame ai . In our setting , Nk is a subset of the 2k temporally adjacent frames ( plus/minus k frames from the anchor ) . Classification . Classification accuracy is defined as accorig = 1− 1N ∑N i=0 L0/1 ( f ( ai ) , yi ) , where L0/1 is the standard 0-1 loss function . We define the pm-k analog of accuracy as accpmk = 1− 1 N N∑ i=0 max b∈Nk ( ai ) L0/1 ( f ( b ) , yi ) , ( 1 ) which corresponds to picking the worst frame from each set Nk ( ai ) before computing accuracy . Detection . The standard metric for detection is mean average precision ( mAP ) of the predictions at a fixed intersection-over-union ( IoU ) threshold Lin et al . ( 2014 ) . We define the pm-k metric analogous to that for classification : We replace each anchor frame with the nearest frame that minimizes the average precision ( AP , averaged over recall thresholds ) of the predictions , and compute pm-k as the mAP on these worst-case neighboring frames . 3 MAIN RESULTS . We evaluate a testbed of 45 classification and three detection models on ImageNet-Vid-Robust and YTBB-Robust . We first discuss the various types of classification models evaluated with the pm-k classification metric . Second , we evaluate the performance of detection models on ImageNet-Vid-Robust using use the bounding box annotations inherited from ImageNet-Vid using a variant of pm-k for detection . We then analyze the errors made on the detection adversarial examples to isolate the effects of localization errors vs. classification errors . 3.1 CLASSIFICATION . The classification robustness metric is accpmk defined in Equation ( 1 ) . For frames with multiple labels , we count a prediction as correct if the model predicts any of the correct classes for a frame . In Figure 3 , we plot the benign accuracy , accorig , versus the robust accuracy , accpmk , for all classification models in our test bed and find that the relationship between accorig and accpmk is approximately linear . This relationship indicates that improvements in the benign accuracy do result in improvements in the worst-case accuracy , but do not suffice to resolve the accuracy drop due to natural perturbations . Our test bed consists of five model types with increasing levels of supervision . We present results for representative models from each model type in Table 2 and defer the full classification results table to Appendix B.2 . ILSVRC Trained The WordNet hierarchy enables us to repurpose models trained for the 1,000 class ILSVRC dataset on ImageNet-Vid-Robust and YTBB-Robust ( see Appendix A.1 ) . We evaluate a wide array of ILSVRC-2012 models ( available from Cadene ) against our natural perturbations . Since these datasets present a substantial distribution shift from the original ILSVRC2012 validation , we expect the benign accuracy accorig to be lower than the comparable accuracy on the ILSVRC-2012 validation set . However , our main interest here is in the difference between the original and perturbed accuracies accorig - accpmk . A small drop in accuracy would indicate that the model is robust to small changes that occur naturally in videos . Instead , we find significant drops of 15.0 % and 13.2 % in accuracy on our two datasets , indicating sensitivity to such changes . Noise augmentation One hypothesis for the accuracy drop from original to perturbed accuracy is that subtle artifacts and corruptions introduced by video compression schemes could degrade performance when evaluating on these corrupted frames . The worst-case nature of the pm-k metric could then be focusing on these corrupted frames . One model for these corruptions are the perturbations introduced in Hendrycks and Dietterich ( 2019 ) . To test this hypothesis , we evaluate models augmented with a subset of the perturbations ( exactly one of : Gaussian noise , Gaussian blur , shot noise , contrast change , impulse noise , or JPEG compression ) . We found that these augmentation schemes did not improve robustness against our perturbations substantially , and still result in accuracy drop of 15.6 % and 16.6 % on the two datasets . ` ∞ robustness . We evaluate the model from Xie et al . ( 2018 ) , which currently performs best against ` ∞ attacks on ImageNet . We find that this model has a smaller accuracy drop than the two aforementioned model types on both datasets . However , we note that the robust model achieves significantly lower original and perturbed accuracy than either of the two model types above , and the robustness gain is modest ( 3 % compared to models of similar benign accuracy ) . Fine-tuning on video frames . To adapt to the new class vocabulary and the video domain , we fine-tune several network architectures on the ImageNet-Vid and Youtube-BB training sets . For Youtube-BB , we train on the anchor frames used for training in Gu et al . ( 2019 ) , and for ImageNet-Vid we use all frames in the training set . We provide hyperparameters for all models in Appendix K. The resulting models significantly improve in accuracy over their ILSVRC pre-trained counterparts ( e.g. , 13 % on ImageNet-Vid-Robust and 34 % on YTBB-Robust for ResNet-50 ) . This improvement in accuracy results in a modest improvement in the accuracy drop for YTBB-Robust , but a finetuned ResNet-50 still suffers from a significant 9.4 % drop . On ImageNet-Vid-Robust , there is almost no change in the accuracy drop from 15.0 % to 15.1 % . Fine-tuning for detection on video frames . We further analyze whether additional supervision in the form of bounding box annotations improves robustness . To this end , we train the Faster R-CNN detection model Ren et al . ( 2015 ) with a ResNet-50 backbone on ImageNet-Vid . Following standard practice , the detection backbone is pre-trained on ILSVRC-2012 . To evaluate this detector for classification , we assign the class with the most confident bounding box as label to the image . We find that this transformation reduces accuracy compared to the model trained for classification ( 77.6 % vs. 80.8 % ) . While there is a slight reduction in the accuracy drop caused by natural perturbations , the reduction is well within the error bars for this test set .

Summary: In this paper, the authors curated two datasets: ImageNet-Vid and Youtube-BB in order to create human-reviewed perceptibly similar sets (Imagenet-Vid-Robust and YTBB-Robust). The obtained datasets are evaluated over 45 different models pre-trained on ImageNet in order to see their drop in accuracy on natural perturbations. Three detection models are also evaluated and show that not only classification models are sensitive to these perturbations, but that it also yields to localization errors.