Mind Your Outliers! Investigating the Negative Impact of Outliers on Active Learning for Visual Question Answering (2021)

Abstract: Active learning promises to alleviate the massive data needs of supervised machine learning: it has successfully improved sample efficiency by an order of magnitude on traditional tasks like topic classification and object recognition. However, we uncover a striking contrast to this promise: across 5 models and 4 datasets on the task of visual question answering, a wide variety of active learning approaches fail to outperform random selection. To understand this discrepancy, we profile 8 active learning methods on a per-example basis, and identify the problem as collective outliers – groups of examples that active learning methods prefer to acquire but models fail to learn (e.g., questions that ask about text in images or require external knowledge). Through systematic ablation experiments and qualitative visualizations, we verify that collective outliers are a general phenomenon responsible for degrading pool-based active learning. Notably, we show that active learning sample efficiency increases significantly as the number of collective outliers in the active learning pool decreases. We conclude with a discussion and prescriptive recommendations for mitigating the effects of these outliers in future work.

## introduction today, language-equipped vision systems such as vizwiz, taptapsee, bemyeyes, and camfind are actively being deployed across a broad spectrum of users. 1 as underlying methods improve, these systems will be expected to operate over diverse visual environments and understand myriad language inputs @xcite @xcite @xcite . visual question answering (vqa), the task of answering questions about figure 1 : we systematically evaluate active learning on vqa datasets and isolate their inability to perform better than random sampling due to the presence of collective outliers. active learning methods prefer to acquire these outliers, which are hard and often impossible for models to learn. we show that dataset maps, like the one shown here, can heuristically identify these collective outliers as examples assigned low model confidence and prediction variability during training. while active learning is often key to effective data acquisition when such labeled data is difficult to obtain @xcite @xcite , we find that 8 modern active learning methods @xcite @xcite show little to no improvement in sample efficiency across 5 models on 4 vqa datasets -indeed, in some cases performing worse than randomly selecting data to label. this finding is in stark contrast to the successful application of active learning methods on a variety of traditional tasks, such as topic classification @xcite , object recognition @xcite , digit classification @xcite , and named entity recognition @xcite . our negative results hold even when accounting for common active learning ailments: cold starts, correlated sampling, and uncalibrated uncertainty. we mitigate the cold start challenge of needing a representative initial dataset by varying the size of the seed set in our experiments. we account for sampling correlated data within a given batch by including core-set selection @xcite in the set of active learning methods we evaluate. finally, we use deep bayesian active learning to calibrate model uncertainty to high-dimensional data @xcite @xcite . after concluding that negative results are consistent across all experimental conditions, we investigate active learning's ineffectiveness on vqa as a data problem and identify the existence of collective outliers @xcite as the source of the problem. leveraging recent advances in model interpretability, we build dataset maps @xcite , which distinguish between collective outliers and useful data that improve validation set performance (see figure 1 ). while global outliers deviate from the rest of the data and are often a consequence of labeling error, collective outliers cluster together; they may not individually be identifiable as outliers but collectively deviate from other examples in the dataset. for instance, vqa-2 @xcite is riddled with collections of hard questions that require external knowledge to answer (e.g., "what is the symbol on the hood often associated with?") or that ask the model to read text in the images (e.g., "what is the word on the wall?"). similarly, gqa @xcite asks underspecified questions (e.g., "what is the person wearing?" which can have multiple correct answers). collective outliers are not specific to vqa, but can similarly be found in many open-ended tasks, including visual navigation @xcite ) (e.g., "go to the grandfather clock" requires identifying rare grandfather clocks), and open-domain question answering @xcite , amongst others. using dataset maps, we profile active learning methods and show that they prefer acquiring collective outliers that models are unable to learn, explaining their poor improvements in sample efficiency relative to random sampling. building on this, we use these maps to perform ablations where we identify and remove outliers iteratively from the active learning pool, observing correlated improvements in sample efficiency. this allows us to conclude that collective outliers are, indeed, responsible for the ineffectiveness of active learning for vqa. we end with prescriptive suggestions for future work in building active learning methods robust to these types of outliers. ## related work our work tests the utility of multiple recent active learning methods on the open-ended understanding task of vqa. we draw on the dataset analysis literature to identify collective outliers as the bottleneck hindering active learning methods in this setting. active learning. active learning strategies have been successfully applied to image recognition @xcite , information extraction @xcite @xcite , named entity recognition @xcite , semantic parsing @xcite , and text categorization @xcite . however, these same methods struggle to outperform a random baseline when applied to the task of vqa @xcite . to study this discrepancy, we systematically apply 8 diverse active learning methods to vqa, including methods that use model uncertainty @xcite @xcite , bayesian uncertainty @xcite , disagreement @xcite , and core-set selection @xcite . ## active learning experimental setup we adopt the standard pool-based active learning setup from prior work @xcite @xcite , consisting of a model m, initial seed set of labeled examples (x i , y i ) ∈ d seed used to initialize m, an unlabeled pool of data d pool , and an acquisition function a(x, m). we run active learning over a series of acquisition iterations pool size # answers vqa-sports 5,411 [5k] 20 vqa-food 4,082 [4k] 20 vqa-2 411,272 [400k] 3130 gqa 943,000 [900k] 1842 t where at each iteration we acquire a batch of b new examples per: @xmath0, m). acquiring an example often refers to using an oracle or human expert to annotate a new example with a correct label. we follow prior work to simulate an oracle using existing datasets, forming d seed from a fixed percentage of the full dataset, and using the remainder as d pool @xcite @xcite . we re-train m after each acquisition iteration. prior work has noted the impact of seed set size on active learning performance @xcite @xcite . we run multiple active learning evaluations with varying seed set sizes (ranging from 5% to 50% of the full pool size). we keep the size of each acquisition batch b to a constant 10% of the overall pool size. ## analysis via dataset maps the previous section shows that active learning fails to improve over random acquisition on vqa across models and datasets. a simple question remains -why? one hypothesis is that sample inefficiency stems from the data itself: there is only a 2% gain in validation accuracy when training on half versus the whole dataset. working from this, we characterize the underlying datasets using dataset maps @xcite and discover that active learning methods prefer sampling "hard-tolearn" examples, leading to poor performance. mapping vqa datasets. a dataset map @xcite ) is a model-specific graph for profiling the learnability of individual training examples. dataset maps present holistic pictures of classification datasets relative to the training dynamics of a given model; as a model trains for multiple epochs and sees the same examples repeatedly, the mapping process logs statistics about the confidence assigned to individual predictions. maps then visualize these statistics against two axes: the y-axis plots the average model confidence assigned to the correct answer over training epochs, while the x-axis plots the spread, or variability, of these values. this introduces a 2d representation of a dataset (viewed through its relationship with individual model) where examples are placed on the map by coarse statistics describing their "learnability". we show the dataset map for butd trained on vqa-2 in figure 1 . for our work, we build this map post-hoc, training on the entire pool as a means for analyzing what active learning is doing -treating it as a diagnostic tool for identifying the root cause why active learning seems to fail for vqa. in an ideal setting, the majority of examples in the training set should lie in the upper half of the graph -i.e., the mean confidence assigned to the correct answer should be relatively high. examples towards the upper-left side represent the "easy-tolearn" examples, as the variability in the confidence assigned by the model over time is fairly low. a curious feature of vqa-2 and other vqa datasets is the presence of the 25-30% of examples in the bottom-left of the map (shown in red in figure 1 ) -examples that have low confidence and variability. in other words, models are unable to learn a large proportion of training examples. while prior work attributes examples in this quadrant to "labeling errors" @xcite , labeling errors in vqa are sparse, and cannot account for the density of such examples in these maps. ## collective outliers this leaves two questions: 1) can we characterize these "hard" examples, and 2) are these examples responsible for the ineffectiveness of active learning on vqa? we first identify hard-to-learn examples as collective outliers and explain why active learning methods prefer to acquire them. next, we perform ablation experiments, removing these outliers from the active learning pool iteratively, and demonstrate a corresponding boost in sample efficiency relative to random acquisition. hard examples are collective outliers. collective outliers are groups of examples that deviate from the rest of the examples but cluster together @xcite -they often present as fundamental subproblems of a broader task. for instance (figure 7 ), in vqa-2, we identify clusters of hard-to-learn examples that require optical character recognition (ocr) for reasoning about text (e.g., "what is the first word on the black car?"); another cluster requires external knowledge to answer ("what is the symbol on the hood often associated with?"). in gqa, we identify different clusters of collective outliers; one cluster stems from innate underspecification (e.g., "what is on the shelf?" with multiple objects present on the shelf); another cluster requires multiple reasoning hops difficult for current models (e.g., "what is the vehicle that is driving down the road the box is on the side of?"). we sample 100 random "hard-to-learn" examples from both vqa-2 and gqa and find that 100% of the examples belong to one of the two aforementioned collectives. since hard-to-learn examples constitute 25-30% of the data pool, active learning methods cannot avoid them. uncertainty-based methods (e.g., least-confidence, entropy, monte-carlo dropout) identify them as valid acquisition targets because models lack the capacity to correctly answer these examples, assigning low confidence and high uncertainty. disagreementbased methods (e.g., bald) are similar; model confidence is generally low but high variance (lower middle/lower right of the dataset maps). finally, diversity methods (e.g., core-set selection) identify these examples as different enough from the existing pool to warrant acquisition, but fail to learn meaningful representations, fueling a vicious cycle wherein they continue to pick these examples. ablating outliers. to verify that collective outliers are responsible for the degradation of active learning performance, we re-run our experiments using active learning pools with varying numbers of outliers removed. to remove these outliers, we sort and remove all examples in the data pool using the product of their model confidence and prediction variability (x and y-axis values of the dataset maps). we systematically remove examples with a low product value and observe how active learning performance changes (see figure 8 ). we observe a 2-3x improvement in sample efficiency when removing 50% of the entire data pool, consisting mainly of collective outliers (figure 8c ). this improvement decreases if we only remove 25% of the full pool (figure 8b ), and further degrades if we remove only 10% (figure 8a ). this ablation demonstrates that active learning methods are more sample efficient than the random baseline when collective outliers are absent from the unlabelled pool. ## discussion and future work this paper asks a simple question -why does the modern neural active learning toolkit fail when applied to complex, open ended tasks? while we focus on vqa, collective outliers are abundant in tasks such as natural language inference @xcite and opendomain question answering @xcite , amongst others. more insidious is their nature; collective outliers can take multiple forms, requiring external domain knowledge or "commonsense" reasoning, containing underspecification, or requiring capabilities beyond the scope of a given model (e.g., requiring ocr ability). while we perform ablations in this work removing collective outliers, demonstrating that active learning fails as collective outliers take up larger portions of the dataset, this is only an analytical tool; these outliers are, and will continue to be, pervasive in open-ended datasets -and as such, we will need to develop better tools for learning (and performing active learning) in their presence. selective classification. one potential direction for future work is to develop systems that abstain when they encounter collective outliers. historical artificial intelligence systems, such as shrdlu @xcite and qualm @xcite , were designed to flag input sequences that they were not designed to parse. ideas from those methods can and should be resurrected using modern techniques; for example, recent work suggests that a simple classifier can be trained to identify out-ofdomain data inputs, provided a seed out-of-domain dataset @xcite . active learning methods can be augmented with a similar classifier, which re-calibrates active learning uncertainty scores with this classifier's predictions. other work learns to identify novel utterances by learning to intelligently set thresholds in representation space @xcite , a powerful idea especially if combined with other representation-centric active learning methods like core-set sampling @xcite .