Language is increasingly being used to define rich visual recognition problems with supporting image collections sourced from the web . Structured prediction models are used in these tasks to take advantage of correlations between co-occurring labels and visual input but risk inadvertently encoding social biases found in web corpora . In this work , we study data and models associated with multilabel object classification and visual semantic role labeling . We find that ( a ) datasets for these tasks contain significant gender bias and ( b ) models trained on these datasets further amplify existing bias . For example , the activity cooking is over 33 % more likely to involve females than males in a training set , and a trained model further amplifies the disparity to 68 % at test time . We propose to inject corpus-level constraints for calibrating existing structured prediction models and design an algorithm based on Lagrangian relaxation for collective inference . Our method results in almost no performance loss for the underlying recognition task but decreases the magnitude of bias amplification by 47.5 % and 40.5 % for multilabel classification and visual semantic role labeling , respectively . Visual recognition tasks involving language , such as captioning ( Vinyals et al . , 2015 ) , visual question answering ( Antol et al . , 2015 ) , and visual semantic role labeling ( Yatskar et al . , 2016 ) , have emerged as avenues for expanding the diversity of information that can be recovered from images . These tasks aim at extracting rich seman-tics from images and require large quantities of labeled data , predominantly retrieved from the web . Methods often combine structured prediction and deep learning to model correlations between labels and images to make judgments that otherwise would have weak visual support . For example , in the first image of Figure 1 , it is possible to predict a spatula by considering that it is a common tool used for the activity cooking . Yet such methods run the risk of discovering and exploiting societal biases present in the underlying web corpora . Without properly quantifying and reducing the reliance on such correlations , broad adoption of these models can have the inadvertent effect of magnifying stereotypes . In this paper , we develop a general framework for quantifying bias and study two concrete tasks , visual semantic role labeling ( vSRL ) and multilabel object classification ( MLC ) . In vSRL , we use the imSitu formalism ( Yatskar et al . , 2016 ( Yatskar et al . , , 2017 ) ) , where the goal is to predict activities , objects and the roles those objects play within an activity . For MLC , we use MS-COCO ( Lin et al . , 2014 ; Chen et al . , 2015 ) , a recognition task covering 80 object classes . We use gender bias as a running example and show that both supporting datasets for these tasks are biased with respect to a gender binary1 . Our analysis reveals that over 45 % and 37 % of verbs and objects , respectively , exhibit bias toward a gender greater than 2:1 . For example , as seen in Figure 1 , the cooking activity in imSitu is a heavily biased verb . Furthermore , we show that after training state-of-the-art structured predictors , models amplify the existing bias , by 5.0 % for vSRL , and 3.6 % in MLC . Each image is paired with a table describing a situation : the verb , cooking , its semantic roles , i.e agent , and noun values filling that role , i.e. woman . In the imSitu training set , 33 % of cooking images have man in the agent role while the rest have woman . After training a Conditional Random Field ( CRF ) , bias is amplified : man fills 16 % of agent roles in cooking images . To reduce this bias amplification our calibration method adjusts weights of CRF potentials associated with biased predictions . After applying our methods , man appears in the agent role of 20 % of cooking images , reducing the bias amplification by 25 % , while keeping the CRF vSRL performance unchanged . To mitigate the role of bias amplification when training models on biased corpora , we propose a novel constrained inference framework , called RBA , for Reducing Bias Amplification in predictions . Our method introduces corpus-level constraints so that gender indicators co-occur no more often together with elements of the prediction task than in the original training distribution . For example , as seen in Figure 1 , we would like noun man to occur in the agent role of the cooking as often as it occurs in the imSitu training set when evaluating on a development set . We combine our calibration constraint with the original structured predictor and use Lagrangian relaxation ( Korte and Vygen , 2008 ; Rush and Collins , 2012 ) to reweigh bias creating factors in the original model . We evaluate our calibration method on imSitu vSRL and COCO MLC and find that in both instances , our models substantially reduce bias amplification . For vSRL , we reduce the average magnitude of bias amplification by 40.5 % . For MLC , we are able to reduce the average magnitude of bias amplification by 47.5 % . Overall , our calibration methods do not affect the performance of the underlying visual system , while substantially reducing the reliance of the system on socially biased correlations 2 . 2 Code and data are available at https://github . com / uclanlp / reducingbias Structured prediction models can leverage correlations that allow them to make correct predictions even with very little underlying evidence . Yet such models risk potentially leveraging social bias in their training data . In this paper , we presented a general framework for visualizing and quantifying biases in such models and proposed RBA to calibrate their predictions under two different settings . Taking gender bias as an example , our analysis demonstrates that conditional random fields can amplify social bias from data while our approach RBA can help to reduce the bias . Our work is the first to demonstrate structured prediction models amplify bias and the first to propose methods for reducing this effect but significant avenues for future work remain . While RBA can be applied to any structured predictor , it is unclear whether different predictors amplify bias more or less . Furthermore , we presented only one method for measuring bias . More extensive analysis could explore the interaction among predictor , bias measurement , and bias deamplification method . Future work also includes applying bias reducing methods in other structured domains , such as pronoun reference resolution ( Mitkov , 2014 ) .

Challenge: Language is used for visual recognition problems such as captioning to improve performance however it can also encode social biases found in web corpora.
Approach: They propose a framework to quantify bias for visual semantic role labelling and multilabel object classification and a constraint inference framework to calibrate existing models.
Outcome: They find that existing datasets contain gender bias the use of text can amplify it, and the proposed framework can reduce bias without performance loss.