  Attribution map visualization has arisen as one of the most effective
techniques to understand the underlying inference process of Convolutional
Neural Networks. In this task, the goal is to compute an score for each image
pixel related with its contribution to the final network output. In this paper,
we introduce Disentangled Masked Backpropagation (DMBP), a novel gradient-based
method that leverages on the piecewise linear nature of ReLU networks to
decompose the model function into different linear mappings. This decomposition
aims to disentangle the positive, negative and nuisance factors from the
attribution maps by learning a set of variables masking the contribution of
each filter during back-propagation. A thorough evaluation over standard
architectures (ResNet50 and VGG16) and benchmark datasets (PASCAL VOC and
ImageNet) demonstrates that DMBP generates more visually interpretable
attribution maps than previous approaches. Additionally, we quantitatively show
that the maps produced by our method are more consistent with the true
contribution of each pixel to the final network output.
