  The ability to detect edges is a fundamental attribute necessary to truly
capture visual concepts. In this paper, we prove that edges cannot be
represented properly in the first convolutional layer of a neural network, and
further show that they are poorly captured in popular neural network
architectures such as VGG-16 and ResNet. The neural networks are found to rely
on color information, which might vary in unexpected ways outside of the
datasets used for their evaluation. To improve their robustness, we propose
edge-detection units and show that they reduce performance loss and generate
qualitatively different representations. By comparing various models, we show
that the robustness of edge detection is an important factor contributing to
the robustness of models against color noise.
