  An important issue in dealing with Deep Convolutional Neural Networks (DCNN)
is the 'black box problem', which represents the unknowns about internal
information representation and processing, especially in the middle and higher
layers. In this study, we adopted a systems neuroscience methodology to measure
the visual feature selectivity and visualize the spatial receptive field of the
units in VGG16. Orientation and spatial frequency tunings of each unit were
measured using sinusoidal grating stimuli. The image category selectivity of
each unit was also measured using natural image stimuli. The spatial structures
of the receptive fields of all convolutional units were estimated by
activation-weighted average (AWA) and activation-weighted covariance (AWC)
analyses. In the middle layers (convolutional layers in block3 and block4), AWC
analysis successfully reconstructed the receptive field that predicted the
visual feature selectivity of the unit. Those results suggested the possibility
that analyzing the reconstructed receptive field structure can be used to
interpret the functional significance of the units and layers of a DCNN.
