White Noise Analysis of Neural Networks

1 INTRODUCTION . Any vision system , biological or artificial , has its own biases . These biases emanate from different sources . Two common sources include a ) the environment and the data on which the system has been trained , and b ) system constraints ( e.g. , hypothesis class , model parameters ) . Exploring these biases is important from at least two perspectives . First , it allows to better understand a system ( e.g. , explain and interpret its decisions ) . Second , it helps reveal system vulnerabilities and make it more robust against adversarial perturbations and attacks . In this paper , we recruit two popular methods from computational neuroscience to study the inherent biases in deep neural networks . The first one , called classification images technique , was introduced into visual psychophysics by Ahumada Jr ( 1996 ) as a new experimental tool . It has been used to examine visual processing and to understand vision across a variety of tasks including simple detection tasks , visual search , and object recognition . It has also been applied to the auditory domain . See Murray ( 2011 ) for a review of the topic . The second method , known as spike triggered analysis ( Marmarelis , 2012 ) , is often used to discover the best stimulus to which a neuron responds ( e.g. , oriented bars ) . These methods are appealing for our purpose since a ) they are general and can be applied to study any black box system ( so long it emits a response to an input stimulus ) and b ) make a modest number of assumptions . From a system identification point of view , they provide a first-order approximation of a complex system such as the brain or an artificial neural network . By feeding white noise stimuli to a classifier and averaging the ones that are categorized into a particular class , we obtain an estimate of the templates it uses for classification . Unlike classification images experiments in human psychophysics , where running a large number of trials is impractical , artificial systems can often be tested against a large number of inputs . While still a constraint , we will discuss how such problems can be mitigated ( e.g. , by generating stimuli containing faint structures ) . Over four datasets , MNIST ( LeCun et al. , 1998 ) , Fashion-MNIST ( Xiao et al. , 2017 ) , ∗Work done during internship at MarkableAI . 1Code is available at : https : //github.com/aliborji/WhiteNoiseAnalysis.git . CIFAR-10 ( Krizhevsky et al. , 2009 ) , and ImageNet Deng et al . ( 2009 ) , we employ classification images to discover implicit biases of a network , utilize those biases to influence network decisions , and detect adversarial perturbations . We also show how spike triggered averaging can be used to identify and visualize filters in different layers of a CNN . Finally , in a less directly related analysis to classification images , we demonstrate how decisions of a CNN are influenced by varying the signal to noise ratio ( akin to microstimulation experiments in monkey electrophysiology or priming experiments in psychophysics ) . We find that CNNs behave in a similar fashion to their biological counterparts and their responses can be characterized by a psychometric function . This may give insights regarding top-down attention and feedback mechanisms in CNNs ( See Borji & Itti ( 2012 ) ) . 2 RELATED WORKS AND CONCEPTS . Our work relates to a large body of research attempting to understand , visualize , and interpret deep neural networks . These networks have been able to achieve impressive performance on a variety of challenging vision and learning tasks ( e.g. , Krizhevsky et al . ( 2012 ) ; He et al . ( 2016 ) ) . However , they are still not well understood , have started to saturate in performance ( Recht et al. , 2019 ) , are brittle2 , and continue to trail humans in accuracy and generalization . This calls for a tighter confluence between machine learning , computer vision , and neuroscience . In this regard , the proposed tools here are complementary to the existing ones in the deep learning toolbox . Perhaps , the closest work to ours is Vondrick et al . ( 2015 ) where they attempted to learn biases in the human visual system and transfer those biases into object recognition systems . Some other works ( e.g. , Fong et al . ( 2018 ) ) have also used human data ( e.g. , fMRI , cell recording ) to improve the accuracy of classifiers , but have not utilized classification images . Bashivan et al . ( 2019 ) used is activation maximization to iteratively change the pixel values in the direction of the gradient to maximize the firing rate of V4 neurons3 . Unlike these works , here we strive to inspect the biases in classifiers , in particular , neural networks , to improve their interpretability and robustness . 2.1 CLASSIFICATION IMAGES . In a typical binary classification image experiment , on each trial , a signal s ∈ Rd and a noise image z ∈ Rd are summed to produce the stimulus n. The observer is supposed to decide which of the two categories the stimulus belongs to . Classification image is then calculated as : c = ( n̄12 + n̄22 ) − ( n̄11 + n̄21 ) ( 1 ) where n̄sr is the average of noise patterns in a stimulus–response class of trials . For example , n̄12 is the average of the noise patterns over all trials where the stimulus contained signal 1 but 2Current deep neural networks can be easily fooled by subtle image alterations in ways that are imperceptible to humans ; a.k.a adversarial examples ( Szegedy et al. , 2013 ; Goodfellow et al. , 2014b ) . 3See https : //openreview.net/forum ? id=H1ebhnEYDH for a discussion on this . the observer responded 2. c ∈ Rd is an approximation of the template that the observer uses to discriminate between the two stimulus classes . The intuition behind the classification images is that the noise patterns in some trials have features similar to one of the signals , thus biasing the observer to choose that signal . By computing the average over many trials a pattern may emerge . c can also be interpreted as the correlation map between stimulus and response : corr [ n , r ] = E ( n− E [ n ] ) E ( r − E [ r ] ) σnσr ( 2 ) where σn is the pixel-wise standard deviation of the noise n and σr is the standard deviation of response r. High positive correlations occur at spatial locations that strongly influence the observer ’ s responses . Conversely , very low ( close to zero ) correlations occur at locations that have no influence on the observer ’ s responses . Assuming zero-mean noise and an unbiased observer , Eq . 2 reduces to ccorr = n̄∗2 − n̄∗1 , where n̄∗u is the average of the noise patterns over all trials where the observer gave a response u ( See Murray ( 2011 ) for details ) . Thus , ccorr is the average of the noise patterns over all trials where the observer responded r = 2 , minus the average over all trials where the observer responded r = 1 , regardless of which signal was presented . We have illustrated the classification images concept in Fig . 1 with a binary classifier trained to separate two digits . The stimulus is a linear combination of noise plus signal as follows : t = γ × s + ( 1− γ ) × n ; γ ∈ [ 0,1 ] ( 3 ) The computed templates for different γ values4 , using about 10 million trials , highlight regions that are correlated with one of the digits ( here 1 vs. 7 or 5 vs. 6 ) . The template fades away with increasing noise ( e.g. , γ = 0 ) but it still resembles the template in the low-noise condition ( i.e. , γ = 0.3 ) . 2.2 SPIKE TRIGGERED ANALYSIS . The spike-triggered analysis , also known as “ reverse correlation ” or “ white-noise analysis ” , is a tool for characterizing the response properties of a neuron using the spikes emitted in response to a time-varying stimulus . It includes two methods : spike-triggered averaging ( STA ) and spiketriggered covariance ( STC ) . They provide an estimate of a neuron ’ s linear receptive field and are useful techniques for the analysis of electrophysiological data . In the visual system , these methods have been used to characterize retinal ganglion cells ( Meister et al. , 1994 ; Sakai & Naka , 1987 ) , lateral geniculate neurons ( Reid & Alonso , 1995 ) , and simple cells in the primary visual cortex ( DeAngelis et al. , 1993 ; Jones & Palmer , 1987 ) . See Schwartz et al . ( 2006 ) for a review . STA is the average stimulus preceding a spike . It provides an unbiased estimate of a neuron ’ s receptive field only if the stimulus distribution is spherically symmetric ( e.g. , Gaussian white noise ) . STC can be used to identify a multi-dimensional feature space in which a neuron computes its response . It identifies the stimulus features affecting a neuron ’ s response via an eigen-decomposition of the spike-triggered covariance matrix ( Sandler & Marmarelis , 2015 ; Park & Pillow , 2011 ) . Let x ∈ Rd denote a spatio-temporal stimulus vector affecting a neuron ’ s scalar spike response y in a single time bin . The main goal of neural characterization is to find Θ , a low-dimensional projection matrix such that ΘTx captures the neuron ’ s dependence on the stimulus x . The STA and the STC matrix are the empirical first and second moments of the spike-triggered stimulus-response pairs { xi|yi } Ni=1 , respectively . They are defined as : STA : µ = 1 nsp N∑ i=1 yixi , and STC : Λ = 1 nsp N∑ i=1 yi ( xi − µ ) ( xi − µ ) T , ( 4 ) where nsp = ∑ yi is the number of spikes and N is the total number of time bins . The traditional spike triggered analysis gives an estimate for the basis Θ consisting of : ( 1 ) µ , if it is significantly different from zero , and ( 2 ) the eigenvectors of Λ corresponding to those eigenvalues that are significantly different from eigenvalues of the prior stimulus covariance Φ = E [ XXT ] . When a stimulus is not white noise ( i.e. , is correlated in space or time ) , whitened STA can be written as : STAw = N nsp ( XTX ) −1XTy ( 5 ) 4We use classification images , bias map , template , and average noise pattern , interchangeably . Please do not confuse this bias with the bias terms in neural networks . where X is a matrix whose ith row is the stimulus vector xTi and y denotes a column vector whose ith element is yi . The whitened STA is equivalent to linear least-squares regression of the stimulus against the spike train . Classification images and spike triggered analysis are related in the sense that both estimate the terms of a Wiener/Volterra expansion in which the mapping from the stimuli to the firing rate is described using a low-order polynomial ( Marmarelis , 2012 ) . See Dayan et al . ( 2001 ) for a discussion on this . Here , we focus on STA and leave STC to future works .

Summary: This work inspects the bias existing in the neural network classifiers through two techniques from computational neuroscience, the classification image techniques and spike-triggered covariance (STA) analysis. The classification image is generated as the difference map between all the averaged images with predicted classes under noise perturbation. STA is used to generate the eigenvector as the input and response, as a visualization. Both of the tools are the standard psychophysics tools.