Original: S UPPRESSION HELPS : L ATERAL I NHIBITION - INSPIRED  C ONVOLUTIONAL N EURAL N ETWORK FOR I MAGE C LASSIFICATION
Revised: L ATERAL I NHIBITION - INSPIRED STRUCTURE FOR C ONVOLUTIONAL N EURAL N ETWORK ON I MAGE C LASSIFICATION

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Convolutional neural networks (CNNs) have become powerful and popular tools since deep learning emerged for image classification in the computer vision field.
Revised: Convolutional neural networks (CNNs) have become powerful and popular tools since deep learning emerged for image classification in the computer vision field.

Original: For better recognition, the dimensions of  depth and width have been explored, leading to convolutional neural networks with more layers andmore channels.
Revised: For better recognition, the dimension of both depth and width has been explored, leading to convolutional neural networks with more layers and channels.

Original: In addition to these factors, neurobiologyalso suggeststhe widely existing lateral inhibition (e.g., Mach band effect), which increases the contrast  of nearby neuron excitation in the lateral direction, to help recognition.
Revised: In addition to these factors, neurobiology suggests lateral inhibition (lateral antagonism, e.g. Mach band effect), a widely existing phenomenon for vision that increases the contrast and sharpness of nearby neuron excitation in the lateral direction to help recognition.

Original: However, suchan important mechanism has not been well explored in modern convolutional neural networks.
Revised: However, such mechanism has not been well explored in the design of convolutional neural network.

Original: In this paper, we explicitly explore the filter dimension in the lateral direction and propose our lateral inhibition-inspired (LI) design.
Revised: In this paper, we explicitly explore the filter dimension in the lateral direction and propose our lateral inhibition-inspired (LI) structure.

Original: Our naive design incorporates the low-pass filter, while eliminating the central weight to mimic theinhibition strength decay.
Revised: Our naive design uses the low-pass filter to mimic the strength decay of lateral interaction from neighbors regarding the distance.

Original: The inhibition value is computed from the filtering result of the input, with a simple learnableweight parameter per channel for multiplication to decide the strength.
Revised: One learnable parameter per channel is applied to set the amplitude of the low-pass filter by multiplication, which is flexible to model various lateral interactions (including lateral inhibition).

Original: Then the inhibition value is  subtracted from the input as suppression, which could increase the contrast to help recognition. We also suggest an alternative using depthwise convolution, as a general form.
Revised: The convolution result is then subtracted from the input, which could increase the contrast and sharpness for better recognition. Furthermore, a learnable scaling factor and shift are applied to adjust the value after subtraction.

Original: Our design could work on boththe plain convolution and the convolutional block with residual connection, while being compatible with  existing modules.
Revised: Our lateral inhibition-inspired (LI) structure works on both plain convolution and the convolutional block with residual connection, while being compatible with the existing modules.

Original: Without any channel attention along the channel dimension, the preliminary results demonstrate an absolute improvement of 3.68% and 0.69% over AlexNet  and ResNet-18, respectively,in the ImageNet data set, with little increase in parameters, indicating the merits of our  design to help feature learning for image classification.
Revised: Preliminary results demonstrate obvious improvements on the ImageNet dataset for AlexNet (7.58%) and ResNet-18 (0.81%), respectively, with little increase in parameters, indicating the effectiveness of our brain-similar design to help feature learning for image classification from a different perspective.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: In recent years, convolutional neural networks (CNNs) (Hinton et al., 2012; Simonyan & Zisserman, 2015; Szegedy et al., 2015; He et al., 2016) have become powerful and popular tools since deep learning emerged for image classification in the computer vision field.
Revised: In recent years, convolutional neural networks (CNNs) (Hinton et al., 2012; Simonyan & Zisserman, 2015; Szegedy et al., 2015; He et al., 2016) have become powerful and popular tools since deep learning emerged for image classification in the computer vision field.

Original: They have archived record-breaking performance versus traditional methods (Quinlan, 1986; Cortes & Vapnik, 1995) with handcrafted features (Dalal & Triggs, 2005; Lowe, 1999) on the ImageNet LargeScale Visual
Revised: They have recorded record-breaking performance and outperformed traditional methods (Quinlan, 1986; Cortes & Vapnik, 1995) with hand-crafted features (Lowe, 1999; Dalal & Triggs, 2005) on the ImageNet Large

Original:  Recognition Challenge (ILSVRC) (Deng et al., 2009).
Revised: Scale Visual Recognition Challenge (ILSVRC) (Deng et al., 2009).

Original: Today, convolutional neural networks still have unique merit, as they have been studied the most, along with the fact that convolution is clear to understand with a strong connection to human visual perception and image processing, which makes them good models for feature learning research.
Revised: Today, convolutional neural networks still possess unique merits, as they have been studied the most, along with the fact that convolution has a strong connection with the human vision system and image processing, making them good models for feature learning research.

Original: Different factors have been considered for better recognition performance of convolutional neural networks.
Revised: Different factors have been explored to improve recognition performance of convolutional neural networks.

Original: VGGNet (Simonyan & Zisserman, 2015) applies a small convolution kernel size of 3 ×for increased depth of network, while ResNet (He et al., 2016) introduces deep residual learning to make training very deep network feasible.
Revised: VGGNet (Simonyan & Zisserman, 2015) applies a small convolution kernel size ( 3 × 3 ) for increased network depth, while ResNet (He et al., 2016) introduces deep residual learning to make training very deep deep networks feasible.

Original: The success of such networks indicates that depthdimension is  crucial  for recognition performance.
Revised: The success of such networks indicates that depth is a crucial factor for recognition performance.

Original: Wide Residual Networks (Zagoruyko & Komodakis, 2016), on the other hand, demonstrate width as anotherdimension important  to improved performance.
Revised: Wide Residual Networks (Zagoruyko & Komodakis, 2016), on the other hand, demonstrate width as another important factor to improved performance.

Original: In addition to these factors, neurobiologyalso suggests the widely existing lateral inhibition (e.g., Mach band effect,which is shown in Fig. 1), which increases the contrast  of nearby neuron excitation in the lateral direction, as a practical way to help feature learning for better recognition.
Revised: In addition to these factors, neurobiology suggests the widely existing lateral inhibition (lateral antagonism, e.g. Mach band effect, shown in Fig. 1), a phenomenon that increases the contrast and sharpness of nearby neuron excitation in the lateral direction, also important to help feature learning.

Original: However, such an important mechanism has not been well explored inthe design of modern convolutional neural networks.
Revised: However, such an important mechanism has not been well explored in convolutional neural network design.

Original: Although there have been attempts to explore this property, the most influential one (Cao et al., 2018) in recent years relies on fixed rules to compute lateral inhibition using the pre-trained VGGNet model for attention map and weakly supervised saliency detection.
Revised: In recent years, Yang et al. (2015) propose a traditional method using manual doubleopponent design (antagonism) with spatial sparseness constraint for boundary detection, following the color-opponent mechanism in the human vision system, to highlight the color boundary while suppressing texture edges. This double-opponent design is further extended (Gao et al., 2015) for color constancy. Cao et al.

Revised: (2018) apply lateral inhibition mechanism in the inference stage of the pre-trained VGGNet (Simonyan & Zisserman, 2015) for visual attention map and saliency detection.

Original: Proper incorporation and modeling of  inhibition  is still needed during the training process formodern convolutional neural networks on large datasets to serve as backbone models.
Revised: Hasani et al. (2019) propose their surround modulation design, using a manually defined kernel from the difference of Gaussian (DoG) function, to explore the feasibility of incorporating the lateral inhibition mechanism. This fixed design is only applied to half channels of the initial layer (with better performance than applying to the full channels). It is still worth exploring flexible lateral inhibition-inspired design for convolutional neural network, to incorporate such mechanism and make it more similar to the brain in the training stage for image classification.

Original: In this paper,we particularly apply the lateral inhibition mechanism in modern convolutional neural networks, which is inspired by the findings of neurobiology. We explicitly explore the filter dimension in the lateral direction and propose our lateral inhibition-inspired (LI) design.
Revised: In this paper, inspired by the findings of neurobiology, we explicitly explore the filter dimension in the lateral direction and propose our lateral inhibition-inspired (LI) structure, which incorporates the lateral inhibition mechanism in a flexible manner inside modern convolutional neural networks.

Original: Our  naive design incorporates the low-pass filter, while eliminating the central weight  to mimic the  inhibition strength decay.
Revised: Our simple naive design applies the low-pass filter with the central weight eliminated, to mimic the decay of inhibition strength from neighbors regarding the distance.

Original: The inhibition value is computed from the filtering result of the input, with asimple learnableweight parameter per channel  formultiplication to decide the strength.
Revised: To set the amplitude of the low-pass filter by multiplication, a learnable parameter per channel is used, for the flexibility of modeling various lateral interactions (including lateral inhibition).

Original: Then the inhibition value is  subtracted from the input as suppression, which could increase the contrast to help recognition. We also suggest an alternative using depthwise convolution, as a general form.
Revised: The convolution result is then subtracted from the input, which could increase the contrast and sharpness for better recognition. Furthermore, a learnable scaling factor and shift are applied to adjust the value after subtraction.

Original: Our design could work on boththe plain convolution andthe convolutional block with  residual connection, while being compatible with  existing modules.
Revised: Our lateral inhibition-inspired (LI) method works on both plain convolution and convolutional block with a residual connection, while being compatible with the existing modules.

Original: Without any channel attention along the channel dimension, the preliminary resultsdemonstrate an absolute improvement of 3.68% and 0.69% over AlexNet and ResNet-18, respectively, in the ImageNet data set, with little increase in parameters, indicating the merits of our design to help feature learning for image classification.
Revised: Preliminary results of our simple naive design demonstrate obvious improvements on the ImageNet dataset for AlexNet (7.58%), ResNet-18 (0.81%), respectively, with little parameter increase, indicating the effectiveness of our brain-similar to help feature learning for image classification from a different perspective.

Original: Our main contributions can be summarized as:
Revised: Our main contributions can be summarized as:

Original: • To our best knowledge, we are the first to incorporate lateral inhibition into modern convolutional neural networks such as ResNet with little increase in parameters. • We archive obvious improvements on ImageNet dataset with our lateral inhibition design.
Revised: • We propose a lateral inhibition-inspired structure for modern convolutional neural network design, which could participate in the training stage for image classification.

Original: • We propose a novel design using a modified Gaussian low-pass filter for lateral inhibition decay, as well as a more flexible design using depthwise convolution.
Revised: • Our design is flexible compared to the fixed difference of Gaussian (DoG) filter, and could be applied to all layers (and channels) with competitive performance.

Original: • We visualize our learned filter for lateral inhibition in both designs and agree with the neurobiology findings, which could potentially lead to improved network  design.
Revised: • We are the few to explicitly model lateral inhibition as well as other lateral interaction to make the network more brain-similar, given the flexible weight design.

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original:  Lateral Inhibition Convolutional Neural Networks.
Revised: Neurobiology. Lateral inhibition is common in sensory system.

Revised: Mach reports the Mach bands effect RATLIFF (1965) due to lateral inhibition that the human vision system perceives increased contrast and sharpness along the boundary, where the dark area becomes darker and the bright area becomes brighter.

Revised: Blakemore et al.

Revised: (1970) discover that lateral inhibition plays an important role in orientation detection within the human vision system.

Revised: It helps to remove redundancy for the existence of inhibitory signals, as large stimulus covering a whole receptive field would be relatively ineffective in excitation of neurons.

Revised: Traditional method.

Original: Cao et al.
Revised: Yang et al.

Revised: (2015) propose their double-opponent design with spatial sparseness constraint for boundary detection, by mimicking the color-opponent mechanism in the human vision system.

Revised: The single opponent uses a manual antagonism such as red-green and blue-yellow, while the double opponent combines two opposite single opponent in either center-surround or sideby-side manner for increased contrast.

Revised: This design highlights the color boundary while suppressing texture edges.

Original: (2018) propose their LICNN with fixed implementation for lateral inhibition computation uisng pre-trained VGGNet, as top-down step for attention map and weakly supervised saliency detection.
Revised: The double-opponent design is further extended (Gao et al., 2015) for color constancy. Convolutional Neural Network. Local Response Normalization (LRN) (Hinton et al., 2012) is introduced in AlexNetalong and is not trainable. Normalization is performed for the value at each position (x, y) considering neighbors at the same position from nearby channels, with the denominator as a sum of the squared neighbors, given a scaling factor, bias, as well as the exponential value for the denominator.

Original: Their inhibition vlaue consists of an average term from the neighborhood and another term only from a higher activation nearby, and this inhibition formula including term coefficients are intuitively selected and unlearnable.
Revised: It applies lateral inhibition across nearby channels via competition. Batch normalization (Ioffe & Szegedy, 2015) applies normalization within individual channels, using an mini-batch mean (shift) and variance, then a scaling factor and re-shift parameter, to constrain all values in a certain range. Layer normalization (Ba et al., 2016) only consider one single instance, but performs normalization across all channels using the average and variance from each channel.

Original: The average term calculate the mean vaue within predefined neighborhood (7 × 7) to protect the inhibition from a high response zone.
Revised: Group normalization (Wu & He, 2018) further extends layer normalization by dividing channels into groups for consideration. Besides normalization methods, Cao et al. (2018) propose their lateral inhibition design using pretrained VGGNet, as top-down step for attention map and weakly supervised saliency detection.

Original: The other term only takes suppression fromany neighbor with higher activation and decay that suppression with distance. After lateral inhibition,nodes with activation below zero will not be considered for another forward feed. This could result in less noise and background removal.
Revised: The inhibition value consists of an average term from the neighbors within the same channel to protect high response zone, and a differential term only consider the difference from higher activation nearby for inhibition, with weight decay regarding the distance. The whole formula including term coefficients are intuitively decided and unlearnable, and only happens in inference stage.

Original: Ullah et al.
Revised: Hasani et al.

Original: (2021) propose their lateral inhibition design for remote sensing application, with multiple filters in different sizes (3 × 3 and 5 × 5) and perform subtraction using the small filter and the large filter for suppression.
Revised: propose surround modulation for lateral inhibition, to make network closer to the brain. It applies two gaussian filters (5x5) with 1:1 weight (fixed) from the difference of Gaussian (DoG) function.

Original: The limitation is it requires adding more filters to  the network for computation to archive lateral inhibition.
Revised: However, this fixed design lacks flexibility, which is only applied to half channels of the initial layer within a small convolutional neural network, due to the performance drop when applying to full channels or the following layer. The channelwise normalization (center pool) in Weighted normalization (Pan et al., 2021) improves LRN (Hinton et al., 2012) from AlexNet by adding extra parameters such as a scaling factor and bias, while further adding surround pool for normalization within each channel does not help on ImageNet dataset. DivisiveNorm (Miller et al., 2021) also similarly follows the design of LRN at each location along the channel dimension, but changes the numerator to be squared, and adds a weight scaling factor for each unnormalized activations inside the exponentially weighted sum. The hyper parameters are manually set for stable training.

Original:  M ETHOD
Revised:  M ETHOD

Original: Our method aims to explicitly incorporate the lateral inhibition mechanism into the modern convolutional neural network,so that suppression could be applied from active nodes to neighbors in the lateral direction, increasing contrast and sharpness for more accurate perception, as neurobiology indicates.
Revised: Our lateral inhibition-inspired (LI) structure explicitly incorporates the lateral inhibition mechanism into the design of convolutional neural network, to increase contrast and sharpness for more accurate perception.

Original: It is involved in training stage, not as post-processing using pre-trained models.
Revised: Unlike methods only involving in the post-processing stage with the original pre-trained model unchanged, our method participates in the training stage.

Original: Our method takes two steps: first generating the lateral inhibition value based on the input and then decreasing that value from the  input. We propose naive inhibition to mimic neurobiology findings, and a more flexible alternative as a general form.
Revised: Our lateral inhibition-inspired (LI) design takes two steps: first, it generates the lateral interaction value based on the input within the spatial neighborhood of each channel; then it subtracts that value from the original input. Details will be explained in the following parts.

Original: 3.  I NHIBITION D ESIGN3.1. N AIVE D ESIGN FOR I NHIBITION
Revised: 3. L ATERAL I NTERACTION D ESIGN

Original: We design our naive inhibition according to neurobiology findings.
Revised: We design our naive lateral interaction based on neurobiological findings.

Original: It follows  that inhibition of  an active neuron decreases as  distance goes further from that neuron  in the lateral direction.
Revised: It follows the principle that the strength of interaction (including inhibition) from an active neuron decreases as the distance to that neuron increases in the lateral direction.

Original: For implementation, we take the exsiting Gaussian low-pass filter withweights decay from the center as the fundation, and eliminate the central weight since no suppression comes from itself.
Revised: For implementation, we take the exsiting Gaussian low-pass filter with central weight eliminated to mimic the strength decay from neighbors.

Original: Here, we denote the function ofthe lateral inhibition value as LI ,the Gaussian low-pass filter as G , the central weight  of  G as C ,the convolution  as Conv , the learnable weight W with a single parameter per channel for multiplication to  decide the inhibition strength, and the input features as x before lateral inhibition.
Revised: Here, we denote the function of lateral interaction as LI , Gaussian low-pass filter as G , the central weight value of the low-pass filter G as C , convolution operation as Conv , the learnable amplitude weights as W (with one parameter per channel for multiplication to separately decide the channelwise amplitude of Gaussian low-pass filter G ), and the input features as x for our lateral inhibition-inspired structure.

Original: The weights for the 2D Gaussian low-pass filter ( G ) are calculated using Equation 1 (with normalization).
Revised: The weights of the 2D Gaussian low-pass filter ( G ) are calculated using Equation 1.

Original: Here u and v denote the spatial coordinates regarding to the center of the  filter, and σ denotes the standard deviation which decides the speed of decay  (we set filter size as 3 and σ as 1):
Revised: Here u and v denote the spatial coordinates regarding to the center of the Gaussian low-pass filter, and σ denotes the standard deviation for weight decay speed from the center (we set filter size as 3 and σ as 1):

Original: [Equation]
Revised: [Equation]

Original: The lateral inhibition value for the  channel i is calculated using Equation 2, by eliminating the central weight value C of theGaussian low-pass filter G first, to determine no suppression coefficient for itself, then multiplication with the learnable weight for the channel i .
Revised: The lateral interaction function given the input feature x at channel i is defined in Equation 2. First, convolution on input feature using Gaussian low-pass filter G is applied, then the corresponding convolution result form the central weight value C (of the low-pass filter G ) is subtracted, to only have interaction from the neighbors. Finally, multiplication is performed by the amplitude parameter (from the learnable weights W ) to decide the amplitude of the Gaussian low-pass filter G per channel.

Revised: Different from Surround Modulation (Hasani et al., 2019), which applies 1:1 weight using two

Revised: Gaussian low-pass filters in the difference of Gaussian (DoG) function to model lateral inhibition, our learnable weights W is more flexible (could be zero, positive or negative) to fit various lateral interactions (for none, lateral inhibition or even the opposite when necessary).

Original: [Equation]
Revised: [Equation]

Original: 3.1. F LEXIBLE D ESIGN FOR I NHIBITION
Revised: 3. A PPLYING L ATERAL I NTERACTION

Original: Wealso propose a more flexible alternative with depthwise convolution to replace the modified Gaussian low-pass filter as well as the learnable weight for multiplication. Here, we denote the depthwise convolution as DW .
Revised: We denote the output function as O for applying lateral interaction on input feature x at channel i .

Original: Therefore, the filter from the depthwise convolution at channel i will decide both the shape and the strength of the lateral inhibition function for that channel i , as shown in Equation 3. This is a general form.
Revised: Then the output function is defined in Equation 3, with lateral interaction result subtracted.

Original: [Equation]
Revised: [Equation]

Original: Although we do not manually control how to learn it, visualization of the filter coefficient from the depthwise convolution indicates such inhibition has been learned.

Original: 3. O UTPUT AFTER I NHIBITION
Revised: 3. A DDITIONAL PARAMETERS

Original: We denote the output function after inhibition as O , then the output of channel i is the input in channel i with the inhibition value subtracted, as shown in Equation 4.
Revised: Additional parameters are applied since the interaction result will be subtracted from the original feature map. Parameters include a scaling factor ( v ), shift ( m ), and bias ( b ). Combination of these parameters can be explored to find the best setting.

Original: [Equation]
Revised: [Equation]

Original: 3. I NHIBITION P LACE
Revised: 3. S TRUCTURE P LACE

Original: Our lateral inhibition works on  plain convolution as well as convolutional blocks with residual connection.
Revised: Our lateral inhibition-inspired (LI) structure works on both the plain convolution and the convolutional block with residual connection.

Original: For  plain convolution,we place it right after that convolution, as shown in Fig.
Revised: For the plain convolution, it is placed after that convolution (and the activation function), as shown in Fig.

Original: For  convolutional blocks with  residual connection,we place it on the input flow right after the identity saving operation, as shown in Fig.
Revised: For the convolutional block with a residual connection, it is placed inside the residual block before the first convolution, as shown in Fig.

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: Dataset.
Revised: Dataset.

Original: We use the ImageNet-1k data set (Russakovsky et al., 2015) to evaluate our proposed methods.
Revised: We use the ImageNet-1k data set (Russakovsky et al., 2015) to evaluate our proposed methods.

Original: ImageNet-1k dataset consists of 1k object categories, with 1.2 million images for training and 50k images for validation.
Revised: ImageNet-1k dataset consists of 1k object categories, with 1.2 million images for training and 50k images for validation.

Original: Experiment Settings.
Revised: Experiment Settings.

Original: We use the Pytorch (Paszke et al., 2019) deep learning framework and follow the official training example.
Revised: We use the Pytorch (Paszke et al., 2019) deep learning framework and follow the official training example.

Original: We adopt  standard settings: image crop size of 224 × 224, initial learning rate of 0.1 (which decays every 30 epochs by a factor of 10), momentum of 0.9, and weight decay of 0.0001.
Revised: We adopt the standard settings: image crop size of 224 × 224, initial learning rate of 0.01 for AlexNet (Hinton et al., 2012) (which decays every 30 epochs by a factor of 10), momentum of 0.9, and weight decay of 0.0001.

Original: We use standard data augmentation from the training example, which includes random horizontal flipping,with random scale and ratio change in the range of [0.08, 1.0] and [0.75, 1.33], respectively.
Revised: We use standard data augmentation, which includes random horizontal flipping, random scale and ratio changes in the range of [0.08, 1.0] and [0.75, 1.33], respectively.

Original: We train each model with batch size of 128 and batch accumulation of 2, forepochs on 4 GPUs.
Revised: Models are trained with batch size of 128 and batch accumulation of 2. Experiments are performed on 4 GPUs, with 90 epochs for AlexNet following Miller et al.

Revised: (2021) (more epochs show little difference) and 100 epochs for ResNet following common practice.

Original: Results on ImageNet.
Revised: Results.

Original: Table 1 shows the results of  AlexNet (Hinton et al., 2012) and our naive lateral inhibition-inspired (LI) design.
Revised: Table 1 shows the results of the baseline model AlexNet (Hinton et al., 2012), related methods, as well as our naive lateral inhibition-inspired (LI) design on ImageNet dataset.

Original: This naive design uses the low-pass filter while eliminating the central weight for inhibition decay. It is very clear that our design brings asignificant improvement with an absolute value of 3.68% on this large dataset. This demonstrates the effectiveness of our [Table]
Revised: While all listed methods improve the baseline significantly, our method outperforms others by a large margin.

Original: [Table] lateral inhibition, which obviously benefits feature learning on this network with plain convolution to a large extent.
Revised: Layer normalization only consider one instance, which fits better for dynamic input rather than batch input.

Original: Table 2 shows the results of ResNet (He et al., 2016) and our lateral inhibition-inspired (LI) design in both naive and flexible versions. With tiny parameter increase (only one parameter per channel for multiplication), our naive design archives an absolute improvement of 0.69% on this large dataset. This is impressive given the reference improvement of 0.94% from Squeeze-and-Excitation (SE) channel attention (Hu et al., 2018), which requires much more parameter increase.
Revised: Group normalization has further division into groups, therefore limits its performance.

Original: Surprisingly, our flexible design has an absolute boost of 1.23%, with a negligible increase inparameter. Here the performance andparameter increase are compared with the baseline ResNet. Inhibition Filter Weight.
Revised: Weighted normalization and Divisive normalization are similar in the normalization design, except the power number in the numerator and the denominator, as well as the weighted sum.

Original: We manually check the  lateral inhibition filter weight. For our naive design, mostly positive for the value to be multiplied with the modified Gaussian low-pass filter.
Revised: However, most of them lack the consideration of neighbors within the same channel for lateral interaction, which our method does.

Original: That weight value mostly is in range of (0, 2), with the setting of filter size of 3 and standard deviation of 1, we could see the inhibition coefficients are positive at the surrounding but negative at the center, given Equation 2.
Revised: Table 3 shows the results of the baseline model

Original: Similarly in the flexible design, we could also spot such pattern in even numbered layers.
Revised: ResNet (He et al., 2016) and our naive lateral inhibition inspired (LI) design on ImageNet dataset.

Original: A classic pattern is provided below in Equation 5, and the inhibition value needs to be substracted from the original input. This pattern is also agreed with neurobiology findings. [Equation]
Revised: Although ResNet contains batch normalization to improve performance, our design makes the improvement further by 0.81%.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: In this paper, we propose our lateral inhibition-inspired design on convolutional neural networks for image classification.
Revised: In this paper, we propose our lateral inhibition-inspired structure for convolutional neural network to make it more brain-similar on image classification task.

Original: We provide a naive design which is based on a Gaussian low-pass filter for inhibition decay and a more flexible alternative using depthwise convolution as a general form.
Revised: We explicitly explore neurobiology findings, using the guassian low-pass filter with central weight eliminated to mimic the strength decay regarding the distance. With a learnable weight, our deign is flexible to model lateral inhibition as well as other lateral interaction.

Original: Our design works on both plain convolution and  convolutional blocks with residual connection.
Revised: Our design works on both plain convolution and the convolutional block with residual connection, while being compatible with the existing modules.

Original: Preliminary results using AlexNet and ResNet demonstrate obvious improvementswith little increase in parameter on  ImageNet dataset for image classification.
Revised: Preliminary results using AlexNet and ResNet demonstrate obvious improvements on the ImageNet dataset for image classification, with little increase in parameters.

Original: We hope that our research could inspire researchers to reconsider the value of this track for better feature learning.
Revised: We hope that our research could inspire researchers to reconsider the value of this track for better feature learning.

Original: A A PPENDIX
Revised: [Table] Our is

Original: You may include other additional sections here
Revised: [Table]