# High-Frequency Synchronization Improves Firing Rate Contrast and Information Transmission Efficiency in E/I Neuronal Networks

Authors: Fang Han; Zhijie Wang; Hong Fan; Yaopeng Zhang
Journal: Neural Plasticity
 (2020)

---

## Abstract

High-frequency synchronization has been found in many real neural systems and is confirmed by excitatory/inhibitory (E/I) network models. However, the functional role played by it remains elusive. In this paper, it is found that high-frequency synchronization in E/I neuronal networks could improve the firing rate contrast of the whole network, no matter if the network is fully connected or randomly connected, with noise or without noise. It is also found that the global firing rate contrast enhancement can prevent the number of spikes of the neurons measured within the limited time window from being confused by noise, thereby enhancing the information encoding efficiency (quantified by entropy theory here) of the neuronal system. The mechanism of firing rate contrast enhancement is also investigated. Our work implies a possible functional role in information transmission of high-frequency synchronization in neuronal systems.


---

## Body

## 1. Introduction

High-frequency synchronization of neural firing is believed to be one possible origination of high-frequency brain oscillations and gamma oscillations, which reside in the frequency band between 30 and 90 hertz and are found in many regions of the cerebral cortex [1]. Synchronization of neural firing may be relevant to some disorders of neural systems [2], but more importantly, the high-frequency oscillations (synchronization) are found to be related with many cognitive behaviors such as the processing of sensory signals, learning and memory, and attention in many biological experiments [3–7]. To reveal the underlying mechanisms of functional roles of the high-frequency synchronization (oscillations) on these cognitive behaviors, neural network models are constructed. Some functional roles of the high-frequency oscillations are hypothesized and tested with neural network models. One of the possible functional roles of the high-frequency oscillations is the feature binding [8–10]. Various features of the object such as shape, color, and motion are represented by different groups of neurons in different regions of the cerebral cortex. These groups of neurons fire synchronously, thereby distinguishing them from other neurons in the cerebral cortex. Thus, the corresponding features encoded in these synchronized firing of neurons are bound together to form a complete perceptual construct. Another possible function of the high-frequency oscillation is stimulus selection [11–14]. More coherent stimuli oscillating at high frequency are found to have a competitive advantage over less coherent ones; thus, the coherent stimuli are selected to pass down the neural systems.Though these proposed functions of the high-frequency oscillation (synchronization) have been tested by neural network models, they are not producing satisfactory results due to the lack of relevant data and the oversimplicity of these models. Since neural systems are usually composed of both excitatory and inhibitory neurons, it is helpful to have a closer look at the mechanism of the genesis of high-frequency synchronization in E/I networks before we explore its functional role. It is widely believed that ING (interneuron gamma) and PING (pyramidal and interneuron gamma) are two possible mechanisms of the generation of high-frequency synchronization [15]. ING mechanism [16–19] suggests that interneurons (inhibitory neurons) themselves can induce synchronization, and this synchronization of inhibitory neurons makes excitatory neurons fire synchronously through the inhibitory synapses on excitatory neurons. However, as excitatory neurons have excitatory synapses on inhibitory neurons as well, excitatory neurons also affect the firing of the inhibitory neurons. For example, excitatory neurons with sensory input encoded in them may drive inhibitory neurons to fire more frequently through excitatory synapses [20]. The PING mechanism [21] supposes that inhibitory neurons themselves cannot generate high-frequency synchronization. It is the interaction of excitatory drive and inhibition that induces the synchronization. Furthermore, it is believed that only a fraction of neurons fires at each cycle of the synchronization [22, 23]. Therefore, a fraction of excitatory neurons fires at each cycle of the synchronization, which provides synaptic current helping to drive inhibitory neurons to fire. The firing of inhibitory neurons inhibits the firing of all neurons in the network. This reminds us of the lateral inhibition [24–26]. Lateral inhibition suggests that the firing of excitatory neurons drives the firing of the neighboring inhibitory neurons, which inhibits the firing of the neighboring excitatory neurons in turn. Inspired by this idea, in Ref. [27], we found that inhibitory neurons could act as a global recurrent inhibition to enhance the firing rate contrast of the whole neuronal network. This further reminds us that improving firing rate contrast could be a possible function of neural synchronization, and as information may be encoded in the firing rate of neurons [28], neural synchronization could contribute to information transmission of neural systems.We proposed and confirmed this global contrast enhancement mechanism based on the high-frequency synchronization in the E/I network in this paper. More importantly, we confirmed by entropy theory [29–31] that the firing rate contrast enhancement can prevent the numbers of firings of the neurons measured within a limited time window from being confused by noise, thereby enhancing the information encoding efficiency of neural systems. This paper is organized as follows. Model and Methods provides the E/I network model and the methods for quantifying the neuronal information transmission efficiency. Simulation Results presents simulation and quantitative results on the enhancement of the firing rate contrast with relatively simple E/I networks. Mechanism of the Global Contrast Enhancement gives a qualitative analysis of the mechanism of the enhancement of firing rate contrast in E/I networks. Conclusions and Discussions summarizes the results and gives some discussions.


## 2. Model and Methods

The information encoded in the sensory neurons passes through the neural systems layer by layer. It is expected that information can be transmitted through the noisy neural systems with less information loss and less energy consumption. We proposed a method that is based on the entropy theory to quantify the neuronal information transmission efficiency. Suppose there is a group of neurons with population sizeNe (see the excitatory neurons in Figure 1). Each neuron, for example, neuron i, receives an input IiS, the firing rate of which, ri, is measured by limT→∞ki/T where ki is the number of spikes emitted by the neuron during period T. When T is finite, ri is usually a random number due to the uncertainty of the internal and external environment of the neuron. We hope that for any IiS≠IjS, ri≠rj, i.e., for any IiS≠IjS, ki≠kj. However, ki may equal kj due to the limited time window T and the noisy nature of the neural system. This will lead to the misidentification of the input values of IiS and IjS. In other words, the input values IiS and IjS will be mixed or wrongly encoded in the output of neurons i and j. To avoid such confusion, the contrast of the firing rates of neurons i and j should be enhanced. We found in this paper that in an excitatory/inhibitory (E/I) network, the high-frequency synchronization of inhibitory interneurons can globally enhance the firing rate contrast of excitatory neurons, the model of which is described in Model of E/I Network. On the other hand, we hope that the neural system can transmit more information with less energy consumption, i.e., we hope neural systems have high information transmission efficiency. To characterize this efficiency, we propose a method based on the entropy theory, which is described in Methods for Quantifying Neuronal Information Transmission Efficiency.Figure 1
The structure of the E/I network.


### 2.1. Model of E/I Network

The network consists of one population of spiking excitatory neurons with sizeNe and one population of spiking inhibitory neurons with size Ni. Excitatory and inhibitory neurons receive synaptic inputs from both excitatory and inhibitory neurons. Neurons in the network connect to other neurons with a probability ρ. We define WEI=wi,jEINi×Ne as the synaptic connection matrix from excitatory neurons to inhibitory neurons, where “E” in the superscript is the abbreviation of “excitatory neurons” and “I” is the abbreviation of “inhibitory neurons.” If there is a synapse from excitatory neuron j to inhibitory neuron i, wi,jEI=gEI with gEI being the strength of synapse; otherwise, wi,jEI=0. The definitions for WIE=wi,jIENe×Ni, wi,jIE, gIE, WEE=wi,jEENe×Ne, wi,jEE, gEE, WII=wi,jIINi×Ni, wi,jII, and gII are similar to those of WEI=wi,jEINi×Ne, wi,jEI, and gEI.The dynamics of the excitatory and inhibitory neurons are described as follows [29, 32].(1)τmEdviEtdt=−viEt−VL−REIiEEt−REIiIEt+REIiSt,(2)τmEdviItdt=−viIt−VL−RIIiEIt−RIIiIIt+RIIiBt.Here,viEt and viIt are the membrane potentials for the excitatory neurons and inhibitory neurons, respectively. When reaching the threshold potential Vth, the neuron emits a spike, and the membrane potential returns to the rest potential Vrest. τmE and τmI are the time constants of the membrane for the excitatory neurons and inhibitory neurons, respectively. RE and RI are the membrane resistances. VL is the balance potential associated with the leak current. IiEEt is the summation of the synaptic current from all excitatory neurons to excitatory neuron i, and the definitions for IiEIt, IiIEt, and IiIIt are similar to IiEEt. They can be described as in(3)IiEEt=∑jwi,jEEIi,jEEt,IiEIt=∑jwi,jEIIi,jEIt,IiIEt=∑jwi,jIEIi,jIEt,IiIIt=∑jwi,jIIIi,jIIt,where Ii,jEEt is the synaptic current resulting from all the spikes of neuron j [29]:(4)Ii,jEEt=∑kβj,kt,τdE,τrEviEt−EsynE,(5)βj,kt,τdE,τrE=0,t<tj,k+d,1τdE−τrEe−t−tj,k−d/τdE−e−t−tj,k−d/τrE,t≥tj,k+d,where tj,k is the firing time of the k-th spike of neuron j; d is the synaptic delay, τdE and τrE are the decay time constant and rise time constant of the excitatory synaptic current, respectively; and EsynE is the reverse potential of the excitatory synapse.Similar to the definition ofτdE, τrE, and EsynE, we denote τdIτrI as the decay time constant (rise time constant) and EsynI as the reverse potential of the inhibitory synapse. Correspondingly, the descriptions of Ii,jIEt, Ii,jEIt, and Ii,jIIt can be given similarly to that of IiEEt.IiSt is the stimulus presented to the excitatory neuron i. IiSt consists of two parts as follows:(6)IiSt=ciS1+εit,where ciS is the strength of the stimulus. It is a constant, but each neuron has a different strength, i.e., ciS≠cjS. ciS is drawn from a normal distribution with standard deviation δS and mean mS. εit is white noise, which obeys a normal distribution with standard deviation σNS and a mean of zero. The parameter δS controls the contrast of the strength of the stimuli. Larger δS means higher contrast of the strength of the stimuli.Similar toIiSt, the background input to the inhibitory neurons IiBt is defined; thereby, we have parameters δB, mB, and δNB for IiB. The excitatory neurons may have background input as inhibitory neurons, and here, we assume that IiSt includes this background input.


### 2.2. Methods for Quantifying Neuronal Information Transmission Efficiency

To simplify the descriptions of the method, assume that the inputs of the neurons are images, which are denoted byX=xi,i=1,2,⋯,Ne, where X represents the image presented to the neurons, Ne is the number of pixels in an image, and xi is the gray value of the i-th pixel in the image. The output of the neurons represents the output image with the number of firings of neuron i during time window T. ki stands for the gray value of the corresponding pixel. The images are presented to the neurons by letting IjS=xj. Suppose that the input xi is discrete (continuous values of the gray value can be discretized following the degree of the resolution in the neural systems), i.e., xi∈0,Δx,2Δx,⋯,MΔx where Δx is the resolution of the discretization. Let us denote sj=jΔx. Let Lgi,j=1 if the gray value of the i-th pixel is jΔx; otherwise, Lgi,j=0. Let psj=∑i=1NeLgi,j/Ne be the “probability” (proportion) of the pixels whose gray value is jΔx. Then, the entropy of the input image (stimulus) can be calculated as SS=−∑j=1Mpsjlogpsj. Let pk be the “probability” (proportion) of neurons whose spiking number is k. Then, the full entropy of the output image is defined as Sfull=−∑k=0∞pklogpk. If for any pair of neurons with IiS≠IjS, ki≠kj, i.e., if the different inputs of any two neurons can be distinguished by their outputs, then the mutual information, SI, will be equal to the entropy of the input image, SI=SS. This means that the information is transmitted from the input side of the neural system to the output side of the neural system without any loss. However, as we discussed in the first paragraph of Model and Methods, ki may be equal to kj when IiS≠IjS, which results in SI<SS, implying that some information is lost due to the finite time window and noisy nature of the neuronal channels. The corresponding noise entropy, which is denoted by Snoise, can be calculated as Snoise=−∑j=1M∑k=0∞psjpk∣sjlogpk∣sj according to the entropy theory [29]. Here, pk∣sj is defined as pk∣sj=C1/C2, where C1 is the count of neurons whose spike number is k and input is sj, and C2 is the count of neurons whose input is sj. The mutual information is the full entropy subtracted by noise entropy, SI=Sfull−Snoise.Energy consumption [33, 34] should also be considered when neural systems encode information with firing rates [35–37]. It is desired that neural systems carry more information with less energy consumption. Metabolic energy is usually considered when encoding efficiency is investigated, which is quantified by the number of spikes employed to encode the information. Then, the information efficiency is calculated as IE=SI/FS, where FS is the average number of spikes of neurons.


## 2.1. Model of E/I Network

The network consists of one population of spiking excitatory neurons with sizeNe and one population of spiking inhibitory neurons with size Ni. Excitatory and inhibitory neurons receive synaptic inputs from both excitatory and inhibitory neurons. Neurons in the network connect to other neurons with a probability ρ. We define WEI=wi,jEINi×Ne as the synaptic connection matrix from excitatory neurons to inhibitory neurons, where “E” in the superscript is the abbreviation of “excitatory neurons” and “I” is the abbreviation of “inhibitory neurons.” If there is a synapse from excitatory neuron j to inhibitory neuron i, wi,jEI=gEI with gEI being the strength of synapse; otherwise, wi,jEI=0. The definitions for WIE=wi,jIENe×Ni, wi,jIE, gIE, WEE=wi,jEENe×Ne, wi,jEE, gEE, WII=wi,jIINi×Ni, wi,jII, and gII are similar to those of WEI=wi,jEINi×Ne, wi,jEI, and gEI.The dynamics of the excitatory and inhibitory neurons are described as follows [29, 32].(1)τmEdviEtdt=−viEt−VL−REIiEEt−REIiIEt+REIiSt,(2)τmEdviItdt=−viIt−VL−RIIiEIt−RIIiIIt+RIIiBt.Here,viEt and viIt are the membrane potentials for the excitatory neurons and inhibitory neurons, respectively. When reaching the threshold potential Vth, the neuron emits a spike, and the membrane potential returns to the rest potential Vrest. τmE and τmI are the time constants of the membrane for the excitatory neurons and inhibitory neurons, respectively. RE and RI are the membrane resistances. VL is the balance potential associated with the leak current. IiEEt is the summation of the synaptic current from all excitatory neurons to excitatory neuron i, and the definitions for IiEIt, IiIEt, and IiIIt are similar to IiEEt. They can be described as in(3)IiEEt=∑jwi,jEEIi,jEEt,IiEIt=∑jwi,jEIIi,jEIt,IiIEt=∑jwi,jIEIi,jIEt,IiIIt=∑jwi,jIIIi,jIIt,where Ii,jEEt is the synaptic current resulting from all the spikes of neuron j [29]:(4)Ii,jEEt=∑kβj,kt,τdE,τrEviEt−EsynE,(5)βj,kt,τdE,τrE=0,t<tj,k+d,1τdE−τrEe−t−tj,k−d/τdE−e−t−tj,k−d/τrE,t≥tj,k+d,where tj,k is the firing time of the k-th spike of neuron j; d is the synaptic delay, τdE and τrE are the decay time constant and rise time constant of the excitatory synaptic current, respectively; and EsynE is the reverse potential of the excitatory synapse.Similar to the definition ofτdE, τrE, and EsynE, we denote τdIτrI as the decay time constant (rise time constant) and EsynI as the reverse potential of the inhibitory synapse. Correspondingly, the descriptions of Ii,jIEt, Ii,jEIt, and Ii,jIIt can be given similarly to that of IiEEt.IiSt is the stimulus presented to the excitatory neuron i. IiSt consists of two parts as follows:(6)IiSt=ciS1+εit,where ciS is the strength of the stimulus. It is a constant, but each neuron has a different strength, i.e., ciS≠cjS. ciS is drawn from a normal distribution with standard deviation δS and mean mS. εit is white noise, which obeys a normal distribution with standard deviation σNS and a mean of zero. The parameter δS controls the contrast of the strength of the stimuli. Larger δS means higher contrast of the strength of the stimuli.Similar toIiSt, the background input to the inhibitory neurons IiBt is defined; thereby, we have parameters δB, mB, and δNB for IiB. The excitatory neurons may have background input as inhibitory neurons, and here, we assume that IiSt includes this background input.


## 2.2. Methods for Quantifying Neuronal Information Transmission Efficiency

To simplify the descriptions of the method, assume that the inputs of the neurons are images, which are denoted byX=xi,i=1,2,⋯,Ne, where X represents the image presented to the neurons, Ne is the number of pixels in an image, and xi is the gray value of the i-th pixel in the image. The output of the neurons represents the output image with the number of firings of neuron i during time window T. ki stands for the gray value of the corresponding pixel. The images are presented to the neurons by letting IjS=xj. Suppose that the input xi is discrete (continuous values of the gray value can be discretized following the degree of the resolution in the neural systems), i.e., xi∈0,Δx,2Δx,⋯,MΔx where Δx is the resolution of the discretization. Let us denote sj=jΔx. Let Lgi,j=1 if the gray value of the i-th pixel is jΔx; otherwise, Lgi,j=0. Let psj=∑i=1NeLgi,j/Ne be the “probability” (proportion) of the pixels whose gray value is jΔx. Then, the entropy of the input image (stimulus) can be calculated as SS=−∑j=1Mpsjlogpsj. Let pk be the “probability” (proportion) of neurons whose spiking number is k. Then, the full entropy of the output image is defined as Sfull=−∑k=0∞pklogpk. If for any pair of neurons with IiS≠IjS, ki≠kj, i.e., if the different inputs of any two neurons can be distinguished by their outputs, then the mutual information, SI, will be equal to the entropy of the input image, SI=SS. This means that the information is transmitted from the input side of the neural system to the output side of the neural system without any loss. However, as we discussed in the first paragraph of Model and Methods, ki may be equal to kj when IiS≠IjS, which results in SI<SS, implying that some information is lost due to the finite time window and noisy nature of the neuronal channels. The corresponding noise entropy, which is denoted by Snoise, can be calculated as Snoise=−∑j=1M∑k=0∞psjpk∣sjlogpk∣sj according to the entropy theory [29]. Here, pk∣sj is defined as pk∣sj=C1/C2, where C1 is the count of neurons whose spike number is k and input is sj, and C2 is the count of neurons whose input is sj. The mutual information is the full entropy subtracted by noise entropy, SI=Sfull−Snoise.Energy consumption [33, 34] should also be considered when neural systems encode information with firing rates [35–37]. It is desired that neural systems carry more information with less energy consumption. Metabolic energy is usually considered when encoding efficiency is investigated, which is quantified by the number of spikes employed to encode the information. Then, the information efficiency is calculated as IE=SI/FS, where FS is the average number of spikes of neurons.


## 3. Simulation Results

### 3.1. Firing Patterns in a Fully Connected Network

We first use a relatively simple E/I network to focus our presentation on our idea of global contrast enhancement resulting from the high-frequency synchronization, where neurons are fully connected and the background inputs to the inhibitory neurons are homogeneous. The parameter values are set similar to Ref. [29], which are set as follows unless otherwise stated: ρ=1, Ne=450, Ni=150, gEI=gEE=1.2/Ne+Ni, gIE=gII=7/Ne+Ni, τmE=50ms, τmI=10ms, RE=RI=1MΩ, VL=−65mV, τdE=τdI=6ms, τrE=τrI=0.1ms, EsynE=0mV, EsynI=−85mV,d=0.2ms,δS=0.5mS=4, δNS=0.5,δB=0, mB=2.4, σNB=0, and T=300ms. It should be notified that the numbers of excitatory neurons and inhibitory neurons can be changed (the ratio of the two types of neurons falls in a range around 3 : 1~4 : 1 according to Ref. [38], and we choose 3 : 1 in this paper). If the ratio of excitatory neurons and inhibitory neurons is changed, the excitatory and inhibitory synaptic strengths should also be suitably changed to maintain the balance of excitation and inhibition in the network so that the network still oscillates synchronously.Figure2 shows the raster plots of the firing of neurons in the E/I network, where neurons labeled from 1 to 450 are excitatory neurons and neurons labeled from 451 to 600 are inhibitory neurons. Inhibitory neurons fire synchronously and act as the globally inhibitory signal to all the neurons in the E/I network. In each cycle of synchronization, a fraction of excitatory neurons fires, and it is the firings of these neurons that drive the inhibitory neurons to fire in this cycle of synchronization. Neurons with larger input will fire more frequently than those with smaller input, i.e., neurons with larger input will fire in more cycles of synchronization. Thus, the input values are encoded in the firing rates (the number of spikes) of these excitatory neurons.Figure 2
High-frequency oscillation in E/I networks with (a) no noisy input to the excitatory neurons,σNS=0, and (b) noisy input to the excitatory neurons, σNS=0.5.
(a)(b)


### 3.2. Contrast Enhancement in a Fully Connected Network

We first investigate the firing rate contrast in the fully connected network when randomness is not included in the stimulus to the excitatory neurons, i.e.,σNS=0. Figure 3(a) shows the number of spikes of all neurons when T=300 ms in such cases. It is very interesting and unexpected to see that the number of spikes of all the neurons scattered between 0 and 6, as the inputs of all the neurons are clustered around 4 (see Figure 3(c)). For comparison, Figure 4(a) shows the number of spikes of all the neurons when neurons do not connect with each other, i.e., each neuron is isolated and only receives the external input IiSt in Equation (1), and the synaptic inputs IiEEt and IiIEt are both 0. We can see from Figure 4(a) that the number of spikes of isolated neurons is closely around 8 and 9. Stated in other words, the contrast of the number of spikes of excitatory neurons in the E/I network is enhanced compared with that of the isolated neurons. If we measure the number of spikes when T is large enough, we can get the firing rate of each neuron, ri≈ki/T. Figures 3(b) and 4(b) visualize the firing rates of 400 excitatory neurons in the E/I network and a population of isolated neurons, respectively, where the gray value of the small square at the position (i,j) represents the firing rate of the neuron with index 20∗i−1+j. It shows clearly that the gray contrast of Figure 3(b) is much larger than that of Figure 4(b), implying that the firing rate contrast of excitatory neurons in the E/I network is much larger than those in the population of isolated neurons.Figure 3
Firing rates of all excitatory neurons in E/I networks with no noise,σNS=0: (a) spiking numbers of all excitatory neurons within 300 ms; (b) visualization of the firing rates of neurons 1-400 with gray values located at (i, j) representing the firing rate of neurons 20∗i−1+j; (c) inputs of neurons 1-450.
(a)(b)(c)Figure 4
Firing rates of all excitatory neurons of a population of isolated excitatory neurons with no noise,σNS=0: (a) spiking numbers of all excitatory neurons within 300 ms; (b) visualization of the firing rates of neurons 1-400.
(a)(b)The enhancement of the contrast of the firing rates of excitatory neurons in E/I networks when randomness is included is also confirmed in Figures5 and 6, where the spiking numbers of neurons within time window T and visualization of the firing rates are shown. Randomness is introduced to the network by setting σNS=0.5. Figures 7(a) and 7(b) further show the histograms of the firing rates of the excitatory neurons in the E/I network and in the population of isolated neurons, respectively. The histogram of Figure 7(b) is more homogeneous than that of Figure 7(a), also confirming that the firing rate contrast of excitatory neurons in the E/I network is much larger than that in the population of isolated neurons.Figure 5
Firing rates of all excitatory neurons in E/I networks with noisy input to excitatory neurons,σNS=0.5: (a) spiking numbers of all excitatory neurons within 300 ms; (b) visualization of the firing rates of neurons 1-400.
(a)(b)Figure 6
Firing rates of all excitatory neurons of a population of isolated excitatory neurons with noisy input to excitatory neurons,σNS=0.5: (a) spiking numbers of all excitatory neurons within 300 ms; (b) visualization of the firing rates of neurons 1-400.
(a)(b)Figure 7
Histograms of the firing rates of all excitatory neurons in (a) a population of isolated excitatory neurons with noisy input and (b) E/I networks with noisy input.
(a)(b)To get a clearer watch of this contrast enhancement, we arbitrarily choose several neurons between Figures3(a) and 4(a), say neurons 6 to 15, and display the firing rates of the ten neurons in Table 1. We can see that the number of spikes of the isolated neurons only has three values, 7, 8, or 9. Some neurons have an identical number of spikes though their inputs are somewhat different. Namely, the number of spikes of some neurons overlaps, resulting in a failure of the encoding of the inputs of these neurons. For example, both neuron 6 and neuron 15 have 8 spikes though they have different inputs. This is because the contrast of the firing rates of the two neurons (neuron 6 and neuron 15 have firing rates of 25.3 and 27.3, respectively) is so small that the number of the spikes within a small time window T cannot be distinguished. On the other hand, as the contrast of the firing rates of the excitatory neurons in the E/I network is enhanced, the problem of this overlapping is alleviated. The number of spikes of these neurons in the E/I network has more individual values (0, 1, 2, 3, 4, or 5) than those (7, 8, or 9) in a population of isolated neurons (notice that as the synaptic inhibition in the E/I network is stronger than excitation, the number of spikes fired by neurons in the E/I network is less than those in the population of isolated neurons). The numbers of spikes of neuron 6 and neuron 15 are no longer identical (they have 1 and 3 spikes, respectively); this is because they have a relatively large contrast of the firing rates (they have firing rates of 5.3 and 12.7, respectively). Therefore, thanks to the enhancement of the contrast of the firing rates, the output overlapping problem is somewhat relieved. Correspondingly, it is found that the information transmission efficiency is much higher in the E/I network (it is 0.54 in the E/I network of Figure 3(a)) than that in the population of isolated neurons (it is 0.13 in the population of isolated neurons of Figure 4(a)).Table 1
Spiking numbers of 10 isolated neurons and 10 excitatory neurons in E/I networks (see Figures3(a) and 4(a)) with no noise.
Neuron index6789101112131415Strength of the stimuli3.693.534.163.733.873.753.963.704.263.88Spiking numbers of isolated neurons8798888898Spiking numbers of neurons in E/I network1042323253There is another more important problem, the noise disturbance problem, for the information transmission caused by the small contrast of firing rates in a population of isolated neurons. Noise will disturb the encoding of the input information. If the difference of the firing rates of the two neurons is too small, the noise will make the two neurons fire the same number of spikes. Similarly, as in Table1, we choose 10 neurons from Figures 5(a) and 6(a) and display their firing rates in Table 2. It can be seen that the firing rate of neuron 23 (26.5) is a little larger than that of neuron 21 (24.0), but the noise makes neuron 23 emit the same number of spikes (8 spikes) as neuron 21 within the small time window T. As a result, it fails for the identification of the two inputs. This noise disturbance problem can also be alleviated in an E/I network. As the contrast of the firing rates of neuron 23 and neuron 21 is enhanced in an E/I network (they have firing rates of 6.7 and 2, respectively), noise cannot make neuron 23 fire the same spikes as neuron 21 any longer (neuron 23 and neuron 21 fire 3 and 1 spikes, respectively); thereby, we can distinguish the strength of the inputs of the two neurons by their outputs correctly. Therefore, due to the alleviation of the output overlapping problem and noise disturbance problem, the information efficiency in the E/I network (it is 0.39 in the E/I network of Figure 5(a)) is greatly improved compared to that in a population of isolated neurons (it is 0.098 in the population of isolated neurons of Figure 6(a)).Table 2
Spiking numbers of 10 isolated neurons and 10 excitatory neurons in E/I networks (see Figures5(a) and 6(a)) with noise.
Neuron index21222324252627282930Strength of the stimuli3.583.693.804.273.914.033.974.404.283.84Spiking numbers of isolated neurons8781099910108Spiking numbers of neurons in E/I network1234343553


### 3.3. Contrast Enhancement in a Randomly Connected Network

It is worthwhile noting that although the contrast enhancement is explored in the relatively simple E/I network, the results obtained in this paper are also reserved in more realistic E/I networks. Figure8 shows the firing rate distribution and information transmission efficiency in a randomly connected network with the connection probability ρ=0.7 among all the neurons. The input to each inhibitory neuron is different one by one by setting δB=0.1, σNB=0.5. The decay constants of excitatory synapses and inhibitory synapses are set differently as τdE=20ms, τdI=5ms. Figure 8(a) shows that inhibitory neurons fire synchronously, and a fraction of excitatory neurons fires at each cycle like the relatively simple case in Figure 2. Figure 8(b) visualizes the number of spikes of neurons 1~400 similar to Figure 5(b), which reveals the contrast enhancement of the firing rates of the excitatory neurons in the E/I network, compared with Figure 8(c). The information transmission efficiency is also calculated. The information transmission efficiency in the E/I network is 2.3 times of that in a population of isolated neurons.Figure 8
Firing rate contrast enhancement in a randomly connected E/I network: (a) raster plot of the firing of all neurons; (b) visualization of the firing rates of neurons 1-400 in the E/I network; (c) visualization of the firing rates of neurons 1-400 in a population of isolated neurons.ρ=0.7, δB=0.1, σNB=0.5, τdE=20ms, τdI=5ms, gEI=gEI=3/Ne+Ni,gIE=gII=20/Ne+Ni, τmI=25ms, EsynI=−75mV, d=3.5ms, and mS=3. The other parameters are set the same as those in Figure 5.
(a)(b)(c)


### 3.4. Information Transmission Efficiency

Further, we explored the dependency of the information transmission efficiency on the parameter values. Firstly, we checked the effect of two important parameters of the network on the efficiency of information encoding of the excitatory neurons, i.e., the strength of the inhibition to the excitatory neurons and the strength of the synaptic currents from the excitatory neurons to the inhibitory neurons. To compare the information transmission efficiency of the E/I network with that of a population of isolated neurons, we define an information transmission efficiency quantityIC=IE/IS, where IE is the information transmission efficiency of the E/I network and IS is the information transmission efficiency of the corresponding population of isolated neurons. Figure 9 shows that IC declines when the strength of the inhibition to the excitatory neurons decreases (see the solid line). IC is even smaller than 1 when the strength of inhibition is too weak, implying that the information transmission efficiency of an E/I network with no inhibition to the excitatory neurons is worse than a population of isolated neurons. This is because without inhibition, excitatory synaptic currents, which are approximately the same to each neuron, may even homogenize the contrast of the inputs. Therefore, inhibition is important for high information transmission efficiency. Figure 9 also shows that IC declines when the strength of the synaptic current from the excitatory to the inhibitory neurons decreases (see the dashed line). This means that the inputs from the excitatory neurons to the inhibitory neurons are also necessary for high efficiency. Hence, synaptic inhibition and excitation work together to enhance the contrast, which we will explain in the next section.Figure 9
Dependency of information encoding efficiency on (a) strength of the synapse from inhibitory neurons to excitatory neurons, (b) strength of the synapse from excitatory neurons to inhibitory neurons, and (c) standard deviation of the stimuli.


## 3.1. Firing Patterns in a Fully Connected Network

We first use a relatively simple E/I network to focus our presentation on our idea of global contrast enhancement resulting from the high-frequency synchronization, where neurons are fully connected and the background inputs to the inhibitory neurons are homogeneous. The parameter values are set similar to Ref. [29], which are set as follows unless otherwise stated: ρ=1, Ne=450, Ni=150, gEI=gEE=1.2/Ne+Ni, gIE=gII=7/Ne+Ni, τmE=50ms, τmI=10ms, RE=RI=1MΩ, VL=−65mV, τdE=τdI=6ms, τrE=τrI=0.1ms, EsynE=0mV, EsynI=−85mV,d=0.2ms,δS=0.5mS=4, δNS=0.5,δB=0, mB=2.4, σNB=0, and T=300ms. It should be notified that the numbers of excitatory neurons and inhibitory neurons can be changed (the ratio of the two types of neurons falls in a range around 3 : 1~4 : 1 according to Ref. [38], and we choose 3 : 1 in this paper). If the ratio of excitatory neurons and inhibitory neurons is changed, the excitatory and inhibitory synaptic strengths should also be suitably changed to maintain the balance of excitation and inhibition in the network so that the network still oscillates synchronously.Figure2 shows the raster plots of the firing of neurons in the E/I network, where neurons labeled from 1 to 450 are excitatory neurons and neurons labeled from 451 to 600 are inhibitory neurons. Inhibitory neurons fire synchronously and act as the globally inhibitory signal to all the neurons in the E/I network. In each cycle of synchronization, a fraction of excitatory neurons fires, and it is the firings of these neurons that drive the inhibitory neurons to fire in this cycle of synchronization. Neurons with larger input will fire more frequently than those with smaller input, i.e., neurons with larger input will fire in more cycles of synchronization. Thus, the input values are encoded in the firing rates (the number of spikes) of these excitatory neurons.Figure 2
High-frequency oscillation in E/I networks with (a) no noisy input to the excitatory neurons,σNS=0, and (b) noisy input to the excitatory neurons, σNS=0.5.
(a)(b)


## 3.2. Contrast Enhancement in a Fully Connected Network

We first investigate the firing rate contrast in the fully connected network when randomness is not included in the stimulus to the excitatory neurons, i.e.,σNS=0. Figure 3(a) shows the number of spikes of all neurons when T=300 ms in such cases. It is very interesting and unexpected to see that the number of spikes of all the neurons scattered between 0 and 6, as the inputs of all the neurons are clustered around 4 (see Figure 3(c)). For comparison, Figure 4(a) shows the number of spikes of all the neurons when neurons do not connect with each other, i.e., each neuron is isolated and only receives the external input IiSt in Equation (1), and the synaptic inputs IiEEt and IiIEt are both 0. We can see from Figure 4(a) that the number of spikes of isolated neurons is closely around 8 and 9. Stated in other words, the contrast of the number of spikes of excitatory neurons in the E/I network is enhanced compared with that of the isolated neurons. If we measure the number of spikes when T is large enough, we can get the firing rate of each neuron, ri≈ki/T. Figures 3(b) and 4(b) visualize the firing rates of 400 excitatory neurons in the E/I network and a population of isolated neurons, respectively, where the gray value of the small square at the position (i,j) represents the firing rate of the neuron with index 20∗i−1+j. It shows clearly that the gray contrast of Figure 3(b) is much larger than that of Figure 4(b), implying that the firing rate contrast of excitatory neurons in the E/I network is much larger than those in the population of isolated neurons.Figure 3
Firing rates of all excitatory neurons in E/I networks with no noise,σNS=0: (a) spiking numbers of all excitatory neurons within 300 ms; (b) visualization of the firing rates of neurons 1-400 with gray values located at (i, j) representing the firing rate of neurons 20∗i−1+j; (c) inputs of neurons 1-450.
(a)(b)(c)Figure 4
Firing rates of all excitatory neurons of a population of isolated excitatory neurons with no noise,σNS=0: (a) spiking numbers of all excitatory neurons within 300 ms; (b) visualization of the firing rates of neurons 1-400.
(a)(b)The enhancement of the contrast of the firing rates of excitatory neurons in E/I networks when randomness is included is also confirmed in Figures5 and 6, where the spiking numbers of neurons within time window T and visualization of the firing rates are shown. Randomness is introduced to the network by setting σNS=0.5. Figures 7(a) and 7(b) further show the histograms of the firing rates of the excitatory neurons in the E/I network and in the population of isolated neurons, respectively. The histogram of Figure 7(b) is more homogeneous than that of Figure 7(a), also confirming that the firing rate contrast of excitatory neurons in the E/I network is much larger than that in the population of isolated neurons.Figure 5
Firing rates of all excitatory neurons in E/I networks with noisy input to excitatory neurons,σNS=0.5: (a) spiking numbers of all excitatory neurons within 300 ms; (b) visualization of the firing rates of neurons 1-400.
(a)(b)Figure 6
Firing rates of all excitatory neurons of a population of isolated excitatory neurons with noisy input to excitatory neurons,σNS=0.5: (a) spiking numbers of all excitatory neurons within 300 ms; (b) visualization of the firing rates of neurons 1-400.
(a)(b)Figure 7
Histograms of the firing rates of all excitatory neurons in (a) a population of isolated excitatory neurons with noisy input and (b) E/I networks with noisy input.
(a)(b)To get a clearer watch of this contrast enhancement, we arbitrarily choose several neurons between Figures3(a) and 4(a), say neurons 6 to 15, and display the firing rates of the ten neurons in Table 1. We can see that the number of spikes of the isolated neurons only has three values, 7, 8, or 9. Some neurons have an identical number of spikes though their inputs are somewhat different. Namely, the number of spikes of some neurons overlaps, resulting in a failure of the encoding of the inputs of these neurons. For example, both neuron 6 and neuron 15 have 8 spikes though they have different inputs. This is because the contrast of the firing rates of the two neurons (neuron 6 and neuron 15 have firing rates of 25.3 and 27.3, respectively) is so small that the number of the spikes within a small time window T cannot be distinguished. On the other hand, as the contrast of the firing rates of the excitatory neurons in the E/I network is enhanced, the problem of this overlapping is alleviated. The number of spikes of these neurons in the E/I network has more individual values (0, 1, 2, 3, 4, or 5) than those (7, 8, or 9) in a population of isolated neurons (notice that as the synaptic inhibition in the E/I network is stronger than excitation, the number of spikes fired by neurons in the E/I network is less than those in the population of isolated neurons). The numbers of spikes of neuron 6 and neuron 15 are no longer identical (they have 1 and 3 spikes, respectively); this is because they have a relatively large contrast of the firing rates (they have firing rates of 5.3 and 12.7, respectively). Therefore, thanks to the enhancement of the contrast of the firing rates, the output overlapping problem is somewhat relieved. Correspondingly, it is found that the information transmission efficiency is much higher in the E/I network (it is 0.54 in the E/I network of Figure 3(a)) than that in the population of isolated neurons (it is 0.13 in the population of isolated neurons of Figure 4(a)).Table 1
Spiking numbers of 10 isolated neurons and 10 excitatory neurons in E/I networks (see Figures3(a) and 4(a)) with no noise.
Neuron index6789101112131415Strength of the stimuli3.693.534.163.733.873.753.963.704.263.88Spiking numbers of isolated neurons8798888898Spiking numbers of neurons in E/I network1042323253There is another more important problem, the noise disturbance problem, for the information transmission caused by the small contrast of firing rates in a population of isolated neurons. Noise will disturb the encoding of the input information. If the difference of the firing rates of the two neurons is too small, the noise will make the two neurons fire the same number of spikes. Similarly, as in Table1, we choose 10 neurons from Figures 5(a) and 6(a) and display their firing rates in Table 2. It can be seen that the firing rate of neuron 23 (26.5) is a little larger than that of neuron 21 (24.0), but the noise makes neuron 23 emit the same number of spikes (8 spikes) as neuron 21 within the small time window T. As a result, it fails for the identification of the two inputs. This noise disturbance problem can also be alleviated in an E/I network. As the contrast of the firing rates of neuron 23 and neuron 21 is enhanced in an E/I network (they have firing rates of 6.7 and 2, respectively), noise cannot make neuron 23 fire the same spikes as neuron 21 any longer (neuron 23 and neuron 21 fire 3 and 1 spikes, respectively); thereby, we can distinguish the strength of the inputs of the two neurons by their outputs correctly. Therefore, due to the alleviation of the output overlapping problem and noise disturbance problem, the information efficiency in the E/I network (it is 0.39 in the E/I network of Figure 5(a)) is greatly improved compared to that in a population of isolated neurons (it is 0.098 in the population of isolated neurons of Figure 6(a)).Table 2
Spiking numbers of 10 isolated neurons and 10 excitatory neurons in E/I networks (see Figures5(a) and 6(a)) with noise.
Neuron index21222324252627282930Strength of the stimuli3.583.693.804.273.914.033.974.404.283.84Spiking numbers of isolated neurons8781099910108Spiking numbers of neurons in E/I network1234343553


## 3.3. Contrast Enhancement in a Randomly Connected Network

It is worthwhile noting that although the contrast enhancement is explored in the relatively simple E/I network, the results obtained in this paper are also reserved in more realistic E/I networks. Figure8 shows the firing rate distribution and information transmission efficiency in a randomly connected network with the connection probability ρ=0.7 among all the neurons. The input to each inhibitory neuron is different one by one by setting δB=0.1, σNB=0.5. The decay constants of excitatory synapses and inhibitory synapses are set differently as τdE=20ms, τdI=5ms. Figure 8(a) shows that inhibitory neurons fire synchronously, and a fraction of excitatory neurons fires at each cycle like the relatively simple case in Figure 2. Figure 8(b) visualizes the number of spikes of neurons 1~400 similar to Figure 5(b), which reveals the contrast enhancement of the firing rates of the excitatory neurons in the E/I network, compared with Figure 8(c). The information transmission efficiency is also calculated. The information transmission efficiency in the E/I network is 2.3 times of that in a population of isolated neurons.Figure 8
Firing rate contrast enhancement in a randomly connected E/I network: (a) raster plot of the firing of all neurons; (b) visualization of the firing rates of neurons 1-400 in the E/I network; (c) visualization of the firing rates of neurons 1-400 in a population of isolated neurons.ρ=0.7, δB=0.1, σNB=0.5, τdE=20ms, τdI=5ms, gEI=gEI=3/Ne+Ni,gIE=gII=20/Ne+Ni, τmI=25ms, EsynI=−75mV, d=3.5ms, and mS=3. The other parameters are set the same as those in Figure 5.
(a)(b)(c)


## 3.4. Information Transmission Efficiency

Further, we explored the dependency of the information transmission efficiency on the parameter values. Firstly, we checked the effect of two important parameters of the network on the efficiency of information encoding of the excitatory neurons, i.e., the strength of the inhibition to the excitatory neurons and the strength of the synaptic currents from the excitatory neurons to the inhibitory neurons. To compare the information transmission efficiency of the E/I network with that of a population of isolated neurons, we define an information transmission efficiency quantityIC=IE/IS, where IE is the information transmission efficiency of the E/I network and IS is the information transmission efficiency of the corresponding population of isolated neurons. Figure 9 shows that IC declines when the strength of the inhibition to the excitatory neurons decreases (see the solid line). IC is even smaller than 1 when the strength of inhibition is too weak, implying that the information transmission efficiency of an E/I network with no inhibition to the excitatory neurons is worse than a population of isolated neurons. This is because without inhibition, excitatory synaptic currents, which are approximately the same to each neuron, may even homogenize the contrast of the inputs. Therefore, inhibition is important for high information transmission efficiency. Figure 9 also shows that IC declines when the strength of the synaptic current from the excitatory to the inhibitory neurons decreases (see the dashed line). This means that the inputs from the excitatory neurons to the inhibitory neurons are also necessary for high efficiency. Hence, synaptic inhibition and excitation work together to enhance the contrast, which we will explain in the next section.Figure 9
Dependency of information encoding efficiency on (a) strength of the synapse from inhibitory neurons to excitatory neurons, (b) strength of the synapse from excitatory neurons to inhibitory neurons, and (c) standard deviation of the stimuli.


## 4. Mechanism of the Global Contrast Enhancement

One may wonder how the global contrast enhancement happens as neurons receive the same afferent synaptic spikes from the network (notice that each neuron connects to all the other neurons in the network)? We found by close observation of the synaptic currents of two typical neurons (the firing rate of one of them is higher than the other) that the synaptic currents of the two neurons are different although they receive the same afferent synaptic spikes. It is this difference between the synaptic currents that results in the contrast enhancement. The synaptic current of a neuron is divided into two parts. One part is the excitatory current due to spikes from the excitatory neurons (see Figure10(a)), and the other part is the inhibitory current due to spikes from the inhibitory neurons (see Figure 10(b)). Figure 10(a) shows that the excitatory current of the neuron with a higher firing rate receives a larger excitatory synaptic current, and the neuron with a lower firing rate receives a smaller current. On the other hand, Figure 10(b) shows that the neuron with a higher firing rate receives a smaller negative synaptic current while the neuron with a lower firing rate receives a larger negative synaptic current. Combining the two elements together, we can conclude that the neuron with the higher firing rate receives more synaptic excitation and less synaptic inhibition, but the neuron with the lower firing rate receives less excitation and more inhibition. Consequently, the neurons with higher firing rates in the network will fire with even higher rates, but neurons with lower firing rates will fire with even lower rates. This explains the firing rate contrast enhancement we obtained in Simulation Results in this paper.Figure 10
Comparisons of the synaptic currents received by a neuron with a higher firing rate (red) and a neuron with a slower firing rate (blue): (a) total synaptic currents from excitatory neurons; (b) total synaptic currents from inhibitory neurons.
(a)(b)The next interesting question is why synaptic currents are different for the two neurons although the afferent spikes for them are the same? This can be explained by the voltage-dependent properties of the synaptic currents [39]. Let us consider the synaptic currents received by a neuron after it fires at a certain cycle of synchronization. For example, neuron 1 fires at the time tk; therefore, the membrane potential of neuron 1 resumes to the reset potential (-65 mV in the model) immediately after its firing. As synchronization of the firing of the neurons also occurs at roughly tk, the inhibitory currents induced by the synchronized spikes from the inhibitory neurons will be small due to the low membrane potential of neuron 1 according to Equation (4) (notice that for inhibitory synapse, EsynE should be replaced by EsynI). On the other hand, neuron 2, which does not fire at tk, will receive a larger inhibitory current due to the higher membrane potential compared to neuron 1. Namely, a neuron firing at a cycle of synchronization will receive a smaller negative synaptic current than those who do not fire, implying that neurons with higher firing rates will receive smaller negative inhibitory currents.In short words, it is the voltage-dependent property of the synaptic current that causes high firing rate neurons to receive large excitatory synaptic currents and small inhibitory synaptic currents, thereby firing with even higher firing rates. Similarly, low firing rate neurons receive small excitatory synaptic currents and large inhibitory synaptic currents, thereby firing with even lower firing rates. This may be the mechanism that explains the firing rate contrast enhancement exhibited in the synchronized E/I network in this paper.


## 5. Conclusions and Discussions

Synchronization with high frequency in excitatory/inhibitory networks has been found in many real neural systems, which is pertinent to some high-frequency components of neuronal oscillation, for example, gamma oscillation. Some possible functional roles of this high-frequency synchronization have been hypothesized and have been tested in theoretical models [12–14]. Contrast enhancement is a general feature of information processing in neural systems, especially in sensory pathways [23]. We found that high-frequency synchronization in the E/I network can enhance the contrast of the firing rates of excitatory neurons globally. Lateral inhibition is widely believed to be able to enhance the contrast of the firing rates of neighboring neurons [24, 25]. It is supposed that information is encoded in excitatory neurons, and neurons only connect to neighboring neurons. The excitatory neurons activate neighboring inhibitory neurons, which in turn inhibit neighboring excitatory neurons. Thereby, excitatory neurons with a higher firing rate may inhibit neighboring neurons more, therefore causing the following effect: for two neighboring excitatory neurons, the one with a lower firing rate receives more inhibition while the other with a higher firing rate receives less inhibition. This reciprocal inhibition of neighboring neurons enhances their firing rate contrast. Unlike lateral inhibition, where neurons only connect to neighboring ones resulting in local firing rate contrast enhancement, each neuron connects to all the other neurons in the network in our model (or connects to all neurons with a constant probability as in the model), resulting in global contrast enhancement. Namely, the inhibition caused by the synchronized inhibitory neurons enhances the firing rate contrast of all excitatory neurons in the network in our model, whereas lateral inhibition only enhances the firing rate contrast of the corresponding two neighboring neurons.The firing rate contrast of all excitatory neurons in the network can be characterized by the theory of image entropy. If we view the firing rate of each neuron as the gray value of a pixel in an image, then the firing rates of all excitatory neurons can project a gray-valued image. Since an image with high image entropy has high image contrast, we can use the theory of image entropy to characterize the firing rate contrast of all excitatory neurons. Correspondingly, the information transmission efficiency is relevant to the entropy of the inputs and the outputs (firing rates) of the neurons. We found that the global contrast enhancement in this paper can alleviate two problems. One is the output overlap problem that the spiking numbers of neurons with a low firing rate contrast will be identical. The other is the noise disturbance problem that noise will make neurons with different inputs fire the same number of spikes. Thus, the contrast enhancement in the E/I networks can improve the information transmission efficiency, namely, increase the mutual information on the one hand and decrease the energy consumption on the other hand. Another important thing for real neural systems is that they need to distinguish signals with high similarities. In our model, signals with high similarities mean that the parameter valueδS is small (if δS is small, the input contrast of all excitatory neurons is small). We can see from Figure 9 (see the dot dashed line) that the information transmission efficiency increases with the decrease of this parameter, implying that the information transmission efficiency is enhanced more by the E/I network if the input contrast is lower. This has a strong biological implication because the neural systems need to enhance the contrast of sensory input more for clear identification of the signals when the sensory input contrast is smaller.The mechanism of contrast enhancement is also analyzed in the paper. The global contrast enhancement results from the interaction of the excitatory neurons and inhibitory neurons. Inhibitory neurons fire synchronously at each cycle, while only a fraction of excitatory neurons fires at each cycle. It is the firing of these excitatory neurons that drives the synchronized firing of the inhibitory neurons at each cycle of the synchronization, and the firing inhibitory neurons inhibit all neurons in the network. Due to the voltage-dependent properties of the synaptic currents, the inhibitory synaptic currents to the fired excitatory neurons are less than those of the nonfiring excitatory neurons. Thus, neurons that fire more frequently may receive less inhibition, resulting in the enhancement of the contrast of the firing rates.Our work implies a possible functional role in information transmission of high-frequency synchronization in neuronal systems and also might be heuristic for modelling of an artificial neuron model [40] and neural networks [41], which will be explored in our future work.



---

*Source: 8823111-2020-11-09.xml*