# Variance Entropy: A Method for Characterizing Perceptual Awareness of Visual Stimulus

Authors: Meng Hu; Hualou Liang
Journal: Applied Computational Intelligence and Soft Computing
 (2012)

---

## Abstract

Entropy, as a complexity measure, is a fundamental concept for time series analysis. Among many methods, sample entropy (SampEn) has emerged as a robust, powerful measure for quantifying complexity of time series due to its insensitivity to data length and its immunity to noise. Despite its popular use, SampEn is based on the standardized data where the variance is routinely discarded, which may nonetheless provide additional information for discriminant analysis. Here we designed a simple, yet efficient, complexity measure, namely variance entropy (VarEn), to integrate SampEn with variance to achieve effective discriminant analysis. We applied VarEn to analyze local field potential (LFP) collected from visual cortex of macaque monkey while performing a generalized flash suppression task, in which a visual stimulus was dissociated from perceptual experience, to study neural complexity of perceptual awareness. We evaluated the performance of VarEn in comparison with SampEn on LFP, at both single and multiple scales, in discriminating different perceptual conditions. Our results showed that perceptual visibility could be differentiated by VarEn, with significantly better discriminative performance than SampEn. Our findings demonstrate that VarEn is a sensitive measure of perceptual visibility, and thus can be used to probe perceptual awareness of a stimulus.


---

## Body

## 1. Introduction

Over the past decades, entropy [1] has been widely used for analysis of dynamic systems. Among many measures, sample entropy (SampEn) is thought of as an effective, robust method due to its insensitivity to data length and its immunity to noise [2]. Until now, SampEn has been successfully applied for discriminant analysis of cardiovascular data [3], electroencephalogram data [4], and many others [5]. In addition, SampEn has been used in multiscale analysis for computing entropy over multiple time scales inherent in time series. For example, multiscale entropy [6] and adaptive multiscale entropy (AME) [7] both use SampEn to estimate entropy over multiple scales of time series.Despite its popularity, it is not well recognized that there is an inherit drawback of SampEn used for discriminant analysis, that is, the calculation of SampEn is routinely based on the normalized data where the variance of data that may provide additional information for discrimination is discarded [8]. The normalization is essentially to rescale the data, which is appropriate if the analysis is driven by the search for order in the dynamics, but is otherwise inappropriate for discriminant analysis of two data sets as the rescaling can make them appear identical when they clearly are not. In fact, the variance and SampEn represent different aspects of the data: the variance measures concentration only around the mean of the data, whereas the entropy measures diffuseness of the density irrespective of the location of concentration of the data. In this paper, we proposed a new complexity measure, variance entropy (VarEn), to take into account both SampEn and the variance for improved discriminant analysis. Not only can it be used as a single-scale measure, but it can also be adapted for studying nonstationary data over multiple time scales. We applied VarEn to analyze cortical local field potential (LFP) data collected from a macaque monkey while performing a generalized flash suppression task [9], in which physical stimulation is dissociated from perceptual experience, to probe perceptual awareness of a visual stimulus. We showed that VarEn performed better than SampEn for both the whole time series (single-scale) and multiscale analysis of LFP data in terms of discriminative ability in distinguishing different perceptual conditions (Visible versus Invisible). Our results suggest that the proposed VarEn measure is a useful technique for discriminant analysis of neural data and can be used to uncover perceptual awareness of a stimulus.


## 2. Method

### 2.1. Sample Entropy

Entropy describes the complexity or irregularity of system, which can be used to classify systems. So far, many attempts have been made for estimation of entropy, such as Kolmogorov entropy [10] and Eckmann-Ruelle entropy [11]. However, these methods usually require very long time series that is not always available. Approximate entropy can be efficiently computed for short and noisy time series [1], but introduces a bias via counting self-match when calculating the pairs of similar epochs. Sample entropy (SampEn) provides a refined version of approximate entropy to reduce the bias [2]. It is defined as the negative natural logarithm of conditional probability that two sequences similar for m points remain similar at the next m + 1 point in the data set within a tolerance r, where self-matches are excluded in calculating the probability.In order to compute SampEn, a time seriesI={i(1),i(2),…,i(N)} is embedded in a delayed m-dimensional space, where the m-dimensional vectors are constructed as xm(k)=(i(k),i(k+1),…,i(k+m-1)),k=1~N-m+1. The match of two vectors in the embedded space is defined as their distance lower than the tolerance r. Bm(r) is the probability that two sequences of m points match within r, whereas Am(r) is similarly defined for an embedded dimension of m + 1. The SampEn is then calculated as(1)SampEn(I,m,r)=-ln⁡(Am(r)Bm(r)).In practice, it is common to set the tolerancer as a fraction of the standard deviation of the data, which effectively rescales the data to have similar dynamic scales. As a result, the normalization process obscures the difference in the scales of data sets, thus rendering the analysis inappropriate if the goal is chiefly to discriminate between data sets. We therefore introduce a variance entropy measure in the following section to rectify this shortcoming.


### 2.2. Variance Entropy

Variance entropy (VarEn) measure is designed for discriminant analysis by combining SampEn and the variance of data. Specifically, VarEn can be treated as inverse-variance weighted entropy to represent system complexity. For a time seriesx, VarEn is defined as(2)VarEn(x,m,r)=∑i=1pSampEn(xi,m,r)×wi∑i=1pwi,
where xi is the ith segment of x, obtained with a window that slides over time, p is the number of sliding windows, wi is inverse variance of xi, m and r are the parameters for calculation of SampEn(xi,m,r). Specifically, to calculate VarEn, we first apply a sliding window over time series x. For a given window (e.g., ith time window, xi), we compute its SampEn, variance, and wi. After computing all p time windows, we then apply above formula to obtain VarEn of this time series x. A schematic representation of the processing steps is shown in Figure 1. It is also straightforward to extend our VarEn measure to the analysis of multitrial neural data in which xi is the ith trial of x, and p is the number of trials. In this study, the parameters m and r were chosen as 2 and 0.2 for minimizing the standard error of entropy estimation [3]. We note that there is no pronounced difference with automatic selection of r [12].Figure 1
Schematic representation of the proposed variance entropy (VarEn).Similar to SampEn, VarEn can also be applied to nonstationary data by considering multiple time scales inherent in the data, on which the entropy can be calculated. By multiple-scale entropy analysis, we compute the VarEn in adaptive multiscale entropy (AME) [7] to demonstrate its performance in comparison with the use of SampEn. AME is a multiscale analysis method in which the scales are adaptively derived directly from the data by virtue of multivariate empirical mode decomposition [13], which is fully data driven, well suited for the analysis of nonlinear/nonstationary neural data. Depending on the consecutive removal of low-frequency or high-frequency components, AME can be estimated at either coarse-to-fine (preserving high-frequency oscillations by progressively removal of low-frequency components) or fine-to-coarse scales (preserving low-frequency oscillations by progressively removal of high-frequency components) over which the sample entropy is performed. The coarse-to-fine and fine-to-coarse AME can be used separately or used in tandem to reveal the underlying dynamics of complex neural data. In this study, we use the VarEn to replace SampEn to perform multiscale analysis.


## 2.1. Sample Entropy

Entropy describes the complexity or irregularity of system, which can be used to classify systems. So far, many attempts have been made for estimation of entropy, such as Kolmogorov entropy [10] and Eckmann-Ruelle entropy [11]. However, these methods usually require very long time series that is not always available. Approximate entropy can be efficiently computed for short and noisy time series [1], but introduces a bias via counting self-match when calculating the pairs of similar epochs. Sample entropy (SampEn) provides a refined version of approximate entropy to reduce the bias [2]. It is defined as the negative natural logarithm of conditional probability that two sequences similar for m points remain similar at the next m + 1 point in the data set within a tolerance r, where self-matches are excluded in calculating the probability.In order to compute SampEn, a time seriesI={i(1),i(2),…,i(N)} is embedded in a delayed m-dimensional space, where the m-dimensional vectors are constructed as xm(k)=(i(k),i(k+1),…,i(k+m-1)),k=1~N-m+1. The match of two vectors in the embedded space is defined as their distance lower than the tolerance r. Bm(r) is the probability that two sequences of m points match within r, whereas Am(r) is similarly defined for an embedded dimension of m + 1. The SampEn is then calculated as(1)SampEn(I,m,r)=-ln⁡(Am(r)Bm(r)).In practice, it is common to set the tolerancer as a fraction of the standard deviation of the data, which effectively rescales the data to have similar dynamic scales. As a result, the normalization process obscures the difference in the scales of data sets, thus rendering the analysis inappropriate if the goal is chiefly to discriminate between data sets. We therefore introduce a variance entropy measure in the following section to rectify this shortcoming.


## 2.2. Variance Entropy

Variance entropy (VarEn) measure is designed for discriminant analysis by combining SampEn and the variance of data. Specifically, VarEn can be treated as inverse-variance weighted entropy to represent system complexity. For a time seriesx, VarEn is defined as(2)VarEn(x,m,r)=∑i=1pSampEn(xi,m,r)×wi∑i=1pwi,
where xi is the ith segment of x, obtained with a window that slides over time, p is the number of sliding windows, wi is inverse variance of xi, m and r are the parameters for calculation of SampEn(xi,m,r). Specifically, to calculate VarEn, we first apply a sliding window over time series x. For a given window (e.g., ith time window, xi), we compute its SampEn, variance, and wi. After computing all p time windows, we then apply above formula to obtain VarEn of this time series x. A schematic representation of the processing steps is shown in Figure 1. It is also straightforward to extend our VarEn measure to the analysis of multitrial neural data in which xi is the ith trial of x, and p is the number of trials. In this study, the parameters m and r were chosen as 2 and 0.2 for minimizing the standard error of entropy estimation [3]. We note that there is no pronounced difference with automatic selection of r [12].Figure 1
Schematic representation of the proposed variance entropy (VarEn).Similar to SampEn, VarEn can also be applied to nonstationary data by considering multiple time scales inherent in the data, on which the entropy can be calculated. By multiple-scale entropy analysis, we compute the VarEn in adaptive multiscale entropy (AME) [7] to demonstrate its performance in comparison with the use of SampEn. AME is a multiscale analysis method in which the scales are adaptively derived directly from the data by virtue of multivariate empirical mode decomposition [13], which is fully data driven, well suited for the analysis of nonlinear/nonstationary neural data. Depending on the consecutive removal of low-frequency or high-frequency components, AME can be estimated at either coarse-to-fine (preserving high-frequency oscillations by progressively removal of low-frequency components) or fine-to-coarse scales (preserving low-frequency oscillations by progressively removal of high-frequency components) over which the sample entropy is performed. The coarse-to-fine and fine-to-coarse AME can be used separately or used in tandem to reveal the underlying dynamics of complex neural data. In this study, we use the VarEn to replace SampEn to perform multiscale analysis.


## 3. Results

In this section, we apply VarEn to analyze local field potentials (LFPs) collected from visual cortex of a macaque monkey while performing a visual illusion task, to characterize neural dynamics of perceptual awareness of a visual stimulus.The visual illusion task used here is called generalized flash suppression (GFS) task, where a salient visual stimulus can be rendered invisible despite continuous retinal input, thus providing a rare opportunity to study neural mechanisms directly related to perception [9]. In the task, as soon as a monkey gained fixation for about 300 msec, the target stimulus indicated by a red disk was presented. At 1400 msec after the target onset, small random-moving dots appeared as the surroundings. With the immediate presence of the surroundings, the red disk could be rendered subjectively invisible. If the target stimulus disappeared from perception, the monkey was trained to release a lever; otherwise, monkey was to hold the lever. Therefore, based on the responses of the animal, the trial was classified as either “Visible” or ‘‘Invisible.” Note that the stimuli in these two conditions were physically identical. Multielectrode LFP recordings were simultaneously collected from multiple cortical areas V1, V2, and V4 while monkeys performed the GFS task [14]. The data were obtained by band-pass filtering the full bandwidth signal between 1 and 500 Hz, and then resampled at 1 KHz. In this study, the LFPs of one second long after surrounding onset on a typical channel of area V4 over 87 trials were used in the analysis.VarEn is first directly applied to the LFP data to discriminate two perceptual conditions: Visible versus Invisible. In Figure2, we can see that the VarEn of the invisible condition is significantly greater than that of the visible condition (P<0.05). As a comparison, SampEn is applied to the same LFP data. However, the result reveals that there is no significant difference between two perceptual conditions (Figure 2). This result suggests that VarEn carries more discriminative information by integrating SampEn and the variance of data.Figure 2
Comparison of the proposed variance entropy (VarEn) and sample entropy (SampEn) in discriminating the different perceptual conditions (Invisible versus Visible). Shown are the means and the standard errors of means of VarEn (left) and SampEn (right). As a result, the VarEn exhibits significant difference between two perceptual conditions (p=7.69e-4, indicated by the sign “*’’), but the SampEn does not (P=0.056).Next, VarEn is applied to the LFP data for performing multiple-scale entropy analysis. As described in the Method, we herein apply the VarEn-based adaptive multiscale analysis (AME) to the LFP data for discriminating two perceptual conditions over different LFP scales. As a comparison, the original AME is also applied to the LFP data, where the SampEn is applied to calculate the AME entropy measure. Figure3 shows the comparison of two methods at the coarse-to-fine scales. We can see from Figures 3(a) and 3(b) that both methods exhibit dominantly increasing trend as the scale increases. However, the VarEn-based AME (Figure 3(a)) clearly provides significantly larger separation between two perceptual conditions than the original AME with SampEn (Figure 3(b)). Specifically, the VarEn-based AME at six significant scales (i.e., scale 1–6) shows significant differences between two perceptual conditions (P<0.05) (Figure 3(a)), whereas the original AME differs only at three scales, that is, scales 4–6 (Figure 3(b)). A detailed comparison of P values between our VarEn and the SampEn along the coarse-to-fine scales is shown in Figure 3(c). These results indicate that VarEn is more sensitive than SampEn to detect perceptual difference between two conditions.Coarse-to-fine comparison for the AME with VarEn (a) and the AME with SampEn (b) for discriminating the Invisible (red circle) and the Visible conditions (blue triangle) over multiple scales. The error bar refers to the standard error of mean. The leftmost red circle and blue triangle are the entropy measures of the raw data. For the AME with VarEn (a), significant differences occur at the 1st–6th scales, while the significant differences only occur at the 4th–6th scales for the AME with SampEn (b). The sign“*’’ refers to P<0.05. The P values based on our VarEn and the SampEn along the coarse-to-fine scales are also compared (c), in which the horizontal black dotted line refers to the significance level (P=0.05).
(a)(b)(c)Similarly, the improvement of discrimination by VarEn occurs at the fine-to-coarse scales as well (Figure4), in which AME with VarEn significant different at scales 1–3 (Figure 4(a)) while AME with SampEn only exhibits significant difference at scale 2 (Figure 4(b)). Detailed comparison of P values between our VarEn and the SampEn along the fine-to-coarse scales is also shown in Figure 4(c) to support our findings. These results, taken together, suggest that our VarEn can be used to obtain improved discriminative information for discriminant analysis of neural data.Fine-to-coarse comparison for the AME with VarEn (a) and the AME with SampEn (b) for discriminating the Invisible (red circle) and the Visible conditions (blue triangle) over multiple scales. For the AME with VarEn (a), significant differences occur at the 1st–3rd scales, whereas for the AME with SampEn (b) significant difference only occurs at the 2nd scale. Conventions are the same as in Figure3. The P values based on our VarEn and the SampEn along the fine-to-coarse scales are also compared (c), in which the horizontal black dotted line refers to the significance level (P=0.05). Note that the P values greater than 0.35 are truncated.
(a)(b)(c)In comparison of the coarse-to-fine AME (Figure3) with the fine-to-coarse AME (Figure 4), the coarse-to-fine AME focuses on the high-frequency oscillations, whereas the fine-to-coarse AME emphasizes the low-frequency oscillations. When applied to the LFP data, we can see that the AME at coarse-to-fine scales exhibit more discriminative scales than those at the fine-to-coarse scales, indicating that the low-frequency scales may not contain as much discriminative information as the high-frequency scales. The low-frequency oscillations mainly correspond to evoked potentials, which are presumably the same as the stimuli in the GFS task are identical; this explains why the fine-to-coarse scales exhibit less discriminative ability for separating two perceptual conditions. Furthermore, among all the significant differences, the AME measure in the invisible condition is higher than that in the visible condition, which suggests that perceptual suppression is likely to be related to more complex neural processes than the normal visible condition.


## 4. Discussion and Conclusion

In this paper, we proposed a simple complexity measure, variance entropy (VarEn), by combining SampEn and the variance to achieve improved discriminant analysis. Our measure was motivated by the observation that the calculation of SampEn is based on the normalized data where the variance is routinely discarded, which may otherwise provide the additional information for discrimination analysis. We applied VarEn to the analysis of cortical local field potential data collected from visual cortex of monkey performing a generalized flash suppression task. We showed that VarEn performed better than SampEn for both the whole time series (single-scale) and multiscale analysis of LFP data in terms of discriminative ability in distinguishing different perceptual conditions. The results suggest that our proposed VarEn measure is a useful measure for discriminant analysis of neural data and can be used to uncover perceptual awareness of a stimulus.To quantify the complexity of a system, our proposed VarEn measure is defined as inverse-variance weighted SampEn. Inverse-variance weighting is typically used in statistical meta-analysis to combine several estimates of an unknown quantity to obtain an estimate of improved precision [15]. While other forms of weights (e.g., amplitude) could be used, such a choice of weight is optimal in providing the unbiased and minimum variance estimator. In comparison of VarEn to SampEn, the key difference is that SampEn discards the variance of the data, whereas VarEn combines the variance with SampEn via inverse-variance weighting. As such, if the main objective is for discriminant analysis, VarEn is preferred as it incorporates the variance information into its estimation. On the other hand, SampEn is appropriate if the analysis is driven by the search for order in the dynamics.



---

*Source: 525396-2012-05-22.xml*