# Calculating Permutation Entropy without Permutations

Authors: Alexander K. Vidybida
Journal: Complexity
 (2020)

---

## Abstract

A method for analyzing sequential data sets, similar to the permutation entropy one, is discussed. The characteristic features of this method are as follows: it preserves information about equal values, if any, in the embedding vectors; it is exempt from combinatorics; and it delivers the same entropy value as does the permutation method, provided the embedding vectors do not have equal components. In the latter case, this method can be used instead of the permutation one. If embedding vectors have equal components, this method could be more precise in discriminating between similar data sets.


---

## Body

## 1. Introduction

Because of technical progress in the areas of sensors and storage devices, a huge amount of raw data about time course of different processes, such as ECG, EEG, climate data recordings, and stock market data, have become available. These data are redundant. The data processing and classification, aimed at extracting meaningful for nonspecialist characteristics, is based on reducing the excess of redundancy. As a result, new data are obtained, small in size and digestible by a human being. Examples of those reduced data for time series can be mean value, variance, Lyapunov exponents, correlation dimension, and attractor dimension.A remarkable method suitable for reducing the excess of redundancy in time series has been proposed by Bandt and Pompe in [1], known as permutation entropy. This method is simple and transparent, is robust with respect to monotonic distortions of the raw data, and is suitable for estimating the dynamical complexity of the underlying dynamical process. Many interesting results, e.g. [2–5], have been obtained with straightforward application of the permutation entropy methodology in its initial form, as it is described in [1]. Nevertheless, this method is subjected to a critique for not taking into account absolute values of the raw data and for not treating properly a possibility of having equal values in the embedding vector (ties), [6, 7]. In this connection, it should be taken into account that any redundancy reduction method leaves out some types of information, which may be useless for one process/task and may carry useful information for another one. In the latter case, the bare idea in [1] about how to treat equal values can/should be modified in order to meet a purpose of concrete situation. Examples of such a modification can be found in [8, 9] for taking into account absolute values or in [10, 11] for treating equal values. Interesting modification of the permutation entropy method has been proposed in [12] for 3-tuple EEG data.In the standard permutation entropy methodology, it is preferable that embedding vectors have all their components different. Otherwise, they cannot be plainly symbolized by a permutation without using additional rules, which actually treat equal values as not being such. Situation with equal values in the embedding vector may arise for high embedding dimension, for crude quantization of measured data, for very long data sequences, and when observed dynamical system has intrinsically only a small number of possible outputs.This note is aimed at discussing a slightly different symbolization technique of embedding vectors, which does not refer to combinatorics and which is capable of preserving information about equal values in embedding vectors. Instead of permutation, an embedding vector is emblematized with a single integer number of baseD, where D is the embedding dimension. In the case of no ties (no equal components in the embedding vectors), the technique is equivalent to the standard permutation entropy methodology. In the opposite case, it may discriminate between similar data sets better than the permutation entropy method does.


## 2. Permutation Entropy

Consider a finite sequence(1)X=x0,x1,…,xN−1,xi∈ℝ1,i=0,1,2,…,N−1,of measurements. By choosing the embedding dimension D<N, the data (1) can be embedded into a D-dimensional space by picking out consecutive D‐tuples from X. As a result, a set of D-dimensional embedding vectors are obtained:(2)V=V0,V1,…,VN−D,Vi∈ℝD,i=0,1,2,…,N−D,where each vector has the following form:(3)V0=x0,x1,x2…,xD−1,…,Vi=xi,xi+1,xi+2,…,xi+D−1,…,VN−D=xN−D,xN+1−D,…,xN−1.An additional parameter of the embedding procedure is delayτ=1,2,…. In the above definition, we put τ=1 for simplicity. With τ≠1 one would have Vi=xi,xi+τ,xi+2τ,…,xi+D−1τ instead of (3).The data represented in (2) and/or (3) are even more redundant than those represented in (1) since for D≪N, most data values from (1) are represented in (3) D times. In the permutation entropy technique [1], each embedding vector from (2) and/or (3) is replaced with a permutation π of D integers {0, 1, 2, …, D−1}, which is defined by the order pattern of values composing the vector. For any embedding vector V=x0,x1,…,xD−1, the permutation π, which symbolizes it, is calculated as follows. Arrange all components of V either in the descending [13]:(4)V=x0,x1,…,xD−1⟶Vπ=xr0,xr1,…,xrD−1,xr0>xr1>⋯>xrD−1,or in the ascending [11,14]:(5)V=x0,x1,…,xD−1⟶Vπ=xr0,xr1,…,xrD−1,xr0<xr1<⋯<xrD−1,order keeping their subscripts unchanged (actually, in (4) and (5), equal values (ties) are as well admitted. Here, we exclude such a possibility for the sake of clarity. The equal values are discussed in the next section.) The permutation π which corresponds to V is obtained as the row of the subscripts in the rearranged vector Vπ from either (4) or (5):(6)π≡πV=r0,r1,…,rD−1.From the set of embedding vectorsV, calculate a new set Π of order patterns by replacing each vector in (2) by the corresponding permutation:(7)Π=π0,π1,…,πN−D.Now, empirical probability of each permutation,pπi, can be obtained by dividing the number of occurrences of πi in Π by the total number of elements in Π. The permutation entropy of V is the Shannon entropy of the probability distribution pπi:(8)HV≡HΠ=−∑i=0K−1pπilogpπi,where K is the number of different permutations in Π.


### 2.1. Treatment of Equal Values

Equal values in an embedding vector are, to an extent, inconvenient. Indeed, ifxr=xs for some 0≤r,s<D in a vector V=x0,x1,…,xD−1, then r and s should be placed side by side in the permutation (6), but which one should go first? Due to the sameness of values, it is impossible to uniquely determine a corresponding permutation without introducing additional rules. In some cases, the possibility of equal values can be ignored due to their low probability. This is reasonable when the embedding dimension is low and/or a chaotic process data are recorded with high precision [1, 15, 16]. If equal values are inevitable, the following rule is applied (in some cases, e.g. [10, 11], the opposite inequality sign is used here):(9)ifxs=xrands>rthensgoes first.The rule (9) has different meaning depending on whether (4) or (5) convention is used. Namely, in the case of (4), an embedding vector with all components equal will be equivalent to a vector with monotonically ascending components. If (5) is adopted, then that same vector will be equivalent to a vector with monotonically descending components (Figure 1).Figure 1
In the standard permutation entropy symbolization, a sequence of same values (a) is equivalent either to ascending (b) or descending (c) sequence, if either (4) or (5) convention is used.
(a)(b)(c)Without knowing a real system, it is not clear which case is better and whether it is good or bad to label a sequence of same values as being decreasing or increasing. Actually, the permutation symbolization technique aims at reducing redundancy. Discrimination between constant and either increasing or decreasing sequences of data may appear to be excessive in some cases. On the contrary, when a system, which generates data, has a few possible outputs, the data were subjected to a crude quantization, or embedding dimension is large, it may happen to be useful if the presence of equal values in the embedding vector results in the order pattern preserving this fact. One possible approach to do this is discussed in the next section.


## 2.1. Treatment of Equal Values

Equal values in an embedding vector are, to an extent, inconvenient. Indeed, ifxr=xs for some 0≤r,s<D in a vector V=x0,x1,…,xD−1, then r and s should be placed side by side in the permutation (6), but which one should go first? Due to the sameness of values, it is impossible to uniquely determine a corresponding permutation without introducing additional rules. In some cases, the possibility of equal values can be ignored due to their low probability. This is reasonable when the embedding dimension is low and/or a chaotic process data are recorded with high precision [1, 15, 16]. If equal values are inevitable, the following rule is applied (in some cases, e.g. [10, 11], the opposite inequality sign is used here):(9)ifxs=xrands>rthensgoes first.The rule (9) has different meaning depending on whether (4) or (5) convention is used. Namely, in the case of (4), an embedding vector with all components equal will be equivalent to a vector with monotonically ascending components. If (5) is adopted, then that same vector will be equivalent to a vector with monotonically descending components (Figure 1).Figure 1
In the standard permutation entropy symbolization, a sequence of same values (a) is equivalent either to ascending (b) or descending (c) sequence, if either (4) or (5) convention is used.
(a)(b)(c)Without knowing a real system, it is not clear which case is better and whether it is good or bad to label a sequence of same values as being decreasing or increasing. Actually, the permutation symbolization technique aims at reducing redundancy. Discrimination between constant and either increasing or decreasing sequences of data may appear to be excessive in some cases. On the contrary, when a system, which generates data, has a few possible outputs, the data were subjected to a crude quantization, or embedding dimension is large, it may happen to be useful if the presence of equal values in the embedding vector results in the order pattern preserving this fact. One possible approach to do this is discussed in the next section.


## 3. Arithmetic Entropy

### 3.1. Symbolization

The following symbolization is aimed to keep information about equal values in embedding vectors. Having a vectorV=x0,x1,…,xD−1, construct a sequence of integers α:(10)V=x0,x1,…,xD−1⟶α=αV=a0,a1,…,aD−1,by using the following rule: find the smallest component, c0, in V. If c0 is found at places r1,r2,…, put number 0 at those places in αV. Find the next smallest component c1, c1>c0 in V. If c1 is found at places s1,s2,…, put number 1 at those places in α. Proceed this way until components of V are exhausted. At this stage, all D components of α will be determined. α obtained this way is used as a symbol of embedding vector V.For example, considerV=7,15,7,25,15. The corresponding symbol, or the order pattern, is α=0,1,0,2,1. Here, information about equal values and their positions is preserved.IfV has no equal components, it can be proven (Appendix A) that α=π−1. This means that α is the inverse permutation of the one obtained for V if convention (5) is used. Since correspondence between permutations and their inverse is one-to-one, it does not matter which one, π or α, is used for calculating entropy. This further means that for a data set and embedding method, which does not deliver equal values in the embedding vectors, symbolization used here is equivalent to the permutation one (it seems that in paper [5], the symbolization method described here is used. But, as it may be concluded from [5] (equation (6)), the issue of equal values is not addressed. Similar approach is used in [12, 17], again without considering equal values.) while calculating entropy.


### 3.2. Arithmetization

Expect that embedding vectorV in (10) has exactly d unique components, where d≤D. In this case, the corresponding symbol αV will be a sequence of D numbers chosen from the set d=0,1,…,d−1 in such a way that not any element from d is missed. The latter can be formulated as the following condition:(11)∧b∈db∈αV.The sequenceαV can be considered as a single integer AV, in a base-D positional numeral system with digits aD−1aD−2…a0:(12)AV≡A=a0+a1D+a2D2+⋯+aD−1DD−1.(For a single embedding vector,d might be chosen as radix instead of D. But d may be different for different vectors. And a same integer may have different representation for different bases with (11) satisfied, e.g., 01123 = 11102.). It is clear that there is one-to-one correspondence between order patterns α and integers obtained as shown in (12). Therefore, a set of order patterns, constructed as described in Section 3.1, can be replaced with a set A of integers obtained as shown in (12):(13)A=A0,A1,…,AN−D,whereAi≡AVi.The empirical probabilitiespAi to find an integer Ai among those in A can be calculated as usual, and we have for the arithmetic entropy:(14)HaV≡HaA=−∑i=0L−1pAilogpAi,where L is the number of different integers in A.For a data sequence and embedding method which does not deliver equal values in the embedding vectors, alldi=D and the integers Ai will represent corresponding permutation order patterns unambiguously. In this case, Amin≤Ai≤Amax, where Amin corresponds to pattern αmin=D−1,D−2,…,1,0:(15)Amin=D−1+D−2D+D−3D2+⋯+DD−2.AndAmax corresponds to pattern αmax=0,1,…,D−1:(16)Amax=D+2D2+3D3+⋯+D−1DD−1.In this case, onlyD! integers will be used from Amin;Amax due to condition (11).


### 3.3. How Many New Possible Order Patterns Are Got?

If it is decided to treat the order patterns generated from embeddingD-vectors with some components equal as not equivalent to those from vectors with all components different, then the number of all possible patterns will be greater than D!. Here we attempt to estimate how many new patterns can be obtained.Any new pattern appears from embeddingD-vector with d different components, where d∈1,2,…,D−1. So, having d fixed, the number of corresponding new patterns is equal to the number ND,d of base-DD-digit integers constructed from digits 0,1,…,d−1 in such a way that each of the d digits is used at least once. This number can be calculated as(17)ND,d=d!Dd,where Dd is the Stirling numbers of the second kind ([18], Part 5, Section 2). Considering all possible values for d, we have for the total number of possible new patterns:(18)ND=∑0<d<Dd!Dd=bD−D!,where bD are known as the ordered Bell numbers, see ([19], p. 337) for naming discussion. Calculating (the Stirling numbers were calculated with stirling2(D, d) function in the “maxima” computer algebra system (http://maxima.sourceforge.net/)) ND for D∈2,3,4,5,6,7, we see that the number of new patterns is normally greater than D!, see Figure 2 and also Table 1. Of course, the possible new patterns may only be significant when they can be observed (see discussion about this in [7]). This depends on the process under study and embedding method.Figure 2
Comparison of the possible number of patterns. (•): equal values are treated as described in Section 2.1, D!; (∗): additional possible patterns due to equal values, ND graph, equation (18).Table 1
Total number of possible patterns in the AE symbolization.
D234567ND+D!31375541468347293


### 3.4. Coding

Certainly, there are several possible implementations of the algorithm discussed in Sections3.1 and 3.2. Here, the one used for the examples in Section 4, and Appendix C, is shown. It is a C++ program. It is expected that the sequence (1) is organized into a one-dimensional array X[N]. For calculating the arithmetic order pattern of vector Vi shown in (3), it is necessary to pass a pointer to X[i] to the function get_numerical_pattern, below, as its third argument: data_point = X + i.In the below example,X[i] is declared as double, but it can be of any type with appropriate sorting defined. The returning value is declared as mpz_class, which is a GNU multiple precision integer (https://gmplib.org/). This is used because for embedding dimensions D>15, the returned number representing an order pattern may exceed 64 bits in size (it makes sense to use large embedding dimensions only for very long sequences of data. Otherwise, any observed pattern appears only once, which is unfavorable for estimating probabilities). For smaller D, mpz_class can be replaced with int or long everywhere in the code.(1)
#include <gmp.h>(2)
#include <gmpxx.h>(3)
#include <forward_list>(4)(5)
/∗∗(6)
Function calculates numerical representation of order pattern(7)
of an embedding vector V_i = {(x_i, x_{i + tau},…).(8)
HereD is the embedding dimension and tau is the delay.(9)
The data_point points to the first component of Vi in the(10)
array of raw data.(11)
∗/(12)
mpz_class get_numerical_pattern (int D, int tau, double∗ data_point)(13)
{(14)
intk;(15)
std:forward_list<double> FL;(16)
auto it = FL.before_begin();(17)
for (k = 0; k < D; k++) it = FL.emplace_after (it, data_point [k∗tau]);(18)
FL.sort ();(19)
FL.unique ();(20)(21)
int∗ pDpnm = new int [D]; //order pattern will be here(22)
int tag = 0;(23)
for (auto it = FL.begin (); it != FL.end (); ++it)(24)
{(25)
for (k = 0; k < D; k++)(26)
if (∗it == data_point [k∗tau]) pDpnm [k] = tag;(27)
tag++;(28)
}(29)(30)
mpz_class pnum = 0; // arithmetic order pattern (initial value)(31)
mpz_class digval = 1; // initial value of a single digit(32)
for (k = 0; k < D; k++)(33)
{(34)
pnum += pDpnm [k]∗digval;(35)
digval∗= D;(36)
}(37)
return pnum;(38)
}This code is transparent and does not refer to combinatorics. At the same time, provided an embedding vector does not have equal components, when loop at lines 23–28 above is complete, we obtain in the arraypDpnm [D] a permutation π−1, where π is the permutation for that vector obtained in accordance with the standard rules of [1] reproduced in Section 2 with (5) adopted.


## 3.1. Symbolization

The following symbolization is aimed to keep information about equal values in embedding vectors. Having a vectorV=x0,x1,…,xD−1, construct a sequence of integers α:(10)V=x0,x1,…,xD−1⟶α=αV=a0,a1,…,aD−1,by using the following rule: find the smallest component, c0, in V. If c0 is found at places r1,r2,…, put number 0 at those places in αV. Find the next smallest component c1, c1>c0 in V. If c1 is found at places s1,s2,…, put number 1 at those places in α. Proceed this way until components of V are exhausted. At this stage, all D components of α will be determined. α obtained this way is used as a symbol of embedding vector V.For example, considerV=7,15,7,25,15. The corresponding symbol, or the order pattern, is α=0,1,0,2,1. Here, information about equal values and their positions is preserved.IfV has no equal components, it can be proven (Appendix A) that α=π−1. This means that α is the inverse permutation of the one obtained for V if convention (5) is used. Since correspondence between permutations and their inverse is one-to-one, it does not matter which one, π or α, is used for calculating entropy. This further means that for a data set and embedding method, which does not deliver equal values in the embedding vectors, symbolization used here is equivalent to the permutation one (it seems that in paper [5], the symbolization method described here is used. But, as it may be concluded from [5] (equation (6)), the issue of equal values is not addressed. Similar approach is used in [12, 17], again without considering equal values.) while calculating entropy.


## 3.2. Arithmetization

Expect that embedding vectorV in (10) has exactly d unique components, where d≤D. In this case, the corresponding symbol αV will be a sequence of D numbers chosen from the set d=0,1,…,d−1 in such a way that not any element from d is missed. The latter can be formulated as the following condition:(11)∧b∈db∈αV.The sequenceαV can be considered as a single integer AV, in a base-D positional numeral system with digits aD−1aD−2…a0:(12)AV≡A=a0+a1D+a2D2+⋯+aD−1DD−1.(For a single embedding vector,d might be chosen as radix instead of D. But d may be different for different vectors. And a same integer may have different representation for different bases with (11) satisfied, e.g., 01123 = 11102.). It is clear that there is one-to-one correspondence between order patterns α and integers obtained as shown in (12). Therefore, a set of order patterns, constructed as described in Section 3.1, can be replaced with a set A of integers obtained as shown in (12):(13)A=A0,A1,…,AN−D,whereAi≡AVi.The empirical probabilitiespAi to find an integer Ai among those in A can be calculated as usual, and we have for the arithmetic entropy:(14)HaV≡HaA=−∑i=0L−1pAilogpAi,where L is the number of different integers in A.For a data sequence and embedding method which does not deliver equal values in the embedding vectors, alldi=D and the integers Ai will represent corresponding permutation order patterns unambiguously. In this case, Amin≤Ai≤Amax, where Amin corresponds to pattern αmin=D−1,D−2,…,1,0:(15)Amin=D−1+D−2D+D−3D2+⋯+DD−2.AndAmax corresponds to pattern αmax=0,1,…,D−1:(16)Amax=D+2D2+3D3+⋯+D−1DD−1.In this case, onlyD! integers will be used from Amin;Amax due to condition (11).


## 3.3. How Many New Possible Order Patterns Are Got?

If it is decided to treat the order patterns generated from embeddingD-vectors with some components equal as not equivalent to those from vectors with all components different, then the number of all possible patterns will be greater than D!. Here we attempt to estimate how many new patterns can be obtained.Any new pattern appears from embeddingD-vector with d different components, where d∈1,2,…,D−1. So, having d fixed, the number of corresponding new patterns is equal to the number ND,d of base-DD-digit integers constructed from digits 0,1,…,d−1 in such a way that each of the d digits is used at least once. This number can be calculated as(17)ND,d=d!Dd,where Dd is the Stirling numbers of the second kind ([18], Part 5, Section 2). Considering all possible values for d, we have for the total number of possible new patterns:(18)ND=∑0<d<Dd!Dd=bD−D!,where bD are known as the ordered Bell numbers, see ([19], p. 337) for naming discussion. Calculating (the Stirling numbers were calculated with stirling2(D, d) function in the “maxima” computer algebra system (http://maxima.sourceforge.net/)) ND for D∈2,3,4,5,6,7, we see that the number of new patterns is normally greater than D!, see Figure 2 and also Table 1. Of course, the possible new patterns may only be significant when they can be observed (see discussion about this in [7]). This depends on the process under study and embedding method.Figure 2
Comparison of the possible number of patterns. (•): equal values are treated as described in Section 2.1, D!; (∗): additional possible patterns due to equal values, ND graph, equation (18).Table 1
Total number of possible patterns in the AE symbolization.
D234567ND+D!31375541468347293


## 3.4. Coding

Certainly, there are several possible implementations of the algorithm discussed in Sections3.1 and 3.2. Here, the one used for the examples in Section 4, and Appendix C, is shown. It is a C++ program. It is expected that the sequence (1) is organized into a one-dimensional array X[N]. For calculating the arithmetic order pattern of vector Vi shown in (3), it is necessary to pass a pointer to X[i] to the function get_numerical_pattern, below, as its third argument: data_point = X + i.In the below example,X[i] is declared as double, but it can be of any type with appropriate sorting defined. The returning value is declared as mpz_class, which is a GNU multiple precision integer (https://gmplib.org/). This is used because for embedding dimensions D>15, the returned number representing an order pattern may exceed 64 bits in size (it makes sense to use large embedding dimensions only for very long sequences of data. Otherwise, any observed pattern appears only once, which is unfavorable for estimating probabilities). For smaller D, mpz_class can be replaced with int or long everywhere in the code.(1)
#include <gmp.h>(2)
#include <gmpxx.h>(3)
#include <forward_list>(4)(5)
/∗∗(6)
Function calculates numerical representation of order pattern(7)
of an embedding vector V_i = {(x_i, x_{i + tau},…).(8)
HereD is the embedding dimension and tau is the delay.(9)
The data_point points to the first component of Vi in the(10)
array of raw data.(11)
∗/(12)
mpz_class get_numerical_pattern (int D, int tau, double∗ data_point)(13)
{(14)
intk;(15)
std:forward_list<double> FL;(16)
auto it = FL.before_begin();(17)
for (k = 0; k < D; k++) it = FL.emplace_after (it, data_point [k∗tau]);(18)
FL.sort ();(19)
FL.unique ();(20)(21)
int∗ pDpnm = new int [D]; //order pattern will be here(22)
int tag = 0;(23)
for (auto it = FL.begin (); it != FL.end (); ++it)(24)
{(25)
for (k = 0; k < D; k++)(26)
if (∗it == data_point [k∗tau]) pDpnm [k] = tag;(27)
tag++;(28)
}(29)(30)
mpz_class pnum = 0; // arithmetic order pattern (initial value)(31)
mpz_class digval = 1; // initial value of a single digit(32)
for (k = 0; k < D; k++)(33)
{(34)
pnum += pDpnm [k]∗digval;(35)
digval∗= D;(36)
}(37)
return pnum;(38)
}This code is transparent and does not refer to combinatorics. At the same time, provided an embedding vector does not have equal components, when loop at lines 23–28 above is complete, we obtain in the arraypDpnm [D] a permutation π−1, where π is the permutation for that vector obtained in accordance with the standard rules of [1] reproduced in Section 2 with (5) adopted.


## 4. Example

The discussed methodology has been tested at two surrogate sequences. The purpose was to demonstrate that for a pair of sequences, the standard permutation entropy method gives roughly the same entropy, whereas the arithmetic entropy may be considerably different.For calculating standard permutation entropy in situation when equal components in embedding vectors are possible, we replace the following fragment:(i)
for (auto it = FL.begin (); it != FL.end (); ++it)(ii)
{(iii)
for (k = 0; k < D; k++)(iv)
if (∗it == data_point [k∗tau]) pDpnm [k] = tag;(v)
tag++;(vi)
},in the code of Section3.4, with the following one:(i)
for (auto it = FL.begin (); it != FL.end (); ++it)(ii)
{(iii)
for (k = D−1; k >= 0; k--)(iv)
if (∗it == data_point [k∗tau]) pDpnm [k] = tag++;(v)
}With such a replacement, we get in the arraypDpnm [D] above the permutation, which is inverse to the one obtained for Vi in the standard permutation entropy symbolization with rules (5) and (9) adopted. As it was mentioned above, usage of inverse permutations instead of the initial ones delivers the same value for the standard permutation entropy.The two sequences, S1 and S2, are obtained as follows: by means of function gsl_rng_uniform_int from the GNU Scientific Library [20], we generate random numbers from the set 0,1,…,4, which are equally probable. Each obtained random number “val” is written into the S1. The same number is written into S2 provided it is not equal to the number written to S2 at the previous step. If it does, then the number (val + 1) (mod 5) is written instead. This introduces a nonzero correlation between consecutive values in S2. For example, in S2, any two consecutive values are always different. Examples of S1 and S2 are as follows:(19)S1=2,2,0,3,4,0,1,3,4,4,0,3,3,2,2,4,4,2,0,1,…,S2=2,3,0,3,4,0,1,3,4,0,1,3,4,2,3,4,0,2,0,1,….1 000 000 long S1 and S2 were produced and both permutation and arithmetic entropy have been calculated. The results are shown in Tables2 and 3.Table 2
Comparison of permutation entropy (PE) and arithmetic entropy (AE) for embedding delayτ=1.
D=3PEAED=4PEAES12.4973.684S14.3906.165S22.3682.919S24.1875.238Entropy is given in bits.Table 3
Same as Table2 for embedding delay τ=2.
D=3PEAED=4PEAES12.4983.684S14.3936.166S22.4073.676S24.2246.098Notice that arithmetic entropy is considerably greater than the permutation one. This is due to the high frequency of embedding vectors with equal components. Also, from Table2 with τ=1, it can be seen that arithmetic entropy discriminates better between S1 and S2. However, the case with delay τ=2 shown in Table 3 is not similarly conclusive. This might be due to the construction method of the S2 sequence. Namely, by pulling from S2 embedding vectors with delay 2, we may get vectors with equal adjacent components, similarly to the S1 case. This alleviates difference between S1 and S2. For τ=1, embedding vectors for S2 do not have equal adjacent components. One more example is in Appendix C.


## 5. Conclusions and Discussion

In this note, we have discussed a method for calculating entropy in a sequence of data, which is similar to the permutation entropy method. The characteristic features of this method are as follows:(i)
It treats equal components in the embedding vectors as being equal instead of ordering them artificially(ii)
It is entirely exempt from combinatorics, labeling order patterns by integers instead of permutations(iii)
If embedding vectors do not have equal components, this method delivers exactly the same value for the entropy as does the standard permutation entropy oneIn the symbolization procedure discussed in Section3.1, new order patterns may appear as compared to the standard permutation method (Section 3.3). Those new patterns arise from embedding vectors with some components being equal to each other. In the standard permutation entropy method, the embedding vectors characterized by those new patterns, if any, are labeled by permutations as if there were no equal components. This is made possible through ordering equal values in accordance with the rule (9).Mathematically, replacing embedding vectors with their order patterns means constructing a quotient set from the set of all embedding vectors with respect to some equivalence relation [10, 21, 22]. In the case of permutation entropy, the corresponding equivalence relation is defined by using (9) and either (4) or (5). Denote it by ∼P. For arithmetic entropy, the corresponding equivalence relation is defined by using the algorithm described in the first paragraph of Section 3.1. Denote it by ∼A. It is clear that for two embedding vectors U and V, if U∼AV, then U∼PV. Namely, if U and V have the same arithmetic order pattern, then they do have the same permutation order pattern. That means that ∼P is coarser relation than ∼A. Other equivalence relations could be offered, which are courser than ∼P, finer than ∼A, lying in between, or incomparable with the both, see [12]. A symbolization which still uses permutations, but is equivalent to the one discussed here, as regards the treatment of equal values in embedding vectors, has been proposed in [11], see discussion in Appendix B. Which one is better depends on the data sequence and which kind of redundancy is intended to strip.



---

*Source: 7163254-2020-10-23.xml*