# The Reciprocal Influence Criterion: An Upgrade of the Information Quality Ratio

Authors: Riccardo Rossi; Michela Gelfusa; Filippo De Masi; Matteo Ossidi; Andrea Murari
Journal: Complexity
 (2021)

---

## Abstract

Understanding and quantifying the mutual influence between systems remain crucial but challenging tasks in any scientific enterprise. The Pearson correlation coefficient, the mutual information, and the information quality ratio are the most widely used indicators, only the last two being valid for nonlinear interactions. Given their limitations, a new criterion is proposed, the reciprocal influence criterion, which is very simple conceptually and does not make any assumption about the statistics of the stochastic variables involved. In addition to being normalised as the information quality ratio, it provides a much better resilience to noise and much higher stability to the issues related to the determination of the involved probability distribution functions. A conditional version, to counteract the effects of confounding variables, has also been developed, showing the same advantages compared to the more traditional indicators. A series of systematic tests with numerical examples is reported, to compare the properties of the new indicator with the more traditional ones, proving its clear superiority in practically all respects.


---

## Body

## 1. Determining Statistical Dependence between Quantities

The investigation of the reciprocal influence between systems is a major scientific objective in practically all disciplines. Both linear and nonlinear effects can be important, the latter becoming often dominant in complex phenomena [1]. The information about systems is typically obtained with measurements affected by various forms of uncertainties. The measurands can therefore be considered as random variables, and their mutual influence is partly deterministic and partly probabilistic in nature.Quantifying the dependence between random variables is an essential statistical task for both bivariate and multivariate data. The most popular measure of dependence between two quantities is the Pearson product-moment correlation coefficient or “Pearson’s correlation coefficient” (PCC) [2]. To alleviate its limitations (see Section 2), at the beginning of the last century, various alternative indicators based on ranked variables were developed, such as the Spearman ρ or Spearman rank correlation coefficient [3] and the rank correlation coefficient or Kendall τ [4]. After the Second World War information theoretic criteria, such as mutual information, became quite popular [5]. At the dawn of the new century, a lot of work was devoted to consolidating distance correlation, an indicator that is zero only when the two variables are not correlated [6]. In the last years, the advances in genetics have also motivated the development of various techniques for the analysis of the dependence between vectors in high dimensions. For this multivariate inference, very sophisticated approaches have been proposed, ranging from projection correlation [7] to Ball covariance [8] and Brownian distance [9]. All these techniques are based on specific assumptions about the statistical properties of the vectors of random variables to consider. They are also quite involved, both conceptually and numerically.The indicators proposed in this paper are meant to deal only with bivariate cases, for which no prior information is known about the statistics of the data, so fully general techniques are required. A conditional version to counteract the effects of confounding variables has also been developed. The indicators are also very simple from the point of view of the conceptual background, interpretation of the results, and implementation requirements.Regarding the structure of the paper, Section2 introduces the most consolidated indicators, typically used to assess the dependence between two variables. Section 3 discusses the main rationale behind the proposed alternative indicator: the reciprocal influence criterion. The systematic tests, comparing the properties of RIC with PCC and IQR, are reported in Sections 4 and 5, for functional and nonfunctional dependencies, respectively. The robustness against the noise of statistics, different from the Gaussian and against outliers, is discussed in Section 6. The conditional version of RIC is introduced in Section 7, before conclusions are drawn in Section 8.


## 2. Traditional and Alternative Criteria to Quantify the Correlations between Quantities

In order to fully appreciate the potential of the new indicators proposed in the paper, it is worth starting the discussion with a review of the traditional and alternative tools most used by practitioners for the analysis of bivariate dependence. The linear correlations between two quantities are typically calculated with the Pearson correlation coefficient (PCC). For a couple of random variablesX and Y, indicating with cov as the covariance and with σ as the standard deviation, the PCC is calculated as [10](1)PCC=covX,Yσxσy.Even if it is very useful in practice, some limitations of the PCC, particularly, its vulnerability to noise, are very often overlooked. In any case, the detection of nonlinear correlations between two variables is a much more serious matter. To quantify nonlinear dependencies, historically, the first techniques developed were based on ranking variables (see Section2.1). After the Second World War, indicators based on the probability distribution function (pdf) of the data have been developed; particularly, information theoretic criteria are popular (Section 2.2). Distance correlation is a much more recent development, first introduced in 2005, to remedy the problem that the Pearson correlation coefficient can be zero for dependent variables (Section 2.3).


### 2.1. Rank-Based Criteria

In this paper, the symbolρS indicates the extension of the Pearson correlation coefficient introduced by Charles Spearman. Spearman’s ρS is a nonparametric measure of the statistical dependence between the rankings of two variables; as such, it quantifies how well the relationship between two variables can be modelled by a monotonic function [11]. Spearman’s ρS is calculated as the Pearson correlation between the rank values of the quantities involved and assesses how monotonic the relation between them is (whether linear or not). If there are no repeated data values, ρS assumes the values +1 or −1 when the two variables are perfect monotone functions of each other.To calculate Spearman’sρS, the raw data Xi and Yi are converted into ranks first: rankX and rankY; the standard deviations of these two ranked variables are indicated with the symbols σrankX and σrankY. Spearman’s ρS is then defined as(2)ρS=covrankx,rankyσrankxσranky.The Kendall rank correlation coefficient, named after Maurice Kendall, was developed at the end of the 30 s, and it is usually indicated with the Greek letterτ. It is meant to measure the rank correlation, i.e., the similarity of the ordering of the data when ranked. Intuitively, the higher the Kendall correlation between two variables, the more similar their rank; the Kendall τ is also normalised in the sense that it ranges between 1 and −1 [11].Mathematically, the Kendall rank correlation coefficient is calculated as(3)τ=Number of concordant pairs−Number of discordant pairsn2,where the denominator is the binomial coefficient n2=nn−1/2.


### 2.2. Information Theoretic Criteria

A widely used indicator to investigate nonlinear interactions is the mutual information, which quantifies the information shared by two systems. The Shannon or discrete version is defined as [12, 13](4)MIShan=∑∑pxylogpxypxpy,where px and py are the discrete probabilities of two random variables X and Y and pxy is their joint probability.A differential version of the indicator can be formulated in terms of the probability densitiesfx and fy:(5)fx=pxΔx,fy=pyΔy,fxy=pxyΔxΔy,where Δx and Δy are the dimensions of the bins. The differential mutual information, MIdiff, is defined as(6)MIDiff=∫∫fxylogfxyfxfydxdy.It can be easily demonstrated that the two relations (4) and (6) are equivalent; the mutual information is, therefore, referred to as MI in the rest of the paper. MI has various positive properties but presents the main limitation of not being normalised. To obviate this drawback, the mutual information is typically divided by the joint entropy to obtain the so-called information quality ratio (IQR) [14]:(7)IQR=MIHX,Y.This quantity is normalised in the sense that it assumes only values between zero and one. It is important to remember that, in equation (7), the discrete or Shannon version of the joint entropy is to be used:(8)HS,XY=−∑∑pxylogpxy.The Shannon entropy is to be compared with the differential one, which can also assume negative values:(9)HD,XY=−∫∫fxylogfxydxdy.Equation (7) is the commonly accepted version of the information quality ratio, normally adopted because the differential version of the joint entropy can be negative, with the obvious related problems and difficulties.Unfortunately, mainly due to the denominator, the IQRS has some limitations, particularly, a lack of robustness to noise and a strong dependence on the choice of the bins.


### 2.3. Distance Correlation

The objective of distance correlation (Dcorr) consists of quantifying the dependence between two random vectors, which do not need to have necessarily equal dimension. Dcorr has the clear advantage, compared to the PCC, that the population distance correlation coefficient assumes a zero value only if the two random vectors are independent. Therefore, distance correlation is meant to quantify both linear and nonlinear association between two random variables or random vectors [5].The calculation ofDcorr requires the definition of some other preliminary quantities. Indicating with (Xk, Yk), k = 1, 2, …, n, a sample from a pair of real-valued or vector-valued random variables (X, Y), the elements of the n by n distance matrices (aj, k) and (bj, k) are all pairwise distances:(10)aj,k=Xj−Xk,j,k=1,2,…,n,bj,k=Yj−Yk,j,k=1,2,…,n,where || || denotes the Euclidean norm. Defining a¯j and a¯kas the jth row mean and the kth column mean, respectively, and with a¯ the grand mean of the first vector mutual distance X matrix, one can then calculate all doubly centred distances (with the same notation for the Y vector matrix):(11)Aj,k=aj,k−a¯j−a¯k+a¯,Bj,k=bj,k−b¯j−b¯k+b¯.The desired sample distance covariance is then simply the arithmetic average of the productAj,kBj,k:(12)dCovn2X,Y≔1n2∑j=1n∑k=1nAj,kBj,k.Indicating with distance variance,(13)dVarn2X,X≔1n2∑k,l=1nAk,l2,finally, the distance correlation is(14)dCorX,Y≔dCovX,YdVarXdVarY.The main properties ofdCor are that it assumes values between 0 and 1, and it is zero only if the two vectors are independent. The distance correlation software used in this work is the one published by Shen Liu [15].


## 2.1. Rank-Based Criteria

In this paper, the symbolρS indicates the extension of the Pearson correlation coefficient introduced by Charles Spearman. Spearman’s ρS is a nonparametric measure of the statistical dependence between the rankings of two variables; as such, it quantifies how well the relationship between two variables can be modelled by a monotonic function [11]. Spearman’s ρS is calculated as the Pearson correlation between the rank values of the quantities involved and assesses how monotonic the relation between them is (whether linear or not). If there are no repeated data values, ρS assumes the values +1 or −1 when the two variables are perfect monotone functions of each other.To calculate Spearman’sρS, the raw data Xi and Yi are converted into ranks first: rankX and rankY; the standard deviations of these two ranked variables are indicated with the symbols σrankX and σrankY. Spearman’s ρS is then defined as(2)ρS=covrankx,rankyσrankxσranky.The Kendall rank correlation coefficient, named after Maurice Kendall, was developed at the end of the 30 s, and it is usually indicated with the Greek letterτ. It is meant to measure the rank correlation, i.e., the similarity of the ordering of the data when ranked. Intuitively, the higher the Kendall correlation between two variables, the more similar their rank; the Kendall τ is also normalised in the sense that it ranges between 1 and −1 [11].Mathematically, the Kendall rank correlation coefficient is calculated as(3)τ=Number of concordant pairs−Number of discordant pairsn2,where the denominator is the binomial coefficient n2=nn−1/2.


## 2.2. Information Theoretic Criteria

A widely used indicator to investigate nonlinear interactions is the mutual information, which quantifies the information shared by two systems. The Shannon or discrete version is defined as [12, 13](4)MIShan=∑∑pxylogpxypxpy,where px and py are the discrete probabilities of two random variables X and Y and pxy is their joint probability.A differential version of the indicator can be formulated in terms of the probability densitiesfx and fy:(5)fx=pxΔx,fy=pyΔy,fxy=pxyΔxΔy,where Δx and Δy are the dimensions of the bins. The differential mutual information, MIdiff, is defined as(6)MIDiff=∫∫fxylogfxyfxfydxdy.It can be easily demonstrated that the two relations (4) and (6) are equivalent; the mutual information is, therefore, referred to as MI in the rest of the paper. MI has various positive properties but presents the main limitation of not being normalised. To obviate this drawback, the mutual information is typically divided by the joint entropy to obtain the so-called information quality ratio (IQR) [14]:(7)IQR=MIHX,Y.This quantity is normalised in the sense that it assumes only values between zero and one. It is important to remember that, in equation (7), the discrete or Shannon version of the joint entropy is to be used:(8)HS,XY=−∑∑pxylogpxy.The Shannon entropy is to be compared with the differential one, which can also assume negative values:(9)HD,XY=−∫∫fxylogfxydxdy.Equation (7) is the commonly accepted version of the information quality ratio, normally adopted because the differential version of the joint entropy can be negative, with the obvious related problems and difficulties.Unfortunately, mainly due to the denominator, the IQRS has some limitations, particularly, a lack of robustness to noise and a strong dependence on the choice of the bins.


## 2.3. Distance Correlation

The objective of distance correlation (Dcorr) consists of quantifying the dependence between two random vectors, which do not need to have necessarily equal dimension. Dcorr has the clear advantage, compared to the PCC, that the population distance correlation coefficient assumes a zero value only if the two random vectors are independent. Therefore, distance correlation is meant to quantify both linear and nonlinear association between two random variables or random vectors [5].The calculation ofDcorr requires the definition of some other preliminary quantities. Indicating with (Xk, Yk), k = 1, 2, …, n, a sample from a pair of real-valued or vector-valued random variables (X, Y), the elements of the n by n distance matrices (aj, k) and (bj, k) are all pairwise distances:(10)aj,k=Xj−Xk,j,k=1,2,…,n,bj,k=Yj−Yk,j,k=1,2,…,n,where || || denotes the Euclidean norm. Defining a¯j and a¯kas the jth row mean and the kth column mean, respectively, and with a¯ the grand mean of the first vector mutual distance X matrix, one can then calculate all doubly centred distances (with the same notation for the Y vector matrix):(11)Aj,k=aj,k−a¯j−a¯k+a¯,Bj,k=bj,k−b¯j−b¯k+b¯.The desired sample distance covariance is then simply the arithmetic average of the productAj,kBj,k:(12)dCovn2X,Y≔1n2∑j=1n∑k=1nAj,kBj,k.Indicating with distance variance,(13)dVarn2X,X≔1n2∑k,l=1nAk,l2,finally, the distance correlation is(14)dCorX,Y≔dCovX,YdVarXdVarY.The main properties ofdCor are that it assumes values between 0 and 1, and it is zero only if the two vectors are independent. The distance correlation software used in this work is the one published by Shen Liu [15].


## 3. The Reciprocal Influence Criterion: Rationale

As will be shown in the rest of the paper, all criteria summarised in the previous section have several drawbacks. In this work, a new indicator, able to quantify linear and nonlinear correlations between variables, is introduced. The new indicator, based on information theoretic quantities and named reciprocal influence criterion (RIC), has been designed to have the following properties.


### 3.1. Property 1

The indicator ranges from zero (no correlation) to one (perfectly correlated). The definition of correlation for this indicator is the following.Two variablesi and j are correlated when the knowledge of i helps to predict j and vice versa. The correlation indicator tends to one when the uncertainty of i (j) known j (i) goes to zero. Correlation must be equal to zero when the uncertainty of i (j) does not change when the j (i) is known. Note that, in this definition, symmetric equations cannot reach the value of one; as, for example, for the function y=x2, since y is known, there are two valid values of x (excluded x = 0 and y = 0).


### 3.2. Property 2

The indicator does not vary as a function of the binning used for the probability density or distribution function calculations, i.e.,(15)ΔNbinNbin≫ΔRICRIC,where ΔRIC denotes the difference between the indicator values calculated with and without the outliers.


### 3.3. Property 3

The indicator can be calculated indifferently for either discrete or continuous random variables.


### 3.4. Property 4

Small sensitivity to outliers is quantified as(16)NoutliersN≫ΔRICRIC.The formulation of the RIC that ensures the former properties is(17)RIC=1−AHxyAHx+Hy=1−AHxyAHx+Hy=1−1AMI.The RIC indicator is based on the ratio between the amount of information (uncertainty) shared by the two variables with respect to the sum of their individual information (uncertainty). In some ways, its definition is very similar to the IQR indicator, with the main difference that the RIC is based on the exponential of the entropyAH. This approach allows obtaining a much more reliable indicator, which satisfies the aforementioned four desirable properties, as demonstrated in the following, and presents some additional positive qualities, namely, symmetry, asymptotic consistency, and unbiasedness, which are discussed in Appendix A.With regard to the free parameter A, the choice of its numerical value can be optimised, depending on the nature of the data and the objectives of the investigation. The results, reported in the rest of the paper, have been obtained by settingA = 10, to maximise the coherence of RIC with PCC in case of linear correlations; the equations in the following are also particularised for this numerical value. A detailed discussion of the RIC behaviour with the parameter A is provided in Appendix B.Some of the aforementioned four properties can be analytically demonstrated. First, property 1 can be proven as follows. For not correlated variables,Hx+Hy=Hxy and MI ⟶ 0; thus,(18)limMI⟶0RIC=limMI⟶01−10HD,xy10HD,x+HD,y=limMI⟶01−110MI=0.In the case of partially correlated quantities, RIC is always positive, and for high mutual information, it tends to one. More specifically, it is known that, for high correlation levels, the joint entropy tends to the lowest entropy of the two variables (Hxy⟶minHx,Hy) [16], which means that MI⟶maxHx,Hy. Assuming Hy > Hx, it follows(19)limHxy⟶HxRIC=limHxy⟶Hx1−10Hxy10Hx+Hy=limHxy⟶Hx1−10Hx10Hx+Hy=limHxy⟶Hx1−110Hy>0.Property 3 can also be proved. Indeed, writing the RIC using the differential information theoretic quantities and remembering that the sum of the entropies is equal to the sum of the mutual information and the joint entropy, it is possible to demonstrate that RICD is the same as RICS:(20)RICD=1−10HD,xy10HD,x+HD,y=1−110MI=1−10HS,xy10HS,x+HS,y=RICS.Moreover, RIC is directly correlated to the mutual information value; also, property 2 is expected to be satisfied (and it is validated in Section4).In terms of interpretation, RIC remains an information theoretic indicator, since it relies on the mutual information. On the contrary, the monotonic transformation of (15) provides a practically normalised indicator without having to make recourse to the joint entropy. The advantages of such a reformulation will be illustrated in detail, with the help of numerical tests, in the next sections.


## 3.1. Property 1

The indicator ranges from zero (no correlation) to one (perfectly correlated). The definition of correlation for this indicator is the following.Two variablesi and j are correlated when the knowledge of i helps to predict j and vice versa. The correlation indicator tends to one when the uncertainty of i (j) known j (i) goes to zero. Correlation must be equal to zero when the uncertainty of i (j) does not change when the j (i) is known. Note that, in this definition, symmetric equations cannot reach the value of one; as, for example, for the function y=x2, since y is known, there are two valid values of x (excluded x = 0 and y = 0).


## 3.2. Property 2

The indicator does not vary as a function of the binning used for the probability density or distribution function calculations, i.e.,(15)ΔNbinNbin≫ΔRICRIC,where ΔRIC denotes the difference between the indicator values calculated with and without the outliers.


## 3.3. Property 3

The indicator can be calculated indifferently for either discrete or continuous random variables.


## 3.4. Property 4

Small sensitivity to outliers is quantified as(16)NoutliersN≫ΔRICRIC.The formulation of the RIC that ensures the former properties is(17)RIC=1−AHxyAHx+Hy=1−AHxyAHx+Hy=1−1AMI.The RIC indicator is based on the ratio between the amount of information (uncertainty) shared by the two variables with respect to the sum of their individual information (uncertainty). In some ways, its definition is very similar to the IQR indicator, with the main difference that the RIC is based on the exponential of the entropyAH. This approach allows obtaining a much more reliable indicator, which satisfies the aforementioned four desirable properties, as demonstrated in the following, and presents some additional positive qualities, namely, symmetry, asymptotic consistency, and unbiasedness, which are discussed in Appendix A.With regard to the free parameter A, the choice of its numerical value can be optimised, depending on the nature of the data and the objectives of the investigation. The results, reported in the rest of the paper, have been obtained by settingA = 10, to maximise the coherence of RIC with PCC in case of linear correlations; the equations in the following are also particularised for this numerical value. A detailed discussion of the RIC behaviour with the parameter A is provided in Appendix B.Some of the aforementioned four properties can be analytically demonstrated. First, property 1 can be proven as follows. For not correlated variables,Hx+Hy=Hxy and MI ⟶ 0; thus,(18)limMI⟶0RIC=limMI⟶01−10HD,xy10HD,x+HD,y=limMI⟶01−110MI=0.In the case of partially correlated quantities, RIC is always positive, and for high mutual information, it tends to one. More specifically, it is known that, for high correlation levels, the joint entropy tends to the lowest entropy of the two variables (Hxy⟶minHx,Hy) [16], which means that MI⟶maxHx,Hy. Assuming Hy > Hx, it follows(19)limHxy⟶HxRIC=limHxy⟶Hx1−10Hxy10Hx+Hy=limHxy⟶Hx1−10Hx10Hx+Hy=limHxy⟶Hx1−110Hy>0.Property 3 can also be proved. Indeed, writing the RIC using the differential information theoretic quantities and remembering that the sum of the entropies is equal to the sum of the mutual information and the joint entropy, it is possible to demonstrate that RICD is the same as RICS:(20)RICD=1−10HD,xy10HD,x+HD,y=1−110MI=1−10HS,xy10HS,x+HS,y=RICS.Moreover, RIC is directly correlated to the mutual information value; also, property 2 is expected to be satisfied (and it is validated in Section4).In terms of interpretation, RIC remains an information theoretic indicator, since it relies on the mutual information. On the contrary, the monotonic transformation of (15) provides a practically normalised indicator without having to make recourse to the joint entropy. The advantages of such a reformulation will be illustrated in detail, with the help of numerical tests, in the next sections.


## 4. The Reciprocal Influence Criterion: Numerical Tests for Functional Dependencies

In this section, the properties of RIC are investigated with the help of a series of numerical tests with synthetic data. The reference indicators to benchmark the performance of RIC are the ones reviewed in Section2, the most used by the practitioners. Only bivariate dependences due to functional relations are considered; nonfunctional dependencies are the subject of Section 4.1. A general overview of the results is provided first and some specific aspects are discussed in dedicated sections.


### 4.1. Comparison Overview

A first comparison between the RIC and the other criteria has been performed for the functional dependencies linear, quadratic, sinusoidal, and exponential. The results for a series of representative cases are reported in Table1. A graphical overview is provided in Figure 1. Random noise of Gaussian distribution, with standard deviation equal to 10% of the quantity value, has been added to all variables.Table 1
The value of the various indicators for the cases of Figure1.
y=xy=x2y=exy=sin3xPearson0.990.030.930.01Spearman0.990.030.980.03Kendall0.930.020.900.01Distance correlation0.990.490.960.35IQR0.470.350.460.33RIC0.970.930.960.90Figure 1
The main classes of functions tested. (a) Examples of synthetic data. (b) The numerical values of the indicators.
(a)(b)The values of Table1 and inspection of Figure 1 reveal that RIC never performs significantly worse than the other criteria for linear dependencies. RIC starts outperforming all other indicators in the case of nonlinear functions. Moreover, as expected, RIC provides much more reliable and reasonable results whenever the functional dependence is nonmonotonic and when even the ranked methods fail miserably. Also, the indicators based on the pdf, IQR, and distance correlation show significant difficulties to provide acceptable results for the nonmonotonic dependencies. All these are general properties not only true for the examples reported but also confirmed in all cases tested.


### 4.2. Linear Correlations: Effects of Gaussian Noise and Binning

This section, with the help of Figure2, is simply aimed at supporting the statement that RIC can reproduce well the values of the PCC for linear correlations. The behaviour of the other indicators is also shown for completeness. The plots of Figure 2 refer to the case of perfect linear correlation between quantities: y = x. On the x-axis, the standard deviation of additive noise and zero mean and sampled randomly from a Gaussian distribution is reported. The results are fully general. The RIC reproduces quite well the values of the PCC, whereas the IQR is significantly more vulnerable to both noise and binning. A similar analysis indicates that RIC is also more robust against the presence of outliers; indeed, it can tolerate about even one order of magnitude more outliers than IQR, confirming that property 4 of Section 3 is satisfied (with 5% of outliers, the average Pearson coefficient variation is about 10% and the IQR variation is 12%, while the ΔRIC is 0.7%).Figure 2
(a) Behavior of PCC, IQR, and RIC with the standard deviation of the additive noise for linear correlations between two variables:y = x. (b) Behavior of PCC, IQR, and RIC with the standard deviation of the additive noise when varying the binning.
(a)(b)


### 4.3. Nonlinear Correlations: Effects of Gaussian Noise and Binning

The competitive advantages of RIC become even more evident in the case of nonlinear correlations. Three exemplificative cases are reported in Figure3, in which the proposed new criterion is compared with IQRS. The functional dependencies reported are y = x2, y = sin (x), and y = exp (x).Figure 3
Behavior of PCC, IQR, and RIC with the standard deviation of the additive noise for various nonlinear dependencies. (a)y = x2. (b) y = sin (x). (c) y = exp (x).
(a)(b)(c)As expected, IQR is much more sensitive to the choice of the binning and the level of noise. RIC remains stable at a value very close to 1 for a much wider range of these factors. Moreover, IQR does not output a value of 1 even for perfect correlation between the two variables. This is a consequence of the denominator not being a normalised quantity. Again, also in the case of nonlinear correlations, RIC provides much more consistent results also in the presence of a significant number of outliers (the comparative resilience is similar to the case of linear correlations).The positive qualities of RIC, compared to PCC and IQR, are not a negligible matter in practice because, in real-life applications, the effects of the noise and the uncertainties about the details of the pdfs can have a strong effect on the conclusions.It should be noted that, from the analysis performed as a function of the binning, it is clear that property 2 of Section3 is satisfied, i.e., ΔNbin/Nbin≫ΔRIC/RIC.


## 4.1. Comparison Overview

A first comparison between the RIC and the other criteria has been performed for the functional dependencies linear, quadratic, sinusoidal, and exponential. The results for a series of representative cases are reported in Table1. A graphical overview is provided in Figure 1. Random noise of Gaussian distribution, with standard deviation equal to 10% of the quantity value, has been added to all variables.Table 1
The value of the various indicators for the cases of Figure1.
y=xy=x2y=exy=sin3xPearson0.990.030.930.01Spearman0.990.030.980.03Kendall0.930.020.900.01Distance correlation0.990.490.960.35IQR0.470.350.460.33RIC0.970.930.960.90Figure 1
The main classes of functions tested. (a) Examples of synthetic data. (b) The numerical values of the indicators.
(a)(b)The values of Table1 and inspection of Figure 1 reveal that RIC never performs significantly worse than the other criteria for linear dependencies. RIC starts outperforming all other indicators in the case of nonlinear functions. Moreover, as expected, RIC provides much more reliable and reasonable results whenever the functional dependence is nonmonotonic and when even the ranked methods fail miserably. Also, the indicators based on the pdf, IQR, and distance correlation show significant difficulties to provide acceptable results for the nonmonotonic dependencies. All these are general properties not only true for the examples reported but also confirmed in all cases tested.


## 4.2. Linear Correlations: Effects of Gaussian Noise and Binning

This section, with the help of Figure2, is simply aimed at supporting the statement that RIC can reproduce well the values of the PCC for linear correlations. The behaviour of the other indicators is also shown for completeness. The plots of Figure 2 refer to the case of perfect linear correlation between quantities: y = x. On the x-axis, the standard deviation of additive noise and zero mean and sampled randomly from a Gaussian distribution is reported. The results are fully general. The RIC reproduces quite well the values of the PCC, whereas the IQR is significantly more vulnerable to both noise and binning. A similar analysis indicates that RIC is also more robust against the presence of outliers; indeed, it can tolerate about even one order of magnitude more outliers than IQR, confirming that property 4 of Section 3 is satisfied (with 5% of outliers, the average Pearson coefficient variation is about 10% and the IQR variation is 12%, while the ΔRIC is 0.7%).Figure 2
(a) Behavior of PCC, IQR, and RIC with the standard deviation of the additive noise for linear correlations between two variables:y = x. (b) Behavior of PCC, IQR, and RIC with the standard deviation of the additive noise when varying the binning.
(a)(b)


## 4.3. Nonlinear Correlations: Effects of Gaussian Noise and Binning

The competitive advantages of RIC become even more evident in the case of nonlinear correlations. Three exemplificative cases are reported in Figure3, in which the proposed new criterion is compared with IQRS. The functional dependencies reported are y = x2, y = sin (x), and y = exp (x).Figure 3
Behavior of PCC, IQR, and RIC with the standard deviation of the additive noise for various nonlinear dependencies. (a)y = x2. (b) y = sin (x). (c) y = exp (x).
(a)(b)(c)As expected, IQR is much more sensitive to the choice of the binning and the level of noise. RIC remains stable at a value very close to 1 for a much wider range of these factors. Moreover, IQR does not output a value of 1 even for perfect correlation between the two variables. This is a consequence of the denominator not being a normalised quantity. Again, also in the case of nonlinear correlations, RIC provides much more consistent results also in the presence of a significant number of outliers (the comparative resilience is similar to the case of linear correlations).The positive qualities of RIC, compared to PCC and IQR, are not a negligible matter in practice because, in real-life applications, the effects of the noise and the uncertainties about the details of the pdfs can have a strong effect on the conclusions.It should be noted that, from the analysis performed as a function of the binning, it is clear that property 2 of Section3 is satisfied, i.e., ΔNbin/Nbin≫ΔRIC/RIC.


## 5. The Reciprocal Influence Criterion: Numerical Tests for Nonfunctional Dependencies

The cases treated in this section, to exemplify the properties of RIC for nonfunctional dependencies (with additive Gaussian noise of mean equal to zero and standard deviation equal to 0.1), are shown in Figure4. These types of dependencies are quite involved and difficult to resolve. They are fully nonlinear and they cannot even be represented by functions. For all these cases, RIC performs significantly better than all other indicators. A synthetic overview of the results is reported in Table 2.Figure 4
Main examples of the main functional dependencies investigated. (a) Examples of synthetic data. (b) The numerical values of the indicators.
(a)(b)Table 2
The value of the various indicators for the case of Figure4.
CircularDouble squaredRhombusPearson0.000.000.03Spearman0.000.000.04Kendall0.000.000.02Distance correlation0.190.310.15IQR0.210.310.05RIC0.800.890.38Inspection of Table2 and Figure 4 reveals that the RIC criterion is always higher than the others by a factor. A part of the case of the rhomboid dependence always provides a value of 0.8 or higher, whereas the other indicators are closer to zero. The RIC, therefore, provides a much more reliable indication that there is a strong correlation between the two variables involved. Even the two other most sophisticated criteria, the IQR and distance correlation, perform significantly worse for all the examples investigated.


## 6. Robustness to Noise of Different Statistics and Outliers

The signals and data acquired in many scientific disciplines are typically affected by noise. The assumption of Gaussian statistics is often justified, but there are also other important types of noise of great practical and theoretical importance. Two of the most relevant distributions are certainly the Poisson and gamma.Poisson distribution:(21)fx=λnn!e−λ,∀n∈ℕ.Gamma distribution:(22)fx=xk−1e−x/θθkΓk,∀x>0,k,θ∈ℕ.Figure5 reports some comparative examples of the performance of the various indicators for these two distributions. The cases reported are full representatives of a series of systematic tests performed to investigate this point. In general, as for the Gaussian distribution reported in Figure 5 as a reference, RIC is also much less affected by additive noise of different distributions.Figure 5
Comparison of RIC and the other criteria for nonlinear correlations,y = 30 sin (3x), in which the signals are affected by additive noise of different statistics.Another potential source of data contamination, of great practical relevance, is the presence of outliers. If the user is aware of the problem and has some prior information about the statistics of the outliers, some measures to remedy the situation can be taken before applying the dependence indicators [17, 18]. These measures belong to the family of robust statistics and can be quite effective. On the contrary, it is not always the case that the practitioner is aware of the issue and therefore investigating the robustness of the various dependence criteria to outliers remains a significant subject. Some representative results of a series of systematic tests, performed to assess this aspect, are shown in Figure 6 for the main classes of functional dependencies. In these cases, the outliers are generated as random Gaussian points with a mean equal to zero and standard deviation comparable to the range of the function. As expected, the most vulnerable indicator is the PCC. The rank-based criteria and the distance correlation are slightly more insensitive; the outliers must be relatively high both in number and amplitude to affect the ranking, before they have a detrimental effect on the values of these indicators. In any case, even in the presence of outliers, RIC remains the most robust criterion.Figure 6
Robustness of RIC and other criteria against outliers. Thex-axis reports the percentage of outliers generated randomly using a Gaussian distribution of large standard deviation.


## 7. The Conditional Version of the Reciprocal Influence Criterion

In many applications of correlation analysis, one fundamental objective consists of determining the mutual influence between variables in the presence of confounding factors. It is therefore natural to investigate the potential of a conditional version of RIC:(23)RICcondX,Z|Y=1−110MIcondX,Z|Y.In the following plots, the performances of RICcond are compared with the ones of a conditional version of IQR:(24)IQRcondX,Z|Y=MIcondX,Z|YHXZ.The plots of Figure7 report only the cases of nonlinear correlations (z = x2+ y, z = x3+ y, z = ex + y), but the same conclusions apply also to linear effects.Figure 7
Comparison of RICcond (X, Z |Y) and IQRcond (X, Z |Y) for various types of mutual influences in the presence of confounders.In addition, in this application, RIC provides much better resilience to noise, and it is less dependent on the choice of the binning to determine the pdfs of the quantities involved. Moreover, even for very low levels of uncertainty, in the limit of no noise, it manages to identify more clearly the mutual correlations actually at play.


## 8. Discussion and Conclusions

To quantify the mutual influence between quantities, a new indicator has been introduced, the reciprocal influence criterion. A conditional version to separate the effects of confounding factors has also been devised. RIC reproduces the results of PCC in the case of linear correlations but is more robust against the influence of additive noise and outliers. In the case of nonlinear influences, RIC outperforms not only the ranked criteria and the distance covariance but also the information theoretic indicators such as the IQR in many respects; it provides more interpretable results and is more robust against noise and less sensitive to the choice of the binning of the pdfs involved. All these competitive advantages can be quite important in practice.Other aspects, not to be neglected in the perspective of a wide application of the proposed indicator, are the fact that RIC is conceptually very simple, easy to implement, and fully general, in the sense that it does not rely on specific assumptions about the properties of the stochastic variables involved. In terms of requirements on the data, of course, enough examples must be available to properly calculate the pdfs, but again RIC is more parsimonious than the other indicators, which require estimating the probability distribution functions of the quantities involved.Regarding future developments, it is planned to investigate whether alternative versions of the entropy and, therefore, of the derived quantities, can help improve the performance of RIC [19–22]. Furthermore, additional formulations, more suited to the investigation of actual causal relations than simple correlations, are also under consideration [23–26]. In terms of practical applications, some of the most immediate range from the investigation of synchronization experiments and disruptions in thermonuclear fusion [27–37] to the refinement of measurement techniques and data analysis methods in earth sciences and plasma physics [38–40].



---

*Source: 9426547-2021-08-12.xml*