# Dynamically Measuring Statistical Dependencies in Multivariate Financial Time Series Using Independent Component Analysis

Authors: Nauman Shah; Stephen J. Roberts
Journal: ISRN Signal Processing
 (2013)

---

## Abstract

We present a computationally tractable approach to dynamically measure statistical dependencies in multivariate non-Gaussian signals. The approach makes use of extensions of independent component analysis to calculate information coupling, as a proxy measure for mutual information, between multiple signals and can be used to estimate uncertainty associated with the information coupling measure in a straightforward way. We empirically validate relative accuracy of the information coupling measure using a set of synthetic data examples and showcase practical utility of using the measure when analysing multivariate financial time series.


---

## Body

## 1. Introduction

The task of accurately inferring the statistical dependency structure (association) in multivariate systems has been an area of active research for many years, with a wide range of practical applications [1]. Many of these applications require real-time sequential analysis of dependencies in multivariate data streams with dynamically changing properties. However, most existing measures of dependence have some serious limitations; in terms of the type of data sets they are suitable for or in their computational complexities. If the data being analysed is generated using a known stable process, with known marginal and multivariate distributions, the degree of dependence can be estimated relatively easily. However, most real-world data sets have dynamically changing properties to which a single distribution cannot be assigned. Multivariate data generated in global financial markets is an example of such complex data sets. Financial data exhibits rapidly changing dynamics and is non-Gaussian in nature; this is especially true for financial data recorded at high frequencies [2]. In fact, as the scale over which financial returns are calculated decreases, their distribution becomes increasingly non-Gaussian, a feature referred to as aggregational Gaussianity. The recent explosive growth in availability and use of financial data sampled at high frequencies therefore requires the use of computationally efficient algorithms which are suitable for dynamically analysing dependencies in non-Gaussian data streams.The most commonly used measure of statistical dependence is linear correlation. However, practical use of the linear correlation measure has three main limitations; that is, it cannot accurately model dependencies between signals with non-Gaussian distributions [3]; it is restricted to measuring linear statistical dependencies and is very sensitive to outliers [4]. Rank correlation is another frequently used measure of association. However, it is only valid for monotonic functions and is not suitable for large data sets, as assigning ranks to a large number of observations is computationally demanding. Financial returns often have a large fraction of zero values, which result in tied ranks [5]. Rank correlation measures cannot accurately deal with the presence of tied ranks and hence the results obtained can be misleading [6]. Another widely used method for multivariate dependence analysis in the financial sector is the use of copulas [7]. However, copula-based methods also suffer from major limitations in practice; for example, computation of copula functions involves calculating multiple moments as well as integration of joint distributions, which require use of numerical methods and hence become computationally complex [8]. Copula-based methods suffer from other major limitations as well, namely, the difficulties in accurate estimation of the copula functions, the empirical choice of the type of copulas, and problems in the design and use of time-dependent copulas [9]. Mutual information, the canonical measure of statistical dependence, is also used in practice. However, accurate computation of mutual information using finite data sets can be computationally complex (as we discuss later). In this paper, we present a computationally efficient independent component analysis (ICA) based approach to dynamically measure information coupling in multivariate non-Gaussian data streams as a proxy measure for mutual information.The paper is organised as follows. We first discuss the need for developing an ICA-based information coupling measure and present the theoretical framework underlying the development of our approach. We then present a brief introduction to the principles of ICA and discuss our method of choice for accurately and efficiently inferring the ICA unmixing matrix in a dynamic environment. We then proceed to present the ICA-based information coupling metric and describe its properties. Finally, we present a set of synthetic and financial data examples which make use of the information coupling measure to estimate dependencies in non-Gaussian signals.


## 2. Measuring Dependencies Using ICA: A Conceptual Overview

Mutual information is the canonical measure of statistical dependence in multivariate systems [10]. It is a quantitative measurement of how much information the observation of one variable gives us regarding another variable. Whilst the computation of mutual information is conceptually straightforward when the full probability density functions (pdf) of the variables under consideration are available, it is often difficult to accurately estimate mutual information directly using finite data sets. This is especially true in high-dimensional spaces, in which computation of mutual information requires the estimation of multivariate joint distributions, a process which is unreliable (being exquisitely sensitive to the joint pdf over the variables of interest) as well as computationally expensive [11]. Existing approaches to compute mutual information include methods that are based on ranking of variables, kernel density estimation, k-nearest neighbours, and the Edgeworth approximation of differential entropy [12, 13]. However, for most finite data sets, none of these techniques, all of which impose a trade-off between computational complexity and accuracy, outperforms the other methods and all these approaches can be extremely sensitive to the presence of noise [12]. The accuracy of these approaches is also highly sensitive to the choice of the model parameters, such as the number of kernels or neighbours. Therefore, in most practical cases, the direct use of mutual information is not feasible. However, as we discuss below, it is possible to make use of information encoded in the ICA unmixing matrix to calculate information coupling as a proxy measure for mutual information.Let us first take a look at the conceptual basis on which we can use ICA as a tool for measuring statistical dependencies. According to its classical definition, ICA estimates an unmixing matrix such that the mutual information between the independent source signals is minimised [14]. Hence, we can consider the unmixing matrix to contain information about the degree of mutual information between the observed signals. Although the direct computation of mutual information can be very expensive, alternative efficient approaches to ICA, which do not involve direct computation of mutual information, exist. Hence, it is possible to indirectly obtain an estimate for mutual information by using the ICA-based information coupling measure as a proxy. Now let us consider some properties of financial returns which make them well-suited to be analysed using ICA (as this paper is focused on financial applications, therefore we consider the case of measuring dependencies in financial data; however, similar ideas can be applied to most real-world systems which give rise to non-Gaussian data). Financial markets are influenced by many independent factors, all of which have some finite effect on any specific financial time series. These factors can include, among others, news releases, price trends, macroeconomic indicators, and order flows. We hypothesise that the observed multivariate financial data may hence be generated as a result of linear combination of some hidden (latent) variables [15, 16]. This process can be quantitatively described by using a linear generative model, such as principal component analysis (PCA), factor analysis (FA), or ICA. As financial returns have non-Gaussian distributions with heavy tails, therefore PCA and FA are not suitable for modelling multivariate financial data, as both these second-order approaches are based on the assumption of Gaussianity [17]. ICA, in contrast, takes into account non-Gaussian nature of the data being analysed by making use of higher-order statistics. ICA has proven applicability for multivariate financial data analysis; some interesting applications are presented in [15, 16, 18]. These, and other similar studies, make use of ICA primarily to extract the underlying latent source signals. However, all relevant information about the source mixing process is contained in the ICA unmixing matrix, which hence encodes dependencies. Therefore, in our analysis we only make use of the ICA unmixing matrix (without extracting the independent components) to measure information coupling. The ICA-based information coupling model we present in this paper can be used to directly measure statistical dependencies in high-dimensional spaces. This makes it particularly attractive for a range of practical applications in which relying solely on pair-wise analysis of dependencies is not feasible (there is surprisingly little work done towards addressing the important issue of estimating the dependency structure in high-dimensional multivariate systems, although there has been interest in this field for a long time [19]. High-dimensional analysis of information coupling has various important applications in the financial sector, including active portfolio management, multivariate financial risk analysis, statistical arbitrage, and pricing and hedging of various instruments [9]).


## 3. Independent Components, Unmixing and Non-Gaussianity

Mixing two or more unique signals, to a set of mixed observations, results in an increase in the dependency of the pdfs of the mixed signals. The marginal pdfs of the observed mixed signals become more Gaussian due to the central limit theorem [20]. The mixing process also results in a reduction in the independence of the mixed signal distribution and hence increase in mutual information associated with it. Moreover, there is a rise in the stationarity of the mixed signals, which have flatter spectra as compared to the original sources [21]. Consider a set of N observed signals x(t)=[x1(t),x2(t),…,xN(t)]⊤ at the time instant t, which are a mixture of M source signals s(t)=[s1(t),s2(t),…,sM(t)]⊤, mixed linearly using a mixing matrix A, with observation noise n(t), as per
(1)x(t)=As(t)+n(t).
Independent component analysis (ICA) attempts to find an unmixing matrix W, such that the M recovered source signals b(t)=[b1(t),b2(t),…,bM(t)]⊤ are given by
(2)b(t)=W(x(t)-n(t)).
For the case where observation noise n(t) is assumed to be normally distributed with a mean of zero, the least squares expected value of the recovered source signals is given by:
(3)b^(t)=Wx(t),
where W is the pseudo-inverse of A; that is,
(4)W=A+=(A⊤A)-1A⊤.
In the case of square mixing, W=A-1.Most ICA approaches make implicit or explicit assumptions regarding the parametric model of the pdfs of the independent sources [21]; for example, Gaussian mixture distributions are used as source models in [22, 23], while [24] makes use of a flexible source density model given by the generalised exponential distribution. In our analysis, we use a reciprocal cosh source model as a canonical heavy-tailed distribution; namely [21],
(5)p(si)=1Z0cosh(si),
where si is the ith source and Z0 is a normalising constant. This analytical fixed source model has no adjustable parameters; therefore, it has considerable computational advantages over alternative source models. Also, as this source model is heavier in the tails, it is able to accurately model heavy-tailed unimodal non-Gaussian distributions, such as financial returns.


### 3.1. Inference

For our analysis, we make use of theicadec algorithm [21, 24] to infer the unmixing matrix. This algorithm constrains the unmixing matrix to the manifold of decorrelating matrices, thus offering rapid computation. The algorithm also gives accurate results compared to other related ICA approaches [24] and allows us to obtain a confidence measure for the unmixing matrix. Here we present a brief overview of this algorithm; an in-depth description is presented in [24].The independent source signals obtained using ICA,b(t), must be at least linearly decorrelated for them to be classed as independent. The icadec algorithm makes use of this property of the independent components to efficiently infer the unmixing matrix. For a set of observed signals, X, where X=[x(t)]t=1t=T; the set of recovered independent components, B=[b(t)]t=1t=T, is given by B=WX. The independent components are linearly decorrelated if
(6)BB⊤=WXX⊤W⊤=D2,
where D is a diagonal matrix of scaling factors. The singular value decomposition of the set of observed signals is given by
(7)X=UΣV⊤,
where U and V are orthogonal matrices with the columns of U being the principal components of X, and Σ is a diagonal matrix of the singular values of X. It can be shown that the decorrelating matrix, W, can then be written as [24]
(8)W=DQΣ-1U⊤,
where Q is a real orthogonal matrix. To obtain an estimate for the ICA unmixing matrix, we need to optimise a given contrast function (we use log-likelihood of the data, as described later). There are a variety of optimisation approaches which can be used; our approach of choice is the Broyden-Fletcher-Golfarb-Shanno (BFGS) quasi-Newton method, which gives the best estimate of the minimum of the negative log-likelihood in a computationally efficient manner and also provides us with an estimate for the Hessian matrix. However, parameterising the optimisation problem directly by the elements of Q makes it a constrained minimisation problem for which BFGS is not applicable. Therefore, to convert it into an unconstrained minimisation problem, we constrain Q to be orthonormal by parameterising its elements as the matrix exponential of a skew-symmetric matrix J (nonzero elements of this matrix are known as the Cayley coordinates), whose above diagonal elements parameterise Q [21] (for M sources and N observed signals, the ICA unmixing matrix may be optimised in the (1/2)M(M+1) dimensional space of decorrelating matrices rather than in the full MN dimensional space, as (1/2)M(M-1) and M parameters are required to specify Q and D respectively. This feature offers considerable computational benefits (especially in high-dimensional spaces) and the resulting matrix hence obtained is guaranteed to be decorrelating [24]):(9)Q=exp(J).
Using this parameterisation makes it possible to apply BFGS to any contrast function; the contrast function used as part of the icadec algorithm is an expression for the log-likelihood of the data, as described below.Using (1) and assuming that the observation noise (n) is normally distributed with a mean of zero and having an isotropic covariance matrix with precision β, the distribution of the observations (as a preprocessing step, we normalise each observed signal to have a mean of zero and unit variance) conditioned on A and s (where we drop the time index t for ease of presentation) is given by
(10)p(x∣A,s)=𝒩(x;As,β-1I),
where As is the mean of the normal distribution and β-1I is its covariance. The likelihood of an observation occurring is given by
(11)p(x∣A)=∫‍p(x∣A,s)p(s)ds.
Assuming that the distribution over sources has a single dominant peak, in this case given by the maximum likelihood source estimates s^=(A⊤A)-1A⊤x, the integral in (11) can be analysed by using a simplified (computationally efficient) variant of Laplace’s method, as shown in [21, 25]:
(12)p(x∣A)=∫‍p(x∣A,s)p(s)ds≈p(x∣A,s^)p(s^)(2π)M/2det(G)-1/2,
where G is the Hessian matrix:
(13)G=-[∂2logp(x∣A,s)∂si∂sj]s=s^.
Taking log of the expanded form of (10) gives
(14)logp(x∣A,s)=N2log(β2π)-β2(x-As)⊤(x-As)
which, via (13), results in G=βA⊤A. The log-likelihood, ℓ≡logp(x∣A), is therefore
(15)ℓ=N-M2log(β2π)-β2(x-As^)⊤(x-As^)+logp(s^)-12logdet(ATA).
By using (8), we obtain A=UΣQ⊤D-1. Hence, the log-likelihood becomes [21]
(16)ℓ=N-M2log(β2πe)+logp(s^)+logdet(Σ-1D),
where e is the average reconstruction error. Noting that we use a reciprocal cosh source model (as given by (5)), it can be shown that taking the derivative of this log-likelihood expression with respect to D and J (which parameterises Q), and by following the resulting likelihood gradient using a BFGS optimiser, makes it possible to efficiently compute an optimum value for the ICA unmixing matrix; details of this procedure are presented in [24].


### 3.2. Dynamic Mixing

The standard (offline) ICA model uses all available data samples at timest=1,2,…,T of the observed signals, x(t), to estimate a single static unmixing matrix, W. The unmixing matrix obtained provides a good estimate of the mixing process for the complete time series and is well suited for offline data analysis. However, many time series, such as financial data streams, are highly dynamic in nature with rapidly changing properties and therefore require a source separation method that can be used in a sequential manner. This issue is addressed here by using a sliding-window ICA model [26]. This model makes use of a sliding-window approach to sequentially update the current unmixing matrix using information contained in the previous window and can easily handle nonstationary data. The unmixing matrix for the current window, W(t), is used as a prior for computing the unmixing matrix for the next window, W(t+1). This results in significant computational efficiency as fewer iterations are required to obtain an optimum value for W(t+1). The algorithm also results in an improvement in the source separation results obtained when the mixing process is drifting and addresses the ICA permutation and sign ambiguity issues [21], by maintaining a fixed (but of course arbitrary) ordering of recovered sources through time.


## 3.1. Inference

For our analysis, we make use of theicadec algorithm [21, 24] to infer the unmixing matrix. This algorithm constrains the unmixing matrix to the manifold of decorrelating matrices, thus offering rapid computation. The algorithm also gives accurate results compared to other related ICA approaches [24] and allows us to obtain a confidence measure for the unmixing matrix. Here we present a brief overview of this algorithm; an in-depth description is presented in [24].The independent source signals obtained using ICA,b(t), must be at least linearly decorrelated for them to be classed as independent. The icadec algorithm makes use of this property of the independent components to efficiently infer the unmixing matrix. For a set of observed signals, X, where X=[x(t)]t=1t=T; the set of recovered independent components, B=[b(t)]t=1t=T, is given by B=WX. The independent components are linearly decorrelated if
(6)BB⊤=WXX⊤W⊤=D2,
where D is a diagonal matrix of scaling factors. The singular value decomposition of the set of observed signals is given by
(7)X=UΣV⊤,
where U and V are orthogonal matrices with the columns of U being the principal components of X, and Σ is a diagonal matrix of the singular values of X. It can be shown that the decorrelating matrix, W, can then be written as [24]
(8)W=DQΣ-1U⊤,
where Q is a real orthogonal matrix. To obtain an estimate for the ICA unmixing matrix, we need to optimise a given contrast function (we use log-likelihood of the data, as described later). There are a variety of optimisation approaches which can be used; our approach of choice is the Broyden-Fletcher-Golfarb-Shanno (BFGS) quasi-Newton method, which gives the best estimate of the minimum of the negative log-likelihood in a computationally efficient manner and also provides us with an estimate for the Hessian matrix. However, parameterising the optimisation problem directly by the elements of Q makes it a constrained minimisation problem for which BFGS is not applicable. Therefore, to convert it into an unconstrained minimisation problem, we constrain Q to be orthonormal by parameterising its elements as the matrix exponential of a skew-symmetric matrix J (nonzero elements of this matrix are known as the Cayley coordinates), whose above diagonal elements parameterise Q [21] (for M sources and N observed signals, the ICA unmixing matrix may be optimised in the (1/2)M(M+1) dimensional space of decorrelating matrices rather than in the full MN dimensional space, as (1/2)M(M-1) and M parameters are required to specify Q and D respectively. This feature offers considerable computational benefits (especially in high-dimensional spaces) and the resulting matrix hence obtained is guaranteed to be decorrelating [24]):(9)Q=exp(J).
Using this parameterisation makes it possible to apply BFGS to any contrast function; the contrast function used as part of the icadec algorithm is an expression for the log-likelihood of the data, as described below.Using (1) and assuming that the observation noise (n) is normally distributed with a mean of zero and having an isotropic covariance matrix with precision β, the distribution of the observations (as a preprocessing step, we normalise each observed signal to have a mean of zero and unit variance) conditioned on A and s (where we drop the time index t for ease of presentation) is given by
(10)p(x∣A,s)=𝒩(x;As,β-1I),
where As is the mean of the normal distribution and β-1I is its covariance. The likelihood of an observation occurring is given by
(11)p(x∣A)=∫‍p(x∣A,s)p(s)ds.
Assuming that the distribution over sources has a single dominant peak, in this case given by the maximum likelihood source estimates s^=(A⊤A)-1A⊤x, the integral in (11) can be analysed by using a simplified (computationally efficient) variant of Laplace’s method, as shown in [21, 25]:
(12)p(x∣A)=∫‍p(x∣A,s)p(s)ds≈p(x∣A,s^)p(s^)(2π)M/2det(G)-1/2,
where G is the Hessian matrix:
(13)G=-[∂2logp(x∣A,s)∂si∂sj]s=s^.
Taking log of the expanded form of (10) gives
(14)logp(x∣A,s)=N2log(β2π)-β2(x-As)⊤(x-As)
which, via (13), results in G=βA⊤A. The log-likelihood, ℓ≡logp(x∣A), is therefore
(15)ℓ=N-M2log(β2π)-β2(x-As^)⊤(x-As^)+logp(s^)-12logdet(ATA).
By using (8), we obtain A=UΣQ⊤D-1. Hence, the log-likelihood becomes [21]
(16)ℓ=N-M2log(β2πe)+logp(s^)+logdet(Σ-1D),
where e is the average reconstruction error. Noting that we use a reciprocal cosh source model (as given by (5)), it can be shown that taking the derivative of this log-likelihood expression with respect to D and J (which parameterises Q), and by following the resulting likelihood gradient using a BFGS optimiser, makes it possible to efficiently compute an optimum value for the ICA unmixing matrix; details of this procedure are presented in [24].


## 3.2. Dynamic Mixing

The standard (offline) ICA model uses all available data samples at timest=1,2,…,T of the observed signals, x(t), to estimate a single static unmixing matrix, W. The unmixing matrix obtained provides a good estimate of the mixing process for the complete time series and is well suited for offline data analysis. However, many time series, such as financial data streams, are highly dynamic in nature with rapidly changing properties and therefore require a source separation method that can be used in a sequential manner. This issue is addressed here by using a sliding-window ICA model [26]. This model makes use of a sliding-window approach to sequentially update the current unmixing matrix using information contained in the previous window and can easily handle nonstationary data. The unmixing matrix for the current window, W(t), is used as a prior for computing the unmixing matrix for the next window, W(t+1). This results in significant computational efficiency as fewer iterations are required to obtain an optimum value for W(t+1). The algorithm also results in an improvement in the source separation results obtained when the mixing process is drifting and addresses the ICA permutation and sign ambiguity issues [21], by maintaining a fixed (but of course arbitrary) ordering of recovered sources through time.


## 4. Information Coupling

We now proceed to derive the ICA-based information coupling metric. Later in this section we discuss the practical advantages this metric offers when used to analyse multivariate financial time series.


### 4.1. Coupling Metric

LetW be any arbitrary square ICA unmixing matrix (for the purpose of brevity and clarity, we only consider the case of square mixing while deriving the metric here. However, the metric derived in this section is valid for nonsquare mixing as well, and the corresponding derivation can be undertaken using a similar approach as presented here but converting the nonsquare ICA unmixing matrices in each instance into square matrices by padding them with zeros):(17)W∈ℝN×N.
Since multiplication of W by a diagonal matrix does not affect the mutual information of the recovered sources; therefore, we row normalise the unmixing matrix in order to address the ICA scale indeterminacy problem (most ICA algorithms, including icadec, suffer from the scale indeterminacy problem; that is, the variances of the independent components cannot be determined; this is because both the unmixing matrix and the source signals are unknown and any scalar multiplication on either will be lost in the mixing process). Row normalisation implies that the elements wij of the unmixing matrix W are constrained, such that each row of the matrix is of unit length; that is,
(18)∑j=1N‍wij2=1
for all rows i. For a set of observed signals to be completely decoupled, their latent independent components must be the same as the observed signals; therefore, the row-normalised unmixing matrix for decoupled signals (W0) must be a permutation of the identity matrix (I):
(19)W0=PI∈ℝN×N,
where P is a permutation matrix. For the case where the observed signals are completely coupled, all the latent independent components must be the same; therefore, the row-normalised unmixing matrix for completely coupled signals (W1) is given by
(20)W1=1NK∈ℝN×N,
where K is the unit matrix (a matrix of ones).To calculate coupling, we need to consider thedistance between any arbitrary unmixing matrix (W) and the zero coupling matrix (W0). The distance measure we use is the generalised 2-norm, also called the spectral norm, of the difference between the two matrices [27], although we can use some other norms as well to get similar results. The spectral norm of a matrix corresponds to its largest singular value and is the matrix equivalent of the vector Euclidean norm. Hence, the distance, d(W,W0), between the two matrices can be written as
(21)d(W,W0)=∥W-W0∥2,
where ∥·∥2 is the spectral norm of the matrix. As W0 is a permutation of the identity matrix, therefore,
(22)d(W,W0)=∥W-PI∥2.
As the spectral norm of a matrix is independent of its permutations, therefore, we may define another permutation matrix (P^) such that
(23)d(W,W0)=∥P^W-I∥2.
For this equation, the following equality holds:
(24)d(W,W0)=∥P^W∥2-1.
Again, noting that ∥P^W∥2=∥W∥2, we have
(25)d(W,W0)=∥W∥2-1.
We normalise this measure with respect to the range over which the distance measure can vary, that is, the distance between matrices representing completely coupled (W1) and decoupled (W0) signals. From (20) we have that W1=(1/N)K; therefore,
(26)d(W1,W0)=∥W1-W0∥2=∥1NK-PI∥2.
Using the same analysis as presented previously, this equation can be simplified to
(27)d(W1,W0)=1N∥K∥2-1.
For a N-dimensional square unit matrix, the spectral norm is given by ∥K∥2=N. Therefore, for a row-normalised unit matrix, the spectral norm is (1/N)∥K∥2=N. Hence, if W is row-normalised, (27) can be written as
(28)d(W1,W0)=N-1.
The normalised information coupling metric (η) is then defined as
(29)η=d(W,W0)d(W1,W0).
Substituting (25) and (28) into (29), the normalised information coupling between N observed signals is given by
(30)η=∥W∥2-1N-1.We can consider the bounds ofη as described below. Suppose M is an arbitrary real matrix. We can look upon the spectral norm of this matrix (∥M∥2=∥M-0∥2) as a measure of departure (distance) of M from a null matrix (0) [28]. The bounds on this norm are given by
(31)0≤∥M∥2≤∞.
If M is a row-normalised ICA unmixing matrix (W), then (as discussed earlier) it lies between W0 and W1. Hence, the bounds on W are
(32)∥W0∥2≤∥W∥2≤∥W1∥2.
Using (19) and (20), we can write
(33)∥PI∥2≤∥W∥2≤∥1NK∥2
which can be simplified to
(34)1≤∥W∥2≤N.
Rearranging terms in this inequality gives
(35)0≤∥W∥2-1N-1≤1
which gives us the same coupling metric as in (30) and shows that the metric is normalised; that is, 0≤η≤1. For real-valued W, ∥W∥2 can be written as
(36)∥W∥2=λmax(W⊤W),
where λmax(W⊤W) is the maximum eigenvalue of W⊤W. The unmixing matrix obtained using most ICA algorithms, including icadec, suffers from row permutation and sign ambiguity problems; that is, the rows are arranged in a random order and the sign of elements in each row is unknown [21, 24]. We note that as ∥W∥2 is independent of the sign and permutations of the rows of W, therefore our measure of information coupling straightaway addresses the problems of ICA sign and permutation ambiguities. Also, as the metric’s value is independent of the row permutations of W, therefore it provides symmetric results. The information coupling metric is valid for all dimensions of the unmixing matrix, W. This implies that information coupling can be easily measured in high-dimensional spaces.It is possible to obtain a measure of uncertainty in our estimation of the information coupling measure. We make use of the BFGS quasi-Newton optimisation approach over the most probable skew-symmetric matrix,J, of (9), from which estimates for the unmixing matrix, W, can be obtained, and thence the coupling measure η calculated. We also estimate the Hessian (inverse covariance) matrix, H, for J, as part of this process. Hence, it is possible to draw samples, J′, say from the distribution over J, as a multivariate normal:
(37)J′~ℳ𝒩(J,H-1).
These samples can be readily transformed to samples in η using (9), (8), and (30), respectively. Confidence bounds (and here we use the 95% bounds) may then be easily obtained from the set of samples for η (in our analysis we use 100 samples).


### 4.2. Computational Complexity

The information coupling algorithm achieves computational efficiency by making use of the sliding-window based decorrelating manifold approach to ICA. Making use of the reciprocal cosh based ICA source model also results in significant computational advantages. We now take a look at the comparative computational complexity of the information coupling measure and three frequently used measures of statistical dependence, that is, linear correlation, rank correlation, and mutual information. For bivariate data (ns samples long), for which these four measures are directly comparable, linear correlation and rank correlation have time complexities of order O(ns) and O(nslogns), respectively [29], while mutual information and information coupling scale as O(ns2) and O(ns), respectively (there have been various estimation algorithms proposed for efficient computation of mutual information; however, they all result in increased estimation errors and require careful selection of various user-defined parameters [30]) [31]. Hence, even though the time complexity of the information coupling measure is of the same order as linear correlation, it can still accurately capture statistical dependencies in non-Gaussian data streams and is a computationally efficient proxy for mutual information.ForN-dimensional multivariate data, direct computation of mutual information has time complexity of order O(nsN) compared to O(nsN3) for the information coupling measure. In high-dimensions, even an approximation for mutual information can be computationally very costly. For example, using a Parzen-window density estimator, the mutual information computational complexity can be reduced to O(nsnbN), where nb is the number of bins used for estimation [32], which will incur a very high computational cost even for relatively small values of N, nb, and ns. As a simple example, Table 1 shows a comparison of computation time (in seconds) taken by mutual information and information coupling measures for analysing bivariate data sets of varying lengths. As expected, mutual information estimation using the Parzen window based approach (which is considered to be a relatively efficient approach to compute mutual information) becomes computationally very demanding with an increase in the number of samples of the bivariate data set. In contrast, the information coupling measure is computationally efficient, even when used to analyse very large high-dimensional multivariate data sets.Table 1
Example showing comparison of average computation time (in seconds) of mutual information and information coupling, when these measures are used to analyse bivariate data sets containing different number of samples (ns). The approach used to estimate mutual information is based on a Parzen window based algorithm, as described in [45]. The computational cost of this algorithm is dependent on the window-size (h). The values of h used for the simulations are h=20 for ns=102, and h=100 for all other simulations. Results are obtained using a 2.66 GHz processor as an average of 100 simulations.
Computation time (sec)
n
s
=
1
0
2
n
s
=
1
0
3
n
s
=
1
0
4
n
s
=
1
0
5
Mutual information
0.0214
2.8289
17.0101
119.5088
Information coupling
0.0073
0.0213
0.0561
0.5543


### 4.3. Capturing Market Dynamics

To dynamically analyse information coupling in multivariate financial data streams, we need to make use of windowing techniques. Financial markets give rise to well-defined events, such as orders, trades and quote revisions. These events are irregularly spaced inclock-time; that is, they are asynchronous. Statistical models in clock-time make use of data aggregated over fixed intervals of time [33]. The time at which these events are indexed is called the event-time. Hence, for dynamic modelling, in event-time the number of data points can be regarded as fixed while time varies, while in clock-time the time period is considered to be fixed with variable number of data points. Although we may need adaptive windows in clock-time, we can use sliding-windows of fixed length in event-time. Using fixed length sliding-windows in event-time can be useful for obtaining consistent results when developing and testing different statistical models. Also, statistical models deployed for online analysis of financial data operate best in event-time as they often need to make decisions as soon as some new market information (such as quote update etc.) becomes available. Consider an online trading model making use of an adaptive window in event-time. At specific times of the day, for example, at times of major news announcements, trading volume can significantly increase. Hence, more data will be available to the algorithm and thus results obtained can be misleading [34]. Using a sliding-window of fixed length in event-time can overcome this problem. The length of the sliding-window needs to be selected appropriately. The financial application for which the model is being used is one of the factors which drives the choice of window length. As a general rule, for trading models a window of approximately the same size as the average time period between placing trades (inverse of trading frequency) is often used. This makes it possible to accurately capture the rapidly evolving dynamics of the markets over the corresponding period, without being too long so as to only capture major trends or too short to capture noise in the data.


### 4.4. Discussion

The information coupling model offers us with multiple advantages when used to analyse multivariate financial data. Here we summarise some of the main properties the model, while the empirical results presented in the next section showcase some of its practical benefits.(i)
The information coupling measure, a proxy for mutual information, is able to accurately pick up statistical dependencies in data sets with non-Gaussian distributions (such as financial returns).(ii)
The information coupling algorithm is computationally efficient, which makes it particularly suitable for use in an online dynamic environment. This makes the algorithm especially attractive when dealing with data sampled at high frequencies. This is because with the ever-increasing use of high-frequency data, overcoming sources of latency is of utmost importance in a variety of applications in modern financial markets.(iii)
It gives confidence levels on the information coupling measure. This allows us to estimate the uncertainty associated with the measurements.(iv)
The metric provides normalised results; that is, information coupling ranges from0 for decoupled systems to 1 for completely coupled systems. This makes it easier to analyse results obtained using the metric and to compare its performance with other similar measures of association. The metric also gives symmetric results.(v)
The metric is valid for any number of signals in high-dimensional spaces; that is, it consistently gives accurate results irrespective of the number of time series between which information coupling is being computed. This makes it suitable for a range of financial applications.(vi)
It is not data intensive; that is, it gives relatively accurate results even when a small sample size is used. This allows the metric to model the complex and rapidly changing dynamics of financial markets.(vii)
It does not depend on user-defined parameters which can restrict its practical utility, as the evolving market conditions may require the parameters to be constantly updated, which may not be practical.


## 4.1. Coupling Metric

LetW be any arbitrary square ICA unmixing matrix (for the purpose of brevity and clarity, we only consider the case of square mixing while deriving the metric here. However, the metric derived in this section is valid for nonsquare mixing as well, and the corresponding derivation can be undertaken using a similar approach as presented here but converting the nonsquare ICA unmixing matrices in each instance into square matrices by padding them with zeros):(17)W∈ℝN×N.
Since multiplication of W by a diagonal matrix does not affect the mutual information of the recovered sources; therefore, we row normalise the unmixing matrix in order to address the ICA scale indeterminacy problem (most ICA algorithms, including icadec, suffer from the scale indeterminacy problem; that is, the variances of the independent components cannot be determined; this is because both the unmixing matrix and the source signals are unknown and any scalar multiplication on either will be lost in the mixing process). Row normalisation implies that the elements wij of the unmixing matrix W are constrained, such that each row of the matrix is of unit length; that is,
(18)∑j=1N‍wij2=1
for all rows i. For a set of observed signals to be completely decoupled, their latent independent components must be the same as the observed signals; therefore, the row-normalised unmixing matrix for decoupled signals (W0) must be a permutation of the identity matrix (I):
(19)W0=PI∈ℝN×N,
where P is a permutation matrix. For the case where the observed signals are completely coupled, all the latent independent components must be the same; therefore, the row-normalised unmixing matrix for completely coupled signals (W1) is given by
(20)W1=1NK∈ℝN×N,
where K is the unit matrix (a matrix of ones).To calculate coupling, we need to consider thedistance between any arbitrary unmixing matrix (W) and the zero coupling matrix (W0). The distance measure we use is the generalised 2-norm, also called the spectral norm, of the difference between the two matrices [27], although we can use some other norms as well to get similar results. The spectral norm of a matrix corresponds to its largest singular value and is the matrix equivalent of the vector Euclidean norm. Hence, the distance, d(W,W0), between the two matrices can be written as
(21)d(W,W0)=∥W-W0∥2,
where ∥·∥2 is the spectral norm of the matrix. As W0 is a permutation of the identity matrix, therefore,
(22)d(W,W0)=∥W-PI∥2.
As the spectral norm of a matrix is independent of its permutations, therefore, we may define another permutation matrix (P^) such that
(23)d(W,W0)=∥P^W-I∥2.
For this equation, the following equality holds:
(24)d(W,W0)=∥P^W∥2-1.
Again, noting that ∥P^W∥2=∥W∥2, we have
(25)d(W,W0)=∥W∥2-1.
We normalise this measure with respect to the range over which the distance measure can vary, that is, the distance between matrices representing completely coupled (W1) and decoupled (W0) signals. From (20) we have that W1=(1/N)K; therefore,
(26)d(W1,W0)=∥W1-W0∥2=∥1NK-PI∥2.
Using the same analysis as presented previously, this equation can be simplified to
(27)d(W1,W0)=1N∥K∥2-1.
For a N-dimensional square unit matrix, the spectral norm is given by ∥K∥2=N. Therefore, for a row-normalised unit matrix, the spectral norm is (1/N)∥K∥2=N. Hence, if W is row-normalised, (27) can be written as
(28)d(W1,W0)=N-1.
The normalised information coupling metric (η) is then defined as
(29)η=d(W,W0)d(W1,W0).
Substituting (25) and (28) into (29), the normalised information coupling between N observed signals is given by
(30)η=∥W∥2-1N-1.We can consider the bounds ofη as described below. Suppose M is an arbitrary real matrix. We can look upon the spectral norm of this matrix (∥M∥2=∥M-0∥2) as a measure of departure (distance) of M from a null matrix (0) [28]. The bounds on this norm are given by
(31)0≤∥M∥2≤∞.
If M is a row-normalised ICA unmixing matrix (W), then (as discussed earlier) it lies between W0 and W1. Hence, the bounds on W are
(32)∥W0∥2≤∥W∥2≤∥W1∥2.
Using (19) and (20), we can write
(33)∥PI∥2≤∥W∥2≤∥1NK∥2
which can be simplified to
(34)1≤∥W∥2≤N.
Rearranging terms in this inequality gives
(35)0≤∥W∥2-1N-1≤1
which gives us the same coupling metric as in (30) and shows that the metric is normalised; that is, 0≤η≤1. For real-valued W, ∥W∥2 can be written as
(36)∥W∥2=λmax(W⊤W),
where λmax(W⊤W) is the maximum eigenvalue of W⊤W. The unmixing matrix obtained using most ICA algorithms, including icadec, suffers from row permutation and sign ambiguity problems; that is, the rows are arranged in a random order and the sign of elements in each row is unknown [21, 24]. We note that as ∥W∥2 is independent of the sign and permutations of the rows of W, therefore our measure of information coupling straightaway addresses the problems of ICA sign and permutation ambiguities. Also, as the metric’s value is independent of the row permutations of W, therefore it provides symmetric results. The information coupling metric is valid for all dimensions of the unmixing matrix, W. This implies that information coupling can be easily measured in high-dimensional spaces.It is possible to obtain a measure of uncertainty in our estimation of the information coupling measure. We make use of the BFGS quasi-Newton optimisation approach over the most probable skew-symmetric matrix,J, of (9), from which estimates for the unmixing matrix, W, can be obtained, and thence the coupling measure η calculated. We also estimate the Hessian (inverse covariance) matrix, H, for J, as part of this process. Hence, it is possible to draw samples, J′, say from the distribution over J, as a multivariate normal:
(37)J′~ℳ𝒩(J,H-1).
These samples can be readily transformed to samples in η using (9), (8), and (30), respectively. Confidence bounds (and here we use the 95% bounds) may then be easily obtained from the set of samples for η (in our analysis we use 100 samples).


## 4.2. Computational Complexity

The information coupling algorithm achieves computational efficiency by making use of the sliding-window based decorrelating manifold approach to ICA. Making use of the reciprocal cosh based ICA source model also results in significant computational advantages. We now take a look at the comparative computational complexity of the information coupling measure and three frequently used measures of statistical dependence, that is, linear correlation, rank correlation, and mutual information. For bivariate data (ns samples long), for which these four measures are directly comparable, linear correlation and rank correlation have time complexities of order O(ns) and O(nslogns), respectively [29], while mutual information and information coupling scale as O(ns2) and O(ns), respectively (there have been various estimation algorithms proposed for efficient computation of mutual information; however, they all result in increased estimation errors and require careful selection of various user-defined parameters [30]) [31]. Hence, even though the time complexity of the information coupling measure is of the same order as linear correlation, it can still accurately capture statistical dependencies in non-Gaussian data streams and is a computationally efficient proxy for mutual information.ForN-dimensional multivariate data, direct computation of mutual information has time complexity of order O(nsN) compared to O(nsN3) for the information coupling measure. In high-dimensions, even an approximation for mutual information can be computationally very costly. For example, using a Parzen-window density estimator, the mutual information computational complexity can be reduced to O(nsnbN), where nb is the number of bins used for estimation [32], which will incur a very high computational cost even for relatively small values of N, nb, and ns. As a simple example, Table 1 shows a comparison of computation time (in seconds) taken by mutual information and information coupling measures for analysing bivariate data sets of varying lengths. As expected, mutual information estimation using the Parzen window based approach (which is considered to be a relatively efficient approach to compute mutual information) becomes computationally very demanding with an increase in the number of samples of the bivariate data set. In contrast, the information coupling measure is computationally efficient, even when used to analyse very large high-dimensional multivariate data sets.Table 1
Example showing comparison of average computation time (in seconds) of mutual information and information coupling, when these measures are used to analyse bivariate data sets containing different number of samples (ns). The approach used to estimate mutual information is based on a Parzen window based algorithm, as described in [45]. The computational cost of this algorithm is dependent on the window-size (h). The values of h used for the simulations are h=20 for ns=102, and h=100 for all other simulations. Results are obtained using a 2.66 GHz processor as an average of 100 simulations.
Computation time (sec)
n
s
=
1
0
2
n
s
=
1
0
3
n
s
=
1
0
4
n
s
=
1
0
5
Mutual information
0.0214
2.8289
17.0101
119.5088
Information coupling
0.0073
0.0213
0.0561
0.5543


## 4.3. Capturing Market Dynamics

To dynamically analyse information coupling in multivariate financial data streams, we need to make use of windowing techniques. Financial markets give rise to well-defined events, such as orders, trades and quote revisions. These events are irregularly spaced inclock-time; that is, they are asynchronous. Statistical models in clock-time make use of data aggregated over fixed intervals of time [33]. The time at which these events are indexed is called the event-time. Hence, for dynamic modelling, in event-time the number of data points can be regarded as fixed while time varies, while in clock-time the time period is considered to be fixed with variable number of data points. Although we may need adaptive windows in clock-time, we can use sliding-windows of fixed length in event-time. Using fixed length sliding-windows in event-time can be useful for obtaining consistent results when developing and testing different statistical models. Also, statistical models deployed for online analysis of financial data operate best in event-time as they often need to make decisions as soon as some new market information (such as quote update etc.) becomes available. Consider an online trading model making use of an adaptive window in event-time. At specific times of the day, for example, at times of major news announcements, trading volume can significantly increase. Hence, more data will be available to the algorithm and thus results obtained can be misleading [34]. Using a sliding-window of fixed length in event-time can overcome this problem. The length of the sliding-window needs to be selected appropriately. The financial application for which the model is being used is one of the factors which drives the choice of window length. As a general rule, for trading models a window of approximately the same size as the average time period between placing trades (inverse of trading frequency) is often used. This makes it possible to accurately capture the rapidly evolving dynamics of the markets over the corresponding period, without being too long so as to only capture major trends or too short to capture noise in the data.


## 4.4. Discussion

The information coupling model offers us with multiple advantages when used to analyse multivariate financial data. Here we summarise some of the main properties the model, while the empirical results presented in the next section showcase some of its practical benefits.(i)
The information coupling measure, a proxy for mutual information, is able to accurately pick up statistical dependencies in data sets with non-Gaussian distributions (such as financial returns).(ii)
The information coupling algorithm is computationally efficient, which makes it particularly suitable for use in an online dynamic environment. This makes the algorithm especially attractive when dealing with data sampled at high frequencies. This is because with the ever-increasing use of high-frequency data, overcoming sources of latency is of utmost importance in a variety of applications in modern financial markets.(iii)
It gives confidence levels on the information coupling measure. This allows us to estimate the uncertainty associated with the measurements.(iv)
The metric provides normalised results; that is, information coupling ranges from0 for decoupled systems to 1 for completely coupled systems. This makes it easier to analyse results obtained using the metric and to compare its performance with other similar measures of association. The metric also gives symmetric results.(v)
The metric is valid for any number of signals in high-dimensional spaces; that is, it consistently gives accurate results irrespective of the number of time series between which information coupling is being computed. This makes it suitable for a range of financial applications.(vi)
It is not data intensive; that is, it gives relatively accurate results even when a small sample size is used. This allows the metric to model the complex and rapidly changing dynamics of financial markets.(vii)
It does not depend on user-defined parameters which can restrict its practical utility, as the evolving market conditions may require the parameters to be constantly updated, which may not be practical.


## 5. Results

We now present a set of synthetic and financial data examples showing the relative accuracy and practical utility of the information coupling measure. The following notations are used for different measures of statistical dependence in this paper: ICA-based information coupling (η), linear correlation (ρ), rank correlation (ρR), and normalised mutual information (IN). Normalisation of mutual information values (I) is achieved using the following transformation [35]:
(38)IN=1-exp(-2I).The financial data examples we present in this paper primarily make use of spot foreign exchange (FX) data. Spot price or spot rate is the price which is actually quoted for a currency transaction to take place. Return is the profit realised when a currency is traded. It is common practice to use the normalised log-returns of financial data in statistical analysis, instead of the raw data itself [36]. Using log-returns makes it possible to convert exponential problems into linear ones, thus significantly simplifying relevant analysis. A normalised log-returns data set, with a mean of zero and unit variance, can generally be regarded as a locally stationary process [37]. Therefore, many signal processing techniques meant solely for stationary processes can be successfully applied to the normalised log-returns time series in an adaptive environment. Denoting the mid-price, at time t, of a financial time series by Pt, the log-return value is given by
(39)rt=ln(PtPt-1).
The normalisation of log-returns is achieved by converting the data to a form such that it has a mean of zero and unit variance. This is easily achieved by removing the mean and dividing by the standard deviation of the data.


### 5.1. Synthetic Data

There is no single distribution which fits financial returns, especially those sampled at higher frequencies which tend to be highly non-Gaussian [2], although there have been attempts to model returns using a variety of distributions [38]. In this paper, we aim to capture the heavy-tailed, skewed, properties of financial returns using a Pearson type IV distribution [39, 40], which can be used to represent distributions with varying degrees of skewness and kurtosis and thus are useful for representing distributions of financial returns [41, 42]. The first four moments of the distribution can be uniquely determined by setting four parameters which characterise the distribution [43]. Until recently, due to its mathematical and computational complexity, this distribution has not been widely used in financial literature, although this is rapidly changing with advances in computational power and proposal of new, improved analytical methods [39, 42, 44].To test accuracy of various measures of dependence, we need to generate coupled non-Gaussian synthetic data with known, predefined, correlation values. There is no straightforward way to simulate correlated random variables when their joint distribution is not known [46], as is the case with multivariate financial returns. One possible method that can be used to induce any desired predefined correlation between independent, randomly distributed variables, irrespective of their distributions, is commonly known as the Iman-Conover method, as presented in [47]. This method is based on inducing a known dependency structure in samples taken from the input independent marginal distributions using reordering techniques. The multivariate coupled structure obtained as the output can thus be used as the input data in various models of dependency analysis to test their relative accuracies.We set parameters of the Pearson type IV distribution such that the coupled synthetic data we obtain using the Iman-Conover method has similar properties to financial returns. But first we need to consider some properties of financial returns which we want to mimic. Figures1(a) and 1(b) show the distributions of two higher-order moments, that is, skewness (γ) and kurtosis (κ), for FX spot data sets sampled at three different frequencies. The plots are obtained using a sliding-window of length 50 data points, as an average of all G10 (Group of Ten) currency pairs, covering a period of 8 hours in the case of 0.25 second and 0.5 second sampled data and 2 years in case of 0.5 hour sampled data. The non-Gaussian (heavy-tailed, skewed) nature of the data is clearly visible. It is interesting to note that the kurtosis value almost never goes below three for any of the data sets, signifying the temporal persistence of non-Gaussianity. We now make use of the Iman-Conover method to induce varying levels of correlation between 1000 samples taken from independent, randomly distributed, Pearson type IV distributions. A 1000 sample data set makes it easier to accurately induce predefined correlations in the system as well as makes it possible to generate data with relatively accurate average kurtosis and skewness values, hence, allowing us to accurately capture the higher-order moments of financial returns. As an example, Figures 1(c) and 1(d) show distributions of the kurtosis and skewness of two variables for 1000 independent simulations. Note the similarity of these plots with the average of the corresponding distributions of higher-order moments for financial data, as presented in Figures 1(a) and 1(b). This shows the effectiveness of using synthetic data sampled from a Pearson type IV distribution for capturing higher-order moments of financial returns.(a, b) Plots showing the normalised pdfs of (a) kurtosis (κ) and (b) skewness (γ), obtained using a sliding-window of length 50 data points, as an average of all G10 currency pairs, covering a period of 8 hours in case of 0.25 second and 0.5 second sampled data and 2 years in case of 0.5 hour sampled data. The plots clearly show the heavy-tailed, skewed, nature of FX returns at all three sampling frequencies. (c, d) An example of the normalised pdf plots showing the average kurtosis and skewness values, respectively, for synthetic data generated using Pearson type IV distributions. The data has properties similar to the average of the distributions presented in (a) and (b). The vertical lines indicate the kurtosis (κ=3) and skewness (γ=0) values for a Gaussian distribution.
(a)
Financial data
(b)
Financial data
(c)
Synthetic data
(d)
Synthetic dataFour different approaches are now used to estimate the level of dependence between the output coupled data. The process is repeated 1000 times for each level oftrue correlation (ρTC). Table 2 presents the results obtained. The results show the accuracy of the information coupling measure when used to analyse non-Gaussian data. For this synthetic data example, on average, the information coupling measure was 53.7% more accurate than the linear correlation measure and 25.6% more accurate with respect to the rank correlation measure. The normalised mutual information provided the least accurate results.Table 2
Table showing accuracy of four measures of dependence, that is, information coupling (η), linear correlation (ρ), rank correlation (ρR), and normalised mutual information (IN), when used to estimate the level of dependence in a coupled system with varying levels of true correlation (ρTC). ρTC is induced in an independent bivariate, randomly distributed, system using the Iman-Conover method as described in the text. The dependence estimates, together with their standard deviation values, shown in the table are obtained using 1000 independent simulations using 1000 data points long data sets for each simulation. The last row of the table gives values for the mean absolute error (MAE).
ρ
TC
η
ρ
ρ
R
I
N
|
ρ
TC
-
η
|
|
ρ
TC
-
ρ
|
|
ρ
TC
-
ρ
R
|
|
ρ
TC
-
I
N
|
0
0.0046
±
0.0026
0.0038
±
0.0022
0.0147
±
0.0115
0.1851
±
0.0401
0.0046
0.0038
0.0147
0.1851
0.1
0.1079
±
0.0073
0.0917
±
0.0058
0.0942
±
0.0184
0.1687
±
0.0412
0.0079
0.0083
0.0058
0.0687
0.2
0.2145
±
0.0102
0.1862
±
0.0093
0.1899
±
0.0177
0.1131
±
0.0464
0.0145
0.0138
0.0111
0.0869
0.3
0.3176
±
0.0138
0.2812
±
0.0130
0.2863
±
0.0167
0.1644
±
0.0538
0.0176
0.0188
0.0137
0.1356
0.4
0.4166
±
0.0210
0.3764
±
0.0165
0.3835
±
0.0153
0.2787
±
0.0500
0.0166
0.0236
0.0165
0.1213
0.5
0.5135
±
0.0196
0.4720
±
0.0197
0.4818
±
0.0136
0.3819
±
0.0475
0.0135
0.0280
0.0182
0.1181
0.6
0.6070
±
0.0347
0.5688
±
0.0226
0.5815
±
0.0117
0.4788
±
0.0458
0.0070
0.0312
0.0185
0.1212
0.7
0.7011
±
0.0232
0.6670
±
0.0242
0.6830
±
0.0093
0.5728
±
0.0483
0.0011
0.0330
0.0170
0.1272
0.8
0.7936
±
0.0239
0.7676
±
0.0249
0.7864
±
0.0066
0.6652
±
0.0490
0.0064
0.0324
0.0136
0.1348
0.9
0.8864
±
0.0252
0.8713
±
0.0249
0.8919
±
0.0034
0.7595
±
0.0501
0.0136
0.0287
0.0081
0.1405
1.0
1.0000
±
0.0000
1.0000
±
0.0000
1.0000
±
0.0000
0.9601
±
0.0185
0.0000
0.0000
0.0000
0.0399
MAE
0.0093
0.0201
0.0125
0.1163We now extend this example by incorporating data dynamics. The same data generation process, as described above, is now used to construct a 32000 samples long bivariate data set in which the induced true correlation changes every8000 time steps; that is, ρTC=0.2 when t=1 : 8000, ρTC=0.4 when t=8001 : 16000, ρTC=0.6 when t=16001 : 24000, and ρTC=0.8 when t=24001 : 32000. A 1000 data points wide sliding-window is used to dynamically measure dependencies in the data set. The resulting temporal information coupling plot, together with the 5th and 95th percentile confidence intervals, is presented in Figure 2(a). The four different coupling regions are clearly visible, together with the step changes in coupling after every 8000 time steps, showing ability of the algorithm to detect abrupt changes in coupling. The normalised empirical probability distributions over η for the four coupling regions are shown in Figure 2(b). Also plotted in the same figure are the normalised empirical pdfs for linear correlation (ρ) and rank correlation (ρR). The mutual information pdf is omitted for clarity as it gives relatively less accurate results, as presented in Table 2. It is interesting to see how the peaks of the η distribution correspond very closely to ρTC values, showing ability of the information coupling model to accurately capture statistical dependencies in a dynamic environment. The least accurate measure in this example is the linear correlation.(a) Temporal variation of information coupling,η(t), plotted as a function of time. Step changes in coupling are visible after every 8000 time steps; that is, ρTC=0.2 when t=1 : 8000, ρTC=0.4 when t=8001 : 16000, ρTC=0.6 when t=16001 : 24000, and ρTC=0.8 when t=24001 : 32000. Also plotted are the median, [η(t)]median, and the 5% and 95% confidence interval contours, [η(t)]5%95%. (b) Normalised empirical pdf plots for information coupling (η), linear correlation (ρ), and rank correlation (ρR). The vertical lines represent the true correlation (ρTC) values. The relative accuracy of the information coupling measure is evident from these results.
(a)
(b)


### 5.2. Financial Data

We first present a simple application of the information coupling algorithm to a section of 0.5 second sampled FX spot log-returns data set (in this paper, all currencies are referred by their standardised international three-letter codes, as described by the ISO 4217 standard. For the currencies mentioned in this paper, the three-letter codes are USD (U.S. dollar), EUR (Euro), JPY (Japanese yen), GBP (British pound), CHF (Swiss franc), and AUD (Australian dollar)). Figure3 shows the variation of information coupling and linear correlation with time for EURUSD and GBPUSD. The results are obtained using a 5-minute wide sliding-window. We note that the two measures of dependence frequently give different results, which reflects on the inability of linear correlation to capture dependencies in non-Gaussian data streams. We also note that dependencies in FX log-returns exhibit rapidly changing dynamics, often characterised by regions of quasi-stability punctuated by abrupt changes. In [48] we show that these regions of persistence in statistical dependence may be captured using a hidden Markov ICA model, which is a hidden Markov model with an ICA observation model. The information thus obtained can be useful for a range of financial applications, such as finding regions of financial market volatility clustering and persistence [49].Figure 3
Information coupling (ηt) and linear correlation (ρt) plotted as a function of time for a section of 0.5 second sampled EURUSD and GBPUSD log-returns data set. A 5-minute wide sliding-window is used to obtain the results.There have been numerous academic studies on the causes and effects of the 2008 financial crisis [50, 51]. However, very few of these have focused on the impact of the crisis on interdependencies in the global FX market. Here we present a set of examples which give a unique insight into the effects of the crisis on the nature of statistical dependencies in the spot FX market. Accurate estimation of dependencies at times of financial crises is of utmost importance, as these estimates are used by financial practitioners for a range of tasks, such as rebalancing portfolios, accurately pricing options, and deciding on the level of risk-taking. We first present an application of the information coupling model for detecting temporal changes in dependencies in bivariate FX data streams at times of financial crises. Figure 4 shows the adjusted daily closing mid-prices (Pt) for AUDUSD and USDJPY from January 2005 till April 2010. The two plots clearly show an abrupt change in the exchange rates in September-October 2008. This was caused at the height of the 2008 global financial crisis due to the unwinding of carry trades [52]. Figure 5(a) displays three plots showing the temporal variation of information coupling (ηt), linear correlation (ρt), and rank correlation (ρR,t) between AUDUSD and USDJPY log-returns. The plots are obtained using a six-month long sliding-window. We notice the rise in uncertainty of the information coupling measure (Figure 5(b)) right before the crash, with uncertainty decreasing gradually thereafter; this information may be useful to systematically predict upheavals in the market, although we do not carry out this study in detail in this paper. Information about the level of uncertainty can be used as a measure of confidence in the information coupling values and can be useful in various practical decision making scenarios, such as deciding on the capital to deploy for the purpose of trading or selecting stocks (or currencies) for inclusion in a portfolio. As daily sampled data is generally less non-Gaussian than data sampled at higher frequencies, therefore, the three plots in Figure 5(a) are somewhat similar during certain time periods. However, right after the September 2008 crash, the plots significantly deviate from each other. We believe that this is because the nature of the data, in particular its level of non-Gaussianity, has changed. As shown in Figure 6, the distance measure, (ηt-|ρt|)2, between information coupling and linear correlation closely matches the non-Gaussianity of the data under consideration. The two plots are adjusted for ease of comparison. The degree of non-Gaussianity is calculated using the multivariate Jarque-Bera statistic (JBMV) which we define for a N-dimensional multivariate data set as
(40)JBMV=∑j=1N‍[ns6(γj2+(κj-3)24)]2,
where ns is the number of data points (in this case the size of the sliding-window), γ is the skewness of the data under analysis, and κ is its kurtosis. This shows that relying solely on correlation measures to model dependencies in multivariate financial time series, even when using data sampled at relatively lower frequencies, can potentially lead to inaccurate results. In contrast, the information coupling model takes into account properties of the data being analysed, resulting in an accurate approach to measure statistical dependencies.Figure 4
Daily closing mid-prices (Pt) of AUDUSD and USDJPY. The two plots are adjusted for ease of comparison. The vertical line corresponds to the September 2008 financial meltdown.(a) Three approaches used to measure temporal dependencies between AUDUSD and USDJPY log-returns. (b) Range of confidence intervals (Δ[η]5%95%=η95%-η5%) plotted as a function of time, showing the uncertainty associated with the information coupling measurements. The vertical lines correspond to the September 2008 financial meltdown.
(a)
(b)Figure 6
Difference between information coupling and linear correlation plotted as a function of time. Also plotted is a measure of non-Gaussianity of the two time series as defined by (40). The two plots are adjusted for ease of comparison. The vertical line corresponds to the September 2008 financial meltdown.We now show utility of the information coupling model for analysing multivariate statistical dependencies. Figure7 shows the temporal variation of information coupling between four major liquid currency pairs (EURUSD, GBPUSD, USDCHF, and USDJPY). The results are obtained using daily log-returns for a seven-year period and a six-month long sliding-window. Also plotted on the same figure is the FTSE-100 (Financial Times Stock Exchange 100) index for the corresponding time period, which has been adjusted for ease of comparison. The plot clearly shows an abrupt upward shift in coupling between the four currency pairs right at the time of the September 2008 financial meltdown, with gradual decrease in coupling over the next year. We also notice an increase in the uncertainty associated with the information coupling measure before the 2008 crash. The increase in dependence of financial instruments in times of financial crises has been observed for other asset classes as well [53]. Our unique example, showing the dynamics of multivariate dependencies within the spot FX space, provides further insight into the nature of interdependencies in times of financial crisis.Figure 7
Information coupling between EURUSD, GBPUSD, USDCHF, and USDJPY log-returns plotted as a function of time. Also plotted is the FTSE-100 index, which is adjusted for ease of comparison. The vertical line corresponds to the September 2008 financial meltdown.


## 5.1. Synthetic Data

There is no single distribution which fits financial returns, especially those sampled at higher frequencies which tend to be highly non-Gaussian [2], although there have been attempts to model returns using a variety of distributions [38]. In this paper, we aim to capture the heavy-tailed, skewed, properties of financial returns using a Pearson type IV distribution [39, 40], which can be used to represent distributions with varying degrees of skewness and kurtosis and thus are useful for representing distributions of financial returns [41, 42]. The first four moments of the distribution can be uniquely determined by setting four parameters which characterise the distribution [43]. Until recently, due to its mathematical and computational complexity, this distribution has not been widely used in financial literature, although this is rapidly changing with advances in computational power and proposal of new, improved analytical methods [39, 42, 44].To test accuracy of various measures of dependence, we need to generate coupled non-Gaussian synthetic data with known, predefined, correlation values. There is no straightforward way to simulate correlated random variables when their joint distribution is not known [46], as is the case with multivariate financial returns. One possible method that can be used to induce any desired predefined correlation between independent, randomly distributed variables, irrespective of their distributions, is commonly known as the Iman-Conover method, as presented in [47]. This method is based on inducing a known dependency structure in samples taken from the input independent marginal distributions using reordering techniques. The multivariate coupled structure obtained as the output can thus be used as the input data in various models of dependency analysis to test their relative accuracies.We set parameters of the Pearson type IV distribution such that the coupled synthetic data we obtain using the Iman-Conover method has similar properties to financial returns. But first we need to consider some properties of financial returns which we want to mimic. Figures1(a) and 1(b) show the distributions of two higher-order moments, that is, skewness (γ) and kurtosis (κ), for FX spot data sets sampled at three different frequencies. The plots are obtained using a sliding-window of length 50 data points, as an average of all G10 (Group of Ten) currency pairs, covering a period of 8 hours in the case of 0.25 second and 0.5 second sampled data and 2 years in case of 0.5 hour sampled data. The non-Gaussian (heavy-tailed, skewed) nature of the data is clearly visible. It is interesting to note that the kurtosis value almost never goes below three for any of the data sets, signifying the temporal persistence of non-Gaussianity. We now make use of the Iman-Conover method to induce varying levels of correlation between 1000 samples taken from independent, randomly distributed, Pearson type IV distributions. A 1000 sample data set makes it easier to accurately induce predefined correlations in the system as well as makes it possible to generate data with relatively accurate average kurtosis and skewness values, hence, allowing us to accurately capture the higher-order moments of financial returns. As an example, Figures 1(c) and 1(d) show distributions of the kurtosis and skewness of two variables for 1000 independent simulations. Note the similarity of these plots with the average of the corresponding distributions of higher-order moments for financial data, as presented in Figures 1(a) and 1(b). This shows the effectiveness of using synthetic data sampled from a Pearson type IV distribution for capturing higher-order moments of financial returns.(a, b) Plots showing the normalised pdfs of (a) kurtosis (κ) and (b) skewness (γ), obtained using a sliding-window of length 50 data points, as an average of all G10 currency pairs, covering a period of 8 hours in case of 0.25 second and 0.5 second sampled data and 2 years in case of 0.5 hour sampled data. The plots clearly show the heavy-tailed, skewed, nature of FX returns at all three sampling frequencies. (c, d) An example of the normalised pdf plots showing the average kurtosis and skewness values, respectively, for synthetic data generated using Pearson type IV distributions. The data has properties similar to the average of the distributions presented in (a) and (b). The vertical lines indicate the kurtosis (κ=3) and skewness (γ=0) values for a Gaussian distribution.
(a)
Financial data
(b)
Financial data
(c)
Synthetic data
(d)
Synthetic dataFour different approaches are now used to estimate the level of dependence between the output coupled data. The process is repeated 1000 times for each level oftrue correlation (ρTC). Table 2 presents the results obtained. The results show the accuracy of the information coupling measure when used to analyse non-Gaussian data. For this synthetic data example, on average, the information coupling measure was 53.7% more accurate than the linear correlation measure and 25.6% more accurate with respect to the rank correlation measure. The normalised mutual information provided the least accurate results.Table 2
Table showing accuracy of four measures of dependence, that is, information coupling (η), linear correlation (ρ), rank correlation (ρR), and normalised mutual information (IN), when used to estimate the level of dependence in a coupled system with varying levels of true correlation (ρTC). ρTC is induced in an independent bivariate, randomly distributed, system using the Iman-Conover method as described in the text. The dependence estimates, together with their standard deviation values, shown in the table are obtained using 1000 independent simulations using 1000 data points long data sets for each simulation. The last row of the table gives values for the mean absolute error (MAE).
ρ
TC
η
ρ
ρ
R
I
N
|
ρ
TC
-
η
|
|
ρ
TC
-
ρ
|
|
ρ
TC
-
ρ
R
|
|
ρ
TC
-
I
N
|
0
0.0046
±
0.0026
0.0038
±
0.0022
0.0147
±
0.0115
0.1851
±
0.0401
0.0046
0.0038
0.0147
0.1851
0.1
0.1079
±
0.0073
0.0917
±
0.0058
0.0942
±
0.0184
0.1687
±
0.0412
0.0079
0.0083
0.0058
0.0687
0.2
0.2145
±
0.0102
0.1862
±
0.0093
0.1899
±
0.0177
0.1131
±
0.0464
0.0145
0.0138
0.0111
0.0869
0.3
0.3176
±
0.0138
0.2812
±
0.0130
0.2863
±
0.0167
0.1644
±
0.0538
0.0176
0.0188
0.0137
0.1356
0.4
0.4166
±
0.0210
0.3764
±
0.0165
0.3835
±
0.0153
0.2787
±
0.0500
0.0166
0.0236
0.0165
0.1213
0.5
0.5135
±
0.0196
0.4720
±
0.0197
0.4818
±
0.0136
0.3819
±
0.0475
0.0135
0.0280
0.0182
0.1181
0.6
0.6070
±
0.0347
0.5688
±
0.0226
0.5815
±
0.0117
0.4788
±
0.0458
0.0070
0.0312
0.0185
0.1212
0.7
0.7011
±
0.0232
0.6670
±
0.0242
0.6830
±
0.0093
0.5728
±
0.0483
0.0011
0.0330
0.0170
0.1272
0.8
0.7936
±
0.0239
0.7676
±
0.0249
0.7864
±
0.0066
0.6652
±
0.0490
0.0064
0.0324
0.0136
0.1348
0.9
0.8864
±
0.0252
0.8713
±
0.0249
0.8919
±
0.0034
0.7595
±
0.0501
0.0136
0.0287
0.0081
0.1405
1.0
1.0000
±
0.0000
1.0000
±
0.0000
1.0000
±
0.0000
0.9601
±
0.0185
0.0000
0.0000
0.0000
0.0399
MAE
0.0093
0.0201
0.0125
0.1163We now extend this example by incorporating data dynamics. The same data generation process, as described above, is now used to construct a 32000 samples long bivariate data set in which the induced true correlation changes every8000 time steps; that is, ρTC=0.2 when t=1 : 8000, ρTC=0.4 when t=8001 : 16000, ρTC=0.6 when t=16001 : 24000, and ρTC=0.8 when t=24001 : 32000. A 1000 data points wide sliding-window is used to dynamically measure dependencies in the data set. The resulting temporal information coupling plot, together with the 5th and 95th percentile confidence intervals, is presented in Figure 2(a). The four different coupling regions are clearly visible, together with the step changes in coupling after every 8000 time steps, showing ability of the algorithm to detect abrupt changes in coupling. The normalised empirical probability distributions over η for the four coupling regions are shown in Figure 2(b). Also plotted in the same figure are the normalised empirical pdfs for linear correlation (ρ) and rank correlation (ρR). The mutual information pdf is omitted for clarity as it gives relatively less accurate results, as presented in Table 2. It is interesting to see how the peaks of the η distribution correspond very closely to ρTC values, showing ability of the information coupling model to accurately capture statistical dependencies in a dynamic environment. The least accurate measure in this example is the linear correlation.(a) Temporal variation of information coupling,η(t), plotted as a function of time. Step changes in coupling are visible after every 8000 time steps; that is, ρTC=0.2 when t=1 : 8000, ρTC=0.4 when t=8001 : 16000, ρTC=0.6 when t=16001 : 24000, and ρTC=0.8 when t=24001 : 32000. Also plotted are the median, [η(t)]median, and the 5% and 95% confidence interval contours, [η(t)]5%95%. (b) Normalised empirical pdf plots for information coupling (η), linear correlation (ρ), and rank correlation (ρR). The vertical lines represent the true correlation (ρTC) values. The relative accuracy of the information coupling measure is evident from these results.
(a)
(b)


## 5.2. Financial Data

We first present a simple application of the information coupling algorithm to a section of 0.5 second sampled FX spot log-returns data set (in this paper, all currencies are referred by their standardised international three-letter codes, as described by the ISO 4217 standard. For the currencies mentioned in this paper, the three-letter codes are USD (U.S. dollar), EUR (Euro), JPY (Japanese yen), GBP (British pound), CHF (Swiss franc), and AUD (Australian dollar)). Figure3 shows the variation of information coupling and linear correlation with time for EURUSD and GBPUSD. The results are obtained using a 5-minute wide sliding-window. We note that the two measures of dependence frequently give different results, which reflects on the inability of linear correlation to capture dependencies in non-Gaussian data streams. We also note that dependencies in FX log-returns exhibit rapidly changing dynamics, often characterised by regions of quasi-stability punctuated by abrupt changes. In [48] we show that these regions of persistence in statistical dependence may be captured using a hidden Markov ICA model, which is a hidden Markov model with an ICA observation model. The information thus obtained can be useful for a range of financial applications, such as finding regions of financial market volatility clustering and persistence [49].Figure 3
Information coupling (ηt) and linear correlation (ρt) plotted as a function of time for a section of 0.5 second sampled EURUSD and GBPUSD log-returns data set. A 5-minute wide sliding-window is used to obtain the results.There have been numerous academic studies on the causes and effects of the 2008 financial crisis [50, 51]. However, very few of these have focused on the impact of the crisis on interdependencies in the global FX market. Here we present a set of examples which give a unique insight into the effects of the crisis on the nature of statistical dependencies in the spot FX market. Accurate estimation of dependencies at times of financial crises is of utmost importance, as these estimates are used by financial practitioners for a range of tasks, such as rebalancing portfolios, accurately pricing options, and deciding on the level of risk-taking. We first present an application of the information coupling model for detecting temporal changes in dependencies in bivariate FX data streams at times of financial crises. Figure 4 shows the adjusted daily closing mid-prices (Pt) for AUDUSD and USDJPY from January 2005 till April 2010. The two plots clearly show an abrupt change in the exchange rates in September-October 2008. This was caused at the height of the 2008 global financial crisis due to the unwinding of carry trades [52]. Figure 5(a) displays three plots showing the temporal variation of information coupling (ηt), linear correlation (ρt), and rank correlation (ρR,t) between AUDUSD and USDJPY log-returns. The plots are obtained using a six-month long sliding-window. We notice the rise in uncertainty of the information coupling measure (Figure 5(b)) right before the crash, with uncertainty decreasing gradually thereafter; this information may be useful to systematically predict upheavals in the market, although we do not carry out this study in detail in this paper. Information about the level of uncertainty can be used as a measure of confidence in the information coupling values and can be useful in various practical decision making scenarios, such as deciding on the capital to deploy for the purpose of trading or selecting stocks (or currencies) for inclusion in a portfolio. As daily sampled data is generally less non-Gaussian than data sampled at higher frequencies, therefore, the three plots in Figure 5(a) are somewhat similar during certain time periods. However, right after the September 2008 crash, the plots significantly deviate from each other. We believe that this is because the nature of the data, in particular its level of non-Gaussianity, has changed. As shown in Figure 6, the distance measure, (ηt-|ρt|)2, between information coupling and linear correlation closely matches the non-Gaussianity of the data under consideration. The two plots are adjusted for ease of comparison. The degree of non-Gaussianity is calculated using the multivariate Jarque-Bera statistic (JBMV) which we define for a N-dimensional multivariate data set as
(40)JBMV=∑j=1N‍[ns6(γj2+(κj-3)24)]2,
where ns is the number of data points (in this case the size of the sliding-window), γ is the skewness of the data under analysis, and κ is its kurtosis. This shows that relying solely on correlation measures to model dependencies in multivariate financial time series, even when using data sampled at relatively lower frequencies, can potentially lead to inaccurate results. In contrast, the information coupling model takes into account properties of the data being analysed, resulting in an accurate approach to measure statistical dependencies.Figure 4
Daily closing mid-prices (Pt) of AUDUSD and USDJPY. The two plots are adjusted for ease of comparison. The vertical line corresponds to the September 2008 financial meltdown.(a) Three approaches used to measure temporal dependencies between AUDUSD and USDJPY log-returns. (b) Range of confidence intervals (Δ[η]5%95%=η95%-η5%) plotted as a function of time, showing the uncertainty associated with the information coupling measurements. The vertical lines correspond to the September 2008 financial meltdown.
(a)
(b)Figure 6
Difference between information coupling and linear correlation plotted as a function of time. Also plotted is a measure of non-Gaussianity of the two time series as defined by (40). The two plots are adjusted for ease of comparison. The vertical line corresponds to the September 2008 financial meltdown.We now show utility of the information coupling model for analysing multivariate statistical dependencies. Figure7 shows the temporal variation of information coupling between four major liquid currency pairs (EURUSD, GBPUSD, USDCHF, and USDJPY). The results are obtained using daily log-returns for a seven-year period and a six-month long sliding-window. Also plotted on the same figure is the FTSE-100 (Financial Times Stock Exchange 100) index for the corresponding time period, which has been adjusted for ease of comparison. The plot clearly shows an abrupt upward shift in coupling between the four currency pairs right at the time of the September 2008 financial meltdown, with gradual decrease in coupling over the next year. We also notice an increase in the uncertainty associated with the information coupling measure before the 2008 crash. The increase in dependence of financial instruments in times of financial crises has been observed for other asset classes as well [53]. Our unique example, showing the dynamics of multivariate dependencies within the spot FX space, provides further insight into the nature of interdependencies in times of financial crisis.Figure 7
Information coupling between EURUSD, GBPUSD, USDCHF, and USDJPY log-returns plotted as a function of time. Also plotted is the FTSE-100 index, which is adjusted for ease of comparison. The vertical line corresponds to the September 2008 financial meltdown.


## 6. Conclusions

We present an ICA-based approach to dynamically measure information coupling, as a proxy for mutual information. This approach makes use of ICA as a tool to capture information in the tails of the underlying distributions and is suitable for efficiently and accurately measuring statistical dependencies between multiple non-Gaussian signals. As far as we know, this is the first attempt to quantify multivariate dependencies using information encoded in the ICA unmixing matrix. Our proposed information coupling model has multiple other benefits associated with its practical use. It provides a framework for estimating confidence bounds on the information coupling metric, can be efficiently used to directly model dependencies in high-dimensional spaces, and gives normalised, symmetric results. It has the added advantage of not depending on any user-defined parameters and is not data intensive; that is, it can be used even with relatively small data sets without significantly affecting its performance, an important requirement for analysing data streams with rapidly changing dynamics such as financial returns. The model makes use of a sliding-window based decorrelating manifold approach to ICA, with a reciprocal cosh source model, to infer the ICA unmixing matrix, which results in increased accuracy and efficiency of the algorithm. This gives the information coupling model the computational complexity similar to that of linear correlation with the accuracy of mutual information.



---

*Source: 434832-2013-06-02.xml*