# Correlation Angles and Inner Products: Application to a Problem from Physics

Authors: Adam Towsley; Jonathan Pakianathan; David H. Douglass
Journal: ISRN Applied Mathematics
 (2011)

---

## Abstract

Covariance is used as an inner product on a formal vector space built onn random variables to define measures of correlation Md across a set of vectors in a d-dimensional space. For d=1, one has the diameter; for d=2, one has an area. These concepts are directly applied to correlation studies in climate science.


---

## Body

## 1. Introduction

In a study of the earth's climate system, Douglass [1] considered the correlation among a set of N climate indices. A distance d between two indices i and j was defined as(1.1)dij(t)=cos⁡-1⁡(|φij(t)|),
where φij is the Pearson correlation coefficient. It was stated that d satisfies the conditions to be a metric. The measure of correlation, or closeness, among the N indices was taken to be the diameter D(1.2)DI0(t)=max⁡⁡{dij(t)∣i,j∈I0}.
Equation (1.2) was applied to the data from a global set of four climate indices to determine the correlation among them (minimum in D) and to infer 18 changes in the state since 1970 (see Section 8). It was pointed out that the topological diameter D, as a measure of phase locking among the indices, is convenient for computation but was probably not the best measure. It was suggested that a better measure of correlation among the N indices could be based upon the area of the spherical triangles created by the N vectors on the unit sphere.This paper gives a proof thatdij is a metric and generalizes the diameter to higher dimensions. In addition, the data of [1] are analyzed using this generalization to areas (see Section 8), and many new abrupt climate changes are identified.


## 2. Probability

LetX and Y be random variables with expected values E(X)=μ and E(Y)=ν. With these values, we make several standard definitions.Definition 2.1.
The variance ofX is defined as
(2.1)Var⁡[X]=E[(X-μ)2].Definition 2.2.
The covariance ofX and Y is defined as
(2.2)Covar[X,Y]=E[(X-μ)(Y-ν)].We now list a few basic properties of variance and covariance (found in [2]).Properties 2.3
ForX and Y as above. (i)
Covar is symmetric.(ii)
Covar is bilinear.(iii)
Var[-] is a quadratic form.(iv)
Covar[X,Y]=E[XY]-E[X]E[Y].(v)
Covar[X,X]=Var[X], the variance of X.Proof.
(i) See [2, page 323]. (ii) Follows easily from the definition. (iii) See [2, page 323]. (iv) See [2, page 323].


## 3. Vector Spaces

The first way most students learn to compare two vectors is through the dot product. The dot product is one example of the more general idea of an inner product. Here we define an inner product and prove that covariance is an inner product.Definition 3.1.
For any real vector spaceV, an inner product is a map
(3.1)〈-,-〉:V×V⟶R
that satisfies the following properties for every u,v,w∈V, and a∈ℝ: (i)
〈u+v,w〉=〈u,w〉+〈v,w〉(ii)
〈av,w〉=a〈v,w〉(iii)
〈v,w〉=〈w,v〉(iv)
〈v,v〉≥0 and 〈v,v〉=0 if and only if v=0.We will now construct a vector space for which covariance is an inner product. Let{X1,X2,…,Xn} be a set of n random variables. Also let V=Spanℝ(X1,X2,…,Xn), the formal ℝ-vector space with basis elements {X1,X2,…,Xn}. We must put one mild hypothesis upon V in order for it to have the desired properties. The hypothesis is that the vectors must be “probabilistically independent.” That is, for any c1,…,cn∈ℝ, we have that Var[c1X1+⋯+cnXn]=0 if and only if c1=⋯=cn=0. It should be noted that this independence is in no way related to the linear independence of the random variables.Proposition 3.2.
LetV=Spanℝ(X1,X2,…,Xn), the formal ℝ-vector space generated by the random variables {X1,X2,…,Xn} which are probabilistically independent, then covariance is an inner product on V.Proof.
We must prove the four properties from Definition3.1.
(i), (ii), and (iii) follow immediately from Properties2.3.
(iv)Covar(X,X)=E[(x-μ)2]≥0. The nonnegativity is obvious as we are squaring a real number. The condition that Covar(X,X)=0⇔X=0 follows from the probabilistic independence of {X1,…,Xn}.The proposition implies thatV is an inner product space (a vector space equipped with an inner product), and as such it has a norm defined by ∥X∥=Covar(X,X)=SD(X), where SD(X) is the standard deviation of X. Additionally it follows from the Cauchy-Schwartz inequality [3] that |Covar(X,Y)|≤SD(X)SD(Y).Using the inner product onV, we are able to define an angle between two vectors. To do this, we first define a new map ρ:(V∖{0})×(V∖{0})→ℝ using the standard definition of correlation (3.2)ρ(X,Y)=Covar(X,Y)SD(X)SD(Y).
By the Cauchy-Schwartz inequality, we can easily see that |ρ(X,Y)|≤1, as such we implicitly define Γ, the angle between X and Y, as follows:
(3.3)Covar(X,Y)=SD(X)SD(Y)cos⁡⁡(Γ).
Therefore, ρ(X,Y)=Covar(X,Y)/SD(X)SD(Y)=cos(Γ).Definition 3.3.
Γ(X,Y)=cos-1(ρ(X,Y)) is the “Correlation Angle” of X and Y.Our definition ofΓ is the standard method of defining an angle from the covariance (or any other) inner product. We will show that Γ is a “metric” on the unit sphere of V.Definition 3.4.
For any setS, a map d:S×S→ℝ is a metric if for any x,y,z∈S the following properties are satisfied:
(a)
d(x,y)≥0withd(x,y)=0⇔x=y(positive definite),(b)
d(x,y)=d(y,x)(symmetry),(c)
d(x,z)≤d(x,y)+d(y,z)(triangle inequality).Theorem 3.5.
The mapΓ:V×V→ℝ from Definition 3.3 is a metric on S(V)= the unit sphere of V.Proof.
We must prove thatΓ satisfies the 3 conditions in Definition 3.4. (a)
cos-1:[-1,1]→[0,π] so the nonnegativity is satisfied trivially. It remains to show that Γ(X,Y)=0⇔X=Y. This is true because if the angle between two vectors is zero, then they are (positive) scalar multiples of each other. Thus since X and Y are unit vectors, if Γ(X,Y)=0, we must have X=Y.(b)
Γ(X,Y)=cos-1(ρ(X,Y))=cos-1(Covar(X,Y)/SD(X)SD(Y))=cos-1(Covar(Y,X)/SD(Y)SD(X))=cos-1(ρ(Y,X))=Γ(Y,X).(c)
To prove the triangle inequality, a geometric idea in itself, we delve into the geometry being defined. We will complete this part of the proof in Section4.Our metricΓ allows us to measure the correlation between two vectors.Definition 3.6.
ForX, Y, Γ, and ρ as above:(i)
ifΓ=0(ρ=1), then X and Y are maximally positively correlated.(ii)
IfΓ=π(ρ=-1), then they are maximally negatively correlated.(iii)
IfΓ=π/2(ρ=0), then X and Y are uncorrelated.It should be noted that cases (i) and (ii) are both considered to be “maximally correlated.”


## 4. A Geometric Interpretation

The vector spaceV with inner product Covar lends itself nicely to a geometric interpretation. First we must establish a small amount of background.ConsiderS, the standard unit sphere in Euclidean n-space (ℝn). Great circles are the intersection of a plane through the origin and S. They share many properties with the standard idea of lines in Euclidean space, including the property that they define the shortest path between any two points. For a thorough treatment of great circles as lines on a sphere, see [4–6] or [7].For any two nonzero vectorsv1 and v2 in ℝn, let θ be the (minimal) angle formed by v1 and v2. The unit vectors v̂1 and v̂2, corresponding to v1 and v2, define two points p1 and p2 on S. In order to measure the distance from p1 to p2 along S, we take the length of the arc on great circle between the two points. By definition, this is the radian measure of θ.IfV, the vector space considered in Section 3, is thought of as ℝn with v1 and v2 any two vectors, then we can compute the spherical distance between v1 and v2, namely, the distance between p1 and p2 on S. We call this quantity Γ(4.1)dspherical(v1,v2)=arccos⁡(ρ(v̂1,v̂2))=Γ.Thus far we have identified the inner product space(V,Covar) as ℝn. We solidify this intuition with the following proposition. First we define A=(Ai,j)=(Covar(Xi,Xj)), a real valued symmetric matrix. As in [3], we use A to create the inner product on ℝn.Proposition 4.1.
The inner product space〈Spanℝ(X1,…,Xn),Covar〉≅〈ℝn,·A〉, where ·A is a “twisted dot product” defined for two vectors (c1,…,cn) and (d1,…,dn) as
(4.2)(c1,…,cn)⋅A(d1,…,dn)∶=(c1,…,cn)A(d1⋮dn).Proof.
This follows from the standard method of representing an inner product by a matrix (see [3, chapter 8.1]).Now we return to our proof of Theorem3.5.Proof of Theorem3.5(iii).
LetX,Y,Z∈V be unit vectors. We have left to show that Γ(X,Y)+Γ(Y,Z)≥Γ(X,Z).
BecauseX and Z are unit vectors, Γ(X,Z) is the geodesic distance between X and Z. Since geodesic distance satisfies the triangle inequality, Γ must as well.


## 5. Projective Metric

For scientists,ρ=±1 (equivalently Γ=0 or Γ=π) are often both considered to be “maximally correlated,” for example, see [1]. To take this into account, we modify our metric on the unit sphere of V. We think of V as a projective space, the space of lines through the origin of V. We denote this space as ℙ(V).Our original correlation angleΓ is modified to be (5.1)Γ′=arccos⁡|ρ(X,Y)|={Γ:0≤Γ≤π2π-Γ:π2≤Γ≤π.Proposition 5.1.
Γ′(X,Y) is a metric on ℙ(ℝn).Proof.
We must show that the three conditions of Definition3.4 are met.
(i)
Γ′=0 corresponds to a correlation angle of 0 or π. The two vectors are either in the same direction or opposite direction. In either case, they determine the same line through the origin and hence correspond to the same point in projective space.(ii)
As in Definition3.4, the symmetry of Γ′ follows from the symmetry of ρ.(iii)
As before, the triangle inequality follows asΓ′ is the geodesic distance for a projective space.The metricΓ′(X,Y) gives the angular distance between X and Y. If ρ(X,Y)=±1 (what we called a “maximal correlation”) then Γ′=0, however, if ρ(X,Y)=0, which we called orthogonality or noncorrelation, then Γ′=π/2.Proposition 5.2.
LetΓ′ be the metric cos-1(ρ′(X,Y)), then the pair (ℙ(V),Γ′) is a projective metric space.Proof.
This is by construction.


## 6. Time Dependence

Until this point, we have treated our random variables{X1,…,Xn} as being time independent. However, random variables often depend on time. Therefore, we will now consider each random variable as depending discretely on time. It should be noted that what follows is essentially a replication of what has come before, however, X and Y are now treated as vectors instead of singleton points. Vectors, however, are just points of V. The additional theory and notation is simply a means of dealing with the additional information.To make ourn random variables time dependent, they will now be given as(6.1)X1={X1(t),X1(t+1),X1(t+2),…},X2={X2(t),X2(t+1),X2(t+2),…},⋮Xn={Xn(t),Xn(t+1),Xn(t+2),…}.We must now redefine the covariance. We do this by looking at a time window starting at timet with a duration of K, where K is called the summation window
(6.2)Covar(Xi,Xj)=1K∑l=tt+K-1(Xi(l)-μ)(Xj(l)-ν),
where μ and ν are the sample means in the summation window of Xi and Xj, respectively. That is, μ=(1/K)∑l=tp=t+K-1Xi(l).If we think ofXî and Xĵ as the vectors Xî=(Xi(t),…,Xi(t+K-1)) (resp., for Xĵ), then we get that (6.3)Covar(Xi,Xj)=1K(Xî-μ̂)⋅(Xĵ-ν̂),
where “·” is the standard Euclidean dot product, and μ̂ is the length K vector (μ,…,μ) (resp., for ν̂). This is called the “Pearson Covariance.”In other words, if we define the vectorsXî̂=(Xî(t)-μ)/K, and Xĵ̂=(Xĵ(t)-μ)/K, then we define the Pearson Correlation as follows.Definition 6.1.
CovarPearson(Xi,Xj)=Xî̂·Xĵ̂, where “·” is the usual Euclidean inner product.Now we define the Pearson Correlation as(6.4)ρ̂(Xi,Xj)=CovarPearson(Xi,Xj)CovarPearson(Xi,Xi)CovarPearson(Xj,Xj)=cos⁡⁡(Γ̂).
Here again Γ̂ corresponds to the standard Euclidean angle, known as the Pearson Correlation Angle, and the resulting metric is the standard metric studied in classical spherical geometry (see [4–6] or [7]).Remark 6.2.
The angleΓ̂ between Xî̂ and Xĵ̂ is the same as the angle between Xi¯ and Xj¯, the unit vectors corresponding to Xî̂ and Xĵ̂.


## 7. Correlation Measures:Mn and Mn,a

To this point, we have developed a method that will numerically tell us the correlation between two vectors. In this section, we will create two sets of functions that allow us to measure the correlation across a set of vectors. The first set,{Mi,a}, is based upon taking the volumes of i-simplices (a 1-simplex is a line, a 2-simplex a triangle, a 3-simplex is a tetrahedron, etc.). The set of Mi,a benefits from computability but is not as precise as the second set of measures {Mi}, that measure the volume of i-dimensional convex hulls.Given a set of vectors{X1,…,Xm}⊆V, let U={U1,…,Um} be the set of corresponding unit vectors. We will define a way to measure the closeness of the Ui to each other using the metric Γ. To do this, we define the diameter of U as (7.1)D=max⁡i,j⁡{Γ(Ui,Uj)}.
If all of the vectors are taken in the standard way to be points on the unit sphere, then the diameter is a measure of the overall spread of the points. If the diameter is small, then the vectors are all close together, hence highly correlated. Whereas if the diameter is large at least some of the points are far apart, hence not highly correlated. The benefit of the diameter is that it is an easy quantity to calculate; however, it can be somewhat misleading. If, for instance, a large number of points are clustered together and there is one outlying point, the diameter can be quite large despite the fact that the points are generally quite correlated.We now proceed to generalize the correlation measure defined byD. Let T be a collection of t points on the n-sphere, and let D be the set of n-simplices made up of points in T.Definition 7.1.
Mn,a(T)=maxΔ∈D{Vol(Δ)}.This maximum is taken over theC=C(T,n+1)=(tn+1) different n-simplices made of points in T.Definition 7.2.
Mn(T)=Vol(H).The volume used in Definition7.2 is the spherical volume, and H is the convex hull of the points of T with respect to the spherical measure. That is, it is the smallest geodesically convex set containing T. (Geodesically convex means that any two points in the set have the minimal geodesic between them completely in the set as well.)The volume is computed by constructing the convex hull ofT, then disregarding all the points of T not contributing to the hull. The hull is then divided into its “essential” n-simplices, and the volumes of these simplices are summed.Mn and Mn,a are each measures of n-dimensional volume. Mn,a benefits from being easily computable. Mn, though harder to compute, gives a better measure of the overall spread of the vectors. However, in the one dimensional case, we have that M1=M1,a=D, the diameter. The reason for this is that when making the hull to compute M1 all, but the two furthermost points will be disregarded. This equality is not true in general, a fact which can be easily observed by plotting four points forming a quadrilateral, where M2,a<M2. In the general case, however, we do have the inequality Mn,a(T)≤Mn(T). This follows since the maximal simplex will necessarily be a subset of the convex hull. Since volume is monotonic, we have the inequality.Assume thats of the t points of T are essential to the convex hull. There is a constant B=B(s,n) defining the number of essential simplices that compose the convex hull. That is, (7.2)Mn(T)=the sum of the volume ofB-essential simplices.
Replacing the volume of each spherical simplex with the maximal one, that is, Mn,a(T), we get the following inequalities (7.3)Mn,a(T)≤Mn(T)≤B⋅Mn,a(T).
Since B depends only on the number of points in T, we see that, for a fixed data set, Mn and Mn,a differ by at most a fixed constant.To relateMn and Mn,a to Section 6, we note that when T time-dependent random variables are looked at over a summation window of length K=n+1, then we get T points on the n-sphere. In this situation, we can apply the measures of spread given by Mn(T) or Mn,a(T) or Mk,a(T) for k<n.


## 8. Topology of Earth's Climate Indices and Phase-Locked States

In this section, we apply our new correlation measure to data from Douglass's paper [1]. In [1], the diameter (M1=M1,a) is used to analyze a set of climate data; in this section, we use M2,a to analyze the same data. Comparing the results of the new analysis to Douglass's original analysis shows the increased effectiveness of the new correlation measure.Various regions of the Earth's climate system are characterized by temperature and pressure indices. Douglass [1], in a study of a global set of four indices, defines a distance(8.1)Γ̂ij(t)=cos⁡-1⁡(|ρ̂[Xi(t),Xj(t)]|)
between indices that satisfies the properties required to be a metric (Definition 3.4), where ρ(Xi(t),Xj(t)) is the Pearson correlation coefficient. Note that the distance Γ is an angle.In Section7 the correlation among a set of indices can be measured, using Mi,a by taking the volumes of i-simplices. In [1], Douglass uses the diameter of the metric space (I0,Γ), defined as
(8.2)DI0(t)=max⁡i,j⁡{Γ[Xi(t),Xj(t)]∣i,j∈I0}.In the notation of Section7,  D=M1,a. Geometrically, D selects the largest angle Γ(Xi,Xj) among the set. The diameter D may be considered a “dissimilarity” index because large D means weak correlation. Thus, the minima in D are associated with high correlation among the elements of the set. In Douglass, [1], two cases were considered: (1) the set of 3 Pacific ocean indices and (2) the global set of 4 indices (6 independent pairs). The D of the global set is shown (in red) in Figure 1.(a) 1870–1940. (b)1940–2010. The plots are for two different correlation measures among a set of four global climate indices—the diameterD=M1,a (in red) and the area A=M2,a (in blue) are defined in the text. Minima correspond to high correlation. The diameter D plot shows 18 identified minima while the area A plot shows 30. Comparisons are given in Table 1.
(a)(b)The maximal areaM2,a, the generalized correlation measure, was computed for the same four indices of [1]. The plot for the calculation is shown (blue) in Figures 1(a) and 1(b). Comparison of the two plots shows that the area measure reveals more minima (30) than the diameter (18). The various minima are indicated by arrows in Figures 1(a) and 1(b), and a list of dates is given in Table 1.Table 1
Date of various minima in plots of diameterD=M1,a or area A=M2,a. (Minima are identified with a change in the phase-locked state of the Earth's climate system).
DiameterD(from the 3 Pacific indices [1])AreaA (this paper)11875-18761875-187621880-18811880-188131882–18841883-188441889–189118894a1892New51894-18951894-189561897-189818976a1904New71908-19091907-190881912-19131912-191391916-1917XNo minimum101919191910a1925New10b1928New1119311931121941–19451941The broad minimum in [1] has been resolved into two.12a194412b1948New12c1953New12d1955New131956–19591959141964–19661965The minimum in [1] has been resolved into two.14a19661519691968-196915a1972-73New15b1974-75New161976-19771976-7716a1984-85New171986-19871986-8717a1991New182001-20022001Number of minima1830


## 9. Summary

By using covariance on a set of time-independent random variables or the covariance defined by the Pearson correlation on a set of time-dependent variables, we create metricsΓ and Γ̂ (resp.) on the unit sphere (resp., projective space) of the corresponding formal vector spaces. If V is the n-dimensional formal vector space whose basis is the set of random variables {X1,…,Xn}, we use Γ or Γ̂ to create Mn or Mn,a, two measures of spread on values taken by the Xi. In Section 8, we give an explicit example of showing the use of M2,a on a global set of climate indices.The two measures of spread differ by at most a fixed multiplicative constant, so for theoretical purposes, they are of equivalent use. However, when applied, they can have different values. The volume of the convex hull created of{X1,…,Xn}, given by Mn, is the most precise measure of the correlation of the Xi; however, it is computationally difficult. The maximal volume of all possible n-simplices defined by the Xi, given by Mn,a, is a rougher measure of correlation. However, Mn,a is a simpler computation than Mn.In the 2-dimensional example, where all the vectors lie on the 2-sphere, one can applyM2,a, M2, or M1,a=Diameter. But in general M1,a is coarser than M2,a but is significantly easier to compute. For example, in [1] and Section 8, the use of M2,a yields much finer and cleaner results than the use of M1,a. More generally in n-dimensions Ml and Ml,a for any l≤n, and one sacrifices accuracy for ease.



---

*Source: 323864-2011-10-17.xml*