# Novel Approaches to Identify Clusters Using Independent Components Analysis with Application

Authors: Saima Afzal; Muhammad Mutahir Iqbal; Ayesha Afzal; Hassan S. Bakouch; Sadiah M. A. Aljeddani
Journal: Mathematical Problems in Engineering
 (2023)

---

## Abstract

As a statistical and computational technique, independent component analysis (ICA) is employed to separate the source variables into statistically independent components. ICA methods have received growing attention as effective data mining tools. In this paper, two novel ICA-based approaches are proposed to identify the clusters of variables. The identified clusters reduce the dimensionality of the data in a natural way. The first approach, namely “Estimated Mixing Coefficients,” is based on the sum of squares of mixing coefficients, and the second approach, namely “RankedR¯2,” uses the ranking pattern of R¯2 of the original and reconstructed series at predefined threshold levels. The proposed techniques are applied to financial time series data to validate their effectiveness. The main focus of the study is on the clustering of multivariate time series datasets using two new proposed approaches based on independent component analysis. The internal and external structures of clusters are also explored using different metrics. Both proposed techniques are compared with some existing clustering techniques. The experimental evaluation results show that the performance of the proposed techniques is better than the existing techniques.


---

## Body

## 1. Introduction

Clustering, as a dimension reduction technique, is quite helpful in deciding the number and structure of the classes. These classes are suitable representatives of the data, which are internally maximally homogeneous. The mutually exclusive and collectively exhaustive classes are termed clusters. Clustering is particularly useful in exploratory data analysis for summarization and as a preprocessing step in complex data mining tasks. In general, an effective clustering scheme produces internally homogeneous but externally heterogeneous clusters of sufficiently large size without using any prior knowledge of data divisions. The produced clusters, therefore, may differ from any theoretical division already available for the data [1].A variety of clustering algorithms has been proposed in the literature. Clustering algorithms are typically categorized into partitioning methods, hierarchical methods, density-based methods, and grid-based methods [2, 3]. Most of the algorithms are developed for clustering of observations rather than dimensions or variables in a multivariate dataset. Our focus in this research is on the clustering of multivariate time series datasets, and we present two new approaches for such a clustering based on independent component analysis.The independent component analysis (ICA) has been used for clustering different kinds of data, e.g., in works by Keck et al. [4], Jamal and Kent [5], and Islam et al. [6]. The ICA is a statistical and computational technique in which the objective is to find a linear projection of the data in which the source signals or components are statistically independent or as independent as possible. Essentially, the ICA linearly transforms the data in a way that the resulting components can be grouped into clusters. Each component is dependent within a cluster and independent across clusters. Among its numerous applications, ICA is the most natural tool for blind source separation in instantaneous linear mixtures when the source signals are assumed to be independent. The main reason for the increased interest of researchers in ICA is mainly due to the plausibility of the statistical independence assumption in a wide variety of fields, including sales, finance, telecom, weather forecasting, and biomedical engineering.In this work, we propose two ICA-based approaches for variable clustering. ICA supports cluster identification by reducing the dimensionality of the data in a natural way. The first approach, the “Estimated Mixing Coefficients Approach,” is based on the sum of squares of mixing coefficients. The second approach, namely “RankedR¯2” uses the ranking pattern of R¯2 of the original and reconstructed series at predefined threshold levels. In order to validate the performance of our proposed techniques, we applied these approaches to a financial time series dataset with the objective of exploring the internal and external structures of identified clusters.Financial time series represent data on asset valuation as a function of time and usually include parameters, such as stock market index values, currency exchange rates, electricity prices, and interest rates. Data mining of financial time series has established very effective and useful results. Financial time series is affected by some underlying factors, such as news (good or bad), government interference, natural or artificial disasters, and political upheaval. These underlying factors affect the volatility of time series. Clustering could be very helpful in analyzing the time series of a group including several stocks. The analysis of the financial time series of a portfolio including several stocks, can be carried out by clustering the stocks. The performance of an investment portfolio is not necessarily determined by the stock that formulates the largest monetary share of the investment. ICA can be applied to discover the underlying or hidden components, factors (e.g., some good or bad news, government interference, any natural or man-made disasters, political disorder, and response to massive trading), and to remove any noise.The performance of the proposed approaches is compared with two existing approaches, namely Ward’s method and the average linkage method. The supremacy of the proposed approaches is confirmed by the findings of comparative evaluation.The primary contributions of this paper are as follows:(i)
Development of two new approaches for clustering based on ICA, namely the estimated mixing coefficients-based approach and the rankedR¯2-based approach(ii)
Experimental validation of the effectiveness of proposed approaches by application on a financial time series dataset for clustering of stock returns(iii)
Finding interpretable factors for stock returns in terms of ICsNow, we discuss the notation used in the paper and formally describe the basic ICA model.Consider a multivariate time series,xit=x1t,x2t,…,xrt with r random variables at some time point t, modeled as linear combinations of m random variables s1t,s2t,…,smt given by the following:(1)xit=wi1s1t+wi2s2t+…+wimsmt,∀i=1,2,…,r,m≤r.With eachwij:i=1,…,r and j=1,…,m being some real unknown parameter.By definition eachsit are statistically mutually independent and nonGaussian distributed components.Using vector-matrix notation equation (1) can simply be written as follows:(2)X=WS,where X is an r×t matrix of observations, W is an r×m matrix of unknown parameters and is called the “Mixing Matrix,” and S is an m×t matrix of nonGaussian and mutually independent hidden components called independent components (ICs).The main objective of ICA is to estimate from the given sample of observationsX, the mixing matrix W as well as the independent components, S. Thus, ICA attempts to find a linear transformation of the data as follows:(3)S^=AX,where a demixing matrix A of size r×m is to be identified such that the components (rows) of S^ become as independent of each other as possible. Principal components analysis (PCA) has been a very common practice for identifying clusters in multivariate data over the past more than two decades. There is also some work on clustering using ICA or hybrid approaches where ICs are computed after applying PCA. For example, Reza et al. [7] proposed an approach to identify clusters through PCs, ICs, and ICs after PCs. Islam et al. [6] compared clusters formed by ICs, PCs, and ICs after PCs using four simulated datasets and three real-life datasets.Bach and Jordan [8] proposed an approach where a transformation was searched to fit the estimated sources to a forest-structured graphical model. The optimal transformation for the nonGaussian temporally independent case was obtained by a mutual information-based contrast function. That mutual information-based contrast function extends the contrast function used for the classical ICA.Keck et al. [9] proposed an algorithm to cluster signals using the incomplete ICA. In this approach, first, the ICA is applied to the dataset without reducing the dimensions; then, in the second step, dimension reduction is performed for clustering using similarity in elements of the mixing matrix.Keck et al. [10] employed the ICA to identify clusters from functional magnetic resonance imaging (fMRI) data. The idea is to identify clusters by comparing the ICs computed at different levels of reduced dimensions. First, a set of ICs is computed without reducing the dimensions of the data. In the next iteration, the second set of ICs is computed from the dataset with reduced dimensions. The approach employs PCA for dimension reduction. After comparing the results of each iteration, matching ICs are retained to form clusters.In another work on multivariate time series clustering, Wu and Yu [11] first employ the FastICA algorithm to transform multivariate time series into ICs and then select the dominant ICs (based upon loadings). Clusters are then identified based on the similarity of the dominant ICs. We also use FastICA in addition to other algorithms for estimating the ICs and computing the mixing matrix. However, our approach can use any efficient ICA algorithm.Based on the fact that departure from Gaussianity helps in calculating ICs, some attempts have also been made to reduce Gaussianity as much as possible. This departure from Gaussianity is common in real-life situations. Inducing nonGaussianity can maximize the absolute kurtosis, which leads to some approaches that move in the positive or negative direction of kurtosis to attain sub-Gaussianity or super-Gaussianity. If the distribution is super-Gaussian, then it is least likely to have more than one mode located, whereas sub-Gaussianity increases the chances of having more than one mode identified. Jamal and Kent [5] proposed a clustering technique based on the fact that the clusters are formed when kurtosis is usually negative, i.e., the distribution is sub-Gaussian. Using the sub-ICA algorithm, ICs can be obtained by minimizing kurtosis and increasing the chances of locating modes. The one-dimensional projection of the so-calculated ICs would suggest modes, and each mode will center a cluster. This is how clusters are formed in this approach.Lu and Chang [12] proposed a hybrid sales forecasting scheme by combining the ICA, K-means clustering, and support vector regression (SVR). The proposed scheme first applies ICA to extract hidden information from the observed sales data. In the next step, the K-means clustering algorithm is applied to extracted features. The SVR forecasting models are applied as the last step to each group to generate final forecasting results. The proposed approach provides forecasting models based on ICA and k-means clustering.Azam and Bouguila [13] proposed a speaker classification method based on supervised hierarchical clustering. A bounded generalized Gaussian mixture model with the ICA is used for statistical learning with some modifications in the clustering framework. Using the training data, the ICA mixture model is learned, and posterior probability is used to divide the training data into clusters. The researchers proposed a supervised hierarchical clustering approach, which could be a complex procedure because supervised learning is more complex as compared to unsupervised learning.Nascimento et al. [14] proposed an ICA-based clustering approach, namely ICAclust, to cluster gene expression data. It is a two-step clustering method that relies upon ICA and a hierarchical method for clustering at the same time. The performance of the ICA-based clustering was compared with k-means clustering. Overall their proposed method performed better than the k-means clustering method, but it was also observed that it performed better for the small number of temporal observations.Gultepe & Makrehchi [15] used K-means, spectral clustering, graph regularized non-negative matrix factorization, and K-means with principal components analysis algorithms. They applied blind source separation (BSS) using the ICA were used for each clustering algorithm. They evaluated the performance of their proposed method using six benchmark datasets, which include five image datasets used in object, face, digit recognition tasks, and one text document dataset used in topic recognition. It was concluded that maximum clustering performance in four out of six datasets was achieved by applying ICA BSS after the initial matrix factorization step. The main drawback of this approach is the processing speed of the similarity graph and the matrix factorization due to the initial eigendecomposition.Durieux and Wilderjans [16] worked on three-way fMRI data. They proposed a two-step procedure. In the first step, the ICA was applied to extract functional connectivity patterns from the data, and in the second step, a clustering algorithm was applied to identify the clusters of patients with similar functional connectivity patterns. The approach suffers from a model selection problem. While conducting the simulation study, the true number of clusters was assumed, and for reduction using the ICA or PCA, the true number of components for the original data was known. Furthermore, the number of components for each patient’s fMRI data was assumed to be the same for every patient. The optimal number of cluster components in the empirical application for a dataset is not known a priori and has to be determined by the researcher. Incorrect specification of the true number of components may negatively affect the identification of the true cluster organization for a given dataset.Shahina and Kumar [17] proposed a clustering approach based on similarity, which grouped the sensor node with similar data as a cluster for combining data. After that, an algorithm is proposed which combines the data making use of ICA, which is applied on cluster head sensor nodes. Data combining procedure was implemented on clusters having similarity of data. The study did not gain much as a very slight improvement of results is achieved in terms of aggregation ratio when compared with existing systems of self-organizing map (SOM) and PCA-based aggregation.Boonyakitanont et al. [18] presented a work that performs subject group identification, latent source magnetoencephalography (MEG) estimation, and discriminatory source visualization. They applied hierarchical clustering on principal components (HCPCs) to identify cluster subject groups, which were based upon cognitive scores, and the ICA was implemented on MEG-evoked responses in such a way that not only higher-order statistics but also sample dependence within sources was considered. The proposed approach is specific to identifying the clusters for MEG data.Most of the existing ICA-based clustering techniques available in the literature are based upon loadings or estimated mixing coefficients of dominant ICs alone and do not take remaining loadings into account. The main disadvantage of choosing only the dominant components is that the remaining components often include some important information that is lost. Our proposed techniques are also built over the ICA. In our first estimated mixing coefficients approach, we utilize the information provided by all the ICs. In the second rankedR¯2 approach, we reconstruct the original series using dominant ICs only. The evaluation results show that considering all ICs significantly improves the clustering results.After providing some background definitions, a formal statement of the problem, and a brief review of the related literature, the rest of the paper is organized as follows. Section2 presents two new approaches to cluster the stock data. The application of the proposed approaches is presented in Section 3. Section 4 provides the analysis of the identified clusters. Finally, Section 5 concludes the paper.


## 2. The Proposed Clustering Approaches

In this section, we discuss in detail the two ICA-based approaches we have developed for clustering. The first approach utilizes all of the mixing coefficients and the second one is based on the reconstruction of the series with dominant ICs. The computation of ICs is the first step for both proposed approaches.


### 2.1. Computation of ICs

Many approaches exist in the literature for estimating ICs and the mixing matrix, including maximization of nonGaussianity, information theoretic measures, maximum likelihood estimation method, and tensor-based methods. In this work, we make use of three prominent algorithms proposed for specific applications to financial data [19, 20] including JADE, SOBI, and FastICA, for a comparative assessment. We have briefly discussed these algorithms in the article.


#### 2.1.1. Joint Approximation Diagonalization of Eigenmatrices Algorithm (JADE)

JADE [21] was developed following the seminal work of Back & Weigend [19], who first proposed ICA for exploring the structure of stock returns. JADE is based on higher-order statistics. Higher-order statistics-based algorithms rely on the characteristics of the data distribution to perform the separation. This makes such algorithms robust to additive Gaussian noise. The rule working behind the algorithm is the solution to the problem of equal eigenvalues of the cumulant tensor. The main quality of JADE is its computational efficiency for blind estimation of directional vectors, which is based on joint diagonalization of fourth-order cumulant matrices.


#### 2.1.2. Fixed-Point Algorithm (FastICA)

FastICA [22, 23] is also a higher-order statistic-based algorithm. It makes use of kurtosis for the estimation of ICs. Data whitening is a pre-processing step for the algorithm. Mainly, FastICA works on the principle of the maximization of nonGaussianity to obtain independence. FastICA is known to be computationally very efficient with parallel implementations. However, the main drawback of FastICA is the loss of temporal information and higher memory requirements in the case of nonparallel implementations.


#### 2.1.3. Second-Order Blind Identification Algorithm (SOBI)

SOBI [24] is based on second-order statistics. SOBI is a three-step algorithm that makes use of time-frequency information for decomposition. In the first step, data whitening is performed; in the next step, lagged correlation matrices are computed; and in the third step, blind source separation is performed by approximate joint diagonalization of time-delayed covariance matrices.Table1 summarizes the main features of the above three algorithms.Table 1
Main features of the JADE, FastICA, and SOBI.
CategoryMethodAlgorithmsProsConsHigher order statistic-based approachNonGaussianityJADE(i) Computationally efficient on the low dimensional datasets in terms of running time requirements(i) Does not consider the temporal characteristics of the dataset(ii) Stable in terms of memory space requirements(ii) Inefficient for high dimensional datasets in terms of computational speed.FastICA(i) Computationally efficient in terms of running time(i) Does not consider the temporal characteristics of the dataset(ii) Capability for parallel implementation(ii) Not robust when criteria for nonGaussianity measurement is kurtosis(iii) Higher memory space requirementsSecond-order statistic-based approachTemporal dependenceSOBI(i) Time-delayed covariance matrices of estimated independent components are closest to the diagonalDoes not consider the selection of auto-covariance order(ii) Computationally efficient both in terms of memory space and execution time


### 2.2. Estimated Mixing Coefficients Approach: The First Approach

Our first approach utilizes all the mixing coefficients. Basically, this approach is based upon the reconstruction of variables with reduced dimensions and concentrates on the comparison of the ICs themselves. Algorithm 1 outlines the basic steps in our approach.Algorithm 1: EMC_clustering_algorithm.
Input:X: r×t matrix of observations, k: number of clustersOutput:C: clusters(1)
S=computeICsX;/∗ execute ICA algorithm to generate ICs ∗//∗compute estimated mixing matrix W and the corresponding demixing matrix A∗/(2)
W=computeMixingMatrixX;(3)
A=inverseMatrixW;/∗compute sum of squares of mixing coefficients in A∗/(4)
fori=1 to m(5)
fork=1 to r(6)
sumVecti=sumVecti+squareOfAik(7)
end for(8)
end for(9)
sortAscending(sumVect);/∗ cluster the ordered rows in sumVect into k clusters, using an arbitrary clustering scheme ∗/(10)
C=performClusteringsumVect,kIn the first step (line 1 of Algorithm 1), we compute the ICs for the given input series given as anr×t matrix X using any ICA algorithm, such as FastICA, JADE, or SOBI. Let, the matrix of ICs be given by the following:(4)S=sit=s11s12…s1,Ts21s22…s2,T⋮⋮⋱⋮sm,1sm,2…sm,T=s1s2⋮sm.In the second step (lines 2 and 3 of Algorithm 1), we compute the estimated mixing matrixW and the corresponding separating matrix A given as follows:(5)W=wik=w1,1w1,2w2,1w2,2⋯w1,r⋯w2,r⋮⋮wm,1wm,2⋱⋮⋯wm,r=w1w2⋮wm,(6)A=W−1=aik=a1,1a1,2a2,1a2,2⋯a1,r⋯a2,r⋮⋮am,1am,2⋱⋮⋯am,r=a1a2⋮am.As discussed by Back & Weigend [19]; for A=W−1, we have three basic assumptions: (i) all sources sit are statistically independent, (ii) at most one source has a Gaussian distribution, and (iii) the observations are stationary.Note that when the ICA is applied for dimension reduction, one main issue is how to rank or order the ICs and rows of the mixing matrix in terms of significance to select dominant components. Our approach for obtaining such an ordering of ICs is to use the sum of squares of mixing coefficients and reordering the rows in the obtained sum of squares vector in ascending order.Therefore, the next step in our approach is to compute the sum of squares of mixing coefficients in matrixA. For each row ai of A, we compute the sum of squares, ∑k=1raik2 (lines 4–8 of Algorithm 1).Finally, we partition the ordered rows obtained in the previous step intok equal sized clusters (line 8 of Algorithm 1). Several criteria are available in the literature to determine a reasonable k for clustering. We follow the criterion given by Mardia et al. [25]; i.e., k≈m/2, where k is the number of clusters and m is the number of objects/variables. Any robust criterion for determining the value of k may be adopted.


### 2.3. RankedR¯2 Approach: The Second Approach

The key idea behind our second approach is to compare the reconstruction of the original variables at different threshold levels of dimension reduction.The step-by-step procedural details of this approach are discussed as follows:(1)
Similar to our first approach, perform the ICA for the input series, given as anr×t matrix X to obtain the matrix of ICs as given by equation (4). Then compute the mixing and separating matrices, W and A, respectively.(2)
Arrange the computed ICs in an appropriate order. For this, we apply a regression-based method proposed by Afzal and Iqbal [26]. Given the m independent components and the mixing matrix, each row i=1,…,m in the original series is regressed on all m independent components (here, we have r=m) to obtain all regression coefficients except the intercept.(i)
Using the corresponding mixing matrix row for theith original series used above, rank 1 is assigned to an element of ith row of the mixing matrix whose magnitude is closest to the magnitude of the first regression coefficient. The pair of the regression coefficient and the element of the ith row of the mixing matrix, which has just been assigned rank 1 are set aside.(ii)
Similarly, rank 2 is assigned to the element of theith row of the mixing matrix whose magnitude is closest to the magnitude of the second regression coefficient. The second pair of regression coefficients and the element of the ith row of the mixing matrix just ranked 2 are set aside.(iii)
The procedure is repeated till all the elements of theith row of the mixing matrix are ranked.Assigned ranks are then used to arrange the corresponding ICs. The ICs matrix with ordered rows using the above process is given by(7)S′=sit′=s11′s12′…s1,t′s21′s22′…s2,t′⋮⋮⋱⋮sm,1′sm,2′…sm,t′=s1′Ts2′T⋮st′T.The mixing matrix with ordered rows is given by(8)A′=aik′=a11′a12′…a1,m′a21′a22′…a2,m′⋮⋮⋱⋮am,1′am,2′…am,m′=a′1Ta′2T⋮a′mT.(3)
Reconstruct the series by using the Back and Weigend [19] procedure, and do the reconstruction of each of the series at different arbitrary threshold levels (say p). Weighted ICs and threshold ICs are computed to reconstruct the series.The matrixW′ of weighted ICs is computed by using the procedure followed by Back and Weigend [19]. The elements of the ith row are used as weights to compute weighted ICs. For the ith variable the weighted ICs are computed by multiplying ai1′ (which is a scalar quantity) to s1′ vector, ai2′ to s2′ and so on. The matrix of weighted ICs is given as follows:(9)W′=aik′skt′=ai1′s11′ai1′s12′…ai1′s1,t′ai2′s21′ai2′s22′…ai2′s2,t′⋮⋮⋱⋮aim′sm1′aim′sm2′…ai,m′sm,t′.The threshold ICs are also computed by following Back and Weigend [19]. An arbitrary threshold level is used here. The m−l components from bottom are excluded where m is the total number of components and l is the number of components to be retained.For theith variable, and at time t, the threshold IC is computed as x^it=∑k=1laik′skt′. (Here x^it is an estimated value of xit).(4)
Use the original series as original data points and reconstructed series as fitted points and their difference as an error. Note that we need to summarize how close the original and reconstructed series are.(5)
Compare each of the reconstructed series with the original series and compute the adjusted coefficient of determinationR¯2(6)
For each of the given series,p values of R¯2s are available. Rank these p values of R¯2 in ascending order for each variable.(7)
Check the ranking patterns of all variables to find similarities. Form clusters of the variables with similar ranking patterns. This automatically defines the number as well as the internal structure of the cluster.Algorithm 2 outlines our rankedR¯2 approach.Algorithm 2: RR2_clustering_algorithm.
Input:X: r×t matrix of observationsOutput:C: clusters(1)
S=computeICsX;/∗ execute ICA algorithm to generate ICs ∗//∗compute estimated mixing matrix W and the corresponding de-mixing matrix A∗/(2)
W=computeMixingMatrixX;(3)
A=inverseMatrixW;/∗ determine a ranking of ICs in S∗/(4)
S′=orderICsS;(5)
Perform reconstruction of the original series at arbitrary threshold levelsP=pi:i=1tor(6)
Compare each of the reconstructed series with original series and compute adjusted coefficient of determinationR¯2(7)
Rank thep values ∈P of computed R¯2s in ascending order for each variable in X./∗Perform clustering of variables based on similar ranking patterns.∗/(8)
C=performClusteringX,P;


## 2.1. Computation of ICs

Many approaches exist in the literature for estimating ICs and the mixing matrix, including maximization of nonGaussianity, information theoretic measures, maximum likelihood estimation method, and tensor-based methods. In this work, we make use of three prominent algorithms proposed for specific applications to financial data [19, 20] including JADE, SOBI, and FastICA, for a comparative assessment. We have briefly discussed these algorithms in the article.


### 2.1.1. Joint Approximation Diagonalization of Eigenmatrices Algorithm (JADE)

JADE [21] was developed following the seminal work of Back & Weigend [19], who first proposed ICA for exploring the structure of stock returns. JADE is based on higher-order statistics. Higher-order statistics-based algorithms rely on the characteristics of the data distribution to perform the separation. This makes such algorithms robust to additive Gaussian noise. The rule working behind the algorithm is the solution to the problem of equal eigenvalues of the cumulant tensor. The main quality of JADE is its computational efficiency for blind estimation of directional vectors, which is based on joint diagonalization of fourth-order cumulant matrices.


### 2.1.2. Fixed-Point Algorithm (FastICA)

FastICA [22, 23] is also a higher-order statistic-based algorithm. It makes use of kurtosis for the estimation of ICs. Data whitening is a pre-processing step for the algorithm. Mainly, FastICA works on the principle of the maximization of nonGaussianity to obtain independence. FastICA is known to be computationally very efficient with parallel implementations. However, the main drawback of FastICA is the loss of temporal information and higher memory requirements in the case of nonparallel implementations.


### 2.1.3. Second-Order Blind Identification Algorithm (SOBI)

SOBI [24] is based on second-order statistics. SOBI is a three-step algorithm that makes use of time-frequency information for decomposition. In the first step, data whitening is performed; in the next step, lagged correlation matrices are computed; and in the third step, blind source separation is performed by approximate joint diagonalization of time-delayed covariance matrices.Table1 summarizes the main features of the above three algorithms.Table 1
Main features of the JADE, FastICA, and SOBI.
CategoryMethodAlgorithmsProsConsHigher order statistic-based approachNonGaussianityJADE(i) Computationally efficient on the low dimensional datasets in terms of running time requirements(i) Does not consider the temporal characteristics of the dataset(ii) Stable in terms of memory space requirements(ii) Inefficient for high dimensional datasets in terms of computational speed.FastICA(i) Computationally efficient in terms of running time(i) Does not consider the temporal characteristics of the dataset(ii) Capability for parallel implementation(ii) Not robust when criteria for nonGaussianity measurement is kurtosis(iii) Higher memory space requirementsSecond-order statistic-based approachTemporal dependenceSOBI(i) Time-delayed covariance matrices of estimated independent components are closest to the diagonalDoes not consider the selection of auto-covariance order(ii) Computationally efficient both in terms of memory space and execution time


## 2.1.1. Joint Approximation Diagonalization of Eigenmatrices Algorithm (JADE)

JADE [21] was developed following the seminal work of Back & Weigend [19], who first proposed ICA for exploring the structure of stock returns. JADE is based on higher-order statistics. Higher-order statistics-based algorithms rely on the characteristics of the data distribution to perform the separation. This makes such algorithms robust to additive Gaussian noise. The rule working behind the algorithm is the solution to the problem of equal eigenvalues of the cumulant tensor. The main quality of JADE is its computational efficiency for blind estimation of directional vectors, which is based on joint diagonalization of fourth-order cumulant matrices.


## 2.1.2. Fixed-Point Algorithm (FastICA)

FastICA [22, 23] is also a higher-order statistic-based algorithm. It makes use of kurtosis for the estimation of ICs. Data whitening is a pre-processing step for the algorithm. Mainly, FastICA works on the principle of the maximization of nonGaussianity to obtain independence. FastICA is known to be computationally very efficient with parallel implementations. However, the main drawback of FastICA is the loss of temporal information and higher memory requirements in the case of nonparallel implementations.


## 2.1.3. Second-Order Blind Identification Algorithm (SOBI)

SOBI [24] is based on second-order statistics. SOBI is a three-step algorithm that makes use of time-frequency information for decomposition. In the first step, data whitening is performed; in the next step, lagged correlation matrices are computed; and in the third step, blind source separation is performed by approximate joint diagonalization of time-delayed covariance matrices.Table1 summarizes the main features of the above three algorithms.Table 1
Main features of the JADE, FastICA, and SOBI.
CategoryMethodAlgorithmsProsConsHigher order statistic-based approachNonGaussianityJADE(i) Computationally efficient on the low dimensional datasets in terms of running time requirements(i) Does not consider the temporal characteristics of the dataset(ii) Stable in terms of memory space requirements(ii) Inefficient for high dimensional datasets in terms of computational speed.FastICA(i) Computationally efficient in terms of running time(i) Does not consider the temporal characteristics of the dataset(ii) Capability for parallel implementation(ii) Not robust when criteria for nonGaussianity measurement is kurtosis(iii) Higher memory space requirementsSecond-order statistic-based approachTemporal dependenceSOBI(i) Time-delayed covariance matrices of estimated independent components are closest to the diagonalDoes not consider the selection of auto-covariance order(ii) Computationally efficient both in terms of memory space and execution time


## 2.2. Estimated Mixing Coefficients Approach: The First Approach

Our first approach utilizes all the mixing coefficients. Basically, this approach is based upon the reconstruction of variables with reduced dimensions and concentrates on the comparison of the ICs themselves. Algorithm 1 outlines the basic steps in our approach.Algorithm 1: EMC_clustering_algorithm.
Input:X: r×t matrix of observations, k: number of clustersOutput:C: clusters(1)
S=computeICsX;/∗ execute ICA algorithm to generate ICs ∗//∗compute estimated mixing matrix W and the corresponding demixing matrix A∗/(2)
W=computeMixingMatrixX;(3)
A=inverseMatrixW;/∗compute sum of squares of mixing coefficients in A∗/(4)
fori=1 to m(5)
fork=1 to r(6)
sumVecti=sumVecti+squareOfAik(7)
end for(8)
end for(9)
sortAscending(sumVect);/∗ cluster the ordered rows in sumVect into k clusters, using an arbitrary clustering scheme ∗/(10)
C=performClusteringsumVect,kIn the first step (line 1 of Algorithm 1), we compute the ICs for the given input series given as anr×t matrix X using any ICA algorithm, such as FastICA, JADE, or SOBI. Let, the matrix of ICs be given by the following:(4)S=sit=s11s12…s1,Ts21s22…s2,T⋮⋮⋱⋮sm,1sm,2…sm,T=s1s2⋮sm.In the second step (lines 2 and 3 of Algorithm 1), we compute the estimated mixing matrixW and the corresponding separating matrix A given as follows:(5)W=wik=w1,1w1,2w2,1w2,2⋯w1,r⋯w2,r⋮⋮wm,1wm,2⋱⋮⋯wm,r=w1w2⋮wm,(6)A=W−1=aik=a1,1a1,2a2,1a2,2⋯a1,r⋯a2,r⋮⋮am,1am,2⋱⋮⋯am,r=a1a2⋮am.As discussed by Back & Weigend [19]; for A=W−1, we have three basic assumptions: (i) all sources sit are statistically independent, (ii) at most one source has a Gaussian distribution, and (iii) the observations are stationary.Note that when the ICA is applied for dimension reduction, one main issue is how to rank or order the ICs and rows of the mixing matrix in terms of significance to select dominant components. Our approach for obtaining such an ordering of ICs is to use the sum of squares of mixing coefficients and reordering the rows in the obtained sum of squares vector in ascending order.Therefore, the next step in our approach is to compute the sum of squares of mixing coefficients in matrixA. For each row ai of A, we compute the sum of squares, ∑k=1raik2 (lines 4–8 of Algorithm 1).Finally, we partition the ordered rows obtained in the previous step intok equal sized clusters (line 8 of Algorithm 1). Several criteria are available in the literature to determine a reasonable k for clustering. We follow the criterion given by Mardia et al. [25]; i.e., k≈m/2, where k is the number of clusters and m is the number of objects/variables. Any robust criterion for determining the value of k may be adopted.


## 2.3. RankedR¯2 Approach: The Second Approach

The key idea behind our second approach is to compare the reconstruction of the original variables at different threshold levels of dimension reduction.The step-by-step procedural details of this approach are discussed as follows:(1)
Similar to our first approach, perform the ICA for the input series, given as anr×t matrix X to obtain the matrix of ICs as given by equation (4). Then compute the mixing and separating matrices, W and A, respectively.(2)
Arrange the computed ICs in an appropriate order. For this, we apply a regression-based method proposed by Afzal and Iqbal [26]. Given the m independent components and the mixing matrix, each row i=1,…,m in the original series is regressed on all m independent components (here, we have r=m) to obtain all regression coefficients except the intercept.(i)
Using the corresponding mixing matrix row for theith original series used above, rank 1 is assigned to an element of ith row of the mixing matrix whose magnitude is closest to the magnitude of the first regression coefficient. The pair of the regression coefficient and the element of the ith row of the mixing matrix, which has just been assigned rank 1 are set aside.(ii)
Similarly, rank 2 is assigned to the element of theith row of the mixing matrix whose magnitude is closest to the magnitude of the second regression coefficient. The second pair of regression coefficients and the element of the ith row of the mixing matrix just ranked 2 are set aside.(iii)
The procedure is repeated till all the elements of theith row of the mixing matrix are ranked.Assigned ranks are then used to arrange the corresponding ICs. The ICs matrix with ordered rows using the above process is given by(7)S′=sit′=s11′s12′…s1,t′s21′s22′…s2,t′⋮⋮⋱⋮sm,1′sm,2′…sm,t′=s1′Ts2′T⋮st′T.The mixing matrix with ordered rows is given by(8)A′=aik′=a11′a12′…a1,m′a21′a22′…a2,m′⋮⋮⋱⋮am,1′am,2′…am,m′=a′1Ta′2T⋮a′mT.(3)
Reconstruct the series by using the Back and Weigend [19] procedure, and do the reconstruction of each of the series at different arbitrary threshold levels (say p). Weighted ICs and threshold ICs are computed to reconstruct the series.The matrixW′ of weighted ICs is computed by using the procedure followed by Back and Weigend [19]. The elements of the ith row are used as weights to compute weighted ICs. For the ith variable the weighted ICs are computed by multiplying ai1′ (which is a scalar quantity) to s1′ vector, ai2′ to s2′ and so on. The matrix of weighted ICs is given as follows:(9)W′=aik′skt′=ai1′s11′ai1′s12′…ai1′s1,t′ai2′s21′ai2′s22′…ai2′s2,t′⋮⋮⋱⋮aim′sm1′aim′sm2′…ai,m′sm,t′.The threshold ICs are also computed by following Back and Weigend [19]. An arbitrary threshold level is used here. The m−l components from bottom are excluded where m is the total number of components and l is the number of components to be retained.For theith variable, and at time t, the threshold IC is computed as x^it=∑k=1laik′skt′. (Here x^it is an estimated value of xit).(4)
Use the original series as original data points and reconstructed series as fitted points and their difference as an error. Note that we need to summarize how close the original and reconstructed series are.(5)
Compare each of the reconstructed series with the original series and compute the adjusted coefficient of determinationR¯2(6)
For each of the given series,p values of R¯2s are available. Rank these p values of R¯2 in ascending order for each variable.(7)
Check the ranking patterns of all variables to find similarities. Form clusters of the variables with similar ranking patterns. This automatically defines the number as well as the internal structure of the cluster.Algorithm 2 outlines our rankedR¯2 approach.Algorithm 2: RR2_clustering_algorithm.
Input:X: r×t matrix of observationsOutput:C: clusters(1)
S=computeICsX;/∗ execute ICA algorithm to generate ICs ∗//∗compute estimated mixing matrix W and the corresponding de-mixing matrix A∗/(2)
W=computeMixingMatrixX;(3)
A=inverseMatrixW;/∗ determine a ranking of ICs in S∗/(4)
S′=orderICsS;(5)
Perform reconstruction of the original series at arbitrary threshold levelsP=pi:i=1tor(6)
Compare each of the reconstructed series with original series and compute adjusted coefficient of determinationR¯2(7)
Rank thep values ∈P of computed R¯2s in ascending order for each variable in X./∗Perform clustering of variables based on similar ranking patterns.∗/(8)
C=performClusteringX,P;


## 3. Application of the Proposed Approaches

We apply the ICA for analyzing financial time series data of the Karachi Stock Exchange 100 Index (KSE-100 index) in order to measure the effectiveness of the proposed methods for clustering variables. An effective time-series clustering can be achieved if and only if the price fluctuations of stocks within a group or cluster are maximally correlated, but the price fluctuations of stocks between different groups are uncorrelated [1]. This is the key assumption that forms the basis of clustering stocks data.The KSE-100 index is a benchmark for comparing stock price performance in Pakistan over a period of time. The dataset covers the daily closing rates of 161 companies of KSE for the period of June 11, 2004, to February 15, 2012. Each of the 161 companies consists of 2004 observations . Rates for the closed market days (other than Saturday and Sunday) are taken on the basis of the last day’s closing rates.Let the matrix of closing rates of 161 companies at 2004 time points be given by(10)Y=yit=y10y11…y1,2003y20y21…y2,2003⋮⋮⋮⋮y161,0y161,1…y161,2003=y1Ty2T⋮y161T.Each value ofyit and yit−1 denote the closing rates of ith company’s stock for two sequential days in the market.


### 3.1. Preprocessing

Stationarity is a standard requirement for most modeling approaches including the ICA. Note that stationary signals have a constant expected value which is not the case with stock prices. Therefore, we first convert the nonstationary stock prices, i.e., the closing rates,yit (where i=1,2,…161 and t=1,2,…,2003) to stock returns. This is typically accomplished by taking the difference between consecutive values of the stock prices as the change in stock prices is relatively higher over the years [19]. Therefore, we compute relative returns to obtain a transformed stationary series xit (where i=1,2,…,161 and t=1,2,…,2003) by describing geometric growth taking instead of additive for the sake of efficiency using equation (11) as follows:(11)xit=lnyityi,t−1.The matrix of transformed series (relative returns) is given by(12)X=xit=x11x12…x1,2003x21x22…x2,2003⋮⋮⋱⋮x161,1x161,2…x161,2003=x1Tx2T⋮x161T.


### 3.2. Application of the Estimated Mixing Coefficients Approach

The ICA is applied to 161 mixed signals, i.e., stock returns of companies, in this case, each having a sample size of 2003. As discussed earlier, this approach is based upon the sum of squares of mixing coefficients. Different algorithms to compute ICs produce a different matrix of mixing coefficients, but every algorithm produces the same sum of squares of mixing coefficients of rows; therefore, any of the algorithms discussed in Section4.1 can be used.Following the main assumption in the experimental setting for financial time series dataset analysis by Back and Weigend [19], we also assume that the number of mixed signals is equal to the number of source signals in all the experiments. Here, we have 161 mixed signals (companies), each having 2003 stock returns.Algorithm 1 is supplied with all 161 stocks as input. The ICA algorithm returns 161 source signals in the form of ICs. Matrix of estimated ICs,S, and the estimated mixing matrix, A, are obtained. The number of clusters is defined using the rule given by Mardia et al. [25]. In our case of the 161 stock companies’ dataset, this rule suggests over 9 clusters which are rounded to 10.The sum of squares∑k=1161aik2 for each row ai of A is computed. The rows denoting different companies are reordered in ascending order of the sum of squares. The ordered rows (companies) are divided into equal parts. The 161 companies are then divided into nine groups, each of size 16. The tenth group is of size 17. Each group is considered a cluster. Application of the estimated mixing coefficients approach on this dataset returns the clusters presented in Table 2.Table 2
The ICA-based clusters using the estimated mixing coefficients approach.
Cluster no.Cluster sizeCompanies116HUBC, AGTL, OGDC, GLAXO, FFC, KOHE, NESTLE, ABOT, SHEL, IBFL, SITC, PSO, PKGS, INDU, HINOON, and GHGL216FFBL, SIEM, PAKT, PTC, ENGRO, ICI, PSMC, AGIL, CPL, POL, ACPL, FEROZ, MTL, SEARL, JDWS, and LUCK316BAHL, MEBL, IDYM, NRL, KSBP, GTYR, PAKD, PRL, ATBA, MCB, SCM, CEPB, MUREB, NBP, FHAM, and SEL416PNSC, PICT, GADT, ATLH, HINO, GLPL, NML, SHFA, CSAP, PECO, DGKC, HABSM, BIFO, CLOV, FABL, and SAZEW516HICL, PCAL, MIRKS, OLPL, PAEL, CHCC, BWHL, ALNRS, ATRL, MARI, DYNO, INIL, FCCL, HSPI, EFUL, and BNWM616SING, ADOS, CRTM, CENI, BOP, NCL, KOHC, PIOC, HCAR, RICL, HAL, AGIC, BYCO, EFUG, FECTC, and MLCF716FHBM, KASBB, PIAA, GHNL, REWM, ANL, KESC, FCSC, DAWH, LOTPTA, SANSM, KTML, SEPCO, GASF, SHSML, and NIB816CHAS, JOPP, GWLC, FNBM, FDIBL, FUDLM, ADMM, JOVC, SGML, DSFL, ECOP, JPGL, NATF, DFML, PAKRI, and PSYL916NICL, KOHP, JSCL, IDRT, MACFL, SAIF, ESBL, PMI, EMCO, FFLM, DWSM, KOIL, FEM, BGL, DNCC, and MZSM1017PNGRS, PTEC, CPMFI, FECM, QUICE, FPJM, SIBL, RAVT, GENP, MODAM, FRCL, MFTM, HADC, PAKMI, COTT, AICL, and MUKT


### 3.3. Application of RankedR¯2 Approach

The rows of the matrix of ICs and mixing matrix, as given in equations (5) and (7), are ordered using the regression-based ordering method proposed by Afzal and Iqbal [26]. The original series are reconstructed with reduced dimensions following the Back and Weigend [19] procedure. The reconstruction of the original series is performed at nine different threshold levels, i.e., using 10, 20, 30, 40, 50, 60, 70, 80, and 90 percent of ICs for the purpose following Afzal et al. [27].LetY^ be the matrix of the reconstructed series of closing rates of 161 companies of KSE, each having 2004 observations, then for a given retention level using equation (11), we have the following relationship:(13)Y^=y^it−1⋅antilogx⌢it.Using this relationship,y^i0 is required to proceed any further, which is borrowed as the starting point from the original series.Each of the reconstructed series is then compared with the original series and using the original series, as original data points and reconstructed series as fitted points and their difference as error,R¯2 is calculated. Thus, nine values of R¯2 for every company are obtained. For a given company, say ABOT, nine values of R¯2s are available. These nine values of R¯2 are ranked in ascending order for each company. The ranking pattern of all the companies is checked to find similarities. Clusters of companies sharing similar patterns are formed. This automatically defined the number and size of clusters. The identified clusters based upon ranked R¯2 approach are presented in Tables 3–5 for JADE, FastICA, and SOBI algorithms, respectively. Twenty-two clusters were identified using JADE and FastICA each, and 10 clusters using the SOBI algorithm.Table 3
The ICA-based clusters using rankedR¯2 approach (case: JADE).
Cluster no.Cluster sizeCompanies134ACPL, ALNRS, BAHL, BNWM, BWHL, CHCC, DAWH, DGKC, DNCC, DSFL, EFUG, ESBL, FABL, GLAXO, GTYR, HINO, HINOON, HUBC, IDRT, JDWS, KESC, KOIL, LUCK, MARI, MFTM, MIRKS, MLCF, MODAM, MUKT, OGDC, PAEL, PIAA, SAIF, and SCM212ATBA, BGL, GADT, GLPL, JOPP, JSCL, KASBB, NESTLE, OLPL, PAKMI, PAKT, and PSO311AGIC, CPL, FPJM, IBFL, ICI, IDYM, PAKRI, PKGS, RICL, SAZEW, and SEPCO49ATLH, CLOV, EFUL, FDIBL, FUDLM, HICL, MUREB, NATF, and SING56AGIL, FCCL, GHNL, KOHE, PIOC, and SHFA66FHAM, KSBP, NCL, SHEL, SIEM, and SITC75GWLC, HSPI, PAKD, PCAL, and SIBL84ANL, FECTC, FNBM, and PRL93ABOT, BYCO, and MZSM103ADOS, ATRL, and ECOP113BIFO, MCB, and SANSM123CPMFI, FEROZ, and FHBM133ENGRO, FFC, and PNGRS143GASF, PNSC, and SEARL152ADMM, and CEPB162AICL and FFLM172CENI and FRCL182CHAS and CRTM192CSAP and DFML202FECM and PMI212FEM and NIB222PICT and RAVTTable 4
The ICA-based clusters using rankedR¯2 approach (case: FastICA).
Cluster no.Cluster sizeCompanies139AGIC, AGTL, ANL, BNWM, CHCC, COTT, DNCC, DWSM, EFUL, ESBL, FABL, FCCL, FCSC, FECTC, FFLM, FHBM, GHNL, HCAR, HICL, HINO, HINOON, INIL, JDWS, JSCL, KASBB, KOHC, LUCK, MARI, MTL, NATF, NBP, PAKRI, PIAA, PNGRS, POL, PRL, SAZEW, SEARL, and SIEM210ABOT, AICL, CLOV, DAWH, ICI, IDRT, MIRKS, NESTLE, OLPL, and PSMC38ATLH, ATRL, BWHL, DFML, GASF, MCB, MODAM, and PTEC47CPL, EFUG, FUDLM, JOVC, NCL, PCAL, and REWM56BAHL, KOHP, KTML, MACFL, SAIF, and SANSM65GWLC, FNBM, NICL, PAKD, and QUICE74CENI, KSBP, PICT, and SIBL84DYNO, ENGRO, IDYM, and SHFA94FDIBL, FEM, JOPP, and PAEL103DGKC, JPGL, and MUREB113FFBL, GLAXO, and HADC123HSPI, NIB, and RICL132ACPL and MEBL142AGIL and GLPL152BYCO and DSFL162CEPB and GTYR172CHAS and GHGL182FPJM and PMI192HAL and SEPCO202HUBC and PKGS212KESC and PSO222MLCF and SCMTable 5
The ICA-based clusters using rankedR¯2 approach (case: SOBI).
Cluster no.Cluster sizeCompanies15HUBC, PAKT, PSO, PSYL, and SITC24AGTL, FECTC, HSPI, and NRL33ATBA, DGKC, and ICI42AICL and POL52BOP and NML62BWHL and NATF72EMCO and SAIF82GLPL and NBP92MIRKS and SANSM102PECO and SAZEW


## 3.1. Preprocessing

Stationarity is a standard requirement for most modeling approaches including the ICA. Note that stationary signals have a constant expected value which is not the case with stock prices. Therefore, we first convert the nonstationary stock prices, i.e., the closing rates,yit (where i=1,2,…161 and t=1,2,…,2003) to stock returns. This is typically accomplished by taking the difference between consecutive values of the stock prices as the change in stock prices is relatively higher over the years [19]. Therefore, we compute relative returns to obtain a transformed stationary series xit (where i=1,2,…,161 and t=1,2,…,2003) by describing geometric growth taking instead of additive for the sake of efficiency using equation (11) as follows:(11)xit=lnyityi,t−1.The matrix of transformed series (relative returns) is given by(12)X=xit=x11x12…x1,2003x21x22…x2,2003⋮⋮⋱⋮x161,1x161,2…x161,2003=x1Tx2T⋮x161T.


## 3.2. Application of the Estimated Mixing Coefficients Approach

The ICA is applied to 161 mixed signals, i.e., stock returns of companies, in this case, each having a sample size of 2003. As discussed earlier, this approach is based upon the sum of squares of mixing coefficients. Different algorithms to compute ICs produce a different matrix of mixing coefficients, but every algorithm produces the same sum of squares of mixing coefficients of rows; therefore, any of the algorithms discussed in Section4.1 can be used.Following the main assumption in the experimental setting for financial time series dataset analysis by Back and Weigend [19], we also assume that the number of mixed signals is equal to the number of source signals in all the experiments. Here, we have 161 mixed signals (companies), each having 2003 stock returns.Algorithm 1 is supplied with all 161 stocks as input. The ICA algorithm returns 161 source signals in the form of ICs. Matrix of estimated ICs,S, and the estimated mixing matrix, A, are obtained. The number of clusters is defined using the rule given by Mardia et al. [25]. In our case of the 161 stock companies’ dataset, this rule suggests over 9 clusters which are rounded to 10.The sum of squares∑k=1161aik2 for each row ai of A is computed. The rows denoting different companies are reordered in ascending order of the sum of squares. The ordered rows (companies) are divided into equal parts. The 161 companies are then divided into nine groups, each of size 16. The tenth group is of size 17. Each group is considered a cluster. Application of the estimated mixing coefficients approach on this dataset returns the clusters presented in Table 2.Table 2
The ICA-based clusters using the estimated mixing coefficients approach.
Cluster no.Cluster sizeCompanies116HUBC, AGTL, OGDC, GLAXO, FFC, KOHE, NESTLE, ABOT, SHEL, IBFL, SITC, PSO, PKGS, INDU, HINOON, and GHGL216FFBL, SIEM, PAKT, PTC, ENGRO, ICI, PSMC, AGIL, CPL, POL, ACPL, FEROZ, MTL, SEARL, JDWS, and LUCK316BAHL, MEBL, IDYM, NRL, KSBP, GTYR, PAKD, PRL, ATBA, MCB, SCM, CEPB, MUREB, NBP, FHAM, and SEL416PNSC, PICT, GADT, ATLH, HINO, GLPL, NML, SHFA, CSAP, PECO, DGKC, HABSM, BIFO, CLOV, FABL, and SAZEW516HICL, PCAL, MIRKS, OLPL, PAEL, CHCC, BWHL, ALNRS, ATRL, MARI, DYNO, INIL, FCCL, HSPI, EFUL, and BNWM616SING, ADOS, CRTM, CENI, BOP, NCL, KOHC, PIOC, HCAR, RICL, HAL, AGIC, BYCO, EFUG, FECTC, and MLCF716FHBM, KASBB, PIAA, GHNL, REWM, ANL, KESC, FCSC, DAWH, LOTPTA, SANSM, KTML, SEPCO, GASF, SHSML, and NIB816CHAS, JOPP, GWLC, FNBM, FDIBL, FUDLM, ADMM, JOVC, SGML, DSFL, ECOP, JPGL, NATF, DFML, PAKRI, and PSYL916NICL, KOHP, JSCL, IDRT, MACFL, SAIF, ESBL, PMI, EMCO, FFLM, DWSM, KOIL, FEM, BGL, DNCC, and MZSM1017PNGRS, PTEC, CPMFI, FECM, QUICE, FPJM, SIBL, RAVT, GENP, MODAM, FRCL, MFTM, HADC, PAKMI, COTT, AICL, and MUKT


## 3.3. Application of RankedR¯2 Approach

The rows of the matrix of ICs and mixing matrix, as given in equations (5) and (7), are ordered using the regression-based ordering method proposed by Afzal and Iqbal [26]. The original series are reconstructed with reduced dimensions following the Back and Weigend [19] procedure. The reconstruction of the original series is performed at nine different threshold levels, i.e., using 10, 20, 30, 40, 50, 60, 70, 80, and 90 percent of ICs for the purpose following Afzal et al. [27].LetY^ be the matrix of the reconstructed series of closing rates of 161 companies of KSE, each having 2004 observations, then for a given retention level using equation (11), we have the following relationship:(13)Y^=y^it−1⋅antilogx⌢it.Using this relationship,y^i0 is required to proceed any further, which is borrowed as the starting point from the original series.Each of the reconstructed series is then compared with the original series and using the original series, as original data points and reconstructed series as fitted points and their difference as error,R¯2 is calculated. Thus, nine values of R¯2 for every company are obtained. For a given company, say ABOT, nine values of R¯2s are available. These nine values of R¯2 are ranked in ascending order for each company. The ranking pattern of all the companies is checked to find similarities. Clusters of companies sharing similar patterns are formed. This automatically defined the number and size of clusters. The identified clusters based upon ranked R¯2 approach are presented in Tables 3–5 for JADE, FastICA, and SOBI algorithms, respectively. Twenty-two clusters were identified using JADE and FastICA each, and 10 clusters using the SOBI algorithm.Table 3
The ICA-based clusters using rankedR¯2 approach (case: JADE).
Cluster no.Cluster sizeCompanies134ACPL, ALNRS, BAHL, BNWM, BWHL, CHCC, DAWH, DGKC, DNCC, DSFL, EFUG, ESBL, FABL, GLAXO, GTYR, HINO, HINOON, HUBC, IDRT, JDWS, KESC, KOIL, LUCK, MARI, MFTM, MIRKS, MLCF, MODAM, MUKT, OGDC, PAEL, PIAA, SAIF, and SCM212ATBA, BGL, GADT, GLPL, JOPP, JSCL, KASBB, NESTLE, OLPL, PAKMI, PAKT, and PSO311AGIC, CPL, FPJM, IBFL, ICI, IDYM, PAKRI, PKGS, RICL, SAZEW, and SEPCO49ATLH, CLOV, EFUL, FDIBL, FUDLM, HICL, MUREB, NATF, and SING56AGIL, FCCL, GHNL, KOHE, PIOC, and SHFA66FHAM, KSBP, NCL, SHEL, SIEM, and SITC75GWLC, HSPI, PAKD, PCAL, and SIBL84ANL, FECTC, FNBM, and PRL93ABOT, BYCO, and MZSM103ADOS, ATRL, and ECOP113BIFO, MCB, and SANSM123CPMFI, FEROZ, and FHBM133ENGRO, FFC, and PNGRS143GASF, PNSC, and SEARL152ADMM, and CEPB162AICL and FFLM172CENI and FRCL182CHAS and CRTM192CSAP and DFML202FECM and PMI212FEM and NIB222PICT and RAVTTable 4
The ICA-based clusters using rankedR¯2 approach (case: FastICA).
Cluster no.Cluster sizeCompanies139AGIC, AGTL, ANL, BNWM, CHCC, COTT, DNCC, DWSM, EFUL, ESBL, FABL, FCCL, FCSC, FECTC, FFLM, FHBM, GHNL, HCAR, HICL, HINO, HINOON, INIL, JDWS, JSCL, KASBB, KOHC, LUCK, MARI, MTL, NATF, NBP, PAKRI, PIAA, PNGRS, POL, PRL, SAZEW, SEARL, and SIEM210ABOT, AICL, CLOV, DAWH, ICI, IDRT, MIRKS, NESTLE, OLPL, and PSMC38ATLH, ATRL, BWHL, DFML, GASF, MCB, MODAM, and PTEC47CPL, EFUG, FUDLM, JOVC, NCL, PCAL, and REWM56BAHL, KOHP, KTML, MACFL, SAIF, and SANSM65GWLC, FNBM, NICL, PAKD, and QUICE74CENI, KSBP, PICT, and SIBL84DYNO, ENGRO, IDYM, and SHFA94FDIBL, FEM, JOPP, and PAEL103DGKC, JPGL, and MUREB113FFBL, GLAXO, and HADC123HSPI, NIB, and RICL132ACPL and MEBL142AGIL and GLPL152BYCO and DSFL162CEPB and GTYR172CHAS and GHGL182FPJM and PMI192HAL and SEPCO202HUBC and PKGS212KESC and PSO222MLCF and SCMTable 5
The ICA-based clusters using rankedR¯2 approach (case: SOBI).
Cluster no.Cluster sizeCompanies15HUBC, PAKT, PSO, PSYL, and SITC24AGTL, FECTC, HSPI, and NRL33ATBA, DGKC, and ICI42AICL and POL52BOP and NML62BWHL and NATF72EMCO and SAIF82GLPL and NBP92MIRKS and SANSM102PECO and SAZEW


## 4. Analysis of the Quality and Structure of Clusters

In this section, we analyze the internal structure of the clusters returned by the proposed approaches by first comparing them with the sectors defined by KSE, and then we check the validity of clusters, i.e., exploring them on their own.


### 4.1. Structural Comparison with Sectors Defined by KSE

The KSE has defined 33 sectors altogether based on the primary activities of listed companies. In this section, we compare the clusters returned by the proposed approaches with the sector-wise grouping provided by KSE. Table6 shows the grouping of the 161 companies in our dataset in these sectors.Table 6
Sector-wise list of the 161 KSE companies.
Sr. no.SectorCompaniesS1Oil and gasBYCO, MARI, NRL, OGDC, POL, PRL, PSO, and SHELS2ChemicalsCPL, DAWH, DSFL, DYNO, ENGRO, FFBL, FFC, ICI, LOTPTA, NICL, and SITCS3Forestry (paper and board)CEPBS4Industrial metals and miningHSPI and INILS5Construction and materials (cement)CHCC, DGKC, DNCC, EMCO, FCCL, FECTC, FRCL, GWLC, HADC, KOHC, LUCK, MLCF, and PIOCS6General industrialsGHGL, MACFL, PKGS, and SIEMS7Electronic and electrical goodsPCALS8EngineeringAGTL, HINO, KSBP, MTL, and PECOS9Industrial transportationPNSCS10Automobile and partsATBA, ATLH, BWHL, DFML, GHNL, GTYR, HCAR, INDU, PSMC, and SAZEWS11BeveragesMUREBS12Food producersCHAS, CLOV, DWSM, HABSM, HAL, JDWS, MIRKS, MZSM, NATF, NESTLE, PNGRS, QUICE, SANSM, SGML, and SHSMLS13Household goodsPAEL and SINGS14Personal goods (textile)ANL, BNWM, COTT, CRTM, GADT, GLPL, IBFL, IDRT, IDYM, KOIL, KTML, LOTPTA, MFTM, MUKT, NCL, NML, PSYL, RAVT, REWM, and SAIFS15TobaccoPAKTS16Healthcare equipment and servicesSHFAS17Pharma and biotechFEROZ, GLAXO, HINOON, and SEARLS18Travel and leisurePIAAS19Fixed line telecommunicationPTCS20ElectricityHUBC, JPGL, KESC, KOHE, KOHP, SEL, and SEPCOS21Commercial banksBOP, ESBL, FABL, FDIBL, KASBB, MCB, MEBL, NBP, and NIBS22Nonlife insuranceAICL, CENI, EFUG, HICL, PAKRI, and RICLS23Life insuranceEFULS24Financial servicesFCSC, FECM, JOVC, JSCL, OLPL, and SIBLS25Equity investment instrumentsFFLM, FHAM, FHBM, FPJM, FUDLM, GASF, FNBM, MODAM, PAKMI, PMI, and SCMS26Technology hardware and equipmentPTECIt can be viewed from Tables2–4 that five sectors (mentioned in Table 5) including commercial banks, nonlife insurance, life insurance, financial services, and equity investment instruments (S21, S22, S23, S24, and S25), are moving together and form a group which we term as “Money & Bank” group.


#### 4.1.1. Clustering by the Estimated Mixing Coefficients Approach

If individual clusters formed by the estimated mixing coefficients approach in Table2 are analyzed, then it is apparent that six companies from the oil and gas sector (S1) and pharma and biotech Sector (S17) are combined in Cluster 1. Cluster 2 has a majority of companies from chemicals sector. Cluster 3 has six companies from money and bank Group. About half of the companies from the industrial metal and mining sector are part of cluster 4. Four companies from the sector general industries are gathered in cluster 6. Cluster 7 is comprised of 5 companies from money and bank group. The remaining clusters do not exhibit any such pattern.


#### 4.1.2. Clustering by the RankedR¯2 Approach

The rankedR¯2 approach using the JADE algorithm returned 22 clusters, as shown in Table 3. The first cluster consists of 34 companies, of which six belong to the money and bank group, whereas one set of five are from the construction and material sector, another set of five are from the personal goods and textile sector, and the remaining 18 companies form smaller groups from other sectors such as food producers, automobile, and parts and chemicals. The first cluster thus takes the shape of a contrast where inversely related groups of sectors get put together, which negate each other in the sense that the positive behavior of the money and bank group, for example, causes a negative impact on the construction and material sector, that is people try to deposit money in the banks rather than consuming it on construction and material. The rationale visible in this cluster does not persist in the remaining clusters, so the argument cannot be forwarded ahead. Negation to the argument is quite obvious in the subsequent clusters, wherein in the second cluster, four out of twelve companies belong to money and bank group; two companies are from the automobile and parts sector. Similarly, four out of eleven companies in the third cluster are from money and bank group, and two companies are from the automobile and parts sector. In the fourth, cluster four out of nine companies belong to money and bank group. Two of them are from the food producers sector.The results in Table4 show that the calculation based upon FastICA produced similar results where, in the first cluster of size 39, ten companies belong to the money and bank group, six to the construction and materials sector, and three to the personal goods (textile) sector. The second cluster of 10 companies does not show a good internal structure as it includes three companies from the food producers sector and two from the chemical sector, whereas the remaining do not form any group. The third cluster is relatively smaller in size, where three companies are from the automobile and parts sector and two from money and bank group. Cluster 4 includes seven companies out of which three are from money and bank group and two from personal goods (textile) sector.Among the three algorithms, SOBI’s results presented in Table5 are the worst in the sense that too many clusters of relatively very small size are formed. The largest cluster identified contains only five companies. That is, the whole spirit of clustering is ruined.The comparison shows that the already defined sectors cannot be used as clusters. The discrepancy can be justified on the ground that the closing rates of the company do not follow a pattern governed by sectors rather they play their role independently. The stock market is based on perspicacity, which has become even more important in modern times because of online trading. Due to this, a lot of inexperienced day-traders have moved towards stock market trading; for example, HINO is classified by KSE in the engineering sector because it earns the largest portion of its revenue from this sector. If most of the investors recognize HINO as part of the automobile and parts sector then its price fluctuation will follow the behavior of the automobile and parts group. Clustering could also be very helpful in analyzing the time series of a group including several stocks. The behavior of an investment group is not necessarily determined by the stock that makes up the largest monetary share of the investment. Clustering the stock data could identify which groups have the greatest influence on the portfolio. It is difficult to identify the cluster of stocks as their appropriate sectors because of the uncertain behavior of some stocks. Similar results were presented by Wittman [28]. Thus clustering should not be confused with pre-defined grouping whatsoever.The next section concentrates on the exploration of the internal structure of clusters on their own.


### 4.2. Validity of Clusters

In this section, we present an evaluation of the quality of clusters identified by our proposed clustering techniques by comparing them with the quality of clusters identified using two of the most widely used clustering methods, including Ward’s method [29] and the average linkage method [30].The quality of clustering can be gauged by measuring the internal homogeneity and external heterogeneity of the clusters. The clustering of a financial time series can be considered credible only when the stock prices within a cluster are maximally correlated, but different clusters are minimally correlated [31].Various indices are available in the literature to determine the validity of identified clusters. Two of the popular and fundamental ones are given as follows:(i)
Calinski–Harabasz Index (CHI): Calinski and Harabasz [32] introduced this index to assess the quality of the clustering solution by analyzing the similarity of the objects within each cluster and the dissimilarity of different clusters. This index is also called the variance ratio criterion (VRC). The larger value of CHI indicates better data partition. The CH index for K number of clusters on a data set X=x1,x2,…xN is given as(14)CH=∑k=1Knkvk−v/K−1∑k=1Kxi−vk/N−k,where,nk and vk are the number of points and centroid of the kth cluster, respectively. v is the overall centroid and N is the total number of data points.(ii)
Davies–Bouldin Index (DBI): this index was introduced by Davies and Bouldin [33] and is based on the ratio of within-cluster-distance to between-cluster-distance. The lower value of the index indicates a better cluster structure. The DBI is calculated for K clusters as follows:(15)DB=1K∑k=1KRk,where,Rk=maxRkl;k,l=1,2,…,K,k≠l,Rkl=sk+sldkl,where,dkl=dvk,vl,si=1ck∑xεckdx,vk.dx,z is the Euclidean distance between xandz, ck is the kth cluster, vk is the kth cluster centroid, and ck refers to norm of ck.Clusters are also identified using hierarchical clustering methods. Only the average linkage method and Ward’s method performed well as other hierarchical methods identified a large number of clusters consisting of one or two members. Hence the two indices are calculated for the clusters formed by the two proposed approaches and two existing approaches, i.e., the average linkage method and Ward’s method to cluster variables. The results are presented in Table7. The relative position of each of the indexes is presented in parentheses for quick comparison.Table 7
Validity indices for estimated mixing coefficients and rankedR¯2 clustering.
Clustering approachValidity index (ranks)Calinski–HarabaszDavies–BouldinProposed techniquesEstimated mixing coefficients44.1597 (1)4.0552 (1)RankedR¯2JADE15.6017 (2)4.1257 (2)FastICA12 (3)4.2017 (3)SOBI11.5935 (4)4.2707 (4)Existing techniquesAverage linkage method10.2368 (6)5.3113 (6)Ward’s method14.9963 (5)5.2687 (5)As depicted in Table7, the performance of both proposed methods is better than the existing techniques. The performance of the estimated mixing coefficient approach is the best. As discussed earlier, the large value of the Calinski–Harabasz index and the small value of Davies–Bouldin are considered better. Both indices awarded rank 1 to the proposed estimated mixing coefficients approach. The performance of JADE is better for the ranked R¯2 approach. Results of FastICA with ranked R¯2 are on the third place, algorithm SOBI with the same approach are placed on number 4. Among the existing approaches, the results of Ward’s method are better than the average linkage method.


## 4.1. Structural Comparison with Sectors Defined by KSE

The KSE has defined 33 sectors altogether based on the primary activities of listed companies. In this section, we compare the clusters returned by the proposed approaches with the sector-wise grouping provided by KSE. Table6 shows the grouping of the 161 companies in our dataset in these sectors.Table 6
Sector-wise list of the 161 KSE companies.
Sr. no.SectorCompaniesS1Oil and gasBYCO, MARI, NRL, OGDC, POL, PRL, PSO, and SHELS2ChemicalsCPL, DAWH, DSFL, DYNO, ENGRO, FFBL, FFC, ICI, LOTPTA, NICL, and SITCS3Forestry (paper and board)CEPBS4Industrial metals and miningHSPI and INILS5Construction and materials (cement)CHCC, DGKC, DNCC, EMCO, FCCL, FECTC, FRCL, GWLC, HADC, KOHC, LUCK, MLCF, and PIOCS6General industrialsGHGL, MACFL, PKGS, and SIEMS7Electronic and electrical goodsPCALS8EngineeringAGTL, HINO, KSBP, MTL, and PECOS9Industrial transportationPNSCS10Automobile and partsATBA, ATLH, BWHL, DFML, GHNL, GTYR, HCAR, INDU, PSMC, and SAZEWS11BeveragesMUREBS12Food producersCHAS, CLOV, DWSM, HABSM, HAL, JDWS, MIRKS, MZSM, NATF, NESTLE, PNGRS, QUICE, SANSM, SGML, and SHSMLS13Household goodsPAEL and SINGS14Personal goods (textile)ANL, BNWM, COTT, CRTM, GADT, GLPL, IBFL, IDRT, IDYM, KOIL, KTML, LOTPTA, MFTM, MUKT, NCL, NML, PSYL, RAVT, REWM, and SAIFS15TobaccoPAKTS16Healthcare equipment and servicesSHFAS17Pharma and biotechFEROZ, GLAXO, HINOON, and SEARLS18Travel and leisurePIAAS19Fixed line telecommunicationPTCS20ElectricityHUBC, JPGL, KESC, KOHE, KOHP, SEL, and SEPCOS21Commercial banksBOP, ESBL, FABL, FDIBL, KASBB, MCB, MEBL, NBP, and NIBS22Nonlife insuranceAICL, CENI, EFUG, HICL, PAKRI, and RICLS23Life insuranceEFULS24Financial servicesFCSC, FECM, JOVC, JSCL, OLPL, and SIBLS25Equity investment instrumentsFFLM, FHAM, FHBM, FPJM, FUDLM, GASF, FNBM, MODAM, PAKMI, PMI, and SCMS26Technology hardware and equipmentPTECIt can be viewed from Tables2–4 that five sectors (mentioned in Table 5) including commercial banks, nonlife insurance, life insurance, financial services, and equity investment instruments (S21, S22, S23, S24, and S25), are moving together and form a group which we term as “Money & Bank” group.


### 4.1.1. Clustering by the Estimated Mixing Coefficients Approach

If individual clusters formed by the estimated mixing coefficients approach in Table2 are analyzed, then it is apparent that six companies from the oil and gas sector (S1) and pharma and biotech Sector (S17) are combined in Cluster 1. Cluster 2 has a majority of companies from chemicals sector. Cluster 3 has six companies from money and bank Group. About half of the companies from the industrial metal and mining sector are part of cluster 4. Four companies from the sector general industries are gathered in cluster 6. Cluster 7 is comprised of 5 companies from money and bank group. The remaining clusters do not exhibit any such pattern.


### 4.1.2. Clustering by the RankedR¯2 Approach

The rankedR¯2 approach using the JADE algorithm returned 22 clusters, as shown in Table 3. The first cluster consists of 34 companies, of which six belong to the money and bank group, whereas one set of five are from the construction and material sector, another set of five are from the personal goods and textile sector, and the remaining 18 companies form smaller groups from other sectors such as food producers, automobile, and parts and chemicals. The first cluster thus takes the shape of a contrast where inversely related groups of sectors get put together, which negate each other in the sense that the positive behavior of the money and bank group, for example, causes a negative impact on the construction and material sector, that is people try to deposit money in the banks rather than consuming it on construction and material. The rationale visible in this cluster does not persist in the remaining clusters, so the argument cannot be forwarded ahead. Negation to the argument is quite obvious in the subsequent clusters, wherein in the second cluster, four out of twelve companies belong to money and bank group; two companies are from the automobile and parts sector. Similarly, four out of eleven companies in the third cluster are from money and bank group, and two companies are from the automobile and parts sector. In the fourth, cluster four out of nine companies belong to money and bank group. Two of them are from the food producers sector.The results in Table4 show that the calculation based upon FastICA produced similar results where, in the first cluster of size 39, ten companies belong to the money and bank group, six to the construction and materials sector, and three to the personal goods (textile) sector. The second cluster of 10 companies does not show a good internal structure as it includes three companies from the food producers sector and two from the chemical sector, whereas the remaining do not form any group. The third cluster is relatively smaller in size, where three companies are from the automobile and parts sector and two from money and bank group. Cluster 4 includes seven companies out of which three are from money and bank group and two from personal goods (textile) sector.Among the three algorithms, SOBI’s results presented in Table5 are the worst in the sense that too many clusters of relatively very small size are formed. The largest cluster identified contains only five companies. That is, the whole spirit of clustering is ruined.The comparison shows that the already defined sectors cannot be used as clusters. The discrepancy can be justified on the ground that the closing rates of the company do not follow a pattern governed by sectors rather they play their role independently. The stock market is based on perspicacity, which has become even more important in modern times because of online trading. Due to this, a lot of inexperienced day-traders have moved towards stock market trading; for example, HINO is classified by KSE in the engineering sector because it earns the largest portion of its revenue from this sector. If most of the investors recognize HINO as part of the automobile and parts sector then its price fluctuation will follow the behavior of the automobile and parts group. Clustering could also be very helpful in analyzing the time series of a group including several stocks. The behavior of an investment group is not necessarily determined by the stock that makes up the largest monetary share of the investment. Clustering the stock data could identify which groups have the greatest influence on the portfolio. It is difficult to identify the cluster of stocks as their appropriate sectors because of the uncertain behavior of some stocks. Similar results were presented by Wittman [28]. Thus clustering should not be confused with pre-defined grouping whatsoever.The next section concentrates on the exploration of the internal structure of clusters on their own.


## 4.1.1. Clustering by the Estimated Mixing Coefficients Approach

If individual clusters formed by the estimated mixing coefficients approach in Table2 are analyzed, then it is apparent that six companies from the oil and gas sector (S1) and pharma and biotech Sector (S17) are combined in Cluster 1. Cluster 2 has a majority of companies from chemicals sector. Cluster 3 has six companies from money and bank Group. About half of the companies from the industrial metal and mining sector are part of cluster 4. Four companies from the sector general industries are gathered in cluster 6. Cluster 7 is comprised of 5 companies from money and bank group. The remaining clusters do not exhibit any such pattern.


## 4.1.2. Clustering by the RankedR¯2 Approach

The rankedR¯2 approach using the JADE algorithm returned 22 clusters, as shown in Table 3. The first cluster consists of 34 companies, of which six belong to the money and bank group, whereas one set of five are from the construction and material sector, another set of five are from the personal goods and textile sector, and the remaining 18 companies form smaller groups from other sectors such as food producers, automobile, and parts and chemicals. The first cluster thus takes the shape of a contrast where inversely related groups of sectors get put together, which negate each other in the sense that the positive behavior of the money and bank group, for example, causes a negative impact on the construction and material sector, that is people try to deposit money in the banks rather than consuming it on construction and material. The rationale visible in this cluster does not persist in the remaining clusters, so the argument cannot be forwarded ahead. Negation to the argument is quite obvious in the subsequent clusters, wherein in the second cluster, four out of twelve companies belong to money and bank group; two companies are from the automobile and parts sector. Similarly, four out of eleven companies in the third cluster are from money and bank group, and two companies are from the automobile and parts sector. In the fourth, cluster four out of nine companies belong to money and bank group. Two of them are from the food producers sector.The results in Table4 show that the calculation based upon FastICA produced similar results where, in the first cluster of size 39, ten companies belong to the money and bank group, six to the construction and materials sector, and three to the personal goods (textile) sector. The second cluster of 10 companies does not show a good internal structure as it includes three companies from the food producers sector and two from the chemical sector, whereas the remaining do not form any group. The third cluster is relatively smaller in size, where three companies are from the automobile and parts sector and two from money and bank group. Cluster 4 includes seven companies out of which three are from money and bank group and two from personal goods (textile) sector.Among the three algorithms, SOBI’s results presented in Table5 are the worst in the sense that too many clusters of relatively very small size are formed. The largest cluster identified contains only five companies. That is, the whole spirit of clustering is ruined.The comparison shows that the already defined sectors cannot be used as clusters. The discrepancy can be justified on the ground that the closing rates of the company do not follow a pattern governed by sectors rather they play their role independently. The stock market is based on perspicacity, which has become even more important in modern times because of online trading. Due to this, a lot of inexperienced day-traders have moved towards stock market trading; for example, HINO is classified by KSE in the engineering sector because it earns the largest portion of its revenue from this sector. If most of the investors recognize HINO as part of the automobile and parts sector then its price fluctuation will follow the behavior of the automobile and parts group. Clustering could also be very helpful in analyzing the time series of a group including several stocks. The behavior of an investment group is not necessarily determined by the stock that makes up the largest monetary share of the investment. Clustering the stock data could identify which groups have the greatest influence on the portfolio. It is difficult to identify the cluster of stocks as their appropriate sectors because of the uncertain behavior of some stocks. Similar results were presented by Wittman [28]. Thus clustering should not be confused with pre-defined grouping whatsoever.The next section concentrates on the exploration of the internal structure of clusters on their own.


## 4.2. Validity of Clusters

In this section, we present an evaluation of the quality of clusters identified by our proposed clustering techniques by comparing them with the quality of clusters identified using two of the most widely used clustering methods, including Ward’s method [29] and the average linkage method [30].The quality of clustering can be gauged by measuring the internal homogeneity and external heterogeneity of the clusters. The clustering of a financial time series can be considered credible only when the stock prices within a cluster are maximally correlated, but different clusters are minimally correlated [31].Various indices are available in the literature to determine the validity of identified clusters. Two of the popular and fundamental ones are given as follows:(i)
Calinski–Harabasz Index (CHI): Calinski and Harabasz [32] introduced this index to assess the quality of the clustering solution by analyzing the similarity of the objects within each cluster and the dissimilarity of different clusters. This index is also called the variance ratio criterion (VRC). The larger value of CHI indicates better data partition. The CH index for K number of clusters on a data set X=x1,x2,…xN is given as(14)CH=∑k=1Knkvk−v/K−1∑k=1Kxi−vk/N−k,where,nk and vk are the number of points and centroid of the kth cluster, respectively. v is the overall centroid and N is the total number of data points.(ii)
Davies–Bouldin Index (DBI): this index was introduced by Davies and Bouldin [33] and is based on the ratio of within-cluster-distance to between-cluster-distance. The lower value of the index indicates a better cluster structure. The DBI is calculated for K clusters as follows:(15)DB=1K∑k=1KRk,where,Rk=maxRkl;k,l=1,2,…,K,k≠l,Rkl=sk+sldkl,where,dkl=dvk,vl,si=1ck∑xεckdx,vk.dx,z is the Euclidean distance between xandz, ck is the kth cluster, vk is the kth cluster centroid, and ck refers to norm of ck.Clusters are also identified using hierarchical clustering methods. Only the average linkage method and Ward’s method performed well as other hierarchical methods identified a large number of clusters consisting of one or two members. Hence the two indices are calculated for the clusters formed by the two proposed approaches and two existing approaches, i.e., the average linkage method and Ward’s method to cluster variables. The results are presented in Table7. The relative position of each of the indexes is presented in parentheses for quick comparison.Table 7
Validity indices for estimated mixing coefficients and rankedR¯2 clustering.
Clustering approachValidity index (ranks)Calinski–HarabaszDavies–BouldinProposed techniquesEstimated mixing coefficients44.1597 (1)4.0552 (1)RankedR¯2JADE15.6017 (2)4.1257 (2)FastICA12 (3)4.2017 (3)SOBI11.5935 (4)4.2707 (4)Existing techniquesAverage linkage method10.2368 (6)5.3113 (6)Ward’s method14.9963 (5)5.2687 (5)As depicted in Table7, the performance of both proposed methods is better than the existing techniques. The performance of the estimated mixing coefficient approach is the best. As discussed earlier, the large value of the Calinski–Harabasz index and the small value of Davies–Bouldin are considered better. Both indices awarded rank 1 to the proposed estimated mixing coefficients approach. The performance of JADE is better for the ranked R¯2 approach. Results of FastICA with ranked R¯2 are on the third place, algorithm SOBI with the same approach are placed on number 4. Among the existing approaches, the results of Ward’s method are better than the average linkage method.


## 5. Conclusion

In this paper, we presented two innovative approaches for clustering of multivariable datasets. The first approach is based upon the sum of squares of mixing coefficients, and the second is established using the ranking pattern of coefficient of determination of reconstructed and original series. Internal as well as external structure of clusters is explored. The compatibility of the clusters is contrasted with the available grouping mechanisms. It is concluded that the identification of clusters of stocks in their appropriate sectors is difficult because of the uncertain behavior of some stocks. Thus clustering should not be mixed up with predefined grouping whatsoever.Gauging the cluster quality using Calinski–Harabasz index and Davies–Bouldin index, we conclude that the performance of both proposed techniques is better than the existing traditional techniques. Our evaluation indicates that the estimated mixing coefficients approach can be regarded as a better approach among the proposed techniques.In the future, the current study can be extended to evaluate the performance of our suggested approaches on different types of datasets, e.g., biomedical, chemometrics, and signal processing datasets. Moreover, a criterion based on the level of independence of ICs may be explored for the identification of clusters. The proposed approach may also be explored for datasets having noise and outliers. The ICA-based identification of a cluster of observations may also be attempted.



---

*Source: 4830716-2023-01-31.xml*