Detecting Meaningful Clusters From High-Dimensional Data: A Strongly Consistent Sparse Center-Based Clustering Approach

In context to high-dimensional clustering, the concept of <italic>feature weighting</italic> has gained considerable importance over the years to capture the relative degrees of importance of different features in revealing the cluster structure of the dataset. However, the popular techniques in this area either fail to perform feature selection or do not preserve the simplicity of Lloyd’s heuristic to solve the <inline-formula><tex-math notation="LaTeX">$k$</tex-math><alternatives><mml:math><mml:mi>k</mml:mi></mml:math><inline-graphic xlink:href="chakraborty-ieq1-3047489.gif"/></alternatives></inline-formula>-means problem and the like. In this paper, we propose a Lasso Weighted <inline-formula><tex-math notation="LaTeX">$k$</tex-math><alternatives><mml:math><mml:mi>k</mml:mi></mml:math><inline-graphic xlink:href="chakraborty-ieq2-3047489.gif"/></alternatives></inline-formula>-means (<inline-formula><tex-math notation="LaTeX">$LW$</tex-math><alternatives><mml:math><mml:mrow><mml:mi>L</mml:mi><mml:mi>W</mml:mi></mml:mrow></mml:math><inline-graphic xlink:href="chakraborty-ieq3-3047489.gif"/></alternatives></inline-formula>-<inline-formula><tex-math notation="LaTeX">$k$</tex-math><alternatives><mml:math><mml:mi>k</mml:mi></mml:math><inline-graphic xlink:href="chakraborty-ieq4-3047489.gif"/></alternatives></inline-formula>-means) algorithm, as a simple yet efficient sparse clustering procedure for high-dimensional data where the number of features (<inline-formula><tex-math notation="LaTeX">$p$</tex-math><alternatives><mml:math><mml:mi>p</mml:mi></mml:math><inline-graphic xlink:href="chakraborty-ieq5-3047489.gif"/></alternatives></inline-formula>) can be much higher than the number of observations (<inline-formula><tex-math notation="LaTeX">$n$</tex-math><alternatives><mml:math><mml:mi>n</mml:mi></mml:math><inline-graphic xlink:href="chakraborty-ieq6-3047489.gif"/></alternatives></inline-formula>). The <inline-formula><tex-math notation="LaTeX">$LW$</tex-math><alternatives><mml:math><mml:mrow><mml:mi>L</mml:mi><mml:mi>W</mml:mi></mml:mrow></mml:math><inline-graphic xlink:href="chakraborty-ieq7-3047489.gif"/></alternatives></inline-formula>-<inline-formula><tex-math notation="LaTeX">$k$</tex-math><alternatives><mml:math><mml:mi>k</mml:mi></mml:math><inline-graphic xlink:href="chakraborty-ieq8-3047489.gif"/></alternatives></inline-formula>-means method imposes an <inline-formula><tex-math notation="LaTeX">$\ell _1$</tex-math><alternatives><mml:math><mml:msub><mml:mi>ℓ</mml:mi><mml:mn>1</mml:mn></mml:msub></mml:math><inline-graphic xlink:href="chakraborty-ieq9-3047489.gif"/></alternatives></inline-formula> regularization term involving the feature weights directly to induce feature selection in a sparse clustering framework. We develop a simple block-coordinate descent type algorithm with time-complexity resembling that of Lloyd’s method, to optimize the proposed objective. In addition, we establish the strong consistency of the <inline-formula><tex-math notation="LaTeX">$LW$</tex-math><alternatives><mml:math><mml:mrow><mml:mi>L</mml:mi><mml:mi>W</mml:mi></mml:mrow></mml:math><inline-graphic xlink:href="chakraborty-ieq10-3047489.gif"/></alternatives></inline-formula>-<inline-formula><tex-math notation="LaTeX">$k$</tex-math><alternatives><mml:math><mml:mi>k</mml:mi></mml:math><inline-graphic xlink:href="chakraborty-ieq11-3047489.gif"/></alternatives></inline-formula>-means procedure. Such an analysis of the large sample properties is not available for the conventional sparse <inline-formula><tex-math notation="LaTeX">$k$</tex-math><alternatives><mml:math><mml:mi>k</mml:mi></mml:math><inline-graphic xlink:href="chakraborty-ieq12-3047489.gif"/></alternatives></inline-formula>-means algorithms, in general. <inline-formula><tex-math notation="LaTeX">$LW$</tex-math><alternatives><mml:math><mml:mrow><mml:mi>L</mml:mi><mml:mi>W</mml:mi></mml:mrow></mml:math><inline-graphic xlink:href="chakraborty-ieq13-3047489.gif"/></alternatives></inline-formula>-<inline-formula><tex-math notation="LaTeX">$k$</tex-math><alternatives><mml:math><mml:mi>k</mml:mi></mml:math><inline-graphic xlink:href="chakraborty-ieq14-3047489.gif"/></alternatives></inline-formula>-means is tested on a number of synthetic and real-life datasets and through a detailed experimental analysis, we find that the performance of the method is highly competitive against the baselines as well as the state-of-the-art procedures for center-based high-dimensional clustering, not only in terms of clustering accuracy but also with respect to computational time.