  The majority of finite mixture models suffer from not allowing asymmetric
tail dependencies within components and not capturing non-elliptical clusters
in clustering applications. Since vine copulas are very flexible in capturing
these types of dependencies, we propose a novel vine copula mixture model for
continuous data. We discuss the model selection and parameter estimation
problems and further formulate a new model-based clustering algorithm. The use
of vine copulas in clustering allows for a range of shapes and dependency
structures for the clusters. Our simulation experiments illustrate a
significant gain in clustering accuracy when notably asymmetric tail
dependencies or/and non-Gaussian margins within the components exist. The
analysis of real data sets accompanies the proposed method. We show that the
model-based clustering algorithm with vine copula mixture models outperforms
the other model-based clustering techniques, especially for the non-Gaussian
multivariate data.
