  As the data size in Machine Learning fields grows exponentially, it is
inevitable to accelerate the computation by utilizing the ever-growing large
number of available cores provided by high-performance computing hardware.
However, existing parallel methods for clustering or regression often suffer
from problems of low accuracy, slow convergence, and complex
hyperparameter-tuning. Furthermore, the parallel efficiency is usually
difficult to improve while striking a balance between preserving model
properties and partitioning computing workloads on distributed systems. In this
paper, we propose a novel and simple data structure capturing the most
important information among data samples. It has several advantageous
properties supporting a hierarchical clustering strategy that is irrelevant to
the hardware parallelism, well-defined metrics for determining optimal
clustering, balanced partition for maintaining the compactness property, and
efficient parallelization for accelerating computation phases. Then we combine
the clustering with regression techniques as a parallel library and utilize a
hybrid structure of data and model parallelism to make predictions. Experiments
illustrate that our library obtains remarkable performance on convergence,
accuracy, and scalability.
