  Machine learning has proved to be a useful tool for extracting knowledge from
scientific data in numerous research fields, including astrophysics, genomics,
and molecular dynamics. Often, data sets from these research areas need to be
processed in distributed platforms due to their magnitude. This can be done
using one of the various distributed machine learning libraries available. One
of these libraries is dislib, a distributed machine learning library for Python
especially designed to process large scale data sets on HPC clusters, which
makes dislib an ideal candidate for analyzing scientific data. However,
dislib's main distributed data structure, called Dataset, has some limitations,
including poor performance in certain operations and low flexibility and
usability. In this paper, we propose a novel distributed data structure for
dislib, called ds-array, that addresses dislib's main limitations in data
management. Ds-arrays simplify distributed data management in dislib by
exposing a NumPy-like API, provide more flexibility, and reduce the
computational complexity of some operations. This results in performance
improvements of up to two orders of magnitude over Datasets, while also greatly
improving scalability and usability.
