  Machine learning has shown extensive growth in recent years. However,
previously existing studies highlighted a reproducibility crisis in machine
learning. The reasons for irreproducibility are manifold. Major machine
learning libraries default to the usage of non-deterministic algorithms based
on atomic operations. Solely fixing all random seeds is not sufficient for
deterministic machine learning. To overcome this shortcoming, various machine
learning libraries released deterministic counterparts to the non-deterministic
algorithms. We evaluated the effect of these algorithms on determinism and
runtime. Based on these results, we formulated a set of requirements for
reproducible machine learning and developed a new software solution, the
mlf-core ecosystem, which aids machine learning projects to meet and keep these
requirements. We applied mlf-core to develop fully reproducible models in
various biomedical fields including a single cell autoencoder with TensorFlow,
a PyTorch-based U-Net model for liver-tumor segmentation in CT scans, and a
liver cancer classifier based on gene expression profiles with XGBoost.
