  Medical imaging is an important research field with many opportunities for
improving patients' health. However, there are a number of challenges that are
slowing down the progress of the field as a whole, such optimizing for
publication. In this paper we reviewed several problems related to choosing
datasets, methods, evaluation metrics, and publication strategies. With a
review of literature and our own analysis, we show that at every step,
potential biases can creep in. On a positive note, we also see that initiatives
to counteract these problems are already being started. Finally we provide a
broad range of recommendations on how to further these address problems in the
future. For reproducibility, data and code for our analyses are available on
\url{https://github.com/GaelVaroquaux/ml_med_imaging_failures}
