  Obtaining accurate photometric redshift estimations is an important aspect of
cosmology, remaining a prerequisite of many analyses. In creating novel methods
to produce redshift estimations, there has been a shift towards using machine
learning techniques. However, there has not been as much of a focus on how well
different machine learning methods scale or perform with the ever-increasing
amounts of data being produced. Here, we introduce a benchmark designed to
analyse the performance and scalability of different supervised machine
learning methods for photometric redshift estimation. Making use of the Sloan
Digital Sky Survey (SDSS - DR12) dataset, we analysed a variety of the most
used machine learning algorithms. By scaling the number of galaxies used to
train and test the algorithms up to one million, we obtained several metrics
demonstrating the algorithms' performance and scalability for this task.
Furthermore, by introducing a new optimisation method, time-considered
optimisation, we were able to demonstrate how a small concession of error can
allow for a great improvement in efficiency. From the algorithms tested we
found that the Random Forest performed best in terms of error with a mean
squared error, MSE = 0.0042; however, as other algorithms such as Boosted
Decision Trees and k-Nearest Neighbours performed incredibly similarly, we used
our benchmarks to demonstrate how different algorithms could be superior in
different scenarios. We believe benchmarks such as this will become even more
vital with upcoming surveys, such as LSST, which will capture billions of
galaxies requiring photometric redshifts.
