Properly return the histograms from build_histograms