Fix histogram calculation in equalizeHist

issue 2678