compress: fix sa8dCost comparisons.

After this patch, all sa8d costs are stored in m_sa8dCost, and RD costs in m_totalRDCost