Only calculate averages if necessary