Speed up and reduced memory usage in variance calculations