now compressing batch data to reduce memory