beginning changes to retrieve and store batches of data so we don't blow
mem