changed to absolute paths and added pickle file for tracking batches