NF: some changes to allow for piecemeal data extraction and saving as HDF5 file