We don't use batchSizes any other way other than popping the first element, and we only remove from it after the full batchSizes has been constructed. As such, it is OK for batchSizes to be modified, as it serves as an indirect indication of the current state of the iterator.