Improves disk storage usage and adds a logic to mark documents as already being processed