Introduce batch process to prevent memory issues