Reduced number of async crawls to reduce memory usage