fix mem leak. always harvest links
for custom crawls.