With code now in link_crawler, I think you don't need this anymore.