10 new functions added, modified find_links and crawler_base.