added basic web crawling support