Crawling the web, C style.