Fixed #42, rewrote part of crawl_url to increase readability.