Updated database file naming.
Updated the way content type is determined.
Added an extra check in crawl_url to make sure the current URL hasn't already been crawled.