ignore
added the start url twice, that is not necessary
use the first response for the page, makig sure the first version of response headers are there
Merge branch 'master' of github.com:soulgalore/crawler