Return http status 500 instead of 400 on errors

400 implies that there is a problem with the request. 500 is an internal error.
Return the http status code from the page being crawled