Added an option to ignore authentication check so spidering doesn't halt upon discovering invalid credentials.