A general crawler for checking errors on web pages