Web crawling functions for R, handling idiosyncrasies and contingencies gracefully