Updating scraper to try to guess at when made public and not worry about search indexing.
Updating interface to reflect scraping changes.