Make link crawling more resilient

* Fixed a bug where errors that had no document would be overlooked
* Added ability to register URLs to only warn, not fail the build
Travis: disable default `./gradlew assemble` step