NEXUS-5944: Repository is auto-blocked if "allow file browsing" is disabled on remote

Changes:
1. externalize Page utility class from AR/Scraping, make it reusable
2. using Page testing what remote is in HttpClientRemoteStorage
3. added IT verifying it


Issue
https://issues.sonatype.org/browse/NEXUS-5944

CI
https://bamboo.zion.sonatype.com/browse/NX-OSSF20