fixes #439. The main url for crawling should be without the basic auth i...

Fixes #439.

The problem was that the basic authentication URL was used also as the main URL for clicking and crawling. The fix is to load the page once in the beginning with the basic auth URL and then continue with the normal URL.