Almost-working site scanner.

Redirects drive it crazy at the moment.