bugfix crawler not properly following URLs
bugfix profiling non-200 status pages