web crawler (find temporary files, parse robots.tx, search some folders and google dorks) for recon