more restrictive crawling and added prepare js