added scripts for analysing pagerank and crawl completeness