can determine a lot about how people are accessing your site. Firstly, if anyone is accessing the files you have excluded in your robots.txt, then ban them. A polite bot should read the robots.txt file and comply. If they do not then they deserve to be banned. Browse the file in Access. If you see large chunks of your site being accessed by an IP address, like screens of accesses in a short period of time, search Google for this IP address to see if it is a legitimate search engine or simply wasting your resources. All sorts of people and companies both private and public can visit your site. If you look them up and they are a known spammer, then ban them. Google Search will tell you. Reputation is everything on the internet, as in life. In one of the columns you will see GET or PUT. GET is a request to access and read your content. If you see a lot of PUTs into URLs such as your site's login screen, you know they are up to no good, so ban them. If you see a wall of PUTs into your site's comments, you know they are trying to spam you, so ban them. These steps are actually not difficult to do, but they can take time. The result is that your resources and therefore money are not wasted on those that do you nor the greater Internet community, any favours. Kill off all abusers of your