If there is one thing I have learned about robots, it is that there is absolutely no pattern to them.
Most robots are stupid and wander randomly.
Then they come back a week later, ask for the same thing and then go away, again. This happens over and over again for months. I have never figured it out. If they wanted to see if the Web site was really a Web site, they could just Ping it. This would be much faster and much more efficient.