graemedouglas/sitewatcher — pull request #1 by gburtini (2013-04-16 20:23:04+00:00)
Title: Pull me please!

I've made some changes I probably should have put in multiple commits so you could pick and choose which you wanted and we could maintain different branches of the other functionality if you wanted, but basically:

1. Added a new way of specifying a URL. You can now execute this as ./sitewatcher http://github.com/gburtini
2. Removed the idea of a "maximum number of requests" by default (set it to -1, check for -1).
3. Changed the order of the request/sleep cycle so that an output occurs immediately.
4. Changed outputs to overwrite the previous one if it is just going to output "Nothing has changed." (instead of filling your terminal with "Nothing has changed" it is now just the one line).
5. Added date on output of changes.
6. Added some ideas that I would like this to do but don't have time to try to build right now.
7. The script now keeps running after a change -- because we can keep a coherent log of changes. The only way to exit at the moment is to ungracefully terminate.

I think that is it. There is an issue here in sites that do things like maintain the date live or change ads on every load -- this will report that they have changed every time even if the change isn't "significant." I'm not sure how to identify a "significant" change, but maybe some options could make this more useful: for example, "was there a new link added or removed" is a common thing I'd want to monitor (think about the original use case) or "was more than X% changed" might be useful in some cases.

We should also be identifying network outages and changes that don't occur on the server side -- for example, if your network went down or the page stopped loading entirely, its not cool to say the page changed then changed back when you finish.