Implemented etags in the crawler, the crawler now stores hashes of the xml