Patent ID: 6418453
Filing Date: 2002-07-09
Classification: G06F,Y10S

Abstract:
In a computer network comprising a web server machine and a web crawler machine, a method comprising:a) automatically maintaining a file modification list on the web server, wherein the file modification list comprises a plurality of modification entries describing modifications made to web content files stored on the web server machine, wherein each modification entry comprises a file name, a file modification time, and a file modification type; b) transmitting from the web crawler machine to the web server machine a request for file modification information, wherein the request comprises a last visited time indicating when the web crawler machine last visited the web server machine; c) generating the requested file modification information and transmitting the requested file modification information to the web crawler machine, wherein the requested file modification information comprises file names corresponding to file modification times later than the last visited time of the web crawler machine; d) generating from the requested file modification information transmitted to the web crawler machine a list of new and changed web content files stored on the web server machine; and e) transmitting from the web crawler machine to the web server machine requests for the new and changed web content files.