Patent ID: 9135261
Filing Date: 2015-09-15
Classification: G06F

Abstract:
1. A system for facilitating data discovery on a network, the network having one or more data storage devices, the system comprising: a crawler program configured to scan files stored on the one or more data storage devices, and identify a first set of files and a second set of files as being relevant, the scanning and the identifying being performed at a crawler operating speed, the crawler program being further configured to delay scanning based on at least one of the following conditions: (1) a file path length associated with one or more files in the first set of files exceeds a file length threshold; and 2) one or more filer lengths associated with one or more files of the of the first set of files exceed a filer length threshold; a data fetcher program configured to receive a location of the first set of files identified by the crawler program, the location being on the one or more data storage devices, and copy the first set of files from the received location at a data fetcher operating speed, the data fetcher program being further configured to delay copying the second set of files, thereby causing the crawler program to adjust the crawler operating speed of the scanning and the identifying according to the data fetcher operating speed based on at least one of the following conditions: (1) a file size associated with a file in the second set of files is smaller than a file size threshold, (2) a quantity of files in one of the first set and the second set of files exceeds a file quantity threshold, (3) a file format associated with a file of one of the first set and the second set of files does not belong to a predetermined set of file formats, and (4) an amount of text to index in the first set of files exceeds a text amount threshold; and circuit hardware implementing one or more functions of one or more of the crawler program and the data fetcher program.