Patent ID: 7801894

Claim:
A method for harvesting file system metadata, comprising: interacting with a file system abstraction layer/protocol adaptor to access managed files and directories across file systems that operate under various file system protocols at one or more physical locations; collecting raw metadata of the managed files and directories; filtering the raw metadata in real time; placing the filtered raw metadata in one or more volume metadata caches; synthesizing synthetic metadata from the filtered raw metadata; generating content-based metadata, wherein generating content-based metadata for a managed file comprises processing content of the file according to a type of the file to determine one or more content-specific entities within the file; transforming the filtered raw metadata, the synthetic metadata, and the content-based metadata into metadata records having a common representation, wherein each of the metadata record comprises a set of attributes associated with a file or directory residing on the file systems; processing the metadata records; and placing processed metadata records in volume clusters, wherein each of the volume clusters comprises one or more node data tables and one or more attribute tables, wherein each of the one or more node data tables represents denormalized dense attribute space common to the file systems and is timestamped by an epoch corresponding to a definition of freshness of data contained therein, and wherein each of the one or more attribute tables corresponds to a sparse attribute-volume-epoch combination.