Patent ID: 8819080

Claim:
A system for collection, retrieval, and distribution of data, comprising: a computing device comprising a processor and a memory; a data collector embodied in the computing device to collect data from at least one data source and to tag each group of collected data with an associated metadata tag, wherein the metadata tag includes information about the collected data; an object distribution logic module associated with the data collector, the object distribution logic module comprising one or more caches to store data and a distributed caching strategy structure, the distributed caching strategy structure including a cache collapsing module to only send updated portions of data rather than an entire data set in accordance with data consumer submitted information and a store and forward feature to dynamically reduce flow of the data to data consumers; at least one other computing device; at least one distributed metadata catalogue embodied in the at least one other computing device for containing a plurality of metadata tags, wherein the metadata tags without any associated groups of collected data are stored on the at least one distributed metadata catalogue, wherein only metadata indicating the existence and location of a requested group of data on a network is sent to a requester in response to a data request corresponding to the requested group of data associated with the metadata tag to avoid sending the requested group of data over the network until needed; and wherein the at least one distributed metadata catalogue comprises a metadata catalogue query services module configured to receive a query from the data consumer, wherein the query is received by a local metadata catalogue query services module of the at least one distributed metadata catalogue that is a closest distance from the data consumer, the local catalogue query services module is configured to determine if a scope of the query is set to local, the local catalogue query services module responding to the query in response to the scope of the query being set to local and the local catalogue query services module passing the query to another catalogue query services module of another distributed metadata catalogue capable of responding to the query.