Before it can be used for composition, individual data items have to be processed for correcting the effects of various distortions, including instrument drift and atmospheric effects. This requires effective declustering of a multi-dimensional dataset onto a suitably configured disk farm. Second, the order of magnitude reduction in data size makes it imperative, from a performance perspective, that correction and composition operations be performed on the same machine that the data is stored on. This requires careful coordination of computation and data retrieval to avoid slowing down either process. To meet these requirements, researchers at the University of Maryland have developed Titan, a parallel database designed for handling remote- sensing data.
Titan addresses the problem of low-latency retrieval of very large volumes of data in two ways.
Second, it tries to maximize disk parallelism by declustering the set of chunks onto a large disk farm. For declustering the dataset, the group developed and used the minimax declustering algorithm. The key idea of the algorithm is to extend Prim's minimal spanning tree algorithm to construct as many spanning trees as there are disks in the disk farm, and to assign all the blocks associated with a single spanning tree to a single disk. To generate the edge costs, we chose the proximity index proposed by Kamel and Faloutsos. Titan delivers good performance for both small and large queries.