An eficient and distributed scheme forfile mapping or file lookup scheme is critical in decentralizing metadata management within a group of metadata servers. This paper presents a novel technique called HBA (Hierarchical Bloom Filter Arrays) to mapfile names to the servers holding their metadata. Two levels of probabilistic arrays, i.e., Bloom Filter Arrays, with different accuracies are used on each metadata serve,: One array, with lower accuracy and representing the distribution of the entire metadata, trades accuracy for significantly reduced memory overhead, while the other array, with higher accuracy, caches partial distribution information and exploits the temporal locality offile access patterns. Extensive trace-driven simulations have shown our HBA design to be highly effective and eflcient in improving performance and scalability offile systems in clusters with 1,000 to 10,000 nodes (or super-clusters).
Hierarchical Bloom-Filter Array (HBA) REF uses a two-tier probabilistic array, i.e. Bloom filter array, to calculate corresponding MDS to the file a user want to query.
Hierarchical Bloom filter arrays (HBA): a novel, scalable metadata management system for large cluster-based storage
2004 IEEE International Conference on Cluster Computing (IEEE Cat. No.04EX935)
2004 IEEE International Conference on Cluster Computing (IEEE Cat. No.04EX935)
Computer Science