Adaptive Learned Bloom Filter (Ada-BF): Efficient Utilization of the Classifier

1 INTRODUCTION . Bloom filter ( BF ) is a widely used data structure for low-memory and high-speed approximate membership testing ( Bloom , 1970 ) . Bloom filters compress a given set S into bit arrays , where we can approximately test whether a given element ( or query ) x belongs to a set S , i.e. , x ∈ S or otherwise . Several applications , in particular caching in memory constrained systems , have benefited tremendously from BF ( Broder et al. , 2002 ) . Bloom filter ensures a zero false negative rate ( FNR ) , which is a critical requirement for many applications . However , BF does not have a non-zero false positive rate ( FPR ) ( Dillinger and Manolios , 2004 ) due to hashing collisions , which measures the performance of BF . There is a known theoretical limit to this reduction . To achieve a FPR of , BF costs at least n log2 ( 1/ ) log2 e bits ( n = |S| ) , which is log2 e ≈ 44 % off from the theoretical lower bound ( Carter et al. , 1978 ) . Mitzenmacher ( 2002 ) proposed Compressed Bloom filter to address the suboptimal space usage of BF , where the space usage can reach the theoretical lower bound in the optimal case . To achieve a more significant reduction of FPR , researchers have generalized BF and incorporated information beyond the query itself to break through the theoretical lower bound of space usage . Bruck et al . ( 2006 ) has made use of the query frequency and varied the number of hash functions based on the query frequency to reduce the overall FPR . Recent work ( Kraska et al. , 2018 ; Mitzenmacher , 2018 ) has proposed to improve the performance of standard Bloom filter by incorporating a machine learning model . This approach paves a new hope of reducing false positive rates beyond the theoretical limit , by using context-specific information in the form of a machine learning model ( Hsu et al. , 2019 ) . Rae et al . ( 2019 ) further proposed Neural Bloom Filter that learns to write to memory using a distributed write scheme and achieves compression gains over the classical Bloom filter . The key idea behind Kraska et al . ( 2018 ) is to use the machine learning model as a pre-filter to give each query x a score s ( x ) . s ( x ) is usually positively associated with the odds that x ∈ S. The assumption is that in many practical settings , the membership of a query in the set S can be figured out from observable features of x and such information is captured by the classifier assigned score s ( x ) . The proposal of Kraska et al . uses this score and treats query x with score s ( x ) higher than a pre-determined threshold τ ( high confidence predictions ) as a direct indicator of the correct membership . Queries with scores less than τ are passed to the back-up Bloom filter . Compared to the standard Bloom filter , learned Bloom filter ( LBF ) uses a machine learning model to answer keys with high score s ( x ) . Thus , the classifier reduces the number of the keys hashed into the Bloom filter . When the machine learning model has a reliable prediction performance , learned Bloom filter significantly reduce the FPR and save memory usage ( Kraska et al. , 2018 ) . Mitzenmacher ( 2018 ) further provided a formal mathematical model for estimating the performance of LBF . In the same paper , the author proposed a generalization named sandwiched learned Bloom filter ( sandwiched LBF ) , where an initial filter is added before the learned oracle to improve the FPR if the parameters are chosen optimally . Wastage of Information : For existing learned Bloom filters to have a lower FPR , the classifier score greater than the threshold τ should have a small probability of wrong answer . Also , a significant fraction of the keys should fall in this high threshold regime to ensure that the backup filter is small . However , when the score s ( x ) is less than τ , the information in the score s ( x ) is never used . Thus , there is a clear waste of information . For instance , consider two elements x1 and x2 with τ > s ( x1 ) s ( x2 ) . In the existing solutions , x1 and x2 will be treated in the exact same way , even though there is enough prior to believing that x1 is more likely positive compared to x2 . Strong dependency on Generalization : It is natural to assume that prediction with high confidence implies a low FPR when the data distribution does not change . However , this assumption is too strong for many practical settings . First and foremost , the data distribution is likely to change in an online streaming environment where Bloom filters are deployed . Data streams are known to have bursty nature with drift in distribution ( Kleinberg , 2003 ) . As a result , the confidence of the classifier , and hence the threshold , is not completely reliable . Secondly , the susceptibility of machine learning oracles to adversarial examples brings new vulnerability in the system . Examples can be easily created where the classifier with any given confidence level τ , is incorrectly classified . Bloom filters are commonly used in networks where such increased adversarial false positive rate can hurt the performance . An increased latency due to collisions can open new possibilities of Denial-of-Service attacks ( DoS ) ( Feinstein et al. , 2003 ) . Motivation : For a binary classifier , the density of score distribution , f ( s ( x ) ) shows a different trend for elements in the set and outside the set S. We observe that for keys , f ( s ( x ) |x ∈ S ) shows ascending trend as s ( x ) increases while f ( s ( x ) |x /∈ S ) has an opposite trend . To reduce the overall FPR , we need lower FPRs for groups with a high f ( s ( x ) |x /∈ S ) . Hence , if we are tuning the number of hash functions differently , more hash functions are required for the corresponding groups . While for groups with a few non-keys , we allow higher FPRs . This variability is the core idea to obtaining a sweeter trade-off . Our Contributions : Instead of only relying on the classifier whether score s ( x ) is above a single specific threshold , we propose two algorithms , Ada-BF and disjoint Ada-BF , that rely on the complete spectrum of scores regions by adaptively tuning Bloom filter parameters in different score regions . 1 ) Ada-BF tunes the number of hash functions differently in different regions to adjust the FPR adaptively ; disjoint Ada-BF allocates variable memory Bloom filters to each region . 2 ) Our theoretical analysis reveals a new set of trade-offs that brings lower FPR with our proposed scheme compared to existing alternatives . 3 ) We evaluate the performance of our algorithms on two datasets : malicious URLs and malware MD5 signatures , where our methods reduce the FPR by over 80 % and save 50 % of the memory usage over existing learned Bloom filters . Notations : Our paper includes some notations that need to be defined here . Let [ g ] denote the index set { 1 , 2 , · · · , g } . We define query x as a key if x ∈ S , or a non-key if x /∈ S. Let n denote the size of keys ( n = |S| ) , and m denote the size of non-keys . We denote K as the number of hash functions used in the Bloom filter . 2 REVIEW : BLOOM FILTER AND LEARNED BLOOM FILTER . Bloom Filter : Standard Bloom filter for compressing a set S consists of an R-bits array and K independent random hash function h1 , h2 , · · · , hK , taking integer values between 0 and R− 1 , i.e. , hi : S ⇒ { 0 , 1 , · · · , R − 1 } . The bit array is initialized with all 0 . For every item x ∈ S , the bit value of hi ( x ) = 1 , for all i ∈ { 0 , 1 , · · · , K } , is set to 1 . To check a membership of an item x ′ in the set S , we return true if all the bits hi ( x ′ ) , for all i ∈ { 0 , 1 , · · · , K } , have been set to 1 . It is clear that Bloom filter has zero FNR ( false negative rate ) . However , due to lossy hash functions , x ′ may be wrongly identified to be positive when x ′ /∈ S while all the hi ( x ′ ) s are set to 1 due to random collisions . It can be shown that if the hash functions are independent , the expected FPR can be written as follows E ( FPR ) = ( 1− ( 1− 1 R ) Kn ) K . Learned Bloom filter : Learned Bloom filter adds a binary classification model to reduce the effective number of keys going to the Bloom filter . The classifier is pre-trained on some available training data to classify whether any given query x belongs to S or not based on its observable features . LBF sets a threshold , τ , where x is identified as a key if s ( x ) ≥ τ . Otherwise , x will be inserted into a Bloom filter to identify its membership in a further step ( Figure 1 ) . Like standard Bloom filter , LBF also has zero FNR . And the false positives can be either caused by that false positives of the classification model ( s ( x|x /∈ S ) ≥ τ ) or that of the Bloom filter . It is clear than when the region s ( x ) ≥ τ contains large number of keys , the number of keys inserted into the Bloom filter decreases which leads to favorable FPR . However , since we identify the region s ( x ) ≥ τ as positives , higher values of τ is better . At the same time , large τ decreases the number of keys in the region s ( x ) ≥ τ , increasing the load of the Bloom filter . Thus , there is a clear trade-off . 3 A STRICT GENERALIZATION : ADAPTIVE LEARNED BLOOM FILTER . ( ADA-BF ) With the formulation of LBF in the previous section , LBF actually divides the x into two groups . When s ( x ) ≥ τ , x will be identified as a key directly without testing with the Bloom filter . In other words , it uses zero hash function to identify its membership . Otherwise , we will test its membership using K hash functions . In other view , LBF switches from K hash functions to no hash function at all , based on s ( x ) ≥ τ or not . Continuing with this mindset , we propose adaptive learned Bloom filter , where x is divided into g groups based on s ( x ) , and for group j , we use Kj hash functions to test its membership . The structure of Ada-BF is represented in Figure 1 ( b ) . More specifically , we divide the spectrum into g regions , where x ∈ Group j if s ( x ) ∈ [ τj−1 , τj ) , j = 1 , 2 , · · · , g. Without loss of generality , here , we assume 0 = τ0 < τ1 < · · · < τg−1 < τg = 1 . Keys from group j are inserted into Bloom filter using Kj independent hash functions . Thus , we use different number of universal hash functions for keys from different groups . For a group j , the expected FPR can be expressed as , E ( FPRj ) = ( 1− ( 1− 1 R ) ∑g t=1 ntKt ) Kj = αKj ( 1 ) where nt = ∑n t=1 I ( τt−1 ≤ s ( xi|xi ∈ S ) < τt ) is the number of keys falling in group t , and Kj is the number of hash functions used in group j . By varying Kj , E ( FPRj ) can be controlled differently for each group . Variable number of hash functions gives us enough flexibility to tune the FPR of each region . To avoid the bit array being overloaded , we only increase the Kj for groups with large number of keys nj , while decrease Kj for groups with small nj . It should be noted that f ( s ( x ) |x ∈ S ) shows an opposite trend compared to f ( s ( x ) |x /∈ S ) as s ( x ) increases ( Figure 2 ) . Thus , there is a need for variable tuning , and a spectrum of regions gives us the room to exploit these variability efficiently . Clearly , Ada-BF generalizes the LBF . When Ada-BF only divides the queries into two groups , by setting K1 = K , K2 = 0 and τ1 = τ , Ada-BF reduces to the LBF .

Summary: The paper proposed an adaptive learned bloom filter. Rather than setting a threshold of prediction score, the paper partitions the score into several intervals; for query insider each interval, the paper either uses a group of independent hash functions to hash the query in one unified bloom filter or introduce an independent bloom filter. The paper proposes efficient ways to tune the hyper-parameters, and provides the analysis of the error. Experiments on two applications show the effectiveness of the proposed methods.