Adaptive Learned Bloom Filter (Ada-BF): Efficient Utilization of the Classifier

1 INTRODUCTION . Bloom filter ( BF ) is a widely used data structure for low-memory and high-speed approximate membership testing ( Bloom , 1970 ) . Bloom filters compress a given set S into bit arrays , where we can approximately test whether a given element ( or query ) x belongs to a set S , i.e. , x ∈ S or otherwise . Several applications , in particular caching in memory constrained systems , have benefited tremendously from BF ( Broder et al. , 2002 ) . Bloom filter ensures a zero false negative rate ( FNR ) , which is a critical requirement for many applications . However , BF does not have a non-zero false positive rate ( FPR ) ( Dillinger and Manolios , 2004 ) due to hashing collisions , which measures the performance of BF . There is a known theoretical limit to this reduction . To achieve a FPR of , BF costs at least n log2 ( 1/ ) log2 e bits ( n = |S| ) , which is log2 e ≈ 44 % off from the theoretical lower bound ( Carter et al. , 1978 ) . Mitzenmacher ( 2002 ) proposed Compressed Bloom filter to address the suboptimal space usage of BF , where the space usage can reach the theoretical lower bound in the optimal case . To achieve a more significant reduction of FPR , researchers have generalized BF and incorporated information beyond the query itself to break through the theoretical lower bound of space usage . Bruck et al . ( 2006 ) has made use of the query frequency and varied the number of hash functions based on the query frequency to reduce the overall FPR . Recent work ( Kraska et al. , 2018 ; Mitzenmacher , 2018 ) has proposed to improve the performance of standard Bloom filter by incorporating a machine learning model . This approach paves a new hope of reducing false positive rates beyond the theoretical limit , by using context-specific information in the form of a machine learning model ( Hsu et al. , 2019 ) . Rae et al . ( 2019 ) further proposed Neural Bloom Filter that learns to write to memory using a distributed write scheme and achieves compression gains over the classical Bloom filter . The key idea behind Kraska et al . ( 2018 ) is to use the machine learning model as a pre-filter to give each query x a score s ( x ) . s ( x ) is usually positively associated with the odds that x ∈ S. The assumption is that in many practical settings , the membership of a query in the set S can be figured out from observable features of x and such information is captured by the classifier assigned score s ( x ) . The proposal of Kraska et al . uses this score and treats query x with score s ( x ) higher than a pre-determined threshold τ ( high confidence predictions ) as a direct indicator of the correct membership . Queries with scores less than τ are passed to the back-up Bloom filter . Compared to the standard Bloom filter , learned Bloom filter ( LBF ) uses a machine learning model to answer keys with high score s ( x ) . Thus , the classifier reduces the number of the keys hashed into the Bloom filter . When the machine learning model has a reliable prediction performance , learned Bloom filter significantly reduce the FPR and save memory usage ( Kraska et al. , 2018 ) . Mitzenmacher ( 2018 ) further provided a formal mathematical model for estimating the performance of LBF . In the same paper , the author proposed a generalization named sandwiched learned Bloom filter ( sandwiched LBF ) , where an initial filter is added before the learned oracle to improve the FPR if the parameters are chosen optimally . Wastage of Information : For existing learned Bloom filters to have a lower FPR , the classifier score greater than the threshold τ should have a small probability of wrong answer . Also , a significant fraction of the keys should fall in this high threshold regime to ensure that the backup filter is small . However , when the score s ( x ) is less than τ , the information in the score s ( x ) is never used . Thus , there is a clear waste of information . For instance , consider two elements x1 and x2 with τ > s ( x1 ) s ( x2 ) . In the existing solutions , x1 and x2 will be treated in the exact same way , even though there is enough prior to believing that x1 is more likely positive compared to x2 . Strong dependency on Generalization : It is natural to assume that prediction with high confidence implies a low FPR when the data distribution does not change . However , this assumption is too strong for many practical settings . First and foremost , the data distribution is likely to change in an online streaming environment where Bloom filters are deployed . Data streams are known to have bursty nature with drift in distribution ( Kleinberg , 2003 ) . As a result , the confidence of the classifier , and hence the threshold , is not completely reliable . Secondly , the susceptibility of machine learning oracles to adversarial examples brings new vulnerability in the system . Examples can be easily created where the classifier with any given confidence level τ , is incorrectly classified . Bloom filters are commonly used in networks where such increased adversarial false positive rate can hurt the performance . An increased latency due to collisions can open new possibilities of Denial-of-Service attacks ( DoS ) ( Feinstein et al. , 2003 ) . Motivation : For a binary classifier , the density of score distribution , f ( s ( x ) ) shows a different trend for elements in the set and outside the set S. We observe that for keys , f ( s ( x ) |x ∈ S ) shows ascending trend as s ( x ) increases while f ( s ( x ) |x /∈ S ) has an opposite trend . To reduce the overall FPR , we need lower FPRs for groups with a high f ( s ( x ) |x /∈ S ) . Hence , if we are tuning the number of hash functions differently , more hash functions are required for the corresponding groups . While for groups with a few non-keys , we allow higher FPRs . This variability is the core idea to obtaining a sweeter trade-off . Our Contributions : Instead of only relying on the classifier whether score s ( x ) is above a single specific threshold , we propose two algorithms , Ada-BF and disjoint Ada-BF , that rely on the complete spectrum of scores regions by adaptively tuning Bloom filter parameters in different score regions . 1 ) Ada-BF tunes the number of hash functions differently in different regions to adjust the FPR adaptively ; disjoint Ada-BF allocates variable memory Bloom filters to each region . 2 ) Our theoretical analysis reveals a new set of trade-offs that brings lower FPR with our proposed scheme compared to existing alternatives . 3 ) We evaluate the performance of our algorithms on two datasets : malicious URLs and malware MD5 signatures , where our methods reduce the FPR by over 80 % and save 50 % of the memory usage over existing learned Bloom filters . Notations : Our paper includes some notations that need to be defined here . Let [ g ] denote the index set { 1 , 2 , · · · , g } . We define query x as a key if x ∈ S , or a non-key if x /∈ S. Let n denote the size of keys ( n = |S| ) , and m denote the size of non-keys . We denote K as the number of hash functions used in the Bloom filter . 2 REVIEW : BLOOM FILTER AND LEARNED BLOOM FILTER . Bloom Filter : Standard Bloom filter for compressing a set S consists of an R-bits array and K independent random hash function h1 , h2 , · · · , hK , taking integer values between 0 and R− 1 , i.e. , hi : S ⇒ { 0 , 1 , · · · , R − 1 } . The bit array is initialized with all 0 . For every item x ∈ S , the bit value of hi ( x ) = 1 , for all i ∈ { 0 , 1 , · · · , K } , is set to 1 . To check a membership of an item x ′ in the set S , we return true if all the bits hi ( x ′ ) , for all i ∈ { 0 , 1 , · · · , K } , have been set to 1 . It is clear that Bloom filter has zero FNR ( false negative rate ) . However , due to lossy hash functions , x ′ may be wrongly identified to be positive when x ′ /∈ S while all the hi ( x ′ ) s are set to 1 due to random collisions . It can be shown that if the hash functions are independent , the expected FPR can be written as follows E ( FPR ) = ( 1− ( 1− 1 R ) Kn ) K . Learned Bloom filter : Learned Bloom filter adds a binary classification model to reduce the effective number of keys going to the Bloom filter . The classifier is pre-trained on some available training data to classify whether any given query x belongs to S or not based on its observable features . LBF sets a threshold , τ , where x is identified as a key if s ( x ) ≥ τ . Otherwise , x will be inserted into a Bloom filter to identify its membership in a further step ( Figure 1 ) . Like standard Bloom filter , LBF also has zero FNR . And the false positives can be either caused by that false positives of the classification model ( s ( x|x /∈ S ) ≥ τ ) or that of the Bloom filter . It is clear than when the region s ( x ) ≥ τ contains large number of keys , the number of keys inserted into the Bloom filter decreases which leads to favorable FPR . However , since we identify the region s ( x ) ≥ τ as positives , higher values of τ is better . At the same time , large τ decreases the number of keys in the region s ( x ) ≥ τ , increasing the load of the Bloom filter . Thus , there is a clear trade-off . 3 A STRICT GENERALIZATION : ADAPTIVE LEARNED BLOOM FILTER . ( ADA-BF ) With the formulation of LBF in the previous section , LBF actually divides the x into two groups . When s ( x ) ≥ τ , x will be identified as a key directly without testing with the Bloom filter . In other words , it uses zero hash function to identify its membership . Otherwise , we will test its membership using K hash functions . In other view , LBF switches from K hash functions to no hash function at all , based on s ( x ) ≥ τ or not . Continuing with this mindset , we propose adaptive learned Bloom filter , where x is divided into g groups based on s ( x ) , and for group j , we use Kj hash functions to test its membership . The structure of Ada-BF is represented in Figure 1 ( b ) . More specifically , we divide the spectrum into g regions , where x ∈ Group j if s ( x ) ∈ [ τj−1 , τj ) , j = 1 , 2 , · · · , g. Without loss of generality , here , we assume 0 = τ0 < τ1 < · · · < τg−1 < τg = 1 . Keys from group j are inserted into Bloom filter using Kj independent hash functions . Thus , we use different number of universal hash functions for keys from different groups . For a group j , the expected FPR can be expressed as , E ( FPRj ) = ( 1− ( 1− 1 R ) ∑g t=1 ntKt ) Kj = αKj ( 1 ) where nt = ∑n t=1 I ( τt−1 ≤ s ( xi|xi ∈ S ) < τt ) is the number of keys falling in group t , and Kj is the number of hash functions used in group j . By varying Kj , E ( FPRj ) can be controlled differently for each group . Variable number of hash functions gives us enough flexibility to tune the FPR of each region . To avoid the bit array being overloaded , we only increase the Kj for groups with large number of keys nj , while decrease Kj for groups with small nj . It should be noted that f ( s ( x ) |x ∈ S ) shows an opposite trend compared to f ( s ( x ) |x /∈ S ) as s ( x ) increases ( Figure 2 ) . Thus , there is a need for variable tuning , and a spectrum of regions gives us the room to exploit these variability efficiently . Clearly , Ada-BF generalizes the LBF . When Ada-BF only divides the queries into two groups , by setting K1 = K , K2 = 0 and τ1 = τ , Ada-BF reduces to the LBF .

Summary: This paper extends the Bloom filter learning by using the complete spectrum of the scores regions. It uses multiple thresholds and then varies the number of hash functions among different scores regions to obtain better trade-off. Detailed theoretical analysis provides guaranteed superiority over learned Bloom filter under some conditions. The experiments also show the two proposed methods outperform learned Bloom filter in FPR and memory usage.