Title: Fast Partitioned Learned Bloom Filter
Venue: NeurIPS 2023
Accepted: yes
Keywords: optimization

data structures

algorithms

theory

learned algorithms
Abstract: A Bloom filter is a memory-efficient data structure for approximate membership queries used in numerous fields of computer science.
Recently, learned Bloom filters that achieve better memory efficiency using machine learning models have attracted attention.
One such filter, the partitioned learned Bloom filter (PLBF), achieves excellent memory efficiency.
However, PLBF requires a $\mathcal{O}(N^3k)$ time complexity to construct the data structure, where $N$ and $k$ are the hyperparameters of PLBF.
One can improve memory efficiency by increasing $N$, but the construction time becomes extremely long.
Thus, we propose two methods that can reduce the construction time while maintaining the memory efficiency of PLBF.
First, we propose fast PLBF, which can construct the same data structure as PLBF with a smaller time complexity $\mathcal{O}(N^2k)$.
Second, we propose fast PLBF++, which can construct the data structure with even smaller time complexity $\mathcal{O}(Nk\log N + Nk^2)$.
Fast PLBF++ does not necessarily construct the same data structure as PLBF.
Still, it is almost as memory efficient as PLBF, and it is proved that fast PLBF++ has the same data structure as PLBF when the distribution satisfies a certain constraint.
Our experimental results from real-world datasets show that (i) fast PLBF and fast PLBF++ can construct the data structure up to 233 and 761 times faster than PLBF, (ii) fast PLBF can achieve the same memory efficiency as PLBF, and (iii) fast PLBF++ can achieve almost the same memory efficiency as PLBF.
The codes are available at [this https URL](https://github.com/atsukisato/FastPLBF).

Fast Partitioned Learned Bloom Filter

Abstract

A Bloom filter is a memory-efficient data structure for approximate membership queries used in numerous fields of computer science. Recently, learned Bloom filters that achieve better memory efficiency using machine learning models have attracted attention. One such filter, the partitioned learned Bloom filter (PLBF), achieves excellent memory efficiency. However, PLBF requires a O ( N 3 k ) time complexity to construct the data structure, where N and k are the hyperparameters of PLBF. One can improve memory efficiency by increasing N , but the construction time becomes extremely long. Thus, we propose two methods that can reduce the construction time while maintaining the memory efficiency of PLBF. First, we propose fast PLBF, which can construct the same data structure as PLBF with a smaller time complexity O ( N 2 k ) . Second, we propose fast PLBF++, which can construct the data structure with even smaller time complexity O ( Nk log N + Nk 2 ) . Fast PLBF++ does not necessarily construct the same data structure as PLBF. Still, it is almost as memory efficient as PLBF, and it is proved that fast PLBF++ has the same data structure as PLBF when the distribution satisfies a certain constraint. Our experimental results from real-world datasets show that (i) fast PLBF and fast PLBF++ can construct the data structure up to 233 and 761 times faster than PLBF, (ii) fast PLBF can achieve the same memory efficiency as PLBF, and (iii) fast PLBF++ can achieve almost the same memory efficiency as PLBF. The codes are available at https://github.com/atsukisato/FastPLBF .

1 Introduction

Membership query is a problem of determining whether a given query q is contained within a set S . Membership query is widely used in numerous areas, including networks and databases. One can correctly answer the membership query by keeping the set S and checking whether it contains q . However, this approach is memory intensive because we need to maintain S .

A Bloom filter [1] is a memory-efficient data structure that answers approximate membership queries. A Bloom filter uses hash functions to compress the set into a bitstring and then answers the queries using the bitstring. When a Bloom filter answers q ∈ S , it can be wrong (false positives can arise); when it answers q / ∈ S , it is always correct (no false negatives arise). A Bloom filter has been incorporated into numerous systems, such as networks [2, 3, 4], databases [5, 6, 7], and cryptocurrencies [8, 9].

Atraditional Bloom filter does not care about the distribution of the set or the queries. Therefore, even if the distribution has a characteristic structure, a Bloom filter cannot take advantage of it. Kraska et al. proposed learned Bloom filters (LBFs), which utilize the distribution structure using a machine learning model [10]. LBF achieves a better trade-off between memory usage and false positive rate (FPR) than the original Bloom filter. Since then, several studies have been conducted on various LBFs [11, 12, 13].

𝑓

!

𝑓

"

𝑓

#

𝑡

"

𝑡

#$!

𝑡

!

…

…

Density

score

𝑡

#

= 1

𝑡

%

= 0

keys

non-keys

FPR

BF

!

BF

"

BF

#

…

0.005

0.002

0.008

…

…

…

…

…

…

…

0.2

0.1

0.02

0.05

…

…

…

…

…

…

…

0.001

1

2

𝑖

𝑁

𝐺

!

𝑔

"

𝑡

#

𝑡

$%!

𝑡

!

…

3

𝑡

&

= 0

𝑡

$

= 1

ℎ

"

𝐺

#

𝐺

$

𝐻

!

𝐻

#

𝐻

$

𝑓

!

𝑓

#

𝑓

$

…

FPR

BF

!

BF

#

BF

$

…

Partitioned learned Bloom filter (PLBF) [14] is a variant of the LBF. PLBF effectively uses the distribution and is currently one of the most memory-efficient LBFs. To construct PLBF, a machine learning model is trained to predict whether the input is included in the set. For a given element x , the machine learning model outputs a score s ( x ) ∈ [0 , 1] , indicating the probability that S includes x . PLBF divides the score space [0 , 1] into N equal segments and then appropriately clusters the N segments into k ( < N ) regions using dynamic programming (DP). Then, k backup Bloom filters with different FPRs are assigned to each region. We can obtain a better (closer to optimal) data structure with a larger N . However, PLBF builds DP tables O ( N ) times, and each DP table requires O ( N 2 k ) time complexity to build, so the total time complexity amounts to O ( N 3 k ) . Therefore, the construction time increases rapidly as N increases.

We propose fast PLBF, which is constructed much faster than PLBF. Fast PLBF constructs the same data structure as PLBF but with O ( N 2 k ) time complexity by omitting the redundant construction of DP tables. Furthermore, we propose fast PLBF++, which can be constructed faster than fast PLBF, with a time complexity of O ( Nk log N + Nk 2 ) . Fast PLBF++ accelerates DP table construction by taking advantage of a characteristic that DP tables often have. We proved that fast PLBF++ constructs the same data structure as PLBF when the probability of x ∈ S and the score s ( x ) are ideally well correlated. Our contributions can be summarized as follows.

We propose fast PLBF, which can construct the same data structure as PLBF with less time complexity.

We propose fast PLBF++, which can construct the data structure with even less time complexity than fast PLBF. Fast PLBF++ constructs the same data structure as PLBF when the probability of x ∈ S and the score s ( x ) are ideally well correlated.

Experimental results show that fast PLBF can construct the data structure up to 233 times faster than PLBF and achieves the same memory efficiency as PLBF.

Experimental results show that fast PLBF++ can construct the data structure up to 761 times faster than PLBF and achieves almost the same memory efficiency as PLBF.

2 Preliminaries: PLBF

First, we define terms to describe PLBF [14]. Let S be a set of elements for which the Bloom filter is to be built, and let Q be the set of elements not included in S that is used when constructing PLBF ( S ∩ Q = ∅ ). The elements included in S are called keys, and those not included are called non-keys. To build PLBF, a machine learning model is trained to predict whether a given element x is included in the set S or Q . For a given element x , the machine learning model outputs a score s ( x ) ∈ [0 , 1] . The score s ( x ) indicates 'how likely is x to be included in the set S .'

Next, we explain the design of PLBF. PLBF partitions the score space [0 , 1] into k regions and assigns backup Bloom filters with different FPRs to each region (Figure 1). Given a target overall FPR,

F ∈ (0 , 1) , we optimize t ∈ R k +1 and f ∈ R k to minimize the total memory usage. Here, t is a vector of thresholds for partitioning the score space into k regions, and f is a vector of FPRs for each region, satisfying t 0 = 0 < t 1 < · · · < t k = 1 and 0 < f i ≤ 1 ( i = 1 . . . k ) .

Next, we explain how PLBF finds the optimal t and f . PLBF divides the score space [0 , 1] into N ( > k ) segments and then finds the optimal t and f using DP. Deciding how to cluster N segments into k consecutive regions corresponds to determining the threshold t (Figure 2). We denote the probabilities that the key and non-key scores are contained in the i -th segment by g i and h i , respectively. After we cluster the segments (i.e., determine the thresholds t ), we denote the probabilities that the key and non-key scores are contained in the i -th region by G i and H i , respectively (e.g., g 1 + g 2 + g 3 = G 1 in Figure 2).

We can find the thresholds t that minimize memory usage by solving the following problem for each j = k . . . N (see the appendix for details); we find a way to cluster the 1 st to ( j -1) -th segments into k -1 regions while maximizing

PLBF solves this problem by building a j × k DPtable DP j KL [ p ][ q ] ( p = 0 . . . j -1 and q = 0 . . . k -1 ) for each j . DP j KL [ p ][ q ] denotes the maximum value of ∑ q i =1 G i log 2 ( G i H i ) one can get when you cluster the 1 st to p -th segments into q regions. To construct PLBF, one must find a clustering method that achieves DP j KL [ j -1][ k -1] . DP j KL can be computed recursively as follows:

where the function d KL ( i l , i r ) is the following function defined for integers i l and i r satisfying 1 ≤ i l ≤ i r ≤ N :

The time complexity to construct this DP table is O ( j 2 k ) . Then, by tracing the recorded transitions backward from DP j KL [ j -1][ k -1] , we obtain the best clustering with a time complexity of O ( k ) . As the DP table is constructed for each j = k . . . N , the overall complexity is O ( N 3 k ) . The pseudo-code for PLBF construction is provided in the appendix.

We can divide the score space more finely with a larger N and thus obtain a near-optimal t . However, the time complexity increases rapidly with increasing N .

3 Fast PLBF

We propose fast PLBF, which constructs the same data structure as PLBF more quickly than PLBF by omitting the redundant construction of DP tables. Fast PLBF uses the same design as PLBF and finds the best clustering (i.e., t ) and FPRs (i.e., f ) to minimize memory usage.

PLBF constructs a DP table for each j = k . . . N . We found that this computation is redundant and that we can also use the last DP table DP N KL for j = k . . . N -1 . This is because the maximum value of ∑ k -1 i =1 G i log 2 ( G i H i ) when clustering the 1 st to ( j -1) -th segments into k -1 regions is equal to DP N KL [ j -1][ k -1] . We can obtain the best clustering by tracing the transitions backward from DP N KL [ j -1][ k -1] . The time complexity of tracing the transitions is O ( k ) , which is faster than constructing the DP table.

This is a simple method, but it is a method that only becomes apparent after some organization on the optimization problem of PLBF. PLBF solves N -k +1 optimization problems separately, i.e., for each of j = k, . . . , N , the problem of 'finding the optimal t and f when the k -th region consists of

𝑝

0

1

2

3

…

𝑁 - 1

DP

!"

𝑝 [𝑞 - 1]

DP

!"

𝑝 [𝑞]

(a)

K

￥

(b)

𝑖

1

2

3

…

DP

!"

𝑖 - 1 [𝑞 - 1]

DP

!"

𝑝 [𝑞]

𝑝

1

2

3

…

-∞

-∞

-∞

-∞

-∞

-∞

-∞

-∞

-∞

-∞

𝐴 =

𝑁 - 1

𝑁 - 1

7

2

4

1

5

4

2

6

5

3

5

8

9

7

9

3

2

3

4

1

7

6

9

4

the j, . . . , N th segments' is solved separately. Fast PLBF, on the other hand, solves the problems for j = k, . . . , N -1 faster by reusing the computations in the problem for j = N . The reorganization of the problem in the appendix makes it clear that this reuse does not change the answer.

The pseudo-code for fast PLBF construction is provided in the appendix. The time complexity of building DP N KL is O ( N 2 k ) , and the worst-case complexity of subsequent computations is O ( Nk 2 ) . Because N > k , the total complexity is O ( N 2 k ) , which is faster than O ( N 3 k ) for PLBF, although fast PLBF constructs the same data structure as PLBF.

Fast PLBF extends the usability of PLBF. In any application of PLBF, fast PLBF can be used instead of PLBF because fast PLBF can be constructed quickly without losing the accuracy of PLBF. Fast PLBF has applications in a wide range of computing areas [15, 5], and is significantly superior in applications where construction is frequently performed. Fast PLBF also has the advantage of simplifying hyperparameter settings. When using PLBF, the hyperparameters N and k must be carefully determined, considering the trade-off between accuracy and construction speed. With fast PLBF, on the other hand, it is easy to determine the appropriate hyperparameters because the construction time is short enough, even if N and k are set somewhat large.

4 Fast PLBF++

We propose fast PLBF++, which can be constructed even faster than fast PLBF. Fast PLBF++ accelerates the construction of the DP table DP N KL by taking advantage of a characteristic that DP tables often have. When the transitions recorded in computing DP N KL [ p ][ q ] ( p = 1 . . . N -1) from DP N KL [ p ][ q -1] ( p = 0 . . . N -2) are represented by arrows, as in Figure 3, we find that the arrows rarely 'cross' (at locations other than endpoints). In other words, the transitions tend to have few intersections (Figure 3(a)) rather than many (Figure 3(b)). Fast PLBF++ takes advantage of this characteristic to construct the DP table with less complexity O ( Nk log N ) , thereby reducing the total construction complexity to O ( Nk log N + Nk 2 ) .

DP

!"

𝑝

𝑝 [𝑞 - 1]

DP

!"

𝑝 [𝑞]

0

1

2

3

…

𝑁 - 1

3

2

7

1

4

3

5

(a)

3

2

7

1

4

3

5

(b)

2

3

9

3

2

7

1

4

3

5

3

2

4

6

1

(c)

𝑝 + 1

𝑡

!"#

𝑖

1

2

𝑝

𝑔

$

ℎ

$

𝑡

!"#

𝑖

1

2

𝑝

𝑔

$

ℎ

$

(a)

𝑔

!

ℎ

!

ℎ

!

𝑝 + 1

𝑡

"#$

𝑖

1

2

𝑝

𝑡

"#$

𝑖

1

2

𝑝

𝑔

!

(b)

一

！

1

--

First, we define the terms to describe fast PLBF++. For simplicity, DP N KL is denoted as DP KL in this section. The ( N -1) × ( N -1) matrix A is defined as follows:

Then, from the definition of DP KL ,

The matrix A represents the intermediate calculations involved in determining DP KL [ p ][ q ] from DP KL [ p ][ q -1] (Figure 4).

Following Aggarwal et al. [16], we define the monotone matrix and matrix problem as follows.

Definition 4.1. Let B be an n × m real matrix, and we define a function J : { 1 . . . n } → { 1 . . . m } , where J ( i ) is the j ∈ { 1 . . . m } such that B ij is the maximum value of the i -th row of B . If there is more than one such j , let J ( i ) be the smallest. A matrix B is called a monotone matrix if J ( i 1 ) ≤ J ( i 2 ) for any i 1 and i 2 that satisfy 1 ≤ i 1 < i 2 ≤ n . Finding the maximum value of each row of a matrix is called a matrix problem .

An example of a matrix problem for a monotone matrix is shown in Figure 5. Solving the matrix problem for a general n × m matrix requires O ( nm ) time complexity because all matrix values must be checked. Meanwhile, if the matrix is known to be a monotone matrix , the matrix problem for this matrix can be solved with a time complexity of O ( n + m log n ) using the divide-and-conquer algorithm [16]. Here, the exhaustive search for the middle row and the refinement of the search range are repeated recursively (Figure 6).

We also define an ideal score distribution as follows.

Definition 4.2. A score distribution is ideal if the following holds:

An ideal score distribution implies that the probability of x ∈ S and the score s ( x ) are ideally well correlated. In other words, an ideal score distribution means that the machine learning model learns the distribution ideally.

'Few crossing transitions' in Figure 3 indicates that A is a monotone matrix or is close to it. It is somewhat intuitive that A is a monotone matrix or is close to it. This is because the fact that A is a monotone matrix implies that the optimal t q -1 does not decrease when the number of regions is fixed at q and the number of segments increases by 1 (Figure 7). It is intuitively more likely that t q -1 remains unchanged or increases, as in Figure 7(a), than that t q -1 decreases, as in Figure 7(b). Fast PLBF++ takes advantage of this insight to rapidly construct DP tables.

From Equation (5), determining DP KL [ p ][ q ] ( p = 1 . . . N -1) from DP KL [ p ][ q -1] ( p = 0 . . . N -2) is equivalent to solving the matrix problem of matrix A . When A is a monotone matrix , the divide-andconquer algorithm can solve this problem with O ( N log N ) time complexity. (The same algorithm

0.0

0.2

0.4

0.6

0.8

1.0

Score

10

3

10

4

10

5

Count

Malicious (key)

0.0

0.2

0.4

0.6

0.8

1.0

Score

10

2

10

3

10

4

10

5

Count

Benign (non-key)

(a) Malicious URLs Dataset

0.0

0.2

0.4

0.6

0.8

1.0

Score

10

3

10

4

10

5

Count

Malicious (key)

0.0

0.2

0.4

0.6

0.8

1.0

Score

10

3

10

4

10

5

Count

Benign (non-key)

(b) EMBER Dataset

0

200

400

600

800

1000

i

10

1

10

0

10

1

10

2

gi

/

hi

(a) Malicious URLs Dataset

0

200

400

600

800

1000

i

10

3

10

2

10

1

10

0

10

1

10

2

10

3

gi

/

hi

(b) EMBER Dataset

can obtain a not necessarily correct solution even if A is not a monotone matrix .) By computing DP KL [ p ][ q ] with this algorithm sequentially for q = 1 . . . k -1 , we can construct a DP table with a time complexity of O ( Nk log N ) . The worst-case complexity of the subsequent calculations is O ( Nk 2 ) , so the total complexity is O ( Nk log N + Nk 2 ) . This is the fast PLBF++ construction algorithm, and this is faster than fast PLBF, which requires O ( N 2 k ) computations.

Fast PLBF++ does not necessarily have the same data structure as PLBF because A is not necessarily a monotone matrix . However, as the following theorem shows, we can prove that A is a monotone matrix under certain conditions.

Theorem 4.3. If the score distribution is ideal, A is a monotone matrix.

The proof is given in the appendix. When the distribution is not ideal , matrix A is not necessarily a monotone matrix , but as mentioned above, it is somewhat intuitive that A is close to a monotone matrix . In addition, as will be shown in the next section, experiment results from real-world datasets whose distribution is not ideal show that fast PLBF++ is almost as memory efficient as PLBF.

5 Experiments

This section evaluates the experimental performance of fast PLBF and fast PLBF++. We compared the performances of fast PLBF and fast PLBF++ with four baselines: Bloom filter [1], Ada-BF [11], sandwiched learned Bloom filter (sandwiched LBF) [12], and PLBF [14]. Similar to PLBF, Ada-BF is an LBF that partitions the score space into several regions and assigns different FPRs to each region. However, Ada-BF relies heavily on heuristics for clustering and assigning FPRs. Sandwiched LBF is an LBF that 'sandwiches' a machine learning model with two Bloom filters. This achieves better memory efficiency than the original LBF by optimizing the size of two Bloom filters.

To facilitate the comparison of different methods or hyperparameters results, we have slightly modified the original PLBF framework. The original PLBF was designed to minimize memory usage under the condition of a given false positive rate. However, this approach makes it difficult to compare the results of different methods or hyperparameters. This is because both the false positive rate at test time and the memory usage vary depending on the method and hyperparameters, which often makes it difficult to determine the superiority of the results. Therefore, in our experiments, we used a framework where the expected false positive rate is minimized under the condition of memory usage. This approach makes it easy to obtain two results with the same memory usage and compare them by

Insert keys to Bloom filter

Train machine learning model

Find optimal parameters

100

150

200

Bloom filter

Ada-BF

Sandwiched

LBF

PLBF

Fast PLBF

(Proposed)

Fast PLBF++

(Proposed)

0

2

4

Construction Time [s]

(a) Malicious URLs Dataset

100

150

200

250

Bloom filter

Ada-BF

Sandwiched

LBF

PLBF

Fast PLBF

(Proposed)

Fast PLBF++

(Proposed)

0.0

2.5

5.0

7.5

Construction Time [s]

(b) EMBER Dataset

the false positive rate at test time. See the appendix for more information on how this framework modification will change the construction method of PLBFs.

Datasets : We evaluated the algorithms using the following two datasets.

Malicious URLs Dataset : As in previous papers [11, 14], we used Malicious URLs Dataset [17]. The URLs dataset comprises 223,088 malicious and 428,118 benign URLs. We extracted 20 lexical features such as URL length, use of shortening, number of special characters, etc. We used all malicious URLs and 342,482 (80%) benign URLs as the training set, and the remaining benign URLs as the test set.

EMBER Dataset : We used the EMBER dataset [18] as in the PLBF research. The dataset consists of 300,000 malicious and 400,000 benign files, along with the features of each file. We used all malicious files and 300,000 (75%) benign files as the train set and the remaining benign files as the test set.

While any model can be used for the classifier, we used LightGBM [19] because of its speed in training and inference, as well as its memory efficiency and accuracy. The sizes of the machine learning model for the URLs and EMBER datasets are 312 Kb and 1.19 Mb, respectively. The training time of the machine learning model for the URLs and EMBER datasets is 1.09 and 2.71 seconds, respectively. The memory usage of LBF is the total memory usage of the backup Bloom filters and the machine learning model. Figure 8 shows a histogram of each dataset's score distributions of keys and non-keys. We can see that the frequency of keys increases and that of non-keys decreases as the score increases. In addition, Figure 9 plots g i /h i ( i = 1 . . . N ) when N = 1 , 000 . We can see that g i /h i tends to increase as i increases, but the increase is not monotonic (i.e., the score distribution is not ideal ).

5.1 Construction time

We compared the construction times of fast PLBF and fast PLBF++ with those of existing methods. Following the experiments in the PLBF paper, hyperparameters for PLBF, fast PLBF, and fast PLBF++ were set to N = 1 , 000 and k = 5 .

Figure 10 shows the construction time for each method. The construction time for learned Bloom filters includes not only the time to insert keys into the Bloom filters but also the time to train the machine learning model and the time to compute the optimal parameters ( t and f in the case of PLBF).

Ada-BF and sandwiched LBF use heuristics to find the optimal parameters, so they have shorter construction times than PLBF but have worse accuracy. PLBF has better accuracy but takes more than 3 minutes to find the optimal t and f . On the other hand, our fast PLBF and fast PLBF++ take less than 2 seconds. As a result, Fast PLBF constructs 50.8 and 34.3 times faster than PLBF, and fast PLBF++ constructs 63.1 and 39.3 times faster than PLBF for the URLs and EMBER datasets, respectively. This is about the same construction time as sandwiched LBF, which relies heavily on heuristics and, as we will see in the next section, is much less accurate than PLBF.

Bloom filter

Ada-BF

Sandwiched LBF

PLBF

Fast PLBF (Proposed)

Fast PLBF++ (Proposed)

0.6

0.8

1.0

1.2

1.4

1.6

1.8

2.0

Total Memory Usage [Mbit]

10

3

10

2

10

1

False Positive Rate

(a) Malicious URLs Dataset

1.4

1.6

1.8

2.0

2.2

2.4

2.6

2.8

3.0

Total Memory Usage [Mbit]

10

3

10

2

10

1

False Positive Rate

(b) EMBER Dataset

PLBF

Fast PLBF (Proposed)

Fast PLBF++ (Proposed)

(a) Malicious URLs Dataset

(b) EMBER Dataset

5.2 Memory Usage and FPR

We compared the trade-off between memory usage and FPR for fast PLBF and fast PLBF++ with Bloom filter, Ada-BF, sandwiched LBF, and PLBF. Following the experiments in the PLBF paper, hyperparameters for PLBF, fast PLBF, and fast PLBF++ were always set to N = 1 , 000 and k = 5 .

Figure 11 shows each method's trade-off between memory usage and FPR. PLBF, fast PLBF, and fast PLBF++ have better Pareto curves than the other methods for all datasets. Fast PLBF constructs the same data structure as PLBF in all cases, so it has exactly the same accuracy as PLBF. Fast PLBF++ achieves almost the same accuracy as PLBF. Fast PLBF++ has up to 1.0019 and 1.000083 times higher false positive rates than PLBF for the URLs and EMBER datasets, respectively.

5.3 Ablation study for hyper-parameters

The parameters of the PLBFs are memory size, N , and k . The memory size is specified by the user, and N and k are hyperparameters that are determined by balancing construction time and accuracy. In the previous sections, we set N to 1,000 and k to 5, following the original paper on PLBF. In this section, we perform ablation studies for these hyperparameters to confirm that our proposed methods can construct accurate data structures quickly, no matter what hyperparameter settings are used. We also confirm that the accuracy tends to be better and the construction time increases as N and k are increased, and that the construction time of the proposed methods increases much slower than PLBF.

Figure 12 shows the construction time and false positive rate with various N while the memory usage of the backup Bloom filters is fixed at 500 Kb and k is fixed at 5. For all three PLBFs, the false positive rate tends to decrease as N increases. (Note that this is the false positive rate on test data, so it does not necessarily decrease monotonically. The appendix shows that the expected value of the false positive rate calculated using training data decreases monotonically as N increases.) Also, as N increases, the PLBF construction time increases rapidly, but the fast PLBF construction time increases much more slowly than that, and for fast PLBF++, the construction time changes little. This is because the construction time of PLBF is asymptotically proportional to N 3 , while that of fast PLBF and fast PLBF++ is proportional to N 2 and N log N , respectively. The experimental results

PLBF

Fast PLBF (Proposed)

Fast PLBF++ (Proposed)

(a) Malicious URLs Dataset

(b) EMBER Dataset

show that the two proposed methods can achieve high accuracy without significantly changing the construction time with large N .

Figure 13 shows the construction time and false positive rate with various k while the backup bloom filter memory usage is fixed at 500 Kb and N is fixed at 1,000. For all three PLBFs, the false positive rate tends to decrease as k increases. For the EMBER dataset, the false positive rate stops decreasing at about k = 20 , while for the URLs dataset, it continues to decrease even at about k = 500 . (Just as in the case of experiments with varying N , this decrease is not necessarily monotonic.) In addition, the construction times of all three PLBFs increase proportionally to k , but fast PLBF has a much shorter construction time than PLBF, and fast PLBF++ has an even shorter construction time than fast PLBF. When k = 50 , fast PLBF constructs 233 and 199 times faster than PLBF, and fast PLBF++ constructs 761 and 500 times faster than PLBF for the URLs and EMBER datasets, respectively. The experimental results indicate that by increasing k , the two proposed methods can achieve high accuracy without significantly affecting the construction time.

6 Related Work

Approximate membership query is a query that asks whether the query q is contained in the set S while allowing for false positives with a small probability ε . One can prove that at least |S| log 2 ( 1 ε ) bits of memory must be used to answer the approximate membership query if the elements in the set or queries are selected with equal probability from the universal set [20].

A Bloom filter [1] is one of the most basic data structures for approximate membership queries. It compresses the set S into a bit string using hash functions. This bit string and the hash functions are then used to answer the queries. To achieve an FPR of ε , a Bloom filter requires |S| log 2 ( 1 ε ) log 2 e bits of memory. This is log 2 e times the theoretical lower bound.

Various derivatives have been proposed that achieve better memory efficiency than the original Bloom filter. The cuckoo filter [21] is more memory efficient than the original Bloom filter and supports dynamic addition and removal of elements. Pagh et al. [22] proposed a replacement for Bloom filter that achieves a theoretical lower bound. Various other memory-efficient filters exist, including the vacuum filter [23], xor filter [24], and ribbon filter [25]. However, these derivatives do not consider the structure of the distribution and thus cannot take advantage of it.

Kraska et al.[10] proposed using a machine learning model as a prefilter of a backup Bloom filter. AdaBF [11] extended this design and proposed to exploit the scores output by the machine learning model. PLBF [14] uses a design similar to that of Ada-BF but introduces fewer heuristics for optimization than Ada-BF. Mitzenmacher [12] proposed an LBF that 'sandwiches' a machine learning model with two Bloom filters. This achieves better memory efficiency than the original LBF but can actually be interpreted as a special case of PLBF.

7 Limitation and Future Work

As explained in Section 4, it is somewhat intuitive that A is a monotone matrix or close to it, so it is also intuitive that fast PLBF++ achieves accuracy close to PLBF. Experimental results on the URLs and EMBER datasets also suggest that fast PLBF++ achieves almost the same accuracy as PLBF,

even when the score distribution is not ideal . This experimental rule is further supported by the results of the artificial data experiments described in the appendix. However, there is no theoretical support for the accuracy of fast PLBF++. Theoretical support for how fast PLBF++ accuracy may degrade relative to PLBF is a future issue.

Besides, it is possible to consider faster methods by making stronger assumptions than fast PLBF++. Fast PLBF++ assumes the monotonicity of matrix A . We adopted this assumption because matrix A is proved to be monotone under intuitive assumptions about the data distribution. However, by assuming stronger assumptions about matrix A , the computational complexity of the construction could be further reduced. For example, if matrix A is totally monotone , the matrix problem for matrix A can be solved in O ( N ) using algorithms such as [16, 26]. Using such existing DP algorithms is a promising direction toward even faster or more accurate methods, and is a future work.

8 Conclusion

PLBF is an outstanding LBF that can effectively utilize the distribution of the set and queries captured by a machine learning model. However, PLBF is computationally expensive to construct. We proposed fast PLBF and fast PLBF++ to solve this problem. Fast PLBF is superior to PLBF because fast PLBF constructs exactly the same data structure as PLBF but does so faster. Fast PLBF++ is even faster than fast PLBF and achieves almost the same accuracy as PLBF and fast PLBF. These proposed methods have greatly expanded the range of applications of PLBF.

Acknowledgments and Disclosure of Funding

We thank the anonymous reviewers for their constructive comments. This work was supported by JST AIP Acceleration Research JPMJCR23U2, Japan.

References

Burton H Bloom. Space/time trade-offs in hash coding with allowable errors. Communications of the ACM , 13(7):422-426, 1970.

Andrei Broder and Michael Mitzenmacher. Network applications of bloom filters: A survey. Internet mathematics , 1(4):485-509, 2004.

Sasu Tarkoma, Christian Esteve Rothenberg, and Eemil Lagerspetz. Theory and practice of bloom filters for distributed systems. IEEE Communications Surveys & Tutorials , 14(1):131155, 2011.

Shahabeddin Geravand and Mahmood Ahmadi. Bloom filter applications in network security: A state-of-the-art survey. Computer Networks , 57(18):4047-4064, 2013.

Fay Chang, Jeffrey Dean, Sanjay Ghemawat, Wilson C Hsieh, Deborah A Wallach, Mike Burrows, Tushar Chandra, Andrew Fikes, and Robert E Gruber. Bigtable: A distributed storage system for structured data. ACM Transactions on Computer Systems (TOCS) , 26(2):1-26, 2008.

Michael T Goodrich and Michael Mitzenmacher. Invertible bloom lookup tables. In Allerton Conference on Communication, Control, and Computing (Allerton) , 2011.

Guanlin Lu, Young Jin Nam, and David HC Du. Bloomstore: Bloom-filter based memoryefficient key-value store for indexing of data deduplication on flash. In IEEE Symposium on Mass Storage Systems and Technologies (MSST) , 2012.

Mike Hearn and Matt Corallo. Bips: Connection bloom filtering, 2012. URL https://github.com/bitcoin/bips/blob/master/bip-0037 [Online; accessed 22-December-2022] , 2012.

Jongbeen Han, Mansub Song, Hyeonsang Eom, and Yongseok Son. An efficient multi-signature wallet in blockchain using bloom filter. In Proceedings of the ACM Symposium on Applied Computing , 2021.

Tim Kraska, Alex Beutel, Ed H Chi, Jeffrey Dean, and Neoklis Polyzotis. The case for learned index structures. In Proceedings of the International Conference on Management of Data , 2018.

Zhenwei Dai and Anshumali Shrivastava. Adaptive learned bloom filter (ada-bf): Efficient utilization of the classifier with application to real-time information filtering on the web. In Advances in Neural Information Processing Systems , 2020.

Michael Mitzenmacher. A model for learned bloom filters and optimizing by sandwiching. In Advances in Neural Information Processing Systems , 2018.

Jack Rae, Sergey Bartunov, and Timothy Lillicrap. Meta-learning neural bloom filters. In International Conference on Machine Learning , 2019.

Kapil Vaidya, Eric Knorr, Michael Mitzenmacher, and Tim Kraska. Partitioned learned bloom filters. In International Conference on Learning Representations , 2021.

Yi-Hsuan Feng, Nen-Fu Huang, and Chia-Hsiang Chen. An efficient caching mechanism for network-based url filtering by multi-level counting bloom filters. In 2011 IEEE International Conference on Communications (ICC) , 2011.

Alok Aggarwal, Maria M Klawe, Shlomo Moran, Peter Shor, and Robert Wilber. Geometric applications of a matrix-searching algorithm. Algorithmica , 2(1):195-208, 1987.

Manu Siddhartha. Malicious urls dataset | kaggle. URL https://www.kaggle.com/datasets/sid321axn/malicious-urls-dataset [Online; accessed 22-December-2022] , 2021.

Hyrum S Anderson and Phil Roth. Ember: an open dataset for training static pe malware machine learning models. arXiv preprint arXiv:1804.04637 , 2018.

Guolin Ke, Qi Meng, Thomas Finley, Taifeng Wang, Wei Chen, Weidong Ma, Qiwei Ye, and Tie-Yan Liu. Lightgbm: A highly efficient gradient boosting decision tree. In Advances in Neural Information Processing Systems , 2017.

Larry Carter, Robert Floyd, John Gill, George Markowsky, and Mark Wegman. Exact and approximate membership testers. In Proceedings of the ACM symposium on Theory of computing , 1978.

Bin Fan, Dave G Andersen, Michael Kaminsky, and Michael D Mitzenmacher. Cuckoo filter: Practically better than bloom. In Proceedings of the ACM International Conference on emerging Networking Experiments and Technologies , 2014.

Anna Pagh, Rasmus Pagh, and S Srinivasa Rao. An optimal bloom filter replacement. In Proceedings of the ACM-SIAM Symposium on Discrete Algorithms , 2005.

Minmei Wang, Mingxun Zhou, Shouqian Shi, and Chen Qian. Vacuum filters: More spaceefficient and faster replacement for bloom and cuckoo filters. Proceedings of the VLDB Endowment , 13(2):197-210, 2019.

Thomas Mueller Graf and Daniel Lemire. Xor filters: Faster and smaller than bloom and cuckoo filters. Journal of Experimental Algorithmics (JEA) , 25:1-16, 2020.

Peter C. Dillinger and Stefan Walzer. Ribbon filter: practically smaller than bloom and xor. arXiv preprint arXiv:2103.02515 , 2021.

Zvi Galil and Kunsoo Park. A linear-time algorithm for concave one-dimensional dynamic programming. Information Processing Letters , 33(6):309-311, 1989.

Johan Ludwig William Valdemar Jensen. Sur les fonctions convexes et les inégalités entre les valeurs moyennes. Acta mathematica , 30(1):175-193, 1906.

Appendices

Appendix A details the optimization problem designed in the original PLBF paper [14] and its solution. It includes a comprehensive analysis of the optimization problem and a detailed description of how PLBF's method leads (under certain assumptions) to a solution to the optimization problem, which are not provided in [14]. Appendix B presents details of PLBF and the fast PLBF algorithm along with pseudo-code. Appendix C gives the proof of Theorem 4.3, a theorem about the accuracy of fast PLBF++. In Appendix D, we explain the trivial modifications to the PLBF framework that we made for the experiments. Appendix E describes a detailed ablation study of the PLBFs hyperparameters, N and k . Appendix F describes the experiments on the simplified solution method presented in the PLBF paper. Appendix G describes experiments with artificial datasets to evaluate the experimental performance of fast PLBF++ in detail.

Appendix A Solution of the optimization problems

In the original PLBF paper [14], the analysis with mathematical expressions was conducted only for the relaxed problem, while no analysis utilizing mathematical expressions was performed for the general problem. Consequently, it was unclear which calculations were redundant, leading to the repetition of constructing similar DP tables. Hence, this appendix provides a comprehensive analysis of the general problem. It presents the optimal solution and the corresponding value of the objective function using mathematical expressions. This analysis uncovers redundant calculations, enabling the derivation of fast PLBF.

The optimization problem designed by PLBF [14] can be written as follows:

The objective function represents the total memory usage of the backup Bloom filters. The first constraint equation represents the condition that the expected overall FPR is below F . c ≥ 1 is a constant determined by the type of backup Bloom filters. G and H are determined by t . Once t is fixed, we can solve this optimization problem to find the optimal f and the minimum value of the objective function. In the original PLBF paper [14], the condition f i ≤ 1 ( i = 1 . . . k ) was relaxed; however, here we conduct the analysis without relaxation. We assume G i > 0 , H i > 0 ( i = 1 . . . k ) and |S| > 0 .

The Lagrange function is defined using the Lagrange multipliers µ ∈ R k and ν ∈ R , as follows:

From the KKT condition, there exist ¯ µ and ¯ ν in the local optimal solution ¯ f of this optimization problem, and the following holds:

̸

Wedefine I f =1 and I f< 1 as I f =1 = { i | ¯ f i = 1 } and I f< 1 = { i | ¯ f i < 1 } . I f =1 ∪I f< 1 = { 1 . . . k } and I f =1 ∩I f< 1 = ∅ are satisfied. Furthermore, we assume that I f< 1 = ∅ . This means that there is at least one region that uses a backup Bloom filter with an FPR smaller than 1 .

By introducing I f =1 and I f< 1 , Equations (9, 10, 11) can be organized as follows:

̸

Here, we get ¯ ν > 0 . This is because if we assume ¯ ν = 0 , Equation (13) implies that G i = 0 for i ∈ I f< 1 ( = ∅ ) , which contradicts the assumption that G i > 0 . From ¯ ν > 0 and Equation (13),

From Equations (14, 15) and ¯ ν > 0 ,

Substituting Equations (15, 17) into the objective function of the optimization problem (7), we obtain

We define G f =1 and H f =1 as G f =1 = ∑ i ∈I f =1 G i and H f =1 = ∑ i ∈I f =1 H i , respectively.

PLBF makes the following assumption:

Assumption A.1. For optimal t and f , I f =1 = ∅ or I f =1 = { k } .

In other words, PLBF assumes that there is at most one region for which f = 1 , and if there is one, it is the region with the highest score. PLBF then tries all possible thresholds t k -1 (under the division of the score space).

In the following, we discuss the optimal f and t under the assumption that the j -th to N -th segments are clustered as the k -th region ( j ∈ { k . . . N } ).

In the case of I f =1 = ∅ , because G f =1 = H f =1 = 0 , Equation (19) becomes

Meanwhile, in the case of I f =1 = { k } , Equation (19) becomes

In both cases, the terms other than the last one are constants under the assumption that the j -th to N -th segments are clustered as the k -th region. Therefore, the value of the objective function is minimized when the 1 st to ( j -1) -th segments are clustered in a way that maximizes ∑ k -1 i =1 G i log 2 ( G i H i ) . Thus, the problem of finding t that minimizes memory usage can be reduced to the following problem: for each j = k . . . N , we find a way to cluster the 1 st to ( j -1) -th segments into k -1 regions that maximizes ∑ k -1 i =1 G i log 2 ( G i H i ) .

Algorithm 1 PLBF [14]

Appendix B Algorithm details

In this appendix, we explain the algorithms for PLBF and the proposed method using pseudo-code in detail. This will help to clarify how each method differs from the others.

First, we show the PLBF algorithm in Algorithm 1. For details of OPTIMALFPR and SPACEUSED, please refer to Appendix B and Equation (2) in the original PLBF paper [14], respectively. The worst case time complexities of OPTIMALFPR and SPACEUSED are O ( k 2 ) and O ( k ) , respectively. As the DP table is constructed with a time complexity of O ( j 2 k ) for each j = k . . . N , the overall complexity is O ( N 3 k ) .

Next, we show the fast PLBF algorithm in Algorithm 2. The time complexity of building DP N KL is O ( N 2 k ) , and the worst-case complexity of subsequent computations is O ( Nk 2 ) . Because N > k , the total complexity is O ( N 2 k ) , which is faster than O ( N 3 k ) for PLBF, although fast PLBF constructs the same data structure as PLBF.

Finally, we describe the fast PLBF++ algorithm. The fast PLBF++ algorithm is nearly identical to fast PLBF, but it calculates approximated DP N KL with a computational complexity of O ( Nk log N ) . Since the remaining calculations are unchanged from fast PLBF, the overall computational complexity is O ( Nk log N + Nk 2 ) .

Appendix C Proof of Theorem 4.3

In this appendix, we present a proof of Theorem 4.3. It demonstrates that fast PLBF++ can attain the same accuracy as PLBF and fast PLBF under certain conditions.

The following lemma holds.

Algorithm 2 Fast PLBF

Lemma C.1. Let u 1 , u 2 , v 1 , and v 2 be real numbers satisfying

Furthermore, we define the function D ( x, y ) for real numbers x > 0 and y > 0 as follows:

Proof.

Therefore,

From Equations (22, 23, 24), when 0 < x , 0 < y , and x y ≥ u 1 v 1 , we obtain

Thus,

The transformation to Equation (34) is due to Jensen's inequality [27]. This shows that if x y ≥ u 1 v 1 , then D ( x, y ) ≥ 0 .

Proof. We prove Theorem 4.3 by contradiction.

Assume that the ( N -1) × ( N -1) matrix A is not a monotone matrix . That is, we assume that there exists p ( 1 ≤ p ≤ N -2 ) such that J ( p ) > J ( p +1) . Let J ( i ) be the smallest j such that A ij equals the maximum of the i -th row of A . We define a := J ( p ) and a ′ := J ( p +1) ( a > a ′ from the assumption for contradiction).

From the definitions of A and J ( i ) , we obtain the following:

Using Equations (35, 36, 38), we evaluate the right side of Equation (37) minus the left side of Equation (37).

Equations (35, 36) are used for the transformation to Equation (40), and Equation (42) is used for the transformation to Equation (38).

We define u 1 , u 2 , v 1 , v 2 , x , and y as follows:

From the conditions of ideal score distribution and a ′ < a , the followings hold:

0

.

(51)

Algorithm 3 OptimalFPR_F (Algorithm 1 in [14])

Input:

G ∈ R N : probabilities that the keys are contained in each region

H ∈ R N

: probabilities that the non-keys are contained in each region

F ∈ (0 , 1)

: target overall FPR

k ∈ N

: number of regions

Output:

f ∈ R k : FPRs of each region

return f

Then, from Equations (39 . . . 43), we obtain

However, this contradicts Equation (37). Hence, Theorem 4.3 is proved.

Appendix D Modification of the PLBF framework

In this appendix, we describe the framework modifications we made to PLBF in our experiments. In the original PLBF paper [14], the optimization problem was designed to minimize the amount of memory usage under a given target false positive rate (Equation (7)). However, this framework makes it difficult to compare the results of different methods and hyperparameters. Therefore, we designed the following optimization problem, which minimizes the expected false positive rate under a given memory usage condition:

where M is a parameter that is set by the user to determine the upper bound of memory usage and is set by the user.

Analyzing as in Appendix A, we find that in order to find the optimal thresholds t , we need to find a way to cluster the 1 st to ( j -1) -th segments into k -1 regions that maximizes ∑ k -1 i =1 G i log 2 ( G i H i ) for each j = k . . . N . We can compute this using the same DP algorithm as in the original framework.

Algorithm 4 OptimalFPR_M

Input:

G ∈ R N : probabilities that the keys are contained in each region H ∈ R N : probabilities that the non-keys are contained in each region M ∈ R : upper bound of the total memory usage of backup Bloom filters k ∈ N : number of regions

Output:

f ∈ R k : FPRs of each region

Also similar to Appendix A, introducing I f =1 and I f< 1 for analysis, it follows that the optimal false positive rates ¯ f is

where

In the original framework, the sets I f =1 and I f< 1 are obtained by repeatedly solving the relaxed problem and setting f i to 1 for regions where f i > 1 (for details of this algorithm, please refer to the Section 3.3.3 in the original PLBF paper [14]). In our framework, we can find the sets I f =1 and I f< 1 in the same way.

The pseudo-code for OPTIMALFPR_F in the original PLBF paper [14] and OPTIMALFPR_M, which is conditioned on the memory usage M , is shown in Algorithm 3 and Algorithm 4. For both functions, the best-case complexity is O ( k ) , and the worst-case complexity is O ( k 2 ) .

Also, the fast PLBF algorithm for the case conditioned by memory usage is shown in Algorithm 5. It is basically the same as the original PLBF Algorithm 2, but instead of using the SPACEUSED function, the EXPECTEDFPR function is used here. Then, the algorithm selects the case when the expected overall false positive rate calculated using the training data is minimized.

Algorithm 5

Appendix E Additional ablation study for hyperparameters

This appendix details the ablation studies for the hyperparameters N and k . In this appendix, we discuss the results of the ablation studies that were not presented in Section 5.3. The results confirm that the expected false positive rate decreases monotonically as the hyperparameters N and k increase. We also confirm that the proposed methods are superior to PLBF in terms of hyperparameter determination.

First, we confirm that the expected false positive rate decreases monotonically as N or k increases. The expected false positive rate is computed using training data (objective function in Equation (53)). Figure 14 shows the expected false positive rate at various N and k . The ablation study for N is done with k fixed to 5. The ablation study for k is done with N fixed to 1,000.

The expected false positive rate decreases monotonically as N or k increases for all datasets and methods. However, as observed in Section 5.3, the false positive rate at test time does not necessarily decrease monotonically as N or k increases. Also, the construction time increases as N or k increases. Therefore, it is necessary to set appropriate N and k when practically using PLBFs.

Therefore, we next performed a comprehensive experiment at various N and k . Figure 15 shows the construction time and false positive rate for each method when the memory usage of the backup bloom filter is fixed at 500 Kb, N is set to 8 , 16 , . . . , 1024 , and k is set to 3 , 5 , 10 , and 20 . The curves connect the same k and different N results.

The results show that the proposed methods are superior to PLBF because it is easier to set hyperparameters to construct accurate data structures in a shorter time. PLBF can achieve the same level of accuracy with the same construction time as fast PLBF and fast PLBF++ if the hyperparameters N and k are set well. However, it is not apparent how N and k should be set when applying the algorithm to unseen data. If N or k is set too large, the construction time will be very long, and if N or k is set too small, the false positive rate will be large. The proposed methods can construct the data structure quickly even if the hyperparameters N and k are too large. Therefore, by using the proposed methods instead of PLBF and setting N and k relatively large, it is possible to quickly and reliably construct a data structure with reasonable accuracy.

Furthermore, we performed an ablation study using fast PLBF to obtain guidelines for setting the hyperparameters N and k (Figure 16). We observed false positive rates for N =

PLBF

Fast PLBF (Proposed)

Fast PLBF++ (Proposed)

(a) Malicious URLs Dataset.

(b) EMBER Dataset.

20 , 50 , 100 , 200 , 300 , . . . , 1500 and k = 50 , 100 , 200 , 300 , . . . , 1400 with k < N . For the Malicious URLs Dataset, the false positive rate was lowest when N = 1300 and k = 700 (in this case, the fast PLBF construction is 333 times faster than PLBF). And for the Ember dataset, the false positive rate was lowest when N = 500 and k = 50 (in this case, the fast PLBF construction is 45 times faster than PLBF). The Malicious URLs Dataset showed an almost monotonous decrease in the false positive rate as N and k increased, while the Ember Dataset showed almost no such trend. This is thought to be due to a kind of overlearning that occurs as N and k are increased in the Ember Dataset. Although a method for determining the appropriate N and k has not yet been established, we suggest setting N ∼ 1000 and k ∼ 100 . This is because the false positive rate is considered to be small enough at this level, and the proposed method can be constructed in a few tens of seconds. In any case, the proposed method is useful because it is faster than the original PLBF for any hyperparameters N and k .

Appendix F Experiments on PLBF solution to relaxed problem

The PLBF paper [14] proposes two methods: a simple method and a complete method. In this appendix, we conducted experiments on the simple method and evaluated the difference in accuracy compared to the complete method. The results revealed that, in certain cases, this method exhibited significantly lower accuracy than the complete method. It was then confirmed that fast PLBF and fast PLBF++ are superior to the simple method in terms of the trade-off between construction time and accuracy.

The simpler method, described in section 3.3.3 of [14], does not consider the condition that f i ≤ 1 ( i = 1 . . . k ) when finding the optimal t . In other words, this method solves the relaxed problem

PLBF

Fast PLBF (Proposed)

Fast PLBF++ (Proposed)

𝑘 = 3

𝑘 = 3

𝑘 = 3

𝑘 = 5

𝑘 = 10

𝑘 = 20

𝑘 = 5

𝑘 = 5

𝑘 = 20

𝑘 = 10

𝑘 = 20

𝑘 = 10

(a) Malicious URLs Dataset

𝑘 = 3

𝑘 = 3

𝑘 = 3

𝑘 = 5

𝑘 = 10

𝑘 = 20

𝑘 = 5

𝑘 = 5

𝑘 = 20

𝑘 = 10

𝑘 = 20

𝑘 = 10

(b) EMBER Dataset

that does not consider the condition f i ≤ 1 ( i = 1 . . . k ) . The complete method uses the first method repeatedly as a subroutine to solve the general problem and find the optimal t . Here, the time complexity of the method for the relaxed problem is O ( N 2 k ) , and that of the method for the general problem is O ( N 3 k ) .

We experimented with the simpler method. Figure 17 shows the construction time (the hyperparameters for PLBFs were set to N = 1 , 000 and k = 5 ). The PLBF for the relaxed problem has about the same construction time as our proposed fast PLBF. This is because the time complexity for constructing the two methods is O ( N 2 k ) for both. Figure 18 shows the trade-off between memory usage and FPR. On the URL dataset, the method for the relaxed problem is almost as accurate as the method for the general problem. At most, it has a false positive rate of only 1.06 times greater. On the other hand, on the EMBER dataset, the method for the relaxed problem is significantly less accurate than the method for the general problem, especially when memory usage is small. It has up to 1.48 times higher false positive rate than the complete method.

In summary, the method for the relaxed problem can construct the data structure faster than the method for the general problem, but the accuracy can be significantly worse, especially when memory usage is small. Our proposed fast PLBF has the same accuracy as the complete PLBF in almost the same construction time as the method for the relaxed problem. Our fast PLBF++ can construct the data structure in even less time than the method for the relaxed problem, and the accuracy degradation is relatively small.

50

100

200

300

400

500

600

700

800

900

1000

1100

1200

1300

1400

1500

N

20

50

100

200

300

400

500

600

700

800

900

1000

1100

1200

1300

1400

k

0.0124

0.0126

0.0128

0.0130

0.0132

0.0134

False Positive Rate

(a) Malicious URLs Dataset

50

100

200

300

400

500

600

700

800

900

1000

1100

1200

1300

1400

1500

N

20

50

100

200

300

400

500

600

700

800

900

1000

1100

1200

1300

1400

k

0.0083

0.0084

0.0085

0.0086

0.0087

False Positive Rate

(b) EMBER Dataset

Appendix G Experiments with artificial datasets

We experimented with artificial datasets. Fast PLBF++ is faster to construct than fast PLBF. However, fast PLBF++ is not theoretically guaranteed to be as accurate as PLBF, except when the score distribution is 'ideally monotonic.' Therefore, we evaluated the accuracy of fast PLBF++ by creating a variety of datasets, ranging from data with monotonicity to data with little monotonicity. The results show that the smaller the monotonicity of the score distribution, the larger the difference in accuracy between fast PLBF++ and PLBF. We also observed that in most cases, the false positive rate for fast PLBF++ is within 1.1 times that of PLBF, but in cases where there is little monotonicity in the score distribution, the false positive rate of PLBF++ can be up to 1.85 times that of PLBF.

Here, we explain the process of creating an artificial dataset, which consists of two steps. First, as in the original PLBF paper [14], the key and non-key score distribution is generated using the Zipfian distribution. Figure 19(a) shows a histogram of the distribution of key and non-key scores at this time, and Figure 20(a) shows g i /h i ( i = 1 . . . N ) when N = 1 , 000 . This score distribution is ideally monotonic. Next, we perform swaps to add non-monotonicity to the score distribution. Swap refers to changing the scores of the elements in the two adjacent segments so that the number of keys and non-keys in the two segments are swapped. Namely, an integer i is randomly selected from { 1 , 2 , . . . , N -1 } , and the scores of elements in the i -th segment are changed so that they are included in the ( i +1) -th segment, and the scores of elements in the ( i +1) -th segment are changed to include them in the i -th segment. Figures 19(b) to 19(i) and Figures 20(b) to 20(i) show the histograms of the score distribution and g i /h i ( i = 1 . . . N ) for 10 , 10 2 , . . . , 10 8 swaps, respectively. It can be seen that as the number of swaps increases, the score distribution becomes more non-monotonic. For each case of the number of swaps, 10 different datasets were created using 10 different seeds.

Figure 21 shows the accuracy of each method for each number of swaps, with the seed set to 0. Hyperparameters for PLBFs are set to N = 1 , 000 and k = 5 . It can be seen that fast PLBF and fast PLBF++ achieve better Pareto curves than the other methods for all datasets. It can also be seen that the higher the number of swaps, the more often there is a difference between the accuracy of fast PLBF++ and fast PLBF.

Figure 22 shows the difference in accuracy between fast PLBF++ and fast PLBF for each swap count. Here, the 'relative false positive rate' is the false positive rate of fast PLBF++ divided by that of PLBF constructed with the same hyperparameters and conditions (note that fast PLBF constructs the same data structure as PLBF, so the false positive rate of fast PLBF is the same as that of PLBF). We created 10 different datasets for each swap count and conducted 6 experiments for each dataset and method with a memory usage of 0.25Mb, 0.5Mb, 0.75Mb, 1.0Mb, 1.25Mb, and 1.5Mb. Namely, we compared the false positive rates of fast PLBF and fast PLBF++ under 60 conditions for each swap count. The result shows that fast PLBF++ consistently achieves the same accuracy as PLBF in a total of 240 experiments where the number of swaps is 10 3 or less. It also shows that in the experiments where the number of swaps is 10 7 or less, the false positive rate of fast PLBF++ is less than 1.1 times

Insert keys to Bloom filter

Train machine learning model

Find optimal parameters

100

200

PLBF

PLBF

(for relaxed problem)

Fast PLBF

(Proposed)

Fast PLBF++

(Proposed)

0

2

4

Construction Time [s]

(a) Malicious URLs Dataset

100

200

PLBF

PLBF

(for relaxed problem)

Fast PLBF

(Proposed)

Fast PLBF++

(Proposed)

0.0

2.5

5.0

7.5

Construction Time [s]

(b) EMBER Dataset

that for PLBF, except for 14 cases. However, in the cases of 10 8 swap counts (where there is almost no monotonicity in the score distribution), the false positive rate for fast PLBF++ is up to 1.85 times that for PLBF.

Bloom filter

PLBF (for relaxed problem)

Ada-BF

Fast PLBF (Proposed)

Sandwiched LBF

Fast PLBF++ (Proposed)

PLBF

0.6

0.8

1.0

1.2

1.4

1.6

1.8

2.0

Total Memory Usage [Mbit]

10

3

10

2

10

1

False Positive Rate

(a) Malicious URLs Dataset

1.4

1.6

1.8

2.0

2.2

2.4

2.6

2.8

3.0

Total Memory Usage [Mbit]

10

3

10

2

10

1

False Positive Rate

(b) EMBER Dataset

0.0

0.2

0.4

0.6

0.8

1.0

Score

10

2

10

3

10

4

Count

Positive

0.0

0.2

0.4

0.6

0.8

1.0

Score

10

2

10

3

10

4

Count

Negative

(a)

0

swaps (score distribution is ideally monotonic)

0.0

0.2

0.4

0.6

0.8

1.0

Score

10

2

10

3

10

4

Count

Positive

0.0

0.2

0.4

0.6

0.8

1.0

Score

10

2

10

3

10

4

Count

Negative

(b)

10

swaps

0.0

0.2

0.4

0.6

0.8

1.0

Score

10

2

10

3

10

4

Count

Positive

0.0

0.2

0.4

0.6

0.8

1.0

Score

10

2

10

3

10

4

Count

Negative

(c)

10

2

swaps

0.0

0.2

0.4

0.6

0.8

1.0

Score

10

2

10

3

10

4

Count

Positive

0.0

0.2

0.4

0.6

0.8

1.0

Score

10

2

10

3

10

4

Count

Negative

(d)

10

3

swaps

0.0

0.2

0.4

0.6

0.8

1.0

Score

10

2

10

3

10

4

Count

Positive

0.0

0.2

0.4

0.6

0.8

1.0

Score

10

2

10

3

10

4

Count

Negative

(e)

10

4

swaps

0.0

0.2

0.4

0.6

0.8

1.0

Score

10

2

10

3

10

4

Count

Positive

0.0

0.2

0.4

0.6

0.8

1.0

Score

10

2

10

3

10

4

Count

Negative

(f)

10

5

swaps

0.0

0.2

0.4

0.6

0.8

1.0

Score

10

2

10

3

10

4

Count

Positive

0.0

0.2

0.4

0.6

0.8

1.0

Score

10

2

10

3

10

4

Count

Negative

(g)

10

6

swaps

0.0

0.2

0.4

0.6

0.8

1.0

Score

10

3

10

4

Count

Positive

0.0

0.2

0.4

0.6

0.8

1.0

Score

10

3

10

4

Count

Negative

(h)

10

7

swaps

0.0

0.2

0.4

0.6

0.8

1.0

Score

10

3

10

4

Count

Positive

0.0

0.2

0.4

0.6

0.8

1.0

Score

10

3

10

4

Count

Negative

(i)

10

8

swaps

0

200

400

600

800

1000

i

10

3

10

2

10

1

10

0

10

1

10

2

10

3

gi

/

hi

(a)

0

swaps (score distribution is ideally monotonic)

0

200

400

600

800

1000

i

10

3

10

2

10

1

10

0

10

1

10

2

10

3

gi

/

hi

(b)

10

swaps

0

200

400

600

800

1000

i

10

3

10

2

10

1

10

0

10

1

10

2

10

3

gi

/

hi

(c)

10

2

swaps

0

200

400

600

800

1000

i

10

3

10

2

10

1

10

0

10

1

10

2

10

3

gi

/

hi

(d)

10

3

swaps

0

200

400

600

800

1000

i

10

3

10

2

10

1

10

0

10

1

10

2

10

3

gi

/

hi

(e)

10

4

swaps

0

200

400

600

800

1000

i

10

3

10

2

10

1

10

0

10

1

10

2

10

3

gi

/

hi

(f)

10

5

swaps

0

200

400

600

800

1000

i

10

3

10

2

10

1

10

0

10

1

10

2

10

3

gi

/

hi

(g)

10

6

swaps

0

200

400

600

800

1000

i

10

3

10

2

10

1

10

0

10

1

10

2

10

3

gi

/

hi

(h)

10

7

swaps

0

200

400

600

800

1000

i

10

3

10

2

10

1

10

0

10

1

10

2

10

3

gi

/

hi

(i)

10

8

swaps

Bloom filter

Fast PLBF (Proposed)

Ada-BF

Fast PLBF++ (Proposed)

Sandwiched LBF

0.0

0.2

0.4

0.6

0.8

1.0

1.2

1.4

Total Memory Usage [Mbit]

10

4

10

3

10

2

10

1

False Positive Rate

(a)

0

swaps (score distribution is ideally monotonic)

0.0

0.2

0.4

0.6

0.8

1.0

1.2

1.4

Total Memory Usage [Mbit]

10

4

10

3

10

2

10

1

False Positive Rate

(b)

10

swaps

0.0

0.2

0.4

0.6

0.8

1.0

1.2

1.4

Total Memory Usage [Mbit]

10

4

10

3

10

2

10

1

False Positive Rate

(c)

10

2

swaps

0.0

0.2

0.4

0.6

0.8

1.0

1.2

1.4

Total Memory Usage [Mbit]

10

4

10

3

10

2

10

1

False Positive Rate

(d)

10

3

swaps

0.0

0.2

0.4

0.6

0.8

1.0

1.2

1.4

Total Memory Usage [Mbit]

10

4

10

3

10

2

10

1

False Positive Rate

(e)

10

4

swaps

0.0

0.2

0.4

0.6

0.8

1.0

1.2

1.4

Total Memory Usage [Mbit]

10

4

10

3

10

2

10

1

False Positive Rate

(f)

10

5

swaps

0.0

0.2

0.4

0.6

0.8

1.0

1.2

1.4

Total Memory Usage [Mbit]

10

4

10

3

10

2

10

1

False Positive Rate

(g)

10

6

swaps

0.0

0.2

0.4

0.6

0.8

1.0

1.2

1.4

Total Memory Usage [Mbit]

10

4

10

3

10

2

10

1

False Positive Rate

(h)

10

7

swaps

0.2

0.4

0.6

0.8

1.0

1.2

1.4

1.6

Total Memory Usage [Mbit]

10

4

10

3

10

2

10

1

False Positive Rate

(i)

10

8

swaps

0

10 1

10 2

10 3

10 4

10 5

10 6

10 7

10 8

Number of Swaps

1.0

1.2

1.4

1.6

1.8

Relative False Positive Rate

Peer reviews:
comments: SUMMARY:
The authors propose two methods to reduce the construction time of the partitioned learned Bloom filter (PLBF): 
1. Fast PLBF, which can construct the same data structure as PLBF but with a smaller time complexity of O(N^2k) and 2. Fast PLBF++, which can construct the data structure with a time complexity of O(Nk log N + Nk^2) but may not necessarily construct the same data structure as PLBF. Fast PLBF++ is almost as memory efficient as PLBF. 

The authors prove that fast PLBF++ has the same data structure as PLBF when the distribution satisfies a certain constraint.

STRENGTHS:
1. The algorithm and presentation are straightforward and strong theoretical results are provided. 

2. The author shows empirically that their construction keeps an edge in practice and can lead to faster PLBF construction.

WEAKNESSES:
The authors may provide further ablation studies on the sensitivity of hyper-parameters in practice and discuss more about societal impact.

QUESTIONS:
My concerns are raised above.

LIMITATIONS:
The authors may include a discussion on the potential negative societal impact of their work.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-06-27

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose faster dynamic programming variants of the learned partitioned Bloom filter data structure of Vaidya et al. that requires $O(N^3 k)$ time.\nTwo solutions are proposed, the first constructs the same data structure in time $O(N^2 k) $ time, and another, that constructs a potentially different structure in $O(Nk\\log N+Nk^2)$ time.\n\nSTRENGTHS:\nA significant speedup of a recent paper on a hot and interesting topic. Also, a nice empirical gains in the construction speed.\n\nWEAKNESSES:\nIn essence, the authors propose DP acceleration techniques without explaining the difference from off-the-shelf methods.\nMany such solutions are known for decades (e.g., see https://courses.engr.illinois.edu/cs473/sp2016/notes/06-sparsedynprog.pdf), and while it is possible that they don't cover the exact problem, they seem very similar. \n\nWhile the authors propose to use the SMAWK algorithm, it seems that the wrapper needed to get from it to the actual dynamic program might also be known.\nFor example, (https://www.sciencedirect.com/science/article/pii/002001909090215J) solves the following problem:\n\n$$ E[j] = \\min_{0\\le i<j} D[i] + w(i,j)\\ .$$\n\nThis looks nearly identical to your Equation (2), and I believe that a small massaging of the notations might yield your $O(N^2 k)$ solution.\nWithout explaining the difference from known DP acceleration methods, it is very hard to assess the novelty of the presented algorithms.\n\n\nI am also unclear about how you set $N$ in practice.\n\nQUESTIONS:\n1. How do you set $N$?\n\n2. Is the error monotone in $N$? (If so, the answer to (1) is simple -- use the largest feasible $N$. I'm not sure otherwise.)\n\n3. Can you explain the differences between your solutions and existing DP acceleration techniques?\n\n4. Vaidya et al. also present an algorithm with runtime $O(N^2 k)$ in their paper, under some assumptions. Can you please comment about whether this is the same conditions you use in Fast PLBF++?\n\n5. In cases where Fast PLBF++ yields a different data structure than PLBF, can you bound the increase in error?\n\nLIMITATIONS:\nNone", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents two improvements of the algorithm PLBF (Partitioned Learned Bloom Filter), called fast PLBF and fast PLBF++. PLBF learns the distribution structure and uses it to minimize the memory allocation. An ML model (LightGBM is used in this paper) is trained to predict a score between [0,1] indicating set membership probability. The score space is divided into N segments and then grouped into k<N regions that will each use a backup Bloom filter. The grouping of N segments into k region is formulated as an optimization problem and solved by dynamic programming, with time complexity O(N^3 k) for PLBF. The new method fast PLBF carefully refines the dynamic programming steps (by avoiding some redundant computation) and builds the same data structure but in O(N^2 k). The fast PLBF++ can construct a slightly different data structure, but faster; under some conditions, fastPLBF++ also constructs the same data structure. Experimental evaluation shows the performance of the two schemes.\n\nSTRENGTHS:\nLearned Bloom filters are a useful data structure. Improving the construction time for PLBF allows for the construction of filters for larger values of N, which improves their performance. The experimental evaluation is good.\n\nWEAKNESSES:\nIt is not clearly explained what computations are redundant in PLBF and how they are avoided by fast PLBF.\n\nQUESTIONS:\nI would like to see more space devoted in the paper to the explanation of fast PLBF compared to PLBF. Can you explain in more detail, or even intuitively, what computation is redundant in PLBF and how it is avoided by the new scheme? All of it is deferred to the appendix, but even there it is not very clear. This is one of the main points of the paper and I think more justification should be included in the paper itself, not in the appendix.\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-05"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper improves upon the Partitioned Learned Bloom Filter (PLBF), which is an essentially optimal learned Bloom filter introduced in ICML 2021. Construction time of PLBF is $O(N^3k)$, where $N > k$ are hyper-parameters. The authors show how to construct the same filter much more quickly; their Fast Partitioned Learned Bloom Filter (FPLBF) produces the same filter as PLBF and takes only $O(N^2k)$ time to construct. They also propose the fast PLBF++ algorithm that runs in even quicker $O(Nk\\log N + Nk^2)$ time. It outputs the same filter under natural assumptions on the input and is essentially just as good experimentally. Extensive experimental evaluation with real world datasets and ablation studies support the claims and demonstrate significant practical speedups.\n\nSTRENGTHS:\n1) Bloom filters are a ubiquitous approximate set membership data structure. Their combination with learned predictors has been an active and fruitful direction of research in the past years. This paper is another significant step.\n2) The authors are right (and the first to discover to the best of my knowledge) spot the redundancy in the PLBF paper and save an $O(N)$ factor.\n3) Fast PLBF++ is a good connection with the monotone matrix max problem. The authors prove that under natural assumptions it provides the same output even quicker. It works well with real data even if the assumptions are not 100% satisfied. \n4) Detailed experimental evaluation, especially kudos for the ablation in Section 5.3.\n\nWEAKNESSES:\n1) The PLBF authors were rather sloppy by rebuilding the same dynamic programming table from scratch when all they needed was adding another ''row'' to it. In fact looking at eq (2) in the paper it's quite obvious that superscript $j$ plays no role in $\\mbox{DP}^j$ and a single DP table suffices. Nevertheless, credit is due for spotting this.\n2) Practical runtime gains are potentially overstated as they depend on the choice of hyper-parameters $N$ and $k$. The paper shows runtime improvements for $N=1000$ and $k=5$, these hyperparameters are copied from the PLBF ICML 2021 paper, where they appeared without much justification. The (excellent) ablation experiments suggests that much smaller $N$ and much larger $k$ are optimal, which would decrease the runtime gaps in practice.\n\nQUESTIONS:\nFigure 12 shows that $N=50$ or $N=100$ max are sufficient for minimizing the false positive rate (for a given memory budget) for $k=5$. Figure 13 indicates that $k=100$ or even higher is required for minimizing the false positive rate for $N=1000$. Could you please find the $(N,k)$ combination that approximately minimizes the false positive rate and then measure and show the construction times for that? Could you please also zoom into the FPR plots around their minima without using log-scale on the x-axis (perhaps in the appendix) as I could not determine the location of the FPR minima in terms of $N$ even when I viewed the enlarged pdf on my screen,\n\nLIMITATIONS:\nYes, it's adequate.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-14"}