dedoc/paragraph_dataset huggingface dataset:
≤ x. (2)
With a change of variable y = 1− md̃ exp(−
ℓ x), inequality (2) becomes:
∀y ∈
0, 1− md̃ exp(−
, λ(y) ≤ − ℓ
md̃
ln(1− y).
10 Matthieu Finiasz and Kannan Ramchandran
Fig. 4. Column weight distribution for the harmonic distribution of order 7.
The optimal choice is thus the harmonic distribution, consisting in a normalized Taylor series
expansion of − ln(1− x) truncated at order D. It is given by:
λ (x) D =
H(D)
D ∑
i=2
i− 1
i−1 x with H(D) =
These λ (x) D distributions will satisfy inequality 2) ( if the expansion factor ℓ
m is greater than 1+ 1
Any expansion ratio ℓ
m = 1+ ϵ can thus be chosen by the user, and using the harmonic distribution
of order 1
ϵ will asymptotically allow to recover most documents.
The Enhanced Harmonic Distribution. One problem with the harmonic distribution is the presence
of columns of weight 2 (see Fig. 4). Any distribution containing columns of weight 2 will have
a non-zero asymptotic probability of having identical 6 columns . Indeed, the collision probability
Pcol that 2 of the m matching columns of H are identical is Pcol ≃
( d̃λi
. If the ratio ℓ
is constant and m tends to inﬁnity, all the terms in this sum tend to 0, except the term i = 2.
Asymptotically, Pcol
m→∞ −→
d̃λ m 2
2ℓ
)2
, which is not negligible for the harmonic distribution.
This means that, using the harmonic distribution, decoding will often end with a few (O(1)
asymptotically) unrecovered documents. In order to deal with this issue, one solution is to combine
this distribution with a constant weight distribution to obtain what we call the enhanced harmonic
distribution. Each column of H is the concatenation of a column of length ℓ − ℓ3 following the
harmonic distribution and a column of length ℓ3 of constant weight 3.
In practice, the proportion of collisions tends to 0 when m grows, so the length ℓ3 can be
chosen such that ℓ3
m tends to 0 and the probability of full recovery still tends to 1. For example,
l3 = O(
ℓ) would be a reasonable choice. Asymptotically, these ℓ3 additional rows in H do not
increase the expansion factor. Using this enhanced harmonic distribution with an order D harmonic
distribution, the probability of recovering all m documents tends to 1 when ℓ
m > 1 + 1
D and m
tends to inﬁnity.
Communication Cost. We compared the diﬀerent distributions through simulations and the
results we obtained are shown in Fig. 5. One can clearly see the limitation of the weight 3 distri-
bution: for m > .8ℓ ≃ ℓ
1.22 the probability of full recovery drops. However, the enhanced harmonic
6 If two documents fi generate the exact same column H , i our decoding algorithm will be unable to recover any of
them as no singleton can ever be obtained.
Communication Optimal Private Stream Search 11
average ratio of recovered documents
full recovery probability
5000 6000 7000 8000 9000 10000
500 600 700 800 900 1000
50 60 70 80 90 100
(a) (b) (c)
(d) (e) (f)
weight 3
Distributions:
Fig. 5. Simulation results for diﬀerent LDPC weight distributions. The curves represent the average
ratio of recovered documents and the probability of recovering all m documents as a function of the
number of matching documents m for diﬀerent buﬀer sizes ℓ. In (a) and (d) ℓ = 100 and ℓ3 = 10,
in (b) and (e) ℓ = 1000 and ℓ3 = 35, and in (c) and (f) ℓ = 10000 and ℓ3 = 100.
distribution holds to our expectations, it behaves as expected with a probability of recovering all
m matching documents close to 1 even for expansion rates smaller than 5%.
With this algorithm and the enhanced harmonic distribution, the asymptotic communication
cost of private stream search with m matching documents of S bits is thus 2m(S+64+O(log |Ω|))
using Paillier’s cryptosystem, or m(S + 1088 + O(log |Ω|)) using the Damg̊ard-Jurik extension,
which corresponds to an expansion rate of 1 compared to a non-private search.
4 Further Improvements
An Oﬄine-Online Construction. Using any of our two new schemes, it is possible to reduce
the size of the reply from the server almost to the size of a non-private search result. However, the
size of the query the user sends remains large. The size of a private query is linear in |Ω| whereas it
is logarithmic for a non-private query. To improve this, we propose an oﬄine-online scheme, where
the linear query is sent oﬄine and a logarithmic query is sent online.
A Single Keyword Scheme. We ﬁrst focus on queries containing a single keyword. In this case, any
query can be obtained as a cyclic shift of any other query. The scheme works as follows:
– oﬄine, the user generates a query Qj = {q , 1 . . . , q } |Ω| where qj = E(1) and qi = E(0) otherwise
(with j picked uniformly at random), and sends it to the server,
– oﬄine, the server computes all possible cyclic shifts of Qj by i ∈ [0, [Ω| − 1] positions and
executes the corresponding queries. It stores each result in a separate buﬀer B . i
– online, the user wants to query the server for the ′ j -th keywords and sends ′ j − j mod |Ω| to
the server,
– online, the server sends ′ Bj −j to the user and discards the other B , i
– the user decrypts/decodes buﬀer ′ Bj −j normally.
With this scheme, the online work on the server side is a simple table lookup and the amount
of online communication is very close to the non-private case: the query is only log |Ω| bits long,
and with the PSS schemes we have presented ′ Bj −j can also be small.
12 Matthieu Finiasz and Kannan Ramchandran
The oﬄine amount of communication is still the same as for the standard scheme, but the
amount of computation on the server side is multiplied by |Ω|. However, as this is oﬄine work,
it can easily be outsourced to distant server farm and does not have to be run on the “online”
low-latency servers. Of course, if the amount of oﬄine work is too high, it is also possible to treat
the shifted query online as in the standard scheme: the amount of work the server has to do will
then be the same as in the normal scheme, but most of the communication will be done oﬄine.
Dealing with Multiple Keywords. To maintain privacy, the oﬄine query the user sends has to be
completely random and, at the same time, it should be possible to modify it into any other query the
user might later want to ask. For a single keyword, cyclic shifts work well whatever the dictionary
size. However, for several keywords (say k), the user should be able to transform any random query
Qj ,...,j 1 k into any chosen query ′ Qj ,...,j 1
, by simply giving the index of a permutation.
When k = 2, a solution is to transform each index j into Aj+B mod |Ω|, where A ∈ [1, |Ω|−1]
and B ∈ [0, |Ω| − 1] are the “permutation index” that the user will send to the server in the online
phase. If |Ω| is prime, then any pair j , 1 j2 can be transformed into any pairs ′ j , 1 j
2 by choosing
A = j −j 2 1
′ ′ j −j 2 1
mod |Ω| and B = Aj1 − ′ j1 mod |Ω|.
Building such families of permutations for larger values of k is not always simple, and the
number of permutations in the family will always have to be at least
(|Ω|
. This means that the
oﬄine work on the server side will be k O(|Ω| ) times more than in the standard online scheme.
Values of k larger than 1 or 2 are therefore not very realistic, and for these values the solutions we
presented work ﬁne.
Using Fully Homomorphic Encryption. The PSS schemes we presented do not need fully
homomorphic encryption (FHE) to work: computing a syndrome only requires addition and multi-
plication by a scalar. However, having an eﬃcient FHE scheme would allow AND queries, whereas
the current schemes are limited to OR queries. Also, the query size could be reduced to O(log |Ω|)
encrypted bits without having to use an oﬄine-online 7 scheme . For instance, when querying for
keyword w , i the user could decompose the index i in log |Ω| bits and send two ciphertexts E(0)
and E(1) (or E(1) and E(0)) for each of these bits. The server can then regenerate the whole query
Q by homomorphically multiplying the encrypted bits corresponding to each index. Continuing on
this idea, it would even be possible to get rid of the dictionary by simply querying a bit string: a
query of 2n encrypted bits would be enough to search any pattern of n bits.
One interesting aspect of this application of FHE, is that the number of homomorphic multi-
plications that have to be done is independent of the database size: multiplications are required to
reconstruct the query, but not for the stream search itself. Even if homomorphic multiplication is
very expensive, or if the chosen scheme only allows for a few multiplications, it is still possible use
it for very large databases: only the dictionary size is limited.
Also, one should note that the homomorphic encryption scheme used in PSS does not have to
be a public key scheme: only the user encrypts and decrypts elements (the server simply has to be
able to initialize the buﬀer to E(0), but the user could send him this initial value). A symmetric
homomorphic encryption scheme with a limit on the number of possible multiplications could thus
also be of interest if it allows a small message expansion rate.
Application to Set Reconciliation. An interesting aspect of the LDPC scheme we presented is
that, even though it behaves like a randomized scheme, it is fully deterministic. What this means
7 Not that a query of O(log |Ω|) bits is probably not achievable in practice as current FHE schemes all require some
important message expansion, but a poly-logarithmic query is possible.