  In this paper, we consider the distributed mean estimation problem where the
server has access to some side information, e.g., its local computed mean
estimation or the received information sent by the distributed clients at the
previous iterations. We propose a practical and efficient estimator based on an
r-bit Wynzer-Ziv estimator proposed by Mayekar et al., which requires no
probabilistic assumption on the data. Unlike Mayekar's work which only utilizes
side information at the server, our scheme jointly exploits the correlation
between clients' data and server' s side information, and also between data of
different clients. We derive an upper bound of the estimation error of the
proposed estimator. Based on this upper bound, we provide two algorithms on how
to choose input parameters for the estimator. Finally, parameter regions in
which our estimator is better than the previous one are characterized.
