  Genomic data provides researchers with an invaluable source of information to
advance progress in biomedical research, personalized medicine, and drug
development. At the same time, however, this data is extremely sensitive, which
makes data sharing, and consequently availability, problematic if not outright
impossible. As a result, organizations have begun to experiment with sharing
synthetic data, which should mirror the real data's salient characteristics,
without exposing it. In this paper, we provide the first evaluation of the
utility and the privacy protection of five state-of-the-art models for
generating synthetic genomic data.
  First, we assess the performance of the synthetic data on a number of common
tasks, such as allele and population statistics as well as linkage
disequilibrium and principal component analysis. Then, we study the
susceptibility of the data to membership inference attacks, i.e., inferring
whether a target record was part of the data used to train the model producing
the synthetic dataset. Overall, there is no single approach for generating
synthetic genomic data that performs well across the board. We show how the
size and the nature of the training dataset matter, especially in the case of
generative models. While some combinations of datasets and models produce
synthetic data with distributions close to the real data, there often are
target data points that are vulnerable to membership inference. Our measurement
framework can be used by practitioners to assess the risks of deploying
synthetic genomic data in the wild, and will serve as a benchmark tool for
researchers and practitioners in the future.
