We need Synthetic Data
By: Joseph Allen
Tags: Towards Data Science, Data Science, Machine Learning, Synthetic Data, Editors Pick

Sensitive data, without the sensitivities.

We need Synthetic Data

The benefits and methods of synthetic data generation

Synthetic data is an upcoming technology in the data industry. There are many reasons to explore synthetic data, which I will go into shortly.

Sometimes the most effective way to collect more data is to make it yourself.

Photo by Markus Spiske from Pexels

A computer program manufactures synthetic data. We no longer measure real-world events. There are many methods for generating synthetic data. These methods can range from find and replace, all the way up to modern machine learning. The synthesis starts easy, but complexity rises with the complexity of our data. To use synthetic data you need domain knowledge. You need to understand what personal data is, and dependence between features. Only with domain knowledge could you answer the following:

Are there relationships between our columns? Are they important enough to maintain?

Should our synthetic data mimic the real-world, or try to fix a problem?

Should we anonymise our data? By how much?

Benefits

Anonymisation

We can change our data in many ways to protect the anonymity of any individuals captured in a dataset. One method is replacing common personal details like emails, names and addresses. Depending on your dataset, your data may become less useful without accurate addresses. In this case, reducing granularity could be acceptable.

There has been an increase in black-box attacks on models to infer details of the training data. A model trained on synthetic data is less at risk.

Improve your model

Generating more data could be an effective way to improve the results of your model. In image data, you can improve your training set by performing some transformations. Shearing, shifting, rotating and all combinations of these can be useful. An ImageDataGenerator exists in Keras to handle this. An example of this applying to the MNIST dataset is available here.

Edge cases

If you want to detect fraudulent transactions you cannot buy more fraud cases. You may not have the luxury of waiting for more to occur. Using undersampling we can create more fraud cases. A more dangerous example might be collecting data on car crashes, or cancer scans. In some cases, waiting isn’t an option.

Cost

How would you collect a dataset of a thousand chess games? You could hire players, record games and convert moves into chess notation. You could pay for a larger dataset, or dedicate many hours to transcribing games yourself. The rules, notation and bots for chess are already known. Synthetic data, instead, might be the most cost and time-effective method here.

Photo by Alex Green from Pexels

Access

I consider accessibility to be the most important aspect of open data. If data can’t be accessed without filling out paperwork, signing EULA’s and more potential is limited. In the exploratory world of data, experts are often keen to make use of public data, for free. Let’s try our best not to block them.

If your synthetic data contains no sensitive data it can comfortably be shared outside of your organisation. Even sharing a handful of rows to show the structure of the data, alongside any documentation is a great step forward. We should be striving to make our data easier to access. Too many public datasets require a large investment into the unknown. Accessible data can be more easily reproduced and criticised by other experts.

An Example

Let’s take a look at the above data, we have:

id — 2999

— 2999 first_name — Grazia

— Grazia last_name — Midden

— Midden email — gmidden0@163

— gmidden0@163 gender — Female

— Female address — 10 Pawling Center, M3 6GA

We need domain knowledge to recognise how dangerous the ID column is. Does your organisation have data connected to this ID? If yes, we need to replace this. Let’s reindex these.

We could apply anonymisation to each column in isolation. That is generating a new and unconnected first_name, last_name, email, gender and address. A big problem in synthetic data is maintaining the distributions of a dataset. For example, should we have an email that makes use of the first and last name? Is it good enough to replace the address with a new address? If we need to know something about the address of an individual this might not be appropriate.

Instead, a better approach here would be to generate an email and other features using our fake data. For the user “Joe Bang” An email like Jbang@gmail.com might appear more genuine. We coarsen the address to maintain the general area, without identifying the individual.

Now again you must understand the context of this dataset. If for example there is only one individual in the postcode M3, we have not made them less anonymous. The same holds if there was only one female in our dataset. You must not skip the exploratory phase before making these decisions.

Synthesis Techniques

The data in government blog outlines the techniques used in data synthesis here. The following categorise these techniques.

Masking

Replacing parts of the dataset with randomly generated information. For example replacing real names, with names generated by the Faker Python library.

Here we have masked the last_name column.

Redaction

Removing sensitive rows from the dataset. Often, outliers in a dataset are sensitive by being outliers. If you have a list of addresses, the address of military bases may need redaction. In a list of children in a school, one student may have a higher number of siblings by far. This alone can identify this student. In our fraudulent transaction case from earlier, removing outliers removes our target. Again context is key in this decision.

Anonymising the names, and adding to the number of siblings is not enough to protect Waring Mineghelli. In this case, it is more appropriate to redact Waring’s data altogether. If our business product is a number of siblings predictor then we can make a case to our data officer that we need this important outlier.

Coarsening

Coarsening is about reducing the precision of the data. For example, instead of listing a full name, store only the initials. For a postcode, we could store only the first two digits.

In this example, we have coarsened the first_name column. While this data is slightly more anonymous, depending on the dataset.

Mimicking

Generation of a dataset close to real data, but with added randomness. For example, here add or subtract a random amount to heights and weights.

In this example, we have mimicked the number of siblings by adding a random value from -1 to 1. As you can see this is not enough to protect our outlier.

Simulation

The most difficult, but the most effective method is generating a new dataset. Using machine learning methods we can understand the relationships between features. We can then generate new rows that maintain these relationships. Now there is no need to obfuscate our individuals, as they no longer exist. Common methods here include GANs, VAEs and more.

We could generate an entirely new dataset using machine learning.

Synthesis Tooling

Mockaroo

While there are many data mocking libraries, that doesn’t mean you need to know how to code. Mockaroo allows you to generate 1000s of rows of structured data. You can generate 1000 rows at a time on the free tier. No programming required.

Data Mocking

Data mocking involves generating realistic test cases. These are usually generated by libraries containing large sets of common values. These values could be names, emails, phone numbers, social security numbers and more. These standard values cover most use cases. Where they don’t often a combination of these values can.

One great library for this is Faker. Faker comes with some standard providers. On top of this Faker also supports community providers, so if what you need doesn’t exist you can make it for the next user. Faker also supports multiple locales.

from faker import Faker

fake = Faker(['it_IT', 'en_US', 'ja_JP'])

for _ in range(10):

print(fake.name())



# 鈴木 陽一

# Leslie Moreno

# Emma Williams

# 渡辺 裕美子

# Marcantonio Galuppi

# Martha Davis

# Kristen Turner

# 中津川 春香

# Ashley Castillo

# 山田 桃子

Data Augmentation

Sometimes small adjustments to our data are enough to improve our dataset. We use the domain knowledge of this data to inform which adjustments are appropriate. In an image example of identifying cats or dogs, you may need to classify the following image.

Photo by Anel Rossouw from Pexels

You now have a dataset of size 1. This isn’t much to work with. Using data augmentation we can generate more data from this image. We do this by applying zooming, shearing, flipping, translating, rotating and more.

Rotated, flipped, shifted, saturated and more.

It’s not too much effort to make these. As we said before, an ImageDataGenerator exists in Keras to handle this. An example of this applying to the MNIST dataset is available here. An alternative library to look into here is Albumentations.

What if our dataset is text-based? Back translation is one example technique we could use. This involves translating from one language, into another and then back. This can be done in Google Sheets.

SMOTE

SMOTE stands for Synthetic Minority Over-Sampling Technique. To simplify that, in a problem where we are trying to predict a minority class such as our fraudulent transactions example. We might have a thousand transactions, with only ten labelled as fraudulent. SMOTE assumes that points on the lines connecting elements of this minority class are valid potential additions to the class. This post goes into more detail on SMOTE.

SMOTE demo by Khyati Mahendru

This credit card fraud exploration outlines a great application of SMOTE. The imblearn library has some documentation on the SMOTE algorithm here.

VAEs

VAE stands for Variable Auto-Encoder. Often we want to generate new and random data. With VAEs you can alter the data you generate with intention. A VAE is a neural network consisting of an encoder and a decoder. The encoder reduces the number of dimensions from our input set, to a single value. The decoder converts this single value back to the original data format. What’s powerful about this is that it’s realistic to ask this model to generate an image that is 90% cat and 10% dog. You can also follow this image-based exploration here. An example implementation in Keras is available here.

GANs

Two Neural networks make up a Generative Adversarial Network, or a GAN.

A Generator, responsible for trying to generate the best fake data it can. A Discriminator, responsible for learning to discriminate genuine data from synthetic data.

These networks work in tandem. The generator works to trick the discriminator. Both networks enjoy the data generated by the other. Generally, the more data we have for each network the better the other performs. You could imagine a child drawing cats and dogs, as a critical adult corrects them. With this, we can build networks that given an input, create an output with the desired property. You may be familiar with some of the Google DeepDream paintings.

From Shweta Goyal’s GAN post

As a note GANs are temperamental. Data generated by two neural networks, feeding into one another, is asking to go awry. GANs are right on the cutting edge and need a high degree of surveillance. For a more technical guide check out Shweta Goyal’s brief introduction.

Conclusion

Synthetic data is vital in the world of open data. If we want experts who are willing to donate their time to solve public problems we need to make it easy. Synthetic data solves the need for paperwork and data accounting. We can also take some steps to lower the commitment needed in a data exploration phase. If you own some data that could be open here are my suggested next steps:

Provide an anonymised header of a couple of rows. Revealing the structure of the data.

Provide a synthetic open dataset of ~1000 rows. Experts can perform some test visualisations before committing to the data’s privacy requirements.

Provide a fully synthetic dataset that maintains all the distributions of the original dataset. The real dataset might enjoy the findings of this synthetic data.

There are countless methods of data synthesis. We haven’t yet solved general-case data synthesis. This means that when you rise to this challenge you need domain knowledge of your dataset. It’s not yet safe to assume that columns don’t contain sensitive data. Make sure you understand the context so that the people who make use of your data don’t have to.