  Algorithms learn rules and associations based on the training data that they
are exposed to. Yet, the very same data that teaches machines to understand and
predict the world, contains societal and historic biases, resulting in biased
algorithms with the risk of further amplifying these once put into use for
decision support. Synthetic data, on the other hand, emerges with the promise
to provide an unlimited amount of representative, realistic training samples,
that can be shared further without disclosing the privacy of individual
subjects. We present a framework to incorporate fairness constraints into the
self-supervised learning process, that allows to then simulate an unlimited
amount of representative as well as fair synthetic data. This framework
provides a handle to govern and control for privacy as well as for bias within
AI at its very source: the training data. We demonstrate the proposed approach
by amending an existing generative model architecture and generating a
representative as well as fair version of the UCI Adult census data set. While
the relationships between attributes are faithfully retained, the gender and
racial biases inherent in the original data are controlled for. This is further
validated by comparing propensity scores of downstream predictive models that
are trained on the original data versus the fair synthetic data. We consider
representative & fair synthetic data a promising future building block to teach
algorithms not on historic worlds, but rather on the worlds that we strive to
live in.
