Researchers often use artificial data to assess the performance of new econometric methods . In many cases the data generating processes used in these Monte Carlo studies do not resemble real data sets and instead reflect many arbitrary decisions made by the researchers. As a result potential users of the methods are rarely persuaded by these simulations that the new methods are as attractive as the simulations make them out to be. We discuss the use of Wasserstein Generative Adversarial Networks ( WGANs) as a method for systematically generating artificial data that mimic closely any given real data set without the researcher having many degrees of freedom . We apply the methods to compare in three different settings twelve different estimators for average treatment effects under unconfoundedness . We conclude in this example that (i) there is not one estimator that outperforms the others in all three settings,  and (ii) that systematic simulation studies can be helpful for selecting among competing methods  .
When researchers develop new econometric methods it is common practice to compare the performance of the new methods to those of existing methods in Monte Carlo studies . The credibility of such Monte Carlo studies is often limited because of the freedom the researcher has in choosing the design. In recent years a new class of generative models emerged in the machine learning literature, termed Generative Adversarial Networks ( GANs) that can be used to systematically generate artificial data that closely mimics real economic datasets, while limiting the degrees of freedom for the researcher and optionally satisfying privacy guarantees with respect to their training data. In addition if an applied researcher is concerned with the performance of a particular statistical method on a specific data set (beyond its theoretical properties in large samples), she may wish to assess the performance, e.g., the coverage rate of confidence intervals or the bias of the estimator, using simulated data which resembles her setting. Tol illustrate these methods we apply Wasserstein GANs (WGANs) to compare a number of different estimators for average treatment effects under unconfoundedness in three distinct settings (corresponding to three real data sets) and present a methodology for assessing the robustness of the results. In this example, we find that (i) there is not one estimator that outperforms the others in all three settings, so researchers should tailor their analytic approach to a given setting, and (ii) systematic simulation studies can be helpful for selecting among competing methods in this situation .
R
Researchers often use artificial data to assess
When researchers develop new econometric methods it is common practice to compare
R
new econometric methods . In many cases the data generating processes used in these
the new methods to those of existing methods in
R
do not resemble real data sets and instead reflect many arbitrary decisions made by the researchers. As a result potential users of the methods are rarely persuaded by these simulations that the new methods are as attractive as the simulations make them out to be. We discuss the use of Wasserstein
. The credibility of such Monte Carlo studies is often limited because of the freedom the researcher has in choosing the design. In recent years a new class of generative models emerged in the machine learning literature, termed
R
WGANs) as a method for systematically generating
GANs) that can be used to systematically generate
R
mimic closely any given real data set without the researcher having many
closely mimics real economic datasets, while limiting the
R
. We apply the methods to compare in three different settings twelve different
for the researcher and optionally satisfying privacy guarantees with respect to their training data. In addition if an applied researcher is concerned with the performance of a particular statistical method on a specific data set (beyond its theoretical properties in large samples), she may wish to assess the performance, e.g., the coverage rate of confidence intervals or the bias of the estimator, using simulated data which resembles her setting. Tol illustrate these methods we apply Wasserstein GANs (WGANs) to compare a number of different
R
. We conclude in this example
in three distinct settings (corresponding to three real data sets) and present a methodology for assessing the robustness of the results. In this example, we find
A
so researchers should tailor their analytic approach to a given setting,
D
that
A
in this situation