3 9 2 1 P. C. AUSTIN sample. There is a lack of consensus in the literature as to the appropriate statistical methods to use when assessing the statistical signiﬁcance of the estimated treatment effect in a propensity-score matched sample. Schafer and Kang suggest that, within the matched sample, the treated and untreated subjects should be regarded as independent [3]. Thus, if outcomes were continuous, a two-sample t-test could be used for testing the statistical signiﬁcance of a difference in means between treatment groups, while the Pearson Chi-squared test could be used to test the statistical signiﬁcance of the difference in proportions between treatment groups. Recent systematic reviews of propensity-score matching in the medical literature have found that authors frequently use statistical methods for independent samples when assessing the statistical signiﬁcance of the estimated treatment effect in the propensity-score matched sample [4--6]. In contrast to this, it has been argued elsewhere that a propensity-score matched sample does not consist of independent observations [5]. Treated and untreated subjects matched on the propensity score will have observed baseline covariates that come from the same multivariate distribution [1]. Thus, on average, the baseline characteristics of matched treated and untreated subjects will be more similar than the baseline characteristics of randomly selected treated and untreated subjects in the propensity-score matched sample. When confounding is present, baseline covariates are related to the outcome. Therefore, the outcomes of matched treated and untreated subjects are likely to be more similar to one another compared to the outcomes of randomly selected treated and untreated subjects in the propensity-score matched sample. In medical and clinical research, dichotomous outcomes are common [7, 8]. Propensity score methods allow for direct estimation of risk differences (or absolute risk reductions), in which the proportion of subjects in whom the event occurs can be directly compared between treatment groups in the propensity-score matched sample [9]. The objective of the current study was to compare the effect on statistical inference when statistical methods for independent samples are used compared with when statistical methods for paired samples are used for comparing differences in proportions between treatment groups in the propensity-score matched sample. The study had four speciﬁc objectives: First, to compare the empirical type I error rate when using McNemar’s test compared with when using the standard Pearson Chi-squared test for comparing the proportion of subjects in whom the event occurred between treatment groups; second, to compare the empirical coverage rates of 95 per cent conﬁdence intervals when standard errors are estimated using methods for paired data compared with when methods for independent samples are used; third, to compare the precision of estimated 95 per cent conﬁdence intervals when standard errors are estimated using methods for paired data compared with when methods for independent samples are used; and fourth, to compare the variance of the empirical sampling distribution of the risk difference with the estimated variance of the risk difference when using methods for independent samples and methods for paired samples. These four objectives will be addressed using Monte Carlo simulations. 2. Monte Carlo simulations—methods In this section, we describe the Monte Carlo simulations that were used to compare statistical inference when methods for independent samples were used compared with when methods for paired samples were used. 2.1. Data-generating process We used a data-generating process identical to one used in a prior study that examined optimal caliper widths for use with propensity-score matching [10]. Brieﬂy, we simulated data sets such that approximately 25 per cent of the sample was exposed to the treatment. The data-generating process was designed to induce a speciﬁc average treatment effect for the treated (ATT), the measure of effect that is estimated when propensity-score matching is used [11]. Furthermore, in the simulated data sets, the marginal probability of the outcome would be approximately 0.29 if all subjects in the population were not exposed. We then examined scenarios in which the risk differences due to treatment in treated subjects were 0, −0.02, −0.05, −0.10 and −0.15 (i.e. absolute reductions in the probability of the outcome due to treatment were 0, 0.02, 0.05, 0.10 and 0.15). First, we randomly generated 10 independent covariates (X1 − X10) from independent standard normal distributions for each of 10 000 Copyright © 2011 John Wiley & Sons, Ltd. Statist. Med. 2011, 30 1292--1301