1 2 9 2 Research Article Received 6 July 2010, Accepted 21 December 2010 Published online 21 February 2011 in Wiley Online Library (wileyonlinelibrary.com) DOI: 10.1002/sim.4200 Comparing paired vs non-paired statistical methods of analyses when making inferences about absolute risk reductions in propensity-score matched samples Peter C. Austina,b,c∗ † Propensity-score matching allows one to reduce the effects of treatment-selection bias or confounding when estimating the effects of treatments when using observational data. Some authors have suggested that methods of inference appropriate for independent samples can be used for assessing the statistical signiﬁcance of treatment effects when using propensity-score matching. Indeed, many authors in the applied medical literature use methods for independent samples when making inferences about treatment effects using propensity-score matched samples. Dichotomous outcomes are common in healthcare research. In this study, we used Monte Carlo simulations to examine the effect on inferences about risk differences (or absolute risk reductions) when statistical methods for independent samples are used compared with when statistical methods for paired samples are used in propensity-score matched samples. We found that compared with using methods for independent samples, the use of methods for paired samples resulted in: (i) empirical type I error rates that were closer to the advertised rate; (ii) empirical coverage rates of 95 per cent conﬁdence intervals that were closer to the advertised rate; (iii) narrower 95 per cent conﬁdence intervals; and (iv) estimated standard errors that more closely reﬂected the sampling variability of the estimated risk difference. Differences between the empirical and advertised performance of methods for independent samples were greater when the treatment-selection process was stronger compared with when treatment-selection process was weaker. We recommend using statistical methods for paired samples when using propensity-score matched samples for making inferences on the effect of treatment on the reduction in the probability of an event occurring. Copyright © 2011 John Wiley & Sons, Ltd. Keywords: propensity score; propensity-score matching; risk difference; absolute risk reduction; Monte Carlo simulations; statistical inference; hypothesis testing; type I error rate; categorical data analysis 1. Introduction Propensity-score matching is increasingly being used to estimate the effects of treatments, exposures and interventions on outcomes in observational studies. The propensity score is the probability of treatment assignment conditional on the observed baseline covariates [1]. If treatment assignment is strongly ignorable, then conditioning on the propensity score allows one to obtain an unbiased estimate of average treatment effects [1]. Matching on the propensity score allows one to construct a matched sample in which systematic differences in observed baseline covariates are reduced or eliminated between treatment groups [2]. Outcomes often can be directly compared between treatment groups in the propensity-score matched a Institute for Clinical Evaluative Sciences, Toronto, Ont., Canada bDepartment of Health Management, Policy and Evaluation, University of Toronto, Toronto, Ont., Canada cDalla Lana School of Public Health, University of Toronto, Toronto, Ont., Canada ∗Correspondence to: Peter C. Austin, Institute for Clinical Evaluative Sciences, G1 06, 2075 Bayview Avenue, Toronto, Ont., Canada M4N 3M5. †E-mail: peter.austin@ices.on.ca Copyright © 2011 John Wiley & Sons, Ltd. Statist. Med. 2011, 30 1292--1301