1 3 0 0 Table V. Coverage and width of empirical 95 per cent conﬁdence intervals and estimation of sampling variances of treatment effects – 0.15 treatment effect, strong treatment-selection model. Coverage of 95 per cent Lengths of 95 per cent conﬁdence intervals conﬁdence intervals True risk difference Independent Paired Independent Paired Ratio of length of independent CI to paired CI P. C. AUSTIN Ratio of mean estimated variance of treatment effect to variance of empirical sampling distribution Independent Paired Independent normal covariates 0 0.974 0.947 −0.02 0.974 0.952 −0.05 0.972 0.946 −0.1 0.974 0.952 −0.15 0.968 0.948 Correlated normal covariates 0 0.978 0.953 −0.02 0.977 0.950 −0.05 0.973 0.945 −0.1 0.970 0.945 −0.15 0.973 0.949 Mixture covariate scenario 1 0 0.967 0.952 −0.02 0.972 0.953 −0.05 0.973 0.951 −0.1 0.971 0.955 −0.15 0.969 0.948 Mixture covariate scenario 2 0 0.969 0.956 −0.02 0.973 0.957 −0.05 0.973 0.957 −0.1 0.965 0.955 −0.15 0.961 0.950 Independent Bernoulli covariates 0 0.960 0.950 −0.02 0.964 0.954 −0.05 0.963 0.954 −0.1 0.957 0.950 −0.15 0.948 0.939 0.168 0.167 0.164 0.159 0.152 0.188 0.187 0.184 0.179 0.173 0.164 0.162 0.159 0.154 0.147 0.154 0.152 0.148 0.141 0.133 0.146 0.143 0.139 0.131 0.122 0.148 0.147 0.145 0.142 0.138 0.161 0.160 0.159 0.156 0.152 0.147 0.146 0.144 0.139 0.135 0.143 0.141 0.138 0.133 0.126 0.139 0.136 0.133 0.126 0.118 1.135 1.136 1.131 1.120 1.101 1.168 1.169 1.157 1.147 1.138 1.116 1.110 1.104 1.108 1.089 1.077 1.078 1.072 1.060 1.056 1.050 1.051 1.045 1.040 1.034 1.252 1.286 1.257 1.265 1.199 1.360 1.341 1.297 1.262 1.259 1.234 1.236 1.296 1.248 1.208 1.208 1.228 1.234 1.188 1.151 1.117 1.148 1.140 1.079 1.004 0.976 1.006 0.989 1.012 0.984 0.997 0.988 0.963 0.954 0.977 0.993 0.997 1.055 1.030 1.020 1.042 1.061 1.071 1.047 1.039 1.008 1.039 1.037 0.996 0.947 When outcomes are binary, the effect of treatment on outcomes can be described using four different metrics: the risk difference, the relative risk, the number needed to treat (NNT) and the odds ratio. Propensity-score matching has been shown to result in biased estimation of both conditional and marginal odds ratios [16, 17]. The NNT is the reciprocal of the absolute risk reduction. Comparisons of different propensity score methods for estimating absolute risk reductions and relative risks are described in greater detail elsewhere [9, 18]. In conclusion, we recommend that when propensity-score matching is used to reduce or eliminate the effects of treatment selection bias or confounding, that statistical methods for paired samples be used when estimating the effect of treatment or exposure on absolute risk reductions or risk differences. Acknowledgements This study was supported by the Institute for Clinical Evaluative Sciences (ICES), which is funded by an annual grant from the Ontario Ministry of Health and Long-Term Care (MOHLTC). The opinions, results and conclusions reported in this paper are those of the authors and are independent from the funding sources. No endorsement by ICES or the Ontario MOHLTC is intended or should be inferred. Dr. Austin is supported in part by a Career Investigator award from the Heart and Stroke Foundation of Ontario. This study was supported in part by an operating grant from the Canadian Institutes of Health Research (CIHR) (Funding number: MOP 86508). Copyright © 2011 John Wiley & Sons, Ltd. Statist. Med. 2011, 30 1292--1301