In 2009, the U.S. embarked on incentivizing providers to adopt government-certified 53 electronic health record technologies (CEHRTs), aiming to reduce costs and inefficiencies in the 54 health care system, while improving quality [1] (abbreviations are listed in Supplementary 55 Materials, Table 1 ). Electronic clinical quality measures (eCQMs) were introduced along with 56 CEHRTs as an approach to quantify the improvement in clinical performance and outcomes. One administrative data is lacking [10] . HEDIS managed care health care quality measures are 70 calculated at the health plan level and not the individual provider level. [11] . 71 The literature assessing the correlation of quality measures to the use of CEHRT suggest 72 that quality improvement is more likely if the CEHRT functionality tracks closely with the 73 quality metric and if providers use the CEHRT functionality effectively. The functionalities of 74 best-practice alerts, order sets, and panel-level reporting, have led providers to receive 75 statistically significantly higher "meaningful use" quality scores related to the EHR functionality 76 [12]. These "best practice alerts," or clinical decision support (CDS), pertained to the 77 preventative services of tobacco cessation, breast cancer screening, colorectal cancer screening, 78 pneumonia vaccination, and body mass index screening [12, 13] . 79 CEHRT-based CDS focuses on prompting providers to take some type of action on 80 particular patients, usually during or immediately surrounding the patient encounter. CDS may 81 be effective as a childhood-vaccine intervention because it takes advantage of the presence of the 82 child during well-and sick-child visits to reduce the likelihood of missed opportunities to 83 administer or catch up on vaccines. How the CDS is implemented within the clinical workflow 84 can differentially impact the effect of CEHRT on vaccination rates, with some studies showing 85 improvements in vaccine rates [14, 15] while others not showing the expected improvements [16 86 With standardized Electronic Health Record (EHR) systems, Medicaid agencies will soon 87 be able to collect patient-level data by provider to compare like providers to like providers 88 longitudinally, allowing for pay-for-outcomes models. However, until the adoption of certified 89 EHRs is widespread, the integration of health-care related data sources is easier, patient 90 attribution is precise and accurate, and, electronic clinical measures become more commonplace, 91 any longitudinal analysis of quality must rely on HEDIS-like quality methodologies. Maryland
Medicaid uses the HEDIS Childhood Immunization Status score, Combination 7 (HEDIS 93 immunization measure), as a standard measure to quantify immunization rates for its child 94 population. The primary objective of this study is to develop a methodology for estimating the effect 
This research used a retrospective, interrupted time series to assess the association 107 between CEHRT use and a provider-specific HEDIS immunization measure (Figure 1 ) [17] . The providers who adopted EHR at any point in this study.
The providers who met MU in 2013 and attested to meeting NQF#0038 (the 136 "intervention group") were compared to PCPs who gave immunizations but had not adopted 137 CEHRT (the "comparison group" Because researchers and health system administrators will likely be using this method A full-variable balance plot provides a better visualization technique to assess the impact 209 of PSM on creating a suitable comparison group for analysis. Figure 3 shows the balance plot of 210 our propensity score matching and comparing each variable pre-and post-matching. Table 6 . 
We restricted each provider in each calendar year to a primary group as providers 305 practice with many groups, and because this analysis recognizes that the intervention group is The relationship between the data used for this analysis is complex due to its hierarchical 331 structure ( Figure 4) . The data includes three nested levels: groups, providers, and time. Providers We specified hierarchical models, expressed as Bayesian analysis Using Gibbs Sampling 345 (BUGS) models and non-informative conjugate priors [28] . A non-informative conjugate prior is 346 a higher-level prior distribution that is within the same family as the underlying distribution it is 347 feeding within the model, but its distribution is specified to be vague [29] . Using BUGS, the 348 model focused on provider vaccination rates as affected by base vaccination rates, behavior of 349 the rest of the provider's group, and other relevant covariates. The second "level" was the 350 difference in vaccination rates across the years, which in turn is a function of CEHRT status.
Likelihood was derived by Equation (2), which states that the likelihood of theta, the 352 parameter you care about, given y, the data you observed, is the product of each of the 353 probabilities of y from y 1 to y n , given theta, a probability whose expression is known (e.g, 
To estimate the effect of CEHRT on HEDIS immunization measure, we monitored the Figure 1 for the trace and kernel density plots. As shown in Table 2 , 413 comparing the intervention group to its comparison group showed a 21% estimated improvement 414 in the odds of meeting HEDIS immunization measure (Combination 7) due to EHR use. This study has some limitations: (1) As with all studies that utilize PSM to develop 510 comparison groups, this study assumes non-ignorability or selection-on-observables to define 511 covariates for propensity score matching. The authors attempted to mitigate this weakness by The authors declare no conflicting interests. 