# **6 Limitations and Robustness**

That data that we have used here has several limitations.

First, there may be measurement error in utilization data due to poor recall or inaccurate records. To benchmark this possibility, the SDI survey data from Kenya was compared to the Kenya DHIS2 survey for the same period ( **Supplemental Appendix Figure 1** ) and found that caseloads were correlated in both reports, with a correlation of 0.79.<sup>8</sup> In addition, time and motion studies have documented similar caseloads in India and Viet Nam; and other studies that measured caseloads report similar averages to ours.<sup>20, 21, 23, 24</sup> Second, SDI data collected caseloads at the level of the facility, not the provider. This is less of a problem in small facilities, which are the majority, but a substantive portion of our results reflect the experience of patients at large, crowded facilities with multiple providers. In these cases, since no data exists on the actual work patterns of individual providers, we split caseloads equally among all providers present, assuming that the present providers were representative of staffing at the facility on a random day. Because the SDI randomly observed in-person providers at the place of practice, our analysis is robust to flexible daily staffing patterns _on average_ , but cannot fully account for changes such as providers who work part-time, providers who move between multiple facilities, and staffing patterns that are responsive to diurnal variation in demand. Ideally, future research would observe detailed work patterns for all clinic staff including precise patient assignments to each provider over a series of work days. In large facilities where we had to estimate the number of providers actually present, there are additional complications; we present a full description of this process and results robust to the exclusion of these facilities in **Supplemental Appendix Tables 2–4** .

Third, in terms of the implications for quality, multiple studies have found a large gap between medical competence and clinical practice.<sup>21, 25–29</sup> Any gains that we compute here (either from reallocation or upskilling) are based on provider competence rather than clinical practice. The data used here are also based on a sample of the health facilities and providers in each country. While previous studies have documented similar variation in caseloads and quality even at the level of the village,<sup>30, 31</sup> we would require a substantial increase in the scale of data collection to accurately compute the gains to reallocation within local areas. These data limitations arguably imply that our results are a lower bound of the gains that could actually ensue from allocative strategies. Larger samples and data on clinical quality (as opposed to knowledge) may further increase the gains from such approaches if the variation in clinical quality is larger than that in knowledge, and larger samples could increase this variation even further.

In contrast, measurement error, either in caseloads or in competence, will overstate the gains from better allocative efficiency. The SDI surveys do not take repeat measurements, which are required to establish the reliability of these measures. For caseloads, the measures track well with other official statistics, but for facilities like central hospitals, where caseloads are very high, small errors can lead to large changes as the reassignment of just a few providers can lead to disproportionately large gains. In both these

**15/30**