metadata: authors: 0: author: Ibnat,Fabliha
1: author: Leonard,Kenneth L
2: author: Bawo,Luke
3: author: Mohammed-Roberts,Rianna L.
count: World
docna: 0: docna: The Three-Gap Model of Health Worker Performance
docty: Policy Research Working Paper
subtopic: Health Care Services Industry,Health Service Management and Delivery,Pharmaceuticals Industry,Pharmaceuticals & Pharmacoeconomics
teratopic: Industry,Health, Nutrition and Population
abstracts: cdata!: The Three-Gap Model examines the
            determinants of low-quality health care by examining the
            patterns and determinants of three gaps. Using four measures
            of performance -- target performance, actual performance,
            capacity to perform, and knowledge to perform -- this paper
            defines three gaps for each health worker: the gap between
            target performance and what they have the knowledge to do
            (the know gap), the gap between their knowledge and their
            capacity to perform (the know-can gap), and the gap between
            their capacity and what they actually do (the can-do gap).
            The paper demonstrates how the patterns of these gaps across
            health workers in a sample can be used to diagnose failures
            in the system as well as evaluate the outcomes of policy
            experiments. Using data on pediatric care from hospitals in
            Liberia, the paper illustrates how the model can be used to
            investigate the potential for improvements in the quality of
            care from several possible policy interventions. The
            analysis of the relationships between these gaps across
            health workers in a health system help to paint a better
            picture of the determinants of performance and can assist
            policy makers in choosing relevant policies to improve
            health worker performance.
repnme: repnme: The Three-Gap Model of Health Worker Performance
display_title: The Three-Gap Model of Health Worker Performance
listing_relative_url: /research/2019/03/30904180/three-gap-model-health-worker-performance
topicv3: Health Workforce m1326573 631,Pharmaceuticals m1327740 1350,Health, Nutrition and Population m1327755 621,Health Worker Performance and Incentives
            m1327756 1332,Health m1504661 3480
publishtoextweb_dt: 2019-03-14T00:00:00Z
historic_topic: Industry,Health, Nutrition and Population
no_of_pages: 47
author_keys: 0: author_key: 915919
model_extractions: Policy Research Working Paper 8782

# The Three-Gap Model of Health Worker Performance

## _Fabliha Ibnat_

_Kenneth L Leonard Luke Bawo Rianna Mohammed-Roberts_ Health, Nutrition and Population Global Practice March 2019
Policy Research Working Paper 8782

# **Abstract**

The Three-Gap Model examines the determinants of low-quality health care by examining the patterns and determinants of three gaps. Using four measures of performance—target performance, actual performance, capacity to perform, and knowledge to perform—this paper defines three gaps for each health worker: the gap between target performance and what they have the knowledge to do (the know gap), the gap between their knowledge and their capacity to perform (the know-can gap), and the gap between their capacity and what they actually do (the can-do gap). The paper demonstrates how the patterns of these gaps across health workers in a sample can be used to diagnose failures in the system as well as evaluate the outcomes of policy experiments. Using data on pediatric care from hospitals in Liberia, the paper illustrates how the model can be used to investigate the potential for improvements in the quality of care from several possible policy interventions. The analysis of the relationships between these gaps across health workers in a health system help to paint a better picture of the determinants of performance and can assist policy makers in choosing relevant policies to improve health worker performance.

This paper is a product of the Health, Nutrition and Population Global Practice. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://www.worldbank.org/research. The authors may be contacted at ibnatf@terpmail.umd.edu, kleonard@arec.umd.edu, and rmohammed@worldbank.org.

_The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent._ Produced by the Research Support Team
# The Three-Gap Model of Health Worker Performance<sup>_∗_</sup>

_† ‡ § ¶_ Fabliha Ibnat, Kenneth L Leonard, Luke Bawo, Rianna Mohammed-Roberts JEL Classification: C93, I1, J41, O1 Key Words: Pay for Quality, Pay for Performance, Health Care Evaluation, Liberia

> _∗_ We acknowledge the funding support from the Health Results Innovation Trust Fund at the World Bank and thank Gil Shapira for his comments on earlier drafts.

> _†_ Department of Agricultural and Resource Economics, 2200 Symons Hall, University of Maryland, College Park 20742 (email:ibnatf@terpmail.umd.edu).

> _‡_ Department of Agricultural and Resource Economics, 2200 Symons Hall, University of Maryland, College Park 20742 (email: kleonard@arec.umd.edu)

> _§_ Ministry of Health and Social Welfare, Monrovia, Liberia

> _¶_ The World Bank, Washington DC
# **1 Introduction**

Despite significant increases in funding directed at improving access to health care in developing and transition countries, important obstacles to quality care remain. Although overwhelming majorities in all countries now have access to a health facility, in many countries and settings, the quality of health care services is low, reducing the value of access. A primary symptom of this low quality is low adherence to medical protocols (Berendes et al., 2011; Das, 2011; Das & Gertler, 2007; Das & Hammer, 2007; Das et al., 2008; Holloway et al., 2013, 2015; Lange et al., 2014; Leonard & Masatu, 2010; Peabody et al., 2017; Rao et al., 2011). Given the significant evidence that increased adherence is one of the most effective ways to improve outcomes and prevent childhood deaths (Chopra et al., 2013; Jaribu et al., 2016; Manzi et al., 2018; Peabody et al., 2014), a continued focus on process quality seems appropriate.

An important element of understanding the determinants of low performance has been the documentation of a know-do gap in health care quality: health worker knowledge of correct protocol is often found to be well above performance, suggesting that lack of knowledge is not the only constraint for improved performance (Kabongo et al., 2017; Lange et al., 2014; Mohanan et al., 2015). This paper demonstrates that this focus on the size of the gap does not fully explain the scope of the problem, nor does it shed light on how to best increase performance. Figure 1 graphically illustrates why. The y-axis shows performance and the x-axis shows level of knowledge for a sample of hypothetical health care workers, plotted in blue. Performance is measured as the percentage of tasks the health worker has performed correctly and knowledge is measured as the corresponding percentage of vignette questions the health worker answered correctly. The dashed line shows the line of best fit for this sample of health workers and the solid 45-degree line represents the ideal level of 2
<!-- Start of picture text --> Average ' ae<br>know-do I ae<br>gap 1 7<br>oc<br>Kove<br><!-- End of picture text --> Kove
relationship between capacity, knowledge, and performance by modeling can as a function of know and do as a function of can.

With the focus now on the relationships between these factors rather than the size of the gaps, our model allows us to ask and answer two key questions. First, what are the slopes? Knowing the slope tells us the gain in performance we can achieve if we increase knowledge or capacity — through training or improving inputs — by one unit. Second, what are the determinants of these slopes? We argue that health worker motivation and improvements in equipment quality are two key inputs that can help increase the slopes and move them closer to the ideal 45-degree line. In this sense, we are modeling the knowcan and can-do relationships as nonlinear functions of inputs by interacting inputs with baseline levels of knowledge and capacity. This allows us to measure how well policy levers like improvements in infrastructure or incentives that increase motivation can improve the _relationships_ between knowledge and capacity, and capacity and performance respectively. In other words, it allows us to examine which policies increase the rate at which health workers can translate knowledge into capacity and capacity into performance.

These inputs can change the know-can and can-do relationships in different ways, leading to different policy implications. Some inputs may improve outcomes for all health workers, causing the pattern to simply shift upwards without changing its slope. In this paper we show that **positive worker behavior** , a motivation score measuring how proactive and positive a health worker is while on the job, is an input that causes the linear can-do function to shift upwards. Increasing **positive worker behavior** increases performance for health workers at all levels of capacity. Other inputs may cause the slope to increase by rotating the function, improving outcomes for some health workers and decreasing it for others. We show that **feeling valued by the facility** , a motivation score measuring the worker’s perception of whether she is valued by the hospital, is an input that causes the can-do function to rotate.

4
The two different ways an input can affect the know-can or can-do relationships have different policy implications. If an input simply shifts the function upwards, it can be implemented as a stand-alone intervention for all health workers meaning it does not need to be targeted to certain types of health workers or implemented alongside complimentary policies. Our empirical results show that a policy that increases **positive worker behavior** will improve performance uniformly for workers at all levels of capacity. Conversely, if an input increases the slope of a function by rotating the function, then it can be most effectively implemented as either a targeted intervention or alongside a complimentary intervention. We show that a policy that increases **feeling valued by the facility** will only increase performance for workers above a certain capacity threshold and will decrease it for workers below the threshold. Therefore, policy makers should either increase **feeling valued by the facility** for workers above this threshold only, or increase **feeling valued by the facility** and capacity through two interventions simultaneously.

In section 2 we introduce the Three-Gap model and outline three specifications that demonstrate how the role each input plays in improving performance can be empirically examined. In section 3 we use data from hospitals in Liberia to illustrate how this model can paint a fuller picture and provide policy makers with better information about how to improve the quality of care. Section 4 concludes.

# **2 The Three-Gap Model of Quality of Care**

The Three-Gap model starts with three measures of performance — performance, capacity and knowledge — and three gaps: the gap between what a health worker should be doing and what they have the knowledge to do (the know gap); the gap between what they have 5
the knowledge to do and their capacity to perform (the know-can gap) and the gap between what they have the capacity to do and what they do (the can-do gap). In this section, we show that these measures are not simply summary statistics for a health worker or system but can be used to analyze the determinants of quality and the possibilities for effective interventions.

# **2.1 A graphical illustration of the model**

To show the overall relationship between knowledge, capacity, and performance and these three gaps, we introduce the Three-Gap model illustrated with four interconnected graphs in Figure 2. Graph 1 in the upper left-hand corner displays capacity (C) and performance (D) and represents the can-do gap. Graph 2 on the upper right displays knowledge (K) and performance (D) and is used to show the know gap. Graph 4 in the lower right displays knowledge and capacity and represents the know-can gap. Graph 3 in the bottom left displays capacity (C) on both axes, which simply reflects capacity between the vertical axis in Graph 4 and the horizontal axis in Graph 1.

The characteristics of each health worker can then be represented by three sets of coordinates, where ( _K, C_ ) is the knowledge-capacity pair, ( _C, D_ ) is the capacity-performance pair, and ( _K, D_ ) is the knowledge-performance pair. In order to interpret Figure 2, first focus on Graph 1, and consider a single representative health worker with capacity _C_ 1 and performance _D_ 1. Using the 45-degree line, we can project (represented by blue arrows) the health worker’s capacity _C_ 1 onto the Y-axis to determine her “capacity to perform” — the level of performance if actual performance were equal to capacity. The can-do gap is represented by the distance labeled _G_ 3. Note that this gap is also the difference between the capacity performance pair ( _C_ 1 _, D_ 1) and the 45-degree line. The same health worker’s 6
<!-- Start of picture text --> Graph 1 Graph2<br>D D<br>Target performance<br>G, G,<br>(know gap)<br>Knowledge to perform<br>Capacity to performel<br>Performance anne nnn nen n nnn ee nn eee ee en ee eee teen ene ee teen ene! seeceesesseeee<br>G3 WA G3 (can-do gap)<br>(C,, 0,) c wy D,) xg<br>Graph3 : Graph 4<br>aiid: aie ie (K,,G,C,)(know-can gap) ThreeG, : know gaps:gap<br>G, : know-can gap<br>G, : can-do gap<br>c K<br><!-- End of picture text -->
between knowledge to perform and target performance in the Y-axis of Graph 1.

Although quantifying the sizes of the average gaps _G_ 1, _G_ 2, and _G_ 3 provides a snapshot of quality for a health worker or system, it is less useful as tool for exploring the possible effectiveness of policy interventions. By modeling capacity as a function of knowledge and performance as a function of capacity, Figure 2 illustrates how we can move away from one-dimensional measures of the gaps to a multi-dimensional approach of exploring the relationships between knowledge, capacity, and performance. Instead of asking how to reduce the size of the gaps of the average health worker, we instead ask how we can improve the relationships between these three measures.

# **2.2 The relationships between performance, capacity and**

# **knowledge**

Figure 3 shows how analyzing relationships rather than gap sizes can give a more complete picture of the drivers of poor performance. In this figure, the dashed lines show the lines of best fit for a hypothetical sample of health care workers. Knowledge and capacity (Graph 4) have a strong positive one-to-one relationship (a one-unit increase in K increases C by one unit) and capacity and performance (Graph 1) have a weak positive relationship (a oneunit increase in C increases D by half a unit). This is illustrated by the dashed lines on Graphs 4 and 1, which show slopes of 1 and 12 respectively. Note that just as the points representing a health worker are connected across the graphs, the slopes are related as well.

Given the relationships between know, can, and do, what is the most effective way of increasing performance? Consider a policy maker with two possible levers: increasing the knowledge of all health workers or increasing their motivation (an input that affects the 8
<!-- Start of picture text --> Graph 1 Graph 2<br>D D<br>-" Fy<br>> “ (C,, Da) a (K2, D2)<br>we (Cy Dy) we (Ky, Dy)<br>= C — K<br>Graph 3 Graph 4<br>Cc Cc<br>”<br>“ (K, C,)<br>a (Ky, Ci)<br>c _ K<br><!-- End of picture text -->
<!-- Start of picture text --> Graph 1 Graph 2<br>D ae)<br>a Fy<br>2 (C,, D,) 2° (Cc, D,) Poa (K, D,)<br>ie oe<br>Di VA)ee aea<br>= - Cc - K<br>C threshold<br>Graph 3 Graph 4<br>Cc Cc<br>ra (K2, C,)<br>“<br>oo (Ky, C,)<br>Cc - K<br><!-- End of picture text -->
<!-- Start of picture text --> Oo<br>—<br>e Gain from increasing effort<br>58 ° ‘<br>83 t. a sa<br>5 ° ® ? Gain from increasing knowledge<br>fl e e<br>5 8 e e Oo<br>= .<br>zg e 0% by ae ag a ~-<br>8 --*7 inal am‘ ~ ae oof% = 0%<br>o a ge os ge @ % e e<br>£° 7<br>oO Lad<br>a<br>o<br>0 10 20 30 40 50 60 70 80 90<br>Knowledge (% of required items)<br>e Individual clinician's knowledge and performance<br>—---—- Predicted quadratic relationship of knowledge to performance<br>——— Performance = Knowledge<br><!-- End of picture text -->
formance and knowledge given the observed distribution assuming zero competence leads to zero performance. If we assume that the average distribution of health workers is the best possible approximation of what would happen as a result of a policy intervention, then the dashed line predicts the result of an increase in knowledge for the average health worker. Consider a training intervention that increases knowledge by 20 percentage points (from 50% to 70%): we can see that the result would be an increase in performance of only 5 percentage points. On the other hand, an intervention that increases effort by shifting the line closer to the ideal 45-degree line can achieve an increase in performance of 15 percentage points. In this setting, it makes more sense to focus first on effort and motivation than on knowledge.

In Figure 5, the slope of the relationship was assumed to be constant. In the real world, this may be too strong an assumption. Alternatively, it is possible to introduce a motivation tool that alters the slope between capacity and performance, as shown in Figure 4. For example, if policy makers implement a reward for achieving a fixed level of performance, it might encourage low performing health workers to improve performance while having no effect on the performance of high capacity workers who already achieved the target.

Figure 5 also highlights the key shortcoming of improving performance via knowledge rather than effort in the current literature (also highlighted by Bucknall, 2012; Kabongo et al., 2017; Mohanan et al., 2015). A systematic review of the effects of a broad range of knowledge and training interventions finds that increasing health workers’ knowledge often has little or ambiguous effects on performance (Pantoja et al., 2017). The small increase in performance from knowledge in Figure 5 reflects these findings, and indicates that health facilities may see larger gains by increasing effort instead. Our model allows us to see this by breaking up the well–documented know-do gap into two components and focusing on changing the slopes that characterize the relationship between knowledge and performance in order to improve performance.

12
# **2.3 A mathematical representation of the Three-Gap model**

The relationships shown graphically in Figure 3 can be represented mathematically: where equation 1 shows the relationship in Graph 1; Equation 2, that of Graph 4; and Equation 3 the relationship in Graph 2. The coefficients _μ_ 1, _μ_ 2, and _μ_ 3 represent the slopes as seen in Figure 3: how well health workers translate capacity into performance, knowledge into capacity, and how far knowledge is from the target level.

One of the advantages of this representation is that these relationships can be more easily understood in the more nuanced space where multiple measures are considered. Equations 4 - 6 expand equations 1 - 3 to show the determinants of both the constants ( _η_ ) and the slopes ( _μ_ ). The policy levers include training ( _T_ ), equipment and infrastructure ( _I_ ) and motivation ( _M_ ) which interacts with the model through the unmeasured effort of health workers ( _E_ ( _M_ )).

13
We can analyze Graph 4 of figure 3 using equation 5. We rewrite equation 5 as: This expression shows that the slope of the dashed line in Graph 4 is _ν_ 2 + _γ_ 2 _I_ , and depends on infrastructure, _I_ . If this slope is close to 1, then the know-can gap is small, and improving knowledge will have a significant effect on improving capacity. This is the case in Figure 3. Similarly, we can analyze Graph 1 using equation 4. We rewrite equation 4 as: This expression shows that the overall slope _ν_ 1 + _γ_ 1 _E_ ( _M_ ) depends on motivation, _M_ . In Figure 3 this slope is not close to 1, indicating that the improvement in capacity implied by equation 7 does not translate into a significant increase in performance. However, we see that the slope depends not only on _ν_ 1, but also on _E_ ( _M_ ). In order to increase performance, the more effective policy is to increase effort through motivation. As discussed in Section 2.2, an increase in _E_ ( _M_ ) may affect the can-do relationship in one of two ways. If _β_ 1, the effect of _E_ ( _M_ ) on the intercept, is positive and _γ_ 1, its effect on the slope, is insignificant, motivation may cause the intercept to increase, shifting the function upwards as illustrated by the movement of function _F_ 1 to _F_ 2 in Figure 4. If _γ_ 1 is positive and _β_ 1 is negative, an increase in motivation may cause the slope to increase, as illustrated by the rotation of function _F_ 1 to _F_ 3 in Figure 4.

14
# **3 An Empirical Example**

We illustrate the uses of this model using data from pediatric care units for a baseline evaluation of hospitals in Liberia (Bawo et al., 2015). In this evaluation, performance is measured with actual patients (do) as well as on vignettes designed to measure both knowledge and capacity. In the vignettes, knowledge (know) is measured by asking health workers to demonstrate how they would treat a patient “assuming you have all the necessary equipment and supplies.” The capacity (can) of health workers is measured by evaluating whether the equipment and supplies the health worker proposed to use in the vignette were actually present. For example, if a health worker knows that protocol requires the use of a particular medicine, but that medicine is out of stock, capacity will be lower than knowledge. The instruments and data collection are discussed in more detail in Bawo et al. (2015) and in Appendix A.

Figure 6 shows the average relationship among the three measures of quality for each health worker in the sample.1 In addition, we plot the average performance, capacity, and knowledge as a green point in each graph and use them to show the average value of each gap. The different interpretations of the average gaps and the underlying relationships between the input measures are informative. The average know gap is about 55%, the average know-can gap is about 2.5%, and the average can-do gap is 8.9%. The imbalance in the gaps might suggest that increasing knowledge is the most important way to improve performance. In addition, the small can-do and know-can gaps might suggest that improvements in equipment and motivation are not important.

However, the relationships between knowledge and capacity and capacity and perfor-

> 1The relationship shown as dashed lines is derived from non-parametric representation of each health worker in the respective two-dimensional spaces. We used locally weighted kernel regression with a bandwidth of 10 percentage points and the Epanechnikov kernel.

15
<!-- Start of picture text --> D Graph 1 D Graph2<br>Know gap ed ood<br>© ©<br>Know- can gap<br>can-do gap ————<br>aa ll eT oo<br>' ae i<br>N ! N '<br>1<br>11<br>11<br>fo) !C Oo i K<br>0 2 4 6 8 1 0 eZ Al 6 8 1<br>' '<br>C Graph3:! C Graph4 |'<br>~<br>' ' i<br>0 <0 y il<br>! 1 i<br>' ' 4<br>as ee e~<br>(7<br>Fe<br>NNa<br>igq ° K<br>0 2 4 6 8 1 0 Pi 4 6 8 1<br><!-- End of picture text -->
Table 1 describes each relationship linearly. The three columns correspond to equations 1-3 outlined in Section 2. Column 1 is given by _K_ = _μ_ 3 1 + _E_ 3, column 2 by _C_ = _η_ 2+ _μ_ 2 _K_ + _E_ 2, and column 3 by _D_ = _η_ 1 + _μ_ 1 _C_ + _E_ 1. Column 1 describes the know gap, which is explained only by a constant because the target level of performance is unchanging. Columns 2 and 3 describe the know-can and can-do relationships, which are graphically represented by Graph 1 and Graph 4 in Figure 6.

Table 1: The know, know-can, and can-do gaps

||(1)<br>Know|(2)<br>Can|(3)<br>Do|
|---|---|---|---|
|Target|0<br>(.)|||
|Know||0.980<sup>_∗∗∗_</sup><br>(0.0300)||
|Can|||0.157<sup>_∗_</sup><br>(0.0850)|
|Constant|0.451<sup>_∗∗∗_</sup>|-0.0156|0.270<sup>_∗∗∗_</sup>|
||(0.0336)|(0.0157)|(0.0407)|
|Observations|831|831|831|
|Adjusted _R_<br>2|0.000|0.937|0.023|

Clustered standard errors in parentheses. _‡_ Target is a constant, so the regression produces only a constant: there cannot be a systematic relationship between knowledge and the target. _∗ p <_ 0 _._ 1,<sup>_∗∗_</sup> _p <_ 0 _._ 05,<sup>_∗∗∗_</sup> _p <_ 0 _._ 01 Ideally, these coefficients would estimate the variance in performance that resulted from exogenous variation in motivation or capacity from, for example, a randomized control trial3 This study, as a baseline, includes only cross-sectional variation, so we cannot make causal

> in hospitals work in teams so a clinician with low levels of knowledge might perform better when part of a team that includes more competent providers. Indeed, when we reanalyze the data focusing only on cases in which health workers worked as individuals, this area of the figure disappears. We retain this particular specification in the analysis because teams matter in this context.

> 3Variation in performance (not volume of services delivered) driven by  exogenous variation in capacity and motivation is seen in Basinga et al. (2011); Brock et al. (2016, 2018) and there are many studies of the link between performance and exogenous variation in training exposure (Jamtvedt et al., 2013).

17
statements. However, the model is designed to be used with exogenous variation, so we demonstrate its value in this context despite the danger of assuming these results are causal.

Taking the coefficients at face value, a one-unit increase in knowledge should lead to a .98 increase in capacity, but this one-unit increase in capacity will lead to only a .16 increase in performance. On the other hand, the 0.84 unit gap in performance for each unit of capacity suggests that significant increases in performance are possible even without any increase in knowledge or capacity. What can we learn about the value of increased knowledge versus that of motivation?

# **3.1 Investigating the role of policy levers**

In this study, we have three sources of data that we can use to examine the role of potential policy inputs on performance: cadre or level of training of each health worker, infrastructure and equipment levels in each hospital and the responses of each health worker to a survey on their attitudes and motivation.

Table 2 shows the effect of the relevant policy levers on the know gap. This regression corresponds to equation 6, _K_ = _β_ 3 _T_ + _ν_ 3 1 + _E_ 3, where _T_ is a vector of dummy variables for each cadre. Note that the target (a proper evaluation of children in the outpatient and emergency wards of a hospital) does not vary with cadre. The know gap does not appear to be strongly affected by cadre: the results show that, compared to the omitted category — physician’s assistants — only certified midwives increase the know gap, meaning that their levels of knowledge are further from the target. Since certified midwives are plausibly the only group whose training does not include pediatric care, this result is not particularly surprising. Using this narrow measure of training, we conclude that, with the possible 18
exception of certified midwives, all clinicians in this setting, are equally qualified to provide high-quality care.

Although we already know that capacity has a one-to-one relationship with knowledge in this setting, we can still examine the role of equipment in the know-can gap in Table 3. This regression corresponds to equation 5, _C_ = _α_ 2 + _β_ 2 _I_ + _ν_ 2 _K_ ( _T_ ) + _γ_ 2 _K_ ( _T_ ) _∗ I_ + _E_ 2. In the analysis of the relationships, it is important to include both the inputs of knowledge _K_ and equipment _I_ and the interaction between the two inputs. The level of equipment might help to explain the intercept on Graph 4 of the Three-Gap model: do increased levels of equipment improve the overall capacity of health workers? The interaction of know and the measure of infrastructure can help to explain the slope: as knowledge increases, does it make a difference if the equipment score is higher? In this case, unsurprisingly, the level of equipment available does not significantly alter either the intercept or the slope. This does not mean every facility has adequate equipment, only that their performance is not explained by the level of equipment.

To examine the determinants of the can-do relationship, we summarize the results of 54 questions on the attitudes and motivation survey, coming up with five summary measures of motivation: **self-satisfaction** , **feeling valued by the facility** , **job satisfaction** , **positive** 4 **facility characteristics** and **positive worker behavior** . The individual questions that make up the summary statistics are listed in Appendix B and each score is described briefly here:

> 4When the survey was conducted, the answers to the motivation questions were scored on a scale of 1 to 5, where 1 corresponds to “strongly disagree” and 5 corresponds to “strongly agree.” Afterwards, we flipped the scores of all negatively worded motivation statements for which answering “strongly disagree” indicated high motivation and “strongly agree” indicated low motivation. For example, the score for the negatively worded statement, “I do not get feedback from my superior so it is hard to improve my performance,” has been flipped so that a 1 means that the worker strongly feels that she does not get feedback from her superior and a 5 means that she feels she does.  Therefore, all motivation questions used in the analysis in this paper are measured on a scale of 1 to 5 where 1 indicates low motivation and 5 indicates high motivation.

19
Table 2: Policy levers that affect the know gap

||(1)<br>Know|
|---|---|
|_‡_<br>Target|0<br>(.)|
|Certified Midwife|-0.191<sup>_∗_</sup><br>(0.101)|
|Registered Nurse|-0.120<br>(0.0974)|
|Associate Degree Nurse|-0.0394<br>(0.148)|
|Bachelors of Science Nurse|-0.0700<br>(0.0879)|
|Medical Doctor|-0.0270<br>(0.0839)|
|Other|-0.101<br>(0.146)|
|Constant|0.508<sup>_∗∗∗_</sup><br>(0.0839)|
|Observations|767|
|Adjusted_R_<br>2|0.057|

Clustered standard errors in parentheses. _∗ p <_ 0 _._ 1,<sup>_∗∗_</sup> _p <_ 0 _._ 05,<sup>_∗∗∗_</sup> _p <_ 0 _._ 01 Note that the category “Physician Assistant” was chosen as the reference category for cadre in column 1 because it has the highest number of observations. It requires the second most years of education and training, after MDs.

_‡_ Target is a constant, so the regression produces only a constant.

20
Table 3: Policy levers that affect the know-can relationship

||(1)<br>Can|
|---|---|
|Know|0.921<sup>_~~∗∗∗~~_</sup><br>(0.114)|
|Average of necessary equipment working|-0.0485<br>(0.106)|
|Know_×_Average of necessary equipment working|0.103<br>(0.199)|
|Constant|0.0119<br>(0.0583)|
|Observations|831|
|Adjusted_R_<br>2|0.937|

Clustered standard errors in parentheses. _∗ p <_ 0 _._ 1,<sup>_∗∗_</sup> _p <_ 0 _._ 05,<sup>_∗∗∗_</sup> _p <_ 0 _._ 01

**Self-satisfaction** : Eight questions related to the worker’s perception of herself and not directly related to the facility or her role in it. Most of them measure her feelings of selfworth. For example, “I feel that I have a number of good qualities.”

**Feeling valued by the facility** : Nine questions related to the worker’s perception of whether she is a valued and useful employee in the facility. For example, “In this organization, I am valuable.”

**Job satisfaction** : Six questions related to the worker’s satisfaction with the facility and feelings of fulfillment in her job or profession. For example, “I would recommend to my children that they choose the health profession” and “I am satisfied with the opportunity to use my abilities in my job.”

**Positive facility characteristics** : Nineteen questions related to the facility’s processes, resources, and coworkers. For example, “This facility has a fair system for rewarding staff”

21
# and “Too often the referral system does not work efficiently.”

**Positive worker behavior** : Twelve questions related to the worker’s behavior on the job. For example, “I am keen to use any new tools to improve my performance” and “When I am not sure how to treat a patient’s condition I look for information or ask for advice.”

Note that the unifying theme for categories 1-3 is that they are subjective questions and capture the health worker’s perception of herself and her work environment. In contrast, categories 4 and 5 are more objective and give information about the characteristics of the facility and the worker’s role in it. All questions are coded so that positive/good responses get higher scores by reversing the score for negatively worded questions such as “Too often the referral system does not work efficiently.” We use confirmatory factor analysis (CFA) to determine the extent to which these five motivation dimensions, defined _a priori_ , are reflected in the data. Following the CFA best practices outlined by Borghi et al. (2018), we use structural equation modeling to assess the standardized factor loading values for each observed motivation variable onto the five latent factors we have identified above. Overall, we find that the factor loadings for most variables were significant at the p _<_ .001 level. Only the factor loading for one variable (“facility provides access to relevant trainings”) onto the fourth latent construct ( **positive facility characteristics** ) is not significant. Goodness of fit measures suggest acceptable fit for the model. Although the chi-square test indicates a poor model fit to the data (chi-square = 14784.18 with p _<_ .001), this measure of fit is sensitive to sample size and it is possible that the test will result in a rejection of an appropriate model when the sample size is large. The sample size of 783 in this analysis is considered to be large in the context of CFA. 5 Therefore, we use the root mean square error of approximation (RMSEA) to assess 22
# goodness of fit, as

5The minimum sample size for CFA is 200 observations (Borghi et al., 2018). The sample in this analysis is almost four times this size.

23
recommended by Borghi et al. (2018). The RMSEA measure of .08 indicates acceptable model fit.

After confirming that each of the five motivation dimensions are reflected in the data, we use factor analysis with varimax rotation to construct a latent variable for each category.6 We then normalize each factor to take on values in the range 0 to 1 in order to make the regression results easier to interpret.

In order to demonstrate the Three-Gap model approach, we use columns 1-3 of Table 4 to show the results of three less nuanced analyses of the effect of motivation on performance, and compare the implications of these results to the Three-Gap model analysis in column 4.

Column 1 shows the simplest analysis of the effect of motivation on performance, ignoring the role of capacity: is performance linked to measures of health worker motivation? This analysis suggests that **self-satisfaction** , **job satisfaction** and **positive facility characteristics** have no effect on performance. In contrast, increasing **feeling valued by the facility** leads to a decrease in performance and increasing **positive worker behavior** leads to an increase in performance. Why would a worker who feels more valued by the facility have lower performance? How should policy makers interpret this result?

In Column 2 we examine the effect of motivation on the size of the can-do gap directly, calculated as the health worker’s performance score subtracted from their capacity score. The results show that neither **positive worker behavior** nor **facility values the worker** now significantly changes the size of the can-do gap. Column 3 examines performance after controlling for capacity and finds similar results to Column 1.

Finally, we examine the effect of motivation using the Three-Gap framework in Column 22
6See Appendix C for rotated tables showing factor loading values.

22
4, by interacting each motivation factor with capacity, thereby allowing the relationship between capacity and performance to change with each measure of motivation. This regression corresponds to equation 4, _D_ = _α_ 1 + _β_ 1 _E_ ( _M_ ) + _ν_ 1 _C_ ( _K, I_ ) + _γ_ 1 _C_ ( _K, I_ ) _∗ E_ ( _M_ ) + _E_ 1, where _M_ is a vector of the five motivation factors outlined above. While columns 1-3 confound the role of motivation in improving performance, the results in Column 4 clarify the role of motivation and help explain the counter-intuitive results of the simple analysis. We see that allowing **feeling valued by the facility** to interact with _Can_ causes the _y-intercept_ to decrease but that the slope of the overall relationship between capacity and performance to significantly increase. This indicates that an increase in **feeling valued by the facility** allows health workers to translate capacity into performance at a higher rate. However, the decrease in the intercept indicates that the can-do gap itself only decreases for workers with high capacity. For workers with low capacity, being more valued by the facility actually decreases performance. In other words, increasing **feeling valued by the facility** causes the can-do function to rotate.

The intercept for **positive worker behavior** is now no longer significant, however, the p-value of the coefficient is 0.114 suggesting some possible positive effect. The coefficient for the slope (the interaction between the factor and capacity) is not significant. This could (cautiously) be interpreted to mean that workers with high levels of **positive worker behavior** provide higher performance independent of their level of capacity. In other words, increasing **positive worker behavior** causes the can-do function to shift upwards.

This has two policy implications for how to use the motivation policy lever and capacity inputs to improve performance. First, if a health facility implements a program that improves how the facility values the health worker, a simultaneous intervention that increases capacity for low capacity workers at baseline is necessary for workers to benefit from the increase in performance from the motivation intervention. If, instead, the policy maker does not have 23
Table 4: Policy levers that affect the can-do relationship

||(1)|(2)|(3)|(4)|
|---|---|---|---|---|
||Do|Can-do Gap|Do|Do|
|self-satisfaction|-0.0375|0.110|-0.0527|0.248|
||(0.0834)|(0.168)|(0.0853)|(0.310)|
|feeling valued by the facility|-0.378<sup>_∗∗_</sup>|0.167|-0.334<sup>_∗∗_</sup>|-1.326<sup>_∗∗_</sup>|
||(0.154)|(0.251)|(0.138)|(0.514)|
|job satisfaction|0.0119|0.234|-0.0394|-0.0112|
||(0.183)|(0.231)|(0.173)|(0.535)|
|positive facility characteristics|0.0782<br>(0.0979)|-0.167<br>(0.141)|0.0968<br>(0.0872)|-0.0261<br>(0.257)|
|positive worker behavior|0.395<sup>_∗∗∗_</sup>|-0.264|0.368<sup>_∗∗_</sup>|0.874|
||(0.146)|(0.298)|(0.149)|(0.548)|
|Can|||0.209<sup>_∗∗_</sup><br>(0.0864)|-0.422<br>(0.570)|
|Can_×_self-satisfaction||||-0.698<br>(0.747)|
|Can_×_feeling valued by the facility||||2.235<sup>_∗∗_</sup><br>(1.016)|
|Can_×_job satisfaction||||-0.195<br>(1.443)|
|Can_×_positive facility characteristics||||0.279<br>(0.645)|
|Can_×_positive worker behavior||||-0.867<br>(1.172)|
|Constant|0.297<sup>_∗∗∗_</sup>|-0.00902|0.237<sup>_∗∗∗_</sup>|0.469<sup>_∗∗_</sup>|
||(0.0844)|(0.115)|(0.0878)|(0.226)|
|Observations|616|616|616|616|
|Adjusted_R_<br>2|0.043|0.022|0.074|0.092|

Clustered standard errors in parentheses _∗ p <_ 0 _._ 1,<sup>_∗∗_</sup> _p <_ 0 _._ 05,<sup>_∗∗∗_</sup> _p <_ 0 _._ 01 24
the ability to increase **feeling valued by the facility** , then this analysis implies that it would also be effective to identify workers who work in facilities where the **feeling valued by the facility** score is high and increase their capacity. Thus, the **feeling valued by the facility** motivation factor works not by increasing the level of health worker performance, but by decreasing the can-do gap for high capacity workers.

On the other hand, if a policy maker knows how to change the way a health worker approaches her job, improving the **positive worker behavior** score, there is no reason to specifically target particular facilities or to pair the intervention with another intervention. Improving this factor will cause increases in performance at every level of capacity, closing the can-do gap for all workers.

# **4 Conclusion**

When the goal of policy makers is to increase the performance of health workers, it is important to understand what can and cannot be done to improve it. Policy makers are beginning to learn that, even though knowledge of proper protocol is low, improving knowledge often fails to have any significant impact on performance. Advancing from that position, we suggest that this model allows for even more nuanced understanding of the ways that policy makers can improve performance.

Using an example from a hospital setting in Liberia to illustrate the model, we show the value of measuring two intermediate inputs into health — knowledge to perform and capacity to perform — and comparing these to target performance as well as actual performance. In this setting, we show that knowledge, capacity and performance are all low, but that performance is only limited (currently) by knowledge and motivation. In other words, in this 25
setting, improvements in capacity are very unlikely to lead to improvements in performance. At first glance it seems that knowledge will not have an important impact on performance because there is a very weak link between knowledge and performance. However, when we use the model to take a closer look, we can see that knowledge might interact with a measure of motivation — the degree to which a health worker feels valued by their facility. As knowledge (and therefore capacity) increases, health workers who feel valued by their facility significantly improve their performance.

By deliberately examining performance through other models that are less nuanced than the Three-Gap model, we can show that this potentially important interpretation of the impact of policy could have been missed. In fact, if we look at performance in isolation, it looks as if feeling valued by one’s health facility significantly decreases performance on average. This demonstrates one of the important advantages of a more nuanced model.

This paper examines baseline data for a pay-for-quality intervention in hospitals in Liberia. The project is providing payments to facilities that improve performance as measured by quality (not quantity) and is paired with a plan to improve both training and equipment and facilities in hospitals. This baseline analysis suggests that the improvements in equipment are unlikely to lead to improvements in quality, but that the improvements in training might improve performance specifically because they are paired with an increase in motivation. If the project succeeds in motivating hospital administrators to pay attention to (value) the health workers within their facilities, then our analysis suggests that improvements in training will lead to improvements in performance.

26