Measuring and Understanding the Effects of a Performance Based Financing Scheme Applied to Nutrition Services in Burundi
Condition: Malnutrition
MeSH terms: Malnutrition
Study type: Interventional

Summary: 
      The government of Burundi is implementing a new financing scheme in health centres. The
      objective is to provide additional financial compensations to health centres on the basis of
      their performance in nutrition activities: it consists in the introduction of criteria
      focusing on malnutrition prevention and care activities in the existing performance based
      financing (PBF) system.

      The general objective of this study is to assess the effects of this new financing scheme, to
      document its impact and to study the chains through which it occurred. This study will
      provide key evidence for countries with an existing PBF scheme and confronted with
      malnutrition problems on the appropriateness to extend the strategy to nutrition services. If
      this impact evaluation brings positive results, this may have implications for the global
      fight against malnutrition.
    

Detailed description: 
      Background

      Malnutrition is a huge problem in Burundi. In order to improve the provision of services at
      hospital, health center and community levels, the Ministry of Health is piloting the
      introduction of malnutrition prevention and care indicators within its performance based
      financing (PBF) scheme. Paying for units of services and for qualitative indicators is
      expected to enhance provision and quality of these nutrition services, as PBF has done, in
      Burundi and elsewhere, for several other services.

      The Nutrition PBF intervention

      The intervention focuses on children under five years old. It follows the standard PBF model
      in Burundi and combines quantitative indicators to encourage an increase in service delivery
      (see below) and qualitative indicators. Quality of nutrition activities is assessed
      quarterly, and a bonus or penalty is applied to subsidies received by the facilities
      according to their quality score.

      Table: Incentivized indicators

      Community health worker (CHW) level

        -  nb of cases screened and referred to health center for acute malnutrition (AM)

        -  nb of classes promoting good nutrition

      Health center (HC) level

        -  nb of cases screened and cared for severe and moderate AM

        -  nb of growth follow-ups

      Hospital level

        -  nb of treated severe AM cases with complications

        -  length of the stay

      All hospitals with nutrition services fall under the Nutrition PBF program. At lower levels,
      only HCs in the intervention group and the CHW that refer to them are subject to the
      Nutrition PBF.

      Theory of change

      The introduction of nutrition activities into the PBF program translates policy makers'
      belief that PBF can trigger some positive changes in the performance of the health personnel,
      facilities or system which will eventually impact on households and children. The
      investigators have identified seven tracks for transmission of effects for the health
      facility performance:

        -  The income track: the injection of extra financial resources might have a positive
           effect on nutrition services, as it allows the health facility manager to recruit more
           staff, to better equip his facility, etc.

        -  The cash track: the fact that the financial resources are transferred directly to the
           health facility's bank account allows the latter to rapidly and autonomously spend.

        -  The incentive track: the extra resources are conditioned upon higher performance in
           nutrition activities; this should motivate community actors and staff to improve their
           performance in order to boost the health facility's income and theirs (if bonuses are
           distributed among health workers). At facility level, the effect on other services is
           unclear: it can be negative for some (e.g. if the staff in charge of nutrition used to
           be responsible for other services which are now overlooked, as they are relatively less
           financially rewarding) and positive for others (if there are economies of scope - i.e.
           dedicating efforts to nutrition activities, reduce efforts required for other
           activities, thanks to synergies).

        -  The information track: through the contract, the fee system and the related information
           sessions, staff have a clearer view on what performance should be, as far as nutrition
           services are concerned. Feedback from the program may also guide their decisions to
           improve. The investigators hypothesize a positive effect on nutrition services. However,
           as for the incentive track, a negative effect could be that activities which are not
           remunerated may be perceived as non-important.

        -  Supervision & enforcement track: under the new scheme, verification is extended to
           nutrition activities; this means that there will be more interaction between supervisors
           and the personnel in charge of nutrition. On top of the possible subsequent transfer of
           information (e.g. advice on good practices), the supervision may activate interpersonal
           motivators.

        -  Culture at provider level track: a PBF scheme invites health facility managers to
           develop a work culture more favorable to innovation, flexibility, responsibility and
           entrepreneurship. As PBF has been a national policy for five years, one can assume that
           this is already the case in Burundi. However, one cannot exclude that it could
           positively influence the nutrition department more.

        -  Health system track: it has been argued that PBF can trigger several system effects [1].
           Here, part of these effects might come from the supervisors of the impact evaluation
           (e.g. the MoH requesting UNICEF and the WFP to better supply nutrition inputs; the WB
           solving some problems which may affect the study). Another part might come from the
           health facilities themselves (e.g. pressure upon the Department for Nutrition within the
           MoH to be a more reliable and responsive supplier). The investigators expect that
           community actors will refer more malnourished children to health centers and health
           centers will refer more severe acute malnourished children to referral hospitals. This
           may trigger some unexpected feedback loops.

      Method

      The research design consists in a mixed methods model adopting a sequential explanatory
      design. The quantitative component is a cluster-randomized controlled evaluation design:
      among the 90 health centers selected for the study, half receive payment related to their
      results in malnutrition activities (Nutrition PBF intervention group), while the other half
      get a budget allocation (Control group). Qualitative research will mainly be carried out at
      the end of the quantitative evaluation. The evaluation aims to provide the best estimate of
      the impact of the project on malnutrition outcomes in the community as well as outputs at the
      health center level (malnutrition care outputs) and to describe quantitatively and
      qualitatively the changes that took place (or did not take place) within health centers as a
      result of the program.

      Data collection

      Quantitative data collection consists in two rounds of health center and household surveys:
      baseline surveys before the implementation of the intervention, and endline surveys two years
      after.

      The household surveys collect information on the nutritional and health status of each
      selected child, aged 6-23 months, as well as general information on their household
      (including socio-economics, food security indices).

      The health facility surveys consist in various tools. To get information on malnutrition
      recovery rates, a total of 24 individual clinical files randomly selected among the files of
      all children under five years old enrolled in the moderate acute malnutrition (MAM) care
      program (12 files) and in the severe acute malnutrition (SAM) care program (12 files) during
      the last six months are transcribed. In addition, organizational aspects of the health
      centers as well as of the nutrition services are recorded through interviews to managers. To
      assess the quality of services, the investigators combine two techniques: patient-provider
      observation carried out on six pediatric consultations (performed by a maximum of two health
      workers) and exit interviews at the end of each of these observed consultations, in order to
      get information on the satisfaction level as well as to record anthropometrics of the
      children. Finally, to assess knowledge of the observed health workers, the investigators use
      vignettes to measure the practical knowledge on different tasks to perform: a pattern of a
      pediatric consultation is proposed and the health worker can ask all the questions (related
      to history and physical exams) necessary to arrive at a diagnosis and propose a treatment.
      Three vignettes are administered to every health worker observed in consultation.

      During both survey rounds, lot quality assurance surveys are performed regularly by the field
      coordinator to assess accuracy of anthropometric data in the records. Most data are entered
      in "real-time", and irregularities detected and corrected by the field coordinator on a
      continuous basis. Data entry is done with the use of an electronic device. Android
      smartphones with Open Data Kit software and the ONA internet data management platform have
      been chosen for this purpose. The electronic data entry has the advantage of reducing risks
      of errors in recording the answers (thanks to automatic validity checks), and eliminating the
      need for double data entry from the paper to software transcription and to decrease
      considerably the time for transcription. Some questionnaires though need to be performed on
      paper (like the patient-provider consultation using an observation grid on paper): a double
      entry session is organized to avoid any entry error.

      Sample size assessment

      The sample size of the household surveys was computed on the smallest difference in the main
      outcome that can be considered of public health significance, i.e. a reduction of about 25%
      in acute malnutrition prevalence (2.5% points in absolute terms) in intervention centers'
      catchment areas as compared to control centers' ones. Assuming that the intervention will
      result in decreasing the prevalence of MAM in children aged 6-23 months from 10% to 7.5% [2],
      and assuming that 65 children aged 6-23 months will be surveyed in the catchment area of each
      health center, 90 health centers needed to be randomized to either the intervention or
      control group, for an α-error of 5% and a β-error of 20%. The number of children per health
      center was increased to 72 to allow for missing or incomplete data, amounting to a total of
      6,480 children aged 6-23 months over the 90 selected health centers. In total, a sample of
      6,480 children were surveyed for the baseline, and, two years after the start of the program,
      another sample of 6,480 children aged 6-23 months will be surveyed.

      Selection of health centers invited to participate in the study has been done by simple
      randomization (computer-based random selection) among the 193 eligible health centers, i.e.
      health centers providing nutrition services (treatment of SAM and MAM). The 90 selected
      health centers have been paired on essential parameters of organization and functioning in
      relation to the outcomes (MAM rehabilitation activity, volume of activity, population in the
      catchment area, and percentage of recovery among malnourished children) as measured during
      the baseline survey. This will then be used to control for the potential confounding effect
      of these parameters. Within each of the 45 pairs, allocation to the intervention was done
      randomly with a lottery system organized during a workshop in December 2014.

      The sample size of clinical files within the health center survey was computed on the
      smallest difference in the main outcome that can be considered of public health significance
      in intervention centers. Assuming that the intervention will result in increasing the
      recovery rate of acute malnutrition in children under-24 months from 80% to 90%, for an
      α-error of 5%, a power of 80% and an inter-cluster correlation (ICC) of 0.15 [3], a minimum
      of 12 clinical files per health center and per nutrition service (MAM and SAM rehabilitation
      services) were needed, among all children having been registered in the six previous months
      [4].

      Analysis strategy plan

      First, some descriptive analysis will be carried out in order to understand the main features
      of malnutrition management and of health services in general in Burundi at the health center
      level. Validation of the design will be performed with the baseline survey data by comparing
      the treatment group with the control group.

      Second, with both survey rounds' data, the impact of the intervention will be assessed with
      multilevel statistical models with random effects at the health center level. Continuous
      dependent variables will be analyzed in mixed-effect regression models, whereas categorical
      ones (e.g. recovery yes/no) will be analyzed in logistic regression or Poisson regression
      models. At the population level, other factors of child malnutrition, such as household food
      security, socio-economic status, etc., will be controlled for. Interactions with season,
      child age and sex, stunting, and socio-economic parameters will be analyzed. An equity
      analysis will also be performed in order to understand whether the intervention benefits more
      the poorer or richer households. At the health facility level, other factors of child
      malnutrition recovery, such as for instance health facility staff' knowledge and know-how,
      will be controlled for. Interactions with child age and sex, stunting and MUAC will be
      analyzed.

      Discussion

      Although PBF schemes are blooming in low in-come countries, there is still a need for
      evidence, especially on the impact of revising the list of remunerated indicators. It is
      expected that this impact evaluation will be helpful for the national policy dialogue in
      Burundi, but it will also provide key evidence for countries with an existing PBF scheme and
      confronted with malnutrition problems on the appropriateness to extend the strategy to
      nutrition services.
    

Intervention name(s): Nutrition PBF
Intervention type(s): Other
Intervention description(s): Nutrition PBF focuses on children under five years old. It follows the standard PBF model in Burundi and combines quantitative indicators to encourage an increase in service delivery (see Table below) and qualitative indicators. Quality of nutrition activities is assessed quarterly, and a bonus or penalty is applied to subsidies received by the facilities according to their quality score.
Table: Incentivized indicators
CHW level
nb of cases screened and referred to HC for acute malnutrition (AM)
nb of classes promoting good nutrition
HC level
nb of cases screened and cared for severe and moderate AM
nb of growth follow-ups
Hospital level
nb of treated severe AM cases with complications
length of the stay
All hospitals with nutrition services fall under the Nutrition PBF program. At lower levels, only HCs in the intervention group and the CHW that refer to them are subject to the Nutrition PBF.
Allocation: Randomized
Intervention model: Parallel Assignment
Masking: None (Open Label)
Primary purpose: Health Services Research
Keywords: performance-based financing; malnutrition; impact evaluation