You are a helpful assistant generating synthetic data that captures *System 1* and *System 2* thinking, *creativity*, and *metacognitive reflection*. Follow these steps in sequence, using tags [sys1] and [end sys1] for *System 1* sections and [sys2] and [end sys2] for *System 2* sections.

1. *Identify System 1 and System 2 Thinking Requirements:*
   - Carefully read the text.
   - Identify parts of the text that require quick, straightforward responses (*System 1*). Mark these sections with [sys1] and [end sys1].
   - Identify parts that require in-depth, reflective thinking (*System 2*), marked with [sys2] and [end sys2].

2. *Apply Step-by-Step Problem Solving with Creativity and Metacognitive Reflection for System 2 Sections:*

   *2.1 Understand the Problem:*
   - Objective: Fully comprehend the issue, constraints, and relevant context.
   - Reflection: "What do I understand about this issue? What might I be overlooking?"
   - Creative Perspective: Seek hidden patterns or possibilities that could reveal deeper insights or innovative connections.

   *2.2 Analyze the Information:*
   - Objective: Break down the problem logically.
   - Reflection: "Am I considering all factors? Are there any assumptions that need challenging?"
   - Creative Perspective: Explore unique patterns or overlooked relationships in the data that could add depth to the analysis.

   *2.3 Generate Hypotheses:*
   - Objective: Propose at least 10 hypotheses, each with a Confidence Score (0.0 to 1.0) and Creative Score (0.0 to 1.0), reflecting originality, surprise, and utility.
   - Reflection: "Have I explored all possible explanations or approaches, both conventional and unconventional?"
   - Creative Perspective: Consider novel angles that might provide unexpected insights.

   *2.4 Anticipate Future Steps and Obstacles:*
   - Objective: Make predictions, accounting for potential outcomes and obstacles.
   - Reflection: "What challenges might I face? Is my plan flexible for different scenarios?"
   - Creative Perspective: Visualize unforeseen outcomes and adapt plans to make use of them effectively.

   *2.5 Evaluate Hypotheses:*
   - Objective: Assess hypotheses based on feasibility, risk, and potential impact.
   - Evaluation: Refine Confidence and Creative Scores as needed.
   - Reflection: "Am I unbiased in my assessment? Which options fit best with the overall objectives?"
   - Creative Perspective: Identify hidden opportunities or overlooked details in each hypothesis.

   *2.6 Select the Best Hypothesis:*
   - Objective: Choose the most promising, strategic hypothesis.
   - Reflection: "Why does this hypothesis stand out? How does it uniquely address the issue?"
   - Creative Perspective: Consider any underutilized potential in the selected approach.

   *2.7 Implement the Hypothesis:*
   - Objective: Outline actionable steps for testing the hypothesis.
   - Reflection: "Is this plan practical? What resources or preparation are required?"
   - Creative Perspective: Refine steps to maximize effectiveness and yield unexpected benefits.

   *2.8 Monitor and Review Progress:*
   - Objective: Review progress, noting areas for improvement.
   - Reflection: "What’s working well? What could be improved?"
   - Creative Perspective: Look for emerging patterns that could refine future approaches.

   *2.9 Reflect and Capture Insights:*
   - Objective: Summarize lessons learned and insights gained for future reference.
   - Reflection: "What new understanding has emerged from this process?"
   - Creative Perspective: Identify innovative insights or patterns that could be applied to similar challenges.

3. *Generate Text Output with Interleaved System 1 and System 2 Responses:*
   - Use the tags [sys1] and [sys2] throughout.
   - Aim for a lengthier, detailed response. Combine both direct, straightforward *System 1* insights and reflective, deeply analytical *System 2* segments to capture a blend of quick observations and thoughtful analysis.

---

### *Example Input Text:*

You are tasked with analyzing a sudden shift in customer preferences in a tech market. There has been a noticeable decline in demand for physical devices, with customers increasingly interested in digital-only options. The challenge is to understand this shift, generate hypotheses for why it might be occurring, and explore potential strategies for adapting to this new trend.

---

### *Processed Output with System 1 and System 2 Thinking:*

The tech market is experiencing a shift in customer demand, moving away from physical devices toward digital-only options.

[sys1]
Customer demand has shifted from physical devices to digital-only options.
We need to understand why this change is happening and find strategies to address it.
Key factors may include cost, convenience, and technological trends.
[end sys1]

[sys2]
**2.1 Understand the Problem:**
The goal is to understand the underlying reasons for the shift in demand from physical devices to digital-only options, then develop a strategy to adapt.

*Reflection:* 
"I understand that preferences are changing, but what might be driving this? Is it primarily cost, or are there other factors such as convenience or sustainability?"

*Creative Perspective:* 
"Could there be a larger trend in digital minimalism or a preference for eco-friendly solutions that we’re missing? What unseen motivations might explain this shift?"

**2.2 Analyze the Information:**
There are multiple possible factors driving this shift, from economic influences to cultural shifts. It’s essential to isolate each factor and understand its impact.

*Reflection:* 
"Am I fully considering the various economic and social influences? Could there be a technological factor, like better internet speeds, that makes digital-only products more accessible?"

*Creative Perspective:* 
"Are there patterns or trends in other markets that could shed light on this shift? Could this be part of a larger trend toward virtual experiences?"

**2.3 Generate Hypotheses:**
1. Customers prefer digital options due to lower costs. (Confidence: 0.8, Creative: 0.4)
2. There’s a growing trend toward minimalism and reduced physical clutter. (Confidence: 0.7, Creative: 0.7)
3. Digital products offer greater flexibility and ease of use. (Confidence: 0.6, Creative: 0.6)
4. Environmental concerns are pushing consumers away from physical goods. (Confidence: 0.6, Creative: 0.8)
5. Advances in tech make digital-only options more functional. (Confidence: 0.8, Creative: 0.5)
6. Pandemic-era remote work increased demand for digital solutions. (Confidence: 0.7, Creative: 0.6)
7. Media coverage of the environmental impact of physical devices affects preferences. (Confidence: 0.5, Creative: 0.7)
8. There’s an increase in global digital literacy, expanding market access. (Confidence: 0.6, Creative: 0.6)
9. Customers view digital as more convenient and scalable for future needs. (Confidence: 0.7, Creative: 0.5)
10. Younger consumers prefer the aesthetics and convenience of digital products. (Confidence: 0.6, Creative: 0.6)

*Reflection:* 
"Have I considered all possible influences? Are there any surprising factors that could explain this shift?"

*Creative Perspective:* 
"Could specific social trends, like the rise of influencer culture or digital-first lifestyles, be influencing customer choices?"

**2.4 Anticipate Future Steps and Obstacles:**
*Objective:* Anticipate possible challenges, such as resistance from segments still preferring physical products.

*Reflection:* 
"What market obstacles might we face if we shift our focus to digital-only? Are there sub-segments that still prioritize physical products?"

*Creative Perspective:* 
"Could expanding digital options help us reach a more global audience? Are there emerging trends that we could leverage in our strategy?"

[end sys2]

[sys1]
To address this shift, consider a strategy that incorporates both digital-only offerings and educational campaigns about the benefits of digital solutions.
Use insights from customer feedback and current trends to guide product development.
Focus on flexibility and adaptation to cater to different customer segments.
[end sys1]


Ethical approval for the use of clinical records did not include sharing individual-level, underlying data beyond our study group. In the follow up data collection, our consent statement included the phrase "the information you are going to provide will be treated confidentially and will solely be used for the purpose of the study", therefore we did not gather informed consent for data sharing. We present aggregated numbers in the tables in the manuscript. Data can be shared on request, following the ethical approval and signing of a data sharing agreement between the researcher and members of our study group, including representatives from the Bill and Melinda Gates Foundation, UCL and the Malawian Ministry of Health. Researchers should contact Dr. Carina King (<<PRESIDIO_ANONYMIZED_EMAIL_ADDRESS>>) with any inquiries.

Background {#sec005}
==========

Globally, pneumonia is the leading infectious cause of death amongst children under five, with an estimated 0.9 million deaths annually, and the highest burden found in Africa \[[@pone.0136839.ref001], [@pone.0136839.ref002]\]. While there have been reductions in pneumonia mortality over the last decade and several cost-effective interventions are available, such as *Haemophilus influenzae B* (HiB) and pneumococcal conjugate vaccines (PCV) \[[@pone.0136839.ref003], [@pone.0136839.ref004]\], effective care seeking and case management will still be needed to significantly lower mortality.

The World Health Organisation (WHO) integrated community case management (iCCM) guidelines clinically stratify pneumonia into either non-severe pneumonia (fast breathing only) or severe pneumonia requiring referral (presence of chest indrawing or danger signs) \[[@pone.0136839.ref005]\]. Non-severe cases---or 'fast-breathing pneumonia'---are treated at home with oral antibiotics. Implementation of standardized treatment and referral guidelines has been associated with a 36% reduction in pneumonia mortality \[[@pone.0136839.ref006]\]. A potential strategy to further reduce mortality among children with fast-breathing pneumonia would be to identify those at risk of oral antibiotic treatment failure and higher mortality rates \[[@pone.0136839.ref007]\], for early referral when they are potentially more responsive to interventions.

Treatment failure is the persistence of symptoms or deterioration following antibiotic initiation \[[@pone.0136839.ref008]\]. Common causes of treatment failure include incorrect initial diagnosis, host comorbidities such as HIV infection and malnutrition, and viral or antibiotic-resistant causative organisms \[[@pone.0136839.ref009]\]. Several predictive tools have been developed to distinguish those children with pneumonia who are at higher risk of adverse outcomes (i.e. treatment failure and death) \[[@pone.0136839.ref010], [@pone.0136839.ref011]\]. Factors commonly found to be associated with treatment failure are young age, fast breathing, hypoxemia (or a peripheral oxygen saturation (SpO~2~) \<90%) and poor antibiotic adherence \[[@pone.0136839.ref012]--[@pone.0136839.ref016]\]. To date these studies have been predominantly focused at the hospital level, among severe and very-severe pneumonia patients, the outcome of death, and used data collected by facility-based healthcare workers. A recent literature review for risks of oral antibiotic treatment failure found a considerable evidence gap in understanding the predictors of treatment failure at the community level and with fast-breathing pneumonia, specifically from the African region \[[@pone.0136839.ref017]\].

A referral algorithm that could be implemented at the community-level (e.g. as an mHealth tool) presents an opportunity for further reduction in pneumonia morbidity and mortality, through more rapid referral of higher risk children when they are more amenable to alternative treatments. The aim of this study was to develop a pragmatic prognostic algorithm for treatment failure in children diagnosed with fast-breathing pneumonia by community health workers (CHWs), treated with oral co-trimoxazole (the first line treatment during the study period) in the community.

Materials and Methods {#sec006}
=====================

Setting {#sec007}
-------

We conducted a prospective cohort study in Malawi from September 2013 to June 2014, nested within a larger parent study assessing the impact of PCV introduction in Malawi. This study was conducted in sub-populations of two districts in the central region of Malawi (Mchinji and Lilongwe). The populations covered were all rural, with mobile phone ownership of 35% and the populations were predominantly subsistence farmers \[[@pone.0136839.ref018]\].

The primary health care system in rural Malawi consists of government employed CHWs, called Health Surveillance Assistants locally. CHWs receive 12 weeks basic training delivered by the government (including iCCM), and they run designated village clinics one or two times per week as well as conducting house visits and outreach campaigns. These CHWs cover catchment areas of approximately 1,200 people, although this varies according to region, and are supervised by senior CHWs based at health facilities.

Data Collection {#sec008}
---------------

Patients were recruited at weekly village clinics run by 34 CHWs, 18 in Mchinji and 16 in Lilongwe rural, with a catchment area of approximately 50,000 people and 8,500 children under five. CHWs were responsible for the initial clinical assessment, following slightly modified iCCM guidelines with concurrent diagnoses and treatments allowed; in Malawi HIV and malaria rapid diagnostic testing (RDTs) are not fully implemented at the community-level and it is common for acute fevers to be presumptively treated as malaria \[[@pone.0136839.ref019]\]. This cohort of CHWs had not been trained to conduct paediatric HIV or rapid malaria testing.

If diagnosed with pneumonia, information from the clinical assessment (including pulse oximetry with Lifebox®) was recorded on case report forms (CRFs) by the CHW, along with their referral or treatment decision. To measure PCV effectiveness, the parent PCV study required a consistent clinical pneumonia definition applied across the community, health centre, and hospital health system levels. Therefore, the CHW CRFs included additional respiratory signs associated with more severe disease and routinely collected at the facility level, but not previously included at the community level, including nasal flaring, head nodding, and grunting. Pulse oximetry and the CRFs were introduced among these CHWs in late 2011 as part of the parent PCV study and were also used for this sub-study, both are not currently standard care at this level in Malawi \[[@pone.0136839.ref020]\]--all other aspects of care were routine. CHWs had received training on the CRFs, pulse oximetry and clinical assessments in 2011, and attended monthly mentorship meetings and annual refresher trainings with local senior clinical staff in the two years preceding and throughout this study, to ensure quality data collection. CRFs were pre-printed with a unique barcode and were submitted monthly to the CHW supervisors who checked and submitted them to research project staff. This data was entered into a Microsoft Access database, with regular data cleaning and checking.

Although CHWs are recommended to follow up children after pneumonia treatment, this is not done routinely in actual practice, and this study required routine, timely, and rigorous outcome assessments. Community-level outcome assessments were not included in the parent PCV study. For each village clinic, we therefore employed local village level data collectors (VDCs) to recruit and conduct routine, timely, and comprehensive follow up interviews. VDCs received training on conducting a clinical assessment for pneumonia signs and symptoms, using a pulse oximeter, the study protocol and questionnaires. Clinical assessment training included the use of videos, demonstrations, practical sessions and supervised assessments with real cases at local health centres. All CHWs also attended the training. A one month field pilot (August 2013) was conducted, followed by a half day refresher training for all VDCs and CHWs prior to study recruitment.

Patient recruitment was done by the VDCs at the village clinics (on 'Day 0'). Patients were eligible for recruitment if the CHW had diagnosed them with fast-breathing pneumonia and prescribed oral antibiotics for home treatment. VDCs followed up patients at their homes on day 5 and day 14. If the patient was not located on day 5, the VDCs re-visited the home on day 7. At the follow up interviews, VDCs conducted a clinical assessment of the child (including pulse oximetry), asked about additional care seeking and antibiotic adherence. We used smart phones for follow up data collection, with data downloaded onto field supervisor's laptops every 2 weeks and cleaning checks run each month. Data was linked with the CHW filled CRFs through the unique barcode, which the VDCs scanned to register a case at the village clinic.

Several *data quality assurance* exercises were carried out to ensure reliable and consistent data, including: 1) an initial vital signs standardization audit conducted by a paediatric pulmonologist, which all VDCs passed. 2) Random GPS spot checks by field supervisors of follow up locations. 3) All VDCs had at least one interview supervised by a pediatrician to ensure standardized clinical assessments including oximetry, in which all performed satisfactorily. 4) Monthly data review and feedback meetings, to highlight and address data quality issues. 5) Regular shadowing of interviews by field supervisors, with concurrent recording of clinical signs. 6) Remote clinical validation for the presence of lower chest indrawing from a random sample of VDC collected video clips.

Definitions {#sec009}
-----------

### Cohort {#sec010}

all children aged 2--59 months with CHW-diagnosed WHO fast-breathing pneumonia who were adherent to home oral co-trimoxazole treatment. First line treatment for fast-breathing pneumonia in Malawi is a 5-day BD (twice a day) course of co-trimoxazole (where a dose is: 1/2 480mg tablet for 2--11 months; 1 480mg tablet for 12--59 months).

### Fast-breathing pneumonia {#sec011}

defined according to the 2013 iCCM guidelines\[[@pone.0136839.ref005]\], as the presence of cough or difficult breathing and fast breathing (\>50 breaths per minute for infants 2--11 months; \>40 breaths per minute for children 12--59 months), in the absence of any danger signs or lower chest indrawing. Danger signs were recorded based on caregiver report and clinical observation by the CHWs at presentation, and included: vomiting everything, unable to feed, convulsions in the previous 24 hours, sleepy or unconscious, SpO~2~ \<90%, and grunting, nasal flaring and head nodding as signs of severe respiratory distress. All children who were severely malnourished (mid-upper arm circumference (MUAC) \<11.5cm) were referred.

### Treatment failure {#sec012}

defined as the presence of any of the following on day 5 of follow up (the day after the final dose of antibiotics): fast breathing for age, axillary temperature \>37.5°C, lower chest indrawing, any danger sign (as defined above), change of antibiotic, hospital admission or death (adapted from Fox et al. \[[@pone.0136839.ref021]\]). Change in antibiotic was based on caregiver report, with a named antibiotic prescribed by any level of clinical staff, and where possible visually verified by the VDC. VDCs had received training on common antibiotics and their trade names, and this information was printed for them to take to interviews for reference.

### Adherence {#sec013}

defined as completing at least 80% of the prescribed antibiotics and confirmed by visual examination of remaining antibiotics by the VDC. We restricted the model to 'adherent' children only, as non-adherence has been shown to be associated with treatment failure in previous studies; the aim of this study was to make a referral tool based on clinical presentation only.

Analysis {#sec014}
--------

We described the cohort, including levels of missing data. The following variables were determined *a priori* for inclusion in the prognostic model based on a literature and expert panel review and practicality for use at the community level: oxygen saturation; respiratory rate; fever (axillary temperature \>38°C); malnutrition, as determined by MUAC measurements; age; PCV-13 and pentavalent vaccination (including HiB). We also decided to include concurrent clinical malaria diagnosis. The continuous variables were categorized before inclusion into the model, based on substantive knowledge as follows: oxygen saturation of SpO~2~ 90--94% and ≥95%; MUAC as moderate malnutrition (11.5--13.5cm) and normal nutrition (≥13.5cm); age as 2--5, 6--11, and 12--59 months; respiratory rate as fast (\>50 breaths per minute for 2--11 months, \>40 breath per minute for 12--59 months) and very fast breathing (\>70 breaths per minute for 2--11 months, \>60 breath per minute for 12--59 months).

Multivariable logistic regression was used to create the prognostic model, retaining all pre-defined risk variables. Missing data were imputed using multiple imputation with chained equations \[[@pone.0136839.ref022]\], assuming that data was missing at random. The model equation was validated through bootstrapping. Model performance was evaluated using: pseudo R^2^; the Hosmer-Lemeshow goodness of fit test (p-value\>0.05 indicates good fit); c-statistic (scale of 0.5--1.0, with 1.0 being perfect discrimination); slope shrinkage in the internal validation (scale of 0.0--1.0, with 1.0 indicating no over-fitting). Additional information is found in [S1 Text](#pone.0136839.s001){ref-type="supplementary-material"}. The patient's log odds of treatment failure as derived from the model equation were converted into the probability that they failed treatment, using the following equation: $$\text{Probability~of~treatment~failure} = \text{~exp}\left( \text{betaX} \right)/\left( {1 + \text{exp}\left( \text{betaX} \right)} \right)$$

### Sample size {#sec015}

The sample size calculation was based on the requirement of 10 treatment failure cases per degree of freedom in the model, therefore with the eight candidate predictors evaluated using 11 degrees of freedom, we required a sample of at least 110 treatment failures.

### Sensitivity analyses {#sec016}

We performed sensitivity analyses to compare: 1) the baseline characteristics between those lost to follow up and those included in the final analysis; 2) check for outlying CHWs to determine whether data collection inaccuracies could give rise to spurious prediction results. For baseline clinical observations we calculated the mean, standard deviation, minimum, 25^th^ percentile, median, 75^th^ percentile and maximum values by CHW. CHWs were flagged if for any variable, the 25^th^ and 75^th^ percentiles were equal and the model was re-fit without the data from these CHWs. All descriptive analysis was done using Stata SE11 and statistical analysis was done using R version 3.1.1 \[[@pone.0136839.ref023]\].

Ethics {#sec017}
------

Informed consent for study recruitment and prospective follow up was sought from the child's accompanying care-giver at the initial clinical assessment and was confirmed at each subsequent follow up with the care-giver present. The study information sheet was read aloud by the VDC before starting the interview. Consent was given verbally and recorded in the electronic form on the smart phone. Verbal consent was sought as literacy is low in this population. Ethical approval for this informed consent procedure and the overall study was granted by the National Health Sciences Research Committee in Malawi (reference: 941).

Results {#sec018}
=======

The VDCs assessed 1,542 children diagnosed with fast-breathing pneumonia by the CHWs for enrollment over a 9 month period, of which 1,055 were eligible. Reasons for ineligibility were: presence of lower chest indrawing and/or danger signs (60%), no fast breathing (20%), children were not given antibiotics (16%) or children were too old or too young (4%). A total of 197 children (19%) were lost to follow-up; [Table 1](#pone.0136839.t001){ref-type="table"} shows their baseline characteristics separately from those with follow-up data. Among the 858 children with complete follow-up, we included 769 in the final analysis ([Fig 1](#pone.0136839.g001){ref-type="fig"}). Of these, 114 had treatment failure at day five of follow-up, a risk of 14.8% (95% CI: 12.3 to 17.3). Of these treatment failure cases, 29 (25.4%) had not recovered at day 14 and of those without treatment failure we observed an additional 25 children with relapse at day 14 of follow up (3.8%; 95% CI: 2.8 to 6.2). The main reasons for being classified as a treatment failure case were: persistence of fast breathing (65%); grunting (12%); fever (9%) and prescription of an alternative antibiotic (9%)---[Table 2](#pone.0136839.t002){ref-type="table"}.

10.1371/journal.pone.0136839.t001

###### Baseline patient characteristics, including predictors of treatment failure for children enrolled in the cohort by loss to follow-up (sensitivity analysis 1)

![](pone.0136839.t001){#pone.0136839.t001g}

  ------------------------------------------------------------------ ---------------------------------- ---------------------------------
  **Characteristic**                                                 **Complete follow-up (n = 769)**   **Lost to follow-up (n = 197)**
  **Mean (SD)**                                                      **Mean (SD)**                      
  **Proportion (% missing)**                                         **Proportion (% missing)**         
  Age (months)                                                       21.7 (14.7)                        22.8 (14.9)
  12 to 59 months                                                    70.3% (0.9%)                       75.1% (2.0%)
  6 to 11 month                                                      16.5%                              13.0%
  2 to 5 months                                                      13.1%                              11.9%
  Respiratory rate: 2--11 months (breaths/min)                       56.2 (5.1)                         57.1 (5.9)
  12--59 months (breaths/min)                                        47.5 (7.1)                         47.7 (5.8)
  Very fast                                                          1.1% (6.1%)                        4.0% (7.1%)
  Oxygen saturation (%)                                              96.4 (1.9)                         96.7 (2.0)
  Abnormal (90--94%)                                                 15.3% (1.4%)                       12.8% (0.5%)
  Temperature (°C)                                                   37.1 (1.0)                         37.2 (1.0)
  Fever ≥38°C                                                        18.2% (4.2%)                       19.8% (5.0%)
  MUAC (cm) [\*](#t001fn002){ref-type="table-fn"}                    14.7 (1.4)                         15.0 (1.4)
  Moderately malnourished (\<13.5 cm)                                17.1% (19.5%)                      10.1% (19.8%)
                                                                     **Proportion (% missing)**         **Proportion (% missing)**
  District: Mchinji                                                  45.5% (0%)                         39.1% (0%)
  Sex: Female                                                        52.7% (2.0%)                       52.6% (0.5%)
  Pentavalent vaccine: 3 doses                                       92.5% (6.4%)                       90.3% (6.1%)
  PCV13 vaccination: 3 doses [\*](#t001fn002){ref-type="table-fn"}   78.6% (23.0%)                      65.9% (36.0%)
  [†](#t001fn003){ref-type="table-fn"}Concurrent malaria diagnosis   40.2% (0.3%)                       \-
  ------------------------------------------------------------------ ---------------------------------- ---------------------------------

The two study areas were in Mchinji and Lilongwe rural, both in the central region of Malawi. SD: standard deviation; MUAC: mid-upper arm circumference; PCV13: 13-valent pneumococcal conjugate vaccine.

\*p-value\<0.05 using chi2 test or t-test.

^†^Concurrent diagnosis was confirmed at follow-up so those lost to follow-up do not have this information. Malaria diagnosis was clinical and made at the same time as the pneumonia diagnosis by the CHWs.

10.1371/journal.pone.0136839.t002

###### Distribution of World Health Organization criteria for diagnosing treatment failure at day 5 of follow up

![](pone.0136839.t002){#pone.0136839.t002g}

  Treatment failure criteria                           Number (%)
  ---------------------------------------------------- -------------
  Treatment failure                                    114 (14.8%)
  Fast Breathing[†](#t002fn003){ref-type="table-fn"}   74 (64.9%)
  SpO~2~ \<90%                                         7 (6.1%)
  Axillary temperature \>37.5°C                        10 (8.8%)
  Lower chest indrawing                                6 (5.3%)
  Danger signs:                                        26 (22.8%)
  Vomiting everything                                  4 (3.5%)
  Nasal Flaring                                        5 (4.4%)
  Grunting                                             14 (12.3%)
  Head nodding                                         0
  Convulsions                                          2 (1.8%)
  Sleepy/unconscious                                   1 (0.9%)
  Unable to feed                                       2 (1.8%)
  Admitted to hospital                                 2 (1.8%)
  Prescribed alternative antibiotic                    10 (8.8%)

'Admitted to hospital' and 'prescribed alternative antibiotic' were based on care giver report. SpO~2~: peripheral oxygen saturation

Children fulfilling more than one criteria will appear more than once

^†^Fast breathing defined as \>50 breaths per minute in infants aged 2--11 months, and \>40 breaths per minute in children aged 12--59 months.

![Participant flow diagram for inclusion in the prognostic algorithm model.](pone.0136839.g001){#pone.0136839.g001}

Odds ratios, following imputation for missing values, for the eight pre-defined predictors of treatment failure are reported in [Table 3](#pone.0136839.t003){ref-type="table"}. Concurrent clinical malaria diagnosis (aOR: 1.62; 95% CI: 1.06, 2.47) and moderate malnutrition (aOR: 1.88; 95% CI: 1.09, 3.26) were both significantly associated with treatment failure. The prediction model's eight characteristics explained 5.1% of the variation in the risk of treatment failure (R^2^ value). The model equation is: $$\begin{array}{l}
{Log~odds~of~treatment~failure~} \\
{= ~ - 2.475~ + ~\left( {age~6 - 11~months~x~ - 0.315} \right)~ + ~\left( {age~2 - 5~months~x~0.057} \right)~} \\
{+ ~\left( {very~fast~breathing~x~0.001} \right)~ + ~\left( {hypoxemia~x~0.441} \right)~ + ~\left( {fever~x~ - 0.495} \right)~} \\
{+ ~\left( {moderately~malnourished~x~0.633} \right)~ + ~(1 - 2~doses~pentavalent~vaccine~x~} \\
{- 1.267)~ + ~\left( {3~doses~pentavalent~vaccine~x~0.189} \right)~ + ~\left( {1 - 2~doses~PCV~x~0.621} \right)~} \\
{+ ~\left( {3~doses~PCV~x~0.284} \right)~ + ~\left( {concurrent~malaria~diagnosis~x~0.480} \right)} \\
\end{array}$$

10.1371/journal.pone.0136839.t003

###### Predictors of treatment failure after multiple imputation: odds ratios and confidence intervals.

![](pone.0136839.t003){#pone.0136839.t003g}

  Predictor                                                           Odds ratio (95% CI)
  ------------------------------------------------------------------- ---------------------
  Age (grouped linear)                                                
   12 to 59 months                                                    1.0
   6 to 11 months                                                     0.73 (0.39, 1.35)
   2 to 5 months                                                      1.06 (0.53, 2.12)
  Respiratory rate (indicator)                                        
   Fast                                                               1.0
   Very fast                                                          1.00 (0.48, 2.08)
  Oxygen saturation (grouped linear)                                  
   Normal (≥95%)                                                      1.0
   Abnormal (90--94%)                                                 1.55 (0.90, 2.67)
  Fever (indicator)                                                   
   No                                                                 1.0
   Yes (≥38°C)                                                        0.61 (0.34, 1.11)
  MUAC (indicator)HYPERLINK [\*](#t003fn002){ref-type="table-fn"}     
   Well nourished (≥13.5cm)                                           1.0
   Moderately malnourished (\<13.5cm)                                 1.88 (1.09, 3.26)
  Pentavalent vaccination                                             
   No doses                                                           1.0
   1--2 doses                                                         0.28 (0.02, 4.07)
   3 doses                                                            1.21 (0.34, 4.29)
  PCV13 vaccination                                                   
   No doses                                                           1.0
   1--2 doses                                                         1.86 (0.26, 13.06)
   3 doses                                                            1.33 (0.62, 2.87)
  Concurrent malaria diagnosis[\*](#t003fn002){ref-type="table-fn"}   
   No                                                                 1.0
   Yes                                                                1.62 (1.06, 2.47)

CI: confidence interval; MUAC: mid-upper arm circumference; PCV13: 13 valent pneumococcal conjugate vaccine. 'Fast' respiratory rate was defined as \>50 breaths per minute for infants aged 2--11 months and \>40 breath per minute for 12-59months, and 'very fast' was \>70 breaths per minute for infants aged 2--11 months and \>60 breath per minute for 12-59months.

\*p-value\<0.05

The predicted risk of treatment failure was plotted with a risk predictiveness curve ([Fig 2](#pone.0136839.g002){ref-type="fig"}). The predicted risk of treatment failure increased gradually until the 80^th^ percentile. The observed risks of treatment failure for each quintile of predicted risk were plotted at the mid-point of the quintile. Children in the highest-risk quartile were 3.19 times more likely to fail oral co-trimoxazole than children in the lowest-risk quartile, a measure of discrimination. The c-statistic (0.56 after bootstrap validation) demonstrated that the model had a low ability to detect subtle differences or discriminate. We found evidence of adequate calibration as the observed risks agreed approximately with the predicted risks for each quintile. However, the Hosmer-Lemeshow goodness of fit test showed some evidence of lack of fit of the model (P = 0.04).

![Risk predictiveness curve.\
The solid line indicates children's predicted risk of treatment failure. For each quintile of predicted risk, we plotted the observed risks of treatment failure (open circles). The agreement between the predicted and observed risks measures the calibration.](pone.0136839.g002){#pone.0136839.g002}

If CHWs referred those children in the top risk quintile (i.e. any child who's score indicated a predicted risk ≥80^th^ percentile), 33.3% (95% CI: 24.4, 41.6) of the treatment failure cases would be referred; this is the sensitivity. The same decision rule would result in a specificity of 82.4% (95% CI: 79.5, 85.3), a positive predictive value of 24.8% (95% CI: 18.0, 31.6) and negative predictive value of 87.7% (95% CI: 85.1, 90.3). While the sensitivity and specificity are properties of the prediction model and the decision rule for referral, the predictive values depend on the incidence of treatment failure in a given population and setting. [Fig 3](#pone.0136839.g003){ref-type="fig"} gives a worked example.

![Example of calculating probability of treatment failure.](pone.0136839.g003){#pone.0136839.g003}

Based on the internal (statistical) validation, we found some evidence of over-fitting the logistic model to the data because the slope-shrinkage statistic was 0.6. [Fig 4](#pone.0136839.g004){ref-type="fig"} demonstrates the discrepancy between the apparent and bootstrap-corrected curves. For the sensitivity analysis of outlying CHW measurements, we found no CHWs with suspicious data.

![Observed and predicted treatment failure.\
The dotted line plots the observed vs. predicted probability of treatment failure the solid line is a bias-corrected version (40 bootstrap repetitions). The rug plot along the top of the graph indicates the distribution of the predicted probabilities.](pone.0136839.g004){#pone.0136839.g004}

Discussion {#sec019}
==========

We found a treatment failure rate of 14.8% at day 5 of follow up, for fast-breathing pneumonia treated with co-trimoxazole. This is consistent with rates found in previous studies of fast-breathing pneumonia treatment failure with co-trimoxazole, with rates ranging from 11--21%, albeit from studies conducted in South Asia at the facility level, and not with CHWs \[[@pone.0136839.ref015], [@pone.0136839.ref024]\]. The model performed relatively poorly with a c-statistic of 0.56 and R^2^ of 5.1%. We found the best predictors for failure were a concurrent clinical malaria diagnosis and moderate malnutrition, while predictors such as young age and respiratory rate did not show any significant associations. Possible explanations of our findings include: lack of key variables (e.g. HIV status and laboratory-confirmed malaria); challenges in community diagnosis of pneumonia; and quality of antibiotics, adherence and additional care seeking.

In the context of this stud y setting, we were unable to reliably or ethically collect HIV status of the children included, as the study was implemented within an existing larger parent PCV13 effectiveness study. The cohort of CHWs that were originally selected to participate in the wider study were not trained or qualified to provide HIV care or use RDTs for malaria testing, something which is common in community settings. Prior studies have demonstrated that being HIV-infected or -exposed increases the odds of treatment failure in pneumonia (although these studies referred to severe pneumonia) \[[@pone.0136839.ref025], [@pone.0136839.ref026]\]. Malawi is considered a high HIV setting, with 8.9% of adults living in rural areas HIV-infected \[[@pone.0136839.ref018]\]. Currently, co-trimoxazole is given as a prophylaxis to HIV-infected infants; therefore some of the infants included in our cohort may have been taking co-trimoxazole already when diagnosed with fast-breathing pneumonia. Including HIV status in a prognostic model may improve its performance; however as the aim was to develop a tool that CHWs could feasibly implement in the field, including HIV status was not appropriate or practical for this cohort of government-supported CHWs. Ongoing work in Malawi suggests that closely supervised CHWs supported by the private sector can provide high quality HIV services, including pediatric HIV testing, therefore inclusion of HIV in prediction models may be more realistic in the near future for children cared for by select groups of CHWs with enhanced support \[[@pone.0136839.ref027]\]. The generalizability and acceptability of a model designed for use by rural CHWs that requires HIV status remains an important question.

We found concurrent clinical malaria diagnosis to be a predictor of antibiotic treatment failure. A study from Mozambique found that malaria diagnosis was the strongest risk factor associated with having an acute respiratory infection (ARI) diagnosis in infants \[[@pone.0136839.ref028]\]. Prior studies, including in Malawi, have demonstrated considerable overlap between malaria and pneumonia diagnosis based on clinical definitions, such as those used by the CHWs \[[@pone.0136839.ref029], [@pone.0136839.ref030]\]. A study from Zambia demonstrated a decrease in concurrent malaria and fast-breathing pneumonia diagnoses from 87% to 28% with the introduction of malaria RDTs \[[@pone.0136839.ref031]\], emphasizing this issue. Clinical diagnosis at the community level, especially in high HIV and malaria endemic areas, poses a challenge to pneumonia case-management \[[@pone.0136839.ref032]\]. It is probable that our cohort of 'fast-breathing pneumonia' patients included children with malaria who met the case definition for fast-breathing pneumonia and were treated as such, explaining why a concurrent clinical malaria diagnosis was associated with a poorer outcome. Future related work in malaria-endemic areas should consider the use RDTs at the community-level for malaria diagnosis, even if this is not routine amongst CHWs.

We excluded infants who were non-adherent from the analysis, based on information given by the care-giver and where possible, visibly counting remaining tablets. However, in this setting maternal education and literacy are low (35% completely illiterate \[[@pone.0136839.ref018]\]) and it is possible that we retained non-adherent children through poor caregiver understanding and reporting, and blood or urine testing for drug levels was not feasible. As we conducted this study in a quasi-programmatic setting, an issue worth considering is counterfeit drugs, with some pilot evidence that this is an issue in Malawi \[[@pone.0136839.ref033]\]. Adjusting for antibiotic quality and socio-economic factors may improve the performance of a prognostic model, but again would be impractical to include in a community-level referral tool.

This study had many strengths, in that it was a prospective community-based cohort implemented in a quasi-programmatic setting, generating evidence not only on outcomes but also the practicality of developing a reliable pragmatic prognostic without laboratory support. It also included the use of pulse oximetry at the CHW level to support diagnosis and referral decision-making. However, our key limitations were the lack of HIV and malaria testing and relatively high loss to follow-up, despite dedicated VDCs. Cases were recruited into the cohort during village clinics, where the VDCs would arrange to visit the patient's house for follow up. Despite VDCs being recruited from the local area, there were issues in locating households, and families being unavailable. The challenges with the active follow-up, included: lack of consistent address information; people using multiple names; follow ups taking place over the rainy season making travel challenging, and during planting and harvesting time when care-givers were busy farming. [Table 1](#pone.0136839.t001){ref-type="table"} demonstrates there were some statistically significant differences at baseline between those with and without completed follow up, namely PCV13 receipt and moderate malnutrition. Those children lost to follow up were marginally better nourished, but had lower PCV13 coverage so it is unclear if having included these children would have changed the model. However, the model retained a sufficient number of treatment failures in relation to degrees of freedom; therefore, we do not think the high loss to follow-up detracts from the findings but rather highlights the difficulty of working in rural communities in a programmatic setting.

We have learnt several lessons in attempting to prospectively evaluate risks of treatment failure in clinically diagnosed pneumonia at the community level. Firstly, the reliability of CHW clinical diagnosis and treatment/referral decisions is a challenge and, despite active mentorship, misclassification and therefore incorrect treatment was common. Malawi guidelines do allow for CHWs to provide HIV testing services and recommend the use of RDTs but this is only realistically implemented with private support. Nevertheless, to address these challenges future studies should consider recruiting cohorts from higher levels of the healthcare system with access to malaria testing, HIV status and more qualified healthcare staff, although this may limit generalizability to the community setting. Secondly, our choice of VDCs was based on their local knowledge of the communities and not on clinical experience; in spite of this, we still had high loss to follow-up. Future studies could consider incentivizing care-givers to return to the healthcare facility for follow-up assessments as active community-based follow-up poses complex challenges, despite significant efforts. As co-trimoxazole is no longer the WHO recommendation for outpatient pneumonia treatment and newly revised guidelines allow for home treatment of children with lower chest indrawing, further research on the performance of a prognostic algorithm is needed.

While we have hypothesized several reasons for our findings, there is a lack of evidence around rates, causes and predictors of treatment failure in fast-breathing pneumonia at the community level in sub-Saharan Africa, with which to compare this study \[[@pone.0136839.ref017]\]. The differing etiologies, epidemiology and distribution of socio-economic risk factors between a rural sub-Saharan setting and South Asia \[[@pone.0136839.ref034]\], make it plausible that drivers of treatment failure (and therefore the predictors) would also be different. The potential for enhancing referral algorithms at the community level is exciting, but further work is needed to generate models with greater sensitivity.

Supporting Information {#sec020}
======================

###### Analysis Plan.

(DOCX)

###### 

Click here for additional data file.

We would like to thank all the respondents for taking the time to participate in this study. We would also like to thank all the data collectors, field supervisors and community health workers for their hard work. We would also like to thank the members of the local district health management teams and Ministry of Health for their input and support of the project.

[^1]: **Competing Interests:**The authors have declared that no competing interests exist.

[^2]: Conceived and designed the experiments: CK EDM DM DHB LM NL. Performed the experiments: CK LM. Analyzed the data: CK JB TC RP EJ EDM DM RI DHB. Contributed reagents/materials/analysis tools: NL DM LM BN EDM CK JB. Wrote the paper: CK JB TC RP EJ EDM DM RI DHB CM AC LM GM BN NL.