# Retention-in-care in the PMTCT cascade: definitions matter! Analyses from the INSPIRE projects in Malawi, Nigeria and Zimbabwe

Helene Font, N. Rollins, S. Essajee, Renaud Becquet, G. Foster, A. Z. Mangwiro, V. Mwapasa, B. Oyeledun, S. Phiri, N. A. Sam-Agudu, et al.

<figure><img src="image_1.png" /></figure>

## To cite this version:

Helene Font, N. Rollins, S. Essajee, Renaud Becquet, G. Foster, et al. Retention-in-care in the PMTCT cascade: definitions matter! Analyses from the INSPIRE projects in Malawi, Nigeria and Zimbabwe. Journal of the International AIDS Society, In press, 23 (10), pp.e25609. 10.1002/jia2.25609. hal-03120423

HAL Id: hal-03120423

https://hal.science/hal-03120423

Submitted on 25 Jan 2021

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
RESEARCH ARTICLE

# Retention-in-care in the PMTCT cascade: definitions matter!
## Analyses from the INSPIRE projects in Malawi, Nigeria and Zimbabwe

Helene Font¹,<fnref n="8" />* ID, Nigel Rollins²*, Shaffiq Essajee³, Renaud Becquet¹, Geoff Foster⁴, Alexio-Zambezio Mangwiro⁵, Victor Mwapasa⁶, Bolanle Oyeledun⁷, Sam Phiri<fnref n="8" />,⁹,¹⁰, Nadia A Sam-Agudu¹¹,¹² ID, Nita B Bellare¹³, Joanna Orne-Gliemann¹,<fnref n="8" />* and for the INSPIRE group

**Abstract**

**Introduction:** Definitions of retention-in-care in Prevention of Mother-to-Child Transmission of HIV (PMTCT) vary substantially between studies and programmes. Some definitions are based on visits missed/made, others on a minimum total number of visits, or attendance at a final clinic visit at a specific time. An agreed definition could contribute to developing evidence-based interventions for improving retention-in-care. In this paper, we estimated retention-in-care rates according to different definitions, and we quantified and visualized the degree of agreement between definitions.

**Methods:** We calculated retention in care rates using nine definitions in the six INSPIRE PMTCT intervention studies, conducted in three sub-Saharan African countries between 2013 and 2017. With data from one of the studies (E4E), we estimated the agreement between definitions using Gwet's agreement coefficient (AC1) and concordance. We calculated positive predictive values (PPV) and negative predictive values (NPV) for all definitions considering successively each definition as the reference standard. Finally, we used a Multiple Correspondence Analysis (MCA) to examine clustering of the way different definitions handle retention-in-care.

**Results:** Retention-in-care rates among 5107 women ranged from 30% to 76% in the complete dataset with Gwet's AC1 being 0.56 [0.53; 0.59] indicating a moderate agreement between all definitions together. Two pairs of definitions with high inner concordance and agreement had either very high PPV or very high NPV, and appeared distinct from the other five definitions on the MCA figures. These pairs of definitions were also the ones resulting in the lowest and highest estimates of retention-in-care. The simplest definition, that only required a final clinic visit to classify women as retained in care, and classified 55% of women as retained in care, had a PPV ranging from 0.7 to 1 and a NPV ranging from 0.69 to 0.98 when excluding the two pairs aforementioned; it resulted in a moderate to substantial agreement and a 70% to 90% concordance with all other definitions.

**Conclusions:** Our study highlights the variability of definitions in estimating retention-in-care. Some definitions are very stringent which may be required in some instances. A simple indicator such as attendance at a single time point may be sufficient for programme planning and evaluation.

**Keywords:** HIV care continuum; retention; Africa; women; PMTCT; outcome assessment

Received 8 November 2019; Accepted 15 July 2020

Copyright © 2020 The Authors. Journal of the International AIDS Society published by John Wiley & Sons Ltd on behalf of the International AIDS Society.

This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited.

# 1 | INTRODUCTION

Lifelong antiretroviral treatment (ART) for mothers living with HIV is critical for the Prevention of Mother-to-Child Transmission of HIV (PMTCT) [1]. ART benefits women's own health and survival and reduces HIV transmission risks for the child [2,3]. To achieve these, health services must reliably deliver care at every step of the PMTCT cascade [4,5] and women must continue to attend facilities, that is to be "retained in care" [6-9].

Measures of "retention-in-care" are sometimes used as a proxy for adherence to ART interventions and ultimately viral suppression [10]. However, there is currently no gold standard definition or metric of PMTCT retention-in-care [11-14]. Concepts such as loss to follow-up [15,16], engagement in care [17] or linkage to care [18] have been used to describe women's attendance and retention in care; these are clearly related but have different inferences and implications. A recent study comparing five definitions of point retention-in-

<footnote n="8">⁸Corresponding author: Helene Font and Joanna Orne-Gliemann, ISPED, Inserm, Bordeaux Population Health Research Center, UMR 1219, University of Bordeaux, 146 Rue Léo Saignat, 33000 Bordeaux, France. Tel: +33 5 57 57 13 93. (helene font@u-bordeaux.fr; joanna.orne-gliemann@u-bordeaux.fr)</footnote>

<footnote>*These authors have contributed equally to the work.</footnote>
care at 12-month post ART initiation, using the same dataset,
found that rates varied from 1.2% to 98% [19]. Similarly, when
using different data sources to estimate various definitions,
another study showed that rates of retention-in-care varied
from 41% to 72% [20]. Both studies underline the importance
of carefully choosing the retention-in-care definition as the
basis for analyses, and of using different definitions on the
same dataset [21].

Because of such variability, comparing rates of retention-in-
care between research or programmes is difficult and impedes
the development of evidence-based guidelines and evaluating
the effectiveness of interventions by settings. Yet, little is
reported about how definitions of retention-in-care are
related and how to choose between them.

We examined facility-attendance data and definitions applied in six intervention studies (known collectively as INSPIRE) that aimed to improve retention-in-care among mothers living with HIV in three sub-Saharan African countries. INSPIRE was an implementation research initiative launched in 2012 by the World Health Organization (WHO) with the goal of testing and integrating effective PMTCT interventions within existing health services [22]. Specifically, our objectives were to estimate rates of retention-in-care according to different definitions, and second to quantify the degree of agreement between retention-in-care definitions.

2 | METHODS

2.1 | Study setting

INSPIRE included five cluster-randomized controlled trials and one prospective cohort study conducted between 2013 and 2017 in Malawi (PURE and PRIME), Nigeria (MoMent and LJM) and Zimbabwe (E4E and EPAZ). Each study implemented different intervention packages, focused on the improvement of the local health system and/or the implementation of peer-support programmes. All six studies investigated retention-in-care as their main outcome, however, they applied different definitions of retention-in-care. Study designs and population characteristics are described elsewhere [23-28].

## 2.2 | Study samples

For the first objective, aiming at estimating rates of retention-
in-care according to different definitions, we used data of all
pregnant and post-partum women enrolled into the six stud-
ies, from both control and intervention arms. Women who
withdrew from the studies were excluded.

For the second objective, exploring agreement and differ-
ences between retention-in-care definitions, we used data
from one study only, in order to reduce variance due to differ-
ences in data collection methods, inclusion/exclusion criteria or
national PMTCT protocols. We used the E4E study data, as it
had a large sample size, high completeness of data regarding
dates of scheduled appointments.

## 2.3 | Data sources

All studies used routine facility data that were abstracted by
research staff from pre-natal and post-natal clinic registers or

patient cards. These data were captured into study-specific
databases and later extracted and merged for the purpose of
these analyses.

Clinic visit dates that were not directly related to prescrib-
ing or reviewing of ART were excluded for consistency. With
the exception of one study, the date of clinic visit by a preg-
nant woman or mother was accompanied by the date of the
next scheduled appointment. Since follow-up duration varied
between studies, we restricted the analysis to the data consis-
tently available, that is one year after study enrolment.

2.4 | Retention-in-care definitions

Nine definitions of retention-in-care were used in the analyses (Table 1). Six definitions were from the INSPIRE studies: four were based on missed visits [29-32]; one on the number of clinic visits [33]; and one on attendance at a final clinic visit [34]. We also used three additional definitions of retention-in-care identified in the literature and formulated around other concepts: visit constancy [35]; gaps in care [36]; and the number of isolated clinic visits [13], which is a South African measure (Health Resources and Services Administration HIV/AIDS Bureau) [37].

2.5 | Statistical analyses

We reclassified each woman, when feasible, according to each
retention-in-care definition.

For the first objective, we calculated retention-in-care rates
according to each definition for each study sample and overall.
We did not stratify results by control and intervention arms.
We applied the study-specific methodology for handling miss-
ing "scheduled appointment date" (e.g. case deletion or impu-
tation). The names of the six INSPIRE studies were replaced
by labels (A to F) as the aim of these analyses is not to com-
pare rates between specific studies but to examine the impor-
tance of chosen definitions on the variability of retention-in-
care rates.

For the second objective, we used different statistical meth-
ods on the E4E dataset, using case deletion for missing data.
The Gwet's first-order agreement coefficient (AC1) [38,39]
score was computed to assess the degree of agreement
among pairs of definitions in the classification of women as
retained or not retained. AC1 scores were interpreted as
"Excellent agreement" for scores over 0.80, "Substantial agree-
ment" for 0.61 to 0.80, "Moderate agreement" for 0.41 to
0.60, "Fair agreement" for 0.21 to 0.40 and "Slight agreement"
under 0.21 [40]. We also estimated the percentage of concor-
dant classifications between pairs of definitions (i.e. the pro-
portion of women that two definitions similarly classified as
retained or not retained). Furthermore, we calculated the Pos-
itive predictive values (PPV) and Negative predictive values
(NPV) of all retention-in-care definitions. As none of the defi-
nition is considered a gold standard, PPVs/NPVs were calcu-
lated, successively using each definition as the reference
standard for the eight others. PPV therefore corresponds to
the proportion of women retained per the reference standard
among those retained by the definition assessed. NPV corre-
sponds to the proportion of women who were considered as
not retained per the reference standard among those not
<table><caption>Table 1. Retention-in-care definitions</caption><thead><tr><th>#</th><th>Definitions</th><th>Reference</th><th>Country</th><th>N</th><th>Definition based on</th></tr></thead><tbody><tr><td>1</td><td>Being in care at 335 days post-delivery or later and no missed visits (&gt;14 days of the scheduled appointment)</td><td>[30]</td><td>Malawi</td><td>1350</td><td>Missed visits</td></tr><tr><td>2</td><td>Final visit (six-months postpartum ± 30 days) and no missed visits (&gt;30 days of the scheduled appointment)</td><td>[31]</td><td>Nigeria</td><td>532</td><td></td></tr><tr><td>3</td><td>No missed visits (≥60 days of the scheduled appointment)</td><td>[32]</td><td>Malawi</td><td>1269</td><td></td></tr><tr><td>4</td><td>Being in care at 335 days post-delivery or later and &lt;25% of missed visits (&gt;14 days of the scheduled appointment) + no gap in care &gt;90 days</td><td>[29]</td><td>Zimbabwe</td><td>1150</td><td></td></tr><tr><td>5</td><td>Attending at month 12 post-delivery [±1 month]</td><td>[34]</td><td>Zimbabwe</td><td>350</td><td>Final visit</td></tr><tr><td>6</td><td>Attending ≥4 times</td><td>[33]</td><td>Nigeria</td><td>497</td><td>Number of visits</td></tr><tr><td>7</td><td>Having ≥1 HIV clinic visit every three months</td><td>[35]</td><td>USA</td><td>782</td><td>Visit constancy</td></tr><tr><td>8</td><td>Time interval between completed clinic visits &lt;3 months</td><td>[36]</td><td>Ethiopia</td><td>346</td><td>Gaps in care</td></tr><tr><td>9</td><td>Having completed ≥2 visits separated by ≥3 months within a 12 months period</td><td>[37]</td><td>USA</td><td>—</td><td>Number of isolated visits</td></tr></tbody></table>

retained by the definition assessed. Lastly, to further understand concordance and differences between retention-in-care definitions, we conducted a Multiple Correspondence Analysis (MCA) considering, for each woman in the E4E study, the retention-in-care status according to the nine definitions. MCA is a descriptive and exploratory method used to visualize how observations, most often patients, are clustered according to multiple qualitative characteristics, and according to unmeasured observed "dimensions" [41]. We used it here to visualize the clustering of retention-in-care definitions according to individual status. As for individual MCAs, we interpreted factors that may explain this clustering.

## 2.6 | Ethics

All studies were approved by their respective national Ethics Committee and the World Health Organization Ethics Review Committee. EPAZ, LJM, MoMent, PRIME and PURE studies obtained written consent before enrolment. E4E study used de-identified data abstracted from registers and did not required individual consent. The Clinical Trials Registration numbers were NCT02070900 (E4E), NCT02216734 (EPAZ), NCT02214875 (LJM), NCT01936753 (MoMent) and NCT02005835 (PURE). The PRIME study registered with Pan African Clinical Trial Registry PACTR201312000678196. These analyses only used de-identified data.

# 3 | RESULTS

## 3.1 | Study population characteristics

After excluding 41 women who withdrew, a total of 5107 women living with HIV from all six INSPIRE studies contributed to the analyses. Median age of enrolled women was 28 years (IQR = 23 to 32) and 12.9% (N = 604) were primipara. Among the 4 360 women who enrolled while pregnant and with a known gestational age at enrolment (85% of the full sample), the majority booked during the second semester of pregnancy (N = 3 131; 72%), 526 (12%) booked late

during the third trimester and 703 (16%) booked during the first semester of pregnancy. Regarding ART initiation, 4242 women (83%) were not on ART at the time of first booking (a high percentage, in part, because it was an inclusion criterion in some INSPIRE studies). Among women with a known delivery outcome (N = 4 059), 3 889 (96%) had a live birth.

For the second analysis exploring the agreement and concordance between retention-in-care definitions, we only used the E4E study population of 1150 women (N = 1 073 after case deletion). Their median age was 26 years (IQR = [22 to 31]) and 22% (N = 233) were primipara. Of these, 68% (N = 719) booked for antenatal care during the second semester of pregnancy, 15% (N = 156) booked later during the third semester and 17% (N = 180) booked early during the first semester of pregnancy. All were ART naïve at enrolment. Live birth rate was 95% with 814 live births among 861 known delivery outcomes.

## 3.2 | Retention-in-care rates according to different definitions

Retention-in-care rates for the INSPIRE study populations, estimated according to the nine definitions, are presented in Table 2. Global retention-in-care rates ranged from 30% (with definition #1 "No missed visit defined by two weeks after an appointment") to 76% (with definition #9 "number of isolated clinic visits"). The variability in rates was even greater when different definitions were applied to individual study datasets, for example rates for study population B ranged between 12% and 79% according to the definition used.

The variability of the estimated retention-in-care rates across different studies also differed according to the definition used. Focusing on study samples A to D (i.e. excluding the samples with missing data), we observed that retention-in-care rates varied little with the definition based on the number of isolated clinic visits (#9), from 73% to 81%. However, when using the "No missed visit defined by two weeks after an appointment" definition (#1), the difference of retention-in-care rates was higher, from 12% to 33%.
<table><caption>Table 2. Retention-in-care rates for each of the six INSPIRE study populations and combined, according to the nine definitions</caption><thead><tr><th>#</th><th>Definitions</th><th>INSPIRE All<br>n = 5107</th><th>Study A<br>sample</th><th>Study B<br>sample</th><th>Study C<br>sample</th><th>Study D<br>sample</th><th>Study E<br>sample*</th><th>Study F<br>sample*</th></tr></thead><tbody><tr><td>1</td><td>No missed visit – 14 days</td><td>29.9</td><td>32.6</td><td>11.6</td><td>32.7</td><td>31.6</td><td>–</td><td>–</td></tr><tr><td>2</td><td>No missed visit – 30 days</td><td>38.1</td><td>38.0</td><td>23.8</td><td>43.3</td><td>38.7</td><td>–</td><td>–</td></tr><tr><td>3</td><td>No missed visit – 60 days</td><td>63.0</td><td>62.8</td><td>52.3</td><td>68.9</td><td>61.4</td><td>–</td><td>–</td></tr><tr><td>4</td><td>&lt;25% missed visit</td><td>54.8</td><td>55.6</td><td>41.6</td><td>60.0</td><td>53.7</td><td>–</td><td>–</td></tr><tr><td>5</td><td>Final visit</td><td>55.0</td><td>56.4</td><td>60.7</td><td>59.9</td><td>51.7</td><td>67.2</td><td>31.8</td></tr><tr><td>6</td><td>Number of visits</td><td>70.3</td><td>66.3</td><td>75.7</td><td>76.9</td><td>74.4</td><td>80.2</td><td>42.3</td></tr><tr><td>7</td><td>Visit constancy</td><td>53.0</td><td>49.6</td><td>56.6</td><td>63.1</td><td>56.4</td><td>62.9</td><td>17.9</td></tr><tr><td>8</td><td>Gap in care</td><td>43.4</td><td>39.4</td><td>48.9</td><td>52.4</td><td>43.9</td><td>53.7</td><td>17.5</td></tr><tr><td>9</td><td>Number of isolated visits</td><td>75.9</td><td>73.0</td><td>79.3</td><td>80.8</td><td>77.0</td><td>84.8</td><td>58.8</td></tr></tbody></table>

*One study did not collect scheduled appointment dates and another had a high proportion of women with at least one missing scheduled appointment dates. As this variable was necessary to compute definitions based on missed visit, rates for these definitions are missing for two studies.

## 3.3 | Agreement and concordance between retention-in-care definitions

The agreement (Gwet's AC1 score – top right of the table) and concordance (% - bottom left of the table) between definitions of retention-in-care are shown in Figure 1. The overall Gwet's AC1 was 0.56 [0.53; 0.59] indicating a moderate agreement between the nine retention-in-care definitions. Pairwise analyses showed a majority of moderate to substantial agreements and concordance levels of 70% to 90%. For example, definitions #5 and #2 had a substantial agreement (AC1 = 0.7) and agreed in categorizing women as “retained” in 85.4% of cases.

The two definitions resulting in the highest rates (#6 and #9) and the two resulting in the lowest rates of retention-in-care (#1 and #2) each demonstrated high levels of agreement and concordance (AC1 = 0.9 and 0.7, concordance = 96.7% and 84.8%, respectively). However, agreement and concordance between these two pairs (#6 and #9 vs. #1 and #2) were, at best, only fair (AC1 from 0.1 to 0.3, concordance from 53.8% to 63.5%).

## 3.4 | Positive and negative predictive values

The PPV/NPV calculated for the nine definitions are shown in Table 3, with definition #1 to #9 successively considered in each column as the reference standard for PPV/NPV of the eight other definitions considered as diagnostic tests. The highest PPVs (0.83 to 1) were found for definitions #1 and #2 (those resulting in the lowest rates of retention-in-care), using any of the other seven as a reference standard; however, definitions #1 and #2 had NPVs ranging from 0.33 to 0.79. In contrast, definitions #6 and #9 (those resulting in the highest retention-in-care estimates) had very high NPVs (0.98 to 1) and lower PPVs (ranging from 0.40 to 0.79). Definition #5, the simplest definition, which only required a final clinic visit to classify women as retained-in-care, showed PPVs ranging from 0.54 to 1 and NPVs ranging from 0.47 to 0.98. However, when excluding as reference standard the four definitions resulting in the lowest and highest rates of retention-in-care

(#1, #2, #6 and #9), PPVs for definition #5 ranged from 0.70 to 0.90 and its NPVs ranged from 0.69 to 0.83.

## 3.5 | Multiple correspondence analysis

Findings of the MCA conducted with the nine retention-in-care definitions are shown in Figure 2a. The nine definitions, represented as points, are distributed roughly in the same area of the graph. More specifically, they are aligned principally along the x-axis, which describes the main underlying dimension common to all nine definitions, and that we thus interpreted as a retention-in-care dimension. This suggests that, despite differences, all definitions are globally similar in measuring retention-in-care. Definitions #1, #2, #6 and #9 however, are grouped slightly apart from the other five along the y-axis. This suggests that there are some differences in the way these four definitions perform in terms of classifying women as retained or not, as compared to the other definitions. Figure 2b shows in more detail how the retention-in-care response relate one to another. The first observation here is that, for all definitions, “retained” is situated on the left on the x-axis whereas “Not retained” is located to the right, confirming the observation in Figure 2a, that all nine definitions generally concur in classifying women for retention in care. The second observation is that definitions #1 and #2 seems to cluster higher on the y-axis in the “retained” group, and definition #6 and #9 cluster similarly in the not-retained group. We interpret this as being consistent with the fact that these two pairs of definitions are similarly more restrictive (#1 and #2) in defining retention in care and more restrictive (#6 and #9) in defining non retention in care.

# 4 | DISCUSSION

In this pooled analysis of data from the six INSPIRE implementation research studies, we observed considerable variability in estimated retention-in-care rates among pregnant women and mothers living with HIV according to the definitions used. When applied to the same dataset, different
<figure><img src="image_3.png" /><figcaption>Figure 1. Retention-in-care definitions and modalities: Representation of the two first dimensions of the MCA (66% and 12% of inertia respectively). (a) Representation of the definitions. (b) Representation of the response modalities.</figcaption></figure>

<table><caption>Table 3. Positive (PPV) and Negative Predictive Values (NPV) for each retention-in-care definition examined with each definition alternatively considered as the diagnostic test and as the reference standard</caption><thead><tr><th rowspan="2">Definitions</th><th rowspan="2"></th><th colspan="9">Considered as the reference standard</th></tr><tr><th>#1</th><th>#2</th><th>#3</th><th>#4</th><th>#5</th><th>#6</th><th>#7</th><th>#8</th><th>#9</th></tr></thead><tbody><tr><td colspan="10">Considered as the test</td></tr><tr><td>#1</td><td>PPV</td><td>0.87</td><td>1</td><td>1</td><td>0.88</td><td>0.99</td><td>0.96</td><td>0.83</td><td>0.99</td></tr><tr><td></td><td>NPV</td><td>0.84</td><td>0.57</td><td>0.68</td><td>0.66</td><td>0.37</td><td>0.61</td><td>0.74</td><td>0.33</td></tr><tr><td>#2</td><td>PPV</td><td>0.71</td><td>0.99</td><td>0.98</td><td>0.98</td><td>0.99</td><td>0.93</td><td>0.82</td><td>0.99</td></tr><tr><td></td><td>NPV</td><td>0.93</td><td>0.63</td><td>0.75</td><td>0.78</td><td>0.41</td><td>0.66</td><td>0.79</td><td>0.37</td></tr><tr><td>#3</td><td>PPV</td><td>0.51</td><td>0.62</td><td>0.84</td><td>0.76</td><td>0.99</td><td>0.88</td><td>0.73</td><td>1</td></tr><tr><td></td><td>NPV</td><td>1</td><td>0.99</td><td>0.95</td><td>0.86</td><td>0.64</td><td>0.93</td><td>1</td><td>0.58</td></tr><tr><td>#4</td><td>PPV</td><td>0.59</td><td>0.71</td><td>0.96</td><td>0.85</td><td>0.99</td><td>0.93</td><td>0.76</td><td>0.99</td></tr><tr><td></td><td>NPV</td><td>1</td><td>0.99</td><td>0.79</td><td>0.87</td><td>0.54</td><td>0.85</td><td>0.92</td><td>0.48</td></tr><tr><td>#5</td><td>PPV</td><td>0.54</td><td>0.73</td><td>0.9</td><td>0.88</td><td>0.99</td><td>0.87</td><td>0.7</td><td>1</td></tr><tr><td></td><td>NPV</td><td>0.93</td><td>0.98</td><td>0.69</td><td>0.83</td><td>0.51</td><td>0.76</td><td>0.83</td><td>0.47</td></tr><tr><td>#6</td><td>PPV</td><td>0.42</td><td>0.51</td><td>0.81</td><td>0.71</td><td>0.68</td><td>0.76</td><td>0.59</td><td>1</td></tr><tr><td></td><td>NPV</td><td>0.99</td><td>0.99</td><td>0.98</td><td>0.99</td><td>0.98</td><td>1</td><td>1</td><td>0.88</td></tr><tr><td>#7</td><td>PPV</td><td>0.53</td><td>0.63</td><td>0.95</td><td>0.88</td><td>0.79</td><td>1</td><td>0.76</td><td>1</td></tr><tr><td></td><td>NPV</td><td>0.97</td><td>0.94</td><td>0.84</td><td>0.91</td><td>0.85</td><td>0.59</td><td>0.97</td><td>0.53</td></tr><tr><td>#8</td><td>PPV</td><td>0.59</td><td>0.71</td><td>1</td><td>0.91</td><td>0.81</td><td>1</td><td>0.97</td><td>1</td></tr><tr><td></td><td>NPV</td><td>0.9</td><td>0.88</td><td>0.7</td><td>0.77</td><td>0.72</td><td>0.46</td><td>0.76</td><td>0.41</td></tr><tr><td>#9</td><td>PPV</td><td>0.4</td><td>0.5</td><td>0.79</td><td>0.69</td><td>0.66</td><td>0.96</td><td>0.73</td><td>0.57</td></tr><tr><td></td><td>NPV</td><td>0.99</td><td>0.99</td><td>0.99</td><td>0.99</td><td>0.99</td><td>0.99</td><td>1</td><td>1</td></tr></tbody></table>
<figure><img src="image_4.png" /><figcaption>Figure 2. Agreement (right side of the diagonal) and concordance (left side of the diagonal) between retention-in-care definitions (using E4E dataset only). Labels of Definitions are in the diagonal. "Excellent agreement" was for scores over 0.80, "Substantial agreement" for 0.61 to 0.80, "Moderate agreement" for 0.41 to 0.60, "Fair agreement" for 0.21 to 0.40 and "Slight agreement" under 0.21 [40].</figcaption></figure>

definitions yielded rates ranging between 30% and 76%. Some definitions, especially more stringent definitions, appeared to result in greater variability of estimated retention-in-care rates according to study settings and populations. Retention-in-care definitions based on missed visits, and with stricter criteria (zero missed visits and short intervals between scheduled and actual clinic visits, definitions #1 and #2), not surprisingly, led to the lowest estimates of retention-in-care. Conversely, the two definitions based on the number of clinic visits attended (#6 and #9) systematically led to the highest retention-in-care rates. This inter-definition variability suggests that some heterogeneity in retention-in-care rates reported in the literature [42,43] could be attributed to differences in definitions applied, as much as or even more than to differences in intervention efficacy or programme quality. Any retention-in-care estimation should thus consider the influence of the definition applied and even explore the effect through sensitivity analyses. Any consensus position, for example determined by UNAIDS or WHO, should consider the purpose of the definition, that is for programme review, national monitoring and comparison or research, and the respective strength and limitation of each definition approach.

The agreement and concordance analyses, as well as the MCA analyses, were consistent in their findings and confirmed the differences between two specific pairs of definitions: those leading to the lowest estimates that were based on the concept of missed visits (#1 and #2) and those leading to the highest estimates that were based on the number of clinic visits (#6 and #9).

Perhaps surprisingly, the definition that was based only on "final clinic visit" (#5) performed very similarly to other definitions. Despite its very simple construct (only using one final clinic visit), definition #5 seemed to capture a large part of the information conveyed by more complicated definitions that emphasized recurrent attendances throughout PMTCT follow-up. Its PPV and NPV indicate that this simplest definition could reflect the women's clinic attendance over the preceding 12 months with similar positive and negative predictive value as other definitions. If found to be robust in other analyses, this single data variable would be easy for health systems

to prioritize and capture accurately. Others have already advocated for a single clinic visit definition [44] because of its simplicity and that it clearly defines "out-of-care patients." Also, a definition based on a specific clinic visit date may be less susceptible to the quality (sometimes poor) and the availability (also sometimes poor) of routine health data.

Our study had several limitations. First, we explored only a limited number of retention-in-care definitions and applied them to a specific research population. Definitions related to linkage to care [18] and loss to follow-up [16,45] have been reported from other HIV-related programmes and may provide valuable insights related to retention-in-care. Second, we limited our analyses to a 12-month period. Exploring the effects of applying retention-in-care definitions over longer periods of time, for example 24 or 36 months may reveal different associations; this may include what happens as mothers move out of early postnatal care and into routine child health or ART services. Actual retention-in-care may differ according to the stage of care – soon after diagnosis or ART initiation, or in ART care [46,47]. Finally, the most significant limitation was the lack of other process and clinical data available at the time of analysis. Pill count and HIV viral load data would have allowed further investigation of the relationship between retention-in-care and women's health status. More work is needed to better understand the link between the regular clinic visits during a specific time period and the final clinic visit at the end of this period. Despite these limitations, one of the major strengths of this study is the detailed scrutiny of this key indicator of programme and intervention success through a combination of several methodological approaches, which as triangulation in qualitative analysis, leads to highly valid data.

The variability in retention-in-care estimates depending on criteria used highlights the importance of understanding how a definition is constructed and what is its primary purpose. Is it based on missed visits or attendance patterns? Is it intended to facilitate comparisons of interventions as part of implementation research or to track the performance of programmes; or does the data collection process aim to identify early defaulters or at-risk populations in order to trigger community-based tracing? Some definitions require more complex data inputs
which will have resource requirements; others will provide
greater insights on the behaviour of individuals within a popu-
lation. When referring to retention-in-care rates, researchers
and programme managers should appreciate the potential for
variance according to criteria used. Failure to do so, may bias
interpretation and comparison of interventions and strategies
aimed at mitigating losses of individuals from treatment pro-
grammes. Different types of retention-in-care definition and
analyses may be needed for research and for programmes.

5 | CONCLUSIONS

In summary, our findings highlight the variability of estimated
PMTCT retention-in-care rates depending on which definition is
applied to the analyses. In the absence of an agreed gold stan-
dard definition for PMTCT retention-in-care and methodologies
for estimating rates, it is important that studies provide detailed
descriptions of their study population, context, data collection
and data management processes in order to accurately inter-
pret findings and compare the effectiveness of relevant inter-
ventions. In contexts where retention-in-care rates may be used
to infer ART adherence and to calculate both antenatal and
postnatal PMTCT risks, for example as part of modelling work
around the estimation of new paediatric HIV infections, the type
of definition used by national programmes will be an important
point to take into consideration. A simple indicator such as
attendance or non-attendance at a single time point, for exam-
ple 12 months postpartum may be sufficient for programme
planning rather than more detailed and complex indicators that
may only be achievable with electronic record systems.

AUTHORS' AFFILIATIONS

¹SPED, Inserm, Bordeaux Population Health Research Center, UMR 1219, University of Bordeaux, Bordeaux, France; ²Department of Maternal, Newborn, Child and Adolescent Health, World Health Organization, Geneva, Switzerland; ³Data and Analytics Section, Unicef, New York, USA; ⁴Family AIDS Caring Trust, World Health Organization, Mutare, Zimbabwe; ⁵Clinton Health Access Initiative, Harare, Zimbabwe; ⁶Department of Public Health, College of Medicine, University of Malawi, Blantyre, Malawi; ⁷Centre for Integrated Health Programs, Abuja, Nigeria; ⁸Lighthouse Trust, Lilongwe, Malawi; ⁹Department of Medicine, University of North Carolina School of Medicine, Chapel Hill, NC, USA; ¹⁰Department of Public Health, School of Public Health and Family Medicine, College of Medicine, University of Malawi, Lilongwe, Malawi; ¹¹International Research Center of Excellence, Institute of Human Virology Nigeria, Abuja, Nigeria; ¹²Division of Epidemiology and Prevention, Institute of Human Virology, University of Maryland School of Medicine, Baltimore, MD, USA; ¹³Strategic Information Department, UNAIDS, Geneva, Switzerland

COMPETING INTERESTS

All authors declare no competing interests.

AUTHORS' CONTRIBUTIONS

HF, NR and JOG designed and implemented the analysis. HF and JOG searched the literature and co-wrote the first draft of the manuscript. HF did the statistical analysis. NR contributed substantive changes to the draft. SE, RB, GF, AM, VM, BO, SP, NS-A, NB reviewed the draft, contributed to the interpretation and presentation of the findings. All authors have read and approved the final version of the manuscript for submission.

ABBREVIATIONS

AC1, Gwet's first-order agreement coefficient; ART, Antiretroviral treatment; E4E, Evidence-for-Elimination study; EPAZ, Elimination of Paediatric AIDS in

Zimbabwe study; HRSA HAB, Human Resources and Services Administration HIV/AIDS Bureau; IQR, Inter-Quartile Range; LJM, Lafiyan Jikin Mata study; MCA, Multiple Correspondence Analysis; MoMent, Mother Mentor study; NPV, Negative predictive values; PMTCT, Prevention of Mother-to-Child Transmission; PPV, Positive predictive values; PRIME, Promoting Retention among Infants and Mothers Effectively study; PURE, Prevention of mother-to-child transmission Uptake and REtention study.

ACKNOWLEDGEMENTS

We thank the INSPIRE co-principal investigators: Andrews Gunda, Talent Maphosa and Pemberai Zambezi. We thank the INSPIRE data managers: Ber-nice Ameh, Alison Erlwanger, Christopher Isah Jessica Joseph, Frank Oronsaye, Tonderai Sengai, Clement Trapence and Hannock Tweya. We thank Olivier Marcy for his help during the revision process.

FUNDING

The work was funded by the World Health Organization through an award from Global Affairs Canada for the INtegrating and Scaling up PMTCT through Implementation REsearch (INSPIRE) initiative.

DISCLAIMER

The findings and conclusions in this paper are those of the authors and do not necessarily represent the views and policies of the authors' institutions or organizations.

REFERENCES

1. Aliyu MH, Blevins M, Audet CM, Kalish M, Gebi UI, Onwujekwe O, et al. Integrated prevention of mother-to-child HIV transmission services, antiretroviral therapy initiation, and maternal and infant retention in care in rural north-central Nigeria: a cluster-randomised controlled trial. Lancet HIV. 2016;3:e202–211.

2. Cohen MS, Chen YQ, McCauley M, Gamble T, Hosseinipour MC, Kumara-samy N, et al. Antiretroviral therapy for the prevention of HIV-1 transmission. N Engl J Med. 2016;375:830–9.

3. World Health Organization. Programmatic update: use of antiretroviral drugs for treating pregnant women and preventing HIV infection in infants: executive summary [cited 2019 Oct 20]. Available from: https://apps.who.int/iris/handle/10665/70892

4. Barker PM, Mphatswe W, Rollins N. Antiretroviral drugs in the cupboard are not enough: the impact of health systems' performance on mother-to-child transmission of HIV. J Acquir Immune Defic Syndr. 2011;56:e45–8.

5. Dionne-Odom J, Massaro C, Jogerst KM, Li Z, Deschamps M-M, Destine CJ, et al. Retention in care among HIV-infected pregnant women in Haiti with PMTCT Option B. AIDS Res Treat. 2016;2016:6284290.

6. Chan AK, Kanike E, Bedell R, Mayuni I, Manyera R, Mlotha W, et al. Same day HIV diagnosis and antiretroviral therapy initiation affects retention in Option B+ prevention of mother-to-child transmission services at antenatal care in Zomba District, Malawi. J Int AIDS Soc. 2016;19:20672.

7. Gerdts SE, Wagenaar BH, Micek MA, Farquhar C, Kariaganis M, Amos J, et al. Linkage to HIV care and antiretroviral therapy by HIV testing service type in Central Mozambique: a retrospective cohort study. J Acquir Immune Defic Syndr. 2014;66:e37–44.

8. Girma M, Wendaferash R, Shibru H, Berhane Y, Hoelscher M, Kroidl A. Uptake and performance of prevention of mother-to-child transmission and early infant diagnosis in pregnant HIV-infected women and their exposed infants at seven health centres in Addis Ababa, Ethiopia. Trop Med Int Health. 2017;22:765–75.

9. Woelk GB, Ndatimana D, Behan S, Mukaminega M, Nyirabahizi E, Hoffman HJ, et al. Retention of mothers and infants in the prevention of mother-to-child transmission of HIV programme is associated with individual and facility-level factors in Rwanda. J Int AIDS Soc. 2016;19:20837.

10. Rollins NC, Becquet R, Orne-Gliemann J, Phiri S, Hayashi C, Baller A, et al. Defining and analyzing retention-in-care among pregnant and breastfeeding HIV-infected women: unpacking the data to interpret and improve PMTCT outcomes. JAIDS J Acquir Immune Defic Syndr. 2014;67:S150–6.

11. Crawford TN, Sanderson WT, Thornton A. A Comparison study of methods for measuring retention in HIV medical care. AIDS Behav. 2013;17:3145–51.

12. Mugavero MJ, Westfall AO, Zinski A, Davila J, Drainoni M-L, Gardner LI, et al. Measuring retention in HIV care: the elusive gold standard. J Acquir Immune Defic Syndr. 2012;61:574–80.
13. Mugavero MJ, Davila JA, Nevin CR, Giordano TP. From access to engagement: measuring retention in outpatient HIV clinical care. AIDS Patient Care STDS. 2010;24:607–13.

14. Rebeiro PF, Horberg MA, Gange SJ, Gebo KA, Yehia BR, Brooks JT, et al. Strong agreement of nationally recommended retention measures from the Institute of Medicine and Department of Health and Human Services. PLoS One. 2014;9:e111772.

15. Johnson LF, Estill J, Keiser O, Cornell M, Moolla H, Schomaker M, et al. Do increasing rates of loss to follow-up in antiretroviral treatment programs imply deteriorating patient retention? Am J Epidemiol. 2014;180:1208–12.

16. Shepherd BE, Blevins M, Vaz LME, Moon TD, Kipp AM, José E, et al. Impact of definitions of loss to follow-up on estimates of retention, disease progression, and mortality: application to an HIV program in Mozambique. Am J Epidemiol. 2013;178:819–28.

17. Myer L, Phillips TK. Beyond "Option B+": Understanding Antiretroviral Therapy (ART) adherence, retention in care and engagement in ART services among pregnant and postpartum women initiating therapy in Sub-Saharan Africa. J Acquir Immune Defic Syndr. 2017;75 Suppl 2:S115–22.

18. Keller SC, Yehia BR, Eberhart MG, Brady KA. Accuracy of definitions for linkage to care in persons living with HIV. J Acquir Immune Defic Syndr. 2013;63:622–30.

19. Ahoua L, Arikawa S, Tiendrebeogo T, Lahuerta M, Aly D, Becquet R, et al. Measuring retention in care for HIV-positive pregnant women in Prevention of Mother-to-Child Transmission of HIV (PMTCT) option B+ programs: the Mozambique experience. BMC Public Health. 2020;20:322.

20. Phillips TK, Orrell C, Brittain K, Zerbe A, Abrams EJ, Myer L. Measuring retention in HIV care: the impact of data sources and definitions using routine data. AIDS. 2020;34:749–59.

21. Sabin LL, Halim N, Hamer DH, Simmons EM, Jonnalagadda S, Larson Williams A, et al. Retention in HIV care among HIV-seropositive pregnant and postpartum women in Uganda: results of a randomized controlled trial. AIDS Behav. 2020. https://doi.org/10.1007/s10461-020-02875-5

22. Blais P, Hirnschall G, Mason E, Shaffer N, Lipa Z, Baller A, et al. Introducing INSPIRE: an implementation research collaboration between the Department of Foreign Affairs, Trade and Development Canada and the World Health Organization. J Acquir Immune Defic Syndr. 2014;67 Suppl 2:S105–107.

23. Foster G, Kangwende A, Magezi V, Maphosa T, Mashapa R, Mukora-Mutseyekwa F, et al. Cluster randomized trial on the effect of mother support groups on retention-in-care and PMTCT outcomes in Zimbabwe: study design, challenges, and national relevance. J Acquir Immune Defic Syndr. 2014;67 Suppl 2:S145–149.

24. Mangwiro A-Z, Makomva K, Bhattacharya A, Bhattacharya G, Gotora T, Owen M, et al. Does provision of point-of-care CD4 technology and early knowledge of CD4 levels affect early initiation and retention on antiretroviral treatment in HIV-positive pregnant women in the context of Option B+ for PMTCT? J Acquir Immune Defic Syndr. 2014;67 Suppl 2:S139–144.

25. Mwapasa V, Pro G, Chinkhumba J, Mukaka M, Kobayashi E, Stuart A, et al. Mother-infant pair clinic and SMS messaging as innovative strategies for improving access to and retention in eMTCT care and option b+ in Malawi: a cluster randomized control trial (The PRIME Study). JAIDS J Acquir Immune Defic Syndr. 2014;67:S120–4.

26. Oyeledun B, Oronsaye F, Oyelade T, Becquet R, Odoh D, Anyaike C, et al. Increasing retention in care of HIV-positive women in PMTCT services through continuous quality improvement-breakthrough (CQI-BTS) series in primary and secondary health care facilities in Nigeria: a cluster randomized controlled trial. The Lafyan Jikin Mata Study. J Acquir Immune Defic Syndr. 2014;67 Suppl 2:S125–131.

27. Rosenberg NE, van Lettow M, Tweya H, Kapito-Tembo A, Bourdon CM, Cataldo F, et al. Improving PMTCT uptake and retention services through novel approaches in peer-based family-supported care in the clinic and community: a 3-arm cluster randomized trial (PURE Malawi). J Acquir Immune Defic Syndr. 2014;67 Suppl 2:S114–119.

28. Sam-Agudu NA, Cornelius LJ, Okundaye JN, Adeyemi OA, Isah HO, Wiwa OM, et al. The impact of mentor mother programs on PMTCT service uptake and retention-in-care at primary health care facilities in Nigeria: a prospective cohort study (MoMent Nigeria). J Acquir Immune Defic Syndr. 2014;67 Suppl 2: S132–138.

29. Joseph J, Gotora T, Erlwanger AS, Mushavi A, Zizhou S, Masuka N, et al. Impact of Point-of-care CD4 testing on retention in care among HIV-positive

pregnant and breastfeeding women in the context of option B+ in Zimbabwe: a cluster randomized controlled trial. J Acquir Immune Defic Syndr. 2017;75 Suppl 2:S190–7.

30. Mwapasa V, Joseph J, Tchereni T, Jousset A, Gunda A. Impact of mother-infant pair clinics and short-text messaging service (SMS) reminders on retention of HIV-infected women and HIV-exposed infants in eMTCT care in Malawi: a cluster randomized trial. J Acquir Immune Defic Syndr. 2017;75 Suppl 2:S123–31.

31. Oyeledun B, Phillips A, Oronsaye F, Alo OD, Shaffer N, Osibo B, et al. The effect of a continuous quality improvement intervention on retention-in-care at 6 months postpartum in a PMTCT program in Northern Nigeria: results of a cluster randomized controlled study. J Acquir Immune Defic Syndr. 2017;75 Suppl 2:S156–64.

32. Phiri S, Tweya H, van Lettow M, Rosenberg NE, Trapence C, Kapito-Tembo A, et al. Impact of facility- and community-based peer support models on maternal uptake and retention in Malawi's option B+ HIV prevention of mother-to-child transmission program: a 3-arm cluster randomized controlled trial (PURE Malawi). J Acquir Immune Defic Syndr. 2017;75 Suppl 2:S140–8.

33. Sam-Agudu NA, Ramadhani HO, Isah C, Anaba U, Erekaha S, Fan-Osuala C, et al. The impact of structured mentor mother programs on 6-month postpartum retention and viral suppression among HIV-positive women in rural Nigeria: a prospective paired cohort study. J Acquir Immune Defic Syndr. 2017;75 Suppl 2:S173–81.

34. Foster G, Orne-Gliemann J, Font H, Kangwende A, Magezi V, Sengai T, et al. Impact of facility-based mother support groups on retention in care and PMTCT outcomes in rural Zimbabwe: the EPAZ cluster-randomized controlled trial. J Acquir Immune Defic Syndr. 2017;75 Suppl 2:S207–15.

35. Sabharwal CJ, Braunstein SL, Robbins RS, Shepard CW. Optimizing the use of surveillance data for monitoring the care status of persons recently diagnosed with HIV in NYC. J Acquir Immune Defic Syndr. 2014;65:571–8.

36. Mitiku I, Arefayne M, Mesfin Y, Gizaw M. Factors associated with loss to follow-up among women in Option B+ PMTCT programme in northeast Ethiopia: a retrospective cohort study. J Int AIDS Soc. 2016;19:20662.

37. HRSA. Performance measure portfolio I HIV/AIDS Bureau 2008 [cited 2017 Sep 27]. Available from: https://hab.hrsa.gov/sites/default/files/hab/clinical-quality-management/archivedmcm.pdf

38. Gwet KL. Computing inter-rater reliability and its variance in the presence of high agreement. Br J Math Stat Psychol. 2008;61:29–48.

39. Wongpakaran N, Wongpakaran T, Wedding D, Gwet KL. A comparison of Cohen's Kappa and Gwet's AC1 when calculating inter-rater reliability coefficients: a study conducted with personality disorder samples. BMC Med Res Methodol. 2013;13:61.

40. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33:159–74.

41. Sourial N, Wolfson C, Zhu B, Quail J, Fletcher J, Karunananthan S, et al. Correspondence analysis is a useful tool to uncover the relationships among categorical variables. J Clin Epidemiol. 2010;63:638–46.

42. Geldsetzer P, Yapa HMN, Vaikath M, Ogbuoji O, Fox MP, Essajee SM, et al. A systematic review of interventions to improve postpartum retention of women in PMTCT and ART care. J Int AIDS Soc. 2016;19:20679.

43. Tudor Car L, Brusamento S, Elmoniry H, van Velthoven MHMMT, Pape UJ, Welch V, et al. The uptake of integrated perinatal prevention of mother-to-child HIV transmission programs in low- and middle-income countries: a systematic review. PLoS One. 2013;8:e56550.

44. Xia Q, Wiewel EW, Torian LV. Comparison of single-visit and multiple-visit measures of retention in care for HIV monitoring and evaluation. J Acquir Immune Defic Syndr. 2016;71:e59.

45. Li MS, Musonda P, Gartland M, Mulenga PL, Mwango A, Stringer JSA, et al. Predictors of patient attrition according to different definitions for loss to follow-up: a comparative analysis from Lusaka, Zambia. J Acquir Immune Defic Syndr. 2013;63:e116–9.

46. Fox MP, Rosen S. Retention of adult patients on antiretroviral therapy in low- and middle-income countries: systematic review and meta-analysis 2008–2013. J Acquir Immune Defic Syndr. 2015;69:98–108.

47. Fox MP, Rosen S. Systematic review of retention of pediatric patients on HIV treatment in low and middle-income countries 2008–2013. AIDS Lond Engl. 2015;29:493–502.