Cost-effectiveness of a smart pillbox intervention for adherence to oral HIV pre-exposure prophylaxis

Domain: neuroscience
Published: 2026-01-28
Editorial outcome: minor_revision

Abstract
Despite proven oral pre-exposure prophylaxis (PrEP) efficacy, adherence barriers among men who have sex with men (MSM) persist. Economic viability of digital adherence tools in low- and middle-income countries (LMICs) requires validation. We assessed the 6-month within-trial cost-effectiveness of a digital intervention for real-time enhancement of oral PrEP adherence (DIRECT-PrEP) in China, from health system, multipayer, and societal perspectives. Costs were derived from hospital records, and effectiveness was evaluated using quality-adjusted life-years (QALYs) and PrEP-adherence days (PADs). Among 442 participants (247 intervention, 195 standard care), incremental costs per QALY gained for the smart pillbox intervention were $19,575.93 (health system), $19,656.66 (multipayer), and $19,844.42 (societal)—all below the cost-effectiveness threshold of three times per capita gross domestic product ($31,200). Costs per PAD gained were $30.46–$30.86, achieving >95% probability of cost-effectiveness at willingness-to-pay thresholds of $47.50–$48.50 per PAD. Notably, the intervention demonstrated greater cost-effectiveness per QALY among participants stratified as more socially vulnerable ($16,259.77–$16,490.41) compared with those classified as less socially vulnerable ($23,514.83–$23,834.38). These results establish the smart-pillbox digital intervention as a cost-effective strategy for oral PrEP adherence within primary health care settings in LMICs, particularly when prioritizing more socially vulnerable populations, though long-term scalability requires further validation.

Introduction
Oral pre-exposure prophylaxis (PrEP) is a cornerstone of HIV prevention for high-risk populations, including men who have sex with men (MSM) in low- and middle-income countries (LMICs)1–3. Despite its efficacy, suboptimal adherence remains a critical barrier, diminishing protection at low adherence levels4 and increasing risks of HIV transmission and drug resistance5. MSM, as a key sexual minority group, experience heightened vulnerability due to Social Determinants of Health (SDH) spanning (1) conditions of daily life (e.g., behavioral patterns, social support) and (2) structural determinants (e.g., economic status, education)6. Consequently, investigating cost-effective interventions for more socially vulnerable MSM is crucial to advance health equity. The urgency of this issue has been further amplified by the COVID-19 pandemic and persistent health inequities have underscored the need for scalable digital solutions in LMICs, where structural constraints—limited healthcare infrastructure, funding shortages, and power supply instability—hinder conventional interventions7.

Achieving equity in HIV prevention requires an understanding of not only which interventions are effective, but also the mechanisms underlying their differential performance across SDH strata. SDH factors—such as socioeconomic status, education, and housing stability—directly influence PrEP persistence8. Furthermore, behavioral risk patterns among MSM9 (e.g., multiple sexual partners, engaging in condomless sex, and substance use) are shaped by these broader structural factors and collectively influence HIV risk and PrEP cost-effectiveness. Across diverse settings, suboptimal PrEP adherence among MSM remains a persistent challenge4, driven by stigma, socioeconomic marginalization, and discrimination10. These barriers are not unique to China but reflect structural determinants common to MSM globally. Moreover, high-risk MSM subgroups may demonstrate greater willingness to adopt PrEP11, yet structural barriers—such as limited psychosocial support or economic resources—often undermine adherence12, leading to variations in cost-effectiveness across different SDH strata.

UNAIDS estimates that adopting innovative technologies, efficiencies, and approaches could reduce annual HIV response costs by approximately US$7 billion13. Although digital adherence technologies [e.g., short message service (SMS) reminders, mobile apps] show promise, evidence of their effectiveness is heterogeneous. For instance, an interactive text-messaging intervention improved PrEP adherence among young MSM in Chicago14, yet SMS-based interventions failed among MSM in China and other vulnerable groups15–19. Moreover, whereas interactive messaging reduced sexual risk behaviors at incremental costs of $19.70–$131.60 per risk-reduction unit20,21, these approaches face scalability barriers: high start-up costs ($1021 per participant)22, declining engagement without reinforcement, and low suitability for populations with limited digital literacy7. Crucially, none adequately address the operational challenges of LMICs as effectively as electronic medication monitors (e.g., smart pillbox). Our randomized controlled trial (RCT) showed telemonitoring using electronic medication monitors increased oral PrEP adherence to 83.1% (21.0% net improvement over controls) over 6 months among Chinese MSM7. This efficacy stems from its user-centered design: real-time dose confirmation, minimal user input, and offline functionality align with people-centered strategies recommended by the World Health Organization (WHO)23.

Technologically, electronic medication monitors circumvent LMIC infrastructural barriers via low operational costs, accurate passive data collection, and seamless system integration. Unlike SMS or app-based platforms requiring continuous connectivity, these devices use durable, battery-powered hardware with predominantly one-time procurement costs24. Their automated monitoring eliminates reliance on self-reporting, thereby mitigating reporting inaccuracies common in LMICs25. Most importantly, their capacity to function independently of grid electricity or internet connectivity is critical in regions such as Sub-Saharan Africa, where mobile network coverage gaps are about 50%26. Nevertheless, cost-effectiveness analyses specific to electronic medication monitors remain scarce. Existing economic evaluations of HIV-related mHealth focus disproportionately on SMS ($132–$314/participant) or video-based interventions ($1021/participant)21,22, overlooking electronic medication monitors’ potential for higher marginal returns at scale. For example, while two-way texting achieved cost savings for antiretroviral therapy retention in Malawi27, similar efficiencies were plausible for electronic medication monitors given their reusable infrastructure. China’s digital infrastructure has seen comparatively greater investment in recent years. However, contextual differences across LMICs may limit the generalizability of our findings to regions such as Sub-Saharan Africa. Evidence shows that LMIC health facilities often lack reliable internet, electricity, and digital infrastructure essential for implementing adherence technologies. For example, a survey of over 5000 facilities in eight LMICs reported very low digital infrastructure scores28. Similar assessments in South and Southeast Asia highlight persistent gaps in electricity, affordable connectivity, and policy support29. These disparities highlight the need for a systematic, human-centered, and contextually grounded approach to implementing digital HIV interventions in LMICs.

This study addresses two critical research gaps in digital health interventions for LMICs. First, it provides the economic evaluation of electronic medication monitors for oral PrEP adherence using LMIC-specific cost metrics, including cost per adherence-unit gained and incremental cost-effectiveness ratios (ICERs). Second, it adopts an equity-focused lens to assess the affordability of electronic medication monitors among more socially vulnerable and less socially vulnerable MSM by examining device durability and maintenance within LMIC resource constraints, thereby aligning with priorities for sustainable, device-led innovations.

Ultimately, by demonstrating the superiority of electronic medication monitors in cost per adherence-unit metrics and scalability within LMIC health systems, this study aims to inform global oral PrEP rollout strategies—advancing health equity through frugal, context-adapted medical devices.

Results
Participant characteristics
Figure 1 presented the overall study design. Among the 516 participants who completed the baseline assessment, 247 were randomized to the smart pillbox intervention group and 195 to the control group (Supplementary Fig. S1). Baseline health utilities were comparable between groups, and no significant between-group differences in direct healthcare utilization costs were observed at baseline (Supplementary Table S1).

Cost components
Table 1 summarized the 6-month cost and health outcome profiles. From the health system, multipayer, and societal perspectives, the mean (standard deviation [SD]) cumulative total costs in the intervention group were $1826.77 (434.88), $1846.00 (434.71), and $1950.76 (434.82), respectively. The corresponding values for the control group were $1373.68 (585.34), $1390.19 (585.21), and $1488.52 (585.23). Healthcare utilization costs represented the predominant cost component for both groups (intervention: 78.37%; control: 78.69%; Supplementary Fig. S2), while intervention-related costs accounted for 8.08% of total intervention expenditures. Unadjusted intergroup cost differences were detailed in Table 2.

Direct costs: testing and intervention
Costs related to testing averaged $140.27 per participant. The mean intervention cost was $157.66 per participant (Table 1), of which $28.99 (18.39%) was allocated to the smart pillbox. Personnel expenditures constituted the majority of intervention costs at $121.32 per participant (76.95%). Costs associated with questionnaire administration and follow-up support were $2.88 per participant, representing 1.83% of total intervention expenditures (Supplementary Fig. S2).

Direct costs: healthcare utilization
The mean total healthcare utilization cost was significantly higher in the intervention group ($1528.85 [434.89]) compared with the control group ($1171.37 [585.35]; P<0.001, Table 2). Healthcare utilization represented the predominant non-intervention cost category across both groups (Supplementary Fig. S2).

Indirect costs
Venue and cohort maintenance costs were comparable between groups. However, the intervention group incurred slightly higher expenditures for food and transportation ($19.23 [1.37]) than the control group ($16.52 [1.38]). Time costs and lost productivity were also higher in the intervention group ($20.27 [1.63] vs. $13.83 [1.45]; Table 1).

Health effect
Table 2 showed significantly greater improvements in both QALYs and PrEP-adherence days (PADs) in the intervention group compared with the control group during the 6-month study period. The unadjusted mean difference was 32.72 (95% CI: 11.94–40.91) for PADs and 0.02 (95% CI: 0.01–0.02) for QALYs (intervention: 0.96 [SD 0.04] vs. control: 0.94 [SD 0.05]).

Cost-effectiveness
Table 3 presented the ICERs calculated from 6-month cost and effectiveness differentials using the SUR model. From the health system, multipayer, and societal perspectives, the smart pillbox intervention yielded ICERs per QALY gained of $19,575.93, $19,656.66, and $19,844.42, respectively. Cost-effectiveness acceptability curves demonstrated a 0.01–98.80% probability of intervention cost-effectiveness across willingness-to-pay (WTP) thresholds ranging from $10,400 to $31,200. For PADs, the intervention produced ICERs of $30.46, $30.58, and $30.86 per PAD gained from these respective perspectives. A WTP threshold between $47.50 and $48.50 per PAD was needed to achieve >95% probability of cost-effectiveness. Notably, at a WTP threshold equivalent to three times China’s per-capita GDP, the intervention demonstrated cost-effectiveness across all examined perspectives and outcomes compared with standard care. Results from the seemingly unrelated regression (SUR) model that was adjusted for baseline characteristics were presented in Table 3 and Supplementary Fig. S7–8.

The smart pillbox intervention was more cost-effective in the more socially vulnerable group, with ICERs ranging from $16,259.77 to $16,490.41 and a cost-effectiveness probability of 2.71–98.99%. In contrast, the less socially vulnerable group exhibited higher ICERs ($23,514.83–$23,834.38) and lower probability (0.08–82.15%). For PADs, the intervention showed ICERs of $26.73–$27.09 in the more socially vulnerable group but higher ICERs ($30.08–$30.47) in the less socially vulnerable group, both evaluated from the three perspectives (Table 4; Supplementary Fig. S3–6).

Sensitivity analyses
Figure 2 displayed the distribution of bootstrap-replicated net cost and QALY estimates on the cost-effectiveness plane. From the health system perspective, the probability of cost-effectiveness ranged from 0.02% (1/5000 replications) at a WTP threshold of 1× GDP per capita ($10,400) to 98.80% (4940/5000) at 3× GDP per capita ($31,200). Similarly, under the multipayer perspective, the probability ranged from 0.02% (1/5000) to 98.75% (4938/5000), while the societal perspective showed a range of 0.01% (1/5000) to 98.63% (4932/5000) across the same WTP thresholds. These findings were further supported by the CEAC illustrated in Fig. 3.

Robustness checks
Our results exhibited consistent robustness across all sensitivity analyses. First, reanalysis adjusted for baseline characteristics maintained significant improvements (PADs: 29.91 [95% CI: 15.68–44.14]); QALYs: 0.03 [95% CI: 0.02–0.03]), demonstrating consistent robustness (Supplementary Table S2). Second, ordinary least squares (OLS) analysis (baseline-adjusted) yielded comparable ICERs (QALYs: $18,897.47–$19,263.99; PADs: $30.71–$31.30; Supplementary Table S3, Figs. S9, 10). Third, generalized linear models (GLM) estimation of incremental costs revealed a 0.00–97.08% cost-effectiveness probability for QALYs at WTP thresholds of 1–3× GDP, with QALY ICERs of $21,121.95–$21,235.81 and PAD ICERs of $34.14–$34.32 across perspectives (Supplementary Table S4; Supplementary Figs. S11, 12). Fourth, treating overhead expenses as variable costs yielded an unadjusted societal ICER of $20,741.40, below three times GDP per capita (Supplementary Table S5). Fifth, using an alternative pooled HIV/AIDS utility value produced an unadjusted societal ICER of $19,793.40, also below this threshold (Supplementary Table S6). Sixth, when using an alternative WTP threshold (Supplementary Tables S7 and S8), the cost-effectiveness probability from the societal perspective among the more socially vulnerable group ranged from 9.30% to 98.57%. Finally, reanalysis using Truvada-specific costs (baseline-adjusted) showed that the probability of cost-effectiveness ranged from 5.64% to 99.94% (Supplementary Table S9). Critically, all analyses converged on a near-100% cost-effectiveness probability at the 3× GDP WTP threshold.

Methods
The prospective interventional China Real-world Oral Intake of PrEP (CROPrEP) study evaluated the HIV prevention efficacy of TDF/FTC among MSM in four cities: Beijing, Shenzhen, Chongqing, and Shenyang48. Participants were stratified by baseline preference into daily or event-driven PrEP subgroups, or a non-PrEP group. Daily PrEP participants received one TDF/FTC tablet per day. The event-driven PrEP regimen involved administering two tablets 2–24 h before sexual intercourse, followed by single tablets 24 and 48 h after the initial dose. If additional sexual activity occurred within 48 h of the initial dose, participants transitioned to daily dosing until 48 h after the final sexual act. Subsequent sexual encounters triggered reinitiation of the event-driven regimen. Cross-over between daily and event-driven groups was permitted during follow-up.

The pragmatic RCT of the digital intervention for real-time enhancement of oral PrEP adherence (DIRECT-PrEP) study7 ran concurrently in the same cities. It was approved by the First Hospital of China Medical University Ethics Committee (No. 2019-253-3). Participants were stratified by PrEP regimen and randomized (1:1) to intervention or control groups using opaque sealed envelopes containing computer-generated random numbers. The trial protocol was previously published49. We adhered to Consolidated Health Economic Evaluation Reporting (CHEERS) 202250 (Supplementary Note 4).

Participants
CHWs and volunteers shared details of DIRECT-PrEP via WeChat to CROPrEP participants. After enrollment, MSM were randomized to intervention or control groups. Eligibility criteria included being biologically male, aged 18–65 years, engaging in anal sex with men, having at least one high-risk behavior in the past six months (condomless anal intercourse with males, ≥2 male partners, or any sexually transmitted infection diagnosis), confirmed HIV-1 negative status, participation in CROPrEP, ability to understand study protocols, and provision of written informed consent (Supplementary Protocol). Recruitment occurred between September 20, 2019, and November 27, 2020.

Five risk factors associated with higher social vulnerability were defined: ≤ high school education, monthly income <$580, relationship instability, having casual partners, and engaging in condomless anal intercourse with casual partners. Participants meeting two or more factors comprised the more socially vulnerable group, while others were classified as the less socially vulnerable group (Supplementary Note 1).

Intervention
This intervention leveraged an integrated technology-support model linking digital monitoring with clinical care to improve oral PrEP adherence. Key components included a multidisciplinary team, tailored participant engagement, structured follow-up, real-time communication, and supplementary resources to ensure sustained adherence and data integrity. Supplementary Fig. S13 illustrates the electronic medication monitoring device and its operation.

(1) Multidisciplinary team collaboration: The intervention involved a coordinated team of researchers, experienced physicians, and technical support personnel. Researchers facilitated proactive monitoring and participant contact, while physicians provided personalized adherence counseling and managed adverse events during follow-up visits. Technical experts ensured smart pillbox functionality. All roles were defined through standardized training before study initiation.

(2) Customized participant engagement protocols: Each participant received individualized support, including tailored instructional materials and demonstration videos for daily and event-driven PrEP use. Training content was refined iteratively based on feedback from pilot testing with researchers and users, emphasizing participant-centered engagement to promote self-management and adherence motivation.

(3) Structured monitoring and follow-up schedule: Participants underwent scheduled follow-ups at 3 and 6 months (±7 days), which included biological testing (e.g., HIV screening, HIV RNA pooling PCR, syphilis serology, HSV-2 antibody testing, routine blood and urine tests, liver and renal function tests, blood glucose and lipid profiles, and bone mineral density assessment) and online questionnaires assessing sociodemographic and behavioral factors. Continuous remote monitoring using the smart pillbox logged adherence data via cloud-based systems accessible through a WeChat Mini Program.

(4) Real-time alert and intervention mechanism: A multi-tiered reminder system was triggered by preset dosing schedules, incorporating multimodal alerts (sound, vibration, light, and WeChat push notifications) for missed doses. Missed-dose reminders escalated to repeated mobile notifications every 30 min (up to twice), followed by researcher alerts. Time-sensitive interventions were employed: doses missed before 22:00 were addressed immediately, while those after were handled by 10:00 the next day. Repeated pillbox openings without intake also triggered system alerts for researcher follow-up.

(5) Supplementary technical and operational resources: The intervention was supported by enhanced training materials, real-time technical assistance, and the provision of NB-IoT-enabled smart pillboxes with 600 mAh rechargeable batteries and automatic cloud logging. Post-study, devices were returned per a predefined protocol to ensure sustainability and data security. Supplementary resources such as customized instructional videos and a dedicated management platform facilitated seamless implementation.

Health effects
To quantify intervention effectiveness, two primary metrics were used: (1) health utility (QALYs) and (2) health effects (PADs) during follow-up.

QALYs provided a longitudinal measure of treatment effectiveness by integrating health-related quality-of-life (HRQoL) states with time-in-state duration. Health utilities were derived from individual PrEP adherence levels. PrEP use as prescribed reduces sexual HIV acquisition risk by approximately 99% among MSM51; thus, perfect adherence was modeled as complete HIV protection. This framework allowed estimation of HIV incidence through adherence metrics. QALYs were calculated as a function of health utilities and estimated HIV incidence. To calculate PADs, PrEP medication adherence was assigned a value of 1 for perfect adherence and 0 for non-adherence. PrEP adherence was treated as a continuous variable, with imperfect adherence representing partial protection. PADs were calculated as (PrEP adherence) × (follow-up days) (Supplementary Note 2 for methodology).

Cost Components
Costs were categorized as testing, healthcare utilization, intervention, and indirect costs.

Direct costs: testing. Biological testing occurred at baseline, 3-month, and 6-month follow-ups. Total testing expenditure equaled the per-participant fixed cost multiplied by the eligible cohort size.

Direct costs: healthcare utilization. At the time of procurement, Truvada (TDF/FTC 300/200 mg; Gilead Sciences) cost $286.96 per 30-tablet bottle. Pill counts at baseline and follow-up visits determined total medication expenditure. Per-participant Truvada costs were calculated by distributing total expenditures across all randomized participants.

Direct costs: intervention. Smart pillbox costs included unit procurement, system development and maintenance (including software design), and charging expenses. Supplementary costs covered training materials, technical support, and NB-IoT-enabled smart pillboxes with rechargeable batteries and cloud auto-logging functionality. These expenditures also supported sustainability and data security through customized instructional videos and a dedicated management platform.

Labor costs included CHWs training and backend management (e.g., manual reminders for missed doses). Salaries were provided for two researchers and nurses conducting pill counts and adherence interventions. Additional costs covered travel, accommodation, and meals for training-related trips. One researcher responsible for pill counting and one nurse performing blood collection received part-time salaries for 15 months. Per-participant labor costs were calculated by dividing total expenditures across randomized participants. Two researchers implementing adherence interventions were compensated for 15 months, with costs allocated only to intervention group participants.

Questionnaire collection costs were based on the annual subscription fee ($579.57) for the Jinshuju data platform (https://jinshuju.net/home). Follow-up support expenses included communication costs covered by a 15-month phone plan at $28.84 per month. Per-participant questionnaire costs were calculated by dividing total expenditures across all randomized participants.

Indirect costs. venue, cohort maintenance, clinical visits, time, and lost productivity. Venue costs represented total expenditures across all four study sites. Cohort maintenance costs (a fixed fee) covered all participants at baseline and follow-up visits. Participant expenditures for food and transportation during clinical visits were extracted from survey data. Lost productivity was estimated by multiplying the number of clinical visits by the average daily wage, assuming a productivity loss of 2.5 h per visit for the intervention group and 2 h for the control group.

All costs were reported in 2020 U.S. dollars, using an exchange rate of 1 USD=6.9 CNY. Given the 6-month time horizon, no discounting was applied to costs or outcomes. Cost categories and analytical perspectives are presented in Supplementary Table S10.

Statistical analysis
We assessed efficacy using intent-to-treat analysis and trial-wide data from DIRECT-PrEP’s economic evaluation. The within-trial cost-effectiveness analysis compared the smart pillbox intervention with the control group across three perspectives: health system, multipayer, and societal. The healthcare perspective incorporated only direct medical expenditures. The multipayer perspective included health sector expenditures, intervention costs, and indirect costs associated with food and transportation. The societal perspective encompassed all additional indirect costs, including participants’ opportunity costs related to healthcare engagement and productivity losses due to illness. As recommended for public health economic evaluations, we prioritized the societal perspective because it provides an inclusive assessment of all costs affecting both beneficiaries and payers, thereby offering a comprehensive evaluation of societal impact (Supplementary Note 3, Supplementary Table S6)40.

We quantified cost-effectiveness using the ICER, calculated by dividing the difference in mean costs between the digital intervention and control groups by the corresponding difference in health outcomes (measured in both QALYs and PADs). The ICER represented the marginal cost required to achieve one additional unit of health benefit when implementing the smart pillbox intervention versus standard care.

Because a specific WTP threshold has not been established in China37, we adopted the cost-effectiveness threshold framework based on WHO-CHOICE and previous publications39–41, corresponding to 1–3 times China’s 2020 per capita gross domestic product (GDP). This yielded an evaluative range of $10,400–$31,200 per QALY gained, based on China’s economic parameters52. WTP thresholds were also estimated using a VSL conversion42, yielding a point estimate of 1.45 times GDP per capita, with a lower bound of about 1.16 times and an upper bound of about 2.90 times GDP per capita. Regarding PADs, in the lack of predefined WTP benchmarks, we derived the theoretical threshold required to attain a 95% probability of cost-effectiveness through probabilistic sensitivity analysis.

We applied the two-sided SUR model to specify cost and effectiveness regression equations, producing incremental cost and utility estimates. To address sampling uncertainty, we used a nonparametric bootstrap resampling approach with 5000 iterations. This method generated an empirical joint distribution of incremental costs and effects by repeatedly sampling with replacement from the original dataset53,54.

These bootstrap replications were used to construct 95% confidence intervals (CIs). The resulting uncertainty estimates were visualized on cost-effectiveness planes, providing a comprehensive depiction of the variability in our cost-effectiveness estimates. There were no missing data in this trial; all essential medical records were fully available and analytically adequate.

We subsequently developed cost-effectiveness acceptability curves (CEACs) to estimate the probability that the intervention was cost-effective across a range of WTP thresholds for QALYs, analyzed from multiple economic perspectives. These CEACs provided policymakers with robust probabilistic evidence of cost-effectiveness, while accounting for key uncertainties inherent in economic evaluations39.

To evaluate the robustness of our primary findings, we conducted comprehensive sensitivity analyses. First, we accounted for residual confounding by controlling for baseline outcome measures (age, income, casual partner status, and PrEP regimen). Second, we reassessed cost-effectiveness using two-sided GLM as well as OLS regression models. Third, we reanalyzed cost-effectiveness after including healthcare expenditures attributable to Truvada. Fourth, we conducted a sensitivity analysis that disaggregated system costs into fixed and variable components, treating overhead expenses as variable. Finally, we performed a sensitivity analysis based on an alternative pooled utility value for HIV/AIDS.

All analyses were conducted using R version 4.4.1. Two-sided independent samples t-tests and two-sided Mann–Whitney U tests were applied for normally and non-normally distributed continuous variables, respectively. Estimates were reported with 95% confidence intervals, using two-tailed tests with α=0.05 (P<0.05). The trial was registered on the Chinese Clinical Trial Registry (ChiCTR1900025604).

Peer review rounds
[{"round": 1, "reviewer_comments": {"reviewer_1": "This study evaluated the cost-effectiveness of a smart pillbox–based electronic medication monitoring intervention (DIRECT-PrEP) to improve oral PrEP adherence among men who have sex with men in China, comparing it with standard care. Using trial data over six months, the analysis assessed costs (from health system, multipayer, and societal perspectives) and effectiveness (QALYs and PrEP-adherence days). Results showed that the intervention produced modest health gains (0.02 QALYs; 32.7 more adherence days) at incremental costs of about $400, yielding ICERs well below the willingness-to-pay threshold of three times China’s GDP per capita, particularly among socially vulnerable groups, supporting its potential as a cost-effective and equity-enhancing strategy in LMIC settings.\nI have some comments below.\n1. In Table 1, the definitions of the health system, multipayer, and societal perspectives are not entirely clear. Based on the cost breakdown, the health system perspective ($1826.77) is limited to direct medical expenditures—drug costs, testing, and intervention-related expenses (smart pillbox, labor, follow-up support). The multipayer perspective adds only patient-incurred food and transportation costs ($19.23), which implicitly assumes that patients receive PrEP services free of charge beyond these access-related expenses. The societal perspective goes further, incorporating not only time and productivity losses ($20.27) and a small learning cost for using the pillbox ($2.88), but also broader indirect system-level costs such as venue expenditures ($76.24) and cohort maintenance costs ($8.26). Together, these additions raise the cost to approximately $1950.76 but not exactly same (calculated as $1826.77 + $19.23 + $20.27 + $2.88 + $76.24 + $8.26 = $1953.65). It would be helpful to note what specific costs were included for each perspective in the Supplementary Note 3 beyond just the definitions.\n2. The authors used Seemingly Unrelated Regression (SUR) as their primary analytic method, allowing adjustment for individual baseline covariates and accounting for correlation between costs and outcomes within the same patient. This regression-based ICER estimation produced incremental cost and effect estimates, and bootstrapping was applied to generate uncertainty distributions. However, authors report the system cost as a point estimate (e.g., software development costs divided by total participants); it is difficult to distinguish which costs are fixed versus variable across different population sizes. This matters because digital health interventions often have high fixed costs that diminish with scale, making cost-effectiveness highly sensitive to the size of the target population. Without clarity on cost structure, the generalizability of findings to other settings is limited. Accordingly, the analysis only accounts the uncertainty based on patients costs but not system side cost. Authors may address this as a limitation or consider additional sensitivity analyses that vary assumptions on system-side costs, in addition to patient-level variation, would better capture the potential economies of scale inherent in digital health delivery.\n3. As the authors note, the observed adherence benefit was modest—0.02 QALYs and an increase in PrEP-adherence days (PADs: 221.47 vs 188.75). This QALY gain was derived through an adherence-based utility mapping, where adherence ≥95% was assigned a utility of 0.95–1.00, 80–94% mapped to 0.90–0.95, and <80% mapped to 0.81–0.90. Using linear interpolation, adherence was then linked to HIV incidence reduction, with the framework assuming complete protection at full adherence and applying a baseline incidence of 5.6 per 100 person-years (rounded to 0.06). While this approach offers a structured way to translate adherence into health outcomes, the justification for the chosen utility values is unclear—particularly the assumption that all adherence <80% falls within the relatively high 0.81–0.90 utility range. This may risk overestimating health benefits in low-adherence groups and warrants greater transparency or sensitivity analyses to test alternative mappings.\n4. The sensitivity analyses primarily examined statistical methods (SUR, OLS, GLM) and cost perspectives (health system, multipayer, societal), but the differences across perspectives were minimal—multipayer added only patient food/transportation costs, while societal added modest productivity losses—resulting in only marginal changes in ICERs. More critical sources of uncertainty likely lie in system-level factors: how health systems adapt to implementing digital interventions in different settings, how cost-effectiveness changes with population scale given fixed costs, and whether the observed adherence difference (221.47 vs 188.75 PADs) can be sustained over the long term. Highlighting these structural uncertainties as limitations would strengthen the discussion, as they may have greater implications for generalizability.\n5. The paper claims to incorporate “behavioral economic principles” by contrasting electronic medication monitors’ adherence-mediated prevention benefits with SMS interventions’ focus on risk-behavior modification. However, the trial design and economic model evaluate only adherence outcomes (PADs, QALYs) and do not measure or compare downstream behavioral risk changes such as sexual risk compensation or reductions in risky behaviors. As a result, the use of “behavioral economics” feels more rhetorical than methodological: the study does not empirically test behavioral constructs but rather frames adherence versus behavior-modification strategies as distinct economic logics.\n(Remarks on code availability)\nThe code runs three regression-based CEA methods (SUR, OLS, GLM) with bootstrapping, estimates incremental cost and effect, calculates ICERs, and visualizes uncertainty with cost-effectiveness planes and CEACs.\nNot directly runnable as-is. The code has placeholders (xxx) for covariates and a hard-coded local path (C:/Users/xxx/xxx.csv). There’s no example dataset, no schema for required columns (e.g., class, cost_m0, effectiveness_m0, units), and no package version pinning.\nNo README / run instructions are shown in the snippet (installation steps, data dictionary, session info, or a one-command pipeline).", "reviewer_2": "Thank you for the opportunity to review this manuscript. This cost-effective analysis of an intervention to improve PrEP adherence is novel and will be of interest to readers. It is mostly well written but requires some editing for readability which can hopefully be improved after copyediting. I have only two main thoughts for the authors to consider:\nSix months doesn’t seem long enough, shouldn’t you extrapolate to consider health outcomes and downstream savings?\nPersonnel costs are a major driver of intervention costs. This raises questions about scalability. Discussion should better distinguish between trial-specific research costs vs. routine implementation costs.", "reviewer_3": "The research investigates a critical economic analysis related to oral PrEP adherence in the MSM population. Focusing on equity is of utmost importance, considering that those in lower socioeconomic strata are disproportionately impacted by HIV. The methodology is generally sound and appropriately applied, however, certain critical areas warrant attention, and the paper does not meet the rigorous standards for Nature Communications.\nMajor comments:\n1. The trial is China-specific, and while often considered a LMIC in public health research, the health system differences may limit generalizability to regions like Sub-Saharan Africa. And especially since the digitial infrastructure differ significantly from many LMICs. This limitation should be acknowledged consistently throughout the manuscript, not just in the discussion.\n2. The use of Seemingly Unrelated Regression (SUR) and bootstrap simulations is unconventional for health economic modeling. While SUR can account for correlated outcomes, it is not standard for cost-effectiveness analysis, which typically relies on decision-analytic models. Can the authors justify why SUR was chosen over more established modelling methods? Also explain its limitations compared to more dynamic or long-term modelling.\n3. The conclusion that \"intervention complexity\" is positively associated with equitable health coverage is not directly supported by the literature review presented in the “Comparison with existing Literature” paragraph, especially since the authors are trying to make this claim by comparing across different settings, time horizons, years. However, this interpretation oversimplifies the literature and possibly conflates methodological differences with causal relationships. The authors should either reframe this as a hypothesis or provide empirical justification.\n4. While the study references willingness to pay (WTP) thresholds to interpret cost-effectiveness, it does not discuss their origin, appropriateness, or limitation, especially in the context of equity. WTP thresholds are often arbitrary and often fail to reflect societal values or budget constraints in LMICs. The authors could strengthen their analysis by discussing alternative approaches (e.g., opportunity cost-based thresholds, equity-weighted thresholds).\nMinor comments:\n1. The study focuses on MSM in China but does not engage with broader literature on PrEP adherence among MSM globally. There is little discussion of behavioral, structural, or psychosocial factors that influence adherence, nor of intersectional vulnerabilities such as stigma, income, or housing. This limits the depth of the equity framing and the relevance of findings to other MSM populations.\n2. Personnel costs are the main driver of the intervention budget (>76%). This raises questions about scalability and whether similar staffing models are feasible in other LMICs.\n3. PADs are a useful intermediate metric, but their translation into long-term HIV prevention outcomes could be better contextualized. The study uses perfect adherence as a binary threshold, yet imperfect adherence still conveys partial protection and this is not accounted for in the modelling. A more nuanced approach is needed. Perhaps modeling adherence as a gradient would better reflect real-world effectiveness.\n4. The reported probability range for cost-effectiveness, eg. 2.71% to 98.99%, is extremely broad, limiting its practicality. This suggests high sensitivity to input assumptions or model structure. The authors should clarify which scenarios drive the lower bound and consider presenting threshold analyses to guide decision-makers more effectively.\n5. The stratification by SDH is descriptive and limited to cost-effectiveness outcomes. The study does not explore underlying mechanisms or contextual factors that explain why cost-effectiveness differs by SDH strata—thus falling short of a deeper equity analysis.\n6. The claim that smart pillboxes \"critically extend\" existing evidence is not fully substantiated. The comparison of incremental costs ($19,844 vs. $24,705 for SMS) lacks context—are these figures derived from comparable methodologies, time horizons, and costing perspectives?\n7. The critique of other digital tools (SMS, apps, hybrid models) is broad and somewhat selective. For example, citing $1021/user for a U.S. MSM app without adjusting for context or scale may unfairly skew the comparison.\n8. Granted that English might not be the first language for the authors, however, spelling errors could be avoided by using a spell checker, e.g. hightened=heightened; siginificant=significant; coverd=covered; trail-wide=trial-wide; to name a few."}, "author_response": "1 -------------------------------------------------------------------------------------------------------\n4 electronic medication monitoring intervention (DIRECT-PrEP) to improve oral PrEP\n5 adherence among men who have sex with men in China, comparing it with standard\n6 care. Using trial data over six months, the analysis assessed costs (from health system,\n7 multipayer, and societal perspectives) and effectiveness (QALYs and PrEP-adherence\n8 days). Results showed that the intervention produced modest health gains (0.02 QALYs;\n9 32.7 more adherence days) at incremental costs of about $400, yielding ICERs well\n10 below the willingness-to-pay threshold of three times China’s GDP per capita,\n11 particularly among socially vulnerable groups, supporting its potential as a cost-\n12 effective and equity-enhancing strategy in LMIC settings.\n16 clarification on the cost-effectiveness analysis of the DIRECT-PrEP. Below, we\n17 respond to your overall feedback in a point-by-point manner.\n18\n21 are not entirely clear. Based on the cost breakdown, the health system perspective\n22 ($1826.77) is limited to direct medical expenditures—drug costs, testing, and\n23 intervention-related expenses (smart pillbox, labor, follow-up support). The multipayer\n24 perspective adds only patient-incurred food and transportation costs ($19.23), which\n25 implicitly assumes that patients receive PrEP services free of charge beyond these\n26 access-related expenses. The societal perspective goes further, incorporating not only\n27 time and productivity losses ($20.27) and a small learning cost for using the pillbox\n28 ($2.88), but also broader indirect system-level costs such as venue expenditures\n29 ($76.24) and cohort maintenance costs ($8.26). Together, these additions raise the cost\n30 to approximately $1950.76 but not exactly same (calculated as $1826.77 + $19.23 +\n31 $20.27 + $2.88 + $76.24 + $8.26 = $1953.65). It would be helpful to note what specific\n32 costs were included for each perspective in the Supplementary Note 3 beyond just the\n33 definitions.\n36 because the subtotal for “time and productivity losses” ($20.27) already included the\n37 $2.88 learning cost. The $2.88 was thus double-counted in your calculation, leading to\n38 the slightly higher figure. The correct aggregated cost is $1950.76, as mentioned in the\n39 manuscript.\n41 breakdown. Accordingly, we have revised Table 1 and Supplementary Note 3 to\n42 explicitly list the cost components included under each perspective (health system,\n43 multipayer, and societal), as you suggested.\n45 “The health system perspective ($1826.77) encompassed only direct medical\n46 expenditures, including drug costs, testing, and intervention-related expenses\n47 (smart pillbox, labor, and follow-up support). The multipayer perspective\n48 additionally incorporated patient-incurred food and transportation costs\n49 ($19.23), thereby implicitly assuming that PrEP services themselves were\n50 provided free of charge aside from these access-related expenses. The societal\n51 perspective further expanded the scope by including time and productivity\n52 losses ($20.27), the learning cost associated with using the pillbox ($2.88), as\n53 well as broader indirect system-level costs such as venue expenditures ($76.24)\n54 and cohort maintenance costs ($8.26).” (Page 36-37, Table 1, line 940-948)\n55 “(1) Health system perspective\n56 ……The health system perspective ($1826.77) encompassed only direct medical\n57 expenditures, including drug costs, testing, and intervention-related expenses\n58 (smart pillbox, labor, and follow-up support).\n59 (2) Multipayer perspective\n60 ……The multipayer perspective additionally incorporated patient-incurred\n2\n61 food and transportation costs ($19.23), thereby implicitly assuming that PrEP\n62 services themselves were provided free of charge aside from these access-\n63 related expenses.\n64 (3) Societal perspective\n65 ……The societal perspective further expanded the scope by including time and\n66 productivity losses ($20.27), the learning cost associated with using the pillbox\n67 ($2.88), as well as broader indirect system-level costs such as venue\n68 expenditures ($76.24) and cohort maintenance costs ($8.26).” (Supplementary\n70\n73 method, allowing adjustment for individual baseline covariates and accounting for\n74 correlation between costs and outcomes within the same patient. This regression-based\n76 was applied to generate uncertainty distributions. However, authors report the system\n77 cost as a point estimate (e.g., software development costs divided by total participants);\n78 it is difficult to distinguish which costs are fixed versus variable across different\n79 population sizes. This matters because digital health interventions often have high fixed\n80 costs that diminish with scale, making cost-effectiveness highly sensitive to the size of\n81 the target population. Without clarity on cost structure, the generalizability of findings\n82 to other settings is limited. Accordingly, the analysis only accounts the uncertainty\n83 based on patients costs but not system side cost. Authors may address this as a\n84 limitation or consider additional sensitivity analyses that vary assumptions on system-\n85 side costs, in addition to patient-level variation, would better capture the potential\n86 economies of scale inherent in digital health delivery.\n89 between fixed and variable components for scalability, which we fully agree is a critical\n90 consideration for digital health interventions. In response, we have added the following\n3\n91 two key actions:\n92 1. We conducted supplementary sensitivity analyses that disaggregated system costs\n93 into fixed and variable components, treating overhead expenses as variable, in\n94 accordance with established economic evaluation frameworks for digital health\n95 interventions. Fixed costs include one-time expenses such as software development,\n96 initial licensing fees, and training costs, while variable costs encompass ongoing\n97 maintenance, technical support, and per-user operational expenses. This distinction is\n98 critical for modeling economies of scale, as highlighted in recent systematic reviews\n99 noting that digital health interventions often exhibit significant cost variability across\n100 implementation scales. Specifically, we allocated 20% of direct testing costs (20%\n101 of 140.27 = 28.05) and direct intervention costs (20% of 157.66 = 31.53 for the\n102 intervention group; 20% of 62.04 = 12.41 for the control group) as variable\n103 overhead to model scale-dependent cost behavior, consistent with methodological\n104 recommendations for cost decomposition in health economic evaluations.\n105 2. We have explicitly acknowledged this methodological limitation in the discussion\n106 section to enhance the transparency and generalizability of our findings.\n108 “Fourth, treating overhead expenses as variable costs yielded an unadjusted\n109 societal ICER of $20,741.40, below three times GDP per capita (Supplementary\n111 “Fourth, system-side costs were reported as point estimates (e.g., per-\n112 participant software development costs) without clarifying fixed versus\n113 variable cost proportions, limiting scalability assessment in larger populations.”\n114 (Page 15, Discussion, line 477-479)\n115 “Fourth, we conducted a sensitivity analysis that disaggregated system costs\n116 into fixed and variable components, treating overhead expenses as variable.”\n117 (Page 23, Methods, line 729-731)\n4\nCost Perspective Cost difference- USD (95% Effectiveness ICER- Probability of Effectiveness ICER- USD WTP per PAD to achieve\nCI) difference- QALYs USD cost- PADs (95% CI) probability of cost-\n(95% CI) effectiveness effectiveness > 95%\nUnadjusted analysis\nHealth System 436.40 (344.83 to 527.97) 0.02 (0.01 to 0.03) 20,472.91 0.00 to 98.14 13.58 (7.47 to 19.69) 31.87 50.00\nMultipayer 438.12 (346.50 to 529.75) 20,553.64 0.00 to 98.07 31.99 50.00\nSocietal 442.13 (350.35 to 533.90) 20,741.40 0.00 to 97.89 32.27 50.50\nAdjusted analysis a\nHealth System 417.93 (331.35 to 504.50) 0.02 (0.01 to 0.03) 19,821.53 0.01 to 98.55 13.51 (7.44 to 19.59) 30.80 48.00\nMultipayer 420.24 (333.62 to 506.85) 19,930.86 0.01 to 98.47 30.97 48.50\nSocietal 425.59 (338.85 to 512.33) 20,184.69 0.00 to 98.27 31.34 49.00\n120 a Adjusted analyses were performed employing a two-sided SUR model, incorporating baseline age, income, the presence of casual partners, and PrEP regimen\n121 adjustment. We allocated 20% of direct testing costs (20% of 140.27 = 28.05) and direct intervention costs (20% of 157.66 = 31.53 for the intervention group;\n122 20% of 62.04 = 12.41 for the control group) as variable overhead. DIRECT-PrEP digital intervention for real-time enhancement of oral PrEP adherence, USD\n124 willingness-to-pay, PrEP pre-exposure prophylaxis, SUR Seemingly Unrelated Regression.\n127 increase in PrEP-adherence days (PADs: 221.47 vs 188.75). This QALY gain was\n128 derived through an adherence-based utility mapping, where adherence ≥95% was\n129 assigned a utility of 0.95–1.00, 80–94% mapped to 0.90–0.95, and <80% mapped to\n130 0.81–0.90. Using linear interpolation, adherence was then linked to HIV incidence\n131 reduction, with the framework assuming complete protection at full adherence and\n132 applying a baseline incidence of 5.6 per 100 person-years (rounded to 0.06). While this\n133 approach offers a structured way to translate adherence into health outcomes, the\n134 justification for the chosen utility values is unclear—particularly the assumption that\n135 all adherence <80% falls within the relatively high 0.81–0.90 utility range. This may\n136 risk overestimating health benefits in low-adherence groups and warrants greater\n137 transparency or sensitivity analyses to test alternative mappings.\n140 our cost-effectiveness analysis of PrEP adherence. We appreciate the opportunity to\n141 clarify our methodology and address the concerns about potential overestimation of\n142 health benefits in low-adherence groups.\n143 1. In our primary analysis, we assigned utilities for adherence levels based on a\n144 structured mapping, with <80% adherence corresponding to a utility range of 0.81–0.90.\n146 diagnosed with HIV promptly initiate antiretroviral therapy (ART). Evidence from\n148 receiving ART (Wei L, et al. Trends in Life Expectancy of HIV-Infected Patients\n150 13;7(24):838-842.). Although a gap with the general population persists, it is narrowing\n151 over time. Informed by this trend and the established utility value of 0.80 for HIV-\n152 positive status at 2 months post-diagnosis (Honiden S, et al. The effect of diagnosis with\n154 we adopted 0.80 as the baseline utility in our primary scenario. Individuals with PrEP\n6\n155 adherence below 80% are not necessarily newly diagnosed with HIV. Therefore, they\n156 are unlikely to experience the significant health-related quality of life (HRQoL) decline\n157 associated with untreated HIV. Based on this, we reasoned their utilities should be\n158 higher than 0.80, which is why the range starts from 0.81.\n160 0.91 to MSM for conditions like genital warts (Kim JJ, et al. Targeted human\n161 papillomavirus vaccination of men who have sex with men in the USA: a cost-\n162 effectiveness modelling analysis. Lancet Infect Dis. 2010 Dec;10(12):845-52.). In their\n163 study, this STI-related utility was applied to a population comparable to ours,\n164 suggesting that non-HIV-related sexual health conditions do not substantially reduce\n168 automatically indicate the occurrence of a quality-of-life-limiting sexually transmitted\n169 infections. Therefore, the HRQoL of these MSM is unlikely to be substantially lower\n170 than that of the general population.\n171 2. We fully acknowledge, however, that this assumption may not hold universally. This\n172 is particularly true in low-resource settings lacking robust “Treat-all” policies, where\n173 delayed ART initiation could result in lower utilities for HIV-positive individuals. To\n174 directly address this and test the robustness of our conclusions, we conducted a\n175 sensitivity analysis based on literature reporting a pooled utility value of\n176 approximately 0.70 for people living with HIV who experience delayed or no\n177 treatment (Tengs TO, et al. A meta-analysis of utility estimates for HIV/AIDS. Med\n179 disease stages, including: identified symptomatic HIV without ART, symptomatic\n181 identified asymptomatic HIV in the first year, unidentified asymptomatic HIV,\n182 and acute HIV infection. Accordingly, in this alternative scenario, we revised the\n183 utility range for <80% adherence to 0.71–0.80. We maintained the same\n184 adherence-based interpolation framework. Following implementation of the new\n7\n186 month period, compared to 0.84 QALYs in the control group. The results\n187 demonstrated that the incremental cost-effectiveness ratio (ICER) remained robust\n188 across both mappings. Consequently, the overall conclusion - that the intervention is\n189 cost-effective under most conditions – remained unchanged.\n191 findings. We have implemented the following revisions to address this concern:\n192 “Fifth, using an alternative pooled HIV/AIDS utility value produced an\n193 unadjusted societal ICER of $19,793.40, also below this threshold\n194 (Supplementary Table S6).” (Page 8, Results, line 247-249)\n195 “Finally, we performed a sensitivity analysis based on an alternative pooled\n196 utility value for HIV/AIDS.” (Page 23, Methods, line 731-732)\n197\n199 concern:\n8\nCost Perspective Cost difference- USD (95% Effectiveness ICER- Probability of Effectiveness ICER- USD WTP per PAD to achieve\nCI) difference- QALYs USD cost- PADs (95% probability of cost-\n(95% CI) effectiveness CI) effectiveness > 95%\nUnadjusted analysis\nHealth System 417.27 (325.70 to 508.85) 0.02 (0.01 to 0.03) 19,525.63 0.02 to 98.84 13.58 (7.47 to 19.69) 30.46 47.50\nMultipayer 419.00 (327.37 to 510.62) 19,606.14 0.02 to 98.79 30.58 48.00\nSocietal 423.00 (331.22 to 514.78) 19,793.40 0.01 to 98.67 30.86 48.50\nAdjusted analysis b\nHealth System 398.81 (312.23 to 485.38) 0.02 (0.01 to 0.03) 18,866.91 0.04 to 99.11 13.51 (7.44 to 19.59) 29.39 46.00\nMultipayer 401.11 (314.50 to 487.73) 18,975.96 0.03 to 99.06 29.55 46.00\nSocietal 406.47 (319.73 to 493.20) 19,229.15 0.02 to 98.93 29.93 47.00\n201 a We conducted a sensitivity analysis based on a pooled utility value of approximately 0.70 for HIV/AIDS. This value encompasses various disease stages,\n202 including: identified symptomatic HIV without ART, symptomatic HIV treated with ART, identified asymptomatic HIV in subsequent years, identified\n203 asymptomatic HIV in the first year, unidentified asymptomatic HIV, and acute HIV infection. b Adjusted analyses were performed employing a two-sided SUR\n204 model, incorporating baseline age, income, the presence of casual partners, and PrEP regimen adjustment. DIRECT-PrEP digital intervention for real-time\n205 enhancement of oral PrEP adherence, USD United States Dollar, CI confidence interval, QALY quality-adjusted life year, ICER incremental cost-effectiveness\n206 ratio, PAD PrEP-adherence day, WTP willingness-to-pay, PrEP pre-exposure prophylaxis, ART antiretroviral therapy, SUR Seemingly Unrelated Regression.\n9\n209 cost perspectives (health system, multipayer, societal), but the differences across\n210 perspectives were minimal—multipayer added only patient food/transportation costs,\n211 while societal added modest productivity losses—resulting in only marginal changes in\n213 health systems adapt to implementing digital interventions in different settings, how\n214 cost-effectiveness changes with population scale given fixed costs, and whether the\n215 observed adherence difference (221.47 vs 188.75 PADs) can be sustained over the long\n216 term. Highlighting these structural uncertainties as limitations would strengthen the\n217 discussion, as they may have greater implications for generalizability.\n220 structural uncertainties beyond statistical and cost perspectives. We agree entirely that\n221 factors such as health system adaptation, economies of scale, and the long-term\n222 sustainability of adherence gains are pivotal for assessing the real-world\n223 generalizability of our findings.\n225 acknowledge these aspects as key limitations. Rather than merely listing them, we have\n226 framed these uncertainties as critical considerations for future implementation and\n227 research, thereby strengthening the interpretation and scope of our conclusions.\n229 “Third, our sensitivity analyses focused on statistical methods and cost\n230 perspectives, yielding minimal ICER changes due to incremental patient costs\n231 and productivity losses. However, these analyses did not fully address\n232 structural uncertainties, such as health system adaptation to digital\n233 interventions, scale effects on cost-effectiveness given high fixed costs, and long-\n234 term sustainability of adherence differences, which may affect generalizability.”\n235 (Page 15, Discussion, line 472-477)\n236\n239 electronic medication monitors’ adherence-mediated prevention benefits with SMS\n240 interventions’ focus on risk-behavior modification. However, the trial design and\n10\n241 economic model evaluate only adherence outcomes (PADs, QALYs) and do not\n242 measure or compare downstream behavioral risk changes such as sexual risk\n243 compensation or reductions in risky behaviors. As a result, the use of “behavioral\n244 economics” feels more rhetorical than methodological: the study does not empirically\n245 test behavioral constructs but rather frames adherence versus behavior-modification\n246 strategies as distinct economic logics.\n249 \"behavioral economic principles\" in the manuscript. Your comment has prompted us to\n250 carefully reconsider our conceptual framing, and we agree that this terminology was\n251 misapplied in our original submission. Consequently, we have removed the sentence\n252 from the manuscript.\n254 intervention paradigms. Our smart pillbox intervention functions as a monitoring and\n255 feedback mechanism to improve adherence. However, it does not incorporate the\n256 deliberate modifications to choice architecture that define genuine behavioral economic\n257 approaches.\n259 manipulating cognitive biases and decision-making heuristics to alter choices. They are\n260 not defined simply by using different intervention modalities. Our trial design measured\n261 adherence outcomes (PADs and QALYs) but did not assess downstream behavioral\n262 changes. These changes — such as sexual risk compensation or reductions in risky\n263 behaviors — are key elements that would be necessary to substantiate a claim of\n264 engaging with behavioral economic principles.\n266 “Second, the analysis incorporates behavioral economic principles by distinguishing\n267 electronic medication monitors' adherence-mediated HIV prevention benefits from\n269 benefit profiles.” (Page 5, Introduction, line 147-150)\n270\n272 (Remarks on code availability):\n274 bootstrapping, estimates incremental cost and effect, calculates ICERs, and visualizes\n11\n275 uncertainty with cost-effectiveness planes and CEACs.\n276\n278 coded local path (C:/Users/xxx/xxx.csv). There’s no example dataset, no schema for\n279 required columns (e.g., class, cost_m0, effectiveness_m0, units), and no package\n280 version pinning.\n281\n283 dictionary, session info, or a one-command pipeline).\n287 available at https://github.com/ZhenhaoWu853/Cost-effectiveness-of-real-time-\n288 digital-intervention. Key updates include:\n289 1) Data import\n291 dataset <- read.csv (\"example dataset.csv\", header = TRUE)\n292 2) Covariates specification\n294 age + group + income + casual\n296 perspective for over 6 months; QALYs, Quality-adjusted life years; societal_m0,\n298 = <30 years; 2 = ≥30 years; group, 1 = daily PrEP, 2 = event-driven PrEP; income,\n299 1 = ≤3999, 2 = 4000–7999, 3 = ≥8000 (CNY); casual:, 1 = has casual male\n300 partner, 2 = none; units, currency = USD.\n301 3) Outcome variable naming\n303 consistency with health economic standards.\n306 4) Model syntax updated\n307 societal ~ class + societal_m0 + age + group + income + casual\n12\n309 5) Version control and reproducibility\n312 execution guidance.\n314 “# README: Cost-Effectiveness Analysis (CEA) Pipeline\n315\n316 # This script runs three regression-based CEA methods (SUR, OLS, GLM)\n317 # using bootstrap resampling. It estimates incremental costs, incremental\n318 # QALYs, ICERs, and visualizes uncertainty via cost-effectiveness planes\n319 # and cost-effectiveness acceptability curves (CEACs). We have provided an\n320 example dataset (example dataset.csv, n=100).\n321 # -----------------------------\n322 # 1. Required Packages & Setup\n323 # -----------------------------\n325 required_packages <- c(\n326 \"mice\", \"systemfit\", \"car\", \"boot\", \"ggplot2\",\n327 \"ggpointdensity\", \"readxl\", \"tidyverse\"\n328 )\n329\n330 installed <- rownames(installed.packages())\n331 for (pkg in required_packages) {\n332 if (!(pkg %in% installed)) install.packages(pkg, dependencies = TRUE)\n333 }\n334\n335 library(mice) # v3.18.0\n336 library(systemfit) # v1.1-30\n337 library(car) # v3.1-2\n338 library(boot) # v1.3-30\n339 library(ggplot2) # v3.5.0\n340 library(ggpointdensity) # v0.2.0\n341 library(readxl) # v1.4.3\n13\n342 library(tidyverse) # v2.0.0\n343\n344 # -----------------------------\n345 # 2. Data Import\n346 # -----------------------------\n347 dataset <- read.csv(\"example dataset.csv\", header = TRUE)\n348\n349 # -----------------------------\n350 # 3. Data Dictionary\n351 # -----------------------------\n352 # class: 0 = control, 1 = intervention\n353 # societal: cost from the societal perspective for over 6 months\n354 # QALYs: Quality-adjusted life years\n355 # societal_m0: Baseline cost from the societal perspective\n356 # QALYs_m0: Baseline QALYs\n357 # age: 1 = <30 years; 2 = ≥30 years\n358 # group: 1 = daily PrEP; 2 = event-driven PrEP\n359 # income: 1 = ≤3999; 2 = 4000–7999; 3 = ≥8000 (CNY)\n360 # casual: 1 = has casual male partner; 2 = none\n361 # units: currency = USD\n362 ” (README.md)\n14\n364 effective analysis of an intervention to improve PrEP adherence is novel and will be of\n365 interest to readers. It is mostly well written but requires some editing for readability\n366 which can hopefully be improved after copyediting.\n369 our work. We are pleased to know that you found the cost-effectiveness analysis novel\n370 and of interest to the readers. The revised manuscript has undergone professional\n372\n375 outcomes and downstream savings?\n378 evaluation. Your comment reflects a fundamental tension in economic evaluation of\n379 digital health interventions. This is the challenge of balancing the use of short-term\n380 empirical evidence against the needs for long-term projections. We appreciate the\n381 opportunity to clarify our methodological approach and its justification.\n382 1. Our study is a within-trial economic evaluation conducted alongside a\n383 randomized controlled trial (RCT) with a 6-month follow-up period. This design\n384 was intentionally chosen to align with the empirical data collected during the trial, and\n385 we acknowledge that it does not involve extrapolation to long-term health outcomes or\n386 downstream savings. Our evaluation relies solely on observed data from the\n387 randomized controlled trial. This contrasts with decision-analytic models, which\n388 require numerous assumptions about long-term adherence trajectories. This provides a\n389 transparent, conservative estimate of the intervention's cost-effectiveness during the\n390 period when efficacy was empirically demonstrated.\n391 2. The 6-month horizon is not arbitrary but is supported by external evidence. As\n392 documented in Zhang’s study (Discontinuation, suboptimal adherence, and reinitiation\n393 of oral HIV pre-exposure prophylaxis: a global systematic review and meta-analysis.\n395 was 38% within the first 6 months, 42% during 7-12 months, and 43% beyond 13\n396 months. This pattern indicates a marked decline in adherence shortly after the initial 6-\n15\n397 month period, after which it stabilizes at a level comparable to the 6-month benchmark.\n399 a consistent pattern (see the figures below). A significant effect was observed within 6\n400 months [RR = 1.18, 95% CI (1.06, 1.30), p < 0.01]. However, this effect was not\n401 sustained in trials with longer follow-up periods (9, 12, or 24 months) [RR = 1.21, 95%\n403 interventions may not be sustained beyond 6 months. Moreover, the long-term\n404 efficacy remains uncertain, making extrapolation from existing trial data\n405 challenging and potentially unreliable.\n407 evaluating the initial efficacy of adherence interventions.\n408\n409 6-month PrEP adherence intervention\n410\n411 over 6-month PrEP adherence intervention\n412\n413 1) Liu AY, et al. Clin Infect Dis. 2019 May 30;68(12):2010-2017.\n414 2) Buchbinder SP, et al. AIDS Behav. 2023 Feb;27(2):719-732.\n415 3) Chu ZX, et al. NPJ Digit Med. 2025 Jun 11;8(1):349.\n16\n416 4) Horvath KJ, et al. AIDS Behav. 2024 Aug;28(8):2804-2820.\n417 5) Songtaweesin WN, et al. J Int AIDS Soc. 2020 Sep;23 Suppl 5:e25564.\n418 6) Lin B, et al. J Med Internet Res. 2022 Aug 11;24(8):e37936.\n419 7) Whiteley L, et al. AIDS Behav. 2021 Jun;25(6):1884-1889.\n420 8) Wray TB, et al. AIDS Behav. 2024 Apr;28(4):1356-1369.\n421 9) Kawichai S, et al. JMIR Mhealth Uhealth. 2022 Apr 21;10(4):e25561.\n422 10) Hightow-Weidman LB, et al. AIDS Behav. 2025 Feb;29(2):652-663.\n423 11) Velloza J, et al. Lancet HIV. 2025 Feb;12(2):e105-e117.\n424 12) Marins LMS, et al. J Med Internet Res. 2025 Jul 18;27:e72360.\n425 13) Moore DJ, et al. Clin Infect Dis. 2018 May 2;66(10):1566-1572.\n426 14) Colson PW, et al. AIDS Behav. 2020 Dec;24(12):3436-3455.\n427 15) van den Elshout MAM, et al. AIDS. 2021 Sep 1;35(11):1823-1834.\n428 16) Liu Z, et al. J Infect. 2022 Feb;84(2):248-288.\n429 3. Unlike dynamic or long-term modeling approaches, such as decision-analytic models\n430 (e.g., Markov models or discrete event simulations) that are standard for cost-\n431 effectiveness analysis, our within-trial evaluation offers several distinct advantages in\n432 this context.\n434 analytic models include:\n435 ⚫ Direct empirical basis: It relies solely on observed RCT data. This minimizes\n436 assumptions, reduces the uncertainty of long-term projections, and provides a\n437 transparent, conservative estimate of cost-effectiveness during the trial period.\n438 ⚫ Avoidance of speculative assumptions: By not extrapolating beyond the trial\n439 duration, we avoid potential biases from unverified assumptions about the\n440 sustainability of intervention effects, which is particularly important for behavioral\n441 interventions like PrEP adherence where effects may wane over time.\n442 ⚫ Focus on short-term efficacy: It is optimally suited for assessing immediate\n443 resource use and outcomes, which is critical for initial decision-making regarding\n444 intervention implementation, especially when long-term efficacy is uncertain.\n445 4. We agree that this relatively short time horizon may underestimate long-term costs\n446 and outcomes. In the revised manuscript, we have expanded the Limitations section to\n447 acknowledge that our analysis reflects robust short-term cost-effectiveness but lacks\n448 longitudinal data on maintenance-phase outcomes. We now also emphasize that many\n449 prior evaluations rely on static models, which risk overstating cost-effectiveness\n17\n450 by ignoring longer-term effects, while dynamic models suggest that digital health\n451 tools may generate sustained value (El Arab RA, et al. Systematic review of cost\n452 effectiveness and budget impact of artificial intelligence in healthcare. NPJ Digit\n454 research is needed using extended follow-up, dynamic modeling frameworks, and\n455 inclusion of broader cost elements and subgroup analyses to assess the long-term\n456 clinical and economic sustainability of such interventions across the full\n457 technology lifecycle.\n458 5. In conclusion, while long-term models are valuable for a comprehensive view, our\n459 within-trial approach provides a methodologically sound and evidence-based\n460 assessment of the intervention’s cost-effectiveness within its empirically demonstrated\n461 period of efficacy.\n463 “First, the 6-month follow-up likely underestimates long-term costs and\n464 outcomes. Although our evaluation demonstrates robust short-term cost-\n465 effectiveness, it does not capture maintenance-phase impacts. Many digital\n466 intervention studies rely on static models that may overestimate cost-\n467 effectiveness by ignoring long-term adherence and system adaptations. In\n468 contrast, dynamic models suggest sustained value, but further research is\n469 needed to incorporate full cost components, subgroup heterogeneity, and the\n470 entire technology lifecycle into long-term assessments47.” (Page 14-15,\n472\n475 scalability. Discussion should better distinguish between trial-specific research costs\n476 vs. routine implementation costs.\n479 intervention expenditures and raise important questions regarding scalability.\n481 labor costs arose from trial-specific activities, such as intensive monitoring, frequent\n482 follow-up visits, and data collection. These activities significantly inflated personnel\n483 costs compared to what would be required in routine implementation. Personnel costs\n18\n484 typically decrease when digital tools transition from research to programmatic settings,\n485 as research-specific activities are eliminated. These costs are unlikely to recur at the\n486 same magnitude during routine implementation. Furthermore, task-shifting and\n487 integration into existing health service structures can further reduce personnel\n488 requirements.\n490 “Personnel expenditures constituted the dominant share of intervention costs\n491 and raised concerns regarding scalability. Much of this burden reflected trial-\n492 specific activities (e.g., additional monitoring, intensive data collection, and\n493 study-related counseling), which are not fully representative of routine service\n494 delivery. Evidence from LMIC studies suggests that task-shifting to community\n495 or paraprofessional workers can reduce personnel costs substantially, while\n496 maintaining acceptable quality of care43. For digital adherence interventions\n497 such as smart pillboxes or digital pillbox programmes (as observed in\n498 tuberculosis adherence studies), once infrastructure is in place and activities\n499 move from trial-specific to more routine settings, labor cost offsets and\n500 economies of scale tend to emerge, lowering the per-participant personnel\n501 cost44. Taken together, while personnel expenditures were a major driver of\n502 our trial costs, they are likely to diminish during programmatic roll-outs,\n503 enhancing the feasibility of implementing this intervention at scale in LMIC\n504 health systems. However, in settings with extreme resource constraints or\n505 fragmented health infrastructure, further adaptation and contextualization of\n506 the staffing model would be essential to ensure sustainability.” (Page 12-13,\n508\n509\n19\n512 considering that those in lower socioeconomic strata are disproportionately impacted\n513 by HIV. The methodology is generally sound and appropriately applied, however,\n514 certain critical areas warrant attention, and the paper does not meet the rigorous\n515 standards for Nature Communications.\n518 acknowledging the soundness and appropriateness of the methodology applied. We\n519 fully recognize that several critical areas required further attention, and we have\n520 carefully revised the manuscript to address these issues in detail, including clarifying\n521 methodological justifications, strengthening the discussion of generalizability and\n522 equity, and incorporating additional sensitivity analyses. We believe these revisions\n523 have substantially improved the rigor and transparency of the manuscript.\n524\n528 the health system differences may limit generalizability to regions like Sub-Saharan\n531 not just in the discussion.\n534 across different LMIC contexts.\n535 1. You are absolutely correct that while China is often classified as an LMIC, its health\n536 system and digital infrastructure differ significantly from many other LMICs,\n537 particularly in Sub-Saharan Africa. We recognize that these differences fundamentally\n538 impact how digital adherence interventions can be deployed.\n540 was conducted in China, where urban areas have benefited from extensive investment.\n542 integrated health information systems. However, implementation contexts vary widely\n543 across LMICs. In many settings, such as parts of Sub-Saharan Africa, constraints\n20\n544 related to electricity reliability, connectivity, and health system integration may\n545 necessitate context-specific adaptation rather than direct replication of our approach.\n546 2. In response, we have undertaken a comprehensive revision to ensure this limitation\n547 is consistently acknowledged throughout the manuscript, as you suggested. we have\n548 revised the Introduction section to explicitly acknowledge these contextual differences\n549 and to situate our study within the broader global HIV prevention landscape. In addition,\n550 we have expanded the Discussion section to note that contextual adaptation of cost\n551 parameters and implementation strategies will be critical when extrapolating our results\n552 to other LMICs. We believe these revisions provide a more consistent acknowledgment\n553 of this limitation throughout the manuscript.\n555 “China’s digital infrastructure has seen comparatively greater investment in\n556 recent years. However, contextual differences across LMICs may limit the\n557 generalizability of our findings to regions such as Sub-Saharan Africa.\n559 electricity, and digital infrastructure essential for implementing adherence\n560 technologies. For example, a survey of over 5000 facilities in eight LMICs\n561 reported extremely low digital infrastructure scores28. Similar assessments in\n563 connectivity, and policy support29. These disparities highlight the need for a\n564 systematic, human-centered, and contextually grounded approach to\n565 implementing digital HIV interventions in LMICs.” (Page 5, Introduction, line\n566 133-142)\n567 “These findings support the implementation of the smart pillbox as a cost-effective\n568 adherence strategy for oral PrEP among MSM, while underscoring the need for\n569 contextual adaptation in diverse global settings.” (Page 9, Discussion, line 271-\n570 273)\n571 “Last, our China-focused sampling strategy may limit generalizability to other\n573 such as Sub-Saharan Africa. Differences in digital infrastructure, health\n574 system capacity, and resource availability mean that both costs and\n575 effectiveness may not translate directly. Contextual adaptation of cost\n576 parameters, informed by local epidemiological and economic profiles, will\n577 therefore be critical to ensure that both efficacy and cost-effectiveness are\n21\n578 maintained when scaling up implementation in diverse settings.” (Page 15,\n580\n583 unconventional for health economic modeling. While SUR can account for correlated\n584 outcomes, it is not standard for cost-effectiveness analysis, which typically relies on\n585 decision-analytic models. Can the authors justify why SUR was chosen over more\n586 established modelling methods? Also explain its limitations compared to more dynamic\n587 or long-term modelling.\n590 and bootstrap simulations in health economic modeling. We appreciate the opportunity\n591 to clarify our rationale for selecting SUR in the context of trial-based economic\n592 evaluations and to discuss its limitations compared to more dynamic or long-term\n593 modeling approaches. Below, we provide a detailed justification based on the specific\n594 needs of analyzing correlated cost-effectiveness data within clinical trials, followed by\n595 an explanation of SUR's constraints in addressing dynamic processes.\n596 1. In trial-based economic evaluations, particularly those embedded within clinical\n597 trials, cost and effectiveness outcomes (e.g., quality-adjusted life-years or utilities) are\n598 often intrinsically correlated. This correlation arises from shared patient characteristics,\n599 treatment effects, or underlying health conditions. For instance:\n600 ⚫ Higher treatment costs may correlate with better health outcomes (positive\n601 correlation), such as when more expensive therapies yield superior efficacy.\n602 ⚫ Poorer health status could lead to increased resource utilization (higher costs) but\n603 reduced utility (negative correlation), as seen in patients with comorbidities\n604 requiring intensive care.\n606 effects, ultimately compromising the accuracy of the incremental cost-effectiveness\n607 ratio (ICER) and leading to misleading policy recommendations. Traditional methods,\n608 such as separate ordinary least squares (OLS) or generalized linear model (GLM)\n609 regressions, assume independence between costs and effects. This assumption is\n610 inefficient and risks biased uncertainty estimates, as it fails to account for the covariance\n611 structure in the error terms.\n22\n613 multiple regression equations (e.g., for costs and effects) while allowing their error\n614 terms to be correlated. This approach provides several advantages in trial-based settings:\n615 ⚫ Enhanced efficiency: SUR leverages the correlation between outcomes to\n616 produce more precise parameter estimates than separate regressions, as it utilizes\n617 generalized least squares or maximum likelihood methods. This is particularly\n618 valuable in cost-effectiveness analysis, where accurate uncertainty estimation is\n619 critical for ICER calculation and confidence interval derivation.\n620 ⚫ Robustness to real-world Data: In trial-based evaluations, patient-level data\n621 often exhibit heterogeneity and correlation driven by unobserved factors (e.g.,\n622 patient adherence). SUR models this explicitly, reducing bias compared to\n623 methods that assume independence.\n624 ⚫ Empirical support: SUR is recognized in health economics literature for\n625 handling correlated outcomes in multivariate settings. For example, El Alili and\n626 colleagues (El Alili M, et al. A scoping review of statistical methods for trial-\n627 based economic evaluations: The current state of play. Health Econ.\n628 2022;31(12):2680–2699) identified SUR and bootstrapping of cost-effect\n629 pairs as a common statistical method in trial-based economic evaluations.\n631 distribution of ICERs, generating cost-effectiveness acceptability curves (CEACs) and\n632 confidence intervals that reflect full uncertainty, including correlation effects. This\n633 combination is particularly suited to trial-based analyses where individual-level data\n634 are available, and the focus is on short-term within-trial results rather than long-term\n635 extrapolation.\n636 2. You are absolutely correct that SUR has limitations compared to dynamic modeling\n637 approaches, and we appreciate this opportunity to clarify these boundaries. While SUR\n638 is effective for handling correlation in trial-based analyses, it has inherent limitations\n639 when compared to dynamic or long-term modeling approaches, such as decision-\n640 analytic models (e.g., Markov models, discrete event simulations). We agree that this\n641 relatively short time horizon may underestimate long-term costs and outcomes. In\n642 the revised manuscript, we have expanded the Limitations section to acknowledge\n643 that our analysis reflects robust short-term cost-effectiveness but lacks\n644 longitudinal data on maintenance-phase outcomes. We now also emphasize that\n645 many prior evaluations rely on static models, which risk overstating cost-\n646 effectiveness by ignoring longer-term effects, while dynamic models suggest that\n647 digital health tools may generate sustained value (El Arab RA, et al. Systematic\n648 review of cost effectiveness and budget impact of artificial intelligence in healthcare. NPJ\n650 further research is needed using extended follow-up, dynamic modeling\n651 frameworks, and inclusion of broader cost elements and subgroup analyses to\n652 assess the long-term clinical and economic sustainability of such interventions\n653 across the full technology lifecycle. These limitations stem from SUR's static,\n654 regression-based nature:\n655 ⚫ SUR is designed for cross-sectional or short-term longitudinal data, modeling\n656 relationships at a fixed point or over a limited follow-up period. It does not\n657 inherently account for time-varying covariates, disease progression, or long-term\n658 transitions between health states. In contrast, dynamic models simulate outcomes\n659 over extended horizons (e.g., lifetime), incorporating processes like aging,\n660 treatment sequencing, and accumulating costs or effects.\n661 ⚫ SUR relies exclusively on observed trial data, making it unsuitable for\n662 extrapolating outcomes beyond the trial period without additional assumptions.\n664 epidemiologic studies, expert opinions) to project long-term cost-effectiveness,\n665 which is essential for chronic diseases or interventions with delayed effects.\n666 ⚫ SUR typically assumes linear relationships and specific error distributions (e.g.,\n667 normality), which may not hold for cost data (often skewed) or effectiveness\n668 measures (e.g., bounded utilities). While transformations or robust variants can\n669 mitigate this, dynamic models offer greater flexibility through non-parametric\n670 structures, time-dependent parameters, and microsimulation techniques.\n671 ⚫ Although SUR with bootstrap captures parameter uncertainty well, it may not\n672 fully address structural uncertainty or patient heterogeneity over time. Dynamic\n673 models facilitate comprehensive sensitivity analyses, including probabilistic\n674 sensitivity analysis (PSA) and scenario testing, to explore uncertainty in long-\n675 term parameters (e.g., discount rates, mortality trends).\n677 question and analytical approach. Our primary objective was to assess short-term cost-\n678 effectiveness within the trial period and to provide actionable evidence for immediate\n679 implementation decisions. Therefore, SUR provided the most transparent and\n24\n680 assumption-light approach. We wanted to avoid the speculative long-term projections\n681 that often characterize economic evaluations but lack empirical support.\n683 “First, the 6-month follow-up likely underestimates long-term costs and\n684 outcomes. Although our evaluation demonstrates robust short-term cost-\n685 effectiveness, it does not capture maintenance-phase impacts. Many digital\n686 intervention studies rely on static models that may overestimate cost-\n687 effectiveness by ignoring long-term adherence and system adaptations. In\n688 contrast, dynamic models suggest sustained value, but further research is\n689 needed to incorporate full cost components, subgroup heterogeneity, and the\n690 entire technology lifecycle into long-term assessments47.” (Page 14-15,\n692\n695 health coverage is not directly supported by the literature review presented in the\n696 “Comparison with existing Literature” paragraph, especially since the authors are\n697 trying to make this claim by comparing across different settings, time horizons, years.\n699 methodological differences with causal relationships. The authors should either\n700 reframe this as a hypothesis or provide empirical justification.\n703 oversimplification and was not directly supported by the evidence presented in the\n704 paragraph.\n706 introduced a more nuanced discussion that frames this idea as a testable hypothesis for\n707 future investigation, as suggested by your opinion. The new conclusion now\n708 acknowledges the limitations of comparing disparate studies and proposes that\n709 interventions like smart pillboxes, might offer a sustainable balance in resource-limited\n710 settings, a concept that warrants direct empirical testing.\n712 “Our findings on the cost-effectiveness of smart pillboxes add to the growing\n713 evidence on digital PrEP adherence tools by highlighting scalability and\n25\n714 sustainability in LMICs. Although methodological differences limit direct\n715 comparison, digital interventions show wide economic variability. For\n716 instance, a two-way texting program in Malawi reported an ICER of $24,705\n717 per additional client retained27, while our analysis estimated cost per QALY\n718 gained—underscoring the contextual diversity of evaluation approaches.\n720 at small scale27, and app-based platforms with high upfront costs and limited\n721 reach in low-resource settings22,26, smart pillboxes represent a mid-complexity\n722 alternative. Operating offline and requiring minimal user input, they may\n723 provide a more sustainable and equitable solution, especially where\n724 infrastructure gaps persist. This hypothesis warrants validation through\n725 future comparative effectiveness studies. Notably, smart pillboxes embed an\n726 equity logic: automated data capture mitigates self-report bias7 and their\n727 offline functionality bridges the digital divide that often excludes SDH-\n728 vulnerable populations26, supporting the value of passive monitoring tools in\n729 marginalized groups.” (Page 13, Discussion, line 406-421)\n730\n733 effectiveness, it does not discuss their origin, appropriateness, or limitation, especially\n734 in the context of equity. WTP thresholds are often arbitrary and often fail to reflect\n735 societal values or budget constraints in LMICs. The authors could strengthen their\n736 analysis by discussing alternative approaches (e.g., opportunity cost-based thresholds,\n737 equity-weighted thresholds).\n741 careful attention. The choice of threshold in economic evaluation represents more than\n742 a technical detail. It embodies fundamental value judgments about resource allocation,\n743 and these judgments directly impact implementation decisions.\n745 and Discussion. Below is a summary of our response.\n746 ⚫ In our Methods section, we described the adoption of the WHO-recommended\n747 cost-effectiveness threshold framework. This framework corresponds to 1–3 times\n26\n749 parameters, this yielded an evaluative range of $10,400–$31,200 per QALY gained.\n751 as a de facto benchmark in the Chinese context.\n752 ⚫ We acknowledge that this WTP threshold has limitations. In particular, it does not\n753 address equity considerations, nor does it necessarily reflect societal values,\n754 budget constraints, or opportunity costs in low- and middle-income settings. The\n755 choice of 1–3× GDP is somewhat arbitrary and may lead to decisions that are not\n756 aligned with maximizing overall population health or with fair distribution of\n757 health benefits. We have now explicitly stated in the revised manuscript that\n758 adoption of this threshold alone is insufficient to address equity concerns.\n759 ⚫ In the revised version, we added a sensitivity analysis and a new subsection\n760 discussing alternative threshold approaches, including the estimation of QALY\n761 thresholds through conversion from the Value of Statistical Life (VSL) (Cai D, et\n762 al. Estimation of the cost-effective threshold of a quality-adjusted life year in China\n763 based on the value of statistical life. Eur J Health Econ. 2022 Jun;23(4):607-615).\n765 was approximately 1.45 times GDP per capita, with an upper bound of about 2.90\n766 times and a lower bound of about 1.16 times GDP per capita.\n768 “Because a specific WTP threshold has not been established in China37, we\n769 adopted the cost-effectiveness threshold framework based on WHO-CHOICE\n770 and previous publications39–41, corresponding to 1–3 times China's 2020 per\n771 capita gross domestic product (GDP). This yielded an evaluative range of\n772 $10,400–$31,200 per QALY gained, based on China's economic parameters53.\n774 conversion42, yielding a point estimate of 1.45 times GDP per capita, with a\n775 lower bound of about 1.16 times and an upper bound of about 2.90 times GDP\n776 per capita.” (Page 22, Methods, line 693-701)\n777 “A critical consideration in interpreting our results concerns the choice of\n780 41, adopting a cost-effectiveness threshold of 1–3 times China’s 2020 per capita\n27\n782 inherently arbitrary. They may fail to capture societal values or budget\n783 constraints in LMICs, especially from an equity perspective. To address this\n784 limitation, we conducted sensitivity analyses using alternative thresholds.\n786 1.45 times GDP per capita (range: 1.16–2.90). Under this threshold, the\n787 probability range of cost-effectiveness narrowed, which enhanced the\n788 practicality of the results, but the intervention was no longer considered cost-\n789 effective at the point estimate. This finding suggests that conclusions drawn\n790 from the conventional 3× GDP rule must be interpreted with caution, because\n791 such thresholds have limited external validity and often fail to reflect the\n792 societal preferences or budget realities of LMIC settings.” (Page 12,\n28\nCost Perspective Cost difference- USD (95% Effectiveness ICER- Probability of Effectiveness ICER- USD WTP per PAD to achieve\nCI) difference- QALYs USD cost- PADs (95% CI) probability of cost-\n(95% CI) effectiveness a CI) effectiveness > 95%\nUnadjusted analysis\nHealth System 417.28 (325.71 to 508.85) 0.02 (0.01 to 0.03) 19,575.93 0.28 to 98.29 13.58 (7.47 to 19.69) 30.46 47.50\nMultipayer 419.00 (327.38 to 510.63) 19,656.66 0.25 to 98.22 30.58 48.00\nSocietal 423.01 (331.23 to 514.78) 19,844.42 0.21 to 98.04 30.86 48.50\nAdjusted analysis b\nHealth System 398.81 (312.23 to 485.38) 0.02 (0.01 to 0.03) 18,914.71 0.52 to 98.69 13.51 (7.44 to 19.59) 29.39 46.00\nMultipayer 401.12 (314.50 to 487.73) 19,024.04 0.46 to 98.62 29.55 46.00\nSocietal 406.47 (319.73 to 493.21) 19,277.87 0.35 to 98.43 29.93 47.00\n796 a We employed a lower bound of approximately 1.16 times ($12064) and an upper bound of around 2.90 times GDP ($30160) per capita to define the WTP\n797 threshold. b Adjusted analyses were performed employing a two-sided SUR model, incorporating baseline age, income, the presence of casual partners, and\n799 interval, QALY quality-adjusted life year, ICER incremental cost-effectiveness ratio, PAD PrEP-adherence day, WTP willingness-to-pay, PrEP pre-exposure\n800 prophylaxis, SUR Seemingly Unrelated Regression.\n29\n805 structural, or psychosocial factors that influence adherence, nor of intersectional\n806 vulnerabilities such as stigma, income, or housing. This limits the depth of the equity\n807 framing and the relevance of findings to other MSM populations.\n810 sufficiently engage with the broader global literature on PrEP adherence among MSM,\n811 which limited the depth of our equity framing.\n813 within the global discourse on MSM health. Specifically:\n814 ⚫ In the Introduction section, we now frame the challenge of PrEP adherence not just\n815 as an issue in China, but as a global one, by incorporating evidence on universal\n816 barriers such as stigma and intersectional vulnerabilities.\n817 ⚫ In the Discussion section, we have expanded the comparison with existing literature\n818 to include studies from other LMIC settings. We now discuss how behavioral,\n819 structural (e.g., unstable housing, low income), and psychosocial factors identified\n820 in global research similarly influence adherence patterns in our study. This allows\n821 us to draw parallels and highlight common challenges faced by MSM worldwide.\n823 “Across diverse settings, suboptimal PrEP adherence among MSM remains a\n824 persistent implementation challenge4, driven by stigma, socioeconomic\n825 marginalization, and discrimination10. These barriers are not unique to China\n826 but reflect structural determinants common to MSM globally.” (Page 3,\n828 “These factors jointly shape PrEP uptake, persistence, and the feasibility of\n829 digital health tools. Evidence shows that high-risk behavioral profiles are\n830 strongly associated with elevated HIV acquisition among MSM9, while\n831 socioeconomic constraints such as unstable housing and limited income\n832 significantly undermine PrEP adherence in LMIC settings12. Psychosocial\n833 factors, particularly stigma and discrimination, further exacerbate adherence\n834 barriers and contribute to intersectional inequities10.” (Page 14, Discussion,\n30\n835 line 425-431)\n836\n839 questions about scalability and whether similar staffing models are feasible in other\n843 of the intervention budget and for raising concerns regarding scalability in other LMIC\n844 settings.\n845 1. Upon reflection, we recognize that our trial design intentionally included intensive\n846 personnel support to ensure methodological rigor. This decision, however, significantly\n847 inflated our cost estimates beyond what would be required in routine implementation.\n849 monitoring visits, detailed data collection, study-specific counseling) rather than what\n850 would be needed for actual program delivery.\n852 high personnel costs we observed reflect additional expenditures inherent to research\n853 evaluation rather than the sustainable operational costs of routine implementation. In\n854 real-world settings, these costs would be substantially lower through integration with\n855 existing services and streamlined workflows.\n856 2. Rather than simply noting that costs would decrease, we've added specific, practical\n857 pathways to reduce personnel requirements in actual implementation. We've also\n858 acknowledged that in some LMIC settings with more severe resource constraints,\n859 additional adaptation would be necessary. Your comment has prompted us to be more\n860 realistic about implementation challenges while still highlighting the intervention's\n861 potential value when properly adapted to local contexts.\n863 “Personnel expenditures constituted the dominant share of intervention costs\n864 and raised concerns regarding scalability. Much of this burden reflected trial-\n865 specific activities (e.g., additional monitoring, intensive data collection, and\n866 study-related counseling), which are not fully representative of routine service\n867 delivery. Evidence from LMIC studies suggests that task-shifting to community\n868 or paraprofessional workers can reduce personnel costs substantially, while\n31\n869 maintaining acceptable quality of care43. For digital adherence interventions\n870 such as smart pillboxes or digital pillbox programmes (as observed in\n871 tuberculosis adherence studies), once infrastructure is in place and activities\n872 move from trial-specific to more routine settings, labor cost offsets and\n873 economies of scale tend to emerge, lowering the per-participant personnel\n874 cost44. Taken together, while personnel expenditures were a major driver of\n875 our trial costs, they are likely to diminish during programmatic roll-outs, which\n876 would enhance the feasibility of implementing this intervention at scale in\n878 or fragmented health infrastructure, further adaptation and contextualization\n879 of the staffing model would be essential to ensure sustainability.” (Page 12-13,\n881\n884 prevention outcomes could be better contextualized. The study uses perfect adherence\n885 as a binary threshold, yet imperfect adherence still conveys partial protection and this\n886 is not accounted for in the modelling. A more nuanced approach is needed. Perhaps\n887 modeling adherence as a gradient would better reflect real-world effectiveness.\n890 adherence days (PADs) as an intermediate outcome measure. You've identified a\n891 potential source of confusion in our original manuscript. This phrasing could have led\n892 readers to misunderstand our actual approach.\n894 misinterpretation. Specifically, the phrase “PrEP adherence binary [1 = perfect; 0 =\n895 non-adherent]” did not accurately reflect our methodology. As clarified in\n897 threshold. We apologize for the misleading phrasing in the main text. Our framework\n898 explicitly recognizes an important principle: imperfect adherence to oral PrEP still\n899 provides partial protection. Instead, we treated PrEP-adherence days (PADs) as an\n900 intermediate outcome. This approach enables the translation of varying adherence\n901 levels into long-term HIV prevention benefits.\n32\n903 “To calculate PADs, PrEP medication adherence was assigned a value of 1 for\n904 perfect adherence and 0 for non-adherence. PrEP adherence was treated as a\n905 continuous variable, with imperfect adherence representing partial protection.\n909 binary dichotomy. We believe this approach better reflects real-world effectiveness. We\n910 appreciate the reviewer’s comment, which allowed us to clarify this important\n911 methodological point.\n912\n915 extremely broad, limiting its practicality. This suggests high sensitivity to input\n916 assumptions or model structure. The authors should clarify which scenarios drive the\n917 lower bound and consider presenting threshold analyses to guide decision-makers\n918 more effectively.\n921 effectiveness (2.71% to 98.99%) was based on the WHO definition of cost-\n922 effectiveness categories. These categories define an intervention as highly cost-\n923 effective when the ICER is <1 times per capita GDP, cost-effective at 1–3 times, and\n924 not cost-effective when >3 times. Accordingly, the lower bound probability was\n925 primarily driven by the lower WTP threshold of 1× GDP. However, the ICER estimates\n926 themselves also contributed to this lower probability.\n927 1. To directly address this and provide more decisive guidance, we conducted\n928 additional sensitivity analyses using an alternative, evidence-based threshold derived\n929 from the Value of Statistical Life (VSL). This yielded a point estimate of 1.45× GDP\n930 per capita, with a lower bound of 1.16× and an upper bound of 2.90× GDP per capita.\n931 2. This refined analysis yielded a critical insight. Although the probability range\n932 narrowed (from 9.30% to 98.57%), the intervention was not cost-effective at the\n934 conclusion under the 3× GDP threshold is not robust when tested against a more\n935 empirically grounded valuation of health.\n33\n937 caution, as this threshold has limitations in terms of external validity and often fails to\n938 reflect societal values or budget constraints in LMIC settings. Incorporating alternative\n939 thresholds through sensitivity analysis enhances the practical relevance of our results\n940 and provides more informative guidance for decision-makers.\n942 “Sixth, when using an alternative WTP threshold (Supplementary Tables S7\n943 and S8), the cost-effectiveness probability from the societal perspective among\n944 the SDH-vulnerable group ranged from 9.30% to 98.57%.” (Page 8, Results,\n945 line 249-252)\n946 “WTP thresholds were also estimated using a Value of Statistical Life (VSL)\n947 conversion42, yielding a point estimate of 1.45 times GDP per capita, with a\n948 lower bound of about 1.16 times and an upper bound of about 2.90 times GDP\n949 per capita.” (Page 22, Methods, line 698-701)\n950 “A critical consideration in interpreting our results concerns the choice of\n953 41, adopting a cost-effectiveness threshold of 1–3 times China’s 2020 per capita\n955 inherently arbitrary. They may fail to capture societal values or budget\n956 constraints in LMICs, especially from an equity perspective. To address this\n957 limitation, we conducted sensitivity analyses using alternative thresholds.\n959 1.45 times GDP per capita (range: 1.16–2.90). Under this threshold, the\n960 probability range of cost-effectiveness narrowed, which enhanced the\n961 practicality of the results, but the intervention was no longer considered cost-\n962 effective at the point estimate. This finding suggests that conclusions drawn\n963 from the conventional 3× GDP rule must be interpreted with caution, because\n964 such thresholds have limited external validity and often fail to reflect the\n965 societal preferences or budget realities of LMIC settings.” (Page 12,\n34\nCost Perspective Cost difference- USD (95% Effectiveness ICER- Probability of Effectiveness ICER- USD WTP per PAD to achieve\nCI) difference- QALYs USD cost- PADs (95% CI) probability of cost-\n(95% CI) effectiveness a CI) effectiveness > 95%\nSDH advantage group among CROPrEP MSM (intervention group=120, control group=82)\nHealth System 463.22 (324.12 to 602.32) 0.02 (0.01 to 0.03) 23,514.83 0.53 to 79.38 15.47 (7.27 to 23.67) 30.08 53.50\nMultipayer 465.22 (326.09 to 604.34) 23,616.19 0.49 to 79.02 30.20 53.50\nSocietal 469.51 (330.29 to 608.73) 23,834.38 0.43 to 78.23 30.47 54.00\nSDH vulnerability group among CROPrEP MSM (intervention group=127, control group=113)\nHealth System 368.01 (245.19 to 490.83) 0.02 (0.01 to 0.03) 16,259.77 10.47 to 98.70 13.45 (4.99 to 21.92) 26.73 53.00\nMultipayer 369.53 (246.64 to 492.42) 16,326.77 10.11 to 98.66 26.84 53.00\nSocietal 373.24 (250.10 to 496.38) 16,490.41 9.30 to 98.57 27.09 53.50\n969 aWe employed a lower bound of approximately 1.16 times ($12,064) and an upper bound of around 2.90 times GDP ($30,160) per capita to define the WTP\n970 threshold. DIRECT-PrEP digital intervention for real-time enhancement of oral PrEP adherence, SDH Social Determinants of Health, WTP willingness-to-pay,\n35\n975 study does not explore underlying mechanisms or contextual factors that explain why\n976 cost-effectiveness differs by SDH strata—thus falling short of a deeper equity analysis.\n979 descriptive and limited to cost-effectiveness outcomes. In the revised manuscript, we\n980 expanded the introduction and discussion to emphasize how SDH factors intersect with\n981 implementation contexts in shaping intervention efficiency, and we reference evidence\n982 showing that socioeconomic barriers often erode the nominal gains of health\n983 interventions. This provides a stronger equity lens for interpreting our findings.\n985 “Achieving equity in HIV prevention requires an understanding of not only\n986 which interventions are effective, but also the mechanisms underlying their\n987 differential performance across SDH strata. SDH factors—such as\n988 socioeconomic status, education, and housing stability—directly influence\n990 multiple sexual partners, engaging in condomless sex, and substance use) are\n991 shaped by these broader structural factors and collectively influence HIV risk\n992 and PrEP cost-effectiveness. Across diverse settings, suboptimal PrEP\n993 adherence among MSM remains a persistent challenge4, driven by stigma,\n994 socioeconomic marginalization, and discrimination10. These barriers are not\n995 unique to China but reflect structural determinants common to MSM globally.\n997 adopt PrEP11, yet structural barriers—such as limited psychosocial support\n998 or economic resources—often undermine adherence12, leading to variations\n999 in cost-effectiveness across different SDH strata.” (Page 3, Introduction, line\n1000 74-87)\n1001 “In our study, cost-effectiveness differed across SDH strata, underscoring the\n1002 need to consider the mechanisms driving these disparities. As a key sexual\n1003 minority group, MSM experience heightened vulnerability due to SDH factors\n1004 that encompass both behavioral patterns and structural determinants. These\n1005 factors jointly shape PrEP uptake, persistence, and the feasibility of digital\n1006 health tools. Evidence shows that high-risk behavioral profiles are strongly\n36\n1007 associated with elevated HIV acquisition among MSM9, while socioeconomic\n1008 constraints such as unstable housing and limited income significantly\n1009 undermine PrEP adherence in LMIC settings12. Psychosocial factors,\n1010 particularly stigma and discrimination, further exacerbate adherence\n1011 barriers and contribute to intersectional inequities10. Illustratively, a door-to-\n1012 door HIV testing initiative in Uganda45, although costing only $8.29 per\n1013 participant, failed to reach daily-wage workers who feared losing income\n1014 during clinic waits, highlighting how unaddressed socioeconomic barriers\n1015 erode nominal efficiency gains. Against this backdrop, smart pillboxes\n1016 enhance adherence by automating data capture during routine use, aligning\n1017 with findings that user-friendly digital devices can improve uptake among\n1018 vulnerable groups in resource-limited settings46. Collectively, these findings\n1019 suggest that stratified cost-effectiveness reflects broader contextual burdens\n1020 rather than intervention efficacy alone. In our trial, the greater cost-\n1021 effectiveness among socially vulnerable groups likely resulted from the smart\n1022 pillbox reducing barriers such as forgetfulness in unstable living\n1023 environments and privacy concerns linked to stigma, which are often\n1024 insufficiently addressed in conventional care.” (Page 13-14, Discussion, line\n1025 422-442)\n1026\n1029 substantiated. The comparison of incremental costs ($19,844 vs. $24,705 for SMS)\n1030 lacks context—are these figures derived from comparable methodologies, time horizons,\n1031 and costing perspectives?\n1034 have revised the manuscript to address these concerns.\n1036 “contribute to the growing body of evidence”, reflecting a more accurate description of\n1037 our study's contribution.\n1039 that a direct, head-to-head comparison is challenging due to differences in\n1040 methodologies, outcome measures (e.g., our study's incremental cost per unit vs. the\n37\n1042 implementation contexts. The comparison is now framed as illustrative of the diverse\n1043 cost structures of digital health interventions, rather than a definitive statement of our\n1044 intervention's superiority.\n1046 “Our findings on the cost-effectiveness of smart pillboxes add to the growing\n1047 evidence on digital PrEP adherence tools by highlighting scalability and\n1048 sustainability in LMICs. Although methodological differences limit direct\n1049 comparison, digital interventions show wide economic variability. For\n1050 instance, a two-way texting program in Malawi reported an ICER of $24,705\n1051 per additional client retained27, while our analysis estimated cost per QALY\n1052 gained—underscoring the contextual diversity of evaluation approaches.”\n1053 (Page 13, Discussion, line 406-412)\n1054\n1057 selective. For example, citing $1021/user for a U.S. MSM app without adjusting for\n1058 context or scale may unfairly skew the comparison.\n1061 broad and lacked the necessary context for a fair comparison. We have substantially\n1062 revised this section to be more balanced and specific.\n1064 setting for the app-based intervention, and we cite the original paper's own observation\n1065 that per-user costs would decrease significantly with scale (De P, et al. Cost Analysis\n1066 of Implementing a Video-Based eHealth Intervention for HIV-Positive Gay, Bisexual,\n1067 and Other Men Who Have Sex With Men. AIDS Educ Prev. 2018 Aug;30(4):301-308).\n1069 technology type faces within its typical implementation environment. For instance, we\n1070 now highlight that the utility of smartphone apps in certain Sub-Saharan African regions\n1071 can be limited by device ownership and connectivity issues (GSM Association. The\n1073 hybrid models supporting healthcare workers may face sustainability issues under high\n1074 workloads (Jeilani A, et al. Impact of digital health technologies adoption on healthcare\n38\n1075 workers' performance and workload: perspective with DOI and TOE models. BMC\n1077 more nuanced and fair assessment of the digital health landscape.\n1079 “Our findings on the cost-effectiveness of smart pillboxes add to the growing\n1080 evidence on digital PrEP adherence tools by highlighting scalability and\n1081 sustainability in LMICs. Although methodological differences limit direct\n1082 comparison, digital interventions show wide economic variability. For instance,\n1083 a two-way texting program in Malawi reported an ICER of $24,705 per additional\n1084 client retained27, while our analysis estimated cost per QALY gained—\n1085 underscoring the contextual diversity of evaluation approaches. Compared\n1086 with SMS systems reliant on continuous network access and costly at small scale27,\n1087 and app-based platforms with high upfront costs and limited reach in low-resource\n1088 settings22,26, smart pillboxes represent a mid-complexity alternative. Operating\n1089 offline and requiring minimal user input, they may provide a more sustainable and\n1090 equitable solution, especially where infrastructure gaps persist.” (Page 13,\n1092\n1095 errors could be avoided by using a spell checker, e.g. hightened=heightened;\n1096 siginificant=significant; coverd=covered; trail-wide=trial-wide; to name a few.\n1099 manuscript.\n1101 highest standard of English. The revised manuscript has undergone professional\n1103 (attached below).\n1105 included correcting the specific errors you kindly highlighted (e.g., \"hightened\" to\n1106 \"heightened,\" \"siginificant\" to \"significant,\" \"coverd\" to \"covered,\" and \"trail-wide\" to\n1107 \"trial-wide\", etc.). This process has significantly improved the clarity, consistency, and\n1108 overall readability of our work.\n39\n1110 “As a key sexual minority group, MSM experience heightened vulnerability due to\n1112 behavioral patterns, social support) and (2) structural determinants (e.g., economic\n1113 status, education)6.” (Page 3, Introduction, line 65-69)\n1114 “Because the advantaged group derived no significant benefit, no widening of\n1115 health disparities occurred across SDH strata.” (Page 9, Discussion, line 268-270)\n1116 “Additional costs covered travel, accommodation, and meals for training-related\n1117 trips.” (Page 20, Methods, line 635-637)\n1118 “We assessed efficacy using intent-to-treat analysis and trial-wide data from\n[Redacted]\n1120\n40"}, {"round": 2, "reviewer_comments": {"reviewer_1": "The authors have addressed the initial concerns by acknowledging key limitations and adding relevant sensitivity analyses. However, the analytic design still does not sufficiently engage with the major system-level drivers—particularly the role of fixed versus variable program costs. Instead, it focuses primarily on relatively small variations in patient-level costs, which yield very similar results across perspectives. This narrow focus constrains the overall significance and policy relevance of the findings. Several tables therefore present only marginal differences, offering limited additional insight despite the application of innovative and comprehensive statistical methods.\nThat said, the study provides detailed program information and demonstrates a rigorous analytic process using rich data. These contributions may support future work that addresses more consequential system-level questions and explores scenarios where meaningful differences are more likely to emerge.", "reviewer_2": "Thank you for responding so thoroughly to the reviewer comments. I have no further suggestions.", "reviewer_3": "Thank you to the authors for considering my feedback and making the necessary changes. I have reviewed the responses and the manuscript. The authors have provided thoughtful and comprehensive responses to all comments raised, adequately addressing all core methodological, structural, and generalizability concerns and making changes to the manuscript. No further feedback from myself."}, "author_response": "Responds to the reviewers’ comments:\nReviewer #1 (Remarks to the Author): The authors have addressed the initial concerns by\nacknowledging key limitations and adding relevant sensitivity analyses. However, the analytic\ndesign still does not sufficiently engage with the major system-level drivers—particularly the role\nof fixed versus variable program costs. Instead, it focuses primarily on relatively small variations\nin patient-level costs, which yield very similar results across perspectives. This narrow focus\nconstrains the overall significance and policy relevance of the findings. Several tables therefore\npresent only marginal differences, offering limited additional insight despite the application of\ninnovative and comprehensive statistical methods.\nThat said, the study provides detailed program information and demonstrates a rigorous analytic\nprocess using rich data. These contributions may support future work that addresses more\nconsequential system-level questions and explores scenarios where meaningful differences are\nmore likely to emerge.\nResponse:\nWe sincerely appreciate the reviewer’s thoughtful comments regarding the limited engagement\nwith system-level cost drivers in our sensitivity analyses. We agree that our current analytic\nframework primarily examined variations in patient-level costs. Due to their relatively small\nmagnitude, these variations led to only minimal changes in the ICER estimates, which limited the\nadditional policy insight that could be drawn. We acknowledge this as an important limitation of\nour study.\nIn response, we have revised the manuscript to explicitly acknowledge this issue and to clarify\nthat the present analyses did not fully explore structural uncertainties related to fixed versus\nvariable program costs, economies of scale, and long-term system adaptation to digital\ninterventions. We also emphasize that these system-level factors may have substantially larger\nimpacts on cost-effectiveness than the incremental patient-level variations we tested.\nMoreover, we now highlight in the discussion that future research should incorporate scenario-\nbased modeling, alternative implementation scales, and health-system capacity constraints to\nbetter capture meaningful differences in program efficiency and policy implications. We\nappreciate the reviewer’s guidance, which has strengthened the clarity and relevance of our\nlimitations section.\nWe have implemented the following revisions to address this concern:\n“Third, although our sensitivity analyses examined multiple statistical methods and cost\nperspectives, they primarily varied patient-level costs and therefore produced only\nmarginal changes in ICER estimates. This approach did not adequately account for\nsystem-level uncertainties, such as the balance of fixed and variable program costs,\npotential scale effects in digital interventions, and the health system’s ability to maintain\nlong-term adherence gains. These structural drivers may have a much larger impact on\ncost-effectiveness and thus limit the generalizability and policy relevance of our findings.\nFuture studies should incorporate scenario analyses and system-level modeling to identify\nconditions under which more substantive economic differences may emerge.” (Page 8,\nResults, line 397-406)\nReviewer #1 (Remarks on code availability): The revised codes provide detailed instructions with\nan example dataset.\nResponse:\nThank you for your positive assessment regarding the revised code materials. We are pleased\nthat the updated code repository and example dataset now provide clear, detailed instructions that\nfacilitate reproducibility and implementation.\nReviewer #2 (Remarks to the Author): Thank you for responding so thoroughly to the reviewer\ncomments. I have no further suggestions.\nResponse:\nWe appreciate your careful evaluation and are grateful for your confirmation that no further\nsuggestions are needed. Thank you for acknowledging our efforts to address all previous\ncomments thoroughly.\nReviewer #3 (Remarks to the Author): Thank you to the authors for considering my feedback and\nmaking the necessary changes. I have reviewed the responses and the manuscript. The authors\nhave provided thoughtful and comprehensive responses to all comments raised, adequately\naddressing all core methodological, structural, and generalizability concerns and making changes\nto the manuscript. No further feedback from myself.\nResponse:\nThank you for your thoughtful review and for recognizing the revisions made in response to\nyour earlier feedback. We appreciate your acknowledgment that the methodological, structural,\nand generalizability concerns have been fully addressed.\nOnce again, we thank all reviewers for their constructive comments, which have significantly\nstrengthened the manuscript."}]

Decision letter