A randomised survey of the quality of antibiotics and other essential medicines in Indonesia, with volume-adjusted estimates of the prevalence of substandard medicines
Authors: Pisani Elizabeth , Rahmawati Ayu , Mulatsari Esti , Rahmi Mawaddati , Nathanial William , Anggriani Yusi 
Published: October 2023

Background

The World Health Organization warns that substandard and falsified medicines threaten public health in low- and middle-income countries. However, medicine quality surveys are often small and unrepresentative of the market, and the true scale of the problem remains unknown. We investigated how weighting survey results by market volume altered estimates of medicine quality.

Methods

We collected 1274 samples of allopurinol, amlodipine, cefixime, amoxicillin and dexamethasone from the internet and a randomised sample of all outlet-types where medicines are sold or dispensed in seven districts across Indonesia, the world’s fourth most populous nation. We conducted compendial testing for identity, assay, dissolution and uniformity, using USP standards. Samples that failed any chemical test were considered substandard. We compared raw prevalence of substandard medicines with prevalence adjusted by the national sales volume of each brand, relative to its weight among survey samples.

Findings

The weighted prevalence of substandard products was 4.4%, 46.9% lower than the raw estimate (8.2%). More antibiotics failed testing than other medicines (weighted prevalence 8.5 vs 3.1; raw prevalence 13.6 vs 4.9, both p<0.000). We found no relationship between quality and price; branded status; public procurement status; or outlet type.

Interpretation

The weighted prevalence of substandard medicine more closely resembled the 4.0% reported by the Indonesian medicine in surveillance of 13,539 samples of a wider range of medicines. Weighting survey results by sales volume likely improves robustness of estimates of medicine quality measured in field surveys.

Funding

The UK National Institute for Health Research funded the study.

Research in Context

Evidence before this study

In 2017, the World Health Organization (WHO) estimated that 10.5% of medicines in all low- and middle-income countries were substandard (they did not meet the standards laid out in their market authorisation paperwork) or falsified (they deliberately misrepresented content, identity or source). More recent meta-analyses focusing on specific therapeutic groups report overall prevalences of poor quality antibiotic, antiretrovirals, cardiovascular and diabetes medicines in “low- and middle-income countries” of 17.4, 1.4, 15.4, and 10.8% respectively. The authors of these reviews all warn that these aggregate prevalences are not generalisable because of unrepresentative sampling and variations in medicines included; tests performed; reference standards and pharmacopeia used; and definitions used when translating multiple quality parameters into a single pass/fail measure.

No reviewed study randomised at the level of the medicine; indeed, many deliberately aimed to collect a variety of brands. None sampled from all the outlets from which patients buy medicines. All report raw prevalence only: the number of samples considered substandard, divided by the number tested. A single, more recent study in DRC Congo, which tested 239 samples of 3 medicines sampled from wholesalers, weighted results by sales volume of each brand; this weighting reduced prevalence estimates from 27.2% to 1.3%, suggesting that sales of lower-quality brands are limited. However, the authors acknowledge that the adjusted estimate likely under-represents true prevalence because degradation in the supply chain is not fully captured in samples collected from wholesalers.

Added value of this study

This study collected medicines from a random selection of all the types of outlets where patients get medicines in seven socio-economically diverse geographic areas of Indonesia, the world’s fourth most populous country. Mystery shoppers bought five medicines from retail pharmacies, over-the-counter drug shops and the internet. We also sampled from public and private hospitals, primary health centres, doctors and nurses. We conducted full compendial testing on 1274 samples using internationally-recognised USP standards and monographs, and verified product data (batch numbers and expiry dates) with producers.

We weighted our raw estimates using brand-level sales data which covered both the retail sector and private and public health facilities, adding estimates for internet sales not captured in other data sources.

In this large, middle-income setting, weighting raw survey results by market size reduced the estimated prevalence of medicines classified as substandard by 47%, bringing our estimate considerably closer to that reported by the national medicine regulator, which randomised at the level of the medicine, and tested over 10 times as many samples. We also found that simple visual inspection identified a high proportion of samples confirmed as falsified; these samples were concentrated among branded products sold by unregulated internet vendors.

Implications for policy and practice

Aggregation of data on medicine quality from studies which do not consider market size, especially across countries with widely different medicine markets, is likely to provide a misleading picture of the threat posed by substandard or falsified medicines. The adjusted data in our study in a large and diverse middle-income country suggests that current global estimates likely overstate the problem, but this should be confirmed in other settings, by routinely weighting survey or post-market surveillance data by market size of different brands and products. These data can be harvested from customs and excise records, production and distribution reports, public procurement systems and other sources.

Our study also highlights important differences between medicines, and points to concentration of falsified products in specific settings. A more nuanced understanding of the true distribution of substandard products and of fake medicines, achieved initially through weighted estimates, would allow regulators to target prevention and future detection efforts more effectively to reduce circulation of the products most likely to be harming patients in specific country settings.