From principles to practice: benchmarking government guidance on health apps
Authors: Agata Ferretti, Elettra Ronchi, Effy Vayena
Journal: The Lancet Digital Health (2019)

Patient-facing mobile health applications (apps) hold the promise to change the way individuals take responsibility for their own health by enabling more effective delivery of health information, allowing better monitoring of symptoms, and encouraging healthier lifestyles.1Boulos MNK Brewer AC Karimkhani C Buller DB Dellavalle RP Mobile medical and health apps: state of the art, concerns, regulatory control and certification.Online J Public Health Inform. 2014; 5: 229PubMed Google Scholar Enthusiasm about the potential of health apps has grown rapidly, generating uncertainty as to who should regulate such apps and how. In most countries, medical device regulation applies only to a subset of high-risk health apps that have well defined medical purposes. However, most health apps available on the market target a wide range of health-related issues, including diet and exercise, pregnancy, and mental health, while still being considered non-medical devices. These apps can collect a variety of personal data2Vayena E Dzenowagis J Brownstein JS Sheikh A Policy implications of big data in the health sector.Bull World Health Organ. 2018; 96: 66-68Crossref PubMed Scopus (54) Google Scholar and, because they are designed to affect health, it is important to ensure their safety, validity, reliability, privacy, and security. Many low-quality health apps exist that, as well as providing advice that is incomplete, misleading, or wrong, might also fall short of meeting the expected standards in privacy and security.3Huckvale K Prieto JT Tilney M Benghozi PJ Car J Unaddressed privacy risks in accredited health and wellness apps: a cross-sectional systematic assessment.BMC Med. 2015; 13: 214Crossref PubMed Scopus (240) Google Scholar, 4Powell AC Torous J Chan S et al.Interrater reliability of mHealth app rating measures: analysis of top depression and smoking cessation apps.JMIR Mhealth Uhealth. 2016; 4: e15Crossref PubMed Scopus (83) Google Scholar In a 2017 report, the Organisation for Economic Co-Operation and Development (OECD) concluded that the use of low-quality, non-medical health apps raises a wide range of ethical, legal, and governance issues, and pointed to the need for international agreement on minimum standards in quality assurance controls.5Organisation for Economic Co-operation and DevelopmentMobile technology-based services for global health and wellness: opportunities and challenges.http://www.oecd.org/sti/ieconomy/Summary-Mobile-Health-Oct2016.pdfDate: 2017Date accessed: April 10, 2019Google Scholar We examined guidance for the development of safe, secure, and reliable apps issued by data protection authorities in nine OECD countries (see appendix for methodological details). All these authorities had reported guidance relating to mobile apps in response to the 2017 Census of the International Conference on Data Protection and Privacy Commissioners. We also examined guidance by national health authorities in the same nine countries, and international guidance issued by WHO and the European Commission (EC). We did a comparative assessment of the guidance against the qualitative indicators and the principles and best practices set out in the 2013 OECD Privacy Guidelines and the 2016 OECD Recommendation on Health Data Governance (appendix). The documents issued by data protection authorities are largely similar to one another. Most address app developers, providing information in question format or checklists, and are easily accessible online. All the documents from data protection authorities focus mainly on privacy and data protection, and include the core principles of the two OECD recommendations. These documents aim to clarify how to comply with relevant national legislation for app development in general, but not specifically for non-medical health apps. Among the documents from data protection authorities, the extent to which explanatory and operational guidance is included varies. Although most documents cover rules on user data (eg, guidance on user's consent, right to access their data, and data portability), only four documents address privacy impact assessment. The documents also included few specifications concerning third party access and use of data. In particular, the documents did not contain advice on good practice to adopt when data is used for marketing and in-app advertising. Furthermore, few documents discuss good practice on apps for children or disabled people. We also searched the websites of the national health authorities in the same nine countries for specific guidance, and contacted their respective OECD representatives for confirmation (appendix). Most European ministries and the Australian department of health confirmed that their countries had not developed specific guidance. Guidance might have been produced at a territory level or by professional organisations, but reviewing such documents was not in the scope of this report. Only two national health authorities have issued specific guidance: the UK National Health Service (NHS) and the French Haute Autorité de Santé (HAS). The types of guidance delivered by these authorities differ substantially. HAS good practice guidelines are aimed at app developers and evaluators (evaluating bodies, consumer associations, or medical professional organisations), and the guidance is based on five categories of good practice requirements: informing users, health content, technical content, security and reliability, and usability. The guidance includes a risk matrix to help tailor and assess good practice according to the app's intended uses. In the UK, the NHS has developed a review process for the selection of non-medical apps to be included in the NHS app library. The review process is based on an online questionnaire listing good practice goals in eight core areas: clinical effectiveness, regulatory approval, clinical safety, data protection, security, usability and accessibility, interoperability, and technical stability. At the EU level, in 2016, the EC issued the draft Privacy Code of Conduct on health apps for public consultation. This document includes most of the criteria listed by the UK and French health authorities, in addition to specific and practical guidance on privacy and data protection principles to be taken into account by health app developers. At international level, the mHealth Evidence Reporting Assessment, developed by the WHO Technical Evidence Review Group in 2016, also includes most of the criteria listed by the NHS and HAS, with a focus on assessment of an app's technical features, rather than non-technical aspects such as clinical safety and privacy. In summary, although governments are developing regulation and guidance for app developers, our report shows that this guidance is siloed and not comprehensive. Data protection authorities are focused on privacy and data protection issues, whereas health authorities introduce safety and efficacy considerations. The spread of different sources of guidance across agencies leaves it up to developers to navigate the complex regulatory environment. As apps are often global products, multiple guidelines and different agencies and requirements make compliance onerous and accountability measures unclear. Although our analysis cannot directly link this fragmented landscape with the poor quality standards of non-medical health apps that has been documented, fragmentation and lack of comprehensive guidance is probably not conducive to effective governance.6Bates DW Landman A Levine DM Health apps and health policy: what is needed?.JAMA. 2018; 320: 1975-1976Crossref PubMed Scopus (60) Google Scholar Professional organisations, academics, and the private sector have stepped in to provide additional guidance. For example, guidelines have been produced by Xcertia to support consumers and clinicians in choosing mobile health apps, and to help developers in complying with industry-wide accepted standards. The British Publicly Available Specification 277 (health and wellness apps—quality criteria across the life cycle) provides a wide variety of specifications on health apps, including topics such as fitness for purpose, risk management, quality criteria, and support. Additionally, medical associations, such as the Royal Dutch Medical Association,7Royal Dutch Medical AssociationMedical app checker: evaluation of mobile medical apps.https://www.knmg.nl/web/file?uuid=39ce54ca-ceec-4d91-b242-05bb8cee5e23&owner=5c945405-d6ca-4deb-aa16-7af2088aa173&contentid=3915Date: 2016Date accessed: April 10, 2019Google Scholar have provided sector-specific guidance and standards to evaluate effectiveness and safety of health apps. At the national level in the USA, the Agency for Healthcare Research and Quality issued an evaluation of the efficacy, usability, and features of commercially available apps for diabetes self-management in 2018.8Agency for Healthcare Research and QualityMobile health applications for self-management of diabetes.https://effectivehealthcare.ahrq.gov/sites/default/files/pdf/technical-brief-31_mobile-health-applications-for-self-management-of-diabetes.pdfDate: 2018Date accessed: April 10, 2019Google Scholar In the UK, the NHS, in collaboration with the National Institute for Health and Care Excellence, has issued an evidence standards framework. Finally, although standardised voluntary self-certification or star rating tools could help consumers and clinicians to reach informed decisions regarding app use, these tools rely on developers' accountability and competence and have yet to be proven reliable.9Chan SR Misra S Certification of mobile apps for health care.JAMA. 2014; 312: 1155-1156Crossref PubMed Scopus (25) Google Scholar Greater policy coordination for the governance of health apps is needed to reduce guidance gaps (eg, on data access and user autonomy), to make quality standards visible and clear, and to create an accessible common reference for developers, users, and payors. As countries move to develop strategies for greater patient-centred care, we must enable cross-country agreement on minimum quality assurance standards to guide app development and use. An OECD-led multi-stakeholder initiative, building on OECD's ongoing work on health data governance, would be a crucial step towards global consensus. We declare no competing interests. AF is supported by the Swiss National Science Foundation (grant number 407540_167223; NRP 75 Big Data). ER is a Senior Policy Analyst for the Organisation for Economic Co-Operation and Development (OECD). The opinions expressed and arguments employed herein do not necessarily reflect the official views of the OECD or of the governments of its member countries. Download .pdf (.22 MB) Help with pdf files Supplementary appendix