# Understanding the Stakeholders’ Expectations About an Adherence App: A Case Study

Anna Spagnolli, Luciano Gamberini, Enrico D’agostini, Giulia Cenzato

► To cite this version:

Anna Spagnolli, Luciano Gamberini, Enrico D'agostini, Giulia Cenzato. Understanding the Stakeholders' Expectations About an Adherence App: A Case Study. 18th IFIP Conference on Human-Computer Interaction (INTERACT), Aug 2021, Bari, Italy. pp.73-81, 10.1007/978-3-030-85607-6_5 hal-04291273

HAL Id: hal-04291273

https://inria.hal.science/hal-04291273v1

Submitted on 17 Nov 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
<figure><img src="image_2.png" /></figure>

ifip

International Federation for Information Processing

This document is the original author manuscript of a paper submitted to an IFIP
conference proceedings or other IFIP publication by Springer Nature. As such, there
may be some differences in the official published version of the paper. Such
differences, if any, are usually due to reformatting during preparation for publication or
minor corrections made by the author(s) during final proofreading of the publication
manuscript.
# Understanding the stakeholders' expectations about an adherence app: A case study.

Anna Spagnolli¹[0000-0002-7964-7186], Luciano Gamberini¹[0000-0003-0461-2752], Enrico D'Agostini¹, Giulia Cenzato¹

¹ Human Inspired Technologies Research Centre, Università degli Studi di Padova, via Venezia 8, 35131 Padova, Italy

anna.spagnolli@unipd.it, luciano.gamberini@unipd.it, enrico.dag@hotmail.it, giulia.cenzato@studenti.unipd.it

**Abstract.** Digital health assistants are increasingly used to improve adherence to pharmaceutical treatments because of their intuitiveness, timeliness, and ubiquity. These applications serve the goals of different kinds of stakeholders, all interested in ensuring adherence: the patients, the physicians treating the patient, the pharmaceutical companies sponsoring the treatment, and the software developers selling the application. If unquestioned, different expectations can be reflected in digital assistants pursuing erratic, confusing goals.

In this case study, we focus on an application called PatchAi (PA), which assists the collection of patients’ data during medical treatments; we aimed to understand the way in which different stakeholders conceive the role of PA. We carried out 14 interviews, with patients, physicians, pharmaceutical companies, and software developers. The interviews were recorded, transcribed, and analyzed with a two-stage thematic analysis, yielding in the end 76.8% inter-coders’ agreement. We identified six roles of PA, i.e., guide, interlocutor, safe box, lifesaver, secretary, and travel mate. We also break down the frequency with which each role is mentioned by the different classes of stakeholders involved in the interviews. We conclude with some formative design implications.

**Keywords:** Digital health assistant, Adherence apps, Stakeholders, Interviews, Expectations.

## 1 Introduction

Successful medical therapy is based on patient's medication adherence that, according to Cramer et al. [8], can be defined as the “act of conforming to the recommendations made by the provider with respect to timing, dosage, and frequency of medication-taking during the prescribed length of time” (p.46). The World Health Organization in 2003 defined adherence as a global problem of striking magnitude both in terms of the number of patients affected and of the costs it involves for the health system [14]. Non-
compliance with a treatment includes taking incorrect doses, taking medications at wrong times, being discontinuous in taking the treatment as well as stopping the treatment too soon; these behaviors can result from involuntary forgetfulness of medication, miscommunication with the care-provider, the complexity of medication, wrong beliefs and lack of motivation [11].

Smartphone applications are increasingly used to improve adherence [6]. As opposed to applications that provide medical recommendations and are often questioned for the authenticity and reliability of their sources [1, 17], adherence applications pursue the adherence to the treatment prescribed by a physician. Adherence applications support the daily routine of treatment by reminding patients to take their medication, recording all taken and missed doses, and sharing the patients' medication regimens and medication-taking with the physician [9].

Ahmed et al. (2018) identified three strategies currently implemented by adherence apps: reminders, education and behaviors. Reminders include alerts, notifications and text messages to the patients' phones informing them that the time has come to take a given medication or provide some data. Reminders improve the accuracy of the data collected because they help overcome the typical memory biases plaguing other data collection methods [2]. Educational tools provide information about the medicine prescribed in order to dispel misbeliefs; the reliability and authenticity of the application's sources are crucial in this case [17]. Behavioral strategies are persuasive strategies meant to motivate adherence, hopefully driving the patient towards the state where they feel empowered [13]. Behavioral strategies include tracking the treatment progress, setting goals and receiving scores and badges. Data illustrating the patients' treatment history also was found to increase treatment adherence significantly [10, 20]. A recent meta-analysis by Wiecek et al. [21] suggests that a comprehensive multi-component mobile intervention is the most effective long-term solution, considering that non-adherence is a complex phenomenon with several determinants.

One strategy to improve the effectiveness of adherence apps is the use of a conversational format. A meta-analysis by Wald et al. [19] compared eight randomized trials using text messages to improve treatment adherence, either as one-way reminders or as two-way reminders. In the latter case, the patient would send replies confirming whether a given medication has been taken. A 23% improvement in adherence was found when this last strategy was applied, compared with a 4% improvement using one-way reminders. For these reasons, adherence apps use conversational interfaces. Some can interpret and respond to the patients' input delivered as free text (or speech), using natural language processing; other conversational interfaces use predetermined messages in both the application's turns and the patient's replies [7]. Conversational agents deployed in health services and applications do not "chat" with patients but engage in goal-oriented transactions with them. These transactions aim to support the patients' treatment (i.e., treatment implementation, management, adherence, support, and monitoring) or a behavior change program; connect patients to health care services; answer health-related queries or establish a diagnosis/triage [7]. Regardless of their goals, they are generally well-received, especially for their timeliness and ease of use [7].
The patient is typically considered the target of adherence application; however, physicians and pharmaceutical companies study the data collected, or prepare the treatment protocols. The heterogenous agendas of these different stakeholders can be conveyed in the app and make the nature of the transaction with the app confusing to the user. A formative evaluation to identify and compare the stakeholders’ expectations can help avoid such confusion.

## 1.3 Study goal

In the present study, we focus on one adherence app, called PatchAI, henceforth PA. PA can be installed on the patients’ smartphones to report their health conditions during the whole duration of a treatment. Its companion web application allows physicians to monitor their patient’s data remotely. The smartphone interface relies on an intuitive interface to improve the patients’ retention in the monitoring program. In particular, it uses a conversational digital assistant to prompt data collection with a set of predefined options, for instance, asking which symptom was experienced that day and how acutely. It also ushers the patients to the various features relevant to them or reminds them of data that was scheduled to be provided. The conversation interface resembles an instant messaging application. ( Screenshots can be found here: https://patchai.io/en). Formative evaluations are carried out on a product while still formed, often with qualitative methods [4]. Our study has a formative goal, i.e., identifying the stakeholder’s expectations about PA role and possible frictions or gaps between them.

# 2. Method

## 2.1 Participants

Four classes of stakeholders had a direct interest in PA: patients, researchers or physicians monitoring the patient’s treatment, pharmaceutical companies sponsoring a treatment, and developers working at the application. The main requirement when recruiting members of those classes was having a familiarity with treatments involving data monitoring. We could not reach any patient who was using PA at the time because they were bound to confidentiality constraints established before the start of our study. Therefore, the kind of patient’s expectations we collected here are those of patients who would consider using the application prospectively. They were explained the basic features of the application during a hands-on demonstration meeting organized some weeks earlier by the company. Only one of them gathered the information on the app from the website. Physicians, pharmaceutical representatives and developers were already familiar with PA, being its direct stakeholders.

The stakeholders reached for this study were 14: six patients familiar with clinical trials involving data monitoring (oncological patients and patients suffering from migraine); three physicians running a medical trial with the PA at the time of the interview; three
developers from PA company; two representatives of pharmaceutical companies using
PA. No compensation was given to participate in the study.

## 2.2 Interviews

The interviews took place in August and September 2020. They were carried out indi-
vidually and remotely, at a time agreed upon with the participants. Each interview
lasted about 30 minutes, and the audio was recorded. The interviews focused on the
application in general and on the virtual assistant in particular. The interviewees were
encouraged to answer from their perspectives. The interview format was semi-struc-
tured, with a few predefined questions and additional follow-up questions to clarify or
elaborate. The predefined questions were:

- what do you think could be the usefulness of an application like PA?

- do you think that PA application can be useful also to other categories such as physicians/patients?

- what are the advantages/disadvantages of using this application compared with other tools you used to collect health data in the past?

- why would patients discontinue sending their data via PA?

- what would be the perfect digital health assistant?

Then the participant was thanked and greeted.

## 2.3 Ethics

The study complies with the Declaration of Helsinki (2013) and the European General Data Protection Regulation (2016/679, GDPR). Before the interview, each participant received the information note and a consent form via email; they agreed to participate by signing the consent and returning it via email to the principal investigator. No interviewee was obliged to participate; participation or withdrawal did not affect their participation in future or current treatments. The information note described the goal of the study (i.e., improving PA), the focus, modality, and duration of the interview, the data protection policy, the participants' rights, and the contact information of the research team carrying out the study. The interview transcripts are anonymous, and their association with participants' identification data was deleted at the end of the study; also, the audio files were deleted once transcribed.

## 2.4 Analysis

A two-stage thematic analysis [4, 18] was carried out on the transcribed interviews to identify the roles attributed to PA. The first phase proceeded in a bottom-up direction: the principal investigator and another research team member read aloud the interviews together and identified their recurring themes. Six different themes (i.e., PA roles) were found. The second phase was top-down. Two members of the research team who did not participate in the bottom-up phase served as independent coders. They were trained on two interviews and then coded the remaining 12 interviews autonomously. For each sentence in the interviews, they decided whether it mentioned any of the six roles or
not; they were instructed to create a new role if the six predefined ones did not fit the sentence. No new theme emerged during the coding process, thereby indicating that data saturation was reached. The agreement between coders was 76.8%. Then, the two coders jointly decided how to code the sentences on which they initially disagreed. Overall, 352 sentences were coded, containing 373 mentions to PA roles.

## 3. Results

Overall, the stakeholders referred to six possible roles of PA in the interviews; we named them “guide,” “interlocutor,” “safe box,” “lifesaver,” “secretary,” and “travel mate.” We described them using a rhetorical storyline and some extracts exemplifying them. Rhetorical storylines are “synthetic statements which parsimoniously summarize the unique topics invoked in the corpus” [12].

**Secretary:** Interviewees refer to PA as an assistant to overloaded physicians, taking on the burden of collecting, storing, and analyzing the data from the patients. “At any point in time, we can say ‘let see how that patient is doing, let’s see how that other patient is doing’” or “I’m interested in knowing, for instance, the frequency and length of the events in a given month, the medicines taken, everything, in a blink.” Such an assistant is void of any medical responsibility.

**Guide:** Interviewees refer to PA as a guide supporting the patients in providing health data. It helps the patients keep their intended data collection schedule and simplify the data entry process through a usable interface. This remark is expressed by sentences such as “[PA] can guide the patient who might have some difficulties in using an application” or “PA is patient-friendly as they say, I mean it is very close to their [the patients’] language.”

**Interlocutor:** Interviewees refer to PA as an interlocutor due to the conversational format. This role transpires from sentences such as: “PA can communicate with some empathy, through which one perceives some sort of closeness.” Being an interlocutor, PA would also enliven the data collection routine by making it more personal and various, “there must be some variation…or some personalization even, so it is not always the same over and over.”

**Safe box:** Interviewees refer to PA as a means to collect and preserve data. The emphasis is here on the data and the quality of their collection, storage, and access. PA allows collecting accurate data, keeping them safely in compliance with privacy norms, and making them conveniently inspectable from both patients and physicians. This role transpires from sentences such as “not to lose, or squander, relevant information about their symptoms” or “data are managed really accurately, and are always used for positive goals, for their stated goals.”
**Lifesaver:** Interviewees refer to PA as a necessary component of the treatment allowing the patients to play an active role, become more aware of their health condition, and be useful to themselves and others. This role is expressed by sentences such as “having to provide regular information becomes part of the treatment itself” or “the patient feels that their opinion, judgment, disclosures – so to speak – matter.”

**Travel mate:** Interviewees refer to PA as a companion being always with the patients during their sickness. “Surely, a support like a chatbot makes you feel constantly assisted.” The installation on personal smartphones and the regular opening of conversations with the patient contribute to this impression.

For each class of stakeholders, we calculated the number of mentions to each role; then, we averaged it by the number of stakeholders in that class. The average frequency per stakeholder class is reported in Table 1 and allows to appreciate the emphases given by different stakeholders to each role.

<table><caption>**Table 1.** The average number of mentions to the six roles within each stakeholder class (a mention is a transcripts' sentences referring to a role).</caption><thead><tr><th></th><th>Guide</th><th>Interlocutor</th><th>Safe box</th><th>Lifesaver</th><th>Secretary</th><th>Travel mate</th></tr></thead><tbody><tr><td>Patients</td><td>5.33</td><td>6.00</td><td>2.17</td><td>1.33</td><td>6.50</td><td>2.50</td></tr><tr><td>Physicians</td><td>5.67</td><td>0.67</td><td>8.67</td><td>6.33</td><td>10.67</td><td>0.00</td></tr><tr><td>Pharma rep.</td><td>5.50</td><td>4.00</td><td>3.50</td><td>2.00</td><td>16.50</td><td>5.50</td></tr><tr><td>Developers</td><td>5.33</td><td>8.67</td><td>1.67</td><td>2.00</td><td>2.00</td><td>0.33</td></tr></tbody></table>

Given the scope of this study and its small sample size, the data in Table 1 will be commented qualitatively. It seems that no role is paramount or exclusive to a specific stakeholder class; at the same time, each class puts a different emphasis on each role. The patients mainly referred to three roles: PA as a guide, interlocutor, or secretary; less often did they mention safe box and lifesaver roles. Conversely, safe box and lifesaver are frequently mentioned by the physicians, in addition to the secretary role; this suggests that physicians are very interested in the medical data provided by PA and in being relieved from the burden of collecting them. Secretary is also the role that the representatives of the pharmaceutical companies mentioned more often; this reflects their interest in making the application useful to the physicians, the class of PA with which they are more in touch. Finally, the developers seem to mention the role of guide and interlocutor more often than the other roles; this suggests their great concern with the usability and intuitiveness of the interface as a key strategy to obtaining adherence. In the next section, we will discuss these results and propose some formative recommendations.
## 4. Discussions and conclusions

The previous section highlights the multifacetedness of the application, whose advantages are distributed over a complex ecology of parties with different, interconnected objectives. This multifacetedness is demonstrated by the six roles that emerged from the analysis of our interviews. At the same time, Table 1 suggests that these roles are perceived differently by different stakeholders. PA can be conceived as a “boundary object” [16] shared by four classes of stakeholders: it represents not only an object they have in common but also one they invest in with different sets of expectations. Our formative contribution to the improvement of this application is then to highlight what roles coexist in the same application and whether there are gaps or frictions between them.

Some roles could become more evident to make adherence more effective. In particular, two roles that physicians already appreciate in PA could be made more visible to patients as well: the creation of a safe dataset of health data, on the one hand, and the close monitoring of a given treatment on specific patients, on the other hand. We named these roles “safe box” and “lifesaving,” respectively. In both cases, the application puts patients in control of the monitoring process and makes them responsible for the treatment’s outcome. These roles then provide a motivational boost to retain the patient in the treatment they contribute to building. Interestingly, however, the “safe box” and “lifesaving box” roles do not come forward in the interviews with the patients. We can assume that it might be even less so to patients who – unlike the ones we interviewed – are administered a treatment without being part of a clinical trial: they are not instructed to appreciate the considerable advantages of constant monitoring. To improve the visibility of these roles to patients, PA could stress the importance of each data received from the patient. The app, for example, could quantify the advancement in the monitoring process enabled by the patients’ data; and the physicians’ and scientists’ access to this data could become visible to the patients, proving to them that their effort is of immediate use.

Some frictions between roles could be prevented from blossoming to keep expectations realistic. For example, there is a potential conflict between the “interlocutor” and “secretary” roles, which could mislead some stakeholder’s expectations. PA is an interlocutor because it adopts a conversational format; this is one critical feature making PA intuitive and pleasant to use and is often mentioned by the patients and developers in our interviews. PA is also a secretary because it alleviates a burden from the physicians’ shoulders by taking over the most mechanical parts of the monitoring process. This role is very often mentioned by the physicians and the representatives of the pharmaceutical companies in our interviews. The friction between these two roles can be generated by the PA’s conversational format and the immediacy of its interactional style, which might lead to overestimating its agency beyond the capabilities of a secretary. The patients might attribute the app the ability to interpret the data collected and even intervene upon them. Indeed, one patient in our sample expected the PA to be backed up by a medical team ready to intervene after receiving bad health updates. To avoid this misunderstanding, PA reminds the user that the app should not be considered an emergency alert. The realistic perception of PA’s skills can also be obtained by checking the way in which the digital health assistant describes itself and its activities
during the conversation with the patient, avoiding the impression of having some clinical agency.

Our sample was heterogeneous but small and did not include patients using or having
used PA. Regardless of these limits, this study can provide insights that can be extended
to other adherence apps using a digital assistant to address the patients/users. Some of
the expectations that emerged from our interviews can be tracked back to functions that
PA shares with other applications (i.e., sending reminders, storing data, engaging the
user with a conversational interface) and are then likely to appear in similar apps. The
same goes for the formative recommendations we provided in this section. Of general
relevance outside this case study is also the method we have adopted, which gives a
voice to the expectations of the different stakeholders having a close role in the appli-
cation. Digital assistants could pave the way to more patient-centered treatment [15];
in order to do so, their design must have a good user model and consider the different
groups of users involved [3].

Acknowledgments

We are very grateful to the informants who accepted to be interviewed and shared
their thoughts with us. We also thank Valentina Irene Missaggia and Marina Smorto
for their help in coding the data. The study was sponsored by PatchAi via the research
agreement "Supporto al design e allo sviluppo dell'app. PatchAi®".

References

1. Alamoodi, A. H., Garfan, S., Zaidan, B. B., Zaidan, A. A., Shuwandy, M. L., Alaa, M., ... & Alobaidi, O. R. (2020). A systematic review into the assessment of medical apps: Motivations, challenges, recommendations and methodological aspect. Health and Technology, 1-17. (DOI: 10.1007/s12553-020-00451-4)

2. Anghel, L. A., Farcas, A. M., & Oprean, R. N. (2019). An overview of the common methods used to measure treatment adherence. Medicine and pharmacy reports, 92(2), 117. (DOI: 10.15386/mpr-1201)

3. Bath P.A. 2008. Health informatics: current issues and challenges. J. Inf. Sci. 34, 4 (August 2008), 501-518. DOI:https://doi.org/10.1177/0165551508092267.

4. Baxter, K., Courage, C., & Caine, K. (2015). Understanding your users: a practical guide to user research methods. Morgan Kaufmann.

5. Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative research in psychology, 3(2), 77-101.

6. Car, J., Tan, W. S., Huang, Z., Sloot, P., & Franklin, B. D. (2017). eHealth in the future of medications management: personalisation, monitoring and adherence. BMC medicine, 15(1), 1-9. (DOI: https://doi.org/10.1186/s12916-017-0838-0)

7. Car, L. T., Dhinagaran, D. A., Kyaw, B. M., Kowatsch, T., Rayhan, J. S., Theng, Y. L., & Atun, R. (2020). Conversational agents in health care: Scoping review and conceptual analysis. Journal of medical Internet research, 22(8), e17158. (DOI: 10.2196/17158)

8. Cramer, J. A., Roy, A., Burrell, A., Fairchild, C. J., Fuldeore, M. J., Ollendorf, D. A., & Wong, P. K. (2008). Medication compliance and persistence: terminology and definitions. Value in health, 11(1), 44-47. (DOI: 10.1111/j.1524-4733.2007.00213.x)
9. Dayer, L., Heldenbrand, S., Anderson, P., Gubbins, P. O., & Martin, B. C. (2013). Smartphone medication adherence apps: potential benefits to patients and providers. Journal of the American Pharmacists Association, 53(2), 172-181. (DOI: 10.1331/JAPhA.2013.12202)

10. Demonceau, J., Ruppar, T., Kristanto, P., Hughes, D. A., Fargher, E., Kardas, P., ... & Vrijens, B. (2013). Identification and assessment of adherence-enhancing interventions in studies assessing medication adherence through electronically compiled drug dosing histories: a systematic literature review and meta-analysis. Drugs, 73(6), 545-562. (DOI: 10.1007/s40265-013-0041-3)

11. Jin, J., Sklar, G. E., Oh, V. M. S., & Li, S. C. (2008). Factors affecting therapeutic compliance: A review from the patient's perspective. Therapeutics and clinical risk management, 4(1), 269. (DOI: 10.2147/tCRM.S1458)

12. Montiel, C. J., Uyheng, J., & Dela Paz, E. (in press). The Language of Pandemic Leadership: Mapping Political Rhetoric During the COVID-19 Outbreak. Political Psychology.

13. Pereira, J., & Diaz (2019). Using health chatbots for behavior change: A mapping study. Journal of medical systems, 43(5), 135.(DOI: 10.1007/s10916-019-1237-1)

14. Sabatè E. Adherence to Long-Term Therapies: Evidence for Action. Geneva, Switzerland: World Health Organization; 2003. https://www.who.int/chp/knowledge/publications/adherence_report_fin.pdf?ua=1

15. Sharma, T., Bamford, M., & Dodman, D. (2015). Person-centred care: an overview of reviews. Contemporary nurse, 51(2-3), 107-120. (DOI: 10.1080/10376178.2016.1150192)

16. Star, Susan; Griesemer, James (1989). "Institutional Ecology, 'Translations' and Boundary Objects: Amateurs and Professionals in Berkeley's Museum of Vertebrate Zoology, 1907-39" (PDF). Social Studies of Science. 19 (3): 387-420.

17. Subhi, Y., Bube, S. H., Bojsen, S. R., Thomsen, A. S. S., & Konge, L. (2015). Expert involvement and adherence to medical evidence in medical mobile phone apps: a systematic review. JMIR mHealth and uHealth, 3(3), e79. (DOI: 10.2196/mhealth.4169)

18. Thomas, D. R. (2006). A general inductive approach for analyzing qualitative evaluation data. American journal of evaluation, 27(2), 237-246.

19. Wald, D. S., Butt, S., & Bestwick, J. P. (2015). One-way Versus Two-way Text Messaging on Improving Medication Adherence: Meta-analysis of Randomized Trials. The American Journal of Medicine, 128(10), 1139.e1–1139.e5

20. van Heuckelum, M., van den Ende, C. H., Houterman, A. E., Heemskerk, C. P., van Dulmen, S., & van den Bemt, B. J. (2017). The effect of electronic monitoring feedback on medication adherence and clinical outcomes: a systematic review. PloS one, 12(10), e0185453. (DOI: 10.1371/journal.pone.0185453)

21. Wiecek E, Tonin FS, Torres-Robles A, Benrimoj SI, Fernandez-Llimos F, Garcia-Cardenas V. Temporal effectiveness of interventions to improve medication adherence: A network meta-analysis. PLoS One. 2019;14(3):e0213432 (DOI: 10.1371/journal.pone.0213432)