# You Can't Touch This: Potential Perils of Patient Interaction with Clinical Medical Devices

Paul Noble, Ann Blandford

<figure><img src="image_1.png" /></figure>

## To cite this version:

Paul Noble, Ann Blandford. You Can't Touch This: Potential Perils of Patient Interaction with Clinical Medical Devices. 14th International Conference on Human-Computer Interaction (INTERACT), Sep 2013, Cape Town, South Africa. pp.395-402, 10.1007/978-3-642-40480-1_26. hal-01501758

HAL Id: hal-01501758

https://inria.hal.science/hal-01501758

Submitted on 4 Apr 2017

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_2.png" /></figure>
# You Can't Touch This: potential perils of patient interaction with clinical medical devices

Paul Noble and Ann Blandford

UCL Interaction Centre, University College London,
Gower Street, London, UK

Paul.Noble@cs.ucl.ac.uk, A.Blandford@ucl.ac.uk

**Abstract.** Clinical medical devices are designed with the explicit assumption that trained medical team members will operate them in appropriate hospital environments. As technological complexity increases, along with the possibility to create specific ward configurations, the potential for unusual interaction combinations poses challenges for safety and training. Resilience engineering proposes that a system should cope with disturbances and unexpected conditions. Consequently, an important consideration for design is to examine medical device interactions that can be considered ‘non-routine’. In recognition of the localised nature of clinical practice, and in order to investigate the broad range and type of non-routine occurrences, a novel interview approach was adopted involving medical researchers and practitioners. Examples of non-routine interaction were obtained across a diverse range of localities. Covert patient interactions and dangerous configuration combinations were identified which adversely affected treatment. Drawing on these concerns the potential role of patient involvement in bolstering system resilience is discussed.

**Keywords:** Medical Devices, Safety, Resilience Engineering, Customisation

## 1 Introduction

Any patient or visitor to a medical facility cannot fail to notice modern technological equipment situated in wards, departments, and operating theatres. These widely used interactive devices undergo natural technology cycles in which manufacturers drive advancement by demonstrating the limitations of yesterday's models, and market the latest feature sets. As a result, it is not uncommon for these devices to increasingly suffer from 'feature bloat', in which new and seldom-used functionality makes operation increasingly confused and complex. An advantage of this approach is that the same medical device can be used across many hospital departments, where specific functionalities are all accessible through the device interface. However, following an institutional drive for simplification, medical device manufacturers have provided the ability to adapt and configure aspects of the user interface; for instance to show or
hide menu items, or to re-order procedural input steps and set default values. This
permits device tailoring by hospital department and ward type.

Human-computer interaction (HCI) has long concerned itself with the inadequacy
of systems that fail to reflect working practice, and are inflexible in use. In particular,
many ethnographic studies observing interactions ‘in the wild’ have emphasised fre-
quently the unplanned and situated nature of technology use [10]. Medical devices
operate within behaviourally complex and emergent environments, and the latest sys-
tems can simply relocate error by introducing as many new issues as existing prob-
lems that they are designed to solve [5]. In this regard, the opportunity for device
configuration and tailoring poses a number of design decisions balancing safety, train-
ing, and efficiency.

Resilience engineering [4] offers an alternate perspective to that of traditional risk
management, examining particular strategies and procedures adopted by those within
a system to allow it to succeed. Of specific interest is how the system is prepared for,
and reacts to, disturbances and unexpected events. This shift in thinking has contrib-
uted to a growing interest in the examination and identification of resilient strategies
surrounding interaction within complex socio-technical systems.

Bearing in mind the highly localised nature of medical device use, the objective of
this paper is to understand the clinical environment by examining unexpected events
and disturbances, and acquire generalisable insights for design. To address this aim,
the paper presents a study of these ‘non-routine’ interactions. We recognise that the
phrase non-routine is not ideal, however we deemed it the best general descriptor
which could convey the temporal, normal, and legal aspects of the following proper-
ties: atypical, infrequent, unapproved, unauthorised, and untrained. Initially, the goal
was to discover the breadth of technological interactions within the local clinical envi-
ronments that were described as non-routine by participants. Non-routine dimensions
were then extracted from these examples in order to facilitate understanding of the
tensions within this space. Secondly, a resilience engineering perspective was applied
in order to uncover areas of system vulnerabilities and weaknesses.

In this paper, existing literature from the fields of resilience engineering, appropri-
ation, customisation, and ethnomethodology are initially reviewed. Subsequently, the
novel methodology used in the study is described where interviews with medical re-
searchers and practitioners are conducted. Situated non-routine dimensions are then
presented and selected examples discussed. The following section then discusses an
identified system area with low resilience (brittleness), and discusses the potential for
unsafe complications with customised devices. The paper concludes by arguing for
patient centered care in device design and practice.

## 2 Previous Work

Health care quality assurance literature [6] describes successful healthcare organisa-
tions as having a requirement to be both flexible and efficient in order to succeed.
Lillrank suggests that this is achieved through a mix of standard, routine, and non-
routine process elements, where repetitive processes can be standardised, routine
processes consisting of both repetition and variability can be executed in dissimilar
ways, and non-routine processes can be considered so variable as to be chaotic. This
definition of non-routine complements resilience engineering's argument that a sys-
tem should handle disturbances and unexpected events. Sujan [11] describes a resili-
ence observation study of a hospital dispensary where individual staff cope with non-
routine occurrences on a daily basis, prioritising tasks in order to anticipate and mod-
erate further disturbances. It is the understanding of the dynamics and differences
from mundane practice to those surrounding the extraordinary event that is important
for the resilient design of medical devices.

Historically, the introduction of new technologies into a variety of work domains has often failed due to misconceptions about local working practice. No system is a perfect fit, and the way in which technologies are successfully adopted, adapted and incorporated into working practice is through the process of appropriation. Interactive technology can also allow some degree of customisation to better meet the needs of particular individuals and groups of users. Therefore, customisation refers to a process of technology ownership where the needs of the individual are balanced with those of the group. Appropriation 'concerns the adoption patterns of technology and the transformation of practice at a deeper level' [2], and differs from customisation in that there is a co-evolution [9] between the fitting of the technology within current organisational structures, and with the adaptation of working practices to support collaboration around the new technology. Randell [8] examined appropriation and customisation within the intensive care unit (ICU), observing situations where technologies are molded into working practices and are made to work in the way that the nurses want them to work, and not as envisaged by the manufacturer. Importantly, there is a strong motivation to share information and develop a working practice around the technology. This is described also by Wenger [12] as how an individual develops membership of a Community of Practice (CoP).

In this paper, a distinction will be made between non-routine incidences occurring
due to local appropriations and customisations, and all others. This is primarily be-
cause departments and wards generally have very good reasons for behaving in a
particular way, based upon a solid history of experience. To an individual observing
outside the particular CoP however, the reasons for these behaviours can be unclear.
In addition, local practices that are not generalisable are not of interest to this study.

Although much work and attention has focused on case studies where medical de-
vice interactions have led to undesirable outcomes, less research has been conducted
on the analysis of situations where devices are used in a non-routine way but do not
directly cause harm. System stability and quality of care may be impacted even
though there is no perceived association with error. The underlying objective of this
work is to analyse unusual socio-technical interactions in order to better understand
how to support patient health care through technology. Prior research leaves much
about resilient situated device use unanswered. In particular, there is little understand-
ing of the experiences of those who are outside the CoP but who still interact with the
situated device, and what the implications are for system resilience and patient safety.
## 3 Methodology

Resilient strategies along with appropriated and customised interactions can be considered non-routine, dependent upon the comprehension of those inside or outside the workgroup community. In order to investigate the breadth of these non-routine medical device interactions a novel interview approach involving medical researchers and practitioners was adopted. A meta-analysis of the experiences of situated researchers and clinicians was conducted, probing their findings and research, and reinterpreting them in a different way. To be clear, this does not simply imply that a literature review was undertaken; the experiences and observations of those who conducted research in particular clinical situations were directly sought.

7 academics and research associates (including 2 clinicians) on the CHI+MED medical project<fnref n="1" /> were chosen as participants in order to allow access to a broad range of professional experiences. All academics had conducted a number of situated medical studies as prior work. To facilitate the collection of rich and varied examples, participants were left to decide for themselves what the term ‘non-routine’ implied. All were invited to prepare some examples of non-routine interaction that they had encountered prior to interview. The interview consisted of two parts where prepared examples of non-routine interaction were initially recounted, before examples and experiences contributed by other participants were shared. In this way, an increasingly rich collection of examples was progressively used to stimulate and sharpen discussion. Researchers were interviewed in ascending order of experience, and prior to clinicians in order to allow examples and interpretations to be challenged or endorsed by final practitioner interviews. Semi-structured interviews with participants lasted approximately one hour and were recorded and transcribed. Initially data was categorised and filtered to remove customisation and appropriation examples. A thematic analysis was then conducted in order to draw out and identify dimensions influencing non-routine interactions. The initial analytic goal was to broadly understand the dynamics of those within the clinical environment in their efforts to manage non-routine events. Subsequently, a resilience engineering perspective was applied in order to elicit design insights from generalisable examples of use.

## 4 Results

The participants together contributed 29 examples of non-routine device interaction. 12 examples described work practice appropriations or customisations, 7 described ‘forced’ situations where there was compelling motivation for interaction within a timeframe, 4 described situations where medical research interactions were conducted by medical staff and patients, 4 described interaction errors primarily due to gaps in knowledge and experience, and the remaining 2 are what we termed ‘covert’ patient interactions.

<footnote n="1">¹ CHI+MED (Computer-Human Interaction for Medical Devices, EP/G059063/1) is an EPSRC-funded project to improve the safety of interactive (programmable) medical devices.</footnote>
## 4.1 Understanding the Clinical Environment

Four main aspects influencing whether an interaction is considered routine or not were identified. In order to assist understanding these have been arranged on a diagram representing a space of potential clinical device interactions (Figure 1). The *legality* of the interaction examines formal procedures, where subsequent approval must be sought if none exist. Work *practice* ascertains if best and local work practice is being applied, or whether workarounds are being followed. The *novelty* of the situation considers the requirement for an entirely original solution, or if a close variation can be adopted. User *training* distinguishes between complete and essential system training. The fluid and temporal nature of each dimension should be noted.

<figure><img src="image_3.png" /><figcaption>**Fig. 1.** Space of potential clinical device interactions</figcaption></figure>

From Figure 1 it is clear that non-routine interactions are only possible in the middle and outer rings, with systemic risk increasing as a function of radius. Any individual dimension has the potential to pull the system into a region of risk. This analysis highlighted a vital role played by those community members who gave *approval* for responses to non-routine interactions. Prompt approval is critical as to how such a system resiliently and legally operates in an unpredictable environment and functions by both providing a damping mechanism through the coordination of an appropriate response, and that of information distribution by disseminating awareness of a particular event.

One participant's example described the use of a newly introduced portable heart monitor to provide visibility of a patient's condition during ward transfer:
On this occasion, the heart monitor failed to activate after charging and displayed a "BATT COND" message. After some experimentation, it was discovered that if the battery was removed and reinserted, the message cleared and the unit functioned once more. Subsequently it was realised that the heart monitors' rechargeable batteries are intended to be discarded after 50 charge cycles in order to guarantee safe operation. The ward does not have the resources to register every time the batteries are recharged, and it has now become practice to remove and reinsert the battery to reset the device when the event reoccurs.

An analysis of this example using the identified non-routine dimensions highlighted that initially this was a completely *novel* situation and so no *training* could have reasonably been provided to tackle this event. However, a solution was found and the team muddled through. Approval was then given for the technically *illegal* procedure, and the workaround became (possibly bad but known) *practice*.

## 4.2 Covert Patient Interactions with Medical Devices

The majority of the examples of non-routine interactions contributed were appropriations and customisations. This is unsurprising considering that these processes are normal means of embedding a technology and tailoring for practice. Of the other examples, those that described interactions in which patients would covertly interact with their clinical device were of particular interest, due to their potential impact on working practices and system resilience.

In this study, there were two main motivators for patient interaction with clinical medical devices; the desire for unrestricted doses of a controlled substance, and for relief from device alarm noise. These examples all involved infusion pumps; devices that intravenously pump medicinal fluids into a patient's circulatory system. Most modern infusion devices provide functionality in which the keypad panel can be locked out in order to restrict access<fnref n="2" />. A physical button located on the casing, or selected through the software menu screen typically activates this type of locking mechanism. Understandably, patients are motivated to attend to this activity, as described by Participant A:

> 'But some patients are also very wise to the fact that there is a keypad lock button...because they watch, and I always teach nurses that if you are going to put the keypad lock on don't make it obvious that you are looking for the button to press that's not in a standard place...and that the patient actually works out what you're doing. You can actually do it on this particular pump by just sort of putting your hand on the top and looking like you're holding the pump as you're doing everything else'

Another strategy used by patients is to look up the device manuals online, and discover the default locking codes or physical locking locations [3]. In these manuals, interactions to be prevented are variously described as ‘unauthorised’, ‘tampering’ and

<footnote n="2">² Functionality generally aims to prevent dosage rate increases</footnote>
even 'malicious tampering'. This may seem a little harsh in the case of the patient who simply seeks relief from a constantly alarming device, however frequently resetting the device alarm can also temporarily shut off the pump with unintended consequences. Participant B:

‘Nobody wants to be shut in a room with something that’s persistently alarming. So the problem was that when they were rechecking the blood of this patient they discovered that clearly he couldn’t have been having all of his therapy. The problem is that if its [the pump] not infusing at a certain rate then you are not getting the target dose within a certain time frame, so the concentration never reaches the critical level’

In this case, the patient had been covertly resetting his alarm so frequently that in the average period that the pump was operating correctly, there was not enough time to build up a particular concentration of the infused drug in his bloodstream.

Looking up device manuals online can appear a responsible alternative for a patient to exercise in preference to device menu exploration. However, the potential consequences of looking up information online can be serious, because as we have discussed it is possible for different wards to have the same medical device but with appropriated and customised configurations. Participant C recalled an example emphasising the dangerous potential for the untrained interaction of a customised device by medical team members or patients:

‘I remember with the volumetric [infusion] pump, the trainer he pointed out that the bolus function on the thing doesn’t mean bolus, or the primer, I can’t remember but there was a specific function that either said prime or bolus or whatever, and he just said it doesn’t do that. Press this button for something else essentially. So I think that’s more to do with how the engineers set up the pumps’

Irrespective of what the particular pump functionality technically does here, the pump has been configured in a non-standard manner where a standard and labelled functional button delivers a different operation.

# 5 Conclusions and Future Work

As interactive medical devices move towards an increasingly feature rich future, it is apparent that some means of interface simplification is necessary. This paper examined potential complications caused by the ability to configure and customise, where the removal of standardisation increases the opportunity for non-routine interaction. The dynamics of non-routine interaction in a clinical medical environment suggest that device customisation will increase the burden of localised training by experienced community members, and impact safety. However, these disadvantages come at the gain of efficiencies for community members, and if possible, a safe balance must be found. One particularly interesting result of this study is the identification of covert patient interactions. These covert interactions already incur safety consequences, but are made potentially riskier when customised devices are operated using standard
manuals found online. Restriction of information as means to control [1] has now
largely been circumvented by these manuals. Using the resilience engineering per-
spective as a lens to uncover the absence of resilience can be illuminating. Particular
areas of concern are situations where humans are prevented from contributing to sys-
tem resilience. According to Participant D: 'The patients aren't there to think about
what's happening with their treatment that's the nurses job'. However, involving
patients in their own treatment would appear to be a particularly resilient action. Con-
sidering the management of false medical device alarms, this has the possibility to
free up significant amounts of nursing time, as well as enormously improving the
wellbeing of the patient. The patient should be integrated into the workgroup commu-
nity and be made aware of interaction implications. Device designers also need to
recognise and anticipate cases of patient interaction. Resilient infusion pump designs
such as that by proposed by Nemeth [7] offer visualisation of a complete cycle of a
treatment, and would have prevented some of the issues discovered in this paper.
Achieving a safe mix of device customisation and safety would appear to be a delicate
balance, and future work will explore the deeper issues surrounding the situated trade-
offs incurred.

References

1. Adams, A., Blandford, A., and Lunt, P. Social empowerment and exclusion: A case study on digital libraries. *ACM Trans ComputHum Interact* **12**, 2 (2005), 174–200.

2. Dourish, P. The Appropriation of Interactive Technologies: Some Lessons from Placeless Documents. *Computer Supported Cooperative Work* **12**, 4 (2003), 465–490.

3. Grissinger, M. A Patient's Wandering Eye and the Internet Can Lead to Pump Tampering. *Pharmacy and Therapeutics* **34**, 6 (2009), 283–328.

4. Hollnagel, E., Woods, D., and Leveson, N. *Resilience Engineering: Concepts and Precepts*. Ashgate, Burlington, VT, 2006.

5. Koppel, R., Metlay, J., and Cohen, A. Role of computerized physician order entry systems in facilitating medication errors. *The Journal of Urology* **293**, 10 (2005), 1197–1203.

6. Lillrank, P. and Liukko, M. Standard, routine and non-routine processes in health care. *Intl Journal Of Health Care QA Inc. Leadership In Health Services* **17**, 1 (2004), 39–46.

7. Nemeth, C.P., Hollnagel, E., and Dekker, S. The Infusion Device as a Source of Healthcare Resilience. In *Resilience engineering perspectives: Preparation and restoration*. 2009.

8. Randell, R. User customisation of medical devices: the reality and the possibilities. *Cogni-tion, Technology & Work* **5**, 3 (2003), 163–170.

9. Rogers, Y. Exploring Obstacles: Integrating CSCW in Evolving Organizations. *Conf on ComputerSupported Cooperative Work CSCW*, 1994, 67–78.

10. Suchman, L. *Plans and Situated Actions*. Cambridge University Press, 1987.

11. Sujan, M., Pozzi, S., and Ingram, C. Resilience as Individual Adaptation: Preliminary Analysis of a Hospital Dispensary. *CEUR Procedings 4th Workshop HCP*, (2011), 48–51.

12. Wenger, E. *Communities of Practice: Learning, Meaning, and Identity*. Cambridge University Press, 1998.