# Usability Heuristics and Qualitative Indicators for the Usability Evaluation of Touch Screen Ventilator Systems

Dinesh Katre, Ganesh Bhutkar, Shekhar Karmarkar

► To cite this version:

Dinesh Katre, Ganesh Bhutkar, Shekhar Karmarkar. Usability Heuristics and Qualitative Indicators for the Usability Evaluation of Touch Screen Ventilator Systems. Second IFIP WG 13.6 Conference on Human Work Interaction Design: Usability in Social, Cultural and Organizational Contexts (HWID), Oct 2009, Pune, India. pp.83-97, 10.1007/978-3-642-11762-6_8. http-01056232

HAL Id: hal-01056232

https://inria.hal.science/hal-01056232

Submitted on 18 Aug 2014

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Usability Heuristics and Qualitative Indicators for the Usability Evaluation of Touch Screen Ventilator Systems

Dinesh Katre<fnref n="1" />, Ganesh Bhutkar<fnref n="2" />, Shekhar Karmarkar<fnref n="3" />

**Abstract.** A ventilator system provides respiratory support to critically ill patients in the Intensive Care Unit. Increasing complexity in the user interface, features and functionalities of ventilator systems can cause medical errors and cost the life of a patient. Therefore, the usability of ventilator systems is most crucial to ensure patient safety. We have evolved a specialized set of heuristics combined with objectively defined usability indicators for the usability evaluation of touch screen based ventilator systems. Our study presents the heuristic evaluation of three touch screen based ventilator systems manufactured by three different companies. The heuristic evaluation has been performed by four different usability evaluators to ensure the reliability of heuristics proposed in this paper. The specialized set of heuristics linked with user interface components and the objectively defined usability indicators are found more reliable in identifying specific usability problems of ventilator systems.

**Keywords:** Touch Screen Ventilator System, Intensive Care Unit, Specialized Heuristics, Usability Indicators, Usability Evaluation, Patient Care

## 1 Introduction

Modern healthcare is supported by variety of complex medical equipments like ventilator system, multi-parameter monitoring system, defibrillator, ECG analyzer, etc. Mechanical age medical equipments are now undergoing major technological upgradation with the advent of embedded electronic equipments, small size displays, information technology and ubiquitous applications wherein the equipments can be networked together. This effort is directed at reducing process inefficiencies, improving the quality of patient care and controlling the healthcare costs. Increasing complexity of functionalities and features in healthcare systems is also resulting in potential usability and design errors.

Medical error is a leading cause of death along with motor vehicle accidents, breast cancer and AIDS [14]. Many medical devices have user interfaces that are so poorly designed and difficult to use that they cause a variety of human errors. Usability of

<footnote n="1">¹Group Coordinator, Human-Centered Design and Computing Group, C-DAC, Pune, India</footnote>

<footnote n="2">²Assistant Professor, Department of Computer Engineering, Vishwakarma Institute of Technology, Pune, India</footnote>

<footnote n="3">³Medical Consultant, Pune, India</footnote>
medical devices is most crucial to ensure safety and to enable physicians to focus on their patients rather than technology [1, 4]. Therefore, it is necessary to consider all such aspects of device design in a practical sense to ensure the optimal usability as well as performance of the medical device.

During our discussions with physicians, many of them highlighted the criticality of ventilator systems from the point of view of usability and recommended it for our usability evaluation. A ventilator system gives respiratory support to critically ill patients [5]. Ventilators can be classified as: mechanical, electronic or touch-screen based. We have specifically considered touch-screen based ventilator systems for our study.

There are many techniques available for usability evaluation [10] such as cognitive walkthrough, expert reviews, focus groups, Delphi technique, heuristic evaluation etc. We observed three ventilator systems manufactured by different companies to find major design problems in all touch screen interfaces. It lead us to define a specific set of heuristics for evaluating the usability of ventilator systems.

<figure><img src="image_2.png" /><figcaption>Fig. 1. Touch screen ventilator system and the environment in the Intensive Care Unit</figcaption></figure>

## 2 Related work

Nielsen proposed 10 broad heuristics of interface design [11]. Also, Ben Shneiderman has described eight golden rules [13] that all good user interface designs should follow. Based on their work, Zhang et al. [14] selected a set of 14 heuristics called as Nielsen-Shneiderman heuristics for evaluation of patient safety of medical devices. They also conclude that such adaptation of heuristic evaluation for medical devices is very useful, efficient and cost effective for evaluating patient safety features.
We have come across several usability evaluations which are carried out using the Nielsen-Shneiderman heuristics. Some examples of these are briefly presented here. Graham et al. carried out heuristic evaluation of infusion pumps [7] using Nielsen-Shneiderman heuristics. The evaluation exercise carried out by 3-5 evaluators is reported to have captured 60-70% of the usability problems [14]. Edwards et al have applied Heuristic Walkthrough (HW) method to evaluate and improve the usability of the Electronic Health Record (EHR) system [4]. In another case study, the usability evaluation of Automatic External Defibrillators (AED) was conducted according to Nielsen-Shneiderman heuristics [2]. Diabetes tele-management system is also evaluated using Nielsen-Shneiderman heuristics [9]. For usability evaluation of this system, they have used 1-5 Likert scale and applied it uniformly to all heuristics.

## 2.1 Need for a specialized set of usability heuristics and indicators

As per our assessment, the interface design heuristics proposed by Nielsen and Shneiderman are meant for general-purpose software applications. Previous research by Nielsen and Molich has already shown that there is vast difference in the findings of usability evaluation by different evaluators [12]. Furthermore, these heuristics tend to miss out the unique nature of user interfaces of ventilator systems such as-

- Combination of touch screen interface and physical interfaces like touch buttons, knobs and LEDs

- Direct, precise and immediate communication and control (less scope for metaphoric representations)

- No scope for trial and error and exploratory approach to figure out the user interface

- Always used in time and life critical situations

- Fatal consequences in case of errors and delay

In case of ventilator systems, we need to specify the user interface components, a set of usability heuristics supported by objectively defined usability indicators [8] so that at least the major usability problems are not missed out during the evaluation. It is an imperative for medical usability because patient safety cannot be compromised and the consequences can be fatal. We have directly mapped the evaluation ratings with the usability indicators. We have attempted to reduce the vagueness and subjectivity in heuristic evaluation.

# 3 Methodology

## ■ Involvement of a physician

The usability experts have limited medical knowledge despite of putting sufficient effort in understanding the functionality and actual usage of a ventilator system. Therefore, it was an imperative step in our usability evaluation to involve a physician with the required medical expertise. The physician was to also help in sharing their expectations, priorities and experiences.

## ■ Ventilator systems
Three touch screen ventilator systems by different manufacturers were selected for usability evaluation. The names of manufacturers and equipment models of these ventilator systems are not disclosed to maintain confidentiality.

* **User interface and usage scenarios**

The usability experts developed adequate familiarity of the ventilator systems [6] with the help of the physicians and medical staff. It was very difficult to observe and evaluate the ventilator systems in the intensive care unit. Therefore, the physician was requested to perform the tasks while explaining the significance of use and this was video recorded for further observations. The video recording was helpful in noting the minute observations and the final usability evaluation. Ethical practices were observed while video recording the ventilator systems in intensive care units.

* **Usability heuristics and indicators**

Usability problems and design deficiencies commonly prevalent among all three ventilator systems were identified based on which the heuristics were formulated. The design priorities and medical priorities were fused together wherever applicable through deliberations between the design / usability experts and the physician. The user interface components and qualitative usability indicators [8] were identified to measure the compliance. Instead of applying the 1-5 Likart scale [9] uniformly across all parameters, we have chosen a indicator based evaluation method. Some heuristic indicators are checked in term of their absence or presence and some are elaborated in terms of their qualitative attributes. Each indicator is rated between 0 and 1.

* **Usability Evaluation**

The heuristic evaluation has been performed by four different usability evaluators to ensure the reliability of heuristics proposed in this paper.

# 4 Introduction to heuristics

## 4.1 Value input interface

<figure><img src="image_3.png" /><figcaption>Fig. 2. Value input interface of ventilator systems</figcaption></figure>
The input values for related parameters can be provided to the system through interface shown in Fig. 2A after selection of ventilator mode. The interface does not indicate valid range of values for parameters with proper upper and lower limits. It does not provide a selection of measuring units for corresponding parameters. Abbreviations are used for describing the parameters like PEEP or I/E and it does not visually represent those values. We found that some of the abbreviations and parameters were unfamiliar to the physicians. For alarm settings shown in 2B, units are not placed next to corresponding values. **Most of the time, the physicians have to input variety of values in the ventilator system and therefore the interface for inputting the values must be error free and user friendly.**

The list of heuristics for evaluating the touch screen interface for value inputting is elaborated in table 1.

<table><caption>**Table 1.** Heuristics for evaluating the touch screen interface for value inputting</caption><thead><tr><th colspan="3">Interface for input of values</th></tr></thead><tbody><tr><td>H1</td><td>Indicate valid range (maximum and minimum thresholds) of values for various parameters</td><td>Indicated (1)<br/>Not Indicated (0)</td></tr><tr><td>H2</td><td>Allow selection of units for measurement</td><td>Allowed (1)<br/>Not Allowed (0)</td></tr><tr><td>H3</td><td>Validate the inputs before acceptance</td><td>Validated (1)<br/>Not Validated (0)</td></tr><tr><td>H4</td><td>Confirm in case of proceeding with default values</td><td>Confirmed (1)<br/>Not Confirmed (0)</td></tr><tr><td>H5</td><td>Highlight the selected text input area and gray out the other text input areas</td><td>Highlights (1)<br/>Grays out (1)<br/>Does not highlight or gray out (0)</td></tr><tr><th colspan="3">Interface for controlling the value input</th></tr><tr><td>H6</td><td>Both on screen controls and physical knobs be provided for adjusting the values</td><td>Both Provided (1)<br/>One is provided (0)</td></tr><tr><td>H7</td><td>(Applicable in case of on screen interface)<br/>The input box and controls for adjusting the values to be co-located for every parameter</td><td>Co-located (1)<br/>Not co-located (0)</td></tr><tr><th colspan="3">Labeling of value input interface</th></tr><tr><td>H8</td><td>Use full form expressions for describing the parameters</td><td>Used (1)<br/>Not used (0)</td></tr><tr><td>H9</td><td>Use full form expressions for describing the units</td><td>Used (1)<br/>Not used (0)</td></tr><tr><td>H10</td><td>Units to be placed next to the value</td><td>Placed (1)<br/>Not placed (0)</td></tr><tr><th colspan="3">Visual Representation</th></tr><tr><td>H11</td><td>Form a visible group of related parameters</td><td>Common color (1)<br/>Boundary (1)<br/>Proximity (1)<br/>Scattered (0)</td></tr><tr><td>H12</td><td>Visually represent the values</td><td>Represented (1)<br/>Not Represented (0)</td></tr><tr><td>H13</td><td>Use unique colour code for quick identification and recall</td><td>Used (1)<br/>Not used (0)</td></tr></tbody></table>

**Culture Specific Preferences**
<table><tr><td>H14</td><td>Date format (dd/mm/yyyy or mm/dd/yyyy)</td><td>Given (1)<br/>Not given (0)</td></tr><tr><td>H15</td><td>Weight measurement unit (Pounds or Kilograms)</td><td>Given (1)<br/>Not given (0)</td></tr><tr><td>H16</td><td>Height measurement unit (Feet or Centimetres)</td><td>Given (1)<br/>Not given (0)</td></tr></table>

## 4.2 Interface for selection of option

Fig. 3A shows the screen for setting the patient configuration that provides two pairs of options namely “invasive or non-invasive” and “pediatric or adult”. From each of these pairs one option needs to be selected. But this expectation is represented in a very ambiguous manner. Fig. 3B provides options for selecting the ventilator modes like (A)CV or PSIMV. Such abbreviations are obscure and unclear for the medical staff. Proper understanding of options and their selection is important. The heuristics for evaluating the touch screen interface for selection of options are enlisted in table 2.

<figure><img src="image_4.png" /><figcaption>Fig. 3. Interface for selection of options</figcaption></figure>

<table><caption>**Table 2.** Heuristics for evaluating the touch screen interface for selection of options</caption><thead><tr><th colspan="2">Interface for structuring of options</th></tr></thead><tbody><tr><td>H17</td><td>Logical sequence of options</td><td>Logically arranged (1)<br/>Randomly arranged (0)</td></tr><tr><td>H18</td><td>Form visibly proximate groups of related options</td><td>Common color (1)<br/>Placed a boundary<br/>around the group (1)<br/>Closeness by distance (1)<br/>Scattered (0)</td></tr></tbody></table>

### Visual Representation of option
<table><tbody><tr><td>H19</td><td>Consistent sizes of buttons by following a grid</td><td>Consistent (1)<br>Inconsistent (0)<br>Grid followed (1)<br>Grid not followed (0)</td></tr><tr><td>H20</td><td>Use appropriate symbols or icons for related options</td><td>Used (1)<br>Not used (0)</td></tr><tr><td colspan="2"><strong>Description of option</strong></td><td></td></tr><tr><td>H21</td><td>Use full expressions for describing the options</td><td>Used (1)<br>Not used (0)</td></tr><tr><td>H22</td><td>Describe the screen / groups of options by precise title</td><td>Title given (1)<br>Title not given (0)</td></tr><tr><td>H23</td><td>Provide tool tips for explaining the options, their implication and the number of options one can select at a time</td><td>Tool tips provided (1)<br>Tool tips<br>not provided (0)</td></tr><tr><td>H24</td><td>Avoid all capital letters for normal text (Acronyms to be excluded)</td><td>Upper-lower case (1)<br>All capital letters (0)</td></tr><tr><td>H25</td><td>Legibility of text</td><td>Legible (1)<br>Not Legible (0)</td></tr><tr><td colspan="2"><strong>Feedback</strong></td><td></td></tr><tr><td>H26</td><td>Highlight the option(s) to indicate the selection</td><td>Highlighted (1)<br>Not highlighted (0)</td></tr><tr><td>H27</td><td>Ask for confirmation before accepting the inputs</td><td>Provided (1)<br>Not provided (0)</td></tr></tbody></table>

## 4.3 Interface for screen locking

<figure><img src="image_5.png" /><figcaption>Fig. 4. Interface for screen locking</figcaption></figure>

Screen locking feature can protect the settings from unintended changes. The touch screen ventilator systems without screen lock facility are prone to the danger of undesired changes in the settings. The touch screen lock is shown in fig. 4. It is obvious that the 'locking and unlocking' icons are not located in a consistent place. The heuristics for evaluating the screen locking interface are enlisted in table 3.
<table><caption>**Table 3.** Heuristics for screen locking interface</caption><thead><tr><th colspan="2">Availability of Lock</th></tr></thead><tbody><tr><td>H28</td><td>Screen locking / unlocking provision at any stage</td><td>Provided (1)<br/>Not provided (0)</td></tr><tr><td colspan="2"><strong>Location of Lock</strong></td></tr><tr><td>H29</td><td>Screen Lock / Unlock toggle buttons to be located in a unique place</td><td>Unique (1)<br/>Not unique (0)</td></tr><tr><td colspan="2"><strong>Visual Representation</strong></td></tr><tr><td>H30</td><td>Clearly visible and understandable iconic representation</td><td>Provided (1)<br/>Not provided (0)</td></tr><tr><td colspan="2"><strong>Functional Behavior</strong></td></tr><tr><td>H31</td><td>Store the settings selected till the stage of locking</td><td>Stores (1)<br/>Does not store (0)</td></tr><tr><td>H32</td><td>Resume from the stage of unlocking</td><td>Resume (1)<br/>Does not resume (0)</td></tr><tr><td colspan="2"><strong>Feedback</strong></td></tr><tr><td>H33</td><td>Visible feedback after locking and unlocking the screen</td><td>Provided (1)<br/>Not provided (0)</td></tr></tbody></table>

## 4.4 Data entry

<figure><img src="image_6.png" /><figcaption>**Fig. 5.** On-screen keyboard and patient record interface</figcaption></figure>

Touch screen ventilator systems require to provide an on-screen keyboard interface for data entry as shown in Fig. 5A. Fig. 5B shows patient record screen with numeric keyboard. The ventilator systems evaluated by us do not provide the facility to store and manage multiple patient records. Also the settings can be stored only once, if you change the settings and save then it overwrites the earlier. The heuristics for evaluating the on-screen keyboard interface and patient records are given in table 4.
<table><caption>**Table 4.** Heuristics for the on-screen keyboard interface and patient records</caption><thead><tr><th colspan="2">On-screen Keyboard</th></tr></thead><tbody><tr><td>H34</td><td>Onscreen keyboard for alphanumeric input</td><td>Provided (1)<br/>Not provided (0)</td></tr><tr><td>H35</td><td>Separate onscreen keyboard only for numeric input</td><td>Provided (1)<br/>Not provided (0)</td></tr><tr><td>H36</td><td>Provision for closing the onscreen keyboard whenever necessary</td><td>Provided (1)<br/>Not provided (0)</td></tr><tr><td>H37</td><td>Key size to be adequately large for fingure touch</td><td>Large (1)<br/>Not large enough (0)</td></tr><tr><td>H38</td><td>Adequate distance between keys so as to avoid wrong key-press</td><td>Adequate (1)<br/>Not adequate (0)</td></tr><tr><td>H39</td><td>QWERTY keyboard layout (with minimum necessary keys)</td><td>Provided (1)<br/>Not provided (0)</td></tr><tr><td>H40</td><td>Allow onscreen movement of keyboard</td><td>Provided (1)<br/>Not provided (0)</td></tr><tr><td colspan="2"><strong>Patient Data Input</strong></td></tr><tr><td>H41</td><td>Allow selection of data format for input</td><td>Provided (1)<br/>Not provided (0)</td></tr><tr><td>H42</td><td>Allow selection of units for measurement</td><td>Provided (1)<br/>Not provided (0)</td></tr><tr><td>H43</td><td>Check in case of proceeding with default values</td><td>Checks (1)<br/>Does not check (0)</td></tr><tr><td>H44</td><td>Validate the inputs before acceptance (e. g. admit date should not be prior to birth date)</td><td>Validates (1)<br/>Does not validate (0)</td></tr><tr><td colspan="2"><strong>Patient Record</strong></td></tr><tr><td>H45</td><td>Patient ID must be assigned to the record</td><td>Assigns (1)<br/>Does not assign (0)</td></tr><tr><td>H46</td><td>Rules may be applied while forming the ID</td><td>Rule-based (1)<br/>Random (0)</td></tr><tr><td>H47</td><td>Every patient record should have unique ID</td><td>Unique (1)<br/>Not unique (0)</td></tr><tr><td>H48</td><td>Storage of patient records</td><td>Stores (1)<br/>Does not store (0)</td></tr><tr><td>H49</td><td>Retrieval / deletion / updation of patient records</td><td>Supported (1)<br/>Not supported (0)</td></tr><tr><td>H50</td><td>Updation and Deletion of record by authorized users only</td><td>Supported (1)<br/>Not supported (0)</td></tr></tbody></table>

## 4.5 System feedback

A ventilator system has to be extremely communicative with the physicians and medical staff. Changes in the settings, internal processing, consequences of actions, warnings, error messages, status updates, alarms, etc have to be communicated from time to time. It is possible to design effective communication with the help of audio, visual, text and mobile messaging. The heuristics for evaluating the system feedback are given in table 5.
<figure><img src="image_7.png" /><figcaption>Fig. 6. Right-handed design of interface</figcaption></figure>

<table><caption>**Table 5.** Heuristics for evaluating the system feedback</caption><thead><tr><th></th><th></th><th></th></tr></thead><tbody><tr><td colspan="3"><strong>Feedback</strong></td></tr><tr><td>H51</td><td>Give feedback to communicate confirmation, status of progress, consequence of action, warnings and errors</td><td>Provided (1)<br/>Not provided (0)</td></tr><tr><td>H52</td><td>Effective use of visual communication</td><td>Appropriate use of signs &amp; symbols (1)<br/>Color code (1)<br/>Animation (1)<br/>Culture-specific depiction (1)<br/>No visual communication (0)</td></tr><tr><td colspan="3"><strong>Alarm</strong></td></tr><tr><td>H53</td><td>Provide alarms in audio, visual and mobile messaging forms</td><td>Audio (1)<br/>Visual (1)<br/>Mobile (1)<br/>None (0)</td></tr></tbody></table>

## 4.6 Neutral interface

Most medical equipments are designed only for right-handed users as seen in Fig. 6. It is clearly reflected in the layout of control panel, placement of knobs and buttons. Such design may not prove efficient for left-handed users. Therefore, neutrality for both left and right-handed users and ergonomic design are most desirable. The heuristics for evaluating neutrality of user interface are provided in table 6.
<table><caption>**Table 6.** Heuristic for evaluating neutrality of interface</caption><thead><tr><th colspan="2">Neutrality</th></tr></thead><tbody><tr><td>H54</td><td>Neutrality towards left and right-handed users</td></tr><tr><td></td><td>Neutral (1)</td></tr><tr><td></td><td>Left-handed design (0)</td></tr><tr><td></td><td>Right-handed design (0)</td></tr></tbody></table>

## 4.7 User manual / online help in local language

Mostly the user manuals are provided in English. Help is not provided as part of the software of ventilator system. The physicians are proficient in English but the assistive staff in the hospitals, which usually operate the ventilator systems are not familiar with English. Therefore, provision of user manuals and online help in English as well as local language is a must for reducing the possible medical errors.

<table><caption>**Table 7.** Heuristics pertaining to user manuals and online help</caption><thead><tr><th colspan="2">User manual and online help in local language</th></tr></thead><tbody><tr><td>H55</td><td>User manual in local language</td><td>Available (1)</td></tr><tr><td></td><td></td><td>Not available (0)</td></tr><tr><td>H56</td><td>Online help in local language</td><td>Available (1)</td></tr><tr><td></td><td></td><td>Not available (0)</td></tr></tbody></table>

# 5 Evaluation of ventilator systems

We have evaluated the usability of three different touch screen ventilator systems using the heuristics and usability indicators with following objectives.

i. Measure the usability and overall efficacy of touch screen ventilator systems

ii. Compare the quality of touch screen interfaces

iii. Study the reliability of the heuristics by involving three more usability evaluators to carry out the evaluation of same set of ventilator systems

This heuristic evaluation was carried out by totally four Usability Evaluators (UE). In this, UE1 are the authors of this paper who have formulated the heuristic guidelines. UE2, UE3, UE4 are other usability evaluators who used our heuristic evaluation method for evaluating the same set of ventilator systems.

We ensured that the usability evaluators had adequate understanding of Human Computer Interaction (HCI). They were sensitized about the proposed heuristics, criticality in the ICU environment and the usability evaluation of ventilator systems. Their queries about the heuristics and related evaluation were discussed and then they carried out the heuristic evaluation of all the three ventilator systems individually.

The total scores of usability evaluations by all four usability evaluators are consolidated in table 8.
<caption>**Table 8.** Heuristic evaluation of three Ventilator Systems (VS) by four different usability evaluators</caption>

<table><thead><tr><th rowspan="2">User Interface for</th><th rowspan="2">Max. Score</th><th rowspan="2">Usability Evaluators</th><th colspan="3">Scores of Touch Screen Ventilator Systems</th></tr><tr><th>VS-I</th><th>VS-II</th><th>VS-III</th></tr></thead><tbody><tr><td rowspan="4">1. Value Input</td><td rowspan="4">21</td><td>UE1</td><td>03</td><td>05</td><td>09</td></tr><tr><td>UE2</td><td>04</td><td>07</td><td>11</td></tr><tr><td>UE3</td><td>05</td><td>05</td><td>09</td></tr><tr><td>UE4</td><td>04</td><td>04</td><td>09</td></tr><tr><td rowspan="4">2. Options</td><td rowspan="4">14</td><td>UE1</td><td>07</td><td>08</td><td>11</td></tr><tr><td>UE2</td><td>06</td><td>11</td><td>9</td></tr><tr><td>UE3</td><td>05</td><td>8</td><td>10</td></tr><tr><td>UE4</td><td>06</td><td>8</td><td>11</td></tr><tr><td rowspan="4">3. Screen Lock</td><td rowspan="4">06</td><td>UE1</td><td>05</td><td>00</td><td>06</td></tr><tr><td>UE2</td><td>06</td><td>00</td><td>05</td></tr><tr><td>UE3</td><td>05</td><td>00</td><td>05</td></tr><tr><td>UE4</td><td>05</td><td>00</td><td>06</td></tr><tr><td rowspan="4">4. Data Entry</td><td rowspan="4">17</td><td>UE1</td><td>00</td><td>07</td><td>02</td></tr><tr><td>UE2</td><td>00</td><td>07</td><td>00</td></tr><tr><td>UE3</td><td>00</td><td>07</td><td>01</td></tr><tr><td>UE4</td><td>00</td><td>10</td><td>01</td></tr><tr><td rowspan="4">5. System Feedback</td><td rowspan="4">08</td><td>UE1</td><td>02</td><td>03</td><td>05</td></tr><tr><td>UE2</td><td>04</td><td>05</td><td>03</td></tr><tr><td>UE3</td><td>04</td><td>04</td><td>04</td></tr><tr><td>UE4</td><td>03</td><td>04</td><td>05</td></tr><tr><td rowspan="4">6. Neutrality</td><td rowspan="4">01</td><td>UE1</td><td>00</td><td>00</td><td>00</td></tr><tr><td>UE2</td><td>00</td><td>00</td><td>00</td></tr><tr><td>UE3</td><td>00</td><td>00</td><td>00</td></tr><tr><td>UE4</td><td>00</td><td>00</td><td>00</td></tr><tr><td rowspan="4">7. Help in local language</td><td rowspan="4">02</td><td>UE1</td><td>00</td><td>00</td><td>00</td></tr><tr><td>UE2</td><td>00</td><td>00</td><td>00</td></tr><tr><td>UE3</td><td>00</td><td>00</td><td>00</td></tr><tr><td>UE4</td><td>00</td><td>00</td><td>00</td></tr><tr><td rowspan="4">Total</td><td rowspan="4">69</td><td>UE1</td><td>17</td><td>23</td><td>33</td></tr><tr><td>UE2</td><td>20</td><td>30</td><td>28</td></tr><tr><td>UE3</td><td>19</td><td>24</td><td>29</td></tr><tr><td>UE4</td><td>18</td><td>26</td><td>32</td></tr></tbody></table>

<footnote>* UE1 are the authors of this paper who have formulated the heuristic guidelines.</footnote>
<figure><img src="image_8.png" /><figcaption>**Fig. 7.** UI component wise usability of all three ventilator systems as per the evaluation of UE1</figcaption></figure>

Considering that UE1 have formulated the usability heuristics and the indicators, their evaluation score is compared with the evaluations by other usability evaluators to find the closeness in their results.

# 6 Reliability of usability heuristics

<figure><img src="image_9.png" /><figcaption>**Fig. 8.** Comparison of usability evaluation of ventilator systems by four usability evaluators (UE1,2,3,4)</figcaption></figure>

The usability evaluation by other usability evaluators differs from UE1 by 11.77% for VS-I, 16.09% for VS-II and -10% for VS-III. The evaluation by UE2 is significantly different than the other usability evaluators because his interpretation of some heuristics (H18, H21, H27) and the importance given is slightly different than expected. On an average the evaluation of other usability evaluators has differed by 5.95% (addition of all % / 3) which is not very significant if compared with the results of Nielsen-Shneiderman heuristics in the context of medical devices [2,7,9,14].
## 7 Conclusion

Our observations of several ventilators systems available in Indian hospitals and the outcomes of heuristic evaluation show that the interface design of touch screen ventilator systems need significant design enhancements.

The specialized set of heuristics linked with user interface components and the objectively defined usability indicators are helpful in identifying specific usability problems of ventilator systems.

Heuristic evaluation in medical context cannot afford to be very subjective and open ended as in case of general-purpose software applications. It must identify specific usability problems in order to ensure patient safety and accuracy of treatment otherwise the consequences can be fatal.

The reliability of our approach in terms of reduced subjectivity and objective definition of UI components, heuristics and usability indicators specifically designed for ventilator systems is much higher.

## Future Work

The UI components and the corresponding heuristics logically seem to be applicable to variety of medical devices. However, which subset of heuristics is more relevant to which medical devices and their significance needs to be explored separately.

We propose to design the prototypes of user interface for a ventilator system which will comply with the heuristic guidelines. We would like to collaborate with the manufacturers of ventilator systems to design more usable interfaces.

## Acknowledgements

This work is supported in part by University of Pune, India under Grant Engg-57 (2008-09). We thank the university and institute authorities for providing a support in terms of this grant. We express our gratitude to physicians and ICU staff at Chintamani Hospital, Navale Hospital and Ranka Hospital at Pune, India for their support to this research work. We specially thank Dr. Shital Joshi, Dr. Shashank Joshi and Dr. Ajit Adangale for their cooperation and help. We also appreciate the help extended by Prof. Priyadarshan Dhabe and Mr. Mangesh Patil in video recording the operation of ventilator systems. We are thankful to Mr. Jalindar Karande, Pallavi Powale, Shrikant Salve and V. S. Subrahmanyam for their support in validation process. We also thank Dr. B. V. Barbadekar, Registrar, Prof. S. Y. Prabhu, Dean, Planning & Development and Prof. M. L. Dhore, Head, Department of Computer Engineering at VIT, Pune for their continuous encouragement to this research work.
References

1. Bhutkar G., Katre D., Rajhans N., Deshmukh S.: Scope of Ergonomic and Usability Issues with Intensive Care Unit (ICU): An Indian Perspective”. HFESA Journal - Ergonomics Australia. 22(1) March–June. 26-32 (2008)

2. Chapgar, A., Jessa, M., Cafazzo, J., Easty, T.: The Danger is in the Details: Human Factors Evaluation of Automatic External Defibrillators. Retrieved on July 3, 2009, from
http://stargate.uwaterloo.ca/~jzelek/teaching/syde361/ficcdat2007/Full%20Papers/C
MBEC30/M0357.pdf

3. Cooper, J., Newbower, R., Long, C., McPeek, B.: Preventable Anesthesia Mishaps: A Study of Human Factors. Quality and Safety in Health Care. 11(3), 277–282. (2002)

4. Edwards, P., Moloney, K., Jacko, J., Sainfort, F.: Evaluating Usability of a Commercial Electronic Health Record: A Case Study. Int. J. Human-Computer Studies. 66, 718-728 (2008)

5. Garmer. K., Ylven. J., Karlsson. IC. M.: User Participation in Requirements Elicitation Comparing Focus Group Interviews and Usability Tests for Eliciting Usability Requirements for a Medical Equipment: A Case Study. Int. J. of Ind. Ergonomics. 33 85-98 (2004)

6. Gould T., de Beer J.M. A.: Principles of Artificial Ventilation. Anesthesia & Intensive Care Medicine. 8(3) 91-101 (2007)

7. Graham, M., Kubose, T., Jordan, D., Zhang, J., Johnson, T., Patel, V.: Heuristic Evaluation of Infusion Pumps: Implications for Patient Safety in Intensive Care Units. Int. J. of Med. Informatics. 73, 771-779 (2004)

8. Katre D. S.: Visualization of Interface Metaphor for Software: Engineering Approach. Ph.D. Dissertation. (2005)

9. Lopez, R., Chagpar, A., White, R., McLean, M., Trudel, M., Cafazzo, J., Logan, A.: Usability of a Diabetes Telemanagement System. Proceedings of 30th Annual Canadian Medical and Biological Engineering Conference. June (2007)

10. Martin. J., Norris. B., Murphy, E., Crowe. J.: Medical Device Development: The Challenge for Ergonomics. Applied Ergonomics. 39 271-283 (2008)

11. Nielsen, J.: Usability Engineering. Boston. AP Professional. (1994)

12. Nielsen, J. & Molich, R. (1990). Heuristic evaluation of user interfaces. Proceedings of the SIGCHI conference on human factors in computing systems: Empowering people. Seattle, WA, USA. April, 1990. 249-256.

13. Shneiderman, B.: Designing the User Interface. Pearson Education. Sixth Indian Reprint. (2005).

14. Zhang, J., Johnson, T., Patel, V., Paige, D., Kubose, T.: Using Usability Heuristics to evaluate Patient Safety of Medical Devices. J. of Biomed. Informatics. 36, 23-30 (2003)