PubMed on Tap: Discovering Design Principles for Online Information
Delivery to Handheld Computers

**Susan E. Hauser, Dina Demner-Fushman, Glenn Ford, George R. Thoma**

*Communications Engineering Branch, Lister Hill National Center for
Biomedical Communications, National Library of Medicine, National
Institutes of Health, Department of Health and Human Services, Bethesda,
Maryland, USA*

**Abstract**

*Online access to biomedical information from handheld computers will be
a valuable adjunct to other popular medical applications if information
delivery systems are designed with handheld computers in mind. The goal
of this project is to discover design principles to facilitate
practitioners' access to online medical information at the
point-of-care. A prototype system was developed to serve as a testbed
for this research. Using the testbed, an initial evaluation has yielded
several user interface design principles. Continued research is expected
to discover additional user interface design principles as well as
guidelines for results organization and system performance.*

***Keywords:***

Handheld Computer; Computer-Assisted Decision Making.

**Introduction**

The practice of evidence based medicine (EBM) has grown dramatically
since the term was introduced in the early 1990's \[1\]. EBM involves
integrating individual clinical expertise with the best available
evidence from systematic research \[2\]. One reason for the growing
interest in EBM is the creation of information systems that permit the
near real-time delivery of clinically relevant information \[3\].

During the same time period, handheld computers, or personal digital
assistants (PDAs), were introduced and have become increasingly popular
for a variety of medical applications \[4\]. Given the desire for access
to current, high-quality information at the point of care and the
convenience of PDAs, it is not surprising that mobile health care
providers see PDAs as a valuable tool for "providing information nuggets
in a just-in-time manner" \[5\].

Current sources of best evidence are electronic, with access via an
internet connection and a web browser \[6\]. However, web pages designed
for display on a desktop computer screen seldom render well on the small
screen of a PDA \[7\]. Providing the same power and flexibility to PDAs
as to desktop-based clients will require system designers to look beyond
the now-traditional model for desk-top computer, browser-based
information access. The goal of the PubMed on Tap[^1] project is to
discover design principles to facilitate practitioners' access to
medical information at the point-of-care, namely those related to user
interface design, results organization and system performance.

**Testbed Design**

A prototype testbed system, PubMed on Tap, was built and tested to
explore user interface design, results organization, and system
performance. The system is designed to access MEDLINE^®^, the National
Library of Medicine's (NLM's) premier database of indexed citations from
biomedical journals through PubMed, NLM's interface to the database. A
system requirement was the need to use PubMed's search and retrieval
capabilities, while at the same time controlling the interface to PubMed
as seen by the PubMed on Tap user. This is accomplished through PubMed's
Entrez Programming Utilities (Esearch, Efetch and Elink \[8\] ). These
tools provide access to Entrez data outside of the regular web query
interface and support search and retrieval from other environments.
Other system design goals include: (a) timely transmission of
information to numerous concurrent users; (b) performance monitoring;
(c) utilization of aggregate usage statistics to improve performance;
(d) the ability to facilitate precision through support of user-specific
preferences (input) and through clustering and/or ranking of citations
(output).

Three designs were considered that would permit using the Utilities for
search and retrieval, while managing the user interface at the handheld
computer. Briefly, they are: (a) develop client software for the PDA to
access PubMed directly; (b) develop a proxy server as an intermediate
website for PDAs, using templates to format html pages for small
screens; (c) develop client software for the PDA to communicate with a
proxy server that, in turn, communicates with PubMed via the Utilities.
Table 1 lists some of the advantages and disadvantages of each design.

The client plus proxy server was selected as the best overall design for
the testbed. The client program permits developer control of the user
interface while the processing power of the proxy server supports
experimenting with methods to facilitate precision and improve response.
Together these allow for flexibility in those aspects of information
delivery that are being researched. An overview of the system is shown
in Figure 1.

*Table 1 - Pros and Cons of three prototype designs*

+----------------+--------------------------+--------------------------+
| **Design**     | **Pros**                 | **Cons**                 |
+----------------+--------------------------+--------------------------+
| **Client       | Control of user          | Not platform independent |
| Only**         | interface                |                          |
|                |                          | User must install new    |
|                | Able to store            | version when client      |
|                | user-specific data       | changes                  |
|                | locally                  |                          |
|                |                          | Client must do all data  |
|                |                          | processing               |
|                |                          |                          |
|                |                          | No opportunity to        |
|                |                          | monitor performance or   |
|                |                          | gather aggregate usage   |
|                |                          | statistics               |
+----------------+--------------------------+--------------------------+
| **Proxy Web    | Platform independent     | Little control of user   |
| Server**       |                          | interface                |
|                | Changes for new versions |                          |
|                | implemented at the       | Must store user-specific |
|                | server only              | data at server,          |
|                |                          | requiring user           |
|                | Can perform data         | identification           |
|                | processing at server     |                          |
|                |                          | Html file \~4 times the  |
|                | Can monitor performance  | size of equivalent XML   |
|                | and gather aggregate     | data                     |
|                | usage statistics         |                          |
|                |                          |                          |
|                | Can store queries,       |                          |
|                | results, citations at    |                          |
|                | server to improve        |                          |
|                | performance              |                          |
+----------------+--------------------------+--------------------------+
| **Client plus  | Control of user          | Not platform independent |
| Proxy Server** | interface                |                          |
|                |                          | User must install new    |
|                | Able to store            | version when client      |
|                | user-specific data       | changes                  |
|                | locally                  |                          |
|                |                          |                          |
|                | Can perform data         |                          |
|                | processing at server     |                          |
|                |                          |                          |
|                | Can monitor performance  |                          |
|                | and gather aggregate     |                          |
|                | usage statistics         |                          |
|                |                          |                          |
|                | Can store queries,       |                          |
|                | results, citations at    |                          |
|                | server to improve        |                          |
|                | performance              |                          |
|                |                          |                          |
|                | Only need to transmit    |                          |
|                | data to be displayed     |                          |
|                | (XML data)               |                          |
+----------------+--------------------------+--------------------------+

![](media/image1.png){width="1.351968503937008in"
height="1.8956692913385826in"}![](media/image2.wmf){width="0.9854330708661417in"
height="0.6818897637795276in"}![](media/image3.wmf){width="1.3692913385826773in"
height="1.3744094488188976in"}*Figure 1 - PubMed on Tap system
components*

![](media/image4.png){width="1.6666666666666667in"
height="1.6666666666666667in"}The client in the selected approach is not
platform independent; it was designed for PDAs running the Palm
operating system, since the literature reports these to be the most
popular among physicians who own PDAs \[9\]. The client program is
written in C/C++ using the Code Warrior development environment. It uses
the PDA's wireless communication interface and the http protocol to
communicate via the internet with a servlet on the proxy server. The
servlet in the proxy server, written in Java, runs in a Tomcat servlet
container on an Apache http server in a UNIX environment.

![](media/image5.png){width="1.6666666666666667in"
height="1.6666666666666667in"}To present a familiar-looking interface to
new users, the design of the client user interface is modeled in part on
ePocrates, the drug database reported to be the most popular application
for PDAs among healthcare professionals \[10\]. Following this model,
functions are organized by tabs and drop down lists are used where
possible to save space. Although space is at a premium on the small
screen, readability trumps space utilization, so whitespace is used to
separate distinct items on the screen. For navigation, small icons are
used in place of larger, text-identified buttons. Search limits are
easily selected from the Profiles tab and are stored locally. Previous
queries are also stored locally and are available for reuse. Figures 2
through 5 show four of the PubMed on Tap screens.

*Figure 2 -- The Search tab*

*Figure 3 -- The Results tab*

![](media/image6.png){width="1.6666666666666667in"
height="1.6666666666666667in"}

*Figure 4 -- The History tab*

![](media/image7.png){width="1.6666666666666667in"
height="1.6666666666666667in"}

*Figure 5 -- The Profile tab*

**Evaluation**

In July 2003, the prototype system was tested in the National Cancer
Institute's Usability Lab with nine volunteers from a variety of
backgrounds. Details of this study are reported in Alexander \[11\].
Insights into a few design principles are noted here:

\(a\) *Retain the icon functions of the target application.* The desktop
PubMed environment uses various icons of pages to indicate "No
Abstract", "Abstract", "Free Full Text Article" and "Article in PubMed
Central". PubMed on Tap used a blank page icon as a link from the
article citation to "Related Articles." Our volunteers, who were
familiar with PubMed, found this confusing.

(b*) Use visual prompts.* Users need a visual clue that tapping at some
location will yield results. Even though our volunteers "knew" that
tapping the abbreviated summary text would display the abstract, they
tended to forget that because there was no visual prompt. Likewise,
because of the widespread familiarity with WWW conventions, users
expected there to be a visual clue that a given abstract has already
been "visited" once it had been viewed.

\(c\) *Focus text entry.* When text is to be entered, users may benefit
by having the text field automatically focused with a blinking cursor.
PubMed on Tap includes a separate screen to select journal titles to
which searches can be limited. There is a "Find" text field at the
bottom of the screen in which users can enter the first few characters
of the journal name to quickly jump to it. Although our volunteers
understood how to use that feature once they saw it, several failed to
notice that the field was present without being prompted.

\(d\) *Avoid complexity* \[12\]. The results screen supports two
navigational paths for obtaining "more information" for an article of
potential interest: one leading to MeSH terms and journal Subject
Headings, and one leading to the abstract. Our volunteers were
accustomed to going from the title to the abstract. Having an additional
choice was confusing rather than helpful.

Following the test session, each volunteer completed an on-line
evaluation form consisting of 29 multiple choice questions. A
quantitative assessment of the user evaluation is reported in Alexander
\[11\]. Overall, the reactions to PubMed on Tap were favorable. Most
participants found it fast, easy to use, easy to read, and said they
would probably use it at work if they could. Features found to be "very
useful" were the ability to limit the search to citations with
abstracts, and to limit the search by date of publication. Opinion was
divided about the ability to display results in "Brief" mode; most
participants thought it was "slightly" or "moderately" useful, but two
thought it was "extremely" useful. Regarding additional features, all
but one participant said that the ability to save selected citations
would be "extremely" useful. Many also thought that clustering citations
by MeSH terms or other criteria would be "very useful".

**Ongoing research**

Based upon these observations, the next version of PubMed on Tap is
being developed to incorporate changes that would yield the greatest
improvement for modest effort. When the new version of the application
is developed and tested, it will be released for beta testing. We also
plan to exhibit the system at conventions of health care practitioners
to elicit feedback directly from our target users.

Initial research has been largely focused on user interface design.
Future research will place additional emphasis on results organization
and system performance.

*Results organization*: This research explores ways to organize the
search results by summarizing, clustering and ranking the resulting
citations. Initial work involves categorizing citations returned in
response to a query, creating multi-document summaries for clusters of
highly related documents, and single-document descriptions containing
features specific only to a given document in the cluster \[13\]. The
research includes developing an appropriate user interface for display
and selection of results categories.

*System performance*: A system design goal is to enable simultaneous use
by hundreds of users with no reduction in reliability or speed.
Therefore, research in this area is focused toward discovering design
factors that ensure the speed and reliability of the hardware and
software required for accurate and timely retrieval of data. Retrieval
time from PubMed is a critical component of PubMed on Tap performance
that is beyond the control of our system. Thus our goal is to minimize
the contribution of the proxy server to the overall retrieval time.
Areas of investigation include choice of parsers, efficient use of a
database to store recent queries and citations, and load testing.

In the longer term, other issues will be addressed: e.g., achieving
platform independence or alternatively, developing multiple versions of
the client; retrieving and displaying full text articles; exploiting
Hotsync to support additional capabilities such as automatically
updating the client software with a new version; linking to an
Electronic Medical Record to use patient data to inform the search.

**References**

1.  Levin A. Evidence-based medicine gaining supporters. Ann Intern Med
    1998 Feb 15. Available from:
    [http://www.acponline.org/shell-cgi/printhappy.pl/journals/annals/15feb98/currebm.htm](http://www.acponline.org/shell-cgo/printhappy.pl/journals/annals/15feb98/currebm.htm)

2.  Sackett DL, Rosenberg WMC, Gray JAM, Haynes RB, Richardson WS.
    Evidence based medicine: what it is and what it isn't \[editorial\].
    BMJ 1996;312:71-72. Available from:
    <http://bmj.com/cgi/content/full/312/7023/71>

3.  Centre for Evidence-Based Medicine. Why the sudden interest in
    EBM? 2000. Available from:
    <http://www.cebm.utoronto.ca/intro/interest.htm>

4.  Fischer S, Stewart TE, Mehta S, Wax R, Lapinsky SE. Handheld
    computing in medicine. JAMIA 2003 Mar/Apr;10(2):139-49.

5.  Greiver M. Hand-held brain extenders \[letter\]. CMAJ 2001 Aug
    7;165(3):273.

6.  Hunt DL, Jaeschke R, McKibbon KA. Users' guide to the medical
    literature. XXI: Using electronic health information resources in
    evidence-based practice. JAMA 2000 Apr 12;283(14):1875-9.

7.  Al-Ubaydli MA. Principles for designing your hospital
    intranet. 2002. Available from:
    <http://www.handheldsfordoctors.com/learn/wireless/principles.htm>

8.  Entrez Programming Utilities. Dec. 2003. Available from:
    http://www.ncbi.nlm.nih.gov/entrez/query/static/eutils_help.html

9.  Criswell DF, Parchman ML. Handheld computer use in U.S. family
    practice residency programs. JAMIA 2002;9:80-6.

10. TNS Intersearch. ePocrates usage study. 2003. Available from:
    https://image.epocrates.com/headlines/story/10133.html

11. Alexander G, Hauser S, Steely K, Ford G, Demner-Fushman D. A
    usability study of the PubMed on Tap user interface for PDAs. To be
    published: 11^th^ World Congress on Medical Informatics; 2004 Sep
    7-11; San Francisco CA, USA.

12. Fleck M, Frid M, Kindberg T, O'Brian-Strain E, Rajani R,
    Spasojevic M. Ubiquitous systems in interactive museums. Pervasive
    Computing 2002 Apr/Jun;:13-21.

13. Demner-Fushman D, Hauser SE, Ford G, Thoma GR. Organizing literature
    information for clinical decision support. To be published: 11^th^
    World Congress on Medical Informatics; 2004 Sep 7-11; San Francisco
    CA, USA.

**Address for correspondence**

Susan E. Hauser, Ph.D., P.E., Lister Hill National Center for Biomedical
Communications, National Library of Medicine, Bethesda, Maryland 20894,
USA. E-mail: hauser@nlm.nih.gov

[^1]: The PubMed on Tap application is available at
    http://archive.nlm.nih.gov/proj/pmot/pmot.php.