Incorporating Usability into the Database Review Process: New Lessons and Possibilities

In summer 2013, Purdue Libraries introduced a modified standard usability concept (heuristic evaluation, or expert review) into an existing yearly electronic resource evaluation process. Introducing more user experience parameters into the process allows librarians to record usability errors to be communicated back to database vendors or to be considered for database renewal and selection in the future at the Libraries. In total, 37 databases were reviewed by eight librarians. This proceeding will review the reported impact the internal process made on librarians’ database decisions. Heuristic Reviews and Purdue University Libraries Yearly Database Review Heuristic reviews, or expert reviews, are a user experience method commonly used to evaluate existing interfaces. While usability tests have been employed throughout libraries, heuristic reviews have not been formally used in libraries’ collection development practices. My aim was to find out answers to the following two questions through a satisfaction survey: 1. Does the incorporation of database heuristic reviews affect the database review process? 2. Is it redundant with other parts of the database evaluation process? Is it perceived as valuable? Every year, the Purdue University Libraries evaluates one-third of their library databases using a form called the “database review.” Roughly 100 databases are reviewed each year by 20 librarians. They evaluate the databases’ intended purpose, usage, audience content, marketing, and cost per use. While the form did have questions that touched upon good usability, the process did not include any explicit guidance on what did or did not make a database a good product. To test whether the introduction of a heuristic review would inform a librarian’s database review, I created the Database Usability Heuristic Form (see Appendix). User Satisfaction Survey Results Overall, satisfaction with the new process was low in this group of librarians. Of the eight librarians surveyed, seven of them said that the database heuristic review had no impact on their final decision. All the librarians found the Database Usability Heuristic Review easy to use, but many had specific problems with structure, content, and organization of the form. One-half of the librarians surveyed (n = 4) felt like the new form was redundant with the existing form, but were not in agreement about which part of the Database Review with which it was redundant. One librarian said that the new process “asked different questions” and was therefore valuable. Another thought the questions were worth asking but perhaps could be asked more efficiently and effectively. When asked if the library should continue the form next year, 75% (n = 6) said no. One librarian commented, “It would be useful in the case of a database that is really difficult to use or inappropriate for the intended audience.” Another viewed the acquisition period as a better time to evaluate a product’s usability. Thirty-seven database heuristic reviews were completed. They included many interesting insights with recorded usability errors which may not have affected librarians’ final decisions, but were nevertheless recorded for posterity for the use of the libraries. The heuristic reviews will be analyzed more thoroughly in a later paper.