USABILITY EVALUATION OF DIGITAL LIBRARIES: OFFLINE AND ONLINE ASSESSMENT COMPARED

Usability is of special importance for every online product. Also a modern digital library or Library 2.0 [1] needs to assure an easy handling and the usability of a library’s website with the integrated online services have to be continuously monitored. An efficient way for usability assessment is the use of short standardized scales either offline (paper pencil, face to face) or online (e.g., by an Internet survey). Offline and online assessment have their special advantages and drawbacks. Thereby, the differences between offline versus online assessment might cause different data even though using the same scales (so-called mode effects). Thus, it has to be assured that the offline measurement delivers equal results like the analogous online measurement. In this contribution I present an empirical investigation on potential mode effects for a questionnaire for the evaluation of a Library 2.0. The concrete use case of a Library 2.0 was the ZBW (www.zbw.eu), the world’s largest information center for economic literature. The questionnaire was applied in an offline versus an online version. Both versions regard to the evaluation of the homepage of the ZBW as well as the three core services of the ZBW (EconBiz for literature search, EconDesk for online help, and EconStor, a publishing portal). As measurement instrument I used the S ystem U sability S cale (SUS, see [2]) as a short standardized usability scale for each, the homepage as well as the three services. In addition to the usability evaluation, for the service EconBiz also a quality evaluation by the standardized short S cale on the Qu ality of L iterature L ists (SQuaLL, see [3]) was implemented. The SUS and the SQuaLL were applied for the offline version and the online version analogously, i.e., with the same wording and the same answering options. This enables a direct comparison of the quantitative data (offline versus online) assessed by the SUS and the SQuaLL. Additionally, both versions comprised the possibility to give open comments. Besides these common elements there were also some differences between the two versions: The offline version was a rather long version because it included also additional questions for heuristic insights and more detailed feedback about the ZBW. The online version was a short condensed version of the questionnaire that comprised the main standardized short scales (SUS, SQuaLL), only. The offline version was filled out by paper-pencil in the presence of an instructor (face-to-face setting). The online survey was announced via mailing lists, via the homepage of the ZBW and via the Facebook-page of the ZBW. The results showed that there were no significant differences between the offline versus online evaluation by means of the SUS and the SQuaLL. However, most online participants ignored the possibility to give open comments whereas the offline participants gave rather exhaustive annotations. Overall, the findings suggest that for a pure evaluation control check a short online survey is sufficient and leads to equal results like a long, more controlled offline questionnaire. However, if deeper insights are needed, an offline evaluation seems to be more appropriate. The theoretical and practical implications of the findings will be discussed.