SCALE ON THE QUALITY OF LITERATURE LISTS (SQUALL): QUICK QUALITY CHECK FROM THE USER PERSPECTIVE

Nowadays, a modern digital library or Library 2.0 [1] usually offers an online portal for literature search. Thereby, it is of special importance to assure a good usability as well as an obvious excellent quality of the search portal. For the assessment of usability, several standardized scales exist, e.g., the S ystem U sability S cale (SUS, see [2]) that offers an economic way for a global usability check. However, the assessment of the quality of the listed results of the literature search is more complicated, because the different professional search portals offer very special and partly unique features. The related assessment instruments are often very detailed and focus more on systems of the digital library in general as on the literature search portals in specific. Beyond this background, this paper describes the newly developed S cale on the Qua lity of L iterature L ists (SQuaLL). The SQuaLL offers a quick and easy way to assess the quality of a literature search portal from the view of the user. The SQuaLL was originally developed with respect to the literature search portal EconBiz (www.econbiz.de ), but can be easily modified for the evaluation of other academic literature search services. The sum-score of the SQuaLL is conceptualized as a general quality index that can be used as an additional supplement to a usability evaluation. The general quality index is defined as a composition of ten selected quality criteria. The identified quality criteria are based on an expert query. The format of the scale is constructed analogous to the SUS. It comprises ten items that regard to ten characteristics that reflect the core quality criteria of academic search portals: Five of these items assess the “must haves”, i.e., the basic characteristics that every literature search portal should offer (e.g., a reasonable ranking and filtering). Another four items regard to the “nice to have”, i.e., features that are part of most modern search portals because they are of special interest for the user (e.g., availability of full text download). One item refers to a rather special feature of EconBiz, i.e., the event search. The wording of this special item can be easily modified with respect to different specifics of other search portals. All ten items are formulated in form of statements that have to be rated on a 5-point Likert scale. The possible sum-scores of the SQuaLL are between 0 (very poor quality) and 100 (very high quality). The practical use and the paralleltest reliability of the SQuaLL were already proven for a controlled offline sample. For the analysis of the paralleltest reliability two forms (A and B) of the SQuaLL were established. The correlation between form A and form B of r = . 777 (p < .001) indicates a good paralleltest reliability of the sum-scores. Additionally, also an online version of the SQuaLL was successfully applied. To sum up, the SQuaLL offers a global quality index of literature search portals from the user perspective. The items of the scale can be easily modified for the evaluation of different academic search portals of multiple scientific domains. The practical and theoretical implications will be discussed.