TopicShop: enhanced support for evaluating and organizing collections of Web sites

TopicShop is an interface that helps users evaluate and organize collections of web sites. The main interface components are site profiles, which contain information that helps users select high-quality items, and a work area, which offers thumbnail images, annotation, and lightweight grouping techniques to help users organize selected sites. The two components are linked to allow task integration. Previous work  demonstrated that subjects who used TopicShop were able to select significantly more highquality sites, in less time and with less effort. We report here on studies that confirm and extend these results. We also show that TopicShop subjects spent just half the time organizing sites, yet still created more groups and more annotations, and agreed more in how they grouped sites. Finally, TopicShop subjects tightly integrated the tasks of evaluating and organizing sites.  In previous work , we motivated an important task for web users – gathering, evaluating, and organizing information resources for a given topic. Current web tools do not support this task well; specifically, they do not make it easy to evaluate collections of web sites to select the best ones or to organize sites for future reuse and sharing. Users have to browse and view sites one after another until they are satisfied they have a good set, or, more likely, they get tired and give up. Browsing a web site is an expensive operation, both in time and cognitive effort. And bookmarks, the most common form of keeping track of web sites, are a fairly primitive organizational technique. We designed and implemented the TopicShop system to provide comprehensive, integrated support for this task. TopicShop aids users in finding a set of relevant sites, in narrowing down the set into a smaller set of high quality sites, and in organizing sites for future use. TopicShop has evolved through a number of design iterations, driven by extensive user testing. We report here on lessons we learned from a pilot study, how these lessons re-shaped our understanding of the task and led to a significant re-design, and the results of a second, larger user study. RELATED WORK Our research program investigates the major information problems faced by users of the World Wide Web: • finding collections of items relevant to their interests; • identifying high-quality items within a collection; • finding items that contain a certain category of information, e.g., episode guides (for a television show) or song lyrics (for a musician); • organizing personalized subsets of items. We have addressed these problems by developing algorithms, implementing them in web crawling and analysis tools, creating interfaces to support users in exploring, comprehending, and organizing collections of web documents, and performing user studies [2, 3, 4, 15]. The work reported here focuses on understanding the user tasks of evaluating and organizing collections of web sites, as ill uminated by the design, evaluation, and re-design of interfaces to support these tasks. Other researchers have investigated these issues. Much recent work has been devoted to algorithms for adding meta-information to collections of web sites to enhance user comprehension, typically by analyzing the structure of links between sites. This approach builds on the intuition that when the author of one site chooses to link to another, this often implies both that the sites have similar content and that the author is endorsing the content of the linked-to site. Pirolli , Pitkow and colleagues [12, 13] experimented with link-based algorithms for clustering and categorizing web pages. Kleinberg’s HITS algorithm  defines authoritative and hub pages within a hypertext collection. Authorities and hubs are mutually dependent: a good authority is a page that is linked to by many hubs, and a good hub is one that links to many authorities. After evaluating items and selecting the interesting ones, users must organize the items for future use. Card, Robertson, and Mackinlay  introduced the concept of information workspaces to refer to environments in which information items can be stored and manipulated. A departure point for most such systems is the file manager popularized by the Apple Macintosh and then in Microsoft Windows. Such systems typically include a list view, which shows various properties of items, and an icon view, which lets users organize icons representing the items in a 2D space. Mander, Salomon, and Wong [10] enhanced the basic metaphor with the addition of “piles” . Users could create and manipulate piles of items. Interesting interaction techniques for displaying, browsing, and searching piles were designed and tested. Bookmarks are the most popular way to create personal information workspaces of web resources. Bookmarks consist of lists of URLs; typically the title of the web page is used as the label for the URL. Users may organize their bookmarks into a hierarchical category structure. Abrams, Baecker, and Chignell  carried out an extensive study of how several hundred web users used bookmarks. They observed a number of strategies for organizing bookmarks, including a flat ordered list, a single level of folders, and hierarchical folders. They also made four design recommendations to help users manage their bookmarks more effectively. First, bookmarks must be easy to organize, e.g., via automatic sorting techniques. Second, visualization techniques are necessary to provide comprehensive overviews of large sets of bookmarks. Third, rich representations of sites are required; many users noted that site titles are not accurate descriptors of site content. Finally, tools for managing bookmarks must be well i ntegrated with web browsers. Many researchers have created experimental information workspace interfaces, often designed expressly for web documents. Card, Robertson, and York  describe the WebBook, which uses a book metaphor to group a collection of related web pages for viewing and interaction, and the WebForager, an interface that lets users view and manage multiple WebBooks. Mackinlay, Rao, and Card  developed a novel user interface for accessing articles from a citation database. The central UI object is a “Butterfly” , which represents an article, its references, and its citers. The interface makes it easy for users to browse among related articles, group articles, and generate queries to retrieve articles that stand in a particular relationship to the current article. The Data Mountain of Robertson et al  represents documents as thumbnail images in a 3D virtual space. Users can move and group the images freely, with various interesting visual and audio cues used to help users arrange the documents. In a study comparing the use of Data Mountain to Internet Explorer Favorites, Data Mountain users retrieved items more quickly, with fewer incorrect or failed retrievals. Our research shares goals with much of the previous work. We focus on designing interfaces that make automatically extracted information about web sites readily accessible to users. We show that this increases users’ abilit y to select high-quality sites. Through ongoing user studies and redesign, we developed easy-to-use annotation and grouping techniques that let users organize items better and more quickly. Finally, we learned how users interleave work on various tasks and re-designed our interface to support such task interleaving.  The TopicShop Explorer is implemented in C++ and runs on Microsoft Windows platforms. Version 1 was based directly on the Macintosh file manager / MS Windows Explorer metaphor (see  for detail of TopicShop Version 1 and the pilot study). Accordingly, users could view collections in either a details  or icons  view. The details view showed site profile information (see below) to help users evaluate sites, and the icons view let users organize sites spatially.  TopicShop Explorer (version 1), details view. Each web site is represented by a small thumbnail image, the site title, and profile data including the links to/from other sites in the collection, and the number of pages, images, and audio files on the site. Users can sort by a property by clicking on the appropriate column.  TopicShop Explorer (version 1), icons view. Each site is represented by a large thumbnail image and the site title. Users can organize sites by arranging them spatially, a technique especially useful in the early stages of exploration. The collections of sites and site profile data used in TopicShop are obtained by running a webcrawler/analyzer. The crawler takes a user-specified set of seed sites as input, and follows links from the seeds to construct a graph of the seed sites, pages contained on these sites, and, optionally, sites determined to be related based on their textual and hyperlink connections to the seeds. Site profiles are built by fetching a large number of pages from each site. Profiles contain content data, including the page title, an estimate of the page count, and a roster of audio files, movie files, and images; they also record links between sites in the collection. In addition, a thumbnail image of each site’s root page is constructed. The first goal of TopicShop is to help users evaluate and identify high quality sites. We sought to achieve this goal by providing site profile data and interface mechanisms for viewing and exploring the data. Making this data visible means that users no longer had to select sites to browse based solely on titles and (sometimes) brief textual annotations. (A chief complaint of subjects in the Abrams et al  study was that titles were inadequate descriptors of site content — and that was for sites that users already had browsed and decided to bookmark.) Instead, users may visit only sites that have been endorsed (linked to) by many other sites or sites that are rich in