Data linking for the Semantic Web

The effective acquisition of (semantic) metadata is crucial for many present day applications. Games with a purpose address this issue by transforming computational problems into computer games. The authors present a novel approach to metadata acquisition via Little Search Game (LSG) – a competitive web search game, whose purpose is the creation of a term relationship network. From a player perspective, the goal is to reduce the number of search results returned for a given search term by adding negative search terms to a query. The authors describe specific aspects of the game’s design, including player motivation and anti-cheating issues. The authors have performed a series of experiments with Little Search Game, acquired real-world player input, gathered qualitative feedback from the players, constructed and evaluated term relationship network from the game logs and examined the types of created relationships. DOI: 10.4018/jswis.2011070102 24 International Journal on Semantic Web and Information Systems, 7(3), 23-45, July-September 2011 Copyright © 2011, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited. and navigation in information spaces (Stewart, Scott, & Zelevinsky, 2008). Rather than use of semantics, more problematic is their acquisition. While automated approaches are able to provide quantity, in comparison with human oriented approaches (expert work, crowdsourcing) they vary in the quality of semantics they provide. The concept of games with a purpose (GWAP) emerged within the human computing initiative in recent years and stresses the use of human problemsolving capabilities via specially engineered games to address so called human intelligence problems (HITs) that are currently too difficult to solve by machine approaches (e.g., image annotation) (Siorpaes & Hepp, 2008a). By transforming computational problems into engaging computer games, GWAPs enable us to take advantage of human computing power without having to pay for expensive human resources while also providing the scalability to web-scale tasks. Several successful games like the image annotation ESP Game (von Ahn & Dabbish, 2008) have already shown the potential of this approach, especially for creation of web semantics (Siorpaes & Hepp, 2008a). However, the creation of GWAPs is not a straightforward process, because it is specific to each human intelligence problem that it tries to solve. Although some effort has been spent on developing generic methodologies for GWAP creation (von Ahn & Dabbish, 2008; Siorpaes & Hepp, 2008a; Vickrey et al., 2008), they remain applicable only to narrow problem domains. In this paper, we examine design aspects of existing GWAPs (e.g., player input validation, anti-cheating, scoring system) – their shared characteristics and specifics to support the effort of devising a broader methodology, at least for the Semantic Web domain. Our own contribution to the field of GWAPs for the Semantic Web includes Little Search Game – a novel game for the discovery of semantic links between terms and the successive construction of a term network. Little Search Game is a web search game, where players compete in reducing the number of search results by entering queries in a special format. The query consists of one normal search term, given to players before the game starts, and several negative terms entered by players, like “star –movie –wars –death”. This query format forces players (in order to be successful in the game) to use negative terms with frequent common co-occurrences with the initial term (i.e., related to it). We collect this information via game logs and aggregate it into a lightweight term network of term relationships. The game has also the unique ability to discover term relationships as perceived by humans, some of which are hard to detect using statistical corpora analysis. Therefore the resulting lightweight term network is suitable for applications like learning support frameworks (Barla et al., 2010), exploratory search tools (Šimko, Tvarožek, & Bieliková, 2010) or as a subject for further semantic enrichment. We present the details of the Little Search Game’s mechanics and term network construction. We further discuss the system of ladders and anti-cheating techniques employed. Next, based on our previous experiments, we briefly present quantitative results of the game deployment and validation of the acquired term network. Next, we discuss the attractiveness of the game with results of a player feedback survey. Lastly, we discuss the extension of the produced term network into a more ontology-like structure through naming of its relationships, and present experimental results examining the relationship types of the term network. SEMAntIc AcQuISItIon tEcHnIQuES Many approaches for knowledge and semantics acquisition have been developed. Automatic approaches to metadata creation address the requirements of web-scale information processing: they extract term relationships by mining text corpora using latent semantic analysis (Park & Ramamohanarao, 2009) or by exploiting existing taxonomies (El Sayed, Hacid, & Zighed, 2007) or folksonomies (Barla & Bieliková, 2009), sometimes within specialized International Journal on Semantic Web and Information Systems, 7(3), 23-45, July-September 2011 25 Copyright © 2011, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited. domains (Nenadić & Ananiadou, 2006; Šimko, 2011; Šimko & Bieliková, 2009). Products of such approaches (and also of our game) are term networks, similar to human-created folksonomies: graphs with terms or concepts as nodes, connected to others by relationships of a single type. More formally the network can be represented by a set of RDF triplets with only one possible predicate (which also raises the issue of posterior acquisition of true relationship types). Other semantics acquisition approaches focus on the discovery of ontology triplets via harvesting of statements (sentences) within a text corpus. Approaches described by Pantel and Pennacchiotti (2008) or Sanchez (2010) extract triplets by searching for occurrences of predicates manually picked beforehand (general, e.g., “part of” or domain specific) and identifying subjects and objects they are bound with. An interesting “inversion” of this approach was developed by Weichselbraun et al. (2010) which labels already existing relationships. The method mines a text corpus looking for co-occurrences of terms coupled in unlabeled relationships and looks up for candidate predicates (Weichselbraun, Wohlgenannt, & Scharl, 2010). While automatic methods are capable of delivering quantity (e.g., number of terms or concepts and relationships), they vary in quality (e.g., relevance of relationships) (Wang, Maynard, Peters, Bontcheva, & Cunningham, 2005) and often cannot cover nuances and unusual situations in corpora thus producing metadata that need additional validation by human experts. Human resources however are usually scarce and too expensive to be practically viable for web-scale tasks. Perhaps the most comprehensive manually created knowledge base and inference system today is CYC, which despite more than 30 years of concentrated human effort (more than a person-century) still has to achieve widespread practical acceptance and impact on the Web (Lenat, 1995). Consequently, the limited availability of metadata impedes the deployment of advanced applications and seriously hinders Semantic Web adoption. GAMES WItH A PurPoSE The basic premise of games with a purpose is that players willingly play a game and are rewarded by non-monetary values (e.g., fun), while they solve a corresponding problem as a side effect of the game (Krause, Takhtamysheva, Wittstock, & Malaka, 2010). To illustrate the potential of GWAPs for web-scale problem solving, we cite statistics provided by the Entertainment Software Association (2011). These indicate that about 273.5 million games were sold in 2009, corresponding to more than 550 million human-hours, if each game was played at least for two hours, still not considering games freely available on the Web (e.g., Farmville on Facebook). We believe that this sheer volume of untapped and potentially available human computing power could be partially harnessed to provide the resources necessary to create the semantic metadata required to bring the Semantic Web vision yet one step closer. Despite the fact that games with a purpose present a recent development enabled by the Web and the number of online users as potential players, they have already been used to perform various tasks, mostly related to semantics and metadata acquisition. GWAPs take advantage of common features such as player consensus and their designers also have to address several common issues (game attractiveness, motivation, cheating prevention) in order to create a successful GWAP. Aspects of GWAP design The basic principle lies in solving a computational problem via a carefully engineered game providing enjoyable experience to players, who are usually unaware of its true purpose. Players engage in the game because it provides some non-monetary incentive, such as fun and entertainment. When playing, players’ actions are logged and subsequently processed to generate useful artifacts, e.g., metadata. The practicality of GWAPs for web-scale problems comes from their properties: 26 International Journal on Semantic Web and Information Systems, 7(3), 23-45, July-September 2011 Copyright © 2011, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited. • Massive parallelization due to the large number of potential players and their willingness to play, • Human intelligence can effectively solve many problems presently unsolvable by machine approaches, • Ability to achieve correct results even on noisy inp