OpenRaiser/Intern-Atlas huggingface dataset:
An analysis of crowd workers mistakes for specific and complex relevance assessment task
Authors: Jesse Anderton, Maryam Bashir, Virgil Pavlu, Javed A. Aslam
Venue: CIKM
Year: 2013

The TREC 2012 Crowdsourcing track asked participants to crowdsource relevance assessments with the goal of replicating costly expert judgements with relatively fast, inexpensive, but less reliable judgements from anonymous online workers. The track used 10 "ad-hoc" queries, highly specific and complex (as compared to web search). The crowdsourced assessments were evaluated against expert judgments made by highly trained and capable human analysts in 1999 as part of ad hoc track collection construction. Since most crowdsourcing approaches submitted to the TREC 2012 track produced assessment sets nowhere close to the expert judgements, we decided to analyze crowdsourcing mistakes made on this task using data we collected via Amazon's Mechanical Turk service. We investigate two types of crowdsourcing approaches: one that asks for nominal relevance grades for each document, and the other that asks for preferences on many (not all) pairs of documents.
Incorporating the surfing behavior of web users into pagerank
Authors: Shatlyk Ashyralyyev, Berkant Barla Cambazoglu, Cevdet Aykanat
Venue: CIKM
Year: 2013

In large-scale commercial web search engines, estimating the importance of a web page is a crucial ingredient in ranking web search results. So far, to assess the importance of web pages, two different types of feedback have been taken into account, independent of each other: the feedback obtained from the hyperlink structure among the web pages (e.g., PageRank) or the web browsing patterns of users (e.g., BrowseRank). Unfortunately, both types of feedback have certain drawbacks. While the former lacks the user preferences and is vulnerable to malicious intent, the latter suffers from sparsity and hence low web coverage. In this work, we combine these two types of feedback under a hybrid page ranking model in order to alleviate the above-mentioned drawbacks. Our empirical results indicate that the proposed model leads to better estimation of page importance according to an evaluation metric that relies on user click feedback obtained from web search query logs. We conduct all of our experiments in a realistic setting, using a very large scale web page collection (around 6.5 billion web pages) and web browsing data (around two billion web page visits).
Retrieval of trending keywords in a peer-to-peer micro-blogging OSN
Authors: H. Asthana, Ingemar J. Cox
Venue: CIKM
Year: 2013

We investigate the problem of identifying trending information in a peer-to-peer micro-blogging online social network. In a distributed decentralized environment, the participating nodes do not have access to global statistics such as the frequencies of the keywords and the information creation rate. We propose a two step solution. First, nodes make a local estimate of the frequency of keywords in the network based on their local information. At each iteration a subset of nodes collect this information from a small subset of random nodes in the network and aggregate the results. The most frequently occurring keywords are identified. In the second step, a node requests another small random subset of nodes to identify when, in the recent past, the more frequently occurring keywords were seen in micro-blogs. Once again this information is aggregated the fraction of time within a consecutive period that keywords were encountered is calculated. If this fraction, referred to as the trending fraction, is close to 1, then the keyword is predicted to be trending. A simulation on a network of 10,000 nodes shows that the solution is capable of detecting multiple trending keywords with a moderate increase in bandwidth.
Correlating medical-dependent query features with image retrieval models using association rules
Authors: Hajer Ayadi, Mouna Torjmen, Mariam Daoud, Maher Ben Jemaa, Jimmy Xiangji Huang
Venue: CIKM
Year: 2013

The increasing quantities of available medical resources have motivated the development of effective search tools and medical decision support systems. Medical image search tools help physicians in searching medical image datasets for diagnosing a disease or monitoring the stage of a disease given previous patient's image screenings. Image retrieval models are classified into three categories: content-based (visual), textual and combined models. In most of previous work, a unique image retrieval model is applied for any user formulated query independently of what retrieval model best suits the information need behind the query. The main challenge in medical image retrieval is to cope the semantic gap between user information needs and retrieval models. In this paper, we propose a novel approach for finding correlations between medical query features and retrieval models based on association rule mining. We define new medical-dependent query features such as image modality and presence of specific medical image terminology and make use of existing generic query features such as query specificity, ambiguity and cohesiveness. The proposed query features are then exploited into association rule mining for discovering rules which correlate query features to visual, textual or combined image retrieval models. Based on the discovered rules, we propose to use an associative classifier that finds the best suitable rule with a maximum feature coverage for a new query. Experiments are performed on Image CLEF queries from 2008 to 2012 where we evaluate the impact of our proposed query features on the classification performance. Results show that combining our proposed specific and generic query features is effective for classifying queries. A comparative study between our classifier, CBA, Naïve Bayes, Bayes Net and decision trees showed that our best coverage associative classifier outperforms existing classifiers where it achieves an improvement of 30%.
Intelligent SSD: a turbo for big data mining
Authors: Duck-Ho Bae, Jin-Hyung Kim, Sang-Wook Kim, Hyunok Oh, Chanik Park
Venue: CIKM
Year: 2013

This paper introduces the notion of intelligent SSDs. First, we present the design considerations of intelligent SSDs, and then examine their potential benefits under various settings in data mining applications.
Towards minimizing the annotation cost of certified text classification
Authors: Mossaab Bagdouri, William Webber, David D. Lewis, Douglas W. Oard
Venue: CIKM
Year: 2013

The common practice of testing a sequence of text classifiers learned on a growing training set, and stopping when a target value of estimated effectiveness is first met, introduces a sequential testing bias. In settings where the effectiveness of a text classifier must be certified (perhaps to a court of law), this bias may be unacceptable. The choice of when to stop training is made even more complex when, as is common, the annotation of training and test data must be paid for from a common budget: each new labeled training example is a lost test example. Drawing on ideas from statistical power analysis, we present a framework for joint minimization of training and test annotation that maintains the statistical validity of effectiveness estimates, and yields a natural definition of an optimal allocation of annotations to training and test data. We identify the development of allocation policies that can approximate this optimum as a central question for research. We then develop simulation-based power analysis methods for van Rijsbergen's F-measure, and incorporate them in four baseline allocation policies which we study empirically. In support of our studies, we develop a new analytic approximation of confidence intervals for the F-measure that is of independent interest.
Cache refreshing for online social news feeds
Authors: Xiao Bai, Flavio Paiva Junqueira, Adam Silberstein
Venue: CIKM
Year: 2013

Several social networking applications enable users to view the events generated by other users, typically friends in the social network, in the form of ``news feeds''. Friends and events are typically maintained per user and cached in memory to enable efficient generation of news feeds. Caching user friends and events, however, raises concerns about the freshness of news feeds as users may not observe the most recent events when cache content becomes stale. Mechanisms to keep cache content fresh are thus critical for user satisfaction while computing news feeds efficiently through caching.
Predicting the impact of expansion terms using semantic and user interaction features
Authors: Anton Bakhtin, Yury Ustinovsky, Pavel Serdyukov
Venue: CIKM
Year: 2013

Query expansion for Information Retrieval is a challenging task. On the one hand, low quality expansion may hurt either recall, due to vocabulary mismatch, or precision, due to topic drift, and therefore reduce user satisfaction. On the other hand, utilizing a large number of expansion terms for a query may easily lead to resource consumption overhead. As web search engines apply strict constraints on response time, it is essential to estimate the impact of each expansion term on query performance at the pre-retrieval time. Our experimental results confirm that a significant part of expansions do not improve query performance, and it is possible to detect such expansions at the pre-retrieval time.
A partially supervised cross-collection topic model for cross-domain text classification
Authors: Yang Bao, Nigel Collier, Anindya Datta
Venue: CIKM
Year: 2013

Cross-domain text classification aims to automatically train a precise text classifier for a target domain by using labelled text data from a related source domain. To this end, one of the most promising ideas is to induce a new feature representation so that the distributional difference between domains can be reduced and a more accurate classifier can be learned in this new feature space. However, most existing methods do not explore the duality of the marginal distribution of examples and the conditional distribution of class labels given labeled training examples in the source domain. Besides, few previous works attempt to explicitly distinguish the domain-independent and domain-specific latent features and align the domain-specific features to further improve the cross-domain learning. In this paper, we propose a model called Partially Supervised Cross-Collection LDA topic model (PSCCLDA) for cross-domain learning with the purpose of addressing these two issues in a unified way. Experimental results on nine datasets show that our model outperforms two standard classifiers and four state-of-the-art methods, which demonstrates the effectiveness of our proposed model.
LearNext: learning to predict tourists movements
Authors: Ranieri Baraglia, Cristina Ioana Muntean, Franco Maria Nardini, Fabrizio Silvestri
Venue: CIKM
Year: 2013

In this paper, we tackle the problem of predicting the "next" geographical position of a tourist given her history (i.e., the prediction is done accordingly to the tourist's current trail) by means of supervised learning techniques, namely Gradient Boosted Regression Trees and Ranking SVM. The learning is done on the basis of an object space represented by a 68 dimension feature vector, specifically designed for tourism related data. Furthermore, we propose a thorough comparison of several methods that are considered state-of-the-art in touristic recommender and trail prediction systems as well as a strong popularity baseline. Experiments show that the methods we propose outperform important competitors and baselines thus providing strong evidence of the performance of our solutions.
Expedited rating of data stores using agile data loading techniques
Authors: Sumita Barahmand, Shahram Ghandeharizadeh
Venue: CIKM
Year: 2013

To benchmark and rate a data store, one must repeat experiments that impose a different amount of load on the data store. Workloads that modify the benchmark database may require the same database to be loaded repeatedly. This may constitute a significant portion of the time to rate a data store. This paper presents several agile data loading techniques to expedite the rating process. These techniques include generating the disk image of the database once and re-using it, restoring the updated data items to their original value, maintaining in-memory state of the database across different experiments to avoid repeated loading of the database all together, and a hybrid of the third technique in combination with the other two. These techniques are general purpose and apply to a variety of cloud benchmarks. We investigate their implementation and evaluation in the context of one, the BG benchmark. Obtained results show a factor of two to twelve speedup in the rating process. As an example, when evaluating MongoDB with a million member BG database, we show these techniques expedite BG's rating from 4 months (123 days) of continuous running to less than 11 days for the first rating experiment. Subsequent ratings of MongoDB with different workloads using the same database is much faster, in the order of hours.
A comparison of two physical data designs for interactive social networking actions
Authors: Sumita Barahmand, Shahram Ghandeharizadeh, Jason Yap
Venue: CIKM
Year: 2013

This paper compares the performance of an SQL solution that implements a relational data model with a document store named MongoDB. We report on the performance of a single node configuration of each data store and assume the database is small enough to fit in main memory. We analyze utilization of the CPU cores and the network bandwidth to compare the two data stores. Our key findings are as follows. First, for those social networking actions that read and write a small amount of data, the join operator of the SQL solution is not slower than the JSON representation of MongoDB. Second, with a mix of actions, the SQL solution provides either the same performance as MongoDB or outperforms it by 20%. Third, a middle-tier cache enhances the performance of both data stores as query result look up is significantly faster than query processing with either system.
Modeling behavioral factors ininteractive information retrieval
Authors: Feza Baskaya, Heikki Keskustalo, Kalervo Järvelin
Venue: CIKM
Year: 2013

In real-life, information retrieval consists of sessions of one or more query iterations. Each iteration has several subtasks like query formulation, result scanning, document link clicking, document reading and judgment, and stopping. Each of the subtasks has behavioral factors associated with them. These factors include search goals and cost constraints, query formulation strategies, scanning and stopping strategies, and relevance assessment behav-ior. Traditional IR evaluation focuses on retrieval and result presentation methods, and interaction within a single-query session. In the present study we aim at assessing the effects of the behavioral factors on retrieval effectiveness. Our research questions include how effective is human behavior employing search strategies compared to various baselines under various search goals and time constraints. We examine both ideal as well as fallible human behavior and wish to identify robust behaviors, if any. Methodologically, we use extensive simulation of human behavior in a test collection. Our findings include that (a) human behavior using multi-query sessions may exceed in effectiveness comparable single-query sessions, (b) the same empirically observed behavioral patterns are reasonably effective under various search goals and constraints, but (c) remain on average clearly below the best possible ones. Moreover, there is no behavioral pattern for sessions that would be even close to winning in most cases; the information need (or topic) in relation to the test collection is a determining factor.
DOLAP 2013 workshop summary
Authors: Ladjel Bellatreche, Alfredo Cuzzocrea, Il-Yeol Song
Venue: CIKM
Year: 2013

The ACM DOLAP workshop presents research on data warehousing and On-Line Analytical Processing (OLAP). The DOLAP 2013 program has three interesting sessions on Design and Exploitation of Social Data Warehouses, ETL and modeling and new trends, as well as a keynote talk on OLAP query processing and a panel on OLAP and DataWarehousing Technology in Big Data era.
Channeling the deluge: research challenges for big data and information systems
Authors: Paul Bennett, C. Lee Giles, Alon Y. Halevy, Jiawei Han, Marti A. Hearst, Jure Leskovec
Venue: CIKM
Year: 2013

With massive amounts of data being generated and stored ubiquitously in every discipline and every aspect of our daily life, how to handle such big data poses many challenging issues to researchers in data and information systems. The participants of CIKM 2013 are active researchers on large scale data, information and knowledge management, from multiple disciplines, including database systems, data mining, information retrieval, human-computer interaction, and knowledge or information management.
Automatically generating descriptions for resources by tag modeling
Authors: Bin Bi, Junghoo Cho
Venue: CIKM
Year: 2013

We have been witnessing an increasing number of social tagging systems on the web. Tags help users understand a resource readily and accurately. In a social tagging system, however, there are typically a fairly large number of resources each associated with a long list of tags. When browsing resources, users are reluctant to read these tags one by one. Instead, users prefer a shorter list of tags as a compact description of a resource. Such a tag description facilitates users to understand the resource accurately and effortlessly. This calls for a generator for a tag description, which selects a set of high-quality tags for a given resource. The tag description condenses the original tag list by retaining the most important tags of the long list.
MetKB: enriching RDF knowledge bases with web entity-attribute tables
Authors: Haoqiong Bian, Yueguo Chen, Xiaoyong Du, Xiaolu Zhang
Venue: CIKM
Year: 2013

There are many entity-attribute tables on the Web that can be utilized for enriching the entities of an RDF knowledge base. This requires the schema mapping (matching) between the Web tables and the RDF knowledge base. In this paper, we propose a feasible solution that is able to automatically search and rank entity-attribute tables from the Web, and effectively map the extracted tables with the RDF knowledge base with very few manual efforts.
Scalable bootstrapping for python
Authors: Peter Birsinger, Richard Xia, Armando Fox
Venue: CIKM
Year: 2013

High-level productivity languages such as Python, Matlab, and R are popular choices for scientists doing data analysis. However, for today's increasingly large datasets, applications written in these languages may run too slowly, if at all. In such cases, an experienced programmer must typically rewrite the application in a less-productive performant language such as C or C++, but this work is intricate, tedious, and often non-reusable. To bridge this gap between programmer productivity and performance, we extend an existing framework that uses just-in-time code generation and compilation. This framework uses the SEJITS methodology, (Selective Embedded Just-In-Time Specialization [11]), converting programs written in domain specific embedded languages (DSELs) to programs in languages suitable for high performance or parallel computation.
Detecting and exploring clusters in attributed graphs: a plugin for the gephi platform
Authors: Brigitte Boden, Roman Haag, Thomas Seidl
Venue: CIKM
Year: 2013

Clustering graph data has gained much attention in recent years, as data represented as graphs is ubiquitous in today's applications. For many applications, besides the mere graph data also further information about the vertices of a graph is available, which can be represented as attribute vectors. Recently, combined clustering approaches were introduced, which consider graph information and attribute vectors simultaneously for clustering. The visualization of clustering results can help users to get a better understanding of the results. In this paper, we introduce the GC-Viz system, which is implemented as a plugin for the Gephi platform. GC-Viz allows the user to test the combined clustering methods GAMer and DB-CSC on their data and to visualize and explore the clustering results. Furthermore, GC-Viz enables the user to visually compare the results of different clustering algorithms on the same dataset.
Accurate and scalable nearest neighbors in large networks based on effective importance
Authors: Petko Bogdanov, Ambuj K. Singh
Venue: CIKM
Year: 2013

Nearest neighbor proximity search in large graphs is an important analysis primitive with a variety of applications in graph data from different domains. We propose a novel proximity measure for weighted graphs called Effective Importance which incorporates multiple paths between nodes and captures the inherent structural clusters within a network. We develop effective bounds on the EI value using a modified small subnetwork around a query node, enabling scalable exact nearest neighbor (NN) search at query time. Our NN search does not require heavy offline analysis or holistic knowledge of the graph, making our method suitable for very large dynamically changing networks or composite network overlays.
A two-phase algorithm for mining sequential patterns with differential privacy
Authors: Luca Bonomi, Li Xiong
Venue: CIKM
Year: 2013

Frequent sequential pattern mining is a central task in many fields such as biology and finance. However, release of these patterns is raising increasing concerns on individual privacy. In this paper, we study the sequential pattern mining problem under the differential privacy framework which provides formal and provable guarantees of privacy. Due to the nature of the differential privacy mechanism which perturbs the frequency results with noise, and the high dimensionality of the pattern space, this mining problem is particularly challenging. In this work, we propose a novel two-phase algorithm for mining both prefixes and substring patterns. In the first phase, our approach takes advantage of the statistical properties of the data to construct a model-based prefix tree which is used to mine prefixes and a candidate set of substring patterns. The frequency of the substring patterns is further refined in the successive phase where we employ a novel transformation of the original data to reduce the perturbation noise. Extensive experiment results using real datasets showed that our approach is effective for mining both substring and prefix patterns in comparison to the state-of-the-art solutions.
Diversified query expansion using conceptnet
Authors: Arbi Bouchoucha, Jing He, Jian-Yun Nie
Venue: CIKM
Year: 2013

Search result diversification (SRD) aims to select diverse documents from the search results in order to cover as many search intents as possible. A prerequisite is that the search results contain diverse documents. For this purpose, we investigate a new approach to SRD by diversifying the query. Expansion terms are selected from ConceptNet so as to cover as diverse aspects as possible. The experimental results on several TREC data sets show that our method can outperform the existing state-of-the-art approaches that do not diversify the query.