OpenRaiser/Intern-Atlas huggingface dataset:
Size and Source Matter: Understanding Inconsistencies in Test Collection-Based Evaluation
Authors: Timothy Jones, Andrew Turpin, Stefano Mizzaro, Falk Scholer, Mark Sanderson
Venue: CIKM
Year: 2014

Past work showed that significant inconsistencies between retrieval results occurred on different test collections, even when one of the test collections contained only a subset of the documents in the other. However, the experimental methodologies in that paper made it hard to determine the cause of the inconsistencies. Using a novel methodology that eliminates the problems with uneven distribution of relevant documents, we confirm that observing a statistically significant improvement between two IR systems can be strongly influenced by the choice of documents in the test collection. We investigate two possible causes of this problem of test collections. Our results show that collection size and document source have a strong influence in the way that a test collection will rank one retrieval system relative to another. This is of particular interest when constructing test collections, as we show that using different subsets of a collection produces differing evaluation results.
On Building Decision Trees from Large-scale Data in Applications of On-line Advertising
Authors: Shivaram Kalyanakrishnan, Deepthi Singh, Ravi Kant
Venue: CIKM
Year: 2014

Decision trees have been used for several decades as simple and effective solutions to supervised learning problems. Their success extends to tasks across a variety of areas. Yet, data collected today through web-domains such as on-line advertising presents many new challenges: sheer size, the prevalence of high-arity categorical features, unknown feature-values, "cold starts", sparse training instances, and imbalance in the class labels. We argue that decision trees remain an ideal choice for applications of on-line advertising as they naturally construct higher-order conjunctive features; we then contribute two ideas to improve tree-building accordingly. First, to handle high-arity categorical features, we introduce a method to cluster feature-values based on their output responses. The result is more "data-dense" trees with relatively small branching factors. Second, we employ cross-validation as a principled approach to derive splitting and stopping criteria: thereby we identify splits that generalize well, and also curb overfitting. Evaluated on three distinct probability-estimation tasks in on-line advertising, our method, "CCDT", shows significant improvements in the accuracy of prediction.
A Meta-reasoner to Rule Them All: Automated Selection of OWL Reasoners Based on Efficiency
Authors: Yong-Bin Kang, Shonali Krishnaswamy, Yuan-Fang Li
Venue: CIKM
Year: 2014

It has been shown, both theoretically and empirically, that reasoning about large and expressive ontologies is computationally hard. Moreover, due to the different reasoning algorithms and optimisation techniques employed, each reasoner may be efficient for ontologies with different characteristics. Based on recently-developed prediction models for various reasoners for reasoning performance, we present our work in developing a meta-reasoner that automatically selects from a number of state-of-the-art OWL reasoners to achieve optimal efficiency. Our preliminary evaluation shows that the meta-reasoner significantly and consistently outperforms 6 state-of-the-art reasoners and it achieves a performance close to the hypothetical gold standard reasoner.
Data/Feature Distributed Stochastic Coordinate Descent for Logistic Regression
Authors: Dongyeop Kang, Woosang Lim, Kijung Shin, Lee Sael, U Kang
Venue: CIKM
Year: 2014

How can we scale-up logistic regression, or L1 regularized loss minimization in general, for Terabyte-scale data which do not fit in the memory? How to design the distributed algorithm efficiently? Although there exist two major algorithms for logistic regression, namely Stochastic Gradient Descent (SGD) and Stochastic Coordinate Descent (SCD), they face limitations in distributed environments. Distributed SGD enables data parallelism (i.e., different machines access different part of the input data), but it does not allow feature parallelism (i.e., different machines compute different subsets of the output), and thus the communication cost is high. On the other hand, Distributed SCD allows feature parallelism, but it does not allow data parallelism and thus is not suitable to work in distributed environments.
Semantic Topology
Authors: Jussi Karlgren, Martin Bohman, Ariel Ekgren, Gabriel Isheden, Emelie Kullmann, David Nilsson
Venue: CIKM
Year: 2014

Semantic spaces, a useful learning framework for lexical resources, are typically treated as black boxes and applied using geometric and linear algebraic processing tools. We have found that topological methods are useful for exploring the makeup of a semantic space.
Efficient Static and Dynamic In-Database Tensor Decompositions on Chunk-Based Array Stores
Authors: Mijung Kim, K. Selçuk Candan
Venue: CIKM
Year: 2014

As the relevant data sets get large, existing in-memory schemes for tensor decomposition become increasingly ineffective and, instead, memory-independent solutions, such as in-database analytics, are necessitated. In this paper, we present techniques for efficient implementations of in-database tensor decompositions on chunk-based array data stores. The proposed static and incremental in-database tensor decomposition operators and their optimizations address the constraints imposed by the main memory limitations when handling large and high-order tensor data. Firstly, we discuss how to implement alternating least squares operations efficiently on a chunk-based data storage system. Secondly, we consider scenarios with frequent data updates and show that compressed matrix multiplication techniques can be effective in reducing the incremental tensor decomposition maintenance costs. To the best of our knowledge, this paper presents the first attempt to develop efficient and optimized in-database tensor decomposition operations. We evaluate the proposed algorithms on tensor data sets that do not fit into the available memory and results show that the proposed techniques significantly improve the scalability of this core data analysis.
TensorDB: In-Database Tensor Manipulation with Tensor-Relational Query Plans
Authors: Mijung Kim, K. Selçuk Candan
Venue: CIKM
Year: 2014

Today's data management systems increasingly need to support both tensor-algebraic operations (for analysis) as well as relational-algebraic operations (for data manipulation and integration). Tensor decomposition techniques are commonly used for discovering underlying structures of multi-dimensional data sets. However, as the relevant data sets get large, existing in-memory schemes for tensor decomposition become increasingly ineffective and, instead, memory-independent solutions, such as in-database analytics, are necessitated. We introduce an in-database analytic system for efficient implementations of in-database tensor decompositions on chunk-based array data stores, so called, TensorDB. TensorDB includes static in-database tensor decomposition and dynamic in-database tensor decomposition operators. TensorDB extends an array database and leverages array operations for data manipulation and integration. TensorDB supports complex data processing plans where multiple relational algebraic and tensor algebraic operations are composed with each other.
Exploiting Knowledge Structure for Proximity-aware Movie Retrieval Model
Authors: Sansung Kim, Keejun Han, Mun Yong Yi, Sinhee Cho, Seongchan Kim
Venue: CIKM
Year: 2014

Current movie title retrieval models, such as IMDB, mainly focus on utilizing structured or semi-structured data. However, user queries for searching a movie title are often based on the movie plot, rather than its metadata. As a solution to this problem, our movie title retrieval model proposes a new way of elaborately utilizing associative relations between multiple key terms that exist in the movie plot, in order to improve search performance when users enter more than one keyword. More specifically, the proposed model exploits associative networks of key terms, called knowledge structures, derived from the movie plots. Using the search query terms entered by Amazon Mechanical Turk users as the golden standard, experiments were conducted to compare the proposed retrieval model with the extant state-of-the-art retrieval models. The experiment results show that the proposed retrieval model consistently outperforms the baseline models. The findings have practical implications for semantic search of movie titles particularly, and of online entertainment contents in general.
Modelling and Detecting Changes in User Satisfaction
Authors: Julia Kiseleva, Eric Crestan, Riccardo Brigo, Roland Ditte
Venue: CIKM
Year: 2014

Informational needs behind queries, that people issue to search engines, are inherently sensitive to external factors such as breaking news, new models of devices, or seasonal changes as 'black Friday'. Mostly these changes happen suddenly and it is natural to suppose that they may cause a shift in user satisfaction with presented old search results and push users to reformulate their queries. For instance, if users issued the query 'CIKM conference' in 2013 they were satisfied with results referring to the page cikm2013.org and this page gets a majority of clicks. However, the confernce site has been changed and the same query issued in 2014 should be linked to the different page cikm2014.fudan.edu.cn. If the link to the fresh page is not among the retrieved results then users will reformulate the query to find desired information.
DEESSE: entity-Driven Exploratory and sErendipitous Search SystEm
Authors: Olivier Van Laere, Ilaria Bordino, Yelena Mejova, Mounia Lalmas
Venue: CIKM
Year: 2014

We present DEESSE [1], a tool that enables an exploratory and serendipitous exploration - at entity level, of the content of two different social media: Wikipedia, a user-curated online encyclopedia, and Yahoo Answers, a more unconstrained question/answering forum. DEESSE represents the content of each source as an entity network, which is further enriched with metadata about sentiment, writing quality, and topical category. Given a query entity, entity results are retrieved from the network by employing an algorithm based on a random walk with restart to the query. Following the emerging paradigm of composite retrieval, we organize the results into topically coherent bundles instead of showing them in a simple ranked list.
CAST: A Context-Aware Story-Teller for Streaming Social Content
Authors: Pei Lee, Laks V. S. Lakshmanan, Evangelos E. Milios
Venue: CIKM
Year: 2014

Online social streams such as Twitter timelines, forum discussions and email threads have emerged as important channels for information propagation. Mining transient stories and their correlations implicit in social streams is a challenging task, since these streams are noisy and surge quickly. In this paper, we propose CAST, which is a context-aware story-teller that discovers new stories from social streams and tracks their structural context on the fly to build a vein of stories. More precisely, we model the social stream as a capillary network, and define stories by a new cohesive subgraph type called (k,d)-Core in the capillary network. We propose deterministic and randomized context search to support the iceberg query, which builds the story vein as social streams flow. We perform detailed experimental study on real Twitter streams and the results demonstrate the creativity and value of our approach.
Using Semantic Role Labeling to Predict Answer Types
Authors: Zuyao Li, Peter Exner, Pierre Nugues
Venue: CIKM
Year: 2014

Most question answering systems feature a step to predict an expected answer type given a question. Li and Roth \cite{li2002learning} proposed an oft-cited taxonomy to the categorize the answer types as well as an annotated data set. While offering a framework compatible with supervised learning, this method builds on a fixed and rigid model that has to be updated when the question-answering domain changes. More recently, Pinchak and Lin \cite{pinchak2006} designed a dynamic method using a syntactic model of the answers that proved more versatile. They used syntactic dependencies to model the question context and evaluated the performance on an English corpus. However, syntactic properties may vary across languages and techniques applicable to English may fail with other languages. In this paper, we present a method for constructing a probability-based answer type model for each different question. We adapted and reproduced the original experiment of Pinchak and Lin \cite{pinchak2006} on a Chinese corpus and we extended their model to semantic dependencies. Our model evaluates the probability that a candidate answer fits into the semantic context of a given question. We carried out an evaluation on a set of questions either drawn from NTCIR corpus \cite{ntcir2005} or that we created manually.
Focusing Decomposition Accuracy by Personalizing Tensor Decomposition (PTD)
Authors: Xinsheng Li, Shengyu Huang, Kasim Selçuk Candan, Maria Luisa Sapino
Venue: CIKM
Year: 2014

Tensor decomposition operation is the basis for many data analysis tasks from clustering, trend detection, anomaly detection, to correlation analysis. One key problem with tensor decomposition, however, is its computational complexity -- especially for dense data sets, the decomposition process takes exponential time in the number of tensor modes; the process is relatively faster for sparse tensors, but decomposition is still a major bottleneck in many applications. While it is possible to reduce the decomposition time by trading performance with decomposition accuracy, a drop in accuracy may not always be acceptable. In this paper, we first recognize that in many applications, the user may have a focus of interest -- i.e., part of the data for which the user needs high accuracy -- and beyond this area focus, accuracy may not be as critical. Relying on this observation, we propose a novel Personalized Tensor Decomposition(PTD) mechanism for accounting for the user's focus: PTD takes as input one or more areas of focus and performs the decomposition in such a way that, when reconstructed, the accuracy of the tensor is boosted for these areas of focus. We discuss alternative ways PTD can be implemented. Experiments show that PTD helps boost accuracy at the foci of interest, while reducing the overall tensor decomposition time.
iMiner: Mining Inventory Data for Intelligent Management
Authors: Lei Li, Chao Shen, Long Wang, Li Zheng, Yexi Jiang, Liang Tang, Hongtai Li, Longhui Zhang, Chunqiu Zeng, Tao Li, Jun Tang, Dong Liu
Venue: CIKM
Year: 2014

Inventory management refers to tracing inventory levels, orders and sales of a retailing business. In the current retailing market, a tremendous amount of data regarding stocked goods (items) in an inventory will be generated everyday. Due to the increasing volume of transaction data and the correlated relations of items, it is often a non-trivial task to efficiently and effectively manage stocked goods. In this demo, we present an intelligent system, called iMiner, to ease the management of enormous inventory data. We utilize distributed computing resources to process the huge volume of inventory data, and incorporate the latest advances of data mining technologies into the system to perform the tasks of inventory management, e.g., forecasting inventory, detecting abnormal items, and analyzing inventory aging. Since 2014, iMiner has been deployed as the major inventory management platform of ChangHong Electric Co., Ltd, one of the world's largest TV selling companies in China.
Probabilistic Classifier Chain Inference via Gibbs Sampling
Authors: Li Li, Longkai Zhang, Guangyi Li, Houfeng Wang
Venue: CIKM
Year: 2014

Multi-label classification is supervised learning, where an instance may be assigned with multiple categories (labels) simultaneously. Recently, a method called Probabilistic Classifier Chain (PCC) was proposed with numerous appealing properties, such as conceptual simplicity, flexibility, and theoretical justification. Nevertheless, PCC suffers from high inference complexity. To address this problem, we propose a novel inference method with gibbs sampling. An acceleration scheme is proposed to accelerate this method further. Our proposed method is based on our claim that PCC is a special case of Bayesian network. This claim may inspire more inference algorithms for PCC. Experiments with real-world data sets show effectiveness of our proposed method.
INK: A Cloud-Based System for Efficient Top-k Interval Keyword Search
Authors: Rui Li, Xiao Zhang, Xin Zhou, Shan Wang
Venue: CIKM
Year: 2014

It is insufficient to search temporal text by only focusing on either time attribute or keywords today as we pay close attention to the evolution of event with time. Both temporal and textual constraints need to be considered in one single query, called Top-k Interval Keyword Query (TIKQ).In this paper, we presents a cloud-based system named INK that supports efficient execution of TIKQs with appropriate effectiveness on Hadoop and HBase. In INK, an Adaptive Index Selector (AIS) is devised to choose the better execution plan for various TIKQs adaptively based on the proposed cost model, and leverage two novel hybrid index modules (TriI and IS-Tree) to combine keyword and interval filtration seamlessly.
CONR: A Novel Method for Sentiment Word Identification
Authors: Jiguang Liang, Xiaofei Zhou, Yue Hu, Li Guo, Shuo Bai
Venue: CIKM
Year: 2014

Sentiment word identification (SWI) is of high relevance to sentiment analysis technologies and applications. Currently most SWI methods heavily rely on sentiment seed words that have limited sentiment information. Even though there emerge non-seed approaches based on sentiment labels of documents, but in which the context information has not been fully considered. In this paper, based on matrix factorization with co-occurrence neighbor regularization which is derived from context, we propose a novel non-seed model called CONR for SWI. Instead of seed words, CONR exploits two important factors: sentiment matching and sentiment consistency for sentiment word identification. Experimental results on four publicly available datasets show that CONR can outperform the state of-the-art methods.
Fast, Accurate, and Space-efficient Tracking of Time-weighted Frequent Items from Data Streams
Authors: Yongsub Lim, Jihoon Choi, U Kang
Venue: CIKM
Year: 2014

How can we discover interesting patterns from time-evolving high speed data streams? How to analyze the data streams quickly and accurately, with little space overhead? High speed data stream has been receiving increasing attentions due to its wide applications such as sensors, network traffic, social networks, etc. One of the most fundamental tasks in the data stream is to find frequent items; especially, finding recently frequent items has become important in real world applications.
Exploring Ensemble of Models in Taxonomy-based Cross-Domain Sentiment Classification
Authors: Cong-Kai Lin, Yang-Yin Lee, Chi-Hsin Yu, Hsin-Hsi Chen
Venue: CIKM
Year: 2014

Most cross-domain sentiment classification techniques consider a domain as a whole set of opinionated instances for training. However, many online shopping websites organize their data in terms of taxonomy. With multiple domains (or, nodes) organized in a tree-structured representation, we propose a general ensemble algorithm which takes into account: 1) the model application, 2) the model weight and 3) the strategies for selecting the most related models with respect to a target node. The traditional sentiment classification technique SVM and the transfer learning algorithm Spectral Features Alignment (SFA) were applied as our model applications. In addition, the model weight takes the tree information and the similarity between domains into account. Finally, two strategies, cosine function and taxonomy-based regression model (TBRM) are proposed to select the most related models with respect to a target node. Experimental results showed both (cosine function and TBRM) proposed strategies outperform two baselines on an Amazon dataset. Three tasks of the proposed methods surpass the gold standard generated by the in-domain classifiers trained on the labeled data from the target nodes. Good results from the three tasks enable this algorithm to shed some new light on eliminating the major difficulties in transfer learning research: the distribution gap.
GPQ: Directly Optimizing Q-measure based on Genetic Programming
Authors: Yuan Lin, Hongfei Lin, Ping Zhang, Bo Xu
Venue: CIKM
Year: 2014

Ranking plays an important role in information retrieval system. In recent years, a kind of research named 'learning to rank' becomes more and more popular, which applies machine learning technology to solve ranking problems. Lots of ranking models belonged to learning to rank have been proposed, such as Regression, RankNet, and ListNet. Inspired by this, we proposed a novel learning to rank algorithm named GPQ in this paper, in which genetic programming was employed to directly optimize Q-measure evaluation metric. Experimental results on OHSUMED benchmark dataset indicated that our method GPQ could be competitive with Ranking SVM, SVMMAP and ListNet, and improve the ranking accuracies.
Towards Pathway Variation Identification: Aligning Patient Records with a Care Pathway
Authors: Haifeng Liu, Yang Liu, Xiang Li, Guo Tong Xie, Geetika T. Lakshmanan
Venue: CIKM
Year: 2014

A Care Pathway is a knowledge-centric process to guide clinicians to provide evidence-based care to patients with specific conditions. One existing problem for care pathways is that they often fail to reflect the best clinical practice as a result of not being adequately updated. A better understanding of the gaps between a care pathway and real practice requires aligning patient records with the pathway. Patient records are unlabeled in practice making it difficult to align them with a care pathway which is inherently complex due to its representation as a hierarchical and declarative process model (HDPM). This paper proposes to solve this problem by developing a Hierarchical Markov Random Field (HMRF) method so that a set of patient records can best fit a given care pathway. We validate the effectiveness of the method with experiments on both synthesized data and real clinical data.
Predicting Search Task Difficulty at Different Search Stages
Authors: Chang Liu, Jingjing Liu, Nicholas J. Belkin
Venue: CIKM
Year: 2014

Knowing, in real time, whether a current searcher in an information retrieval system finds the search task difficult can be valuable for tailoring the system's support for that searcher. This study investigated searcher's behaviors at different stages of the search process; they are: 1) first-round point at the beginning of the search, right before searchers issued their second query; 2) middle point, when searchers proceeded to the middle of the search process, and 3) end point, when searchers finished the whole task. We compared how the behavioral features calculated at these three points were different between difficult and easy search tasks, and identified behavioral features during search sessions that can be used in real-time to predict perceived task difficulty. In addition, we compared the prediction performance at different stages of search process. Our results show that a number of user behavioral measures at all three points differed between easy and difficult tasks. Query interval time, dwell time on viewed documents, and number of viewed documents per query were important predictors of task difficulty. The results also indicate that it is possible to make relatively accurate prediction of task difficulty at the first query round of a search. Our findings can help search systems predict task difficulty which is necessary in personalizing support for the individual searcher.