OpenRaiser/Intern-Atlas huggingface dataset:
Challenges of Multileaved Comparison in Practice: Lessons from NTCIR-13 OpenLiveQ Task
Authors: Makoto P. Kato, Tomohiro Manabe, Sumio Fujita, Akiomi Nishida, Takehiro Yamamoto
Venue: CIKM
Year: 2018

This paper discusses challenges of an online evaluation technique, multileaved comparison, based on the analysis of evaluation results in a community question-answering (cQA) search service. NTCIR-13 OpenLiveQ task offered a shared task in which participants addressed an ad-hoc retrieval task in a cQA service, and evaluated their rankers by multileaved comparison, which combines multiple rankings to generate a single search result page, and simultaneously evaluates the different rankings based on users' clicks on the search result page. Since the number of search result impressions during the evaluation period might not suffice to evaluate a hundred of rankers, we conducted the online evaluation only for rankers that achieved high performance in offline evaluation. The analysis of evaluation results showed that offline and online evaluation results did not fully agree, and a large number of users' clicks were necessary to find a statistically significant difference for every ranker pair. To cope with these problems in large-scale multileaved comparison, we propose a new experimental design that evaluates all the rankers online but intensively tests only the top-k rankers. Simulation-based experiments demonstrated that Copeland counting algorithm could achieve high top-k recall in the top-k identification problem for multileaved comparison.
PolyHJ: A Polymorphic Main-Memory Hash Join Paradigm for Multi-Core Machines
Authors: Omar Khattab, Mohammad Hammoud, Omar Shekfeh
Venue: CIKM
Year: 2018

Relational join is a central data management operation that influences the performance of almost every database query. In this paper, we show that different input features and hardware settings necessitate different main-memory hash join models. Subsequently, we identify four particular models by which hash-based join algorithms can be executed and propose a novel polymorphic paradigm that dynamically subscribes to the best model given workload and hardware characteristics. We refer to our polymorphic paradigm as PolyHJ and suggest a corresponding implementation, which consists of two mechanisms, namely, in-place, cache-aware partitioning (ICP) and collaborative building and probing (ColBP). ICP and ColBP serve substantially in reducing multi-core cache misses, memory bandwidth usage, and cross-socket traffic. Our experimental results demonstrate that PolyHJ can successfully select the right models for the tested workloads and significantly outperform the current state-of-the-art hash-based join schemes.
Weave&Rec: A Word Embedding based 3-D Convolutional Network for News Recommendation
Authors: Dhruv Khattar, Vaibhav Kumar, Vasudeva Varma, Manish Gupta
Venue: CIKM
Year: 2018

An effective news recommendation system should harness the historical information of the user based on her interactions as well as the content of the articles. In this paper we propose a novel deep learning model for news recommendation which utilizes the content of the news articles as well as the sequence in which the articles were read by the user. To model both of these information, which are essentially of different types, we propose a simple yet effective architecture which utilizes a 3-dimensional Convolutional Neural Network which takes the word embeddings of the articles present in the user history as its input. Using such a method endows the model with the capability to automatically learn spatial (features of a particular article) as well as temporal features (features across articles read by a user) which signify the interest of the user. At test time, we use this in combination with a 2-dimensional Convolutional Neural Network for recommending articles to users. On a real-world dataset our method outperformed strong baselines which also model the news recommendation problem using neural networks.
Predicting Personal Life Events from Streaming Social Content
Authors: Maryam Khodabakhsh, Hossein Fani, Fattane Zarrinkalam, Ebrahim Bagheri
Venue: CIKM
Year: 2018

Researchers have shown that it is possible to identify reported instances of personal life events from users' social content, e.g., tweets. This is known as personal life event detection. In this paper, we take a step forward and explore the possibility of predicting users' next personal life event based solely on the their historically reported personal life events, a task which we refer to as personal life event prediction. We present a framework for modeling streaming social content for the purpose of personal life event prediction and describe how various instantiations of the framework can be developed to build a life event prediction model. In our extensive experiments, we find that (i) historical personal life events of a user have strong predictive power for determining the user's future life event; (ii) the consideration of sequence in historically reported personal life events shows inferior performance compared to models that do not consider sequence, and (iii) the number of historical life events and the length of the past time intervals that are taken into account for making life event predictions can impact prediction performance whereby more recent life events show more relevance for the prediction of future life events.
Distinguishing Trajectories from Different Drivers using Incompletely Labeled Trajectories
Authors: Tung Kieu, Bin Yang, Chenjuan Guo, Christian S. Jensen
Venue: CIKM
Year: 2018

We consider a scenario that occurs often in the auto insurance industry. We are given a large collection of trajectories that stem from many different drivers. Only a small number of the trajectories are labeled with driver identifiers, and only some drivers are used in labels. The problem is to label correctly the unlabeled trajectories with driver identifiers. This is important in auto insurance to detect possible fraud and to identify the driver in, e.g., pay-as-you-drive settings when a vehicle has been involved in an incident. To solve the problem, we first propose a Trajectory-to-Image( T2I) encoding scheme that captures both geographic features and driving behavior features of trajectories in 3D images. Next, we propose a multi-task, deep learning model called T2INet for estimating the total number of drivers in the unlabeled trajectories, and then we partition the unlabeled trajectories into groups so that the trajectories in a group belong to the same driver. Experimental results on a large trajectory data set offer insight into the design properties of T2INet and demonstrate that T2INet is capable of outperforming baselines and the state-of-the-art method.
Web Table Understanding by Collective Inference
Authors: San Kim, Guoliang Li, Jianhua Feng, Kaiyu Li
Venue: CIKM
Year: 2018

Web tables have become very popular and important in many real applications, such as search engines and knowledge base enrichment. Due to its benefit, it is very urgent to understand web tables. An important task in web table understanding is the column-type detection, which detects the most likely types (categories) to describe the columns in the web table. Some existing studies use knowledge bases to determine the column types. However, this problem has three challenges. (i) Web tables are too dirty to be understood. (ii) Knowledge bases are not comprehensive enough to cover all the columns. (iii) The size of both knowledge bases and web tables are extremely huge. Thus, traditional approaches encounter the limitations with low quality and poor scalability. Also, they cannot extract the best type from top-k types automatically. To address these limitations, we propose a collective inference approach (CIA) based on Topic Sensitive PageRank, which considers not only the types of detected columns, but also the collective information of web tables to automatically produce more accurate top-k types, especially the top-1 type, for both incorrectly detected columns and undetectable columns whose cells do not exist in the knowledge base. We also propose three methods to improve the inference performance and implemented techniques of CIA in MapReduce. Experimental results on real-world datasets show that our CIA achieves much higher quality in top-1 type detection as well as the entity enrichment, and outperforms state-of-the-art approaches significantly.
Insights from the Long-Tail: Learning Latent Representations of Online User Behavior in the Presence of Skew and Sparsity
Authors: Adit Krishnan, Ashish Sharma, Hari Sundaram
Venue: CIKM
Year: 2018

This paper proposes an approach to learn robust behavior representations in online platforms by addressing the challenges of user behavior skew and sparse participation. Latent behavior models are important in a wide variety of applications: recommender systems; prediction; user profiling; community characterization. Our framework is the first to jointly address skew and sparsity across graphical behavior models. We propose a generalizable bayesian approach to partition users in the presence of skew while simultaneously learning latent behavior profiles over these partitions to address user-level sparsity. Our behavior profiles incorporate the temporal activity and links between participants, although the proposed framework is flexible to introduce other definitions of participant behavior. Our approach explicitly discounts frequent behaviors and learns variable size partitions capturing diverse behavior trends. The partitioning approach is data-driven with no rigid assumptions, adapting to varying degrees of skew and sparsity. A qualitative analysis indicates our ability to discover niche and informative user groups on large online platforms. Results on User Characterization (+6-22% AUC); Content Recommendation (+6-43% AUC) and Future Activity Prediction (+12-25% RMSE) indicate significant gains over state-of-the-art baselines. Furthermore, user cluster quality is validated with magnified gains in the characterization of users with sparse activity.
An Adversarial Approach to Improve Long-Tail Performance in Neural Collaborative Filtering
Authors: Adit Krishnan, Ashish Sharma, Aravind Sankar, Hari Sundaram
Venue: CIKM
Year: 2018

In recent times, deep neural networks have found success in Collaborative Filtering (CF) based recommendation tasks. By parametrizing latent factor interactions of users and items with neural architectures, they achieve significant gains in scalability and performance over matrix factorization. However, the long-tail phenomenon in recommender performance persists on the massive inventories of online media or retail platforms. Given the diversity of neural architectures and applications, there is a need to develop a generalizable and principled strategy to enhance long-tail item coverage. In this paper, we propose a novel adversarial training strategy to enhance long-tail recommendations for users with Neural CF (NCF) models. The adversary network learns the implicit association structure of entities in the feedback data while the NCF model is simultaneously trained to reproduce these associations and avoid the adversarial penalty, resulting in enhanced long-tail performance. Experimental results show that even without auxiliary data, adversarial training can boost long-tail recall of state-of-the-art NCF models by up to 25%, without trading-off overall performance. We evaluate our approach on two diverse platforms, content tag recommendation in Q&A forums and movie recommendation.
Preference-driven Interactive Ranking System for Personalized Decision Support
Authors: Caitlin Kuhlman, MaryAnn Van Valkenburg, Diana Doherty, Malika Nurbekova, Goutham Deva, Zarni Phyo, Elke A. Rundensteiner, Lane Harrison
Venue: CIKM
Year: 2018

Manually constructing rankings is a tedious ad-hoc process, requiring extensive user effort to evaluate data attribute importance, and often leading to undesirable results. Meanwhile, sophisticated learning-to-rank algorithms are able to leverage large amounts of data to generate high quality rankings automatically. In this work we present RanKit, a novel technology that brings the power of automatic learning-to-rank to the public. RanKit serves as a personal recommender system for building rankings from partial user knowledge in the form of item preferences. A user-friendly rank building interface provides rich input modes for preference specification. Visual feedback on the quality of the learned ranking model is given in real time, empowering the user to guide the underlying learn-to-rank algorithm. Users are actively involved with every step of the rank generation process, developing trust in the model and producing personalized rankings suitable for real-world decision making. In this demonstration, the audience works directly with the RanKit system on public domain datasets ranging from college rankings and economic indicators to movies and sports.
Detecting Outliers in Data with Correlated Measures
Authors: Yu-Hsuan Kuo, Zhenhui Li, Daniel Kifer
Venue: CIKM
Year: 2018

Advances in sensor technology have enabled the collection of large-scale datasets. Such datasets can be extremely noisy and often contain a significant amount of outliers that result from sensor malfunction or human operation faults. In order to utilize such data for real-world applications, it is critical to detect outliers so that models built from these datasets will not be skewed by outliers. In this paper, we propose a new outlier detection method that utilizes the correlations in the data (e.g., taxi trip distance vs. trip time). Different from existing outlier detection methods, we build a robust regression model that explicitly models the outliers and detects outliers simultaneously with the model fitting. We validate our approach on real-world datasets against methods specifically designed for each dataset as well as the state of the art outlier detectors. Our outlier detection method achieves better performances, demonstrating the robustness and generality of our method. Last, we report interesting case studies on some outliers that result from atypical events.
Construction of Efficient V-Gram Dictionary for Sequential Data Analysis
Authors: Igor Kuralenok, Natalia Starikova, Aleksandr Khvorov, Julian Serdyuk
Venue: CIKM
Year: 2018

This paper presents a new method for constructing an optimal feature set from sequential data. It creates a dictionary of n-grams of variable length (we call them v-grams), based on the minimum description length principle. The proposed method is a dictionary coder and works simultaneously as both a compression algorithm and as unsupervised feature extraction. The length of constructed v-grams is not limited by any bound and exceeds 100 characters in provided experiments. Constructed v-grams can be used for any sequential data analysis and allows transfer bag-of-word techniques to non-text data types. The method demonstrates a high compression rate on various real-life datasets. Extracted features generate a practical basis for text classification, that shows competitive results on standard text classification collections without using the text structure. Combining extracted character v-grams with the words from the original text we achieved substantially better classification quality than on words or v-grams alone.
When Rank Order Isn't Enough: New Statistical-Significance-Aware Correlation Measures
Authors: Mücahid Kutlu, Tamer Elsayed, Maram Hasanain, Matthew Lease
Venue: CIKM
Year: 2018

Because it is expensive to construct test collections for Cranfield-based evaluation of information retrieval systems, a variety of lower-cost methods have been proposed. The reliability of these methods is often validated by measuring rank correlation (e.g., Kendall's tau) between known system rankings on the full test collection vs. observed system rankings on the lower-cost one. However, existing rank correlation measures do not consider the statistical significance of score differences between systems in the observed rankings. To address this, we propose two statistical-significance-aware rank correlation measures, one of which is a head-weighted version of the other. We first show empirical differences between our proposed measures and existing ones. We then compare the measures while benchmarking four system evaluation methods: pooling, crowdsourcing, evaluation with incomplete judgments, and automatic system ranking. We show that use of our measures can lead to different experimental conclusions regarding reliability of alternative low-cost evaluation methods.
Traffic-Cascade: Mining and Visualizing Lifecycles of Traffic Congestion Events Using Public Bus Trajectories
Authors: Agus Trisnajaya Kwee, Meng-Fen Chiang, Philips Kokoh Prasetyo, Ee-Peng Lim
Venue: CIKM
Year: 2018

As road transportation supports both economic and social activities in developed cities, it is important to maintain smooth traffic on all highways and local roads. Whenever possible, traffic congestions should be detected early and resolved quickly. While existing traffic monitoring dashboard systems have been put in place in many cities, these systems require high-cost vehicle speed monitoring instruments and detect traffic congestion as independent events. There is a lack of low-cost dashboards to inspect and analyze the lifecycle of traffic congestion which is critical in assessing the overall impact of congestion, determining the possible the source(s) of congestion and its evolution. In the absence of publicly available sophisticated road sensor data which measures on-road vehicle speed, we make use of publicly available vehicle trajectory data to detect the lifecycle of traffic congestion, also known as congestion cascade. We have developed Traffic-Cascade, a dashboard system to identify traffic congestion events, compile them into congestion cascades, and visualize them on a web dashboard. Traffic-Cascade unveils spatio-temporal insights of the congestion cascades.
Improving Low-Rank Matrix Completion with Self-Expressiveness
Authors: Minsu Kwon, Han-Gyu Kim, Ho-Jin Choi
Venue: CIKM
Year: 2018

In this paper, we improve the low-rank matrix completion algorithm by assuming that the data points lie in a union of low dimensional subspaces. We applied the self-expressiveness, which is a property of a dataset when the data points lie in a union of low dimensional subspaces, to the low-rank matrix completion. By considering self-expressiveness of low dimensional subspaces, the proposed low-rank matrix completion may perform well even with little information, leading to the robust completion on a dataset with high missing rate. In our experiments on movie rating datasets, the proposed model outperforms state-of-the-art matrix completion models. In clustering experiments conducted on MNIST dataset, the result indicates that our method closely recovers the subspaces of original dataset even with the high missing rate.
Multiperspective Graph-Theoretic Similarity Measure
Authors: Dung D. Le, Hady W. Lauw
Venue: CIKM
Year: 2018

Determining the similarity between two objects is pertinent to many applications. When the basis for similarity is a set of object-to-object relationships, it is natural to rely on graph-theoretic measures. One seminal technique for measuring the structural-context similarity between a pair of graph vertices is SimRank, whose underlying intuition is that two objects are similar if they are connected by similar objects. However, by design, SimRank as well as its variants capture only a single view or perspective of similarity. Meanwhile, in many real-world scenarios, there emerge multiple perspectives of similarity, i.e., two objects may be similar from one perspective, but dissimilar from another. For instance, human subjects may generate varied, yet valid, clusterings of objects. In this work, we propose a graph-theoretic similarity measure that is natively multiperspective. In our approach, the observed object-to-object relationships due to various perspectives are integrated into a unified graph-based representation, stylised as a hypergraph to retain the distinct perspectives. We then introduce a novel model for learning and reflecting diverse similarity perceptions given the hypergraph, yielding the similarity score between any pair of objects from any perspective. In addition to proposing an algorithm for computing the similarity scores, we also provide theoretical guarantees on the convergence of the algorithm. Experiments on public datasets show that the proposed model deals better with multiperspectivity than the baselines.
Disk-based Matrix Completion for Memory Limited Devices
Authors: Dongha Lee, Jinoh Oh, Christos Faloutsos, Byungju Kim, Hwanjo Yu
Venue: CIKM
Year: 2018

More and more data need to be processed or analyzed within mobile devices for efficiency or privacy reasons, but performing machine learning tasks with large data within the devices is challenging because of their limited memory resources. For this reason, disk-based machine learning methods have been actively researched, which utilize storage resources without holding all the data in memory. This paper proposes D-MC2, a novel disk-based matrix completion method that (1) supports incremental data update (i.e., data insertion and deletion) and (2) spills both data and model to disk when necessary; these functionalities are not supported by existing methods. First, D-MC2 builds a two-layered index to efficiently support incremental data update; there exists a trade-off relationship between model learning and data update costs, and our two-layered index simultaneously optimizes the two costs. Second, we develop a window-based stochastic gradient descent (SGD) scheduler to efficiently support the dual spilling; a huge amount of disk I/O is incurred when the size of model is larger than that of memory, and our new scheduler substantially reduces it. Our evaluation results show that D-MC2 is significantly more scalable and faster than other disk-based competitors under the limited memory environment. In terms of the co-optimization, D-MC2 outperforms the baselines that only optimize one of the two costs up to 48x. Furthermore, the window-based scheduler improves the training speed 12.4x faster compared to a naive scheduler.
CAPatternMiner: Mining Ship Collision Avoidance Behavior from AIS Trajectory Data
Authors: Po-Ruey Lei, Li-Pin Xiao, Yu Ting Wen, Wen-Chih Peng
Venue: CIKM
Year: 2018

The improvement of collision avoidance for ship navigation in encounter situation is an important topic in maritime traffic safety. Most research on maritime collision avoidance has focused on planning a safe path for a ship to keep away from the approaching ship under the requirements of the International Regulations for Preventing Collision at Sea (COLREGs). However, the specific anti-collision actions are actually carried out by the navigators' own experience according to the local encounter situation. In this paper, different from the existing works, we discover the collision avoidance behavior from real ships' movement, i.e., AIS trajectory data. However, the uncertainty of maritime trajectory data brings the challenge of collision avoidance behavior mining. ollision behavior, which is effective in the encounter situation, and generate the discovered behavior in form of collision avoidance pattern. Furthermore, a prototype of CAPatternMiner is built for pattern analysis and visualization and also benefits a deeper understanding of collision avoidance behavior on maritime traffic. The proposed framework will be applied to the developing of pattern-aware collision avoidance system to improve the maritime traffic safety.