OpenRaiser/Intern-Atlas huggingface dataset:
A Fine-grained and Noise-aware Method for Neural Relation Extraction
Authors: Jianfeng Qu, Wen Hua, Dantong Ouyang, Xiaofang Zhou, Ximing Li
Venue: CIKM
Year: 2019

Distant supervision is an efficient way to generate large-scale training data for relation extraction without human efforts. However, a coin has two sides. The automatically annotated labels for training data are problematic, which can be summarized as multi-instance multi-label problem and coarse-grained (bag-level) supervised signal. To address these problems, we propose two reasonable assumptions and craft reinforcement learning to capture the expressive sentence for each relation mentioned in a bag. More specifically, we extend the original expressed-at-least-once assumption to multi-label level, and introduce a novel express-at-most-one assumption. Besides, we design a fine-grained reward function, and model the sentence selection process as an auction where different relations for a bag need to compete together to achieve the possession of a specific sentence based on its expressiveness. In this way, our model can be dynamically self-adapted, and eventually implements the accurate one-to-one mapping from a relation label to its chosen expressive sentence, which serves as training instances for the extractor. The experimental results on a public dataset demonstrate that our model constantly and substantially outperforms current state-of-the-art methods for relation extraction.
Learning to be Relevant: Evolution of a Course Recommendation System
Authors: Shivani Rao, Konstantin Salomatin, Gungor Polatkan, Mahesh Joshi, Sneha Chaudhari, Vladislav Tcheprasov, Jeffrey D. Gee, Deepak Kumar
Venue: CIKM
Year: 2019

We present the evolution of a large-scale content recommendation platform for LinkedIn Learning, serving 645M+ LinkedIn users across several different channels (e.g., desktop, mobile). We address challenges and complexities from both algorithms and infrastructure perspectives. We describe the progression from unsupervised models that exploit member similarity with course content, to supervised learning models leveraging member interactions with courses, and finally to hyper-personalized mixed-effects models with several million coefficients. For all the experiments, we include metric lifts achieved via online A/B tests and illustrate the trade-offs between computation and storage requirements.
CRUX: Adaptive Querying for Efficient Crowdsourced Data Extraction
Authors: Theodoros Rekatsinas, Amol Deshpande, Aditya G. Parameswaran
Venue: CIKM
Year: 2019

Crowdsourcing is essential for collecting information about real-world entities. Existing crowdsourced data extraction solutions use fixed, non-adaptive querying strategies that repeatedly ask workers to provide entities from a fixed domain until a desired level of coverage is reached. Unfortunately, such solutions are highly impractical as they yield many duplicate extractions. We design an adaptive querying framework, CRUX, that maximizes the number of extracted entities for a given budget. We show that the problem of budgeted crowdsourced entity extraction is NP-Hard. We leverage two insights to focus our extraction efforts: \em exploiting the structure of the domain of interest, and \em using exclude lists to limit repeated extractions. We develop new statistical tools to reason about the number of new distinct extracted entities of \em additional queries under the presence of little information, and embed them within adaptive algorithms that maximize the distinct extracted entities under budget constraints. We evaluate our techniques on synthetic and real-world datasets, demonstrating an improvement of up to 300% over competing approaches for the same budget.
Augment to Prevent: Short-Text Data Augmentation in Deep Learning for Hate-Speech Classification
Authors: Georgios Rizos, Konstantin Hemker, Björn W. Schuller
Venue: CIKM
Year: 2019

In this paper, we address the issue of augmenting text data in supervised Natural Language Processing problems, exemplified by deep online hate speech classification. A great challenge in this domain is that although the presence of hate speech can be deleterious to the quality of service provided by social platforms, it still comprises only a tiny fraction of the content that can be found online, which can lead to performance deterioration due to majority class overfitting. To this end, we perform a thorough study on the application of deep learning to the hate speech detection problem: a) we propose three text-based data augmentation techniques aimed at reducing the degree of class imbalance and to maximise the amount of information we can extract from our limited resources and b) we apply them on a selection of top-performing deep architectures and hate speech databases in order to showcase their generalisation properties. The data augmentation techniques are based on a) synonym replacement based on word embedding vector closeness, b) warping of the word tokens along the padded sequence or c) class-conditional, recurrent neural language generation. Our proposed framework yields a significant increase in multi-class hate speech detection, outperforming the baseline in the largest online hate speech database by an absolute 5.7% increase in Macro-F1 score and 30% in hate speech class recall.
Towards Stochastic Simulations of Relevance Profiles
Authors: Kevin Roitero, Andrea Brunello, Julián Urbano, Stefano Mizzaro
Venue: CIKM
Year: 2019

Recently proposed methods allow the generation of simulated scores representing the values of an effectiveness metric, but they do not investigate the generation of the actual lists of retrieved documents. In this paper we address this limitation: we present an approach that exploits an evolutionary algorithm and, given a metric score, creates a simulated relevance profile (i.e., a ranked list of relevance values) that produces that score. We show how the simulated relevance profiles are realistic under various analyses.
I-REX: A Lucene Plugin for EXplainable IR
Authors: Dwaipayan Roy, Sourav Saha, Mandar Mitra, Bihan Sen, Debasis Ganguly
Venue: CIKM
Year: 2019

Providing high-level, intuitive explanations of the performance of IR systems is generally difficult due to their complexity, and the various low-level implementation details involved. We present I-REX, a tool built on top of Lucene, that is intended to provide a systematic view into the inner workings of retrieval models and methods (specifically query expansion). This should help researchers study, compare, understand and explain the performance of these models and methods. I-REX can be run either as a Web service accessible through a browser, or as a terminal-based tool with a shell-like interactive interface. In this article, we describe a session that illustrates how I-REX can be used to explain the observed difference in the performance of two variants of the Language Model.
A Sampling-Based System for Approximate Big Data Analysis on Computing Clusters
Authors: Salman Salloum, Yinxu Wu, Joshua Zhexue Huang
Venue: CIKM
Year: 2019

To break the in-memory bottleneck and facilitate online sampling in cluster computing frameworks, we propose a new sampling-based system for approximate big data analysis on computing clusters. We address both computational and statistical aspects of big data across the main layers of cluster computing frameworks: big data storage, big data management, big data online sampling, big data processing, and big data exploration and analysis. We use the new Random Sample Partition (RSP) distributed data model to store a big data set as a set of ready-to-use random sample data blocks in Hadoop Distributed File System (HDFS), called RSP blocks. With this system, only a few RSP blocks are selected and processed using a sequential algorithm in a distributed data-parallel manner to produce approximate results for the entire data set. In this paper, we present a prototype RSP-based system and demonstrate its advantages. Our experiments show that RSP blocks can be used to get approximate models and summary statistics as well as estimate the proportions of inconsistent values without computing the entire data or running expensive online sampling operations. This new system enables big data exploration and analysis where the entire data set cannot be computed.
FLEET: Butterfly Estimation from a Bipartite Graph Stream
Authors: Seyed-Vahid Sanei-Mehri, Yu Zhang, Ahmet Erdem Sariyüce, Srikanta Tirthapura
Venue: CIKM
Year: 2019

We consider space-efficient single-pass estimation of the number of butterflies, a fundamental bipartite graph motif, from a massive bipartite graph stream where each edge represents a connection between entities in two different partitions. We present a space lower bound for any streaming algorithm that can estimate the number of butterflies accurately, as well as FLEET, a suite of algorithms for accurately estimating the number of butterflies in the graph stream. Estimates returned by the algorithms come with provable guarantees on the approximation error, and experiments show good tradeoffs between the space used and the accuracy of approximation. We also present space-efficient algorithms for estimating the number of butterflies within a sliding window of the most recent elements in the stream. While there is a significant body of work on counting subgraphs such as triangles in a unipartite graph stream, our work seems to be one of the few to tackle the case of bipartite graph streams.
Realtime Object Detection via Deep Learning-based Pipelines
Authors: James G. Shanahan, Liang Dai
Venue: CIKM
Year: 2019

Ever wonder how the Tesla Autopilot system works (or why it fails)? In this tutorial we will look under the hood of self-driving cars and of other applications of computer vision and review state-of-the-art tech pipelines for object detection such as two-stage approaches (e.g., Faster R-CNN) or single-stage approaches (e.g., YOLO/SSD). This is accomplished via a series of Jupyter Notebooks that use Python, OpenCV, Keras, and Tensorflow. No prior knowledge of computer vision is assumed (although it will be help!). To this end we begin this tutorial with a review of computer vision and traditional approaches to object detection such as Histogram of oriented gradients (HOG).
Labelling for Venue Visit Detection by Matching Wi-Fi Hotspots with Businesses
Authors: Denis Shaposhnikov, Anastasia A. Bezzubtseva, Ekaterina Gladkikh, Alexey Drutsa
Venue: CIKM
Year: 2019

User behaviour data is essential for modern companies, as it allows them to measure the impact of decisions they make and to gain new insights. A particular type of such data is user location trajectories, which can be clustered into Points of Interest, which, in turn, can be tied to certain venues (restaurants, schools, theaters, etc.). Machine learning is extensively utilized to detect and predict venue visits given the location data, but it requires a sufficient sample of labeled visits. Few Internet services provide a possibility to check-in for a user --- to send a signal that she is visiting a particular venue. However, for the majority of mobile applications it is unreasonable or far-fetched to introduce such a functionality for labeling purposes only. In this paper, we present a novel approach to label large quantities of location data as visits based on the following intuition: if a user is connected to a Wi-Fi hotspot of some venue, she is visiting the venue. Namely, we address the problem of matching Wi-Fi hotspots with venues by means of machine learning achieving 95% precision and 85% recall. The method has been deployed to production of one of the most popular global geo-based web services. We also release our dataset (that we utilize to develop the matching model) to facilitate research in this area.
Cluster-Based Focused Retrieval
Authors: Eilon Sheetrit, Oren Kurland
Venue: CIKM
Year: 2019

The focused retrieval task is to rank documents' passages by their presumed relevance to a query. Inspired by work on cluster-based document retrieval, we present a novel cluster-based focused retrieval method. The method is based on ranking clusters of similar passages using a learning-to-rank approach and transforming the cluster ranking to passage ranking. Empirical evaluation demonstrates the clear merits of the method.
GRLA 2019: The first International Workshop on Graph Representation Learning and its Applications
Authors: Huawei Shen, Jian Tang, Peng Bao
Venue: CIKM
Year: 2019

Graphs are the universal data structures for representing the relationships between interconnected objects. They are ubiquitous in a variety of disciplines and domains ranging from computer science, social science, economics, medicine, to bioinformatics. In Recent years, extensive studies have been conducted on the graph analysis techniques. One of the most fundamental challenges of analyzing graphs is effectively representing graphs, which largely determines the performance of many follow-up tasks. This workshop aims to provide a forum for industry and academia to discuss the latest progress on graph representation learning and their applications in different fields. We hope more advanced technologies can be proposed or inspired, and also we expect that the direction of graph representation learning can catch much more attention in both academic and industry.
Autonomous Driving Towards Mass Production
Authors: Jianping Shi
Venue: CIKM
Year: 2019

Visual recognition technology is very important for autonomous driving especially in direction of mass production. In this talk, we will introduce the algorimic progress for SenseTime in autonoumous driving, as well as our platform foundation for AI technology. Based on this, we illustrate how we make use of these technology into mass production product for autonomous driving.
A Multi-Scale Temporal Feature Aggregation Convolutional Neural Network for Portfolio Management
Authors: Si Shi, Jianjun Li, Guohui Li, Peng Pan
Venue: CIKM
Year: 2019

Financial portfolio management is the process of periodically reallocating a fund into different financial investment products, with the goal of achieving the maximum profits. While conventional financial machine learning methods try to predict the price trends, reinforcement learning based portfolio management methods makes trading decisions according to the price changes directly. However, existing reinforcement learning based methods are limited in extracting the price change information at single-scale level, which makes their performance still not satisfactory. In this paper, inspired by the Inception network that has achieved great success in computer vision and can extract multi-scale features simultaneously, we propose a novel Ensemble of Identical Independent Inception (EI$^3$) convolutional neural network, with the objective of addressing the limitation of existing reinforcement learning based portfolio management methods. With EI$^3$, multiple assets can be processed independently while sharing the same network parameters. Moreover, price movement information for each product can be extracted at multiple scales via wide network and then aggregated to make trading decision. Based on EI$^3$, we further propose a recurrent reinforcement learning framework to provide a deep machine learning solution for the portfolio management problem. Comprehensive experiments on the cryptocurrency datasets demonstrate the superiority of our method over existing competitors, in both upswing and downswing environments.
Document-Level Multi-Aspect Sentiment Classification for Online Reviews of Medical Experts
Authors: Tian Shi, Vineeth Rakesh, Suhang Wang, Chandan K. Reddy
Venue: CIKM
Year: 2019

In the era of big data, online doctor review platforms, which enable patients to give feedback to their doctors, have become one of the most important components in healthcare systems. On one hand, they help patients to choose their doctors based on the experience of others. On the other hand, they help doctors to improve the quality of their service. Moreover, they provide important sources for us to discover common concerns of patients and existing problems in clinics, which potentially improve current healthcare systems. In this paper, we systematically investigate the dataset from one of such review platform, namely, ratemds.com, where each review for a doctor comes with an overall rating and ratings of four different aspects. A comprehensive statistical analysis is conducted first for reviews, ratings, and doctors. Then, we explore the content of reviews by extracting latent topics related to different aspects with unsupervised topic modeling techniques. As the core component of this paper, we propose a multi-task learning framework for the document-level multi-aspect sentiment classification. This task helps us to not only recover missing aspect-level ratings and detect inconsistent rating scores but also identify aspect-keywords for a given review based on ratings. The proposed model takes both features of doctors and aspect-keywords into consideration. Extensive experiments have been conducted on two subsets of ratemds dataset to demonstrate the effectiveness of the proposed model.
Discovering Hypernymy in Text-Rich Heterogeneous Information Network by Exploiting Context Granularity
Authors: Yu Shi, Jiaming Shen, Yuchen Li, Naijing Zhang, Xinwei He, Zhengzhi Lou, Qi Zhu, Matthew Walker, Myunghwan Kim, Jiawei Han
Venue: CIKM
Year: 2019

Text-rich heterogeneous information networks (text-rich HINs) are ubiquitous in real-world applications. Hypernymy, also known as is-a relation or subclass-of relation, lays in the core of many knowledge graphs and benefits many downstream applications. Existing methods of hypernymy discovery either leverage textual patterns to extract explicitly mentioned hypernym-hyponym pairs, or learn a distributional representation for each term of interest based its context. These approaches rely on statistical signals from the textual corpus, and their effectiveness would therefore be hindered when the signals from the corpus are not sufficient for all terms of interest. In this work, we propose to discover hypernymy in text-rich HINs, which can introduce additional high-quality signals. We develop a new framework, named HyperMine, that exploits multi-granular contexts and combines signals from both text and network without human labeled data. HyperMine extends the definition of "context" to the scenario of text-rich HIN. For example, we can define typed nodes and communities as contexts. These contexts encode signals of different granularities and we feed them into a hypernymy inference model. HyperMine learns this model using weak supervision acquired based on high-precision textual patterns. Extensive experiments on two large real-world datasets demonstrate the effectiveness of HyperMine and the utility of modeling context granularity. We further show a case study that a high-quality taxonomy can be generated solely based on the hypernymy discovered by HyperMine.
Recent Developments of Deep Heterogeneous Information Network Analysis
Authors: Chuan Shi, Philip S. Yu
Venue: CIKM
Year: 2019

Recently, there is a surge of research on employing Heterogeneous Information Networks (HIN) to model complex interaction system, where networks compose of different types of nodes or links, since HIN contains richer structure and semantic information. Many researches develop structural analysis approaches by leveraging the rich semantic meaning of structural types of objects and links in the networks. Furthermore, recent advancement on deep learning and network embedding poses new opportunities and challenges to mine HIN, and heterogeneous network embedding, even heterogeneous graph neural network, is becoming a hot topic. In this tutorial, we will give a survey on recent developments of heterogeneous information network analysis, especially on newly emerging heterogeneous network embedding. This tutorial shall help researchers and practitioners to share new techniques for identifying and analyzing relationships in networks that integrate multiple types or sources of information.
HENA 2019: The 3rd Workshop of Heterogeneous Information Network Analysis and Applications
Authors: Chuan Shi, Yanfang Ye, Jiawei Zhang
Venue: CIKM
Year: 2019

The third International Workshop on Heterogeneous Information Network Analysis and Applications is held in Beijing, China on November 3, 2019 and is co-located with the 28th International Conference on Information and Knowledge Management. The goal of this workshop is to bring together people from these different areas and provide an opportunity for researchers and practitioners to share new techniques for identifying and analyzing relationships in networks that integrate multiple types or sources of information. This workshop has an exciting program that spans a number of subareas, including: network construction and mining, network embedding, information diffusion, knowledge graph analysis, community detection, parallel computing for network analysis, and network analysis applications. The program includes several invited speakers, lively discussion on emerging topics, and presentations of accepted original research papers.
Adaptive Feature Sampling for Recommendation with Missing Content Feature Values
Authors: Shaoyun Shi, Min Zhang, Xinxing Yu, Yongfeng Zhang, Bin Hao, Yiqun Liu, Shaoping Ma
Venue: CIKM
Year: 2019

Most recommendation algorithms mainly make use of user history interactions in the model, while these methods often suffer from the cold-start problem (user/item has no history information). On the other sides, content features help on cold-start scenarios for modeling new users or items. So it is essential to utilize content features to enhance different recommendation models. To take full advantage of content features, feature interactions such as cross features are used by some models and outperform than using raw features. However, in real-world systems, many content features are incomplete, e.g., we may know the occupation and gender of a user, but the values of other features (location, interests, etc.) are missing. This missing-feature-value (MFV) problem is harmful to the model performance, especially for models that rely heavily on rich feature interactions. Unfortunately, this problem has not been well studied previously. In this work, we propose a new adaptive "Feature Sampling'' strategy to help train different models to fit distinct scenarios, no matter for cold-start or missing feature value cases. With the help of this strategy, more feature interactions can be utilized. A novel model named CC-CC is proposed. The model takes both raw features and the feature interactions into consideration. It has a linear part to memorize useful variant information from the user or item contents and contexts (Content & Context Module), and a deep attentive neural module that models both content and collaborate information to enhance the generalization ability (Content & Collaborate Module). Both parts have feature interactions. The model is evaluated on two public datasets. Comparative results show that the proposed CC-CC model outperforms the state-of-the-art algorithms on both warm and cold scenarios significantly (up to 6.3%). To the best of our knowledge, this model is the first clear and powerful model that proposed to handle the missing feature values problem in deep neural network frameworks for recommender systems.