OpenRaiser/Intern-Atlas huggingface dataset:
On the Impact of Speech Recognition Errors in Passage Retrieval for Spoken Question Answering
Authors: Georgios Sidiropoulos, Svitlana Vakulenko, Evangelos Kanoulas
Venue: CIKM
Year: 2022

Interacting with a speech interface to query a Question Answering (QA) system is becoming increasingly popular. Typically, QA systems rely on passage retrieval to select candidate contexts and reading comprehension to extract the final answer. While there has been some attention to improving the reading comprehension part of QA systems against errors that automatic speech recognition (ASR) models introduce, the passage retrieval part remains unexplored. However, such errors can affect the performance of passage retrieval, leading to inferior end-to-end performance. To address this gap, we augment two existing large-scale passage ranking and open domain QA datasets with synthetic ASR noise and study the robustness of lexical and dense retrievers against questions with ASR noise. Furthermore, we study the generalizability of data augmentation techniques across different domains; with each domain being a different language dialect or accent. Finally, we create a new dataset with questions voiced by human users and use their transcriptions to show that the retrieval performance can further degrade when dealing with natural ASR noise instead of synthetic ASR noise.
Perturbation Effect: A Metric to Counter Misleading Validation of Feature Attribution
Authors: Ilija Simic, Vedran Sabol, Eduardo E. Veas
Venue: CIKM
Year: 2022

This paper provides evidence indicating that the most commonly used metric for validating feature attribution methods in eXplainable AI (XAI) is misleading when applied to time series data. To evaluate whether an XAI method attributes importance to relevant features, these are systematically perturbed while measuring the impact on the performance of the classifier. The assumption is that a drastic performance reduction with increasing perturbation of relevant features indicates that these are indeed relevant. We demonstrate empirically that this assumption is incomplete without considering low relevance features in the used metrics. We introduce a novel metric, the Perturbation Effect Size, and demonstrate how it complements existing metrics to offer a more faithful assessment of importance attribution. Finally, we contribute a comprehensive evaluation of attribution methods on time series data, considering the influence of perturbation methods and region size selection.
Data Oversampling with Structure Preserving Variational Learning
Authors: Indu Solomon, Senthilnath Jayavelu, Md Meftahul Ferdaus, Uttam Kumar
Venue: CIKM
Year: 2022

Traditional oversampling methods are well explored for binary and multi-class imbalanced datasets. In most cases, the data space is adapted for oversampling the imbalanced classes. It leads to various issues like poor modelling of the structure of the data, resulting in data overlapping between minority and majority classes that lead to poor classification performance of minority class(es). To overcome these limitations, we propose a novel data oversampling architecture called Structure Preserving Variational Learning (SPVL). This technique captures an uncorrelated distribution among classes in the latent space using an encoder-decoder framework. Hence, minority samples are generated in the latent space, preserving the structure of the data distribution. The improved latent space distribution (oversampled training data) is evaluated by training an MLP classifier and testing with unseen test dataset. The proposed SPVL method is applied to various benchmark datasets with i) binary and multi-class imbalance data, ii) high-dimensional data and, iii) large or small-scale data. Extensive experimental results demonstrated that the proposed SPVL technique outperforms the state-of-the-art counterparts.
ST-GAT: A Spatio-Temporal Graph Attention Network for Accurate Traffic Speed Prediction
Authors: Junho Song, Jiwon Son, Dong-Hyuk Seo, Kyungsik Han, Namhyuk Kim, Sang-Wook Kim
Venue: CIKM
Year: 2022

Spatio-temporal models, which combine GNNs (Graph Neural Networks) and RNNs (Recurrent Neural Networks), have shown state-of-the-art accuracy in traffic speed prediction. However, we find that they consider the spatial and temporal dependencies between speeds separately in the two (i.e., space and time) dimensions, thereby unable to exploit the joint-dependencies of speeds in space and time. In this paper, with the evidence via preliminary analysis, we point out the importance of considering individual dependencies between two speeds from all possible points in space and time for accurate traffic speed prediction. Then, we propose an Individual Spatio-Temporal graph (IST-graph) that represents the Individual Spatio-Temporal dependencies (IST-dependencies) very effectively and a Spatio-Temporal Graph ATtention network (ST-GAT), a novel model to predict the future traffic speeds based on the IST-graph and the attention mechanism. The results from our extensive evaluation with five real-world datasets demonstrate (1) the effectiveness of the IST-graph in modeling traffic speed data, (2) the superiority of ST-GAT over 5 state-of-the-art models (i.e., 2-33% gains) in prediction accuracy, and (3) the robustness of our ST-GAT even in abnormal traffic situations.
Exploring and Analyzing Change: The Janus Project
Authors: Divesh Srivastava, Tobias Bleifuß, Leon Bornemann, Dmitri V. Kalashnikov, Felix Naumann
Venue: CIKM
Year: 2022

Data change, all the time. The Janus project seeks to address the Variability dimension of Big Data by modeling, exploring, and analyzing such change, providing valuable insights into the evolving real world and ways in which data about it are collected and used. We start by identifying technical challenges that need to be addressed to realize the Janus vision. Towards this end, we have extracted and worked with the histories of various structured datasets, including DBLP, IMDB, open government data, and Wikipedia, for which a detailed history of every edit is available. Our DBChEx (Database Change Explorer) prototype enables interactive exploration of data and schema changes, and we show how DBChEx can help users gain valuable insights by exploring two real-world datasets, IMDB and Wikipedia infoboxes. Based on an analysis of the history of 3.5M tables on the English Wikipedia for a total of 53.8M table versions, we then illustrate the rich history of structured Wikipedia data: we show that tables are created in certain locations, they change their shape, they move, they grow, they shrink, their data change, they vanish, and they re-appear; indeed, each table has a life of its own. Finally, to help automatically interpret the useful knowledge harbored in the history of Wikipedia tables, we present recent results on two technical problems: (i) identifying Natural Keys, a particularly important piece of metadata, which serves as a primary key in tables over time and consists of attributes inherent to an entity, and (ii) matching tables, infoboxes and lists within a Wikipedia page across page revisions. We solve these problems at scale and make the resulting curated datasets available to the community to facilitate future research.
Targeted Influence with Community and Gender-Aware Seeding
Authors: Maciej Styczen, Bing-Jyue Chen, Ya-Wen Teng, Yvonne-Anne Pignolet, Lydia Y. Chen, De-Nian Yang
Venue: CIKM
Year: 2022

When spreading information over social networks, seeding algorithms selecting users to start the dissemination play a crucial role. The majority of existing seeding algorithms focus solely on maximizing the total number of reached nodes, overlooking the issue of group fairness, in particular, gender imbalance. To tackle the challenge of maximizing information spread on certain target groups, e.g., females, we introduce the concept of the community and gender-aware potential of users. We first show that the network's community structure is closely related to the gender distribution. Then, we propose an algorithm that leverages the information about community structure and its gender potential to iteratively modify a seed set such that the information spread on the target group meets the target ratio. Finally, we validate the algorithm by performing experiments on synthetic and real-world datasets. Our results show that the proposed seeding algorithm achieves not only the target ratio but also the highest information spread, compared to the state-of-the-art gender-aware seeding algorithm.
Cross-domain Recommendation via Adversarial Adaptation
Authors: Hongzu Su, Yifei Zhang, Xuejiao Yang, Hua Hua, Shuangyang Wang, Jingjing Li
Venue: CIKM
Year: 2022

Data scarcity, e.g., labeled data being either unavailable or too expensive, is a perpetual challenge of recommendation systems. Cross-domain recommendation leverages the label information in the source domain to facilitate the task in the target domain. However, in many real-world cross-domain recommendation systems, the source domain and the target domain are sampled from different data distributions, which obstructs the cross-domain knowledge transfer. In this paper, we propose to specifically align the data distributions between the source domain and the target domain to alleviate imbalanced sample distribution and thus challenge the data scarcity issue in the target domain. Technically, our proposed approach builds an adversarial adaptation (AA) framework to adversarially train the target model together with a pre-trained source model. A domain discriminator plays the two-player minmax game with the target model and guides the target model to learn domain-invariant features that can be transferred across domains. At the same time, the target model is calibrated to learn domain-specific information of the target domain. With such a formulation, the target model not only learns domain-invariant features for knowledge transfer, but also preserves domain-specific information for target recommendation. We apply the proposed method to address the issues of insufficient data and imbalanced sample distribution in real-world Click-Through Rate (CTR)/Conversion Rate (CVR) predictions on a large-scale dataset. Specifically, we formulate our approach as a plug-and-play module to boost existing recommendation systems. Extensive experiments verify that the proposed method is able to significantly improve the prediction performance on the target domain. For instance, our method can boost PLE with a performance improvement of 13.88% in terms of Area Under Curve (AUC) compared with single-domain PLE.
Learning and Mining with Noisy Labels
Authors: Masashi Sugiyama, Tongliang Liu, Bo Han, Yang Liu, Gang Niu
Venue: CIKM
Year: 2022

''Knowledge should not be accessible only to those who can pay" said Robert May, chair of UC's faculty Academic Senate. Similarly, machine learning should not be accessible only to those who can pay. Thus, machine learning should benefit to the whole world, especially for developing countries in Africa and Asia. When dataset sizes grow bigger, it is laborious and expensive to obtain clean supervision, especially for developing countries. As a result, the volume of noisy supervision becomes enormous, e.g., web-scale image and speech data with noisy labels. However, standard machine learning assumes that the supervised information is fully clean and intact. Therefore, noisy data harms the performance of most of the standard learning algorithms, and sometimes even makes existing algorithms broken down. There are bunch of theories and approaches proposed to deal with noisy data. As far as we know, learning and mining with noisy labels spans over two important ages in machine learning, data mining and knowledge management community: statistical learning (i.e., shallow learning) and deep learning. In the age of statistical learning, learning and mining with noisy labels focused on designing noise-tolerant losses or unbiased risk estimators. Nonetheless, in the age of deep learning, learning and mining with noisy labels has more options to combat with noisy labels, such as designing biased risk estimators or leveraging memorization effects of deep networks. In this tutorial, we summarize the foundations and go through the most recent noisy-label-tolerant techniques. By participating the tutorial, the audience will gain a broad knowledge of learning and mining with noisy labels from the viewpoint of statistical learning theory, deep learning, detailed analysis of typical algorithms and frameworks, and their real-world data mining applications.
Multi-Aspect Embedding of Dynamic Graphs
Authors: Aimin Sun, Zhiguo Gong
Venue: CIKM
Year: 2022

Graph embedding is regarded as one of the most advanced techniques for graph data analyses due to its significant performance. However, the majority of existing works only focus on static graphs while ignoring the ubiquitous dynamic graphs. In fact, the temporal evolution of edges in a dynamic graph sets a harsh challenge for the traditional embedding algorithms. To solve the problem, in this paper we propose a Dynamic Graph Multi-Aspect Embedding (DGMAE) to automatically learn the proper number of aspects and their distributions in each temporal duration based on a distance dependent Chinese Restaurant Process. The proposed method can encode the inherent property of varying interactions among nodes along the time and present different aspect-influences to nodes embedding. Our extensive experiments on several public datasets show the performance improvement over state-of-the-art works.
Serpens: Privacy-Preserving Inference through Conditional Separable of Convolutional Neural Networks
Authors: Longlong Sun, Hui Li, Yanguo Peng, Jiangtao Cui
Venue: CIKM
Year: 2022

With the extensive usage of convolutional neural networks (CNNs), privacy issues within practical applications have attracted much attention, especially when deep learning services are provided by third-party clouds. Many private inference schemes have been proposed, but their overheads are still too large. In this work, we find that the inference procedure of CNNs can be separated and performed synergistically by many parties. Following this observation, we present a pair of novel notions, namely separable and conditional separable, to tell whether a layer in CNNs can be exactly computed over multiple parties or not. Besides, we also prove that CNNs are conditionally separable. Accordingly, we propose Serpens, a private inference framework under multi-server settings. Serpens reduces the overhead of linear layers to almost zero, and now the computing bottleneck is ReLU. To address that, we design two secure ReLU protocols based on homomorphic encryption and random masks for two- and three-server settings. Experimental results show that Serpens is 78x-105x faster than the state-of-the-art private inference scheme in the two-server setting, and the superiority of Serpens is even larger in the three-server setting, only 11x-64x slower than performing the same inference over plaintext images.
Global and Local Feature Interaction with Vision Transformer for Few-shot Image Classification
Authors: Mingze Sun, Weizhi Ma, Yang Liu
Venue: CIKM
Year: 2022

Image classification is a classical machine learning task and has been widely used. Due to the high costs of annotation and data collection in real scenarios, few-shot learning has become a vital technique to improve image classification performances. However, most existing few-shot image classification methods only focus on modeling the global image feature or image local patches, which ignore the global-local interactions. In this study, we propose a new method, named GL-ViT, to integrate both global and local features to fully exploit the few-shot samples for image classification. Firstly, we design a feature extractor module to calculate the interactions between the global representation and local patch embeddings, where ViT is also adopted to achieve efficient and effective image representation. Then, Earth Mover's Distance is adopted to measure the similarity between two images. Abundant Experimental results on several widely-used open datasets show that GL-ViT outperforms state-of-the-art algorithms significantly, and our ablation studies also verify the effectiveness of both global-local features.
DeepScalper: A Risk-Aware Reinforcement Learning Framework to Capture Fleeting Intraday Trading Opportunities
Authors: Shuo Sun, Wanqi Xue, Rundong Wang, Xu He, Junlei Zhu, Jian Li, Bo An
Venue: CIKM
Year: 2022

Reinforcement learning (RL) techniques have shown great success in many challenging quantitative trading tasks, such as portfolio management and algorithmic trading. Especially, intraday trading is one of the most profitable and risky tasks because of the intraday behaviors of the financial market that reflect billions of rapidly fluctuating capitals. However, a vast majority of existing RL methods focus on the relatively low frequency trading scenarios (e.g., day-level) and fail to capture the fleeting intraday investment opportunities due to two major challenges: 1) how to effectively train profitable RL agents for intraday investment decision-making, which involves high-dimensional fine-grained action space; 2) how to learn meaningful multi-modality market representation to understand the intraday behaviors of the financial market at tick-level. Motivated by the efficient workflow of professional human intraday traders, we propose DeepScalper, a deep reinforcement learning framework for intraday trading to tackle the above challenges. Specifically, DeepScalper includes four components: 1) a dueling Q-network with action branching to deal with the large action space of intraday trading for efficient RL optimization; 2) a novel reward function with a hindsight bonus to encourage RL agents making trading decisions with a long-term horizon of the entire trading day; 3) an encoder-decoder architecture to learn multi-modality temporal market embedding, which incorporates both macro-level and micro-level market information; 4) a risk-aware auxiliary task to maintain a striking balance between maximizing profit and minimizing risk. Through extensive experiments on real-world market data spanning over three years on six financial futures (2 stock index and 4 treasury bond), we demonstrate that DeepScalper significantly outperforms many state-of-the-art baselines in terms of four financial criteria. Furthermore, we conduct a series of exploratory and ablative studies to analyze the contributions of each component in DeepScalper.
TinyRL: Towards Reinforcement Learning on Tiny Embedded Devices
Authors: Tomasz Szydlo, Prem Prakash Jayaraman, Yinhao Li, Graham Morgan, Rajiv Ranjan
Venue: CIKM
Year: 2022

We observe significant interest in reinforcement learning methods for real-world sensing-control scenarios driven by the sensor data streams. However, the delay introduced to the data by the communication channels may degrade the system's performance. It is especially crucial in the internet of things (IoT), where devices with constraint resources and low throughput networks are used. We demonstrate TinyRL framework, a different approach to this problem, by transferring RL algorithms knowledge to resource-limited devices. Our initial experiments point towards a successful demonstration of our technique using common microcontrollers used in IoT systems. Such devices have limited resource capability, and their regulation by processing data directly on devices without their transmission to the cloud can play a crucial role in their lifespan and usefulness.
WARNER: Weakly-Supervised Neural Network to Identify Eviction Filing Hotspots in the Absence of Court Records
Authors: Maryam Tabar, Wooyong Jung, Amulya Yadav, Owen Wilson Chavez, Ashley Flores, Dongwon Lee
Venue: CIKM
Year: 2022

The widespread eviction of tenants across the United States has metamorphosed into a challenging public-policy problem. In particular, eviction exacerbates several income-based, educational, and health inequities in society, e.g., eviction disproportionately affects low-income renting families, many of whom belong to underrepresented minority groups. Despite growing interest in understanding and mitigating the eviction crisis, there are several legal and infrastructural obstacles to data acquisition at scale that limit our understanding of the distribution of eviction across the United States. To circumvent existing challenges in data acquisition, we propose WARNER, a novel Machine Learning (ML) framework that predicts eviction filing hotspots in US counties from unlabeled satellite imagery dataset. We account for the lack of labeled training data in this domain by leveraging sociological insights to propose a novel approach to generate probabilistic labels for a subset of an unlabeled dataset of satellite imagery, which is then used to train a neural network model to identify eviction filing hotspots. Our experimental results show that WARNER acheives a higher predictive performance than several strong baselines. Further, the superiority of WARNER can be generalized to different counties across the United States. Our proposed framework has the potential to assist NGOs and policymakers in designing well-informed (data-driven) resource allocation plans to improve the nationwide housing stability. This work is conducted in collaboration with The Child Poverty Action Lab (a leading non-profit leveraging data-driven approaches to inform actions for relieving poverty and relevant problems in Dallas County, TX). The code can be accessed via https://github.com/maryam-tabar/WARNER.
Geographical Address Models in the Indian e-Commerce
Authors: Ravindra Babu Tallamraju
Venue: CIKM
Year: 2022

Unambiguous customer addresses are important for the e-Commerce companies in timely and accurate delivery of shipments. In many developing countries a prescribed structure is not usually followed in practice. It is observed that the customer addresses contain additional text such as instructions to the delivery team. Further not every address has an associated geolocation information in many countries. Thus address understanding, address classification, and clustering of similar but noisy addresses are critical. In addition to last mile delivery solutions, the models help reduce buyer fraud and understanding returns. In view of noisy nature of customer addresses and non-availability of associated geolocation, the above problems are effectively solved with the help of NLP. The proposed talk traces the challenges in the Indian addresses in the context of e-commerce, solution approaches, and their extensions during the last 8 years. The talk is based on the author's own experience, his publications as well as developments in this topic across the industry over these years.
Temporality- and Frequency-aware Graph Contrastive Learning for Temporal Network
Authors: Shiyin Tan, Jingyi You, Dongyuan Li
Venue: CIKM
Year: 2022

Graph contrastive learning (GCL) methods aim to learn more distinguishable representations by contrasting positive and negative samples. They have received increasing attention in recent years due to their wide application in recommender systems and knowledge graphs. However, almost all GCL methods are applied to static networks and can not be extended to temporal networks directly. Furthermore, recent GCL models treat low- and high-frequency nodes equally in overall training objectives, which hinders the prediction precision. To solve the aforementioned problems, in this paper, we propose a Temporality- and Frequency-aware Graph Contrastive Learning for temporal networks (TF-GCL). Specifically, to learn more diverse representations for infrequent nodes and fully explore temporal information, we first generate two augmented views from the input graph based on topological and temporal perspectives. We then design a temporality and frequency-aware objective function to maximize the agreement between node representations of the two views. Experimental results demonstrate that TF-GCL remarkably achieves more robust node representations and significantly outperforms the state-of-the-art methods on six temporal link prediction benchmark datasets. Considering the reproducibility, we release our code on Github.