pwc-archive/papers-with-abstracts huggingface dataset:
LW-GCN: A Lightweight FPGA-based Graph Convolutional Network Accelerator
Authors: Zhuofu Tao, Chen Wu, Yuan Liang, Lei He
Date: 2021-11-04 00:00:00
Tasks: CPU, GPU, Quantization

Graph convolutional networks (GCNs) have been introduced to effectively process non-euclidean graph data. However, GCNs incur large amounts of irregularity in computation and memory access, which prevents efficient use of traditional neural network accelerators. Moreover, existing dedicated GCN accelerators demand high memory volumes and are difficult to implement onto resource limited edge devices. In this work, we propose LW-GCN, a lightweight FPGA-based accelerator with a software-hardware co-designed process to tackle irregularity in computation and memory access in GCN inference. LW-GCN decomposes the main GCN operations into sparse-dense matrix multiplication (SDMM) and dense matrix multiplication (DMM). We propose a novel compression format to balance workload across PEs and prevent data hazards. Moreover, we apply data quantization and workload tiling, and map both SDMM and DMM of GCN inference onto a uniform architecture on resource limited hardware. Evaluation on GCN and GraphSAGE are performed on Xilinx Kintex-7 FPGA with three popular datasets. Compared to existing CPU, GPU, and state-of-the-art FPGA-based accelerator, LW-GCN reduces latency by up to 60x, 12x and 1.7x and increases power efficiency by up to 912x., 511x and 3.87x, respectively. Furthermore, compared with NVIDIA's latest edge GPU Jetson Xavier NX, LW-GCN achieves speedup and energy savings of 32x and 84x, respectively.
MIRACLE: Causally-Aware Imputation via Learning Missing Data Mechanisms
Authors: Trent Kyono, Yao Zhang, Alexis Bellot, Mihaela van der Schaar
Date: 2021-11-04 00:00:00
Proceeding: NeurIPS 2021 12
Tasks: Imputation

Missing data is an important problem in machine learning practice. Starting from the premise that imputation methods should preserve the causal structure of the data, we develop a regularization scheme that encourages any baseline imputation method to be causally consistent with the underlying data generating mechanism. Our proposal is a causally-aware imputation algorithm (MIRACLE). MIRACLE iteratively refines the imputation of a baseline by simultaneously modeling the missingness generating mechanism, encouraging imputation to be consistent with the causal structure of the data. We conduct extensive experiments on synthetic and a variety of publicly available datasets to show that MIRACLE is able to consistently improve imputation over a variety of benchmark methods across all three missingness scenarios: at random, completely at random, and not at random.
Value Function Spaces: Skill-Centric State Abstractions for Long-Horizon Reasoning
Authors: Dhruv Shah, Peng Xu, Yao Lu, Ted Xiao, Alexander Toshev, Sergey Levine, Brian Ichter
Date: 2021-11-04 00:00:00
Proceeding: ICLR 2022 4
Tasks: Hierarchical Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Zero-shot Generalization

Reinforcement learning can train policies that effectively perform complex tasks. However for long-horizon tasks, the performance of these methods degrades with horizon, often necessitating reasoning over and chaining lower-level skills. Hierarchical reinforcement learning aims to enable this by providing a bank of low-level skills as action abstractions. Hierarchies can further improve on this by abstracting the space states as well. We posit that a suitable state abstraction should depend on the capabilities of the available lower-level policies. We propose Value Function Spaces: a simple approach that produces such a representation by using the value functions corresponding to each lower-level skill. These value functions capture the affordances of the scene, thus forming a representation that compactly abstracts task relevant information and robustly ignores distractors. Empirical evaluations for maze-solving and robotic manipulation tasks demonstrate that our approach improves long-horizon performance and enables better zero-shot generalization than alternative model-free and model-based methods.
Explainable k-means. Don't be greedy, plant bigger trees!
Authors: Konstantin Makarychev, Liren Shan
Date: 2021-11-04 00:00:00
Tasks: Clustering

We provide a new bi-criteria $\tilde{O}(\log^2 k)$ competitive algorithm for explainable $k$-means clustering. Explainable $k$-means was recently introduced by Dasgupta, Frost, Moshkovitz, and Rashtchian (ICML 2020). It is described by an easy to interpret and understand (threshold) decision tree or diagram. The cost of the explainable $k$-means clustering equals to the sum of costs of its clusters; and the cost of each cluster equals the sum of squared distances from the points in the cluster to the center of that cluster. The best non bi-criteria algorithm for explainable clustering $\tilde{O}(k)$ competitive, and this bound is tight. Our randomized bi-criteria algorithm constructs a threshold decision tree that partitions the data set into $(1+\delta)k$ clusters (where $\delta\in (0,1)$ is a parameter of the algorithm). The cost of this clustering is at most $\tilde{O}(1/ \delta \cdot \log^2 k)$ times the cost of the optimal unconstrained $k$-means clustering. We show that this bound is almost optimal.
Compressing Sensor Data for Remote Assistance of Autonomous Vehicles using Deep Generative Models
Authors: Daniel Bogdoll, Johannes Jestram, Jonas Rauch, Christin Scheib, Moritz Wittig, J. Marius Zöllner
Date: 2021-11-05 00:00:00
Tasks: Autonomous Vehicles, Data Compression

In the foreseeable future, autonomous vehicles will require human assistance in situations they can not resolve on their own. In such scenarios, remote assistance from a human can provide the required input for the vehicle to continue its operation. Typical sensors used in autonomous vehicles include camera and lidar sensors. Due to the massive volume of sensor data that must be sent in real-time, highly efficient data compression is elementary to prevent an overload of network infrastructure. Sensor data compression using deep generative neural networks has been shown to outperform traditional compression approaches for both image and lidar data, regarding compression rate as well as reconstruction quality. However, there is a lack of research about the performance of generative-neural-network-based compression algorithms for remote assistance. In order to gain insights into the feasibility of deep generative models for usage in remote assistance, we evaluate state-of-the-art algorithms regarding their applicability and identify potential weaknesses. Further, we implement an online pipeline for processing sensor data and demonstrate its performance for remote assistance using the CARLA simulator.
Augmentations in Graph Contrastive Learning: Current Methodological Flaws & Towards Better Practices
Authors: Puja Trivedi, Ekdeep Singh Lubana, Yujun Yan, Yaoqing Yang, Danai Koutra
Date: 2021-11-05 00:00:00
Tasks: Classification, Contrastive Learning, Data Augmentation, Document Classification, Fake News Detection, Graph Classification, Graph Representation Learning, Inductive Bias, Representation Learning

Unsupervised graph representation learning is critical to a wide range of applications where labels may be scarce or expensive to procure. Contrastive learning (CL) is an increasingly popular paradigm for such settings and the state-of-the-art in unsupervised visual representation learning. Recent work attributes the success of visual CL to use of task-relevant augmentations and large, diverse datasets. Interestingly, graph CL frameworks report strong performance despite using orders of magnitude smaller datasets and employing domain-agnostic graph augmentations (DAGAs). Motivated by this discrepancy, we probe the quality of representations learnt by popular graph CL frameworks using DAGAs. We find that DAGAs can destroy task-relevant information and harm the model's ability to learn discriminative representations. On small benchmark datasets, we show the inductive bias of graph neural networks can significantly compensate for this weak discriminability. Based on our findings, we propose several sanity checks that enable practitioners to quickly assess the quality of their model's learned representations. We further propose a broad strategy for designing task-aware augmentations that are amenable to graph CL and demonstrate its efficacy on two large-scale, complex graph applications. For example, in graph-based document classification, we show task-aware augmentations improve accuracy up to 20%.
Dynamic Data Augmentation with Gating Networks for Time Series Recognition
Authors: Daisuke Oba, Shinnosuke Matsuo, Brian Kenji Iwana
Date: 2021-11-05 00:00:00
Tasks: Data Augmentation, Time Series, Time Series Analysis

Data augmentation is a technique to improve the generalization ability of machine learning methods by increasing the size of the dataset. However, since every augmentation method is not equally effective for every dataset, you need to select an appropriate method carefully. We propose a neural network that dynamically selects the best combination of data augmentation methods using a mutually beneficial gating network and a feature consistency loss. The gating network is able to control how much of each data augmentation is used for the representation within the network. The feature consistency loss gives a constraint that augmented features from the same input should be in similar. In experiments, we demonstrate the effectiveness of the proposed method on the 12 largest time-series datasets from 2018 UCR Time Series Archive and reveal the relationships between the data augmentation methods through analysis of the proposed method.
CGCL: Collaborative Graph Contrastive Learning without Handcrafted Graph Data Augmentations
Authors: Tianyu Zhang, Yuxiang Ren, Wenzheng Feng, Weitao Du, Xuecang Zhang
Date: 2021-11-05 00:00:00
Tasks: Contrastive Learning, Data Augmentation, Graph Representation Learning, Representation Learning, Self-Supervised Learning

Unsupervised graph representation learning is a non-trivial topic. The success of contrastive methods in the unsupervised representation learning on structured data inspires similar attempts on the graph. Existing graph contrastive learning (GCL) aims to learn the invariance across multiple augmentation views, which renders it heavily reliant on the handcrafted graph augmentations. However, inappropriate graph data augmentations can potentially jeopardize such invariance. In this paper, we show the potential hazards of inappropriate augmentations and then propose a novel Collaborative Graph Contrastive Learning framework (CGCL). This framework harnesses multiple graph encoders to observe the graph. Features observed from different encoders serve as the contrastive views in contrastive learning, which avoids inducing unstable perturbation and guarantees the invariance. To ensure the collaboration among diverse graph encoders, we propose the concepts of asymmetric architecture and complementary encoders as the design principle. To further prove the rationality, we utilize two quantitative metrics to measure the assembly of CGCL respectively. Extensive experiments demonstrate the advantages of CGCL in unsupervised graph-level representation learning and the potential of collaborative framework. The source code for reproducibility is available at https://github.com/zhangtia16/CGCL
Graph Denoising with Framelet Regularizer
Authors: Bingxin Zhou, Ruikun Li, Xuebin Zheng, Yu Guang Wang, Junbin Gao
Date: 2021-11-05 00:00:00
Tasks: Denoising

As graph data collected from the real world is merely noise-free, a practical representation of graphs should be robust to noise. Existing research usually focuses on feature smoothing but leaves the geometric structure untouched. Furthermore, most work takes L2-norm that pursues a global smoothness, which limits the expressivity of graph neural networks. This paper tailors regularizers for graph data in terms of both feature and structure noises, where the objective function is efficiently solved with the alternating direction method of multipliers (ADMM). The proposed scheme allows to take multiple layers without the concern of over-smoothing, and it guarantees convergence to the optimal solutions. Empirical study proves that our model achieves significantly better performance compared with popular graph convolutions even when the graph is heavily contaminated.
EpilNet: A Novel Approach to IoT based Epileptic Seizure Prediction and Diagnosis System using Artificial Intelligence
Authors: Shivam Gupta, Virender Ranga, Priyansh Agrawal
Date: 2021-11-05 00:00:00
Tasks: EEG, Electroencephalogram (EEG), Seizure prediction

Epilepsy is one of the most occurring neurological diseases. The main characteristic of this disease is a frequent seizure, which is an electrical imbalance in the brain. It is generally accompanied by shaking of body parts and even leads (fainting). In the past few years, many treatments have come up. These mainly involve the use of anti-seizure drugs for controlling seizures. But in 70% of cases, these drugs are not effective, and surgery is the only solution when the condition worsens. So patients need to take care of themselves while having a seizure and be safe. Wearable electroencephalogram (EEG) devices have come up with the development in medical science and technology. These devices help in the analysis of brain electrical activities. EEG helps in locating the affected cortical region. The most important is that it can predict any seizure in advance on-site. This has resulted in a sudden increase in demand for effective and efficient seizure prediction and diagnosis systems. A novel approach to epileptic seizure prediction and diagnosis system EpilNet is proposed in the present paper. It is a one-dimensional (1D) convolution neural network. EpilNet gives the testing accuracy of 79.13% for five classes, leading to a significant increase of about 6-7% compared to related works. The developed Web API helps in bringing EpilNet into practical use. Thus, it is an integrated system for both patients and doctors. The system will help patients prevent injury or accidents and increase the efficiency of the treatment process by doctors in the hospitals.
Interpretable Personalized Experimentation
Authors: Han Wu, Sarah Tan, Weiwei Li, Mia Garrard, Adam Obeng, Drew Dimmery, Shaun Singh, Hanson Wang, Daniel Jiang, Eytan Bakshy
Date: 2021-11-05 00:00:00

Black-box heterogeneous treatment effect (HTE) models are increasingly being used to create personalized policies that assign individuals to their optimal treatments. However, they are difficult to understand, and can be burdensome to maintain in a production environment. In this paper, we present a scalable, interpretable personalized experimentation system, implemented and deployed in production at Meta. The system works in a multiple treatment, multiple outcome setting typical at Meta to: (1) learn explanations for black-box HTE models; (2) generate interpretable personalized policies. We evaluate the methods used in the system on publicly available data and Meta use cases, and discuss lessons learnt during the development of the system.
Neural Network Based Epileptic EEG Detection and Classification
Authors: Shivam Gupta, Jyoti Meena, O. P Gupta
Date: 2021-11-05 00:00:00
Tasks: Binary Classification, Classification, EEG, Electroencephalogram (EEG), Sensitivity, Specificity

Timely diagnosis is important for saving the life of epileptic patients. In past few years, a lot of treatments are available for epilepsy. These treatments require use of anti-seizure drugs but are not effective in controlling frequency of seizure. There is need of removal of an affected region using surgery. Electroencephalogram (EEG) is a widely used technique for monitoring the brain activity and widely popular for seizure region detection. It is used before surgery for locating affected region. This manual process, using EEG graphs, is time consuming and requires deep expertise. In the present paper, a model has been proposed that preserves the true nature of an EEG signal in form of textual one-dimensional vector. The proposed model achieves a state of art performance for Bonn University dataset giving an average sensitivity, specificity of 81% and 81.4% respectively for classification of EEG data among all five classes. Also for binary classification achieving 99.9%, 99.5% score value for specificity and sensitivity instead of 2D models used by other researchers. Thus, developed system will significantly help neurosurgeons in the increase of their performance.
Automated Human Mind Reading Using EEG Signals for Seizure Detection
Authors: Virender Ranga, Shivam Gupta, Jyoti Meena, Priyansh Agrawal
Date: 2021-11-05 00:00:00
Tasks: EEG, Electroencephalogram (EEG), Seizure Detection

Epilepsy is one of the most occurring neurological disease globally emerged back in 4000 BC. It is affecting around 50 million people of all ages these days. The trait of this disease is recurrent seizures. In the past few decades, the treatments available for seizure control have improved a lot with the advancements in the field of medical science and technology. Electroencephalogram (EEG) is a widely used technique for monitoring the brain activity and widely popular for seizure region detection. It is performed before surgery and also to predict seizure at the time operation which is useful in neuro stimulation device. But in most of cases visual examination is done by neurologist in order to detect and classify patterns of the disease but this requires a lot of pre-domain knowledge and experience. This all in turns put a pressure on neurosurgeons and leads to time wastage and also reduce their accuracy and efficiency. There is a need of some automated systems in arena of information technology like use of neural networks in deep learning which can assist neurologists. In the present paper, a model is proposed to give an accuracy of 98.33% which can be used for development of automated systems. The developed system will significantly help neurologists in their performance.
Dataset of Fake News Detection and Fact Verification: A Survey
Authors: Taichi Murayama
Date: 2021-11-05 00:00:00
Tasks: Fact Verification, Fake News Detection, Satire Detection, Survey

The rapid increase in fake news, which causes significant damage to society, triggers many fake news related studies, including the development of fake news detection and fact verification techniques. The resources for these studies are mainly available as public datasets taken from Web data. We surveyed 118 datasets related to fake news research on a large scale from three perspectives: (1) fake news detection, (2) fact verification, and (3) other tasks; for example, the analysis of fake news and satire detection. We also describe in detail their utilization tasks and their characteristics. Finally, we highlight the challenges in the fake news dataset construction and some research opportunities that address these challenges. Our survey facilitates fake news research by helping researchers find suitable datasets without reinventing the wheel, and thereby, improves fake news studies in depth.
Learning on Random Balls is Sufficient for Estimating (Some) Graph Parameters
Authors: Takanori Maehara, Hoang NT
Date: 2021-11-05 00:00:00
Proceeding: NeurIPS 2021 12
Tasks: Classification, Generalization Bounds, Graph Classification, Graph Learning

Theoretical analyses for graph learning methods often assume a complete observation of the input graph. Such an assumption might not be useful for handling any-size graphs due to the scalability issues in practice. In this work, we develop a theoretical framework for graph classification problems in the partial observation setting (i.e., subgraph samplings). Equipped with insights from graph limit theory, we propose a new graph classification model that works on a randomly sampled subgraph and a novel topology to characterize the representability of the model. Our theoretical framework contributes a theoretical validation of mini-batch learning on graphs and leads to new learning-theoretic results on generalization bounds as well as size-generalizability without assumptions on the input.
DVFL: A Vertical Federated Learning Method for Dynamic Data
Authors: Yuzhi Liang, Yixiang Chen
Date: 2021-11-05 00:00:00
Tasks: Federated Learning, Knowledge Distillation, Privacy Preserving, Vertical Federated Learning

Federated learning, which solves the problem of data island by connecting multiple computational devices into a decentralized system, has become a promising paradigm for privacy-preserving machine learning. This paper studies vertical federated learning (VFL), which tackles the scenarios where collaborating organizations share the same set of users but disjoint features. Contemporary VFL methods are mainly used in static scenarios where the active party and the passive party have all the data from the beginning and will not change. However, the data in real life often changes dynamically. To alleviate this problem, we propose a new vertical federation learning method, DVFL, which adapts to dynamic data distribution changes through knowledge distillation. In DVFL, most of the computations are held locally to improve data security and model efficiency. Our extensive experimental results show that DVFL can not only obtain results close to existing VFL methods in static scenes, but also adapt to changes in data distribution in dynamic scenarios.
Coherent Probabilistic Aggregate Queries on Long-horizon Forecasts
Authors: Prathamesh Deshpande, Sunita Sarawagi
Date: 2021-11-05 00:00:00
Tasks: Time Series, Time Series Analysis, Time Series Forecasting

Long range forecasts are the starting point of many decision support systems that need to draw inference from high-level aggregate patterns on forecasted values. State of the art time-series forecasting methods are either subject to concept drift on long-horizon forecasts, or fail to accurately predict coherent and accurate high-level aggregates. In this work, we present a novel probabilistic forecasting method that produces forecasts that are coherent in terms of base level and predicted aggregate statistics. We achieve the coherency between predicted base-level and aggregate statistics using a novel inference method based on KL-divergence that can be solved efficiently in closed form. We show that our method improves forecast performance across both base level and unseen aggregates post inference on real datasets ranging three diverse domains. (\href{https://github.com/pratham16cse/AggForecaster}{Project URL})
Dual Parameterization of Sparse Variational Gaussian Processes
Authors: Vincent Adam, Paul E. Chang, Mohammad Emtiyaz Khan, Arno Solin
Date: 2021-11-05 00:00:00
Proceeding: NeurIPS 2021 12
Tasks: Computational Efficiency, Gaussian Processes

Sparse variational Gaussian process (SVGP) methods are a common choice for non-conjugate Gaussian process inference because of their computational benefits. In this paper, we improve their computational efficiency by using a dual parameterization where each data example is assigned dual parameters, similarly to site parameters used in expectation propagation. Our dual parameterization speeds-up inference using natural gradient descent, and provides a tighter evidence lower bound for hyperparameter learning. The approach has the same memory cost as the current SVGP methods, but it is faster and more accurate.