pwc-archive/papers-with-abstracts huggingface dataset:
GraphCrop: Subgraph Cropping for Graph Classification
Authors: Yiwei Wang, Wei Wang, Yuxuan Liang, Yujun Cai, Bryan Hooi
Date: 2020-09-22 00:00:00
Tasks: Classification, Data Augmentation, General Classification, Graph Classification, valid

We present a new method to regularize graph neural networks (GNNs) for better generalization in graph classification. Observing that the omission of sub-structures does not necessarily change the class label of the whole graph, we develop the \textbf{GraphCrop} (Subgraph Cropping) data augmentation method to simulate the real-world noise of sub-structure omission. In principle, GraphCrop utilizes a node-centric strategy to crop a contiguous subgraph from the original graph while maintaining its connectivity. By preserving the valid structure contexts for graph classification, we encourage GNNs to understand the content of graph structures in a global sense, rather than rely on a few key nodes or edges, which may not always be present. GraphCrop is parameter learning free and easy to implement within existing GNN-based graph classifiers. Qualitatively, GraphCrop expands the existing training set by generating novel and informative augmented graphs, which retain the original graph labels in most cases. Quantitatively, GraphCrop yields significant and consistent gains on multiple standard datasets, and thus enhances the popular GNNs to outperform the baseline methods.
When Deep Reinforcement Learning Meets Federated Learning: Intelligent Multi-Timescale Resource Management for Multi-access Edge Computing in 5G Ultra Dense Network
Authors: Shuai Yu, Xu Chen, Zhi Zhou, Xiaowen Gong, Di wu
Date: 2020-09-22 00:00:00
Tasks: Decision Making, Deep Reinforcement Learning, Edge-computing, Federated Learning, Management

Ultra-dense edge computing (UDEC) has great potential, especially in the 5G era, but it still faces challenges in its current solutions, such as the lack of: i) efficient utilization of multiple 5G resources (e.g., computation, communication, storage and service resources); ii) low overhead offloading decision making and resource allocation strategies; and iii) privacy and security protection schemes. Thus, we first propose an intelligent ultra-dense edge computing (I-UDEC) framework, which integrates blockchain and Artificial Intelligence (AI) into 5G ultra-dense edge computing networks. First, we show the architecture of the framework. Then, in order to achieve real-time and low overhead computation offloading decisions and resource allocation strategies, we design a novel two-timescale deep reinforcement learning (\textit{2Ts-DRL}) approach, consisting of a fast-timescale and a slow-timescale learning process, respectively. The primary objective is to minimize the total offloading delay and network resource usage by jointly optimizing computation offloading, resource allocation and service caching placement. We also leverage federated learning (FL) to train the \textit{2Ts-DRL} model in a distributed manner, aiming to protect the edge devices' data privacy. Simulation results corroborate the effectiveness of both the \textit{2Ts-DRL} and FL in the I-UDEC framework and prove that our proposed algorithm can reduce task execution time up to 31.87%.
Evolutionary Architecture Search for Graph Neural Networks
Authors: Min Shi, David A. Wilson, Xingquan Zhu, Yu Huang, Yuan Zhuang, Jianxun Liu, Yufei Tang
Date: 2020-09-21 00:00:00
Tasks: AutoML, Neural Architecture Search, Representation Learning

Automated machine learning (AutoML) has seen a resurgence in interest with the boom of deep learning over the past decade. In particular, Neural Architecture Search (NAS) has seen significant attention throughout the AutoML research community, and has pushed forward the state-of-the-art in a number of neural models to address grid-like data such as texts and images. However, very litter work has been done about Graph Neural Networks (GNN) learning on unstructured network data. Given the huge number of choices and combinations of components such as aggregator and activation function, determining the suitable GNN structure for a specific problem normally necessitates tremendous expert knowledge and laborious trails. In addition, the slight variation of hyper parameters such as learning rate and dropout rate could dramatically hurt the learning capacity of GNN. In this paper, we propose a novel AutoML framework through the evolution of individual models in a large GNN architecture space involving both neural structures and learning parameters. Instead of optimizing only the model structures with fixed parameter settings as existing work, an alternating evolution process is performed between GNN structures and learning parameters to dynamically find the best fit of each other. To the best of our knowledge, this is the first work to introduce and evaluate evolutionary architecture search for GNN models. Experiments and validations demonstrate that evolutionary NAS is capable of matching existing state-of-the-art reinforcement learning approaches for both the semi-supervised transductive and inductive node representation learning and classification.
Multi-threaded Memory Efficient Crossover in C++ for Generational Genetic Programming
Authors: W. B. Langdon
Date: 2020-09-22 00:00:00
Tasks: C++ code, Evolutionary Algorithms

C++ code snippets from a multi-core parallel memory-efficient crossover for genetic programming are given. They may be adapted for separate generation evolutionary algorithms where large chromosomes or small RAM require no more than M + (2 times nthreads) simultaneously active individuals.
Complex Vehicle Routing with Memory Augmented Neural Networks
Authors: Marijn van Knippenberg, Mike Holenderski, Vlado Menkovski
Date: 2020-09-22 00:00:00
Tasks: Combinatorial Optimization, Deep Learning

Complex real-life routing challenges can be modeled as variations of well-known combinatorial optimization problems. These routing problems have long been studied and are difficult to solve at scale. The particular setting may also make exact formulation difficult. Deep Learning offers an increasingly attractive alternative to traditional solutions, which mainly revolve around the use of various heuristics. Deep Learning may provide solutions which are less time-consuming and of higher quality at large scales, as it generally does not need to generate solutions in an iterative manner, and Deep Learning models have shown a surprising capacity for solving complex tasks in recent years. Here we consider a particular variation of the Capacitated Vehicle Routing (CVRP) problem and investigate the use of Deep Learning models with explicit memory components. Such memory components may help in gaining insight into the model's decisions as the memory and operations on it can be directly inspected at any time, and may assist in scaling the method to such a size that it becomes viable for industry settings.
Tensor Programs III: Neural Matrix Laws
Authors: Greg Yang
Date: 2020-09-22 00:00:00

In a neural network (NN), *weight matrices* linearly transform inputs into *preactivations* that are then transformed nonlinearly into *activations*. A typical NN interleaves multitudes of such linear and nonlinear transforms to express complex functions. Thus, the (pre-)activations depend on the weights in an intricate manner. We show that, surprisingly, (pre-)activations of a randomly initialized NN become *independent* from the weights as the NN's widths tend to infinity, in the sense of asymptotic freeness in random matrix theory. We call this the Free Independence Principle (FIP), which has these consequences: 1) It rigorously justifies the calculation of asymptotic Jacobian singular value distribution of an NN in Pennington et al. [36,37], essential for training ultra-deep NNs [48]. 2) It gives a new justification of gradient independence assumption used for calculating the Neural Tangent Kernel of a neural network. FIP and these results hold for any neural architecture. We show FIP by proving a Master Theorem for any Tensor Program, as introduced in Yang [50,51], generalizing the Master Theorems proved in those works. As warmup demonstrations of this new Master Theorem, we give new proofs of the semicircle and Marchenko-Pastur laws, which benchmarks our framework against these fundamental mathematical results.
Claraprint: a chord and melody based fingerprint for western classical music cover detection
Authors: Mickaël Arcos
Date: 2020-09-21 00:00:00
Tasks: Information Retrieval, Music Information Retrieval, Retrieval

Cover song detection has been an active field in the Music Information Retrieval (MIR) community during the past decades. Most of the research community focused in solving it for a wide range of music genres with diverse characteristics. Western classical music, a genre heavily based on the recording of "cover songs", or musical works, represents a large heritage, offering immediate application for an efficient fingerprint algorithm. We propose an engineering approach for retrieving a cover song from a reference database thanks to a fingerprint designed for classical musical works. We open a new data set to encourage the scientific community to use it for further researches regarding this genre.
Embedding-based Zero-shot Retrieval through Query Generation
Authors: Davis Liang, Peng Xu, Siamak Shakeri, Cicero Nogueira dos Santos, Ramesh Nallapati, Zhiheng Huang, Bing Xiang
Date: 2020-09-22 00:00:00
Tasks: Passage Retrieval, Retrieval

Passage retrieval addresses the problem of locating relevant passages, usually from a large corpus, given a query. In practice, lexical term-matching algorithms like BM25 are popular choices for retrieval owing to their efficiency. However, term-based matching algorithms often miss relevant passages that have no lexical overlap with the query and cannot be finetuned to downstream datasets. In this work, we consider the embedding-based two-tower architecture as our neural retrieval model. Since labeled data can be scarce and because neural retrieval models require vast amounts of data to train, we propose a novel method for generating synthetic training data for retrieval. Our system produces remarkable results, significantly outperforming BM25 on 5 out of 6 datasets tested, by an average of 2.45 points for Recall@1. In some cases, our model trained on synthetic data can even outperform the same model trained on real data
A Low-Level Index for Distributed Logic Programming
Authors: Thomas Prokosch
Date: 2020-09-22 00:00:00

A distributed logic programming language with support for meta-programming and stream processing offers a variety of interesting research problems, such as: How can a versatile and stable data structure for the indexing of a large number of expressions be implemented with simple low-level data structures? Can low-level programming help to reduce the number of occur checks in Robinson's unification algorithm? This article gives the answers.
TextFuseNet: Scene Text Detection with Richer Fused Features
Authors: Jian Ye, Zhe Chen, Juhua Liu, Bo Du
Date: 2020-05-17 00:00:00
Tasks: Scene Text Detection, Text Detection

Arbitrary shape text detection in natural scenes is an extremely challenging task. Unlike existing text detection approaches that only perceive texts based on limited feature representations, we propose a novel framework, namely TextFuseNet, to exploit the use of richer features fused for text detection. More specifically, we propose to perceive texts from three levels of feature representations, i.e., character-, word- and global-level, and then introduce a novel text representation fusion technique to help achieve robust arbitrary text detection. The multi-level feature representation can adequately describe texts by dissecting them into individual characters while still maintaining their general semantics. TextFuseNet then collects and merges the texts’ features from different levels using a multi-path fusion architecture which can effectively align and fuse different representations. In practice, our proposed TextFuseNet can learn a more adequate description of arbitrary shapes texts, suppressing false positives and producing more accurate detection results. Our proposed framework can also be trained with weak supervision for those datasets that lack character-level annotations. Experiments on several datasets show that the proposed TextFuseNet achieves state-of-the-art performance. Specifically, we achieve an F-measure of 94.3% on ICDAR2013, 92.1% on ICDAR2015, 87.1% on Total-Text and 86.6% on CTW-1500, respectively.
Text-based Person Search via Attribute-aided Matching
Authors: Surbhi Aggarwal R., Venkatesh Babu, Anirban Chakraborty
Date: 2020-03-14 00:00:00
Tasks: Attribute, Person Search, Representation Learning, Retrieval, Text based Person Retrieval, Text based Person Search

Text-based person search aims to retrieve the pedestrian images that best match a given text query. Existing
methods utilize class-id information to get discriminative
and identity-preserving features. However, it is not wellexplored whether it is beneficial to explicitly ensure that the
semantics of the data are retained. In the proposed work, we
aim to create semantics-preserving embeddings through an
additional task of attribute prediction. Since attribute annotation is typically unavailable in text-based person search,
we first mine them from the text corpus. These attributes are
then used as a means to bridge the modality gap between the
image-text inputs, as well as to improve the representation
learning. In summary, we propose an approach for textbased person search by learning an attribute-driven space
along with a class-information driven space, and utilize
both for obtaining the retrieval results. Our experiments on
benchmark dataset, CUHK-PEDES, show that learning the
attribute-space not only helps in improving performance,
giving us state-of-the-art Rank-1 accuracy of 56.68%, but
also yields humanly-interpretable features.
Focused Clinical Query Understanding and Retrieval of Medical Snippets powered through a Healthcare Knowledge Graph
Authors: Maulik R. Kamdar, Michael Carroll, Will Dowling, Linda Wogulis, Cailey Fitzgerald, Matt Corkum, Danielle Walsh, David Conrad, Craig E. Stanley, Jr., Steve Ross, Dru Henke, Mevan Samarasinghe
Date: 2020-09-17 00:00:00
Tasks: Retrieval

Clinicians face several significant barriers to search and synthesize accurate, succinct, updated, and trustworthy medical information from several literature sources during the practice of medicine and patient care. In this talk, we will be presenting our research behind the development of a Focused Clinical Search Service, powered by a Healthcare Knowledge Graph, to interpret the query intent behind clinical search queries and retrieve relevant medical snippets from a diverse corpus of medical literature.
Spatio-Temporal Hybrid Graph Convolutional Network for Traffic Forecasting in Telecommunication Networks
Authors: Marcus Kalander, Min Zhou, Chengzhi Zhang, Hanling Yi, Lujia Pan
Date: 2020-09-17 00:00:00

Telecommunication networks play a critical role in modern society. With the arrival of 5G networks, these systems are becoming even more diversified, integrated, and intelligent. Traffic forecasting is one of the key components in such a system, however, it is particularly challenging due to the complex spatial-temporal dependency. In this work, we consider this problem from the aspect of a cellular network and the interactions among its base stations. We thoroughly investigate the characteristics of cellular network traffic and shed light on the dependency complexities based on data collected from a densely populated metropolis area. Specifically, we observe that the traffic shows both dynamic and static spatial dependencies as well as diverse cyclic temporal patterns. To address these complexities, we propose an effective deep-learning-based approach, namely, Spatio-Temporal Hybrid Graph Convolutional Network (STHGCN). It employs GRUs to model the temporal dependency, while capturing the complex spatial dependency through a hybrid-GCN from three perspectives: spatial proximity, functional similarity, and recent trend similarity. We conduct extensive experiments on real-world traffic datasets collected from telecommunication networks. Our experimental results demonstrate the superiority of the proposed model in that it consistently outperforms both classical methods and state-of-the-art deep learning models, while being more robust and stable.
Force2Vec: Parallel force-directed graph embedding
Authors: Md. Khaledur Rahman, Majedul Haque Sujon, Ariful Azad
Date: 2020-09-17 00:00:00
Tasks: Clustering, Graph Embedding, Link Prediction, Node Classification

A graph embedding algorithm embeds a graph into a low-dimensional space such that the embedding preserves the inherent properties of the graph. While graph embedding is fundamentally related to graph visualization, prior work did not exploit this connection explicitly. We develop Force2Vec that uses force-directed graph layout models in a graph embedding setting with an aim to excel in both machine learning (ML) and visualization tasks. We make Force2Vec highly parallel by mapping its core computations to linear algebra and utilizing multiple levels of parallelism available in modern processors. The resultant algorithm is an order of magnitude faster than existing methods (43x faster than DeepWalk, on average) and can generate embeddings from graphs with billions of edges in a few hours. In comparison to existing methods, Force2Vec is better in graph visualization and performs comparably or better in ML tasks such as link prediction, node classification, and clustering. Source code is available at https://github.com/HipGraph/Force2Vec.
Cross-Modal Alignment with Mixture Experts Neural Network for Intral-City Retail Recommendation
Authors: Po Li, Lei LI, Yan Fu, Jun Rong, Yu Zhang
Date: 2020-09-17 00:00:00
Tasks: cross-modal alignment, Image to text, Recommendation Systems

In this paper, we introduce Cross-modal Alignment with mixture experts Neural Network (CameNN) recommendation model for intral-city retail industry, which aims to provide fresh foods and groceries retailing within 5 hours delivery service arising for the outbreak of Coronavirus disease (COVID-19) pandemic around the world. We propose CameNN, which is a multi-task model with three tasks including Image to Text Alignment (ITA) task, Text to Image Alignment (TIA) task and CVR prediction task. We use pre-trained BERT to generate the text embedding and pre-trained InceptionV4 to generate image patch embedding (each image is split into small patches with the same pixels and treat each patch as an image token). Softmax gating networks follow to learn the weight of each transformer expert output and choose only a subset of experts conditioned on the input. Then transformer encoder is applied as the share-bottom layer to learn all input features' shared interaction. Next, mixture of transformer experts (MoE) layer is implemented to model different aspects of tasks. At top of the MoE layer, we deploy a transformer layer for each task as task tower to learn task-specific information. On the real word intra-city dataset, experiments demonstrate CameNN outperform baselines and achieve significant improvements on the image and text representation. In practice, we applied CameNN on CVR prediction in our intra-city recommender system which is one of the leading intra-city platforms operated in China.
Finding Influential Instances for Distantly Supervised Relation Extraction
Authors: Zifeng Wang, Rui Wen, Xi Chen, Shao-Lun Huang, Ningyu Zhang, Yefeng Zheng
Date: 2020-09-17 00:00:00
Proceeding: COLING 2022 10
Tasks: Relation, Relation Extraction

Distant supervision (DS) is a strong way to expand the datasets for enhancing relation extraction (RE) models but often suffers from high label noise. Current works based on attention, reinforcement learning, or GAN are black-box models so they neither provide meaningful interpretation of sample selection in DS nor stability on different domains. On the contrary, this work proposes a novel model-agnostic instance sampling method for DS by influence function (IF), namely REIF. Our method identifies favorable/unfavorable instances in the bag based on IF, then does dynamic instance sampling. We design a fast influence sampling algorithm that reduces the computational complexity from $\mathcal{O}(mn)$ to $\mathcal{O}(1)$, with analyzing its robustness on the selected sampling function. Experiments show that by simply sampling the favorable instances during training, REIF is able to win over a series of baselines that have complicated architectures. We also demonstrate that REIF can support interpretable instance selection.
Energy-based Surprise Minimization for Multi-Agent Value Factorization
Authors: Karush Suri, Xiao Qi Shi, Konstantinos Plataniotis, Yuri Lawryshyn
Date: 2020-09-16 00:00:00
Tasks: Multi-agent Reinforcement Learning, Q-Learning, Starcraft, Starcraft II

Multi-Agent Reinforcement Learning (MARL) has demonstrated significant success in training decentralised policies in a centralised manner by making use of value factorization methods. However, addressing surprise across spurious states and approximation bias remain open problems for multi-agent settings. Towards this goal, we introduce the Energy-based MIXer (EMIX), an algorithm which minimizes surprise utilizing the energy across agents. Our contributions are threefold; (1) EMIX introduces a novel surprise minimization technique across multiple agents in the case of multi-agent partially-observable settings. (2) EMIX highlights a practical use of energy functions in MARL with theoretical guarantees and experiment validations of the energy operator. Lastly, (3) EMIX extends Maxmin Q-learning for addressing overestimation bias across agents in MARL. In a study of challenging StarCraft II micromanagement scenarios, EMIX demonstrates consistent stable performance for multiagent surprise minimization. Moreover, our ablation study highlights the necessity of the energy-based scheme and the need for elimination of overestimation bias in MARL. Our implementation of EMIX can be found at karush17.github.io/emix-web/.
Hacking with God: a Common Programming Language of Robopsychology and Robophilosophy
Authors: Norbert Bátfai
Date: 2020-09-16 00:00:00
Tasks: Philosophy

This note is a sketch of how the concept of robopsychology and robophilosophy could be reinterpreted and repositioned in the spirit of the original vocation of psychology and philosophy. The notion of the robopsychology as a fictional science and a fictional occupation was introduced by Asimov in the middle of the last century. The robophilosophy, on the other hand, is only a few years old today. But at this moment, none of these new emerging disciplines focus on the fundamental and overall issues of the development of artificial general intelligence. Instead, they focus only on issues that, although are extremely important, play a complementary role, such as moral or ethical ones, rather than the big questions of life. We try to outline a conception in which the robophilosophy and robopsychology will be able to play a similar leading rule in the progress of artificial intelligence than the philosophy and psychology have done in the progress of human intelligence. To facilitate this, we outline the idea of a visual artificial language and interactive theorem prover-based computer application called Prime Convo Assistant. The question to be decided in the future is whether we can develop such an application. And if so, can we build a computer game on it, or even an esport game? It may be an interesting question in order for this game will be able to transform human thinking on the widest possible social scale and will be able to develop a standard mathematical logic-based communication channel between human and machine intelligence.