pwc-archive/papers-with-abstracts huggingface dataset:
Dependently Typed Knowledge Graphs
Authors: Zhangsheng Lai, Aik Beng Ng, Liang Ze Wong, Simon See, Shaowei Lin
Date: 2020-03-08 00:00:00
Tasks: Knowledge Graphs

Reasoning over knowledge graphs is traditionally built upon a hierarchy of languages in the Semantic Web Stack. Starting from the Resource Description Framework (RDF) for knowledge graphs, more advanced constructs have been introduced through various syntax extensions to add reasoning capabilities to knowledge graphs. In this paper, we show how standardized semantic web technologies (RDF and its query language SPARQL) can be reproduced in a unified manner with dependent type theory. In addition to providing the basic functionalities of knowledge graphs, dependent types add expressiveness in encoding both entities and queries, explainability in answers to queries through witnesses, and compositionality and automation in the construction of witnesses. Using the Coq proof assistant, we demonstrate how to build and query dependently typed knowledge graphs as a proof of concept for future works in this direction.
DeepCP: Deep Learning Driven Cascade Prediction Based Autonomous Content Placement in Closed Social Network
Authors: Qiong Wu, Muhong Wu, Xu Chen, Zhi Zhou, Kaiwen He, Liang Chen
Date: 2020-03-09 00:00:00
Tasks: Decision Making, Generative Adversarial Network

Online social networks (OSNs) are emerging as the most popular mainstream platform for content cascade diffusion. In order to provide satisfactory quality of experience (QoE) for users in OSNs, much research dedicates to proactive content placement by using the propagation pattern, user's personal profiles and social relationships in open social network scenarios (e.g., Twitter and Weibo). In this paper, we take a new direction of popularity-aware content placement in a closed social network (e.g., WeChat Moment) where user's privacy is highly enhanced. We propose a novel data-driven holistic deep learning framework, namely DeepCP, for joint diffusion-aware cascade prediction and autonomous content placement without utilizing users' personal and social information. We first devise a time-window LSTM model for content popularity prediction and cascade geo-distribution estimation. Accordingly, we further propose a novel autonomous content placement mechanism CP-GAN which adopts the generative adversarial network (GAN) for agile placement decision making to reduce the content access latency and enhance users' QoE. We conduct extensive experiments using cascade diffusion traces in WeChat Moment (WM). Evaluation results corroborate that the proposed DeepCP framework can predict the content popularity with a high accuracy, generate efficient placement decision in a real-time manner, and achieve significant content access latency reduction over existing schemes.
DeepMAL -- Deep Learning Models for Malware Traffic Detection and Classification
Authors: Gonzalo Marín, Pedro Casas, Germán Capdehourat
Date: 2020-03-03 00:00:00
Tasks: Classification, Deep Learning, General Classification

Robust network security systems are essential to prevent and mitigate the harming effects of the ever-growing occurrence of network attacks. In recent years, machine learning-based systems have gain popularity for network security applications, usually considering the application of shallow models, which rely on the careful engineering of expert, handcrafted input features. The main limitation of this approach is that handcrafted features can fail to perform well under different scenarios and types of attacks. Deep Learning (DL) models can solve this limitation using their ability to learn feature representations from raw, non-processed data. In this paper we explore the power of DL models on the specific problem of detection and classification of malware network traffic. As a major advantage with respect to the state of the art, we consider raw measurements coming directly from the stream of monitored bytes as input to the proposed models, and evaluate different raw-traffic feature representations, including packet and flow-level ones. We introduce DeepMAL, a DL model which is able to capture the underlying statistics of malicious traffic, without any sort of expert handcrafted features. Using publicly available traffic traces containing different families of malware traffic, we show that DeepMAL can detect and classify malware flows with high accuracy, outperforming traditional, shallow-like models.
Overview of Tools Supporting Planning for Automated Driving
Authors: Kailin Tong, Zlatan Ajanovic, Georg Stettinger
Date: 2020-03-09 00:00:00

Planning is an essential topic in the realm of automated driving. Besides planning algorithms that are widely covered in the literature, planning requires different software tools for its development, validation, and execution. This paper presents a survey of such tools including map representations, communication, traffic rules, open-source planning stacks and middleware, simulation, and visualization tools as well as benchmarks. We start by defining the planning task and different supporting tools. Next, we provide a comprehensive review of state-of-the-art developments and analysis of relations among them. Finally, we discuss the current gaps and suggest future research directions.
A Uniform Treatment of Aggregates and Constraints in Hybrid ASP
Authors: Pedro Cabalar, Jorge Fandinno, Torsten Schaub, Philipp Wanko
Date: 2020-03-09 00:00:00

Characterizing hybrid ASP solving in a generic way is difficult since one needs to abstract from specific theories. Inspired by lazy SMT solving, this is usually addressed by treating theory atoms as opaque. Unlike this, we propose a slightly more transparent approach that includes an abstract notion of a term. Rather than imposing a syntax on terms, we keep them abstract by stipulating only some basic properties. With this, we further develop a semantic framework for hybrid ASP solving and provide aggregate functions for theory variables that adhere to different semantic principles, show that they generalize existing aggregate semantics in ASP and how we can rely on off-the-shelf hybrid solvers for implementation.
Beyond Application End-Point Results: Quantifying Statistical Robustness of MCMC Accelerators
Authors: Xiangyu Zhang, Ramin Bashizade, Yicheng Wang, Cheng Lyu, Sayan Mukherjee, Alvin R. Lebeck
Date: 2020-03-05 00:00:00
Tasks: Diagnostic

Statistical machine learning often uses probabilistic algorithms, such as Markov Chain Monte Carlo (MCMC), to solve a wide range of problems. Probabilistic computations, often considered too slow on conventional processors, can be accelerated with specialized hardware by exploiting parallelism and optimizing the design using various approximation techniques. Current methodologies for evaluating correctness of probabilistic accelerators are often incomplete, mostly focusing only on end-point result quality ("accuracy"). It is important for hardware designers and domain experts to look beyond end-point "accuracy" and be aware of the hardware optimizations impact on other statistical properties. This work takes a first step towards defining metrics and a methodology for quantitatively evaluating correctness of probabilistic accelerators beyond end-point result quality. We propose three pillars of statistical robustness: 1) sampling quality, 2) convergence diagnostic, and 3) goodness of fit. We apply our framework to a representative MCMC accelerator and surface design issues that cannot be exposed using only application end-point result quality. Applying the framework to guide design space exploration shows that statistical robustness comparable to floating-point software can be achieved by slightly increasing the bit representation, without floating-point hardware requirements.
Are You Satisfied by This Partial Assignment?
Authors: Roberto Sebastiani
Date: 2020-02-28 00:00:00

Many procedures for SAT and SAT-related problems -- in particular for those requiring the complete enumeration of satisfying truth assignments -- rely their efficiency on the detection of partial assignments satisfying an input formula. In this paper we analyze the notion of partial-assignment satisfiability -- in particular when dealing with non-CNF and existentially-quantified formulas -- raising a flag about the ambiguities and subtleties of this concept, and investigating their practical consequences. This may drive the development of more effective assignment-enumeration algorithms.
Exploration of Surgeons' Natural Skills for Robotic Catheterization
Authors: Olatunji Mumini Omisore, Wenjing Du, Tao Zhou, Shipeng Han, Kamen Ivanov, Yousef Al-Handarish, Lei Wang
Date: 2020-03-06 00:00:00

Despite having the robotic catheter systems which have recently emerged as safe way of performing cardiovascular interventions, a number of important challenges are yet to be investigated. One of them is exploration of surgeons' natural skills during vascular catheterization with robotic systems. In this study, surgeons' natural hand motions were investigated for identification of four basic movements used for intravascular catheterization. Controlled experiment was setup to acquire surface electromyography (sEMG) signals from six muscles that are innervated when a subject with catheterization skills made the four movements in open settings. k-means and k-NN models were implemented over average EMG and root means square features to uniquely identify the movements. The result shows great potentials of sEMG analysis towards designing intelligent cyborg control for safe and efficient robotic catheterization.
Compiling Neural Networks for a Computational Memory Accelerator
Authors: Kornilios Kourtis, Martino Dazzi, Nikolas Ioannou, Tobias Grosser, Abu Sebastian, Evangelos Eleftheriou
Date: 2020-03-05 00:00:00

Computational memory (CM) is a promising approach for accelerating inference on neural networks (NN) by using enhanced memories that, in addition to storing data, allow computations on them. One of the main challenges of this approach is defining a hardware/software interface that allows a compiler to map NN models for efficient execution on the underlying CM accelerator. This is a non-trivial task because efficiency dictates that the CM accelerator is explicitly programmed as a dataflow engine where the execution of the different NN layers form a pipeline. In this paper, we present our work towards a software stack for executing ML models on such a multi-core CM accelerator. We describe an architecture for the hardware and software, and focus on the problem of implementing the appropriate control logic so that data dependencies are respected. We propose a solution to the latter that is based on polyhedral compilation.
Matheuristics to optimize refueling and maintenance planning of nuclear power plants
Authors: Nicolas Dupin, El-Ghazali Talbi
Date: 2018-12-13 00:00:00

Planning the maintenance of nuclear power plants is a complex optimization problem, involving a joint optimization of maintenance dates, fuel constraints and power production decisions. This paper investigates Mixed Integer Linear Programming (MILP) matheuristics for this problem, to tackle large size instances used in operations with a time scope of five years, and few restrictions with time window constraints for the latest maintenance operations. Several constructive matheuristics and a Variable Neighborhood Descent local search are designed. The matheuristics are shown to be accurately effective for medium and large size instances. The matheuristics give also results on the design of MILP formulations and neighborhoods for the problem. Contributions for the operational applications are also discussed. It is shown that the restriction of time windows, which was used to ease computations, induces large over-costs and that this restriction is not required anymore with the capabilities of matheuristics or local search to solve such size of instances. Our matheuristics can be extended to a bi-objective optimization extension with stability costs, for the monthly re-optimization of the maintenance planning in the real-life application.
RSDNE: Exploring Relaxed Similarity and Dissimilarity from Completely-imbalanced Labels for Network Embedding
Authors: ZhengWang, Xiaojun Ye, Chaokun Wang, YuexinWu, ChangpingWang, Kaiwen Liang
Date: 2018-04-25 00:00:00
Proceeding: The Thirty-Second AAAI Conference on Artificial Intelligence 2018 4
Tasks: Network Embedding

Network embedding, aiming to project a network into a low-dimensional space, is increasingly becoming a focus of network research. Semi-supervised network embedding takes advantage of labeled data, and has shown promising performance. However, existing semi-supervised methods would get unappealing results in the completely-imbalanced label setting where some classes have no labeled nodes at all. To alleviate this, we propose a novel semi-supervised network embedding method, termed Relaxed Similarity  and Dissimilarity Network Embedding (RSDNE). Specifically, to benefit from the completely-imbalanced labels, RSDNE guarantees both intra-class similarity and inter-class dissimilarity in an approximate way. Experimental results on several real-world datasets demonstrate the superiority of the proposed method.
Text-attentional convolutional neural network for scene text detection
Authors: Tong He, Weilin Huang, Yu Qiao, Jian Yao
Date: 2016-03-24 00:00:00
Proceeding: IEEE Trans. on Image Processing, 2016 2016 3
Tasks: Multi-Task Learning, Scene Text Detection, text-classification, Text Classification, Text Detection

Recent deep learning models have demonstrated strong capabilities for classifying text and non-text components
in natural images. They extract a high-level feature computed globally from a whole image component (patch), where the cluttered background information may dominate true text features in the deep representation. This leads to less discriminative power and poorer robustness. In this work, we present a new system for scene text detection by proposing a novel Text-Attentional Convolutional Neural Network (Text-CNN) that particularly focuses on extracting text-related regions and features from the image components. We develop a new learning mechanism to train the Text-CNN with multi-level and rich supervised information, including text region mask, character label, and binary text/nontext information. The rich supervision information enables the Text CNN with a strong capability for discriminating ambiguous texts, and also increases its robustness against complicated background components. The training process is formulated as a multi-task learning problem, where low-level supervised information greatly facilitates main task of text/non-text classification. In addition, a powerful low-level detector called ContrastEnhancement Maximally Stable Extremal Regions (CE-MSERs) is developed, which extends the widely-used MSERs by enhancing intensity contrast between text patterns and background. This allows it to detect highly challenging text patterns, resulting in a higher recall. Our approach achieved promising results on the ICDAR 2013 dataset, with a F-measure of 0.82, improving the state-of-the-art results substantially.
ReZero is All You Need: Fast Convergence at Large Depth
Authors: Thomas Bachlechner, Bodhisattwa Prasad Majumder, Huanru Henry Mao, Garrison W. Cottrell, Julian McAuley
Date: 2020-03-10 00:00:00
Tasks: All, Language Modeling, Language Modelling

Deep networks often suffer from vanishing or exploding gradients due to inefficient signal propagation, leading to long training times or convergence difficulties. Various architecture designs, sophisticated residual-style networks, and initialization schemes have been shown to improve deep signal propagation. Recently, Pennington et al. used free probability theory to show that dynamical isometry plays an integral role in efficient deep learning. We show that the simplest architecture change of gating each residual connection using a single zero-initialized parameter satisfies initial dynamical isometry and outperforms more complex approaches. Although much simpler than its predecessors, this gate enables training thousands of fully connected layers with fast convergence and better test performance for ResNets trained on CIFAR-10. We apply this technique to language modeling and find that we can easily train 120-layer Transformers. When applied to 12 layer Transformers, it converges 56% faster on enwiki8.
Cryptanalytic Extraction of Neural Network Models
Authors: Nicholas Carlini, Matthew Jagielski, Ilya Mironov
Date: 2020-03-10 00:00:00
Tasks: Model extraction

We argue that the machine learning problem of model extraction is actually a cryptanalytic problem in disguise, and should be studied as such. Given oracle access to a neural network, we introduce a differential attack that can efficiently steal the parameters of the remote model up to floating point precision. Our attack relies on the fact that ReLU neural networks are piecewise linear functions, and thus queries at the critical points reveal information about the model parameters. We evaluate our attack on multiple neural network models and extract models that are 2^20 times more precise and require 100x fewer queries than prior work. For example, we extract a 100,000 parameter neural network trained on the MNIST digit recognition task with 2^21.5 queries in under an hour, such that the extracted model agrees with the oracle on all inputs up to a worst-case error of 2^-25, or a model with 4,000 parameters in 2^18.5 queries with worst-case error of 2^-40.4. Code is available at https://github.com/google-research/cryptanalytic-model-extraction.
Pruned Neural Networks are Surprisingly Modular
Authors: Daniel Filan, Shlomi Hod, Cody Wild, Andrew Critch, Stuart Russell
Date: 2020-03-10 00:00:00
Tasks: Clustering, Graph Clustering

The learned weights of a neural network are often considered devoid of scrutable internal structure. To discern structure in these weights, we introduce a measurable notion of modularity for multi-layer perceptrons (MLPs), and investigate the modular structure of MLPs trained on datasets of small images. Our notion of modularity comes from the graph clustering literature: a "module" is a set of neurons with strong internal connectivity but weak external connectivity. We find that training and weight pruning produces MLPs that are more modular than randomly initialized ones, and often significantly more modular than random MLPs with the same (sparse) distribution of weights. Interestingly, they are much more modular when trained with dropout. We also present exploratory analyses of the importance of different modules for performance and how modules depend on each other. Understanding the modular structure of neural networks, when such structure exists, will hopefully render their inner workings more interpretable to engineers. Note that this paper has been superceded by "Clusterability in Neural Networks", arxiv:2103.03386 and "Quantifying Local Specialization in Deep Neural Networks", arxiv:2110.08058!
Moving Target Monte Carlo
Authors: Haoyun Ying, Keheng Mao, Klaus Mosegaard
Date: 2020-03-10 00:00:00

The Markov Chain Monte Carlo (MCMC) methods are popular when considering sampling from a high-dimensional random variable $\mathbf{x}$ with possibly unnormalised probability density $p$ and observed data $\mathbf{d}$. However, MCMC requires evaluating the posterior distribution $p(\mathbf{x}|\mathbf{d})$ of the proposed candidate $\mathbf{x}$ at each iteration when constructing the acceptance rate. This is costly when such evaluations are intractable. In this paper, we introduce a new non-Markovian sampling algorithm called Moving Target Monte Carlo (MTMC). The acceptance rate at $n$-th iteration is constructed using an iteratively updated approximation of the posterior distribution $a_n(\mathbf{x})$ instead of $p(\mathbf{x}|\mathbf{d})$. The true value of the posterior $p(\mathbf{x}|\mathbf{d})$ is only calculated if the candidate $\mathbf{x}$ is accepted. The approximation $a_n$ utilises these evaluations and converges to $p$ as $n \rightarrow \infty$. A proof of convergence and estimation of convergence rate in different situations are given.
Multi-SimLex: A Large-Scale Evaluation of Multilingual and Cross-Lingual Lexical Semantic Similarity
Authors: Ivan Vulić, Simon Baker, Edoardo Maria Ponti, Ulla Petti, Ira Leviant, Kelly Wing, Olga Majewska, Eden Bar, Matt Malone, Thierry Poibeau, Roi Reichart, Anna Korhonen
Date: 2020-03-10 00:00:00
Tasks: Cross-Lingual Word Embeddings, Representation Learning, Semantic Similarity, Semantic Textual Similarity, Word Embeddings

We introduce Multi-SimLex, a large-scale lexical resource and evaluation benchmark covering datasets for 12 typologically diverse languages, including major languages (e.g., Mandarin Chinese, Spanish, Russian) as well as less-resourced ones (e.g., Welsh, Kiswahili). Each language dataset is annotated for the lexical relation of semantic similarity and contains 1,888 semantically aligned concept pairs, providing a representative coverage of word classes (nouns, verbs, adjectives, adverbs), frequency ranks, similarity intervals, lexical fields, and concreteness levels. Additionally, owing to the alignment of concepts across languages, we provide a suite of 66 cross-lingual semantic similarity datasets. Due to its extensive size and language coverage, Multi-SimLex provides entirely novel opportunities for experimental evaluation and analysis. On its monolingual and cross-lingual benchmarks, we evaluate and analyze a wide array of recent state-of-the-art monolingual and cross-lingual representation models, including static and contextualized word embeddings (such as fastText, M-BERT and XLM), externally informed lexical representations, as well as fully unsupervised and (weakly) supervised cross-lingual word embeddings. We also present a step-by-step dataset creation protocol for creating consistent, Multi-Simlex-style resources for additional languages. We make these contributions -- the public release of Multi-SimLex datasets, their creation protocol, strong baseline results, and in-depth analyses which can be be helpful in guiding future developments in multilingual lexical semantics and representation learning -- available via a website which will encourage community effort in further expansion of Multi-Simlex to many more languages. Such a large-scale semantic resource could inspire significant further advances in NLP across languages.
Video Caption Dataset for Describing Human Actions in Japanese
Authors: Yutaro Shigeto, Yuya Yoshikawa, Jiaqing Lin, Akikazu Takeuchi
Date: 2020-03-10 00:00:00
Proceeding: LREC 2020 5
Tasks: Caption Generation

In recent years, automatic video caption generation has attracted considerable attention. This paper focuses on the generation of Japanese captions for describing human actions. While most currently available video caption datasets have been constructed for English, there is no equivalent Japanese dataset. To address this, we constructed a large-scale Japanese video caption dataset consisting of 79,822 videos and 399,233 captions. Each caption in our dataset describes a video in the form of "who does what and where." To describe human actions, it is important to identify the details of a person, place, and action. Indeed, when we describe human actions, we usually mention the scene, person, and action. In our experiments, we evaluated two caption generation methods to obtain benchmark results. Further, we investigated whether those generation methods could specify "who does what and where."