pwc-archive/papers-with-abstracts huggingface dataset:
BitMix: Data Augmentation for Image Steganalysis
Authors: In-Jae Yu, Wonhyuk Ahn, Seung-Hun Nam, Heung-Kyu Lee
Date: 2020-06-30 00:00:00
Tasks: Data Augmentation, Steganalysis

Convolutional neural networks (CNN) for image steganalysis demonstrate better performances with employing concepts from high-level vision tasks. The major employed concept is to use data augmentation to avoid overfitting due to limited data. To augment data without damaging the message embedding, only rotating multiples of 90 degrees or horizontally flipping are used in steganalysis, which generates eight fixed results from one sample. To overcome this limitation, we propose BitMix, a data augmentation method for spatial image steganalysis. BitMix mixes a cover and stego image pair by swapping the random patch and generates an embedding adaptive label with the ratio of the number of pixels modified in the swapped patch to those in the cover-stego pair. We explore optimal hyperparameters, the ratio of applying BitMix in the mini-batch, and the size of the bounding box for swapping patch. The results reveal that using BitMix improves the performance of spatial image steganalysis and better than other data augmentation methods.
Accelerating Binarized Neural Networks via Bit-Tensor-Cores in Turing GPUs
Authors: Ang Li, Simon Su
Date: 2020-06-30 00:00:00

Despite foreseeing tremendous speedups over conventional deep neural networks, the performance advantage of binarized neural networks (BNNs) has merely been showcased on general-purpose processors such as CPUs and GPUs. In fact, due to being unable to leverage bit-level-parallelism with a word-based architecture, GPUs have been criticized for extremely low utilization (1%) when executing BNNs. Consequently, the latest tensorcores in NVIDIA Turing GPUs start to experimentally support bit computation. In this work, we look into this brand new bit computation capability and characterize its unique features. We show that the stride of memory access can significantly affect performance delivery and a data-format co-design is highly desired to support the tensorcores for achieving superior performance than existing software solutions without tensorcores. We realize the tensorcore-accelerated BNN design, particularly the major functions for fully-connect and convolution layers -- bit matrix multiplication and bit convolution. Evaluations on two NVIDIA Turing GPUs show that, with ResNet-18, our BTC-BNN design can process ImageNet at a rate of 5.6K images per second, 77% faster than state-of-the-art. Our BNN approach is released on https://github.com/pnnl/TCBNN.
Partial Recovery for Top-$k$ Ranking: Optimality of MLE and Sub-Optimality of Spectral Method
Authors: Pinhan Chen, Chao GAO, Anderson Y. Zhang
Date: 2020-06-30 00:00:00

Given partially observed pairwise comparison data generated by the Bradley-Terry-Luce (BTL) model, we study the problem of top-$k$ ranking. That is, to optimally identify the set of top-$k$ players. We derive the minimax rate with respect to a normalized Hamming loss. This provides the first result in the literature that characterizes the partial recovery error in terms of the proportion of mistakes for top-$k$ ranking. We also derive the optimal signal to noise ratio condition for the exact recovery of the top-$k$ set. The maximum likelihood estimator (MLE) is shown to achieve both optimal partial recovery and optimal exact recovery. On the other hand, we show another popular algorithm, the spectral method, is in general sub-optimal. Our results complement the recent work by Chen et al. (2019) that shows both the MLE and the spectral method achieve the optimal sample complexity for exact recovery. It turns out the leading constants of the sample complexity are different for the two algorithms. Another contribution that may be of independent interest is the analysis of the MLE without any penalty or regularization for the BTL model. This closes an important gap between theory and practice in the literature of ranking.
Deep Learning Based Proactive Multi-Objective Eco-Routing Strategies for Connected and Automated Vehicles
Authors: Lama Alfaseeh, Bilal Farooq
Date: 2020-06-30 00:00:00

This study exploits the advancements in information and communication technology (ICT), connected and automated vehicles (CAVs), and sensing, to develop proactive multi-objective eco-routing strategies. For a robust application, several GHG costing approaches are examined. The predictive models for the link level traffic and emission states are developed using long short term memory deep network with exogenous predictors. It is found that proactive routing strategies outperformed the myopic strategies, regardless of the routing objective. Whether myopic or proactive, the multi-objective routing, with travel time and GHG minimization as objectives, outperformed the single objective routing strategies, causing a reduction in the average travel time (TT), average vehicle kilometre travelled (VKT), total GHG and total NOx by 17%, 21%, 18%, and 20%, respectively. Finally, the additional TT and VKT experienced by the vehicles in the network contributed adversely to the amount of GHG and NOx produced in the network.
Fast OSCAR and OWL Regression via Safe Screening Rules
Authors: Runxue Bao, Bin Gu, Heng Huang
Date: 2020-06-29 00:00:00
Tasks: regression, Sparse Learning

Ordered Weighted $L_{1}$ (OWL) regularized regression is a new regression analysis for high-dimensional sparse learning. Proximal gradient methods are used as standard approaches to solve OWL regression. However, it is still a burning issue to solve OWL regression due to considerable computational cost and memory usage when the feature or sample size is large. In this paper, we propose the first safe screening rule for OWL regression by exploring the order of the primal solution with the unknown order structure via an iterative strategy, which overcomes the difficulties of tackling the non-separable regularizer. It effectively avoids the updates of the parameters whose coefficients must be zero during the learning process. More importantly, the proposed screening rule can be easily applied to standard and stochastic proximal gradient methods. Moreover, we prove that the algorithms with our screening rule are guaranteed to have identical results with the original algorithms. Experimental results on a variety of datasets show that our screening rule leads to a significant computational gain without any loss of accuracy, compared to existing competitive algorithms.
Improving neural network predictions of material properties with limited data using transfer learning
Authors: Schuyler Krawczuk, Daniele Venturi
Date: 2020-06-29 00:00:00
Tasks: Transfer Learning

We develop new transfer learning algorithms to accelerate prediction of material properties from ab initio simulations based on density functional theory (DFT). Transfer learning has been successfully utilized for data-efficient modeling in applications other than materials science, and it allows transferable representations learned from large datasets to be repurposed for learning new tasks even with small datasets. In the context of materials science, this opens the possibility to develop generalizable neural network models that can be repurposed on other materials, without the need of generating a large (computationally expensive) training set of materials properties. The proposed transfer learning algorithms are demonstrated on predicting the Gibbs free energy of light transition metal oxides.
Transition control of a tail-sitter UAV using recurrent neural networks
Authors: Alejandro Flores, Gerardo Flores
Date: 2020-06-29 00:00:00

This paper presents the implementation of a Recurrent Neural Network (RNN) based-controller for the stabilization of the flight transition maneuver (hover-cruise and vice versa) of a tail-sitter UAV. The control strategy is based on attitude and velocity stabilization. For that aim, the RNN is used for the estimation of high nonlinear aerodynamic terms during the transition stage. Then, this estimate is used together with a feedback linearization technique for stabilizing the entire system. Results show convergence of linear velocities and the pitch angle during the transition maneuver. To analyze the performance of our proposed control strategy, we present simulations for the transition from hover to cruise and vice versa.
An EM Approach to Non-autoregressive Conditional Sequence Generation
Authors: Zhiqing Sun, Yiming Yang
Date: 2020-06-29 00:00:00
Proceeding: ICML 2020 1
Tasks: Machine Translation, Translation

Autoregressive (AR) models have been the dominating approach to conditional sequence generation, but are suffering from the issue of high inference latency. Non-autoregressive (NAR) models have been recently proposed to reduce the latency by generating all output tokens in parallel but could only achieve inferior accuracy compared to their autoregressive counterparts, primarily due to a difficulty in dealing with the multi-modality in sequence generation. This paper proposes a new approach that jointly optimizes both AR and NAR models in a unified Expectation-Maximization (EM) framework. In the E-step, an AR model learns to approximate the regularized posterior of the NAR model. In the M-step, the NAR model is updated on the new posterior and selects the training examples for the next AR model. This iterative process can effectively guide the system to remove the multi-modality in the output sequences. To our knowledge, this is the first EM approach to NAR sequence generation. We evaluate our method on the task of machine translation. Experimental results on benchmark data sets show that the proposed approach achieves competitive, if not better, performance with existing NAR models and significantly reduces the inference latency.
Bayesian Sparse learning with preconditioned stochastic gradient MCMC and its applications
Authors: Yating Wang, Wei Deng, Lin Guang
Date: 2020-06-29 00:00:00
Tasks: Sparse Learning

In this work, we propose a Bayesian type sparse deep learning algorithm. The algorithm utilizes a set of spike-and-slab priors for the parameters in the deep neural network. The hierarchical Bayesian mixture will be trained using an adaptive empirical method. That is, one will alternatively sample from the posterior using preconditioned stochastic gradient Langevin Dynamics (PSGLD), and optimize the latent variables via stochastic approximation. The sparsity of the network is achieved while optimizing the hyperparameters with adaptive searching and penalizing. A popular SG-MCMC approach is Stochastic gradient Langevin dynamics (SGLD). However, considering the complex geometry in the model parameter space in non-convex learning, updating parameters using a universal step size in each component as in SGLD may cause slow mixing. To address this issue, we apply a computationally manageable preconditioner in the updating rule, which provides a step-size parameter to adapt to local geometric properties. Moreover, by smoothly optimizing the hyperparameter in the preconditioning matrix, our proposed algorithm ensures a decreasing bias, which is introduced by ignoring the correction term in preconditioned SGLD. According to the existing theoretical framework, we show that the proposed algorithm can asymptotically converge to the correct distribution with a controllable bias under mild conditions. Numerical tests are performed on both synthetic regression problems and learning the solutions of elliptic PDE, which demonstrate the accuracy and efficiency of present work.
Inference in Bayesian Additive Vector Autoregressive Tree Models
Authors: Florian Huber, Luca Rossini
Date: 2020-06-29 00:00:00

Vector autoregressive (VAR) models assume linearity between the endogenous variables and their lags. This assumption might be overly restrictive and could have a deleterious impact on forecasting accuracy. As a solution, we propose combining VAR with Bayesian additive regression tree (BART) models. The resulting Bayesian additive vector autoregressive tree (BAVART) model is capable of capturing arbitrary non-linear relations between the endogenous variables and the covariates without much input from the researcher. Since controlling for heteroscedasticity is key for producing precise density forecasts, our model allows for stochastic volatility in the errors. We apply our model to two datasets. The first application shows that the BAVART model yields highly competitive forecasts of the US term structure of interest rates. In a second application, we estimate our model using a moderately sized Eurozone dataset to investigate the dynamic effects of uncertainty on the economy.
Random Partitioning Forest for Point-Wise and Collective Anomaly Detection -- Application to Intrusion Detection
Authors: Pierre-Francois Marteau
Date: 2020-06-29 00:00:00
Tasks: Anomaly Detection, Intrusion Detection

In this paper, we propose DiFF-RF, an ensemble approach composed of random partitioning binary trees to detect point-wise and collective (as well as contextual) anomalies. Thanks to a distance-based paradigm used at the leaves of the trees, this semi-supervised approach solves a drawback that has been identified in the isolation forest (IF) algorithm. Moreover, taking into account the frequencies of visits in the leaves of the random trees allows to significantly improve the performance of DiFF-RF when considering the presence of collective anomalies. DiFF-RF is fairly easy to train, and excellent performance can be obtained by using a simple semi-supervised procedure to setup the extra hyper-parameter that is introduced. We first evaluate DiFF-RF on a synthetic data set to i) verify that the limitation of the IF algorithm is overcome, ii) demonstrate how collective anomalies are actually detected and iii) to analyze the effect of the meta-parameters it involves. We assess the DiFF-RF algorithm on a large set of datasets from the UCI repository, as well as two benchmarks related to intrusion detection applications. Our experiments show that DiFF-RF almost systematically outperforms the IF algorithm, but also challenges the one-class SVM baseline and a deep learning variational auto-encoder architecture. Furthermore, our experience shows that DiFF-RF can work well in the presence of small-scale learning data, which is conversely difficult for deep neural architectures. Finally, DiFF-RF is computationally efficient and can be easily parallelized on multi-core architectures.
Machine learning in problems of automation of ultrasound diagnostics of railway tracks
Authors: Igonin Andrey, Ulybin Vitaliy
Date: 2020-06-29 00:00:00
Tasks: BIG-bench Machine Learning

The article presents the system architecture for automatic decoding of railway track defectograms in real time. The system includes an ultrasound data preprocessing module, a set of neutral network classifiers, a decision block. Preprocessing of data includes affine transformations of measurement information into a format suitable for the operation of a neural network, as well as a combination of information on measurement channels, depending on the type of defect being defined. The classifier is built on a convolutional neural network. The proposed solution can be effectively implemented on a modern elemental basis for performing parallel computing, including tensor processor and GPUs.
A GRU-based Mixture Density Network for Data-Driven Dynamic Stochastic Programming
Authors: Xiaoming Li, Chun Wang, Xiao Huang, Yimin Nie
Date: 2020-06-26 00:00:00
Tasks: Decision Making, Time Series, Time Series Analysis

The conventional deep learning approaches for solving time-series problem such as long-short term memory (LSTM) and gated recurrent unit (GRU) both consider the time-series data sequence as the input with one single unit as the output (predicted time-series result). Those deep learning approaches have made tremendous success in many time-series related problems, however, this cannot be applied in data-driven stochastic programming problems since the output of either LSTM or GRU is a scalar rather than probability distribution which is required by stochastic programming model. To fill the gap, in this work, we propose an innovative data-driven dynamic stochastic programming (DD-DSP) framework for time-series decision-making problem, which involves three components: GRU, Gaussian Mixture Model (GMM) and SP. Specifically, we devise the deep neural network that integrates GRU and GMM which is called GRU-based Mixture Density Network (MDN), where GRU is used to predict the time-series outcomes based on the recent historical data, and GMM is used to extract the corresponding probability distribution of predicted outcomes, then the results will be input as the parameters for SP. To validate our approach, we apply the framework on the car-sharing relocation problem. The experiment validations show that our framework is superior to data-driven optimization based on LSTM with the vehicle average moving lower than LSTM.
On Finite Entailment of Non-Local Queries in Description Logics
Authors: Tomasz Gogacz, Víctor Gutiérrez-Basulto, Albert Gutowski, Yazmín Ibáñez-García, Filip Murlak
Date: 2020-06-30 00:00:00

We study the problem of finite entailment of ontology-mediated queries. Going beyond local queries, we allow transitive closure over roles. We focus on ontologies formulated in the description logics ALCOI and ALCOQ, extended with transitive closure. For both logics, we show 2EXPTIME upper bounds for finite entailment of unions of conjunctive queries with transitive closure. We also provide a matching lower bound by showing that finite entailment of conjunctive queries with transitive closure in ALC is 2EXPTIME-hard.
Mobile Link Prediction: Automated Creation and Crowd-sourced Validation of Knowledge Graphs
Authors: Mark Christopher Ballandies, Evangelos Pournaras
Date: 2020-06-30 00:00:00
Tasks: Knowledge Graphs, Link Prediction

Building trustworthy knowledge graphs for cyber-physical social systems (CPSS) is a challenge. In particular, current approaches relying on human experts have limited scalability, while automated approaches are often not accountable to users resulting in knowledge graphs of questionable quality. This paper introduces a novel pervasive knowledge graph builder that brings together automation, experts' and crowd-sourced citizens' knowledge. The knowledge graph grows via automated link predictions using genetic programming that are validated by humans for improving transparency and calibrating accuracy. The knowledge graph builder is designed for pervasive devices such as smartphones and preserves privacy by localizing all computations. The accuracy, practicality, and usability of the knowledge graph builder is evaluated in a real-world social experiment that involves a smartphone implementation and a Smart City application scenario. The proposed knowledge graph building methodology outperforms the baseline method in terms of accuracy while demonstrating its efficient calculations on smartphones and the feasibility of the pervasive human supervision process in terms of high interactions throughput. These findings promise new opportunities to crowd-source and operate pervasive reasoning systems for cyber-physical social systems in Smart Cities.
PLATO-2: Towards Building an Open-Domain Chatbot via Curriculum Learning
Authors: Siqi Bao, Huang He, Fan Wang, Hua Wu, Haifeng Wang, Wenquan Wu, Zhen Guo, Zhibin Liu, Xinchao Xu
Date: 2020-06-30 00:00:00
Proceeding: Findings (ACL) 2021 8
Tasks: Chatbot, Response Generation

To build a high-quality open-domain chatbot, we introduce the effective training process of PLATO-2 via curriculum learning. There are two stages involved in the learning process. In the first stage, a coarse-grained generation model is trained to learn response generation under the simplified framework of one-to-one mapping. In the second stage, a fine-grained generative model augmented with latent variables and an evaluation model are further trained to generate diverse responses and to select the best response, respectively. PLATO-2 was trained on both Chinese and English data, whose effectiveness and superiority are verified through comprehensive evaluations, achieving new state-of-the-art results.
FairRec: Fairness-aware News Recommendation with Decomposed Adversarial Learning
Authors: Chuhan Wu, Fangzhao Wu, Xiting Wang, Yongfeng Huang, Xing Xie
Date: 2020-06-30 00:00:00
Tasks: Attribute, Fairness, News Recommendation

News recommendation is important for online news services. Existing news recommendation models are usually learned from users' news click behaviors. Usually the behaviors of users with the same sensitive attributes (e.g., genders) have similar patterns and news recommendation models can easily capture these patterns. It may lead to some biases related to sensitive user attributes in the recommendation results, e.g., always recommending sports news to male users, which is unfair since users may not receive diverse news information. In this paper, we propose a fairness-aware news recommendation approach with decomposed adversarial learning and orthogonality regularization, which can alleviate unfairness in news recommendation brought by the biases of sensitive user attributes. In our approach, we propose to decompose the user interest model into two components. One component aims to learn a bias-aware user embedding that captures the bias information on sensitive user attributes, and the other aims to learn a bias-free user embedding that only encodes attribute-independent user interest information for fairness-aware news recommendation. In addition, we propose to apply an attribute prediction task to the bias-aware user embedding to enhance its ability on bias modeling, and we apply adversarial learning to the bias-free user embedding to remove the bias information from it. Moreover, we propose an orthogonality regularization method to encourage the bias-free user embeddings to be orthogonal to the bias-aware one to better distinguish the bias-free user embedding from the bias-aware one. For fairness-aware news ranking, we only use the bias-free user embedding. Extensive experiments on benchmark dataset show that our approach can effectively improve fairness in news recommendation with minor performance loss.
Correction of Faulty Background Knowledge based on Condition Aware and Revise Transformer for Question Answering
Authors: Xinyan Zhao, Xiao Feng, Haoming Zhong, Jun Yao, Huanhuan Chen
Date: 2020-06-30 00:00:00
Tasks: Missing Values, Question Answering

The study of question answering has received increasing attention in recent years. This work focuses on providing an answer that compatible with both user intent and conditioning information corresponding to the question, such as delivery status and stock information in e-commerce. However, these conditions may be wrong or incomplete in real-world applications. Although existing question answering systems have considered the external information, such as categorical attributes and triples in knowledge base, they all assume that the external information is correct and complete. To alleviate the effect of defective condition values, this paper proposes condition aware and revise Transformer (CAR-Transformer). CAR-Transformer (1) revises each condition value based on the whole conversation and original conditions values, and (2) it encodes the revised conditions and utilizes the conditions embedding to select an answer. Experimental results on a real-world customer service dataset demonstrate that the CAR-Transformer can still select an appropriate reply when conditions corresponding to the question exist wrong or missing values, and substantially outperforms baseline models on automatic and human evaluations. The proposed CAR-Transformer can be extended to other NLP tasks which need to consider conditioning information.