pwc-archive/papers-with-abstracts huggingface dataset:
Taking Care of The Discretization Problem: A Comprehensive Study of the Discretization Problem and A Black-Box Adversarial Attack in Discrete Integer Domain
Authors: Lei Bu, Yuchao Duan, Fu Song, Zhe Zhao
Date: 2019-05-19 00:00:00
Tasks: Adversarial Attack

Numerous methods for crafting adversarial examples were proposed recently with high success rate. Since most existing machine learning based classifiers normalize images into some continuous, real vector, domain firstly, attacks often craft adversarial examples in such domain. However, "adversarial" examples may become benign after denormalizing them back into the discrete integer domain, known as the discretization problem. This problem was mentioned in some work, but has received relatively little attention. In this work, we first conduct a comprehensive study of existing methods and tools for crafting. We theoretically analyze 34 representative methods and empirically study 20 representative open source tools for crafting adversarial images. Our study reveals that the discretization problem is far more serious than originally thought. This suggests that the discretization problem should be taken into account seriously when crafting adversarial examples and measuring attack success rate. As a first step towards addressing this problem in black-box scenario, we propose a black-box method which reduces the adversarial example searching problem to a derivative-free optimization problem. Our method is able to craft adversarial images by derivative-free search in the discrete integer domain. Experimental results show that our method is comparable to recent white-box methods (e.g., FGSM, BIM and C\&W) and achieves significantly higher success rate in terms of adversarial examples in the discrete integer domain than recent black-box methods (e.g., ZOO, NES-PGD and Bandits). Moreover, our method is able to handle models that is non-differentiable and successfully break the winner of NIPS 2017 competition on defense with 95\% success rate. Our results suggest that discrete optimization algorithms open up a promising area of research into effective black-box attacks.
What Do Adversarially Robust Models Look At?
Authors: Takahiro Itazuri, Yoshihiro Fukuhara, Hirokatsu Kataoka, Shigeo Morishima
Date: 2019-05-19 00:00:00
Tasks: Adversarial Robustness, Open-Ended Question Answering

In this paper, we address the open question: "What do adversarially robust models look at?" Recently, it has been reported in many works that there exists the trade-off between standard accuracy and adversarial robustness. According to prior works, this trade-off is rooted in the fact that adversarially robust and standard accurate models might depend on very different sets of features. However, it has not been well studied what kind of difference actually exists. In this paper, we analyze this difference through various experiments visually and quantitatively. Experimental results show that adversarially robust models look at things at a larger scale than standard models and pay less attention to fine textures. Furthermore, although it has been claimed that adversarially robust features are not compatible with standard accuracy, there is even a positive effect by using them as pre-trained models particularly in low resolution datasets.
On Selecting Stable Predictors in Time Series Models
Authors: Avleen S. Bijral
Date: 2019-05-18 00:00:00
Tasks: feature selection, Time Series, Time Series Analysis

We extend the feature selection methodology to dependent data and propose a novel time series predictor selection scheme that accommodates statistical dependence in a more typical i.i.d sub-sampling based framework. Furthermore, the machinery of mixing stationary processes allows us to quantify the improvements of our approach over any base predictor selection method (such as lasso) even in a finite sample setting. Using the lasso as a base procedure we demonstrate the applicability of our methods to simulated and several real time series datasets.
A Case Study: Exploiting Neural Machine Translation to Translate CUDA to OpenCL
Authors: Yonghae Kim, Hyesoon Kim
Date: 2019-05-18 00:00:00
Tasks: GPU, Machine Translation, Translation

The sequence-to-sequence (seq2seq) model for neural machine translation has significantly improved the accuracy of language translation. There have been new efforts to use this seq2seq model for program language translation or program comparisons. In this work, we present the detailed steps of using a seq2seq model to translate CUDA programs to OpenCL programs, which both have very similar programming styles. Our work shows (i) a training input set generation method, (ii) pre/post processing, and (iii) a case study using Polybench-gpu-1.0, NVIDIA SDK, and Rodinia benchmarks.
SAWNet: A Spatially Aware Deep Neural Network for 3D Point Cloud Processing
Authors: Chaitanya Kaul, Nick Pears, Suresh Manandhar
Date: 2019-05-18 00:00:00
Tasks: Benchmarking, Scene Segmentation, Segmentation

Deep neural networks have established themselves as the state-of-the-art methodology in almost all computer vision tasks to date. But their application to processing data lying on non-Euclidean domains is still a very active area of research. One such area is the analysis of point cloud data which poses a challenge due to its lack of order. Many recent techniques have been proposed, spearheaded by the PointNet architecture. These techniques use either global or local information from the point clouds to extract a latent representation for the points, which is then used for the task at hand (classification/segmentation). In our work, we introduce a neural network layer that combines both global and local information to produce better embeddings of these points. We enhance our architecture with residual connections, to pass information between the layers, which also makes the network easier to train. We achieve state-of-the-art results on the ModelNet40 dataset with our architecture, and our results are also highly competitive with the state-of-the-art on the ShapeNet part segmentation dataset and the indoor scene segmentation dataset. We plan to open source our pre-trained models on github to encourage the research community to test our networks on their data, or simply use them for benchmarking purposes.
Scalable Gromov-Wasserstein Learning for Graph Partitioning and Matching
Authors: Hongteng Xu, Dixin Luo, Lawrence Carin
Date: 2019-05-18 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: Clustering, Graph Matching, graph partitioning

We propose a scalable Gromov-Wasserstein learning (S-GWL) method and establish a novel and theoretically-supported paradigm for large-scale graph analysis. The proposed method is based on the fact that Gromov-Wasserstein discrepancy is a pseudometric on graphs. Given two graphs, the optimal transport associated with their Gromov-Wasserstein discrepancy provides the correspondence between their nodes and achieves graph matching. When one of the graphs has isolated but self-connected nodes ($i.e.$, a disconnected graph), the optimal transport indicates the clustering structure of the other graph and achieves graph partitioning. Using this concept, we extend our method to multi-graph partitioning and matching by learning a Gromov-Wasserstein barycenter graph for multiple observed graphs; the barycenter graph plays the role of the disconnected graph, and since it is learned, so is the clustering. Our method combines a recursive $K$-partition mechanism with a regularized proximal gradient algorithm, whose time complexity is $\mathcal{O}(K(E+V)\log_K V)$ for graphs with $V$ nodes and $E$ edges. To our knowledge, our method is the first attempt to make Gromov-Wasserstein discrepancy applicable to large-scale graph analysis and unify graph partitioning and matching into the same framework. It outperforms state-of-the-art graph partitioning and matching methods, achieving a trade-off between accuracy and efficiency.
Disentangled Attribution Curves for Interpreting Random Forests and Boosted Trees
Authors: Summer Devlin, Chandan Singh, W. James Murdoch, Bin Yu
Date: 2019-05-18 00:00:00
Tasks: Feature Engineering, Feature Importance, Interpretable Machine Learning

Tree ensembles, such as random forests and AdaBoost, are ubiquitous machine learning models known for achieving strong predictive performance across a wide variety of domains. However, this strong performance comes at the cost of interpretability (i.e. users are unable to understand the relationships a trained random forest has learned and why it is making its predictions). In particular, it is challenging to understand how the contribution of a particular feature, or group of features, varies as their value changes. To address this, we introduce Disentangled Attribution Curves (DAC), a method to provide interpretations of tree ensemble methods in the form of (multivariate) feature importance curves. For a given variable, or group of variables, DAC plots the importance of a variable(s) as their value changes. We validate DAC on real data by showing that the curves can be used to increase the accuracy of logistic regression while maintaining interpretability, by including DAC as an additional feature. In simulation studies, DAC is shown to out-perform competing methods in the recovery of conditional expectations. Finally, through a case-study on the bike-sharing dataset, we demonstrate the use of DAC to uncover novel insights into a dataset.
Evolving Rewards to Automate Reinforcement Learning
Authors: Aleksandra Faust, Anthony Francis, Dar Mehta
Date: 2019-05-18 00:00:00
Tasks: continuous-control, Continuous Control, Hyperparameter Optimization, MuJoCo, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Many continuous control tasks have easily formulated objectives, yet using them directly as a reward in reinforcement learning (RL) leads to suboptimal policies. Therefore, many classical control tasks guide RL training using complex rewards, which require tedious hand-tuning. We automate the reward search with AutoRL, an evolutionary layer over standard RL that treats reward tuning as hyperparameter optimization and trains a population of RL agents to find a reward that maximizes the task objective. AutoRL, evaluated on four Mujoco continuous control tasks over two RL algorithms, shows improvements over baselines, with the the biggest uplift for more complex tasks. The video can be found at: \url{https://youtu.be/svdaOFfQyC8}.
Quantitative Error Prediction of Medical Image Registration using Regression Forests
Authors: Hessam Sokooti, Gorkem Saygili, Ben Glocker, Boudewijn P. F. Lelieveldt, Marius Staring
Date: 2019-05-18 00:00:00
Tasks: Image Registration, Medical Image Registration, regression

Predicting registration error can be useful for evaluation of registration procedures, which is important for the adoption of registration techniques in the clinic. In addition, quantitative error prediction can be helpful in improving the registration quality. The task of predicting registration error is demanding due to the lack of a ground truth in medical images. This paper proposes a new automatic method to predict the registration error in a quantitative manner, and is applied to chest CT scans. A random regression forest is utilized to predict the registration error locally. The forest is built with features related to the transformation model and features related to the dissimilarity after registration. The forest is trained and tested using manually annotated corresponding points between pairs of chest CT scans in two experiments: SPREAD (trained and tested on SPREAD) and inter-database (including three databases SPREAD, DIR-Lab-4DCT and DIR-Lab-COPDgene). The results show that the mean absolute errors of regression are 1.07 $\pm$ 1.86 and 1.76 $\pm$ 2.59 mm for the SPREAD and inter-database experiment, respectively. The overall accuracy of classification in three classes (correct, poor and wrong registration) is 90.7% and 75.4%, for SPREAD and inter-database respectively. The good performance of the proposed method enables important applications such as automatic quality control in large-scale image analysis.
A Discrete Empirical Interpolation Method for Interpretable Immersion and Embedding of Nonlinear Manifolds
Authors: Samuel E. Otto, Clarence W. Rowley
Date: 2019-05-18 00:00:00

Manifold learning techniques seek to discover structure-preserving mappings of high-dimensional data into low-dimensional spaces. While the new sets of coordinates specified by these mappings can closely parameterize the data, they are generally complicated nonlinear functions of the original variables. This makes them difficult to interpret physically. Furthermore, in data-driven model reduction applications the governing equations may have structure that is destroyed by nonlinear mapping into coordinates on an inertial manifold, creating a computational bottleneck for simulations. Instead, we propose to identify a small collection of the original variables which are capable of uniquely determining all others either locally via immersion or globally via embedding of the underlying manifold. When the data lies on a low-dimensional subspace the existing discrete empirical interpolation method (DEIM) accomplishes this with recent variants employing greedy algorithms based on pivoted QR (PQR) factorizations. However, low-dimensional manifolds coming from a variety of applications, particularly from advection-dominated PDEs, do not lie in or near any low-dimensional subspace. Our proposed approach extends DEIM to data lying near nonlinear manifolds by applying a similar pivoted QR procedure simultaneously on collections of patches making up locally linear approximations of the manifold, resulting in a novel simultaneously pivoted QR (SimPQR) algorithm. The immersion provided by SimPQR can be extended to an embedding by applying SimPQR a second time to a modified collection of vectors. The SimPQR method for computing these `nonlinear DEIM' (NLDEIM) coordinates is successfully applied to real-world data lying near an inertial manifold in a cylinder wake flow as well as data coming from a viscous Burgers equation with different initial conditions.
Semantic flow in language networks
Authors: Edilson A. Corrêa Jr., Vanessa Q. Marinho, Diego R. Amancio
Date: 2019-05-18 00:00:00
Tasks: Community Detection, Philosophy, Semantic Similarity, Semantic Textual Similarity

In this study we propose a framework to characterize documents based on their semantic flow. The proposed framework encompasses a network-based model that connected sentences based on their semantic similarity. Semantic fields are detected using standard community detection methods. as the story unfolds, transitions between semantic fields are represent in Markov networks, which in turned are characterized via network motifs (subgraphs). Here we show that the proposed framework can be used to classify books according to their style and publication dates. Remarkably, even without a systematic optimization of parameters, philosophy and investigative books were discriminated with an accuracy rate of 92.5%. Because this model captures semantic features of texts, it could be used as an additional feature in traditional network-based models of texts that capture only syntactical/stylistic information, as it is the case of word adjacency (co-occurrence) networks.
BERTSel: Answer Selection with Pre-trained Models
Authors: Dongfang Li, Yifei Yu, Qingcai Chen, Xinyu Li
Date: 2019-05-18 00:00:00
Tasks: Answer Selection, Natural Language Inference, Question Answering

Recently, pre-trained models have been the dominant paradigm in natural language processing. They achieved remarkable state-of-the-art performance across a wide range of related tasks, such as textual entailment, natural language inference, question answering, etc. BERT, proposed by Devlin et.al., has achieved a better marked result in GLUE leaderboard with a deep transformer architecture. Despite its soaring popularity, however, BERT has not yet been applied to answer selection. This task is different from others with a few nuances: first, modeling the relevance and correctness of candidates matters compared to semantic relatedness and syntactic structure; second, the length of an answer may be different from other candidates and questions. In this paper. we are the first to explore the performance of fine-tuning BERT for answer selection. We achieved STOA results across five popular datasets, demonstrating the success of pre-trained models in this task.
Microblog Hashtag Generation via Encoding Conversation Contexts
Authors: Yue Wang, Jing Li, Irwin King, Michael R. Lyu, Shuming Shi
Date: 2019-05-18 00:00:00
Proceeding: NAACL 2019 6
Tasks: Topic Models

Automatic hashtag annotation plays an important role in content understanding for microblog posts. To date, progress made in this field has been restricted to phrase selection from limited candidates, or word-level hashtag discovery using topic models. Different from previous work considering hashtags to be inseparable, our work is the first effort to annotate hashtags with a novel sequence generation framework via viewing the hashtag as a short sequence of words. Moreover, to address the data sparsity issue in processing short microblog posts, we propose to jointly model the target posts and the conversation contexts initiated by them with bidirectional attention. Extensive experimental results on two large-scale datasets, newly collected from English Twitter and Chinese Weibo, show that our model significantly outperforms state-of-the-art models based on classification. Further studies demonstrate our ability to effectively generate rare and even unseen hashtags, which is however not possible for most existing methods.
Convolutional Feature Extraction and Neural Arithmetic Logic Units for Stock Prediction
Authors: Shangeth Rajaa, Jajati Keshari Sahoo
Date: 2019-05-18 00:00:00
Tasks: Sentiment Analysis, Stock Prediction

Stock prediction is a topic undergoing intense study for many years. Finance experts and mathematicians have been working on a way to predict the future stock price so as to decide to buy the stock or sell it to make profit. Stock experts or economists, usually analyze on the previous stock values using technical indicators, sentiment analysis etc to predict the future stock price. In recent years, many researches have extensively used machine learning for predicting the stock behaviour. In this paper we propose data driven deep learning approach to predict the future stock value with the previous price with the feature extraction property of convolutional neural network and to use Neural Arithmetic Logic Units with it.
Combining Experience Replay with Exploration by Random Network Distillation
Authors: Francesco Sovrano
Date: 2019-05-18 00:00:00
Tasks: Atari Games, Montezuma's Revenge

Our work is a simple extension of the paper "Exploration by Random Network Distillation". More in detail, we show how to efficiently combine Intrinsic Rewards with Experience Replay in order to achieve more efficient and robust exploration (with respect to PPO/RND) and consequently better results in terms of agent performances and sample efficiency. We are able to do it by using a new technique named Prioritized Oversampled Experience Replay (POER), that has been built upon the definition of what is the important experience useful to replay. Finally, we evaluate our technique on the famous Atari game Montezuma's Revenge and some other hard exploration Atari games.
The Curious Case of Machine Learning In Malware Detection
Authors: Sherif Saad, William Briguglio, Haytham Elmiligi
Date: 2019-05-18 00:00:00
Tasks: BIG-bench Machine Learning, Malware Analysis, Malware Detection

In this paper, we argue that machine learning techniques are not ready for malware detection in the wild. Given the current trend in malware development and the increase of unconventional malware attacks, we expect that dynamic malware analysis is the future for antimalware detection and prevention systems. A comprehensive review of machine learning for malware detection is presented. Then, we discuss how malware detection in the wild present unique challenges for the current state-of-the-art machine learning techniques. We defined three critical problems that limit the success of malware detectors powered by machine learning in the wild. Next, we discuss possible solutions to these challenges and present the requirements of next-generation malware detection. Finally, we outline potential research directions in machine learning for malware detection.
RaFM: Rank-Aware Factorization Machines
Authors: Xiaoshuang Chen, Yin Zheng, Jiaxing Wang, Wenye Ma, Junzhou Huang
Date: 2019-05-18 00:00:00
Tasks: General Classification

Factorization machines (FM) are a popular model class to learn pairwise interactions by a low-rank approximation. Different from existing FM-based approaches which use a fixed rank for all features, this paper proposes a Rank-Aware FM (RaFM) model which adopts pairwise interactions from embeddings with different ranks. The proposed model achieves a better performance on real-world datasets where different features have significantly varying frequencies of occurrences. Moreover, we prove that the RaFM model can be stored, evaluated, and trained as efficiently as one single FM, and under some reasonable conditions it can be even significantly more efficient than FM. RaFM improves the performance of FMs in both regression tasks and classification tasks while incurring less computational burden, therefore also has attractive potential in industrial applications.
Human-like machine thinking: Language guided imagination
Authors: Feng Qi, Wenchuan Wu
Date: 2019-05-18 00:00:00

Human thinking requires the brain to understand the meaning of language expression and to properly organize the thoughts flow using the language. However, current natural language processing models are primarily limited in the word probability estimation. Here, we proposed a Language guided imagination (LGI) network to incrementally learn the meaning and usage of numerous words and syntaxes, aiming to form a human-like machine thinking process. LGI contains three subsystems: (1) vision system that contains an encoder to disentangle the input or imagined scenarios into abstract population representations, and an imagination decoder to reconstruct imagined scenario from higher level representations; (2) Language system, that contains a binarizer to transfer symbol texts into binary vectors, an IPS (mimicking the human IntraParietal Sulcus, implemented by an LSTM) to extract the quantity information from the input texts, and a textizer to convert binary vectors into text symbols; (3) a PFC (mimicking the human PreFrontal Cortex, implemented by an LSTM) to combine inputs of both language and vision representations, and predict text symbols and manipulated images accordingly. LGI has incrementally learned eight different syntaxes (or tasks), with which a machine thinking loop has been formed and validated by the proper interaction between language and vision system. The paper provides a new architecture to let the machine learn, understand and use language in a human-like way that could ultimately enable a machine to construct fictitious 'mental' scenario and possess intelligence.