pwc-archive/papers-with-abstracts huggingface dataset:
DeepAtom: A Framework for Protein-Ligand Binding Affinity Prediction
Authors: Yanjun Li, Mohammad A. Rezaei, Chenglong Li, Xiaolin Li, Dapeng Wu
Date: 2019-12-01 00:00:00
Tasks: Drug Design, Drug Discovery, Feature Engineering, Molecular Docking, Prediction

The cornerstone of computational drug design is the calculation of binding affinity between two biological counterparts, especially a chemical compound, i.e., a ligand, and a protein. Predicting the strength of protein-ligand binding with reasonable accuracy is critical for drug discovery. In this paper, we propose a data-driven framework named DeepAtom to accurately predict the protein-ligand binding affinity. With 3D Convolutional Neural Network (3D-CNN) architecture, DeepAtom could automatically extract binding related atomic interaction patterns from the voxelized complex structure. Compared with the other CNN based approaches, our light-weight model design effectively improves the model representational capacity, even with the limited available training data. With validation experiments on the PDBbind v.2016 benchmark and the independent Astex Diverse Set, we demonstrate that the less feature engineering dependent DeepAtom approach consistently outperforms the other state-of-the-art scoring methods. We also compile and propose a new benchmark dataset to further improve the model performances. With the new dataset as training input, DeepAtom achieves Pearson's R=0.83 and RMSE=1.23 pK units on the PDBbind v.2016 core set. The promising results demonstrate that DeepAtom models can be potentially adopted in computational drug development protocols such as molecular docking and virtual screening.
Identifying Cognitive Radars -- Inverse Reinforcement Learning using Revealed Preferences
Authors: Vikram Krishnamurthy, Daniel Angley, Robin Evans, William Moran
Date: 2019-12-01 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Scheduling, Stochastic Optimization

We consider an inverse reinforcement learning problem involving us versus an enemy radar equipped with a Bayesian tracker. By observing the emissions of the enemy radar,how can we identify if the radar is cognitive (constrained utility maximizer)? Given the observed sequence of actions taken by the enemy's radar, we consider three problems: (i) Are the enemy radar's actions (waveform choice, beam scheduling) consistent with constrained utility maximization? If so how can we estimate the cognitive radar's utility function that is consistent with its actions. We formulate and solve the problem in terms of the spectra (eigenvalues) of the state and observation noise covariance matrices, and the algebraic Riccati equation. (ii) How to construct a statistical test for detecting a cognitive radar (constrained utility maximization) when we observe the radar's actions in noise or the radar observes our probe signal in noise? We propose a statistical detector with a tight Type-II error bound. (iii) How can we optimally probe (interrogate) the enemy's radar by choosing our state to minimize the Type-II error of detecting if the radar is deploying an economic rational strategy, subject to a constraint on the Type-I detection error? We present a stochastic optimization algorithm to optimize our probe signal. The main analysis framework used in this paper is that of revealed preferences from microeconomics.
DeepLofargram: A Deep Learning based Fluctuating Dim Frequency Line Detection and Recovery
Authors: Yina Han, Yuyan Li, Qingyu Liu, Yuanliang Ma
Date: 2019-12-02 00:00:00
Tasks: Line Detection

This paper investigates the problem of dim frequency line detection and recovery in the so-called lofargram. Theoretically, time integration long enough can always enhance the detection characteristic. But this does not hold for irregularly fluctuating lines. Deep learning has been shown to perform very well for sophisticated visual inference tasks. With the composition of multiple processing layers, very complex high level representation that amplify the important aspects of input while suppresses irrelevant variations can be learned. Hence we propose a new DeepLofargram, composed of deep convolutional neural network and its visualization counterpart. Plugging into specifically designed multi-task loss, an end-to-end training jointly learns to detect and recover the spatial location of potential lines. Leveraging on this deep architecture, the performance boundary is -24dB on average, and -26dB for some. This is far beyond the perception of human visual and significantly improves the state-of-the-art.
Inversion of 1D frequency- and time-domain electromagnetic data with convolutional neural networks
Authors: Vladimir Puzyrev, Andrei Swidinsky
Date: 2019-12-02 00:00:00
Tasks: Geophysics

Inversion of electromagnetic data finds applications in many areas of geophysics. The inverse problem is commonly solved with either deterministic optimization methods (such as the nonlinear conjugate gradient or Gauss-Newton) which are prone to getting trapped in a local minimum, or probabilistic methods which are very computationally demanding. A recently emerging alternative is to employ deep neural networks for predicting subsurface model properties from measured data. This approach is entirely data-driven, does not employ traditional gradient-based techniques and provides a guess to the model instantaneously. In this study, we apply deep convolutional neural networks for 1D inversion of marine frequency-domain controlled-source electromagnetic (CSEM) data as well as onshore time-domain electromagnetic (TEM) data. Our approach yields accurate results both on synthetic and real data and provides them instantaneously. Using several networks and combining their outputs from various training epochs can also provide insights into the uncertainty distribution, which is found to be higher in the regions where resistivity anomalies are present. The proposed method opens up possibilities to estimate the subsurface resistivity distribution in exploration scenarios in real time.
Meta-Path Constrained Random Walk Inference for Large-Scale Heterogeneous Information Networks
Authors: Chenguang Wang
Date: 2019-12-02 00:00:00

Heterogeneous information network (HIN) has shown its power of modeling real world data as a multi-typed entity-relation graph. Meta-path is the key contributor to this power since it enables inference by capturing the proximities between entities via rich semantic links. Previous HIN studies ask users to provide either 1) the meta-path(s) directly or 2) biased examples to generate the meta-path(s). However, lots of HINs (e.g., YAGO2 and Freebase) have rich schema consisting of a sophisticated and large number of types of entities and relations. It is impractical for users to provide the meta-path(s) to support the large scale inference, and biased examples will result in incorrect meta-path based inference, thus limit the power of the meta-path. In this paper, we propose a meta-path constrained inference framework to further release the ability of the meta-path, by efficiently learning the HIN inference patterns via a carefully designed tree structure; and performing unbiased random walk inference with little user guidance. The experiment results on YAGO2 and DBLP datasets show the state-of-the-art performance of the meta-path constrained inference framework.
Noisy, Greedy and Not So Greedy k-means++
Authors: Anup Bhattacharya, Jan Eube, Heiko Röglin, Melanie Schmidt
Date: 2019-12-02 00:00:00
Tasks: Open-Ended Question Answering

The k-means++ algorithm due to Arthur and Vassilvitskii has become the most popular seeding method for Lloyd's algorithm. It samples the first center uniformly at random from the data set and the other $k-1$ centers iteratively according to $D^2$-sampling where the probability that a data point becomes the next center is proportional to its squared distance to the closest center chosen so far. k-means++ is known to achieve an approximation factor of $O(\log k)$ in expectation. Already in the original paper on k-means++, Arthur and Vassilvitskii suggested a variation called greedy k-means++ algorithm in which in each iteration multiple possible centers are sampled according to $D^2$-sampling and only the one that decreases the objective the most is chosen as a center for that iteration. It is stated as an open question whether this also leads to an $O(\log k)$-approximation (or even better). We show that this is not the case by presenting a family of instances on which greedy k-means++ yields only an $\Omega(\ell\cdot \log k)$-approximation in expectation where $\ell$ is the number of possible centers that are sampled in each iteration. We also study a variation, which we call noisy k-means++ algorithm. In this variation only one center is sampled in every iteration but not exactly by $D^2$-sampling anymore. Instead in each iteration an adversary is allowed to change the probabilities arising from $D^2$-sampling individually for each point by a factor between $1-\epsilon_1$ and $1+\epsilon_2$ for parameters $\epsilon_1 \in [0,1)$ and $\epsilon_2 \ge 0$. We prove that noisy k-means++ compute an $O(\log^2 k)$-approximation in expectation. We also discuss some applications of this result.
DLTPy: Deep Learning Type Inference of Python Function Signatures using Natural Language Context
Authors: Casper Boone, Niels de Bruin, Arjan Langerak, Fabian Stelmach
Date: 2019-12-02 00:00:00

Due to the rise of machine learning, Python is an increasingly popular programming language. Python, however, is dynamically typed. Dynamic typing has shown to have drawbacks when a project grows, while at the same time it improves developer productivity. To have the benefits of static typing, combined with high developer productivity, types need to be inferred. In this paper, we present DLTPy: a deep learning type inference solution for the prediction of types in function signatures based on the natural language context (identifier names, comments and return expressions) of a function. We found that DLTPy is effective and has a top-3 F1-score of 91.6%. This means that in most of the cases the correct type is within the top-3 predictions. We conclude that natural language contained in comments and return expressions are beneficial to predicting types more accurately. DLTPy does not significantly outperform or underperform the previous work NL2Type for Javascript, but does show that similar prediction is possible for Python.
Financial Market Directional Forecasting With Stacked Denoising Autoencoder
Authors: Shaogao Lv, Yongchao Hou, Hongwei Zhou
Date: 2019-12-02 00:00:00
Tasks: Deep Learning, Denoising

Forecasting stock market direction is always an amazing but challenging problem in finance. Although many popular shallow computational methods (such as Backpropagation Network and Support Vector Machine) have extensively been proposed, most algorithms have not yet attained a desirable level of applicability. In this paper, we present a deep learning model with strong ability to generate high level feature representations for accurate financial prediction. Precisely, a stacked denoising autoencoder (SDAE) from deep learning is applied to predict the daily CSI 300 index, from Shanghai and Shenzhen Stock Exchanges in China. We use six evaluation criteria to evaluate its performance compared with the back propagation network, support vector machine. The experiment shows that the underlying financial model with deep machine technology has a significant advantage for the prediction of the CSI 300 index.
Relating lp regularization and reweighted l1 regularization
Authors: Hao Wang, Hao Zeng, Jiashan Wang
Date: 2019-12-02 00:00:00

We propose a general framework of iteratively reweighted l1 methods for solving lp regularization problems. We prove that after some iteration k, the iterates generated by the proposed methods have the same support and sign as the limit points, and are bounded away from 0, so that the algorithm behaves like solving a smooth problem in the reduced space. As a result, the global convergence can be easily obtained and an update strategy for the smoothing parameter is proposed which can automatically terminate the updates for zero components. We show that lp regularization problems are locally equivalent to a weighted l1 regularization problem and every optimal point corresponds to a Maximum A Posterior estimation for independently and non-identically distributed Laplace prior parameters. Numerical experiments exhibit the behaviors and the efficiency of our proposed methods.
Pythia: AI-assisted Code Completion System
Authors: Alexey Svyatkovskiy, Ying Zhao, Shengyu Fu, Neel Sundaresan
Date: 2019-11-29 00:00:00
Tasks: Code Completion, Language Modeling, Language Modelling

In this paper, we propose a novel end-to-end approach for AI-assisted code completion called Pythia. It generates ranked lists of method and API recommendations which can be used by software developers at edit time. The system is currently deployed as part of Intellicode extension in Visual Studio Code IDE. Pythia exploits state-of-the-art large-scale deep learning models trained on code contexts extracted from abstract syntax trees. It is designed to work at a high throughput predicting the best matching code completions on the order of 100 $ms$. We describe the architecture of the system, perform comparisons to frequency-based approach and invocation-based Markov Chain language model, and discuss challenges serving Pythia models on lightweight client devices. The offline evaluation results obtained on 2700 Python open source software GitHub repositories show a top-5 accuracy of 92\%, surpassing the baseline models by 20\% averaged over classes, for both intra and cross-project settings.
Deep Density: circumventing the Kohn-Sham equations via symmetry preserving neural networks
Authors: Leonardo Zepeda-Núñez, Yixiao Chen, Jiefu Zhang, Weile Jia, Linfeng Zhang, Lin Lin
Date: 2019-11-27 00:00:00
Tasks: Translation

The recently developed Deep Potential [Phys. Rev. Lett. 120, 143001, 2018] is a powerful method to represent general inter-atomic potentials using deep neural networks. The success of Deep Potential rests on the proper treatment of locality and symmetry properties of each component of the network. In this paper, we leverage its network structure to effectively represent the mapping from the atomic configuration to the electron density in Kohn-Sham density function theory (KS-DFT). By directly targeting at the self-consistent electron density, we demonstrate that the adapted network architecture, called the Deep Density, can effectively represent the electron density as the linear combination of contributions from many local clusters. The network is constructed to satisfy the translation, rotation, and permutation symmetries, and is designed to be transferable to different system sizes. We demonstrate that using a relatively small number of training snapshots, Deep Density achieves excellent performance for one-dimensional insulating and metallic systems, as well as systems with mixed insulating and metallic characters. We also demonstrate its performance for real three-dimensional systems, including small organic molecules, as well as extended systems such as water (up to $512$ molecules) and aluminum (up to $256$ atoms).
Experiments with a PCCoder extension
Authors: Dan Hernest
Date: 2019-11-26 00:00:00

Recent research in synthesis of programs written in some Domain Specific Language (DSL) by means of neural networks from a limited set of inputs-output correspondences such as DeepCoder and its PCCoder reimplementation/optimization proved the efficiency of this kind of approach to automatic program generation in a DSL language that although limited in scope is universal in the sense that programs can be translated to basically any programming language. We experiment with the extension of the DSL of DeepCoder/PCCoder with symbols IFI and IFL which denote functional expressions of the If ramification (test) instruction for types Int and List. We notice an increase (doubling) of the size of the training set, the number of parameters of the trained neural network and of the time spent looking for the program synthesized from limited sets of inputs-output correspondences. The result is positive in the sense of preserving the accuracy of applying synthesis on randomly generated test sets.
Capacity of the covariance perceptron
Authors: David Dahmen, Matthieu Gilson, Moritz Helias
Date: 2019-12-02 00:00:00
Tasks: Binary Classification, Classification, General Classification, Time Series, Time Series Analysis

The classical perceptron is a simple neural network that performs a binary classification by a linear mapping between static inputs and outputs and application of a threshold. For small inputs, neural networks in a stationary state also perform an effectively linear input-output transformation, but of an entire time series. Choosing the temporal mean of the time series as the feature for classification, the linear transformation of the network with subsequent thresholding is equivalent to the classical perceptron. Here we show that choosing covariances of time series as the feature for classification maps the neural network to what we call a 'covariance perceptron'; a mapping between covariances that is bilinear in terms of weights. By extending Gardner's theory of connections to this bilinear problem, using a replica symmetric mean-field theory, we compute the pattern and information capacities of the covariance perceptron in the infinite-size limit. Closed-form expressions reveal superior pattern capacity in the binary classification task compared to the classical perceptron in the case of a high-dimensional input and low-dimensional output. For less convergent networks, the mean perceptron classifies a larger number of stimuli. However, since covariances span a much larger input and output space than means, the amount of stored information in the covariance perceptron exceeds the classical counterpart. For strongly convergent connectivity it is superior by a factor equal to the number of input neurons. Theoretical calculations are validated numerically for finite size systems using a gradient-based optimization of a soft-margin, as well as numerical solvers for the NP hard quadratically constrained quadratic programming problem, to which training can be mapped.
Restoring Chaos Using Deep Reinforcement Learning
Authors: Sumit Vashishtha, Siddhartha Verma
Date: 2019-11-27 00:00:00
Tasks: Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

A catastrophic bifurcation in non-linear dynamical systems, called crisis, often leads to their convergence to an undesirable non-chaotic state after some initial chaotic transients. Preventing such behavior has proved to be quite challenging. We demonstrate that deep Reinforcement Learning (RL) is able to restore chaos in a transiently-chaotic regime of the Lorenz system of equations. Without requiring any a priori knowledge of the underlying dynamics of the governing equations, the RL agent discovers an effective perturbation strategy for sustaining the chaotic trajectory. We analyze the agent's autonomous control-decisions, and identify and implement a simple control-law that successfully restores chaos in the Lorenz system. Our results demonstrate the utility of using deep RL for controlling the occurrence of catastrophes and extreme-events in non-linear dynamical systems.
Kernelized Multiview Subspace Analysis by Self-weighted Learning
Authors: Huibing Wang, Yang Wang, Zhao Zhang, Xianping Fu, Zhuo Li, Mingliang Xu, Meng Wang
Date: 2019-11-23 00:00:00
Tasks: Dimensionality Reduction, Image Retrieval, Retrieval

With the popularity of multimedia technology, information is always represented or transmitted from multiple views. Most of the existing algorithms are graph-based ones to learn the complex structures within multiview data but overlooked the information within data representations. Furthermore, many existing works treat multiple views discriminatively by introducing some hyperparameters, which is undesirable in practice. To this end, abundant multiview based methods have been proposed for dimension reduction. However, there are still no research to leverage the existing work into a unified framework. To address this issue, in this paper, we propose a general framework for multiview data dimension reduction, named Kernelized Multiview Subspace Analysis (KMSA). It directly handles the multi-view feature representation in the kernel space, which provides a feasible channel for direct manipulations on multiview data with different dimensions. Meanwhile, compared with those graph-based methods, KMSA can fully exploit information from multiview data with nothing to lose. Furthermore, since different views have different influences on KMSA, we propose a self-weighted strategy to treat different views discriminatively according to their contributions. A co-regularized term is proposed to promote the mutual learning from multi-views. KMSA combines self-weighted learning with the co-regularized term to learn appropriate weights for all views. We also discuss the influence of the parameters in KMSA regarding the weights of multi-views. We evaluate our proposed framework on 6 multiview datasets for classification and image retrieval. The experimental results validate the advantages of our proposed method.
Detecting Unknown Behaviors by Pre-defined Behaviours: An Bayesian Non-parametric Approach
Authors: Jin Watanabe, Takatomi Kubo, Fan Yang, Kazushi Ikeda
Date: 2019-11-25 00:00:00

An automatic mouse behavior recognition system can considerably reduce the workload of experimenters and facilitate the analysis process. Typically, supervised approaches, unsupervised approaches and semi-supervised approaches are applied for behavior recognition purpose under a setting which has all of predefined behaviors. In the real situation, however, as mouses can show various types of behaviors, besides the predefined behaviors that we want to analyze, there are many undefined behaviors existing. Both supervised approaches and conventional semi-supervised approaches cannot identify these undefined behaviors. Though unsupervised approaches can detect these undefined behaviors, a post-hoc labeling is needed. In this paper, we propose a semi-supervised infinite Gaussian mixture model (SsIGMM), to incorporate both labeled and unlabelled information in learning process while considering undefined behaviors. It also generates the distribution of the predefined and undefined behaviors by mixture Gaussians, which can be used for further analysis. In our experiments, we confirmed the superiority of SsIGMM for segmenting and labelling mouse-behavior videos.
Time-aware Gradient Attack on Dynamic Network Link Prediction
Authors: Jinyin Chen, Jian Zhang, Zhi Chen, Min Du, Qi Xuan
Date: 2019-11-24 00:00:00
Tasks: Adversarial Attack, Link Prediction, Network Embedding, Prediction

In network link prediction, it is possible to hide a target link from being predicted with a small perturbation on network structure. This observation may be exploited in many real world scenarios, for example, to preserve privacy, or to exploit financial security. There have been many recent studies to generate adversarial examples to mislead deep learning models on graph data. However, none of the previous work has considered the dynamic nature of real-world systems. In this work, we present the first study of adversarial attack on dynamic network link prediction (DNLP). The proposed attack method, namely time-aware gradient attack (TGA), utilizes the gradient information generated by deep dynamic network embedding (DDNE) across different snapshots to rewire a few links, so as to make DDNE fail to predict target links. We implement TGA in two ways: one is based on traversal search, namely TGA-Tra; and the other is simplified with greedy search for efficiency, namely TGA-Gre. We conduct comprehensive experiments which show the outstanding performance of TGA in attacking DNLP algorithms.