pwc-archive/papers-with-abstracts huggingface dataset:
Minimum Description Length Revisited
Authors: Peter Grünwald, Teemu Roos
Date: 2019-08-21 00:00:00
Tasks: Data Compression, Model Selection, Two-sample testing

This is an up-to-date introduction to and overview of the Minimum Description Length (MDL) Principle, a theory of inductive inference that can be applied to general problems in statistics, machine learning and pattern recognition. While MDL was originally based on data compression ideas, this introduction can be read without any knowledge thereof. It takes into account all major developments since 2007, the last time an extensive overview was written. These include new methods for model selection and averaging and hypothesis testing, as well as the first completely general definition of {\em MDL estimators}. Incorporating these developments, MDL can be seen as a powerful extension of both penalized likelihood and Bayesian approaches, in which penalization functions and prior distributions are replaced by more general luckiness functions, average-case methodology is replaced by a more robust worst-case approach, and in which methods classically viewed as highly distinct, such as AIC vs BIC and cross-validation vs Bayes can, to a large extent, be viewed from a unified perspective.
U-Net Training with Instance-Layer Normalization
Authors: Xiao-Yun Zhou, Peichao Li, Zhao-Yang Wang, Guang-Zhong Yang
Date: 2019-08-21 00:00:00
Tasks: Image Segmentation, Semantic Segmentation

Normalization layers are essential in a Deep Convolutional Neural Network (DCNN). Various normalization methods have been proposed. The statistics used to normalize the feature maps can be computed at batch, channel, or instance level. However, in most of existing methods, the normalization for each layer is fixed. Batch-Instance Normalization (BIN) is one of the first proposed methods that combines two different normalization methods and achieve diverse normalization for different layers. However, two potential issues exist in BIN: first, the Clip function is not differentiable at input values of 0 and 1; second, the combined feature map is not with a normalized distribution which is harmful for signal propagation in DCNN. In this paper, an Instance-Layer Normalization (ILN) layer is proposed by using the Sigmoid function for the feature map combination, and cascading group normalization. The performance of ILN is validated on image segmentation of the Right Ventricle (RV) and Left Ventricle (LV) using U-Net as the network architecture. The results show that the proposed ILN outperforms previous traditional and popular normalization methods with noticeable accuracy improvements for most validations, supporting the effectiveness of the proposed ILN.
Hebbian Graph Embeddings
Authors: Shalin Shah, Venkataramana Kini
Date: 2019-08-21 00:00:00
Tasks: Recommendation Systems, Representation Learning

Representation learning has recently been successfully used to create vector representations of entities in language learning, recommender systems and in similarity learning. Graph embeddings exploit the locality structure of a graph and generate embeddings for nodes which could be words in a language, products of a retail website; and the nodes are connected based on a context window. In this paper, we consider graph embeddings with an error-free associative learning update rule, which models the embedding vector of node as a non-convex Gaussian mixture of the embeddings of the nodes in its immediate vicinity with some constant variance that is reduced as iterations progress. It is very easy to parallelize our algorithm without any form of shared memory, which makes it possible to use it on very large graphs with a much higher dimensionality of the embeddings. We study the efficacy of proposed method on several benchmark data sets and favorably compare with state of the art methods. Further, proposed method is applied to generate relevant recommendations for a large retailer.
Deep Reinforcement Learning for Foreign Exchange Trading
Authors: Yun-Cheng Tsai, Chun-Chieh Wang
Date: 2019-08-21 00:00:00
Tasks: Deep Reinforcement Learning, Q-Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Reinforcement learning can interact with the environment and is suitable for applications in decision control systems. Therefore, we used the reinforcement learning method to establish a foreign exchange transaction, avoiding the long-standing problem of unstable trends in deep learning predictions. In the system design, we optimized the Sure-Fire statistical arbitrage policy, set three different actions, encoded the continuous price over a period of time into a heat-map view of the Gramian Angular Field (GAF) and compared the Deep Q Learning (DQN) and Proximal Policy Optimization (PPO) algorithms. To test feasibility, we analyzed three currency pairs, namely EUR/USD, GBP/USD, and AUD/USD. We trained the data in units of four hours from 1 August 2018 to 30 November 2018 and tested model performance using data between 1 December 2018 and 31 December 2018. The test results of the various models indicated that favorable investment performance was achieved as long as the model was able to handle complex and random processes and the state was able to describe the environment, validating the feasibility of reinforcement learning in the development of trading strategies.
More unlabelled data or label more data? A study on semi-supervised laparoscopic image segmentation
Authors: Yunguan Fu, Maria R. Robu, Bongjin Koo, Crispin Schneider, Stijn van Laarhoven, Danail Stoyanov, Brian Davidson, Matthew J. Clarkson, Yipeng Hu
Date: 2019-08-20 00:00:00
Tasks: Image Segmentation, Segmentation, Semantic Segmentation

Improving a semi-supervised image segmentation task has the option of adding more unlabelled images, labelling the unlabelled images or combining both, as neither image acquisition nor expert labelling can be considered trivial in most clinical applications. With a laparoscopic liver image segmentation application, we investigate the performance impact by altering the quantities of labelled and unlabelled training data, using a semi-supervised segmentation algorithm based on the mean teacher learning paradigm. We first report a significantly higher segmentation accuracy, compared with supervised learning. Interestingly, this comparison reveals that the training strategy adopted in the semi-supervised algorithm is also responsible for this observed improvement, in addition to the added unlabelled data. We then compare different combinations of labelled and unlabelled data set sizes for training semi-supervised segmentation networks, to provide a quantitative example of the practically useful trade-off between the two data planning strategies in this surgical guidance application.
A Deep Actor-Critic Reinforcement Learning Framework for Dynamic Multichannel Access
Authors: Chen Zhong, Ziyang Lu, M. Cenk Gursoy, Senem Velipasalar
Date: 2019-08-20 00:00:00
Tasks: Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

To make efficient use of limited spectral resources, we in this work propose a deep actor-critic reinforcement learning based framework for dynamic multichannel access. We consider both a single-user case and a scenario in which multiple users attempt to access channels simultaneously. We employ the proposed framework as a single agent in the single-user case, and extend it to a decentralized multi-agent framework in the multi-user scenario. In both cases, we develop algorithms for the actor-critic deep reinforcement learning and evaluate the proposed learning policies via experiments and numerical results. In the single-user model, in order to evaluate the performance of the proposed channel access policy and the framework's tolerance against uncertainty, we explore different channel switching patterns and different switching probabilities. In the case of multiple users, we analyze the probabilities of each user accessing channels with favorable channel conditions and the probability of collision. We also address a time-varying environment to identify the adaptive ability of the proposed framework. Additionally, we provide comparisons (in terms of both the average reward and time efficiency) between the proposed actor-critic deep reinforcement learning framework, Deep-Q network (DQN) based approach, random access, and the optimal policy when the channel dynamics are known.
ElectroLens: Understanding Atomistic Simulations Through Spatially-resolved Visualization of High-dimensional Features
Authors: Xiangyun Lei, Fred Hohman, Duen Horng Chau, Andrew J. Medford
Date: 2019-08-20 00:00:00
Tasks: BIG-bench Machine Learning

In recent years, machine learning (ML) has gained significant popularity in the field of chemical informatics and electronic structure theory. These techniques often require researchers to engineer abstract "features" that encode chemical concepts into a mathematical form compatible with the input to machine-learning models. However, there is no existing tool to connect these abstract features back to the actual chemical system, making it difficult to diagnose failures and to build intuition about the meaning of the features. We present ElectroLens, a new visualization tool for high-dimensional spatially-resolved features to tackle this problem. The tool visualizes high-dimensional data sets for atomistic and electron environment features by a series of linked 3D views and 2D plots. The tool is able to connect different derived features and their corresponding regions in 3D via interactive selection. It is built to be scalable, and integrate with existing infrastructure.
Chaotic Time Series Prediction using Spatio-Temporal RBF Neural Networks
Authors: Alishba Sadiq, Muhammad Sohail Ibrahim, Muhammad Usman, Muhammad Zubair, Shujaat Khan
Date: 2019-08-17 00:00:00
Tasks: Time Series, Time Series Analysis, Time Series Prediction

Due to the dynamic nature, chaotic time series are difficult predict. In conventional signal processing approaches signals are treated either in time or in space domain only. Spatio-temporal analysis of signal provides more advantages over conventional uni-dimensional approaches by harnessing the information from both the temporal and spatial domains. Herein, we propose an spatio-temporal extension of RBF neural networks for the prediction of chaotic time series. The proposed algorithm utilizes the concept of time-space orthogonality and separately deals with the temporal dynamics and spatial non-linearity(complexity) of the chaotic series. The proposed RBF architecture is explored for the prediction of Mackey-Glass time series and results are compared with the standard RBF. The spatio-temporal RBF is shown to out perform the standard RBFNN by achieving significantly reduced estimation error.
Neural Plasticity Networks
Authors: Yang Li, Shihao Ji
Date: 2019-08-13 00:00:00
Tasks: image-classification, Image Classification

Neural plasticity is an important functionality of human brain, in which number of neurons and synapses can shrink or expand in response to stimuli throughout the span of life. We model this dynamic learning process as an $L_0$-norm regularized binary optimization problem, in which each unit of a neural network (e.g., weight, neuron or channel, etc.) is attached with a stochastic binary gate, whose parameters determine the level of activity of a unit in the network. At the beginning, only a small portion of binary gates (therefore the corresponding neurons) are activated, while the remaining neurons are in a hibernation mode. As the learning proceeds, some neurons might be activated or deactivated if doing so can be justified by the cost-benefit tradeoff measured by the $L_0$-norm regularized objective. As the training gets mature, the probability of transition between activation and deactivation will diminish until a final hardening stage. We demonstrate that all of these learning dynamics can be modulated by a single parameter $k$ seamlessly. Our neural plasticity network (NPN) can prune or expand a network depending on the initial capacity of network provided by the user; it also unifies dropout (when $k=0$), traditional training of DNNs (when $k=\infty$) and interpolates between these two. To the best of our knowledge, this is the first learning framework that unifies network sparsification and network expansion in an end-to-end training pipeline. Extensive experiments on synthetic dataset and multiple image classification benchmarks demonstrate the superior performance of NPN. We show that both network sparsification and network expansion can yield compact models of similar architectures, while retaining competitive accuracies of the original networks.
Analysis of Wide and Deep Echo State Networks for Multiscale Spatiotemporal Time Series Forecasting
Authors: Zachariah Carmichael, Humza Syed, Dhireesha Kudithipudi
Date: 2019-07-01 00:00:00
Tasks: Time Series, Time Series Analysis, Time Series Forecasting

Echo state networks are computationally lightweight reservoir models inspired by the random projections observed in cortical circuitry. As interest in reservoir computing has grown, networks have become deeper and more intricate. While these networks are increasingly applied to nontrivial forecasting tasks, there is a need for comprehensive performance analysis of deep reservoirs. In this work, we study the influence of partitioning neurons given a budget and the effect of parallel reservoir pathways across different datasets exhibiting multi-scale and nonlinear dynamics.
Deep-learning-assisted diagnosis for knee magnetic resonance imaging: Development and retrospective validation of MRNet
Authors: Nicholas Bien, Pranav Rajpurkar, Robyn L. Ball, Jeremy Irvin, Allison Park, Erik Jones, Michael Bereket, Bhavik N. Patel, Kristen W. Yeom, Katie Shpanskaya, Safwan Halabi, Evan Zucker, Gary Fanton, Derek F. Amanatullah, Christopher F. Beaulieu, Geoffrey M. Riley, Russell J. Stewart, Francis G. Blankenberg, David B. Larson, Ricky H. Jones, Curtis P. Langlotz, Andrew Y. Ng, Matthew P. Lungren
Date: 2018-11-27 00:00:00
Proceeding: Medicine 2018 11
Tasks: Deep Learning, Diagnostic, Multi-Label Classification

Magnetic resonance imaging (MRI) of the knee is the preferred method for diagnosing knee
injuries. However, interpretation of knee MRI is time-intensive and subject to diagnostic
error and variability. An automated system for interpreting knee MRI could prioritize highrisk patients and assist clinicians in making diagnoses. Deep learning methods, in being
able to automatically learn layers of features, are well suited for modeling the complex relationships between medical images and their interpretations. In this study we developed a
deep learning model for detecting general abnormalities and specific diagnoses (anterior
cruciate ligament [ACL] tears and meniscal tears) on knee MRI exams. We then measured
the effect of providing the model’s predictions to clinical experts during interpretation.
A Combination Method for Android Malware Detection Based on Control Flow Graphs and Machine Learning Algorithms
Authors: Zhuo Ma, Haoran Ge, Yang Liu, Meng Zhao, Jianfeng Ma
Date: 2019-01-29 00:00:00
Proceeding: journal 2019 1
Tasks: Android Malware Detection, BIG-bench Machine Learning, Malware Detection, Time Series, Time Series Analysis

Android malware severely threaten system and user security in terms of privilege escalation, remote control, tariff theft, and privacy leakage. Therefore, it is of great importance and necessity to detect Android malware. In this paper, we present a combination method for Android malware detection based on the machine learning algorithm. First, we construct the control flow graph of the application to obtain API information. Based on the API information, we innovatively construct Boolean, frequency, and time-series data sets. Based on these three data sets, three detection models for Android malware detection regarding API calls, API frequency, and API sequence aspects are constructed. Ultimately, an ensemble model is constructed for conformity. We tested and compared the accuracy and stability of our detection models through a large number of experiments. The experiments were conducted on 10010 benign applications and 10683 malicious applications. The results show that our detection model achieves 98.98% detection precision and has high accuracy and stability. All of the results are consistent with the theoretical analysis in this paper.
On an exact method for the constrained shortest path problem
Authors: Leonardo Lozano, Andrés L. Medaglia
Date: 2012-07-16 00:00:00
Proceeding: Computers and Operations Research 2012 7
Tasks: Scheduling

The constrained shortest path (CSP) is a well known NP-Hard problem. Besides from its straightforward application as a network problem, the CSP is also used as a building block under column-generation solution methods for crew scheduling and crew rostering problems. We propose an exact solution method for the CSP capable of handling large-scale networks in a reasonable amount of time. We compared our approach with three different state-of-the-art algorithms for the CSP and found optimal solutions on networks with up to 40,000 nodes and 800,000 arcs. We extended the algorithm to effectively solve the auxiliary problems of a multi-activity shift scheduling problem and a bus rapid transit route design problem tackled with column generation. We obtained significant speedups against alternative column generation schemes that solve the auxiliary problem with state-of-the-art commercial (linear) optimizers. We also present a first parallel version of our algorithm that shows promising results.
Limited Data Rolling Bearing Fault Diagnosis with Few-shot Learning
Authors: Ansi Zhang, Shaobo Li, Yuxin Cui, Wanli Yang, Rongzhi Dong and Jianjun Hu
Date: 2019-08-22 00:00:00
Proceeding: IEEE Access 2019 8
Tasks: Fault Diagnosis, Few-Shot Learning

This paper focuses on bearing fault diagnosis with limited training data. A major challenge in fault diagnosis is the infeasibility of obtaining sufficient training samples for every fault type under all working conditions. Recently deep learning based fault diagnosis methods have achieved promising results. However, most of these methods require large amount of training data. In this study, we propose a deep neural network based few-shot learning approach for rolling bearing fault diagnosis with limited data. Our model is based on the siamese neural network, which learns by exploiting sample pairs of the same or different categories. Experimental results over the standard Case Western Reserve University (CWRU) bearing fault diagnosis benchmark dataset showed that our few-shot learning approach is more effective in fault diagnosis with limited data availability. When tested over different noise environments with minimal amount of training data, the performance of our few-shot learning model surpasses the one of the baseline with reasonable noise level. When evaluated over test sets with new fault types or new working conditions, few-shot models work better than the baseline trained with all fault types. All our models and datasets in this study are open sourced and can be downloaded from https://mekhub.cn/as/fault_diagnosis_with_few-shot_learning/ .
DeeplyTough: Learning Structural Comparison of Protein Binding Sites
Authors: Martin Simonovsky, Joshua Meyers
Date: 2019-04-05 00:00:00
Proceeding: bioRxiv 2019 4
Tasks: Drug Discovery

Protein binding site comparison (pocket matching) is of importance in drug discovery. Identification of similar binding sites can help guide efforts for hit finding, understanding polypharmacology and characterization of protein function. The design of pocket matching methods has traditionally involved much intuition, and has employed a broad variety of algorithms and representations of the input protein structures. We regard the high heterogeneity of past work and the recent availability of large-scale benchmarks as an indicator that a data-driven approach may provide a new perspective.

We propose DeeplyTough, a convolutional neural network that encodes a three-dimensional representation of protein binding sites into descriptor vectors that may be compared efficiently in an alignment-free manner by computing pairwise Euclidean distances. The network is trained with supervision: (i) to provide similar pockets with similar descriptors, (ii) to separate the descriptors of dissimilar pockets by a minimum margin, and (iii) to achieve robustness to nuisance variations. We evaluate our method using three large-scale benchmark datasets, on which it demonstrates excellent performance for held-out data coming from the training distribution and competitive performance when the trained network is required to generalize to datasets constructed independently.
Learning Neural Representations for Network Anomaly Detection
Authors: Van Loi Cao, Miguel Nicolau, and James McDermott
Date: 2019-08-01 00:00:00
Proceeding: IEEE Transactions on Cybernetics 2019 8
Tasks: Anomaly Detection, Intrusion Detection, Model Selection, Unsupervised Anomaly Detection

This paper proposes latent representation models for improving network anomaly detection. Well-known anomaly detection algorithms often suffer from challenges posed by network data, such as high dimension and sparsity, and a lack of anomaly data for training, model selection, and hyperparameter tuning. Our approach is to introduce new regularizers to a classical autoencoder (AE) and a variational AE, which force normal data into a very tight area centered at the origin in the nonsaturating area of the bottleneck unit activations. These trained AEs on normal data will push normal points toward the origin, whereas anomalies, which differ from normal data, will be put far away from the normal region. The models are very different from common regularized AEs, sparse AE, and contractive AE,in which the regularized AEs tend to make their latent representation less sensitive to changes of the input data. The bottleneck feature space is now used as a new data representation. A number of one-class learning algorithms are used for evaluating the proposed models. The experiments testify that our models help these classifiers to perform efficiently and consistently on highdimensional and sparse network datasets, even with relatively few training points. More importantly, the models can minimize the effect of model selection on these classifiers since their performance is insensitive to a wide range of hyperparameter settings.
Human mobility synthesis using matrix and tensor factorizations
Authors: Dezhong Yao, Chen Yu, Hai Jin, Qiang Ding
Date: 2014-06-12 00:00:00
Proceeding: Information Fusion 2014 6
Tasks: Management, Prediction, Tensor Decomposition

Human mobility prediction is of great advantage in route planning and schedule management. However, mobility data is a high-dimensional dataset in which multi-context prediction is difficult in a single model. Mobility data can usually be expressed as a home event, a work event, a shopping event and a traveling event. Previous works have only been able to learn and predict one type of mobility event and then integrate them. As the tensor model has a strong ability to describe high-dimensional information, we propose an algorithm to predict human mobility in tensors of location context data. Using the tensor decomposition method, we extract human mobility patterns with multiple expressions and then synthesize the future mobility event based on mobility patterns. The experiment is based on real-world location data and the results show that the tensor decomposition method has the highest accuracy in terms of prediction error among the three methods. The results also prove the feasibility of our multi-context prediction model.