pwc-archive/papers-with-abstracts huggingface dataset:
Learning to Transfer Learn: Reinforcement Learning-Based Selection for Adaptive Transfer Learning
Authors: Linchao Zhu, Sercan O. Arik, Yi Yang, Tomas Pfister
Date: 2019-08-29 00:00:00
Proceeding: ECCV 2020 8
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Transfer Learning

We propose a novel adaptive transfer learning framework, learning to transfer learn (L2TL), to improve performance on a target dataset by careful extraction of the related information from a source dataset. Our framework considers cooperative optimization of shared weights between models for source and target tasks, and adjusts the constituent loss weights adaptively. The adaptation of the weights is based on a reinforcement learning (RL) selection policy, guided with a performance metric on the target validation set. We demonstrate that L2TL outperforms fine-tuning baselines and other adaptive transfer learning methods on eight datasets. In the regimes of small-scale target datasets and significant label mismatch between source and target datasets, L2TL shows particularly large benefits.
Active Learning for Domain Classification in a Commercial Spoken Personal Assistant
Authors: Xi C. Chen, Adithya Sagar, Justine T. Kao, Tony Y. Li, Christopher Klein, Stephen Pulman, Ashish Garg, Jason D. Williams
Date: 2019-08-29 00:00:00
Tasks: Active Learning, domain classification, General Classification

We describe a method for selecting relevant new training data for the LSTM-based domain selection component of our personal assistant system. Adding more annotated training data for any ML system typically improves accuracy, but only if it provides examples not already adequately covered in the existing data. However, obtaining, selecting, and labeling relevant data is expensive. This work presents a simple technique that automatically identifies new helpful examples suitable for human annotation. Our experimental results show that the proposed method, compared with random-selection and entropy-based methods, leads to higher accuracy improvements given a fixed annotation budget. Although developed and tested in the setting of a commercial intelligent assistant, the technique is of wider applicability.
Deep Learning for Estimating Synaptic Health of Primary Neuronal Cell Culture
Authors: Andrey Kormilitzin, Xinyu Yang, William H. Stone, Caroline Woffindale, Francesca Nicholls, Elena Ribe, Alejo Nevado-Holgado, Noel Buckley
Date: 2019-08-29 00:00:00
Tasks: Binary Classification, Classification, Cultural Vocal Bursts Intensity Prediction, Drug Discovery, General Classification

Understanding the morphological changes of primary neuronal cells induced by chemical compounds is essential for drug discovery. Using the data from a single high-throughput imaging assay, a classification model for predicting the biological activity of candidate compounds was introduced. The image recognition model which is based on deep convolutional neural network (CNN) architecture with residual connections achieved accuracy of 99.6$\%$ on a binary classification task of distinguishing untreated and treated rodent primary neuronal cells with Amyloid-$\beta_{(25-35)}$.
Estimation of Body Mass Index from Photographs using Deep Convolutional Neural Networks
Authors: Adam Pantanowitz, Emmanuel Cohen, Philippe Gradidge, Nigel Crowther, Vered Aharonson, Benjamin Rosman, David M Rubin
Date: 2019-08-29 00:00:00

Obesity is an important concern in public health, and Body Mass Index is one of the useful (and proliferant) measures. We use Convolutional Neural Networks to determine Body Mass Index from photographs in a study with 161 participants. Low data, a common problem in medicine, is addressed by reducing the information in the photographs by generating silhouette images. Results present with high correlation when tested on unseen data.
High Performance Scalable FPGA Accelerator for Deep Neural Networks
Authors: Sudarshan Srinivasan, Pradeep Janedula, Saurabh Dhoble, Sasikanth Avancha, Dipankar Das, Naveen Mellempudi, Bharat Daga, Martin Langhammer, Gregg Baeckler, Bharat Kaul
Date: 2019-08-29 00:00:00
Tasks: CPU, GPU, Vocal Bursts Intensity Prediction

Low-precision is the first order knob for achieving higher Artificial Intelligence Operations (AI-TOPS). However the algorithmic space for sub-8-bit precision compute is diverse, with disruptive changes happening frequently, making FPGAs a natural choice for Deep Neural Network inference, In this work we present an FPGA-based accelerator for CNN inference acceleration. We use {\it INT-8-2} compute (with {\it 8 bit} activation and {2 bit} weights) which is recently showing promise in the literature, and which no known ASIC, CPU or GPU natively supports today. Using a novel Adaptive Logic Module (ALM) based design, as a departure from traditional DSP based designs, we are able to achieve high performance measurement of 5 AI-TOPS for {\it Arria10} and project a performance of 76 AI-TOPS at 0.7 TOPS/W for {\it Stratix10}. This exceeds known CPU, GPU performance and comes close to best known ASIC (TPU) numbers, while retaining the versatility of the FPGA platform for other applications.
Solve fraud detection problem by using graph based learning methods
Authors: Loc Tran, Tuan Tran, Linh Tran, An Mai
Date: 2019-08-29 00:00:00
Tasks: Fraud Detection

The credit cards' fraud transactions detection is the important problem in machine learning field. To detect the credit cards's fraud transactions help reduce the significant loss of the credit cards' holders and the banks. To detect the credit cards' fraud transactions, data scientists normally employ the unsupervised learning techniques and supervised learning techniques. In this paper, we employ the graph p-Laplacian based semi-supervised learning methods combined with the undersampling techniques such as Cluster Centroids to solve the credit cards' fraud transactions detection problem. Experimental results show that the graph p-Laplacian semi-supervised learning methods outperform the current state of the art graph Laplacian based semi-supervised learning method (p=2).
An Ultra-Efficient Memristor-Based DNN Framework with Structured Weight Pruning and Quantization Using ADMM
Authors: Geng Yuan, Xiaolong Ma, Caiwen Ding, Sheng Lin, Tianyun Zhang, Zeinab S. Jalali, Yilong Zhao, Li Jiang, Sucheta Soundarajan, Yanzhi Wang
Date: 2019-08-29 00:00:00
Tasks: Quantization

The high computation and memory storage of large deep neural networks (DNNs) models pose intensive challenges to the conventional Von-Neumann architecture, incurring substantial data movements in the memory hierarchy. The memristor crossbar array has emerged as a promising solution to mitigate the challenges and enable low-power acceleration of DNNs. Memristor-based weight pruning and weight quantization have been seperately investigated and proven effectiveness in reducing area and power consumption compared to the original DNN model. However, there has been no systematic investigation of memristor-based neuromorphic computing (NC) systems considering both weight pruning and weight quantization. In this paper, we propose an unified and systematic memristor-based framework considering both structured weight pruning and weight quantization by incorporating alternating direction method of multipliers (ADMM) into DNNs training. We consider hardware constraints such as crossbar blocks pruning, conductance range, and mismatch between weight value and real devices, to achieve high accuracy and low power and small area footprint. Our framework is mainly integrated by three steps, i.e., memristor-based ADMM regularized optimization, masked mapping and retraining. Experimental results show that our proposed framework achieves 29.81X (20.88X) weight compression ratio, with 98.38% (96.96%) and 98.29% (97.47%) power and area reduction on VGG-16 (ResNet-18) network where only have 0.5% (0.76%) accuracy loss, compared to the original DNN models. We share our models at link http://bit.ly/2Jp5LHJ.
Fairness-Aware Process Mining
Authors: Mahnaz Sadat Qafari, Wil van der Aalst
Date: 2019-08-28 00:00:00
Tasks: Fairness

Process mining is a multi-purpose tool enabling organizations to improve their processes. One of the primary purposes of process mining is finding the root causes of performance or compliance problems in processes. The usual way of doing so is by gathering data from the process event log and other sources and then applying some data mining and machine learning techniques. However, the results of applying such techniques are not always acceptable. In many situations, this approach is prone to making obvious or unfair diagnoses and applying them may result in conclusions that are unsurprising or even discriminating (e.g., blaming overloaded employees for delays). In this paper, we present a solution to this problem by creating a fair classifier for such situations. The undesired effects are removed at the expense of reduction on the accuracy of the resulting classifier. We have implemented this method as a plug-in in ProM. Using the implemented plug-in on two real event logs, we decreased the discrimination caused by the classifier, while losing a small fraction of its accuracy.
Using LSTMs to Model the Java Programming Language
Authors: Brendon Boldt
Date: 2019-08-26 00:00:00
Tasks: model

Recurrent neural networks (RNNs), specifically long-short term memory networks (LSTMs), can model natural language effectively. This research investigates the ability for these same LSTMs to perform next "word" prediction on the Java programming language. Java source code from four different repositories undergoes a transformation that preserves the logical structure of the source code and removes the code's various specificities such as variable names and literal values. Such datasets and an additional English language corpus are used to train and test standard LSTMs' ability to predict the next element in a sequence. Results suggest that LSTMs can effectively model Java code achieving perplexities under 22 and accuracies above 0.47, which is an improvement over LSTM's performance on the English language which demonstrated a perplexity of 85 and an accuracy of 0.27. This research can have applicability in other areas such as syntactic template suggestion and automated bug patching.
Improving Adversarial Robustness via Attention and Adversarial Logit Pairing
Authors: Dou Goodman, Xingjian Li, Ji Liu, Dejing Dou, Tao Wei
Date: 2019-08-23 00:00:00
Tasks: Adversarial Robustness

Though deep neural networks have achieved the state of the art performance in visual classification, recent studies have shown that they are all vulnerable to the attack of adversarial examples. In this paper, we develop improved techniques for defending against adversarial examples. First, we propose an enhanced defense technique denoted Attention and Adversarial Logit Pairing(AT+ALP), which encourages both attention map and logit for the pairs of examples to be similar. When being applied to clean examples and their adversarial counterparts, AT+ALP improves accuracy on adversarial examples over adversarial training. We show that AT+ALP can effectively increase the average activations of adversarial examples in the key area and demonstrate that it focuses on discriminate features to improve the robustness of the model. Finally, we conduct extensive experiments using a wide range of datasets and the experiment results show that our AT+ALP achieves the state of the art defense performance. For example, on 17 Flower Category Database, under strong 200-iteration PGD gray-box and black-box attacks where prior art has 34% and 39% accuracy, our method achieves 50% and 51%. Compared with previous work, our work is evaluated under highly challenging PGD attack: the maximum perturbation $\epsilon \in \{0.25,0.5\}$ i.e. $L_\infty \in \{0.25,0.5\}$ with 10 to 200 attack iterations. To the best of our knowledge, such a strong attack has not been previously explored on a wide range of datasets.
Neural Network Inference on Mobile SoCs
Authors: Siqi Wang, Anuj Pathania, Tulika Mitra
Date: 2019-08-24 00:00:00
Tasks: CPU, GPU

The ever-increasing demand from mobile Machine Learning (ML) applications calls for evermore powerful on-chip computing resources. Mobile devices are empowered with heterogeneous multi-processor Systems-on-Chips (SoCs) to process ML workloads such as Convolutional Neural Network (CNN) inference. Mobile SoCs house several different types of ML capable components on-die, such as CPU, GPU, and accelerators. These different components are capable of independently performing inference but with very different power-performance characteristics. In this article, we provide a quantitative evaluation of the inference capabilities of the different components on mobile SoCs. We also present insights behind their respective power-performance behavior. Finally, we explore the performance limit of the mobile SoCs by synergistically engaging all the components concurrently. We observe that a mobile SoC provides up to 2x improvement with parallel inference when all its components are engaged, as opposed to engaging only one component.
Dynamic Stale Synchronous Parallel Distributed Training for Deep Learning
Authors: Xing Zhao, Aijun An, Junfeng Liu, Bao Xin Chen
Date: 2019-08-16 00:00:00
Tasks: Deep Learning, GPU

Deep learning is a popular machine learning technique and has been applied to many real-world problems. However, training a deep neural network is very time-consuming, especially on big data. It has become difficult for a single machine to train a large model over large datasets. A popular solution is to distribute and parallelize the training process across multiple machines using the parameter server framework. In this paper, we present a distributed paradigm on the parameter server framework called Dynamic Stale Synchronous Parallel (DSSP) which improves the state-of-the-art Stale Synchronous Parallel (SSP) paradigm by dynamically determining the staleness threshold at the run time. Conventionally to run distributed training in SSP, the user needs to specify a particular staleness threshold as a hyper-parameter. However, a user does not usually know how to set the threshold and thus often finds a threshold value through trial and error, which is time-consuming. Based on workers' recent processing time, our approach DSSP adaptively adjusts the threshold per iteration at running time to reduce the waiting time of faster workers for synchronization of the globally shared parameters, and consequently increases the frequency of parameters updates (increases iteration throughput), which speedups the convergence rate. We compare DSSP with other paradigms such as Bulk Synchronous Parallel (BSP), Asynchronous Parallel (ASP), and SSP by running deep neural networks (DNN) models over GPU clusters in both homogeneous and heterogeneous environments. The results show that in a heterogeneous environment where the cluster consists of mixed models of GPUs, DSSP converges to a higher accuracy much earlier than SSP and BSP and performs similarly to ASP. In a homogeneous distributed cluster, DSSP has more stable and slightly better performance than SSP and ASP, and converges much faster than BSP.
Analyzing Cyber-Physical Systems from the Perspective of Artificial Intelligence
Authors: Eric M. S. P. Veith, Lars Fischer, Martin Tröschel, Astrid Nieße
Date: 2019-08-21 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Principles of modern cyber-physical system (CPS) analysis are based on analytical methods that depend on whether safety or liveness requirements are considered. Complexity is abstracted through different techniques, ranging from stochastic modelling to contracts. However, both distributed heuristics and Artificial Intelligence (AI)-based approaches as well as the user perspective or unpredictable effects, such as accidents or the weather, introduce enough uncertainty to warrant reinforcement-learning-based approaches. This paper compares traditional approaches in the domain of CPS modelling and analysis with the AI researcher perspective to exploring unknown complex systems.
FastSpeech: Fast,Robustand Controllable Text-to-Speech
Authors: Yi Ren, Yangjun Ruan, Xu Tan, Tao Qin, Sheng Zhao, Zhou Zhao, Tie-Yan Liu
Date: 2019-05-22 00:00:00
Tasks: Decoder, text-to-speech, Text to Speech, Text-To-Speech Synthesis

Neural network based end-to-end text to speech (TTS) has significantly improved the quality of synthesized speech. Prominent methods (e.g., Tacotron 2) usually first generate mel-spectrogram from text, and then synthesize speech from mel-spectrogram using vocoder such as WaveNet. Compared with traditional concatenative and statistical parametric approaches, neural network based end-to-end models suffer from slow inference speed, and the synthesized speech is usually not robust (i.e., some words are skipped or repeated) and lack of controllability (voice speed or prosody control). In this work, we propose a novel feed-forward network based on Transformer to generate mel-spectrogram in parallel for TTS. Specifically, we extract attention alignments from an encoder-decoder based teacher model for phoneme duration prediction, which is used by a length regulator to expand the source phoneme sequence to match the length of target mel-sprectrogram sequence for parallel mel-sprectrogram generation. Experiments on the LJSpeech dataset show that our parallel model matches autoregressive models in terms of speech quality, nearly eliminates the skipped words and repeated words, and can adjust voice speed smoothly. Most importantly, compared with autoregressive models, our model speeds up the mel-sprectrogram generation by 270x. Therefore, we call our model FastSpeech. We will release the code on Github.
Toward better boundary preserved supervoxel segmentation for 3D point clouds
Authors: Yangbin Lin, Cheng Wang, Dawei Zhai, Wei Li, Jonathan Li
Date: 2019-09-01 00:00:00
Proceeding: ISPRS Journal of Photogrammetry and Remote Sensing 2019 9
Tasks: Point Cloud Segmentation, Segmentation

Supervoxels provide a more natural and compact representation of three dimensional point clouds, and enable the operations to be performed on regions rather than on the scattered points. Many state-of-the-art supervoxel segmentation methods adopt fixed resolution for each supervoxel, and rely on the initialization of seed points. As a result, they may not preserve well the boundaries of the point cloud with a non-uniform density. In this paper, we present a simple but effective supervoxel segmentation method for point clouds, which formalizes supervoxel segmentation as a subset selection problem. We develop an heuristic algorithm that utilizes local information to efficiently solve the subset selection problem. The proposed method can produce supervoxels with adaptive resolutions, and dose not rely the selection of seed points. The method is fully tested on three publicly available point cloud segmentation benchmarks, which cover the major point cloud types. The experimental results show that compared with the state-of-the-art supervoxel segmentation methods, the supervoxels extracted using our method preserve the object boundaries and small structures more effectively, which is reflected in a higher boundary recall and lower under-segmentation error.
Impact of p-GaN Gate Length on Performance of AlGaN/GaN Normally-off HEMT Devices
Authors: Sanjit Kumar Swain, Sudhansu Mohan Biswal, Umakanta Nanda, D. Siva Patro, Suraj Kumar Nayak and Birendra Biswal
Date: 2018-03-31 00:00:00
Proceeding: Springer 2018 3

In this work, we have studied the effect of variation in length of GaN gate
with p-type doping concentration on the DC performances of AlGaN/GaN normallyoff
HEMT using 2D Atlas TCAD simulator. A comprehensive simulation is undertaken
on the proposed device to examine different performance parameters such as
drain current, transconductance factor, energy band diagram, and surface potential
with respect to change in p-type GaN gate lengths. The gate lengths are varied from
60 to 90 nm, and it is noticed from the simulation results that with a decrease in gate
length the drain current increases and transconductance increases. A proper optimization
of gate length is indispensable to preserve the normally-off mode operation
and at the same time enhancing certain performance parameters.
Multi-objective radiomics model for predicting distant failure in lung SBRT
Authors: Zhou Z1, Folkert M, Iyengar P, Westover K, Zhang Y, Choy H, Timmerman R, Jiang S, Wang J.
Date: 2017-06-07 00:00:00
Proceeding: journal 2017 6
Tasks: Specificity

Stereotactic body radiation therapy (SBRT) has demonstrated high local control rates in early stage non-small cell lung cancer patients who are not ideal surgical candidates. However, distant failure after SBRT is still common. For patients at high risk of early distant failure after SBRT treatment, additional systemic therapy may reduce the risk of distant relapse and improve overall survival. Therefore, a strategy that can correctly stratify patients at high risk of failure is needed. The field of radiomics holds great potential in predicting treatment outcomes by using high-throughput extraction of quantitative imaging features. The construction of predictive models in radiomics is typically based on a single objective such as overall accuracy or the area under the curve (AUC). However, because of imbalanced positive and negative events in the training datasets, a single objective may not be ideal to guide model construction. To overcome these limitations, we propose a multi-objective radiomics model that simultaneously considers sensitivity and specificity as objective functions. To design a more accurate and reliable model, an iterative multi-objective immune algorithm (IMIA) was proposed to optimize these objective functions. The multi-objective radiomics model is more sensitive than the single-objective model, while maintaining the same levels of specificity and AUC. The IMIA performs better than the traditional immune-inspired multi-objective algorithm.
Randomized Kaczmarz Algorithm for Massive MIMO Systems with Channel Estimation and Spatial Correlation
Authors: Victor Croisfelt Rodrigues, Jose Carlos Marinello Filho, Taufik Abrao
Date: 2019-04-08 00:00:00
Tasks: Fairness

To exploit the benefits of massive multiple-input multiple-output (M-MIMO) technology in scenarios where base stations (BSs) need to be cheap and equipped with simple hardware, the computational complexity of classical signal processing schemes for spatial multiplexing of users shall be reduced. This calls for suboptimal designs that perform well the combining/precoding steps and simultaneously achieve low computational complexities. An approach based on the iterative Kaczmarz algorithm (KA) has been recently investigated, assuring well execution without the knowledge of second order moments of the wireless channels in the BS, and with easiness, since no tuning parameters, besides the number of iterations, are required. In fact, the randomized version of KA (rKA) has been used in this context due to global convergence properties. Herein, modifications are proposed on this first rKA-based attempt, aiming to improve its performance-complexity trade-off solution for M-MIMO systems. We observe that long-term channel effects degrade the rate of convergence of the rKA-based schemes. This issue is then tackled herein by means of a hybrid rKA initialization proposal that lands within the region of convexity of the algorithm and assures fairness to the communication system. The effectiveness of our proposal is illustrated through numerical results which bring more realistic system conditions in terms of channel estimation and spatial correlation than those used so far. We also characterize the computational complexity of the proposed rKA scheme, deriving upper bounds for the number of iterations. A case study focused on a dense urban application scenario is used to gather new insights on the feasibility of the proposed scheme to cope with the inserted BS constraints.