pwc-archive/papers-with-abstracts huggingface dataset:
CVR-Net: A deep convolutional neural network for coronavirus recognition from chest radiography images
Authors: Md. Kamrul Hasan, Md. Ashraful Alam, Md. Toufick E Elahi, Shidhartho Roy, Sifat Redwan Wahid
Date: 2020-07-21 00:00:00
Tasks: Computed Tomography (CT), Diagnostic

The novel Coronavirus Disease 2019 (COVID-19) is a global pandemic disease spreading rapidly around the world. A robust and automatic early recognition of COVID-19, via auxiliary computer-aided diagnostic tools, is essential for disease cure and control. The chest radiography images, such as Computed Tomography (CT) and X-ray, and deep Convolutional Neural Networks (CNNs), can be a significant and useful material for designing such tools. However, designing such an automated tool is challenging as a massive number of manually annotated datasets are not publicly available yet, which is the core requirement of supervised learning systems. In this article, we propose a robust CNN-based network, called CVR-Net (Coronavirus Recognition Network), for the automatic recognition of the coronavirus from CT or X-ray images. The proposed end-to-end CVR-Net is a multi-scale-multi-encoder ensemble model, where we have aggregated the outputs from two different encoders and their different scales to obtain the final prediction probability. We train and test the proposed CVR-Net on three different datasets, where the images have collected from different open-source repositories. We compare our proposed CVR-Net with state-of-the-art methods, which are trained and tested on the same datasets. We split three datasets into five different tasks, where each task has a different number of classes, to evaluate the multi-tasking CVR-Net. Our model achieves an overall F1-score & accuracy of 0.997 & 0.998; 0.963 & 0.964; 0.816 & 0.820; 0.961 & 0.961; and 0.780 & 0.780, respectively, for task-1 to task-5. As the CVR-Net provides promising results on the small datasets, it can be an auspicious computer-aided diagnostic tool for the diagnosis of coronavirus to assist the clinical practitioners and radiologists. Our source codes and model are publicly available at https://github.com/kamruleee51/CVR-Net.
Disentangling the Gauss-Newton Method and Approximate Inference for Neural Networks
Authors: Alexander Immer
Date: 2020-07-21 00:00:00
Tasks: Deep Learning, Gaussian Processes

In this thesis, we disentangle the generalized Gauss-Newton and approximate inference for Bayesian deep learning. The generalized Gauss-Newton method is an optimization method that is used in several popular Bayesian deep learning algorithms. Algorithms that combine the Gauss-Newton method with the Laplace and Gaussian variational approximation have recently led to state-of-the-art results in Bayesian deep learning. While the Laplace and Gaussian variational approximation have been studied extensively, their interplay with the Gauss-Newton method remains unclear. Recent criticism of priors and posterior approximations in Bayesian deep learning further urges the need for a deeper understanding of practical algorithms. The individual analysis of the Gauss-Newton method and Laplace and Gaussian variational approximations for neural networks provides both theoretical insight and new practical algorithms. We find that the Gauss-Newton method simplifies the underlying probabilistic model significantly. In particular, the combination of the Gauss-Newton method with approximate inference can be cast as inference in a linear or Gaussian process model. The Laplace and Gaussian variational approximation can subsequently provide a posterior approximation to these simplified models. This new disentangled understanding of recent Bayesian deep learning algorithms also leads to new methods: first, the connection to Gaussian processes enables new function-space inference algorithms. Second, we present a marginal likelihood approximation of the underlying probabilistic model to tune neural network hyperparameters. Finally, the identified underlying models lead to different methods to compute predictive distributions. In fact, we find that these prediction methods for Bayesian neural networks often work better than the default choice and solve a common issue with the Laplace approximation.
A Semismooth-Newton's-Method-Based Linearization and Approximation Approach for Kernel Support Vector Machines
Authors: Chen Jiang, Qingna Li
Date: 2020-07-21 00:00:00

Support Vector Machines (SVMs) are among the most popular and the best performing classification algorithms. Various approaches have been proposed to reduce the high computation and memory cost when training and predicting based on large-scale datasets with kernel SVMs. A popular one is the linearization framework, which successfully builds a bridge between the $L_1$-loss kernel SVM and the $L_1$-loss linear SVM. For linear SVMs, very recently, a semismooth Newton's method is proposed. It is shown to be very competitive and have low computational cost. Consequently, a natural question is whether it is possible to develop a fast semismooth Newton's algorithm for kernel SVMs. Motivated by this question and the idea in linearization framework, in this paper, we focus on the $L_2$-loss kernel SVM and propose a semismooth Newton's method based linearization and approximation approach for it. The main idea of this approach is to first set up an equivalent linear SVM, then apply the Nystr\"om method to approximate the kernel matrix, based on which a reduced linear SVM is obtained. Finally, the fast semismooth Newton's method is employed to solve the reduced linear SVM. We also provide some theoretical analyses on the approximation of the kernel matrix. The advantage of the proposed approach is that it maintains low computational cost and keeps a fast convergence rate. Results of extensive numerical experiments verify the efficiency of the proposed approach in terms of both predicting accuracy and speed.
Detecting the Insider Threat with Long Short Term Memory (LSTM) Neural Networks
Authors: Eduardo Lopez, Kamran Sartipi
Date: 2020-07-20 00:00:00

Information systems enable many organizational processes in every industry. The efficiencies and effectiveness in the use of information technologies create an unintended byproduct: misuse by existing users or somebody impersonating them - an insider threat. Detecting the insider threat may be possible if thorough analysis of electronic logs, capturing user behaviors, takes place. However, logs are usually very large and unstructured, posing significant challenges for organizations. In this study, we use deep learning, and most specifically Long Short Term Memory (LSTM) recurrent networks for enabling the detection. We demonstrate through a very large, anonymized dataset how LSTM uses the sequenced nature of the data for reducing the search space and making the work of a security analyst more effective.
Social Learning in Non-Stationary Environments
Authors: Etienne Boursier, Vianney Perchet, Marco Scarsini
Date: 2020-07-20 00:00:00

Potential buyers of a product or service, before making their decisions, tend to read reviews written by previous consumers. We consider Bayesian consumers with heterogeneous preferences, who sequentially decide whether to buy an item of unknown quality, based on previous buyers' reviews. The quality is multi-dimensional and may occasionally vary over time; the reviews are also multi-dimensional. In the simple uni-dimensional and static setting, beliefs about the quality are known to converge to its true value. Our paper extends this result in several ways. First, a multi-dimensional quality is considered, second, rates of convergence are provided, third, a dynamical Markovian model with varying quality is studied. In this dynamical setting the cost of learning is shown to be small.
Computing the Dirichlet-Multinomial Log-Likelihood Function
Authors: Djallel Bouneffouf
Date: 2020-07-17 00:00:00
Tasks: Form

Dirichlet-multinomial (DMN) distribution is commonly used to model over-dispersion in count data. Precise and fast numerical computation of the DMN log-likelihood function is important for performing statistical inference using this distribution, and remains a challenge. To address this, we use mathematical properties of the gamma function to derive a closed form expression for the DMN log-likelihood function. Compared to existing methods, calculation of the closed form has a lower computational complexity, hence is much faster without comprimising computational accuracy.
AM-DCGAN: Analog Memristive Hardware Accelerator for Deep Convolutional Generative Adversarial Networks
Authors: Olga Krestinskaya, Bhaskar Choubey, Alex Pappachen James
Date: 2020-06-20 00:00:00
Tasks: Generative Adversarial Network

Generative Adversarial Network (GAN) is a well known computationally complex algorithm requiring signficiant computational resources in software implementations including large amount of data to be trained. This makes its implementation in edge devices with conventional microprocessor hardware a slow and difficult task. In this paper, we propose to accelerate the computationally intensive GAN using memristive neural networks in analog domain. We present a fully analog hardware design of Deep Convolutional GAN (DCGAN) based on CMOS-memristive convolutional and deconvolutional networks simulated using 180nm CMOS technology.
Comparative Analysis of Polynomial and Rational Approximations of Hyperbolic Tangent Function for VLSI Implementation
Authors: Mahesh Chandra
Date: 2020-07-13 00:00:00
Tasks: object-detection, Object Detection, speech-recognition, Speech Recognition

Deep neural networks yield the state-of-the-art results in many computer vision and human machine interface applications such as object detection, speech recognition etc. Since, these networks are computationally expensive, customized accelerators are designed for achieving the required performance at lower cost and power. One of the key building blocks of these neural networks is non-linear activation function such as sigmoid, hyperbolic tangent (tanh), and ReLU. A low complexity accurate hardware implementation of the activation function is required to meet the performance and area targets of the neural network accelerators. Even though, various methods and implementations of tanh activation function have been published, a comparative study is missing. This paper presents comparative analysis of polynomial and rational methods and their hardware implementation.
Mitigating backdoor attacks in LSTM-based Text Classification Systems by Backdoor Keyword Identification
Authors: Chuanshuai Chen, Jiazhu Dai
Date: 2020-07-11 00:00:00
Tasks: Classification, Data Poisoning, General Classification, text-classification, Text Classification

It has been proved that deep neural networks are facing a new threat called backdoor attacks, where the adversary can inject backdoors into the neural network model through poisoning the training dataset. When the input containing some special pattern called the backdoor trigger, the model with backdoor will carry out malicious task such as misclassification specified by adversaries. In text classification systems, backdoors inserted in the models can cause spam or malicious speech to escape detection. Previous work mainly focused on the defense of backdoor attacks in computer vision, little attention has been paid to defense method for RNN backdoor attacks regarding text classification. In this paper, through analyzing the changes in inner LSTM neurons, we proposed a defense method called Backdoor Keyword Identification (BKI) to mitigate backdoor attacks which the adversary performs against LSTM-based text classification by data poisoning. This method can identify and exclude poisoning samples crafted to insert backdoor into the model from training data without a verified and trusted dataset. We evaluate our method on four different text classification datset: IMDB, DBpedia ontology, 20 newsgroups and Reuters-21578 dataset. It all achieves good performance regardless of the trigger sentences.
E3-targetPred: Prediction of E3-Target Proteins Using Deep Latent Space Encoding
Authors: Seongyong Park, Shujaat Khan, Abdul Wahab
Date: 2020-06-26 00:00:00

Understanding E3 ligase and target substrate interactions are important for cell biology and therapeutic development. However, experimental identification of E3 target relationships is not an easy task due to the labor-intensive nature of the experiments. In this article, a sequence-based E3-target prediction model is proposed for the first time. The proposed framework utilizes composition of k-spaced amino acid pairs (CKSAAP) to learn the relationship between E3 ligases and their target protein. A class separable latent space encoding scheme is also devised that provides a compressed representation of feature space. A thorough ablation study is performed to identify an optimal gap size for CKSAAP and the number of latent variables that can represent the E3-target relationship successfully. The proposed scheme is evaluated on an independent dataset for a variety of standard quantitative measures. In particular, it achieves an average accuracy of $70.63\%$ on an independent dataset. The source code and datasets used in the study are available at the author's GitHub page (https://github.com/psychemistz/E3targetPred).
Proceedings of the KG-BIAS Workshop 2020 at AKBC 2020
Authors: Edgar Meij, Tara Safavi, Chenyan Xiong, Gianluca Demartini, Miriam Redi, Fatma Özcan
Date: 2020-06-18 00:00:00
Tasks: Knowledge Graphs

The KG-BIAS 2020 workshop touches on biases and how they surface in knowledge graphs (KGs), biases in the source data that is used to create KGs, methods for measuring or remediating bias in KGs, but also identifying other biases such as how and which languages are represented in automatically constructed KGs or how personal KGs might incur inherent biases. The goal of this workshop is to uncover how various types of biases are introduced into KGs, investigate how to measure, and propose methods to remediate them.
MRAEA: An Efficient and Robust Entity Alignment Approach for Cross-lingual Knowledge Graph
Authors: Xin Mao, Wenting Wang, Huimin Xu, Man Lan, Yuanbin Wu
Date: 2020-02-03 00:00:00
Proceeding: The International Conference on Web Search and Data Mining (WSDM) 2020 2
Tasks: Entity Alignment, Entity Embeddings, Graph Neural Network, Knowledge Graphs, Relation

Entity alignment to find equivalent entities in cross-lingual Knowledge Graphs (KGs) plays a vital role in automatically integrating multiple KGs. Existing translation-based entity alignment methods jointly model the cross-lingual knowledge and monolingual knowledge into one unified optimization problem. On the other hand, the Graph Neural Network (GNN) based methods either ignore the node differentiations, or represent relation through entity or triple instances. They all fail to model the meta semantics embedded in relation nor complex relations such as n-to-n and multi-graphs. To tackle these challenges, we propose a novel Meta Relation Aware Entity Alignment (MRAEA) to directly model cross-lingual entity embeddings by attending over the node's incoming and outgoing neighbors and its connected relations' meta semantics. In addition, we also propose a simple and effective bi-directional iterative strategy to add new aligned seeds during training. Our experiments on all three benchmark entity alignment datasets show that our approach consistently outperforms the state-of-the-art methods, exceeding by 15%-58% on Hit@1. Through an extensive ablation study, we validate that the proposed meta relation aware representations, relation aware self-attention and bi-directional iterative strategy of new seed selection all make contributions to significant performance improvement. The code is available at https://github.com/MaoXinn/MRAEA.
PD-Sparse : A Primal and Dual Sparse Approach to Extreme Multiclass and Multilabel Classification
Authors: Ian En-Hsu Yen, Xiangru Huang, Pradeep Ravikumar, Kai Zhong, Inderjit S. Dhillon
Date: 2016-06-01 00:00:00
Proceeding: ICML 2016 6
Tasks: Classification, General Classification, Text Classification

We consider Multiclass and Multilabel classification with extremely large number of classes, of which only few are labeled to each instance. In such setting, standard methods that have training, prediction cost linear to the number of classes become intractable. State-of-the-art methods thus aim to reduce the complexity by exploiting correlation between labels under assumption that the similarity between labels can be captured by structures such as low-rank matrix or balanced tree. However, as the diversity of labels increases in the feature space, structural assumption can be easily violated, which leads to degrade in the testing performance. In this work, we show that a margin-maximizing loss with l1 penalty, in case of Extreme Classification, yields extremely sparse solution both in primal and in dual without sacrificing the expressive power of predictor. We thus propose a Fully-Corrective Block-Coordinate Frank-Wolfe (FC-BCFW) algorithm that exploits both primal and dual sparsity to achieve a complexity sublinear to the number of primal and dual variables. A bi-stochastic search method is proposed to further improve the efficiency. In our experiments on both Multiclass and Multilabel problems, the proposed method achieves significant higher accuracy than existing approaches of Extreme Classification with very competitive training and prediction time.
Version Control of Speaker Recognition Systems
Authors: Quan Wang, Ignacio Lopez Moreno
Date: 2020-07-23 00:00:00
Tasks: Speaker Recognition

This paper discusses one of the most challenging practical engineering problems in speaker recognition systems - the version control of models and user profiles. A typical speaker recognition system consists of two stages: the enrollment stage, where a profile is generated from user-provided enrollment audio; and the runtime stage, where the voice identity of the runtime audio is compared against the stored profiles. As technology advances, the speaker recognition system needs to be updated for better performance. However, if the stored user profiles are not updated accordingly, version mismatch will result in meaningless recognition results. In this paper, we describe different version control strategies for speaker recognition systems that had been carefully studied at Google from years of engineering practice. These strategies are categorized into three groups according to how they are deployed in the production environment: device-side deployment, server-side deployment, and hybrid deployment. To compare different strategies with quantitative metrics under various network configurations, we present SpeakerVerSim, an easily-extensible Python-based simulation framework for different server-side deployment strategies of speaker recognition systems.
TrueSkill Through Time: Revisiting the History of Chess
Authors: Pierre Dangauthier, Ralf Herbrich, Tom Minka, Thore Graepel
Date: 2017-12-03 00:00:00
Proceeding: NIPS 2007 12
Tasks: Time Series, Time Series Analysis

We extend the Bayesian skill rating system TrueSkill to infer entire time
series of skills of players by smoothing through time instead of filtering.
The skill of each participating player, say, every year is represented by a
latent skill variable which is affected by the relevant game outcomes that
year, and coupled with the skill variables of the previous and subsequent
year. Inference in the resulting factor graph is carried out by approximate
message passing (EP) along the time series of skills. As before the system
tracks the uncertainty about player skills, explicitly models draws, can deal
with any number of competing entities and can infer individual skills from
team results. We extend the system to estimate player-specific draw margins. Based on these models we present an analysis of the skill curves of
important players in the history of chess over the past 150 years. Results
include plots of players’ lifetime skill development as well as the ability to
compare the skills of different players across time. Our results indicate that
a) the overall playing strength has increased over the past 150 years, and
b) that modelling a player’s ability to force a draw provides significantly
better predictive power.
Guided neural style transfer for shape stylization
Authors: Gantugs Atarsaikhan, Brian Kenji Iwana, Seiichi Uchida
Date: 2020-06-04 00:00:00
Proceeding: PLOS ONE 2020 6
Tasks: BIG-bench Machine Learning, Style Transfer

Designing logos, typefaces, and other decorated shapes can require professional skills. In this paper, we aim to produce new and unique decorated shapes by stylizing ordinary shapes with machine learning. Specifically, we combined parametric and non-parametric neural style transfer algorithms to transfer both local and global features. Furthermore, we introduced a distance-based guiding to the neural style transfer process, so that only the foreground shape will be decorated. Lastly, qualitative evaluation and ablation studies are provided to demonstrate the usefulness of the proposed method.
Deep Co-Training with Task Decomposition for Semi-Supervised Domain Adaptation
Authors: Luyu Yang, Yan Wang, Mingfei Gao, Abhinav Shrivastava, Kilian Q. Weinberger, Wei-Lun Chao, Ser-Nam Lim
Date: 2020-07-24 00:00:00
Proceeding: ICCV 2021 10
Tasks: Domain Adaptation, Semi-supervised Domain Adaptation, Unsupervised Domain Adaptation

Semi-supervised domain adaptation (SSDA) aims to adapt models trained from a labeled source domain to a different but related target domain, from which unlabeled data and a small set of labeled data are provided. Current methods that treat source and target supervision without distinction overlook their inherent discrepancy, resulting in a source-dominated model that has not effectively used the target supervision. In this paper, we argue that the labeled target data needs to be distinguished for effective SSDA, and propose to explicitly decompose the SSDA task into two sub-tasks: a semi-supervised learning (SSL) task in the target domain and an unsupervised domain adaptation (UDA) task across domains. By doing so, the two sub-tasks can better leverage the corresponding supervision and thus yield very different classifiers. To integrate the strengths of the two classifiers, we apply the well-established co-training framework, in which the two classifiers exchange their high confident predictions to iteratively "teach each other" so that both classifiers can excel in the target domain. We call our approach Deep Co-training with Task decomposition (DeCoTa). DeCoTa requires no adversarial training and is easy to implement. Moreover, DeCoTa is well-founded on the theoretical condition of when co-training would succeed. As a result, DeCoTa achieves state-of-the-art results on several SSDA datasets, outperforming the prior art by a notable 4% margin on DomainNet. Code is available at https://github.com/LoyoYang/DeCoTa