pwc-archive/papers-with-abstracts huggingface dataset:
Homogeneous Online Transfer Learning with Online Distribution Discrepancy Minimization
Authors: Yuntao Du, Zhiwen Tan, Qian Chen, Yi Zhang, Chongjun Wang
Date: 2019-12-31 00:00:00
Tasks: Transfer Learning

Transfer learning has been demonstrated to be successful and essential in diverse applications, which transfers knowledge from related but different source domains to the target domain. Online transfer learning(OTL) is a more challenging problem where the target data arrive in an online manner. Most OTL methods combine source classifier and target classifier directly by assigning a weight to each classifier, and adjust the weights constantly. However, these methods pay little attention to reducing the distribution discrepancy between domains. In this paper, we propose a novel online transfer learning method which seeks to find a new feature representation, so that the marginal distribution and conditional distribution discrepancy can be online reduced simultaneously. We focus on online transfer learning with multiple source domains and use the Hedge strategy to leverage knowledge from source domains. We analyze the theoretical properties of the proposed algorithm and provide an upper mistake bound. Comprehensive experiments on two real-world datasets show that our method outperforms state-of-the-art methods by a large margin.
Image Seam-Carving by Controlling Positional Distribution of Seams
Authors: Mahdi Ahmadi, Nader Karimi, Shadrokh Samavi
Date: 2019-12-31 00:00:00
Tasks: Image Quality Assessment, Image Retargeting

Image retargeting is a new image processing task that renders the change of aspect ratio in images. One of the most famous image-retargeting algorithms is seam-carving. Although seam-carving is fast and straightforward, it usually distorts the images. In this paper, we introduce a new seam-carving algorithm that not only has the simplicity of the original seam-carving but also lacks the usual unwanted distortion existed in the original method. The positional distribution of seams is introduced. We show that the proposed method outperforms the original seam-carving in terms of retargeted image quality assessment and seam coagulation measures.
A Modern Introduction to Online Learning
Authors: Francesco Orabona
Date: 2019-12-31 00:00:00
Tasks: All, Multi-Armed Bandits

In this monograph, I introduce the basic concepts of Online Learning through a modern view of Online Convex Optimization. Here, online learning refers to the framework of regret minimization under worst-case assumptions. I present first-order and second-order algorithms for online learning with convex losses, in Euclidean and non-Euclidean settings. All the algorithms are clearly presented as instantiation of Online Mirror Descent or Follow-The-Regularized-Leader and their variants. Particular attention is given to the issue of tuning the parameters of the algorithms and learning in unbounded domains, through adaptive and parameter-free online learning algorithms. Non-convex losses are dealt through convex surrogate losses and through randomization. The bandit setting is also briefly discussed, touching on the problem of adversarial and stochastic multi-armed bandits. These notes do not require prior knowledge of convex analysis and all the required mathematical tools are rigorously explained. Moreover, all the included proofs have been carefully chosen to be as simple and as short as possible.
A Robust Comparison of the KDDCup99 and NSL-KDD IoT Network Intrusion Detection Datasets Through Various Machine Learning Algorithms
Authors: Suchet Sapre, Pouyan Ahmadi, Khondkar Islam
Date: 2019-12-31 00:00:00
Tasks: Intrusion Detection, Network Intrusion Detection

In recent years, as intrusion attacks on IoT networks have grown exponentially, there is an immediate need for sophisticated intrusion detection systems (IDSs). A vast majority of current IDSs are data-driven, which means that one of the most important aspects of this area of research is the quality of the data acquired from IoT network traffic. Two of the most cited intrusion detection datasets are the KDDCup99 and the NSL-KDD. The main goal of our project was to conduct a robust comparison of both datasets by evaluating the performance of various Machine Learning (ML) classifiers trained on them with a larger set of classification metrics than previous researchers. From our research, we were able to conclude that the NSL-KDD dataset is of a higher quality than the KDDCup99 dataset as the classifiers trained on it were on average 20.18% less accurate. This is because the classifiers trained on the KDDCup99 dataset exhibited a bias towards the redundancies within it, allowing them to achieve higher accuracies.
Investigating Wave Energy Potential in Southern Coasts of the Caspian Sea Using Grey Wolf Optimizer Algorithm
Authors: Erfan Amini, Seyed Taghi Omid Naeeni, Pedram Ghaderi, Fereidoun Amini
Date: 2019-12-31 00:00:00

There is a significantly accelerating trend in the application of the marine wave energy converters in recent years. As a result, it is imperative to adopt a suitable point for implementing these systems. Besides, the Caspian Sea, as one of the most important marine renewable energy sources in Asia, is capable of supplying the coastal areas with a large amount of energy. Therefore, areas around nine ports in the southern coasts of the Caspian Sea were selected to measure their wave energy potential. Initially, the amount of energy on these points was measured using the irregular energy theory. It was observed that the wave power was higher in the southwestern areas (within the Kiashahr coast and Anzali port) than the southeastern areas. A new approach was developed to compare these points and measure their fitnesses in supplying the maximum energy using the Grey Wolf optimizer (GWO) algorithm and time history analysis. In this method, the optimal parameters were first extracted from the algorithm for assessing the points within the southern areas of the Caspian Sea. These values were regarded as the assessment indices. Then, the fitness of each point was obtained using the correlation function and the norm vector to present the most optimal position with maximum wave energy exploitation potential. This new approach was validated with analytical data, and its accuracy in predicting and comparing the wave power on different points was approved. Finally, by a side-by-side comparison of the parameters affecting the wave energy, the optimum range of significant wave height and wave energy period was achieved.
AdderNet: Do We Really Need Multiplications in Deep Learning?
Authors: Hanting Chen, Yunhe Wang, Chunjing Xu, Boxin Shi, Chao Xu, Qi Tian, Chang Xu
Date: 2019-12-31 00:00:00
Proceeding: CVPR 2020 6
Tasks: Deep Learning

Compared with cheap addition operation, multiplication operation is of much higher computation complexity. The widely-used convolutions in deep neural networks are exactly cross-correlation to measure the similarity between input feature and convolution filters, which involves massive multiplications between float values. In this paper, we present adder networks (AdderNets) to trade these massive multiplications in deep neural networks, especially convolutional neural networks (CNNs), for much cheaper additions to reduce computation costs. In AdderNets, we take the $\ell_1$-norm distance between filters and input feature as the output response. The influence of this new similarity measure on the optimization of neural network have been thoroughly analyzed. To achieve a better performance, we develop a special back-propagation approach for AdderNets by investigating the full-precision gradient. We then propose an adaptive learning rate strategy to enhance the training procedure of AdderNets according to the magnitude of each neuron's gradient. As a result, the proposed AdderNets can achieve 74.9% Top-1 accuracy 91.7% Top-5 accuracy using ResNet-50 on the ImageNet dataset without any multiplication in convolution layer. The codes are publicly available at: https://github.com/huaweinoah/AdderNet.
Comparison of object detection methods for crop damage assessment using deep learning
Authors: Ali HamidiSepehr, Seyed Vahid Mirnezami, Jason K. Ward
Date: 2019-12-31 00:00:00
Tasks: Disaster Response, Object, object-detection, Object Detection

Severe weather events can cause large financial losses to farmers. Detailed information on the location and severity of damage will assist farmers, insurance companies, and disaster response agencies in making wise post-damage decisions. The goal of this study was a proof-of-concept to detect damaged crop areas from aerial imagery using computer vision and deep learning techniques. A specific objective was to compare existing object detection algorithms to determine which was best suited for crop damage detection. Two modes of crop damage common in maize (corn) production were simulated: stalk lodging at the lowest ear and stalk lodging at ground level. Simulated damage was used to create a training and analysis data set. An unmanned aerial system (UAS) equipped with a RGB camera was used for image acquisition. Three popular object detectors (Faster R-CNN, YOLOv2, and RetinaNet) were assessed for their ability to detect damaged regions in a field. Average precision was used to compare object detectors. YOLOv2 and RetinaNet were able to detect crop damage across multiple late-season growth stages. Faster R-CNN was not successful as the other two advanced detectors. Detecting crop damage at later growth stages was more difficult for all tested object detectors. Weed pressure in simulated damage plots and increased target density added additional complexity.
CASE: Context-Aware Semantic Expansion
Authors: Jialong Han, Aixin Sun, Haisong Zhang, Chenliang Li, Shuming Shi
Date: 2019-12-31 00:00:00
Tasks: Word Sense Disambiguation

In this paper, we define and study a new task called Context-Aware Semantic Expansion (CASE). Given a seed term in a sentential context, we aim to suggest other terms that well fit the context as the seed. CASE has many interesting applications such as query suggestion, computer-assisted writing, and word sense disambiguation, to name a few. Previous explorations, if any, only involve some similar tasks, and all require human annotations for evaluation. In this study, we demonstrate that annotations for this task can be harvested at scale from existing corpora, in a fully automatic manner. On a dataset of 1.8 million sentences thus derived, we propose a network architecture that encodes the context and seed term separately before suggesting alternative terms. The context encoder in this architecture can be easily extended by incorporating seed-aware attention. Our experiments demonstrate that competitive results are achieved with appropriate choices of context encoder and attention scoring function.
PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detection
Authors: Shaoshuai Shi, Chaoxu Guo, Li Jiang, Zhe Wang, Jianping Shi, Xiaogang Wang, Hongsheng Li
Date: 2019-12-31 00:00:00
Proceeding: CVPR 2020 6
Tasks: 3D Object Detection, Object, object-detection, Object Detection, Robust 3D Object Detection

We present a novel and high-performance 3D object detection framework, named PointVoxel-RCNN (PV-RCNN), for accurate 3D object detection from point clouds. Our proposed method deeply integrates both 3D voxel Convolutional Neural Network (CNN) and PointNet-based set abstraction to learn more discriminative point cloud features. It takes advantages of efficient learning and high-quality proposals of the 3D voxel CNN and the flexible receptive fields of the PointNet-based networks. Specifically, the proposed framework summarizes the 3D scene with a 3D voxel CNN into a small set of keypoints via a novel voxel set abstraction module to save follow-up computations and also to encode representative scene features. Given the high-quality 3D proposals generated by the voxel CNN, the RoI-grid pooling is proposed to abstract proposal-specific features from the keypoints to the RoI-grid points via keypoint set abstraction with multiple receptive fields. Compared with conventional pooling operations, the RoI-grid feature points encode much richer context information for accurately estimating object confidences and locations. Extensive experiments on both the KITTI dataset and the Waymo Open dataset show that our proposed PV-RCNN surpasses state-of-the-art 3D detection methods with remarkable margins by using only point clouds. Code is available at https://github.com/open-mmlab/OpenPCDet.
Definitions and Semantic Simulations Based on Object-Oriented Analysis and Modeling
Authors: Robert B. Allen
Date: 2019-12-31 00:00:00

We have proposed going beyond traditional ontologies to use rich semantics implemented in programming languages for modeling. In this paper, we discuss the application of executable semantic models to two examples, first a structured definition of a waterfall and second the cardiopulmonary system. We examine the components of these models and the way those components interact. Ultimately, such models should provide the basis for direct representation.
Modeling Neural Architecture Search Methods for Deep Networks
Authors: Emad Malekhosseini, Mohsen Hajabdollahi, Nader Karimi, Shadrokh Samavi
Date: 2019-12-31 00:00:00
Tasks: Neural Architecture Search

There are many research works on the designing of architectures for the deep neural networks (DNN), which are named neural architecture search (NAS) methods. Although there are many automatic and manual techniques for NAS problems, there is no unifying model in which these NAS methods can be explored and compared. In this paper, we propose a general abstraction model for NAS methods. By using the proposed framework, it is possible to compare different design approaches for categorizing and identifying critical areas of interest in designing DNN architectures. Also, under this framework, different methods in the NAS area are summarized; hence a better view of their advantages and disadvantages is possible.
Diversity Transfer Network for Few-Shot Learning
Authors: Mengting Chen, Yuxin Fang, Xinggang Wang, Heng Luo, Yifeng Geng, Xin-Yu Zhang, Chang Huang, Wenyu Liu, Bo wang
Date: 2019-12-31 00:00:00
Tasks: Diversity, Few-Shot Learning

Few-shot learning is a challenging task that aims at training a classifier for unseen classes with only a few training examples. The main difficulty of few-shot learning lies in the lack of intra-class diversity within insufficient training samples. To alleviate this problem, we propose a novel generative framework, Diversity Transfer Network (DTN), that learns to transfer latent diversities from known categories and composite them with support features to generate diverse samples for novel categories in feature space. The learning problem of the sample generation (i.e., diversity transfer) is solved via minimizing an effective meta-classification loss in a single-stage network, instead of the generative loss in previous works. Besides, an organized auxiliary task co-training over known categories is proposed to stabilize the meta-training process of DTN. We perform extensive experiments and ablation studies on three datasets, i.e., \emph{mini}ImageNet, CIFAR100 and CUB. The results show that DTN, with single-stage training and faster convergence speed, obtains the state-of-the-art results among the feature generation based few-shot learning methods. Code and supplementary material are available at: \texttt{https://github.com/Yuxin-CV/DTN}
Modeling Teacher-Student Techniques in Deep Neural Networks for Knowledge Distillation
Authors: Sajjad Abbasi, Mohsen Hajabdollahi, Nader Karimi, Shadrokh Samavi
Date: 2019-12-31 00:00:00
Tasks: Knowledge Distillation

Knowledge distillation (KD) is a new method for transferring knowledge of a structure under training to another one. The typical application of KD is in the form of learning a small model (named as a student) by soft labels produced by a complex model (named as a teacher). Due to the novel idea introduced in KD, recently, its notion is used in different methods such as compression and processes that are going to enhance the model accuracy. Although different techniques are proposed in the area of KD, there is a lack of a model to generalize KD techniques. In this paper, various studies in the scope of KD are investigated and analyzed to build a general model for KD. All the methods and techniques in KD can be summarized through the proposed model. By utilizing the proposed model, different methods in KD are better investigated and explored. The advantages and disadvantages of different approaches in KD can be better understood and develop a new strategy for KD can be possible. Using the proposed model, different KD methods are represented in an abstract view.
Deep Learning on Image Denoising: An overview
Authors: Chunwei Tian, Lunke Fei, Wenxian Zheng, Yong Xu, WangMeng Zuo, Chia-Wen Lin
Date: 2019-12-31 00:00:00
Tasks: Deep Learning, Denoising, Image Denoising

Deep learning techniques have received much attention in the area of image denoising. However, there are substantial differences in the various types of deep learning methods dealing with image denoising. Specifically, discriminative learning based on deep learning can ably address the issue of Gaussian noise. Optimization models based on deep learning are effective in estimating the real noise. However, there has thus far been little related research to summarize the different deep learning techniques for image denoising. In this paper, we offer a comparative study of deep techniques in image denoising. We first classify the deep convolutional neural networks (CNNs) for additive white noisy images; the deep CNNs for real noisy images; the deep CNNs for blind denoising and the deep CNNs for hybrid noisy images, which represents the combination of noisy, blurred and low-resolution images. Then, we analyze the motivations and principles of the different types of deep learning methods. Next, we compare the state-of-the-art methods on public denoising datasets in terms of quantitative and qualitative analysis. Finally, we point out some potential challenges and directions of future research.
Schrödinger Bridge Samplers
Authors: Espen Bernton, Jeremy Heng, Arnaud Doucet, Pierre E. Jacob
Date: 2019-12-31 00:00:00

Consider a reference Markov process with initial distribution $\pi_{0}$ and transition kernels $\{M_{t}\}_{t\in[1:T]}$, for some $T\in\mathbb{N}$. Assume that you are given distribution $\pi_{T}$, which is not equal to the marginal distribution of the reference process at time $T$. In this scenario, Schr\"odinger addressed the problem of identifying the Markov process with initial distribution $\pi_{0}$ and terminal distribution equal to $\pi_{T}$ which is the closest to the reference process in terms of Kullback--Leibler divergence. This special case of the so-called Schr\"odinger bridge problem can be solved using iterative proportional fitting, also known as the Sinkhorn algorithm. We leverage these ideas to develop novel Monte Carlo schemes, termed Schr\"odinger bridge samplers, to approximate a target distribution $\pi$ on $\mathbb{R}^{d}$ and to estimate its normalizing constant. This is achieved by iteratively modifying the transition kernels of the reference Markov chain to obtain a process whose marginal distribution at time $T$ becomes closer to $\pi_T = \pi$, via regression-based approximations of the corresponding iterative proportional fitting recursion. We report preliminary experiments and make connections with other problems arising in the optimal transport, optimal control and physics literatures.
Efficient Decremental Learning Algorithms for Broad Learning System
Authors: Hufei Zhu
Date: 2019-12-31 00:00:00
Tasks: Incremental Learning

The decremented learning algorithms are required in machine learning, to prune redundant nodes and remove obsolete inline training samples. In this paper, an efficient decremented learning algorithm to prune redundant nodes is deduced from the incremental learning algorithm 1 proposed in [9] for added nodes, and two decremented learning algorithms to remove training samples are deduced from the two incremental learning algorithms proposed in [10] for added inputs. The proposed decremented learning algorithm for reduced nodes utilizes the inverse Cholesterol factor of the Herminia matrix in the ridge inverse, to update the output weights recursively, as the incremental learning algorithm 1 for added nodes in [9], while that inverse Cholesterol factor is updated with an unitary transformation. The proposed decremented learning algorithm 1 for reduced inputs updates the output weights recursively with the inverse of the Herminia matrix in the ridge inverse, and updates that inverse recursively, as the incremental learning algorithm 1 for added inputs in [10].
Transform-Domain Classification of Human Cells based on DNA Methylation Datasets
Authors: Xueyuan Zhao, Dario Pompili
Date: 2019-12-31 00:00:00
Tasks: BIG-bench Machine Learning, Classification, domain classification, General Classification

A novel method to classify human cells is presented in this work based on the transform-domain method on DNA methylation data. DNA methylation profile variations are observed in human cells with the progression of disease stages, and the proposal is based on this DNA methylation variation to classify normal and disease cells including cancer cells. The cancer cell types investigated in this work cover hepatocellular (sample size n = 40), colorectal (n = 44), lung (n = 70) and endometrial (n = 87) cancer cells. A new pipeline is proposed integrating the DNA methylation intensity measurements on all the CpG islands by the transformation of Walsh-Hadamard Transform (WHT). The study reveals the three-step properties of the DNA methylation transform-domain data and the step values of association with the cell status. Further assessments have been carried out on the proposed machine learning pipeline to perform classification of the normal and cancer tissue cells. A number of machine learning classifiers are compared for whole sequence and WHT sequence classification based on public Whole-Genome Bisulfite Sequencing (WGBS) DNA methylation datasets. The WHT-based method can speed up the computation time by more than one order of magnitude compared with whole original sequence classification, while maintaining comparable classification accuracy by the selected machine learning classifiers. The proposed method has broad applications in expedited disease and normal human cell classifications by the epigenome and genome datasets.
C. H. Robinson Uses Heuristics to Solve Rich Vehicle Routing Problems
Authors: Ehsan Khodabandeh, Lawrence V. Snyder, John Dennis, Joshua Hammond, Cody Wanless
Date: 2019-12-31 00:00:00

We consider a wide family of vehicle routing problem variants with many complex and practical constraints, known as rich vehicle routing problems, which are faced on a daily basis by C.H. Robinson (CHR). Since CHR has many customers, each with distinct requirements, various routing problems with different objectives and constraints should be solved. We propose a set partitioning framework with a number of route generation algorithms, which have shown to be effective in solving a variety of different problems. The proposed algorithms have outperformed the existing technologies at CHR on 10 benchmark instances and since, have been embedded into the company's transportation planning and execution technology platform.