pwc-archive/papers-with-abstracts huggingface dataset:
Neural Shuffle-Exchange Networks -- Sequence Processing in O(n log n) Time
Authors: Kārlis Freivalds, Emīls Ozoliņš, Agris Šostaks
Date: 2019-07-18 00:00:00
Tasks: LAMBADA, Language Modeling, Language Modelling, Music Generation, Question Answering

A key requirement in sequence to sequence processing is the modeling of long range dependencies. To this end, a vast majority of the state-of-the-art models use attention mechanism which is of O($n^2$) complexity that leads to slow execution for long sequences. We introduce a new Shuffle-Exchange neural network model for sequence to sequence tasks which have O(log n) depth and O(n log n) total complexity. We show that this model is powerful enough to infer efficient algorithms for common algorithmic benchmarks including sorting, addition and multiplication. We evaluate our architecture on the challenging LAMBADA question answering dataset and compare it with the state-of-the-art models which use attention. Our model achieves competitive accuracy and scales to sequences with more than a hundred thousand of elements. We are confident that the proposed model has the potential for building more efficient architectures for processing large interrelated data in language modeling, music generation and other application domains.
A feasibility study of deep neural networks for the recognition of banknotes regarding central bank requirements
Authors: Julia Schulte, Daniel Staps, Alexander Lampe
Date: 2019-07-18 00:00:00
Tasks: Classification, General Classification, GPU

This paper contains a feasibility study of deep neural networks for the classification of Euro banknotes with respect to requirements of central banks on the ATM and high speed sorting industry. Instead of concentrating on the accuracy for a large number of classes as in the famous ImageNet Challenge we focus thus on conditions with few classes and the requirement of rejection of images belonging clearly to neither of the trained classes (i.e. classification in a so-called 0-class). These special requirements are part of frameworks defined by central banks as the European Central Bank and are met by current ATMs and high speed sorting machines. We also consider training and classification time on state of the art GPU hardware. The study concentrates on the banknote recognition whereas banknote class dependent authenticity and fitness checks are a topic of its own which is not considered in this work.
A Strong Feature Representation for Siamese Network Tracker
Authors: Zhipeng Zhou, Rui Zhang, Dong Yin
Date: 2019-07-18 00:00:00
Tasks: Object Tracking

Object tracking has important application in assistive technologies for personalized monitoring. Recent trackers choosing AlexNet as their backbone to extract features have gained great success. However, AlexNet is too shallow to form a strong feature representation, the tracker based on the Siamese network have an accuracy gap compared with state-of-the-art algorithms. To solve this problem, this paper proposes a tracker called SiamPF. Firstly, the modified pre-trained VGG16 network is fine-tuned as the backbone. Secondly, an AlexNet-like branch is added after the third convolutional layer and merged with the response map of the backbone network to form a preliminary strong feature representation. And then, a channel attention block is designed to adaptively select the contribution features. Finally, the APCE is modified to process the response map to reduce interference and focus the tracker on the target. Our SiamPF only used ILSVRC2015-VID for training, but it achieved excellent performance on OTB-2013 / OTB-2015 / VOT2015 / VOT2017, while maintaining the real-time performance of 41FPS on the GTX 1080Ti.
Post-Earthquake Assessment of Buildings Using Deep Learning
Authors: Dhananjay Nahata, Harish Kumar Mulchandani, Suraj Bansal, G Muthukumar
Date: 2019-07-18 00:00:00
Tasks: Deep Learning, Transfer Learning

Classification of the extent of damage suffered by a building in a seismic event is crucial from the safety perspective and repairing work. In this study, authors have proposed a CNN based autonomous damage detection model. Over 1200 images of different types of buildings-1000 for training and 200 for testing classified into 4 categories according to the extent of damage suffered. Categories are namely, no damage, minor damage, major damage, and collapse. Trained network tested by the application of various algorithms with different learning rates. The most optimum results were obtained on the application of VGG16 transfer learning model with a learning rate of 1e-5 as it gave a training accuracy of 97.85% and validation accuracy of up to 89.38%. The model developed has real-time application in the event of an earthquake.
Autoencoder-Based Incremental Class Learning without Retraining on Old Data
Authors: Euntae Choi, Kyungmi Lee, Ki-Young Choi
Date: 2019-07-18 00:00:00
Tasks: Continual Learning, General Classification

Incremental class learning, a scenario in continual learning context where classes and their training data are sequentially and disjointedly observed, challenges a problem widely known as catastrophic forgetting. In this work, we propose a novel incremental class learning method that can significantly reduce memory overhead compared to previous approaches. Apart from conventional classification scheme using softmax, our model bases on an autoencoder to extract prototypes for given inputs so that no change in its output unit is required. It stores only the mean of prototypes per class to perform metric-based classification, unlike rehearsal approaches which rely on large memory or generative model. To mitigate catastrophic forgetting, regularization methods are applied on our model when a new task is encountered. We evaluate our method by experimenting on CIFAR-100 and CUB-200-2011 and show that its performance is comparable to the state-of-the-art method with much lower additional memory cost.
Understanding Video Content: Efficient Hero Detection and Recognition for the Game "Honor of Kings"
Authors: Wentao Yao, Zixun Sun, Xiao Chen
Date: 2019-07-18 00:00:00
Tasks: Template Matching

In order to understand content and automatically extract labels for videos of the game "Honor of Kings", it is necessary to detect and recognize characters (called "hero") together with their camps in the game video. In this paper, we propose an efficient two-stage algorithm to detect and recognize heros in game videos. First, we detect all heros in a video frame based on blood bar template-matching method, and classify them according to their camps (self/ friend/ enemy). Then we recognize the name of each hero using one or more deep convolution neural networks. Our method needs almost no work for labelling training and testing samples in the recognition stage. Experiments show its efficiency and accuracy in the task of hero detection and recognition in game videos.
Event-based Feature Extraction Using Adaptive Selection Thresholds
Authors: Saeed Afshar, Ying Xu, Jonathan Tapson, André van Schaik, Gregory Cohen
Date: 2019-07-18 00:00:00
Tasks: Benchmarking

Unsupervised feature extraction algorithms form one of the most important building blocks in machine learning systems. These algorithms are often adapted to the event-based domain to perform online learning in neuromorphic hardware. However, not designed for the purpose, such algorithms typically require significant simplification during implementation to meet hardware constraints, creating trade offs with performance. Furthermore, conventional feature extraction algorithms are not designed to generate useful intermediary signals which are valuable only in the context of neuromorphic hardware limitations. In this work a novel event-based feature extraction method is proposed that focuses on these issues. The algorithm operates via simple adaptive selection thresholds which allow a simpler implementation of network homeostasis than previous works by trading off a small amount of information loss in the form of missed events that fall outside the selection thresholds. The behavior of the selection thresholds and the output of the network as a whole are shown to provide uniquely useful signals indicating network weight convergence without the need to access network weights. A novel heuristic method for network size selection is proposed which makes use of noise events and their feature representations. The use of selection thresholds is shown to produce network activation patterns that predict classification accuracy allowing rapid evaluation and optimization of system parameters without the need to run back-end classifiers. The feature extraction method is tested on both the N-MNIST benchmarking dataset and a dataset of airplanes passing through the field of view. Multiple configurations with different classifiers are tested with the results quantifying the resultant performance gains at each processing stage.
Prioritized Guidance for Efficient Multi-Agent Reinforcement Learning Exploration
Authors: Qisheng Wang, Qichao Wang
Date: 2019-07-18 00:00:00
Tasks: Multi-agent Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Exploration efficiency is a challenging problem in multi-agent reinforcement learning (MARL), as the policy learned by confederate MARL depends on the collaborative approach among multiple agents. Another important problem is the less informative reward restricts the learning speed of MARL compared with the informative label in supervised learning. In this work, we leverage on a novel communication method to guide MARL to accelerate exploration and propose a predictive network to forecast the reward of current state-action pair and use the guidance learned by the predictive network to modify the reward function. An improved prioritized experience replay is employed to better take advantage of the different knowledge learned by different agents which utilizes Time-difference (TD) error more effectively. Experimental results demonstrates that the proposed algorithm outperforms existing methods in cooperative multi-agent environments. We remark that this algorithm can be extended to supervised learning to speed up its training.
Growing a Brain: Fine-Tuning by Increasing Model Capacity
Authors: Yu-Xiong Wang, Deva Ramanan, Martial Hebert
Date: 2019-07-18 00:00:00
Proceeding: CVPR 2017 7
Tasks: Developmental Learning

CNNs have made an undeniable impact on computer vision through the ability to learn high-capacity models with large annotated training sets. One of their remarkable properties is the ability to transfer knowledge from a large source dataset to a (typically smaller) target dataset. This is usually accomplished through fine-tuning a fixed-size network on new target data. Indeed, virtually every contemporary visual recognition system makes use of fine-tuning to transfer knowledge from ImageNet. In this work, we analyze what components and parameters change during fine-tuning, and discover that increasing model capacity allows for more natural model adaptation through fine-tuning. By making an analogy to developmental learning, we demonstrate that "growing" a CNN with additional units, either by widening existing layers or deepening the overall network, significantly outperforms classic fine-tuning approaches. But in order to properly grow a network, we show that newly-added units must be appropriately normalized to allow for a pace of learning that is consistent with existing units. We empirically validate our approach on several benchmark datasets, producing state-of-the-art results.
An Adaptive Approach for Anomaly Detector Selection and Fine-Tuning in Time Series
Authors: Hui Ye, Xiaopeng Ma, Qingfeng Pan, Huaqiang Fang, Hang Xiang, Tongzhen Shao
Date: 2019-07-18 00:00:00
Tasks: Anomaly Detection, Time Series, Time Series Analysis, Transfer Learning

The anomaly detection of time series is a hotspot of time series data mining. The own characteristics of different anomaly detectors determine the abnormal data that they are good at. There is no detector can be optimizing in all types of anomalies. Moreover, it still has difficulties in industrial production due to problems such as a single detector can't be optimized at different time windows of the same time series. This paper proposes an adaptive model based on time series characteristics and selecting appropriate detector and run-time parameters for anomaly detection, which is called ATSDLN(Adaptive Time Series Detector Learning Network). We take the time series as the input of the model, and learn the time series representation through FCN. In order to realize the adaptive selection of detectors and run-time parameters according to the input time series, the outputs of FCN are the inputs of two sub-networks: the detector selection network and the run-time parameters selection network. In addition, the way that the variable layer width design of the parameter selection sub-network and the introduction of transfer learning make the model be with more expandability. Through experiments, it is found that ATSDLN can select appropriate anomaly detector and run-time parameters, and have strong expandability, which can quickly transfer. We investigate the performance of ATSDLN in public data sets, our methods outperform other methods in most cases with higher effect and better adaptation. We also show experimental results on public data sets to demonstrate how model structure and transfer learning affect the effectiveness.
Multi-year Long-term Load Forecast for Area Distribution Feeders based on Selective Sequence Learning
Authors: Ming Dong, Jian Shi, QingXin Shi
Date: 2019-07-18 00:00:00

Long-term load forecast (LTLF) for area distribution feeders is one of the most critical tasks frequently performed in electric distribution utility companies. For a specific planning area, cost-effective system upgrades can only be planned out based on accurate feeder LTLF results. In our previous research, we established a unique sequence prediction method which has the tremendous advantage of combining area top-down, feeder bottom-up and multi-year historical data all together for forecast and achieved a superior performance over various traditional methods by real-world tests. However, the previous method only focused on the forecast of the next one-year. In our current work, we significantly improved this method: the forecast can now be extended to a multi-year forecast window in the future; unsupervised learning techniques are used to group feeders by their load composition features to improve accuracy; we also propose a novel selective sequence learning mechanism which uses Gated Recurrent Unit network to not only learn how to predict sequence values but also learn to select the best-performing sequential configuration for each individual feeder. The proposed method was tested on an actual urban distribution system in West Canada. It was compared with traditional methods and our previous sequence prediction method. It demonstrates the best forecasting performance as well as the possibility of using sequence prediction models for multi-year component-level load forecast.
EEG-Based Emotion Recognition Using Regularized Graph Neural Networks
Authors: Peixiang Zhong, Di Wang, Chunyan Miao
Date: 2019-07-18 00:00:00
Tasks: EEG, EEG Emotion Recognition, Electroencephalogram (EEG), Emotion Recognition, Graph Neural Network

Electroencephalography (EEG) measures the neuronal activities in different brain regions via electrodes. Many existing studies on EEG-based emotion recognition do not fully exploit the topology of EEG channels. In this paper, we propose a regularized graph neural network (RGNN) for EEG-based emotion recognition. RGNN considers the biological topology among different brain regions to capture both local and global relations among different EEG channels. Specifically, we model the inter-channel relations in EEG signals via an adjacency matrix in a graph neural network where the connection and sparseness of the adjacency matrix are inspired by neuroscience theories of human brain organization. In addition, we propose two regularizers, namely node-wise domain adversarial training (NodeDAT) and emotion-aware distribution learning (EmotionDL), to better handle cross-subject EEG variations and noisy labels, respectively. Extensive experiments on two public datasets, SEED and SEED-IV, demonstrate the superior performance of our model than state-of-the-art models in most experimental settings. Moreover, ablation studies show that the proposed adjacency matrix and two regularizers contribute consistent and significant gain to the performance of our RGNN model. Finally, investigations on the neuronal activities reveal important brain regions and inter-channel relations for EEG-based emotion recognition.
Comparison of Classical Machine Learning Approaches on Bangla Textual Emotion Analysis
Authors: Md. Ataur Rahman, Md. Hanif Seddiqui
Date: 2019-07-18 00:00:00
Tasks: BIG-bench Machine Learning, Clustering, Emotion Recognition

Detecting emotions from text is an extension of simple sentiment polarity detection. Instead of considering only positive or negative sentiments, emotions are conveyed using more tangible manner; thus, they can be expressed as many shades of gray. This paper manifests the results of our experimentation for fine-grained emotion analysis on Bangla text. We gathered and annotated a text corpus consisting of user comments from several Facebook groups regarding socio-economic and political issues, and we made efforts to extract the basic emotions (sadness, happiness, disgust, surprise, fear, anger) conveyed through these comments. Finally, we compared the results of the five most popular classical machine learning techniques namely Naive Bayes, Decision Tree, k-Nearest Neighbor (k-NN), Support Vector Machine (SVM) and K-Means Clustering with several combinations of features. Our best model (SVM with a non-linear radial-basis function (RBF) kernel) achieved an overall average accuracy score of 52.98% and an F1 score (macro) of 0.3324
Decoding the Style and Bias of Song Lyrics
Authors: Manash Pratim Barman, Amit Awekar, Sambhav Kothari
Date: 2019-07-17 00:00:00

The central idea of this paper is to gain a deeper understanding of song lyrics computationally. We focus on two aspects: style and biases of song lyrics. All prior works to understand these two aspects are limited to manual analysis of a small corpus of song lyrics. In contrast, we analyzed more than half a million songs spread over five decades. We characterize the lyrics style in terms of vocabulary, length, repetitiveness, speed, and readability. We have observed that the style of popular songs significantly differs from other songs. We have used distributed representation methods and WEAT test to measure various gender and racial biases in the song lyrics. We have observed that biases in song lyrics correlate with prior results on human subjects. This correlation indicates that song lyrics reflect the biases that exist in society. Increasing consumption of music and the effect of lyrics on human emotions makes this analysis important.
Unsupervised Task Design to Meta-Train Medical Image Classifiers
Authors: Gabriel Maicas, Cuong Nguyen, Farbod Motlagh, Jacinto C. Nascimento, Gustavo Carneiro
Date: 2019-07-17 00:00:00
Tasks: Classification, Few-Shot Learning, General Classification

Meta-training has been empirically demonstrated to be the most effective pre-training method for few-shot learning of medical image classifiers (i.e., classifiers modeled with small training sets). However, the effectiveness of meta-training relies on the availability of a reasonable number of hand-designed classification tasks, which are costly to obtain, and consequently rarely available. In this paper, we propose a new method to unsupervisedly design a large number of classification tasks to meta-train medical image classifiers. We evaluate our method on a breast dynamically contrast enhanced magnetic resonance imaging (DCE-MRI) data set that has been used to benchmark few-shot training methods of medical image classifiers. Our results show that the proposed unsupervised task design to meta-train medical image classifiers builds a pre-trained model that, after fine-tuning, produces better classification results than other unsupervised and supervised pre-training methods, and competitive results with respect to meta-training that relies on hand-designed classification tasks.
Analysis of Word Embeddings Using Fuzzy Clustering
Authors: Shahin Atakishiyev, Marek Z. Reformat
Date: 2019-07-17 00:00:00
Tasks: Clustering, Semantic Similarity, Semantic Textual Similarity, Word Embeddings

In data dominated systems and applications, a concept of representing words in a numerical format has gained a lot of attention. There are a few approaches used to generate such a representation. An interesting issue that should be considered is the ability of such representations - called embeddings - to imitate human-based semantic similarity between words. In this study, we perform a fuzzy-based analysis of vector representations of words, i.e., word embeddings. We use two popular fuzzy clustering algorithms on count-based word embeddings, known as GloVe, of different dimensionality. Words from WordSim-353, called the gold standard, are represented as vectors and clustered. The results indicate that fuzzy clustering algorithms are very sensitive to high-dimensional data, and parameter tuning can dramatically change their performance. We show that by adjusting the value of the fuzzifier parameter, fuzzy clustering can be successfully applied to vectors of high - up to one hundred - dimensions. Additionally, we illustrate that fuzzy clustering allows to provide interesting results regarding membership of words to different clusters.
Stability selection enables robust learning of partial differential equations from limited noisy data
Authors: Suryanarayana Maddu, Bevan L. Cheeseman, Ivo F. Sbalzarini, Christian L. Müller
Date: 2019-07-17 00:00:00
Tasks: compressed sensing, Model Selection, regression

We present a statistical learning framework for robust identification of partial differential equations from noisy spatiotemporal data. Extending previous sparse regression approaches for inferring PDE models from simulated data, we address key issues that have thus far limited the application of these methods to noisy experimental data, namely their robustness against noise and the need for manual parameter tuning. We address both points by proposing a stability-based model selection scheme to determine the level of regularization required for reproducible recovery of the underlying PDE. This avoids manual parameter tuning and provides a principled way to improve the method's robustness against noise in the data. Our stability selection approach, termed PDE-STRIDE, can be combined with any sparsity-promoting penalized regression model and provides an interpretable criterion for model component importance. We show that in particular the combination of stability selection with the iterative hard-thresholding algorithm from compressed sensing provides a fast, parameter-free, and robust computational framework for PDE inference that outperforms previous algorithmic approaches with respect to recovery accuracy, amount of data required, and robustness to noise. We illustrate the performance of our approach on a wide range of noise-corrupted simulated benchmark problems, including 1D Burgers, 2D vorticity-transport, and 3D reaction-diffusion problems. We demonstrate the practical applicability of our method on real-world data by considering a purely data-driven re-evaluation of the advective triggering hypothesis for an embryonic polarization system in C.~elegans. Using fluorescence microscopy images of C.~elegans zygotes as input data, our framework is able to recover the PDE model for the regulatory reaction-diffusion-flow network of the associated proteins.