pwc-archive/papers-with-abstracts huggingface dataset:
A Comparison of LSTM and BERT for Small Corpus
Authors: Aysu Ezen-Can
Date: 2020-09-11 00:00:00
Tasks: intent-classification, Intent Classification, text-classification, Text Classification, Text Generation, Transfer Learning

Recent advancements in the NLP field showed that transfer learning helps with achieving state-of-the-art results for new tasks by tuning pre-trained models instead of starting from scratch. Transformers have made a significant improvement in creating new state-of-the-art results for many NLP tasks including but not limited to text classification, text generation, and sequence labeling. Most of these success stories were based on large datasets. In this paper we focus on a real-life scenario that scientists in academia and industry face frequently: given a small dataset, can we use a large pre-trained model like BERT and get better results than simple models? To answer this question, we use a small dataset for intent classification collected for building chatbots and compare the performance of a simple bidirectional LSTM model with a pre-trained BERT model. Our experimental results show that bidirectional LSTM models can achieve significantly higher results than a BERT model for a small dataset and these simple models get trained in much less time than tuning the pre-trained counterparts. We conclude that the performance of a model is dependent on the task and the data, and therefore before making a model choice, these factors should be taken into consideration instead of directly choosing the most popular model.
Heterogeneous Domain Generalization via Domain Mixup
Authors: Yufei Wang, Haoliang Li, Alex C. Kot
Date: 2020-09-11 00:00:00
Tasks: Domain Generalization

One of the main drawbacks of deep Convolutional Neural Networks (DCNN) is that they lack generalization capability. In this work, we focus on the problem of heterogeneous domain generalization which aims to improve the generalization capability across different tasks, which is, how to learn a DCNN model with multiple domain data such that the trained feature extractor can be generalized to supporting recognition of novel categories in a novel target domain. To solve this problem, we propose a novel heterogeneous domain generalization method by mixing up samples across multiple source domains with two different sampling strategies. Our experimental results based on the Visual Decathlon benchmark demonstrates the effectiveness of our proposed method. The code is released in \url{https://github.com/wyf0912/MIXALL}
Embodied Visual Navigation with Automatic Curriculum Learning in Real Environments
Authors: Steven D. Morad, Roberto Mecca, Rudra P. K. Poudel, Stephan Liwicki, Roberto Cipolla
Date: 2020-09-11 00:00:00
Tasks: Deep Reinforcement Learning, Navigate, reinforcement-learning, Reinforcement Learning (RL), Visual Navigation

We present NavACL, a method of automatic curriculum learning tailored to the navigation task. NavACL is simple to train and efficiently selects relevant tasks using geometric features. In our experiments, deep reinforcement learning agents trained using NavACL significantly outperform state-of-the-art agents trained with uniform sampling -- the current standard. Furthermore, our agents can navigate through unknown cluttered indoor environments to semantically-specified targets using only RGB images. Obstacle-avoiding policies and frozen feature networks support transfer to unseen real-world environments, without any modification or retraining requirements. We evaluate our policies in simulation, and in the real world on a ground robot and a quadrotor drone. Videos of real-world results are available in the supplementary material.
Evaluation of the Robustness of Visual SLAM Methods in Different Environments
Authors: Joonas Lomps, Artjom Lind, Amnir Hadachi
Date: 2020-09-11 00:00:00
Tasks: Position, Simultaneous Localization and Mapping

Determining the position and orientation of a sensor vis-a-vis its surrounding, while simultaneously mapping the environment around that sensor or simultaneous localization and mapping is quickly becoming an important advancement in embedded vision with a large number of different possible applications. This paper presents a comprehensive comparison of the latest open-source SLAM algorithms with the main focus being their performance in different environmental surroundings. The chosen algorithms are evaluated on common publicly available datasets and the results reasoned with respect to the datasets' environment. This is the first stage of our main target of testing the methods in off-road scenarios.
Semantic Relations and Deep Learning
Authors: Vivi Nastase, Stan Szpakowicz
Date: 2020-09-11 00:00:00
Tasks: Deep Learning, Relation Classification

The second edition of "Semantic Relations Between Nominals" by Vivi Nastase, Stan Szpakowicz, Preslav Nakov and Diarmuid \'O S\'eaghdha has been published in April 2021 by Morgan & Claypool (www.morganclaypoolpublishers.com/catalog_Orig/product_info.php?products_id=1627). A new Chapter 5 of the book, by Vivi Nastase and Stan Szpakowicz, discusses relation classification/extraction in the deep-learning paradigm which arose after the first edition appeared. This is Chapter 5, made public by the kind permission of Morgan & Claypool.
Achieving Adversarial Robustness via Sparsity
Authors: Shufan Wang, Ningyi Liao, Liyao Xiang, Nanyang Ye, Quanshi Zhang
Date: 2020-09-11 00:00:00
Tasks: Adversarial Robustness, Network Pruning

Network pruning has been known to produce compact models without much accuracy degradation. However, how the pruning process affects a network's robustness and the working mechanism behind remain unresolved. In this work, we theoretically prove that the sparsity of network weights is closely associated with model robustness. Through experiments on a variety of adversarial pruning methods, we find that weights sparsity will not hurt but improve robustness, where both weights inheritance from the lottery ticket and adversarial training improve model robustness in network pruning. Based on these findings, we propose a novel adversarial training method called inverse weights inheritance, which imposes sparse weights distribution on a large network by inheriting weights from a small network, thereby improving the robustness of the large network.
Bayesian Screening: Multi-test Bayesian Optimization Applied to in silico Material Screening
Authors: James Hook, Calum Hand, Emma Whitfield
Date: 2020-09-11 00:00:00
Tasks: Bayesian Optimization

We present new multi-test Bayesian optimization models and algorithms for use in large scale material screening applications. Our screening problems are designed around two tests, one expensive and one cheap. This paper differs from other recent work on multi-test Bayesian optimization through use of a flexible model that allows for complex, non-linear relationships between the cheap and expensive test scores. This additional modeling flexibility is essential in the material screening applications which we describe. We demonstrate the power of our new algorithms on a family of synthetic toy problems as well as on real data from two large scale screening studies.
Machine Learning and Data Science approach towards trend and predictors analysis of CDC Mortality Data for the USA
Authors: Yasir Nadeem, Awais Ahmed
Date: 2020-09-11 00:00:00
Tasks: Anomaly Detection, BIG-bench Machine Learning

The research on mortality is an active area of research for any country where the conclusions are driven from the provided data and conditions. The domain knowledge is an essential but not a mandatory skill (though some knowledge is still required) in order to derive conclusions based on data intuition using machine learning and data science practices. The purpose of conducting this project was to derive conclusions based on the statistics from the provided dataset and predict label(s) of the dataset using supervised or unsupervised learning algorithms. The study concluded (based on a sample) life expectancy regardless of gender, and their central tendencies; Marital status of the people also affected how frequent deaths were for each of them. The study also helped in finding out that due to more categorical and numerical data, anomaly detection or under-sampling could be a viable solution since there are possibilities of more class labels than the other(s). The study shows that machine learning predictions aren't as viable for the data as it might be apparent.
IndoNLU: Benchmark and Resources for Evaluating Indonesian Natural Language Understanding
Authors: Bryan Wilie, Karissa Vincentio, Genta Indra Winata, Samuel Cahyawijaya, Xiaohong Li, Zhi Yuan Lim, Sidik Soleman, Rahmad Mahendra, Pascale Fung, Syafri Bahar, Ayu Purwarianti
Date: 2020-09-11 00:00:00
Proceeding: Asian Chapter of the Association for Computational Linguistics 2020
Tasks: Benchmarking, Diversity, Natural Language Understanding, Sentence, Sentence Classification

Although Indonesian is known to be the fourth most frequently used language over the internet, the research progress on this language in the natural language processing (NLP) is slow-moving due to a lack of available resources. In response, we introduce the first-ever vast resource for the training, evaluating, and benchmarking on Indonesian natural language understanding (IndoNLU) tasks. IndoNLU includes twelve tasks, ranging from single sentence classification to pair-sentences sequence labeling with different levels of complexity. The datasets for the tasks lie in different domains and styles to ensure task diversity. We also provide a set of Indonesian pre-trained models (IndoBERT) trained from a large and clean Indonesian dataset Indo4B collected from publicly available sources such as social media texts, blogs, news, and websites. We release baseline models for all twelve tasks, as well as the framework for benchmark evaluation, and thus it enables everyone to benchmark their system performances.
Activation Relaxation: A Local Dynamical Approximation to Backpropagation in the Brain
Authors: Beren Millidge, Alexander Tschantz, Anil. K. Seth, Christopher L. Buckley
Date: 2020-09-11 00:00:00

The backpropagation of error algorithm (backprop) has been instrumental in the recent success of deep learning. However, a key question remains as to whether backprop can be formulated in a manner suitable for implementation in neural circuitry. The primary challenge is to ensure that any candidate formulation uses only local information, rather than relying on global signals as in standard backprop. Recently several algorithms for approximating backprop using only local signals have been proposed. However, these algorithms typically impose other requirements which challenge biological plausibility: for example, requiring complex and precise connectivity schemes, or multiple sequential backwards phases with information being stored across phases. Here, we propose a novel algorithm, Activation Relaxation (AR), which is motivated by constructing the backpropagation gradient as the equilibrium point of a dynamical system. Our algorithm converges rapidly and robustly to the correct backpropagation gradients, requires only a single type of computational unit, utilises only a single parallel backwards relaxation phase, and can operate on arbitrary computation graphs. We illustrate these properties by training deep neural networks on visual classification tasks, and describe simplifications to the algorithm which remove further obstacles to neurobiological implementation (for example, the weight-transport problem, and the use of nonlinear derivatives), while preserving performance.
Meta Learning for Few-Shot One-class Classification
Authors: Gabriel Dahia, Maurício Pamplona Segundo
Date: 2020-09-11 00:00:00
Tasks: Classification, General Classification, Meta-Learning, One-Class Classification

We propose a method that can perform one-class classification given only a small number of examples from the target class and none from the others. We formulate the learning of meaningful features for one-class classification as a meta-learning problem in which the meta-training stage repeatedly simulates one-class classification, using the classification loss of the chosen algorithm to learn a feature representation. To learn these representations, we require only multiclass data from similar tasks. We show how the Support Vector Data Description method can be used with our method, and also propose a simpler variant based on Prototypical Networks that obtains comparable performance, indicating that learning feature representations directly from data may be more important than which one-class algorithm we choose. We validate our approach by adapting few-shot classification datasets to the few-shot one-class classification scenario, obtaining similar results to the state-of-the-art of traditional one-class classification, and that improves upon that of one-class classification baselines employed in the few-shot setting. Our code is available at https://github.com/gdahia/meta_occ
Power Evolution Prediction and Optimization in a Multi-span System Based on Component-wise System Modeling
Authors: Metodi P. Yankov, Uiara Celine de Moura, Francesco Da Ros
Date: 2020-09-11 00:00:00
Tasks: BIG-bench Machine Learning

Cascades of a machine learning-based EDFA gain model trained on a single physical device and a fully differentiable stimulated Raman scattering fiber model are used to predict and optimize the power profile at the output of an experimental multi-span fully-loaded C-band optical communication system.
Disentangling Neural Architectures and Weights: A Case Study in Supervised Classification
Authors: Nicolo Colombo, Yang Gao
Date: 2020-09-11 00:00:00
Tasks: General Classification

The history of deep learning has shown that human-designed problem-specific networks can greatly improve the classification performance of general neural models. In most practical cases, however, choosing the optimal architecture for a given task remains a challenging problem. Recent architecture-search methods are able to automatically build neural models with strong performance but fail to fully appreciate the interaction between neural architecture and weights. This work investigates the problem of disentangling the role of the neural structure and its edge weights, by showing that well-trained architectures may not need any link-specific fine-tuning of the weights. We compare the performance of such weight-free networks (in our case these are binary networks with {0, 1}-valued weights) with random, weight-agnostic, pruned and standard fully connected networks. To find the optimal weight-agnostic network, we use a novel and computationally efficient method that translates the hard architecture-search problem into a feasible optimization problem.More specifically, we look at the optimal task-specific architectures as the optimal configuration of binary networks with {0, 1}-valued weights, which can be found through an approximate gradient descent strategy. Theoretical convergence guarantees of the proposed algorithm are obtained by bounding the error in the gradient approximation and its practical performance is evaluated on two real-world data sets. For measuring the structural similarities between different architectures, we use a novel spectral approach that allows us to underline the intrinsic differences between real-valued networks and weight-free architectures.
The PREVENTION Challenge: How Good Are Humans Predicting Lane Changes?
Authors: A. Quintanar, R. Izquierdo, I. Parra, D. Fernández-Llorca, M. A. Sotelo
Date: 2020-09-11 00:00:00
Tasks: Trajectory Prediction

While driving on highways, every driver tries to be aware of the behavior of surrounding vehicles, including possible emergency braking, evasive maneuvers trying to avoid obstacles, unexpected lane changes, or other emergencies that could lead to an accident. In this paper, human's ability to predict lane changes in highway scenarios is analyzed through the use of video sequences extracted from the PREVENTION dataset, a database focused on the development of research on vehicle intention and trajectory prediction. Thus, users had to indicate the moment at which they considered that a lane change maneuver was taking place in a target vehicle, subsequently indicating its direction: left or right. The results retrieved have been carefully analyzed and compared to ground truth labels, evaluating statistical models to understand whether humans can actually predict. The study has revealed that most participants are unable to anticipate lane-change maneuvers, detecting them after they have started. These results might serve as a baseline for AI's prediction ability evaluation, grading if those systems can outperform human skills by analyzing hidden cues that seem unnoticed, improving the detection time, and even anticipating maneuvers in some cases.
Machine learning-based EDFA Gain Model Generalizable to Multiple Physical Devices
Authors: Francesco Da Ros, Uiara Celine de Moura, Metodi P. Yankov
Date: 2020-09-11 00:00:00
Tasks: 2D Object Detection, BIG-bench Machine Learning

We report a neural-network based erbium-doped fiber amplifier (EDFA) gain model built from experimental measurements. The model shows low gain-prediction error for both the same device used for training (MSE $\leq$ 0.04 dB$^2$) and different physical units of the same make (generalization MSE $\leq$ 0.06 dB$^2$).
Accurate and Intuitive Contextual Explanations using Linear Model Trees
Authors: Aditya Lahiri, Narayanan Unny Edakunni
Date: 2020-09-11 00:00:00
Tasks: Generative Adversarial Network, Marketing, Synthetic Data Generation

With the ever-increasing use of complex machine learning models in critical applications within the finance domain, explaining the decisions of the model has become a necessity. With applications spanning from credit scoring to credit marketing, the impact of these models is undeniable. Among the multiple ways in which one can explain the decisions of these complicated models, local post hoc model agnostic explanations have gained massive adoption. These methods allow one to explain each prediction independent of the modelling technique that was used while training. As explanations, they either give individual feature attributions or provide sufficient rules that represent conditions for a prediction to be made. The current state of the art methods use rudimentary methods to generate synthetic data around the point to be explained. This is followed by fitting simple linear models as surrogates to obtain a local interpretation of the prediction. In this paper, we seek to significantly improve on both, the method used to generate the explanations and the nature of explanations produced. We use a Generative Adversarial Network for synthetic data generation and train a piecewise linear model in the form of Linear Model Trees to be used as the surrogate model.In addition to individual feature attributions, we also provide an accompanying context to our explanations by leveraging the structure and property of our surrogate model.
SoFAr: Shortcut-based Fractal Architectures for Binary Convolutional Neural Networks
Authors: Zhu Baozhou, Peter Hofstee, Jinho Lee, Zaid Al-Ars
Date: 2020-09-11 00:00:00
Tasks: Binarization

Binary Convolutional Neural Networks (BCNNs) can significantly improve the efficiency of Deep Convolutional Neural Networks (DCNNs) for their deployment on resource-constrained platforms, such as mobile and embedded systems. However, the accuracy degradation of BCNNs is still considerable compared with their full precision counterpart, impeding their practical deployment. Because of the inevitable binarization error in the forward propagation and gradient mismatch problem in the backward propagation, it is nontrivial to train BCNNs to achieve satisfactory accuracy. To ease the difficulty of training, the shortcut-based BCNNs, such as residual connection-based Bi-real ResNet and dense connection-based BinaryDenseNet, introduce additional shortcuts in addition to the shortcuts already present in their full precision counterparts. Furthermore, fractal architectures have been also been used to improve the training process of full-precision DCNNs since the fractal structure triggers effects akin to deep supervision and lateral student-teacher information flow. Inspired by the shortcuts and fractal architectures, we propose two Shortcut-based Fractal Architectures (SoFAr) specifically designed for BCNNs: 1. residual connection-based fractal architectures for binary ResNet, and 2. dense connection-based fractal architectures for binary DenseNet. Our proposed SoFAr combines the adoption of shortcuts and the fractal architectures in one unified model, which is helpful in the training of BCNNs. Results show that our proposed SoFAr achieves better accuracy compared with shortcut-based BCNNs. Specifically, the Top-1 accuracy of our proposed RF-c4d8 ResNet37(41) and DRF-c2d2 DenseNet51(53) on ImageNet outperforms Bi-real ResNet18(64) and BinaryDenseNet51(32) by 3.29% and 1.41%, respectively, with the same computational complexity overhead.
A Density-Aware PointRCNN for 3D Object Detection in Point Clouds
Authors: Jie Li, Yu Hu
Date: 2020-09-11 00:00:00
Tasks: 3D Object Detection, object-detection, Object Detection

We present an improved version of PointRCNN for 3D object detection, in which a multi-branch backbone network is adopted to handle the non-uniform density of point clouds. An uncertainty-based sampling policy is proposed to deal with the distribution differences of different point clouds. The new model can achieve about 0.8 AP higher performance than the baseline PointRCNN on KITTI val set. In addition, a simplified model using a single scale grouping for each set-abstraction layer can achieve competitive performance with less computational cost.
CatGCN: Graph Convolutional Networks with Categorical Node Features
Authors: Weijian Chen, Fuli Feng, Qifan Wang, Xiangnan He, Chonggang Song, Guohui Ling, Yongdong Zhang
Date: 2020-09-11 00:00:00
Tasks: Graph Learning, Node Classification, Recommendation Systems

Recent studies on Graph Convolutional Networks (GCNs) reveal that the initial node representations (i.e., the node representations before the first-time graph convolution) largely affect the final model performance. However, when learning the initial representation for a node, most existing work linearly combines the embeddings of node features, without considering the interactions among the features (or feature embeddings). We argue that when the node features are categorical, e.g., in many real-world applications like user profiling and recommender system, feature interactions usually carry important signals for predictive analytics. Ignoring them will result in suboptimal initial node representation and thus weaken the effectiveness of the follow-up graph convolution. In this paper, we propose a new GCN model named CatGCN, which is tailored for graph learning when the node features are categorical. Specifically, we integrate two ways of explicit interaction modeling into the learning of initial node representation, i.e., local interaction modeling on each pair of node features and global interaction modeling on an artificial feature graph. We then refine the enhanced initial node representations with the neighborhood aggregation-based graph convolution. We train CatGCN in an end-to-end fashion and demonstrate it on semi-supervised node classification. Extensive experiments on three tasks of user profiling (the prediction of user age, city, and purchase level) from Tencent and Alibaba datasets validate the effectiveness of CatGCN, especially the positive effect of performing feature interaction modeling before graph convolution.