pwc-archive/papers-with-abstracts huggingface dataset:
Transfer between long-term and short-term memory using Conceptors
Authors: Anthony Strock, Nicolas Rougier, Xavier Hinaut
Date: 2020-03-11 00:00:00

We introduce a recurrent neural network model of working memory combining short-term and long-term components. e short-term component is modelled using a gated reservoir model that is trained to hold a value from an input stream when a gate signal is on. e long-term component is modelled using conceptors in order to store inner temporal patterns (that corresponds to values). We combine these two components to obtain a model where information can go from long-term memory to short-term memory and vice-versa and we show how standard operations on conceptors allow to combine long-term memories and describe their effect on short-term memory.
Improved Binary Artificial Bee Colony Algorithm
Authors: Rafet Durgut
Date: 2020-03-12 00:00:00

The Artificial Bee Colony (ABC) algorithm is an evolutionary optimization algorithm based on swarm intelligence and inspired by the honey bees' food search behavior. Since the ABC algorithm has been developed to achieve optimal solutions by searching in the continuous search space, modification is required to apply this method to binary optimization problems. In this paper, we improve the ABC algorithm to solve binary optimization problems and call it the improved binary Artificial Bee Colony (ibinABC). The proposed method consists of an update mechanism based on fitness values and processing different number of decision variables. Thus, we aim to prevent the ABC algorithm from getting stuck in a local minimum by increasing its exploration ability. We compare the ibinABC algorithm with three variants of the ABC and other meta-heuristic algorithms in the literature. For comparison, we use the wellknown OR-Library dataset containing 15 problem instances prepared for the uncapacitated facility location problem. Computational results show that the proposed method is superior to other methods in terms of convergence speed and robustness. The source code of the algorithm will be available on GitHub after reviewing process
A Novel Hybrid GWO with WOA for Global Numerical Optimization and Solving Pressure Vessel Design
Authors: Hardi M. Mohammed, Tarik A. Rashid
Date: 2020-02-28 00:00:00

A recent metaheuristic algorithm, such as Whale Optimization Algorithm (WOA), was proposed. The idea of proposing this algorithm belongs to the hunting behavior of the humpback whale. However, WOA suffers from poor performance in the exploitation phase and stagnates in the local best solution. Grey Wolf Optimization (GWO) is a very competitive algorithm comparing to other common metaheuristic algorithms as it has a super performance in the exploitation phase while it is tested on unimodal benchmark functions. Therefore, the aim of this paper is to hybridize GWO with WOA to overcome the problems. GWO can perform well in exploiting optimal solutions. In this paper, a hybridized WOA with GWO which is called WOAGWO is presented. The proposed hybridized model consists of two steps. Firstly, the hunting mechanism of GWO is embedded into the WOA exploitation phase with a new condition which is related to GWO. Secondly, a new technique is added to the exploration phase to improve the solution after each iteration. Experimentations are tested on three different standard test functions which are called benchmark functions: 23 common functions, 25 CEC2005 functions and 10 CEC2019 functions. The proposed WOAGWO is also evaluated against original WOA, GWO and three other commonly used algorithms. Results show that WOAGWO outperforms other algorithms depending on the Wilcoxon rank-sum test. Finally, WOAGWO is likewise applied to solve an engineering problem such as pressure vessel design. Then the results prove that WOAGWO achieves optimum solution which is better than WOA and Fitness Dependent Optimizer (FDO).
Implementing a GPU-based parallel MAX-MIN Ant System
Authors: Rafał Skinderowicz
Date: 2020-01-18 00:00:00
Tasks: Combinatorial Optimization, CPU, GPU, Traveling Salesman Problem

The MAX-MIN Ant System (MMAS) is one of the best-known Ant Colony Optimization (ACO) algorithms proven to be efficient at finding satisfactory solutions to many difficult combinatorial optimization problems. The slow-down in Moore's law, and the availability of graphics processing units (GPUs) capable of conducting general-purpose computations at high speed, has sparked considerable research efforts into the development of GPU-based ACO implementations. In this paper, we discuss a range of novel ideas for improving the GPU-based parallel MMAS implementation, allowing it to better utilize the computing power offered by two subsequent Nvidia GPU architectures. Specifically, based on the weighted reservoir sampling algorithm we propose a novel parallel implementation of the node selection procedure, which is at the heart of the MMAS and other ACO algorithms. We also present a memory-efficient implementation of another key-component -- the tabu list structure -- which is used in the ACO's solution construction stage. The proposed implementations, combined with the existing approaches, lead to a total of six MMAS variants, which are evaluated on a set of Traveling Salesman Problem (TSP) instances ranging from 198 to 3,795 cities. The results show that our MMAS implementation is competitive with state-of-the-art GPU-based and multi-core CPU-based parallel ACO implementations: in fact, the times obtained for the Nvidia V100 Volta GPU were up to 7.18x and 21.79x smaller, respectively. The fastest of the proposed MMAS variants is able to generate over 1 million candidate solutions per second when solving a 1,002-city instance. Moreover, we show that, combined with the 2-opt local search heuristic, the proposed parallel MMAS finds high-quality solutions for the TSP instances with up to 18,512 nodes.
Identifying and Correcting Bias from Time- and Severity- Dependent Reporting Rates in the Estimation of the COVID-19 Case Fatality Rate
Authors: Anastasios Nikolas Angelopoulos, Reese Pathak, Rohit Varma, Michael. I. Jordan
Date: 2020-03-19 00:00:00

Background: COVID-19 is an ongoing pandemic with over 400,000 confirmed cases and large variability in its reported case fatality rate (CFR). CFR is an epidemiological measure of severity which can affect policymaking and public health responses to the disease. As we are in the middle of an active outbreak, estimating this measure will necessarily involve correcting for time- and severity- dependent reporting of cases, and time-lags in observed patient outcomes. Methods: We carry out a theoretical analysis of current estimators of the CFR. We adapt a standard statistical technique, expectation maximization (EM), in a form previously developed for pandemic influenzas to correct for time- and severity- dependent reporting in the estimated CFR of COVID-19. Code is available at https://github.com/aangelopoulos/cfr-covid-19 . Findings: We find that the na\"{i}ve estimator of CFR is asymptotically biased for the true CFR. To compensate for both of these variables we apply an expectation maximization strategy previously developed for emerging pathogens such as pandemic influenza. We obtain a CFR estimate of 2.4% for COVID-19. We also show results of our method for relative CFR by nation. Finally, we release our code on GitHub so it can be used as more data becomes available globally. Interpretation: The current strategy of estimating the CFR by dividing the number of deaths by the number of cases should not be used anymore as it is unreliable. Moving forward we suggest instead the use of maximum likelihood models which correct for time-dependent bias. This may allow public health organizations and local, state, and national governments to more accurately direct resources (e.g. test kits and vaccines) to places that would be most in need by compensating for the time delay inherent in this urgent ongoing pandemic.
I-ViSE: Interactive Video Surveillance as an Edge Service using Unsupervised Feature Queries
Authors: Seyed Yahya Nikouei, Yu Chen, Alexander Aved, Erik Blasch
Date: 2020-03-09 00:00:00
Tasks: Descriptive, Face Recognition, Scene Recognition

Situation AWareness (SAW) is essential for many mission critical applications. However, SAW is very challenging when trying to immediately identify objects of interest or zoom in on suspicious activities from thousands of video frames. This work aims at developing a queryable system to instantly select interesting content. While face recognition technology is mature, in many scenarios like public safety monitoring, the features of objects of interest may be much more complicated than face features. In addition, human operators may not be always able to provide a descriptive, simple, and accurate query. Actually, it is more often that there are only rough, general descriptions of certain suspicious objects or accidents. This paper proposes an Interactive Video Surveillance as an Edge service (I-ViSE) based on unsupervised feature queries. Adopting unsupervised methods that do not reveal any private information, the I-ViSE scheme utilizes general features of a human body and color of clothes. An I-ViSE prototype is built following the edge-fog computing paradigm and the experimental results verified the I-ViSE scheme meets the design goal of scene recognition in less than two seconds.
A Comprehensive Survey of Navigation Systems for the Visual Impaired
Authors: Roya Norouzi Kandalan, Kamesh Namuduri
Date: 2019-06-13 00:00:00
Tasks: Navigate

Sight is essential for humans to navigate their surrounding environment independently. Tasks that are simple for the sighted are often close to impossible for the visually impaired. Throughout the years, many researchers dedicated their time and efforts to design and implement technologies and devices that can help the visually impaired to navigate in unknown areas independently. A navigation assistive system for the visually impaired is built on multiple components of which localization, navigation, obstacle avoidance, and human-machine interaction form the essential components. Individual components have been explored extensively in the literature and it is a daunting task to review it in its entirety. In this paper, a compact but comprehensive collection of available methods to build each component has been provided. The discussed methods may not have been implemented for a navigation assistive technology for visually impaired directly but they can be employed to build an individual building block. Given that every approach has its own challenges, methods to combat those challenges have been presented. The purpose of this paper is to provide researchers with a shortcut to review most of the available methods to build each component, provide them with the essential knowledge to understand the nuances in each component and give them a baseline to start with.
A Bird's Eye View of Nonlinear System Identification
Authors: Luis Antonio Aguirre
Date: 2019-07-16 00:00:00

This text aims at providing a bird's eye view of system identification with special attention to nonlinear systems. The driving force is to give a feeling for the philosophical problems facing those that build mathematical models from data. Special attention will be given to grey-box approaches in nonlinear system identification. In this text, grey-box methods use auxiliary information such as the system steady-state data, possible symmetries, some bifurcations and the presence of hysteresis. The text ends with a sample of applications. No attempt is made to be thorough nor to survey such an extensive and mature field as system identification. In most parts references will be provided for a more detailed study.
Hybrid Neural Networks for Learning the Trend in Time Series
Authors: Tao Lin∗，Tian Guo∗，Karl Aberer
Date: 2017-07-01 00:00:00
Proceeding: International Joint Conferences on Artificial Intelligence Organization 2017 7
Tasks: Time Series, Time Series Analysis

Trend of time series characterizes the intermediate upward and downward behaviour of time series. Learning and forecasting the trend in time series data play an important role in many real applica- tions, ranging from resource allocation in data cen- ters, load schedule in smart grid, and so on. In- spired by the recent successes of neural networks, in this paper we propose TreNet, a novel end-to- end hybrid neural network to learn local and global contextual features for predicting the trend of time series. TreNet leverages convolutional neural net- works (CNNs) to extract salient features from local raw data of time series. Meanwhile, considering the long-range dependency existing in the sequence of historical trends, TreNet uses a long-short term memory recurrent neural network (LSTM) to cap- ture such dependency. Then, a feature fusion layer is to learn joint representation for predicting the trend. TreNet demonstrates its effectiveness by out- performing CNN, LSTM, the cascade of CNN and LSTM, Hidden Markov Model based method and various kernel based baselines on real datasets.
Probabilistic Regression for Visual Tracking
Authors: Martin Danelljan, Luc van Gool, Radu Timofte
Date: 2020-03-27 00:00:00
Proceeding: CVPR 2020 6
Tasks: Object Tracking, regression, Visual Tracking

Visual tracking is fundamentally the problem of regressing the state of the target in each video frame. While significant progress has been achieved, trackers are still prone to failures and inaccuracies. It is therefore crucial to represent the uncertainty in the target estimation. Although current prominent paradigms rely on estimating a state-dependent confidence score, this value lacks a clear probabilistic interpretation, complicating its use. In this work, we therefore propose a probabilistic regression formulation and apply it to tracking. Our network predicts the conditional probability density of the target state given an input image. Crucially, our formulation is capable of modeling label noise stemming from inaccurate annotations and ambiguities in the task. The regression network is trained by minimizing the Kullback-Leibler divergence. When applied for tracking, our formulation not only allows a probabilistic representation of the output, but also substantially improves the performance. Our tracker sets a new state-of-the-art on six datasets, achieving 59.8% AUC on LaSOT and 75.8% Success on TrackingNet. The code and models are available at https://github.com/visionml/pytracking.
DA-NAS: Data Adapted Pruning for Efficient Neural Architecture Search
Authors: Xiyang Dai, Dong-Dong Chen, Mengchen Liu, Yinpeng Chen, Lu Yuan
Date: 2020-03-27 00:00:00
Proceeding: ECCV 2020 8
Tasks: GPU, Neural Architecture Search

Efficient search is a core issue in Neural Architecture Search (NAS). It is difficult for conventional NAS algorithms to directly search the architectures on large-scale tasks like ImageNet. In general, the cost of GPU hours for NAS grows with regard to training dataset size and candidate set size. One common way is searching on a smaller proxy dataset (e.g., CIFAR-10) and then transferring to the target task (e.g., ImageNet). These architectures optimized on proxy data are not guaranteed to be optimal on the target task. Another common way is learning with a smaller candidate set, which may require expert knowledge and indeed betrays the essence of NAS. In this paper, we present DA-NAS that can directly search the architecture for large-scale target tasks while allowing a large candidate set in a more efficient manner. Our method is based on an interesting observation that the learning speed for blocks in deep neural networks is related to the difficulty of recognizing distinct categories. We carefully design a progressive data adapted pruning strategy for efficient architecture search. It will quickly trim low performed blocks on a subset of target dataset (e.g., easy classes), and then gradually find the best blocks on the whole target dataset. At this time, the original candidate set becomes as compact as possible, providing a faster search in the target task. Experiments on ImageNet verify the effectiveness of our approach. It is 2x faster than previous methods while the accuracy is currently state-of-the-art, at 76.2% under small FLOPs constraint. It supports an argument search space (i.e., more candidate blocks) to efficiently search the best-performing architecture.
IMAC: In-memory multi-bit Multiplication andACcumulation in 6T SRAM Array
Authors: Mustafa Ali, Akhilesh Jaiswal, Sangamesh Kodge, Amogh Agrawal, Indranil Chakraborty, Kaushik Roy
Date: 2020-03-27 00:00:00

`In-memory computing' is being widely explored as a novel computing paradigm to mitigate the well known memory bottleneck. This emerging paradigm aims at embedding some aspects of computations inside the memory array, thereby avoiding frequent and expensive movement of data between the compute unit and the storage memory. In-memory computing with respect to Silicon memories has been widely explored on various memory bit-cells. Embedding computation inside the 6 transistor (6T) SRAM array is of special interest since it is the most widely used on-chip memory. In this paper, we present a novel in-memory multiplication followed by accumulation operation capable of performing parallel dot products within 6T SRAM without any changes to the standard bitcell. We, further, study the effect of circuit non-idealities and process variations on the accuracy of the LeNet-5 and VGG neural network architectures against the MNIST and CIFAR-10 datasets, respectively. The proposed in-memory dot-product mechanism achieves 88.8% and 99% accuracy for the CIFAR-10 and MNIST, respectively. Compared to the standard von Neumann system, the proposed system is 6.24x better in energy consumption and 9.42x better in delay.
Closed-loop Parameter Identification of Linear Dynamical Systems through the Lens of Feedback Channel Coding Theory
Authors: Ali Reza Pedram, Takashi Tanaka
Date: 2020-03-27 00:00:00

This paper considers the problem of closed-loop identification of linear scalar systems with Gaussian process noise, where the system input is determined by a deterministic state feedback policy. The regularized least-square estimate (LSE) algorithm is adopted, seeking to find the best estimate of unknown model parameters based on noiseless measurements of the state. We are interested in the fundamental limitation of the rate at which unknown parameters can be learned, in the sense of the D-optimality scalarization criterion subject to a quadratic control cost. We first establish a novel connection between a closed-loop identification problem of interest and a channel coding problem involving an additive white Gaussian noise (AWGN) channel with feedback and a certain structural constraint. Based on this connection, we show that the learning rate is fundamentally upper bounded by the capacity of the corresponding AWGN channel. Although the optimal design of the feedback policy remains challenging, we derive conditions under which the upper bound is achieved. Finally, we show that the obtained upper bound implies that super-linear convergence is unattainable for any choice of the policy.
Unpacking Information Bottlenecks: Unifying Information-Theoretic Objectives in Deep Learning
Authors: Andreas Kirsch, Clare Lyle, Yarin Gal
Date: 2020-03-27 00:00:00
Tasks: Deep Learning, Density Estimation

The Information Bottleneck principle offers both a mechanism to explain how deep neural networks train and generalize, as well as a regularized objective with which to train models. However, multiple competing objectives are proposed in the literature, and the information-theoretic quantities used in these objectives are difficult to compute for large deep neural networks, which in turn limits their use as a training objective. In this work, we review these quantities and compare and unify previously proposed objectives, which allows us to develop surrogate objectives more friendly to optimization without relying on cumbersome tools such as density estimation. We find that these surrogate objectives allow us to apply the information bottleneck to modern neural network architectures. We demonstrate our insights on MNIST, CIFAR-10 and Imagenette with modern DNN architectures (ResNets).
Identification of Choquet capacity in multicriteria sorting problems through stochastic inverse analysis
Authors: Renata Pelissari, Leonardo Tomazeli Duarte
Date: 2020-03-27 00:00:00
Tasks: Descriptive

In multicriteria decision aiding (MCDA), the Choquet integral has been used as an aggregation operator to deal with the case of interacting decision criteria. While the application of the Choquet integral for ranking problems have been receiving most of the attention, this paper rather focuses on multicriteria sorting problems (MCSP). In the Choquet integral context, a practical problem that arises is related to the elicitation of parameters known as the Choquet capacities. We address the problem of Choquet capacity identification for MCSP by applying the Stochastic Acceptability Multicriteri Analysis (SMAA), proposing the SMAA-S-Choquet method. The proposed method is also able to model uncertain data that may be present in both decision matrix and limiting profiles, the latter a parameter associated with the sorting problematic. We also introduce two new descriptive measures in order to conduct reverse analysis regarding the capacities: the Scenario Acceptability Index and the Scenario Central Capacity vector.
Generation of Consistent Sets of Multi-Label Classification Rules with a Multi-Objective Evolutionary Algorithm
Authors: Thiago Zafalon Miranda, Diorge Brognara Sardinha, Márcio Porto Basgalupp, Yaochu Jin, Ricardo Cerri
Date: 2020-03-27 00:00:00
Tasks: Classification, General Classification, Multi-Label Classification, MUlTI-LABEL-ClASSIFICATION

Multi-label classification consists in classifying an instance into two or more classes simultaneously. It is a very challenging task present in many real-world applications, such as classification of biology, image, video, audio, and text. Recently, the interest in interpretable classification models has grown, partially as a consequence of regulations such as the General Data Protection Regulation. In this context, we propose a multi-objective evolutionary algorithm that generates multiple rule-based multi-label classification models, allowing users to choose among models that offer different compromises between predictive power and interpretability. An important contribution of this work is that different from most algorithms, which usually generate models based on lists (ordered collections) of rules, our algorithm generates models based on sets (unordered collections) of rules, increasing interpretability. Also, by employing a conflict avoidance algorithm during the rule-creation, every rule within a given model is guaranteed to be consistent with every other rule in the same model. Thus, no conflict resolution strategy is required, evolving simpler models. We conducted experiments on synthetic and real-world datasets and compared our results with state-of-the-art algorithms in terms of predictive performance (F-Score) and interpretability (model size), and demonstrate that our best models had comparable F-Score and smaller model sizes.