pwc-archive/papers-with-abstracts huggingface dataset:
A Multi-View Discriminant Learning Approach for Indoor Localization Using Bimodal Features of CSI
Authors: Tahsina Farah Sanam, Hana Godrich
Date: 2019-08-13 00:00:00
Tasks: Diversity, Indoor Localization, MULTI-VIEW LEARNING

With the growth of location-based services, indoor localization is attracting great interests as it facilitates further ubiquitous environments. Specifically, device free localization using wireless signals is getting increased attention as human location is estimated using its impact on the surrounding wireless signals without any active device tagged with subject. In this paper, we propose MuDLoc, the first multi-view discriminant learning approach for device free indoor localization using both amplitude and phase features of Channel State Information (CSI) from multiple APs. Multi-view learning is an emerging technique in machine learning which improve performance by utilizing diversity from different view data. In MuDLoc, the localization is modeled as a pattern matching problem, where the target location is predicted based on similarity measure of CSI features of an unknown location with those of the training locations. MuDLoc implements Generalized Inter-view and Intra-view Discriminant Correlation Analysis (GI$^{2}$DCA), a discriminative feature extraction approach using multi-view CSIs. It incorporates inter-view and intra-view class associations while maximizing pairwise correlations across multi-view data sets. A similarity measure is performed to find the best match to localize a subject. Experimental results from two cluttered environments show that MuDLoc can estimate location with high accuracy which outperforms other benchmark approaches.
New Approach for Solving The Clustered Shortest-Path Tree Problem Based on Reducing The Search Space of Evolutionary Algorithm
Authors: Huynh Thi Thanh Binh, Pham Dinh Thanh, Ta Bao Thang
Date: 2019-06-10 00:00:00
Tasks: Evolutionary Algorithms

Along with the development of manufacture and services, the problem of distribution network optimization has been growing in importance, thus receiving much attention from the research community. One of the most recently introduced network optimization problems is the Clustered Shortest-Path Tree Problem (CluSTP). Since the problem is NP-Hard, recent approaches often prefer to use approximation algorithms to solve it, several of which used Evolutionary Algorithms (EAs) and have been proven to be effective. However, most of the prior studies directly applied EAs to the whole CluSTP problem, which leads to a great amount of resource consumption, especially when the problem size is large. To overcome these limitations, this paper suggests a method for reducing the search space of the EAs applied to CluSTP by decomposing the original problem into two sub-problems, the solution to one of which is found by an EAs and that to the other is found by another method. The goal of the first sub-problem is to determine a spanning tree which connects among the clusters, while the goal of the second sub-problem is to determine the best spanning tree for each cluster. In addition, this paper proposes a new EAs, which can be applied to solve the first sub-problem and suggests using the Dijkstra's algorithm to solve the second sub-problem. The proposed approach is comprehensively experimented and compared with existing methods. Experimental results prove that our method is more efficient and more importantly, it can obtain results which are close to the optimal results.
Recurrent Neural Networks: An Embedded Computing Perspective
Authors: Nesma M. Rezk, Madhura Purnaprajna, Tomas Nordström, Zain Ul-Abdin
Date: 2019-07-23 00:00:00
Tasks: Time Series, Time Series Analysis

Recurrent Neural Networks (RNNs) are a class of machine learning algorithms used for applications with time-series and sequential data. Recently, there has been a strong interest in executing RNNs on embedded devices. However, difficulties have arisen because RNN requires high computational capability and a large memory space. In this paper, we review existing implementations of RNN models on embedded platforms and discuss the methods adopted to overcome the limitations of embedded systems. We will define the objectives of mapping RNN algorithms on embedded platforms and the challenges facing their realization. Then, we explain the components of RNN models from an implementation perspective. We also discuss the optimizations applied to RNNs to run efficiently on embedded platforms. Finally, we compare the defined objectives with the implementations and highlight some open research questions and aspects currently not addressed for embedded RNNs. Overall, applying algorithmic optimizations to RNN models and decreasing the memory access overhead is vital to obtain high efficiency. To further increase the implementation efficiency, we point up the more promising optimizations that could be applied in future research. Additionally, this article observes that high performance has been targeted by many implementations, while flexibility has, as yet, been attempted less often. Thus, the article provides some guidelines for RNN hardware designers to support flexibility in a better manner.
Analysis of Memory Capacity for Deep Echo State Networks
Authors: Xuanlin Liu, Mingzhe Chen, Changchuan Yin, Walid Saad
Date: 2019-06-11 00:00:00

In this paper, the echo state network (ESN) memory capacity, which represents the amount of input data an ESN can store, is analyzed for a new type of deep ESNs. In particular, two deep ESN architectures are studied. First, a parallel deep ESN is proposed in which multiple reservoirs are connected in parallel allowing them to average outputs of multiple ESNs, thus decreasing the prediction error. Then, a series architecture ESN is proposed in which ESN reservoirs are placed in cascade that the output of each ESN is the input of the next ESN in the series. This series ESN architecture can capture more features between the input sequence and the output sequence thus improving the overall prediction accuracy. Fundamental analysis shows that the memory capacity of parallel ESNs is equivalent to that of a traditional shallow ESN, while the memory capacity of series ESNs is smaller than that of a traditional shallow ESN.In terms of normalized root mean square error, simulation results show that the parallel deep ESN achieves 38.5% reduction compared to the traditional shallow ESN while the series deep ESN achieves 16.8% reduction.
Data Analysis of Wireless Networks Using Classification Techniques
Authors: Daniel Rosa Canêdo, Alexandre Ricardo Soares Romariz
Date: 2019-07-22 00:00:00
Tasks: Classification, General Classification, Intrusion Detection

In the last decade, there has been a great technological advance in the infrastructure of mobile technologies. The increase in the use of wireless local area networks and the use of satellite services are also noticed. The high utilization rate of mobile devices for various purposes makes clear the need to track wireless networks to ensure the integrity and confidentiality of the information transmitted. Therefore, it is necessary to quickly and efficiently identify the normal and abnormal traffic of such networks, so that administrators can take action. This work aims to analyze classification techniques in relation to data from Wireless Networks, using some classes of anomalies pre-established according to some defined criteria of the MAC layer. For data analysis, WEKA Data Mining software (Waikato Environment for Knowledge Analysis) is used. The classification algorithms present a success rate in the classification of viable data, being indicated in the use of intrusion detection systems for wireless networks.
Data-Driven Predictive Modeling of Neuronal Dynamics using Long Short-Term Memory
Authors: Benjamin Plaster, Gautam Kumar
Date: 2019-08-11 00:00:00

Modeling brain dynamics to better understand and control complex behaviors underlying various cognitive brain functions are of interests to engineers, mathematicians, and physicists from the last several decades. With a motivation of developing computationally efficient models of brain dynamics to use in designing control-theoretic neurostimulation strategies, we have developed a novel data-driven approach in a long short-term memory (LSTM) neural network architecture to predict the temporal dynamics of complex systems over an extended long time-horizon in future. In contrast to recent LSTM-based dynamical modeling approaches that make use of multi-layer perceptrons or linear combination layers as output layers, our architecture uses a single fully connected output layer and reversed-order sequence-to-sequence mapping to improve short time-horizon prediction accuracy and to make multi-timestep predictions of dynamical behaviors. We demonstrate the efficacy of our approach in reconstructing the regular spiking to bursting dynamics exhibited by an experimentally-validated 9-dimensional Hodgkin-Huxley model of hippocampal CA1 pyramidal neurons. Through simulations, we show that our LSTM neural network can predict the multi-time scale temporal dynamics underlying various spiking patterns with reasonable accuracy. Moreover, our results show that the predictions improve with increasing predictive time-horizon in the multi-timestep deep LSTM neural network.
Delineating Knowledge Domains in the Scientific Literature Using Visual Information
Authors: Sean Yang, Po-shen Lee, Jevin D. West, Bill Howe
Date: 2019-08-12 00:00:00
Tasks: Recommendation Systems

Figures are an important channel for scientific communication, used to express complex ideas, models and data in ways that words cannot. However, this visual information is mostly ignored in analyses of the scientific literature. In this paper, we demonstrate the utility of using scientific figures as markers of knowledge domains in science, which can be used for classification, recommender systems, and studies of scientific information exchange. We encode sets of images into a visual signature, then use distances between these signatures to understand how patterns of visual communication compare with patterns of jargon and citation structures. We find that figures can be as effective for differentiating communities of practice as text or citation patterns. We then consider where these metrics disagree to understand how different disciplines use visualization to express ideas. Finally, we further consider how specific figure types propagate through the literature, suggesting a new mechanism for understanding the flow of ideas apart from conventional channels of text and citations. Our ultimate aim is to better leverage these information-dense objects to improve scientific communication across disciplinary boundaries.
SAIFE: Unsupervised Wireless Spectrum Anomaly Detection with Interpretable Features
Authors: Sreeraj Rajendran, Wannes Meert, Vincent Lenders, Sofie Pollin
Date: 2018-07-22 00:00:00
Tasks: Anomaly Detection, Data Compression

Detecting anomalous behavior in wireless spectrum is a demanding task due to the sheer complexity of the electromagnetic spectrum use. Wireless spectrum anomalies can take a wide range of forms from the presence of an unwanted signal in a licensed band to the absence of an expected signal, which makes manual labeling of anomalies difficult and suboptimal. We present, Spectrum Anomaly Detector with Interpretable FEatures (SAIFE), an Adversarial Autoencoder (AAE) based anomaly detector for wireless spectrum anomaly detection using Power Spectral Density (PSD) data which achieves good anomaly detection and localization in an unsupervised setting. In addition, we investigate the model's capabilities to learn interpretable features such as signal bandwidth, class and center frequency in a semi-supervised fashion. Along with anomaly detection the model exhibits promising results for lossy PSD data compression up to 120X and semisupervised signal classification accuracy close to 100% on three datasets just using 20% labeled samples. Finally the model is tested on data from one of the distributed Electrosense sensors over a long term of 500 hours showing its anomaly detection capabilities.
Testing for differential abundance in compositional counts data, with application to microbiome studies
Authors: Barak Brill, Amnon Amir, Ruth Heller
Date: 2019-04-18 00:00:00
Tasks: valid

Identifying which taxa in our microbiota are associated with traits of interest is important for advancing science and health. However, the identification is challenging because the measured vector of taxa counts (by amplicon sequencing) is compositional, so a change in the abundance of one taxon in the microbiota induces a change in the number of sequenced counts across all taxa. The data is typically sparse, with zero counts present either due to biological variance or limited sequencing depth (technical zeros). For low abundance taxa, the chance for technical zeros is non-negligible. We show that existing methods designed to identify differential abundance for compositional data may have an inflated number of false positives due to improper handling of the zero counts. We introduce a novel non-parametric approach which provides valid inference even when the fraction of zero counts is substantial. Our approach uses a set of reference taxa that are non-differentially abundant, which can be estimated from the data or from outside information. We show the usefulness of our approach via simulations, as well as on three different data sets: a Crohn's disease study, the Human Microbiome Project, and an experiment with 'spiked-in' bacteria.
On the Resilience of RTL NN Accelerators: Fault Characterization and Mitigation
Authors: Behzad Salami, Osman Unsal, Adrian Cristal
Date: 2018-06-14 00:00:00
Tasks: High-Level Synthesis

Machine Learning (ML) is making a strong resurgence in tune with the massive generation of unstructured data which in turn requires massive computational resources. Due to the inherently compute- and power-intensive structure of Neural Networks (NNs), hardware accelerators emerge as a promising solution. However, with technology node scaling below 10nm, hardware accelerators become more susceptible to faults, which in turn can impact the NN accuracy. In this paper, we study the resilience aspects of Register-Transfer Level (RTL) model of NN accelerators, in particular, fault characterization and mitigation. By following a High-Level Synthesis (HLS) approach, first, we characterize the vulnerability of various components of RTL NN. We observed that the severity of faults depends on both i) application-level specifications, i.e., NN data (inputs, weights, or intermediate), NN layers, and NN activation functions, and ii) architectural-level specifications, i.e., data representation model and the parallelism degree of the underlying accelerator. Second, motivated by characterization results, we present a low-overhead fault mitigation technique that can efficiently correct bit flips, by 47.3% better than state-of-the-art methods.
A Closer Look at Deep Learning Heuristics: Learning rate restarts, Warmup and Distillation
Authors: Akhilesh Gotmare, Nitish Shirish Keskar, Caiming Xiong, Richard Socher
Date: 2018-10-29 00:00:00
Proceeding: ICLR 2019 5
Tasks: Dimensionality Reduction, Knowledge Distillation

The convergence rate and final performance of common deep learning models have significantly benefited from heuristics such as learning rate schedules, knowledge distillation, skip connections, and normalization layers. In the absence of theoretical underpinnings, controlled experiments aimed at explaining these strategies can aid our understanding of deep learning landscapes and the training dynamics. Existing approaches for empirical analysis rely on tools of linear interpolation and visualizations with dimensionality reduction, each with their limitations. Instead, we revisit such analysis of heuristics through the lens of recently proposed methods for loss surface and representation analysis, viz., mode connectivity and canonical correlation analysis (CCA), and hypothesize reasons for the success of the heuristics. In particular, we explore knowledge distillation and learning rate heuristics of (cosine) restarts and warmup using mode connectivity and CCA. Our empirical analysis suggests that: (a) the reasons often quoted for the success of cosine annealing are not evidenced in practice; (b) that the effect of learning rate warmup is to prevent the deeper layers from creating training instability; and (c) that the latent knowledge shared by the teacher is primarily disbursed to the deeper layers.
Demystifying Neural Network Filter Pruning
Authors: Zhuwei Qin, Fuxun Yu, ChenChen Liu, Xiang Chen
Date: 2018-10-29 00:00:00
Proceeding: NIPS Workshop CDNNRIA 2018

Based on filter magnitude ranking (e.g. L1 norm), conventional filter pruning methods for Convolutional Neural Networks (CNNs) have been proved with great effectiveness in computation load reduction. Although effective, these methods are rarely analyzed in a perspective of filter functionality. In this work, we explore the filter pruning and the retraining through qualitative filter functionality interpretation. We find that the filter magnitude based method fails to eliminate the filters with repetitive functionality. And the retraining phase is actually used to reconstruct the remained filters for functionality compensation for the wrongly-pruned critical filters. With a proposed functionality-oriented pruning method, we further testify that, by precisely addressing the filter functionality redundancy, a CNN can be pruned without considerable accuracy drop, and the retraining phase is unnecessary.
Channel-wise pruning of neural networks with tapering resource constraint
Authors: Alexey Kruglov
Date: 2018-12-04 00:00:00
Tasks: Network Pruning, Transfer Learning

Neural network pruning is an important step in design process of efficient neural networks for edge devices with limited computational power. Pruning is a form of knowledge transfer from the weights of the original network to a smaller target subnetwork. We propose a new method for compute-constrained structured channel-wise pruning of convolutional neural networks. The method iteratively fine-tunes the network, while gradually tapering the computation resources available to the pruned network via a holonomic constraint in the method of Lagrangian multipliers framework. An explicit and adaptive automatic control over the rate of tapering is provided. The trainable parameters of our pruning method are separate from the weights of the neural network, which allows us to avoid the interference with the neural network solver (e.g. avoid the direct dependence of pruning speed on neural network learning rates). Our method combines the `rigoristic' approach by the direct application of constrained optimization, avoiding the pitfalls of ADMM-based methods, like their need to define the target amount of resources for each pruning run, and direct dependence of pruning speed and priority of pruning on the relative scale of weights between layers. For VGG-16 @ ILSVRC-2012, we achieve reduction of 15.47 -> 3.87 GMAC with only 1% top-1 accuracy reduction (68.4% -> 67.4%). For AlexNet @ ILSVRC-2012, we achieve 0.724 -> 0.411 GMAC with 1% top-1 accuracy reduction (56.8% -> 55.8%).
RNNFast: An Accelerator for Recurrent Neural Networks Using Domain Wall Memory
Authors: Mohammad Hossein Samavatian, Anys Bacha, Li Zhou, Radu Teodorescu
Date: 2018-11-07 00:00:00
Tasks: speech-recognition, Speech Recognition, Translation

Recurrent Neural Networks (RNNs) are an important class of neural networks designed to retain and incorporate context into current decisions. RNNs are particularly well suited for machine learning problems in which context is important, such as speech recognition and language translation. This work presents RNNFast, a hardware accelerator for RNNs that leverages an emerging class of non-volatile memory called domain-wall memory (DWM). We show that DWM is very well suited for RNN acceleration due to its very high density and low read/write energy. At the same time, the sequential nature of input/weight processing of RNNs mitigates one of the downsides of DWM, which is the linear (rather than constant) data access time.RNNFast is very efficient and highly scalable, with flexible mapping of logical neurons to RNN hardware blocks. The basic hardware primitive, the RNN processing element (PE) includes custom DWM-based multiplication, sigmoid and tanh units for high density and low-energy. The accelerator is designed to minimize data movement by closely interleaving DWM storage and computation. We compare our design with a state-of-the-art GPGPU and find21.8x higher performance with70x lower energy
Evaluating Pruning Methods in Gene Network Inference
Authors: Michael M. Saint-Antoine, Abhyudai Singh
Date: 2019-02-16 00:00:00

One challenge in gene network inference is distinguishing between direct and indirect regulation. Some algorithms, including ARACNE and Phixer, approach this problem by using pruning methods to eliminate redundant edges in an attempt to explain the observed data with the simplest possible network structure. However, we hypothesize that there may be a cost in accuracy to simplifying the predicted networks in this way, especially due to the prevalence of redundant connections, such as feed forward loops, in gene networks. In this paper, we evaluate the pruning methods of ARACNE and Phixer, and score their accuracy using receiver operating characteristic curves and precision-recall curves. Our results suggest that while pruning can be useful in some situations, it may have a negative effect on overall accuracy that has not been previously studied. Researchers should be aware of both the advantages and disadvantages of pruning when inferring networks, in order to choose the best inference strategy for their experimental context.
Kaggle Competition: Expedia Hotel Recommendations
Authors: Gourav G. Shenoy, Mangirish A. Wagle, Anwar Shaikh
Date: 2017-03-06 00:00:00

With hundreds, even thousands, of hotels to choose from at every destination, it's difficult to know which will suit your personal preferences. Expedia wants to take the proverbial rabbit hole out of hotel search by providing personalized hotel recommendations to their users. This is no small task for a site with hundreds of millions of visitors every month! Currently, Expedia uses search parameters to adjust their hotel recommendations, but there aren't enough customer specific data to personalize them for each user. In this project, we have taken up the challenge to contextualize customer data and predict the likelihood a user will stay at 100 different hotel groups.
Asymmetric Non-local Neural Networks for Semantic Segmentation
Authors: Zhen Zhu, Mengde Xu, Song Bai, Tengteng Huang, Xiang Bai
Date: 2019-08-21 00:00:00
Proceeding: ICCV 2019 10
Tasks: GPU, Segmentation, Semantic Segmentation

The non-local module works as a particularly useful technique for semantic segmentation while criticized for its prohibitive computation and GPU memory occupation. In this paper, we present Asymmetric Non-local Neural Network to semantic segmentation, which has two prominent components: Asymmetric Pyramid Non-local Block (APNB) and Asymmetric Fusion Non-local Block (AFNB). APNB leverages a pyramid sampling module into the non-local block to largely reduce the computation and memory consumption without sacrificing the performance. AFNB is adapted from APNB to fuse the features of different levels under a sufficient consideration of long range dependencies and thus considerably improves the performance. Extensive experiments on semantic segmentation benchmarks demonstrate the effectiveness and efficiency of our work. In particular, we report the state-of-the-art performance of 81.3 mIoU on the Cityscapes test set. For a 256x128 input, APNB is around 6 times faster than a non-local block on GPU while 28 times smaller in GPU running memory occupation. Code is available at: https://github.com/MendelXu/ANN.git.