OpenRaiser/Intern-Atlas huggingface dataset:
A Survey of Neuromorphic Computing and Neural Networks in Hardware
Venue: arXiv cs.NE 2017
Year: 2017

Neuromorphic computing has come to refer to a variety of brain-inspired computers, devices, and models that contrast the pervasive von Neumann computer architecture. This biologically inspired approach has created highly connected synthetic neurons and synapses that can be used to model neuroscience theories as well as solve challenging machine learning problems. The promise of the technology is to create a brain-like ability to learn and adapt, but the technical challenges are significant, starting with an accurate neuroscience model of how the brain works, to finding materials and engineering breakthroughs to build devices to support these models, to creating a programming framework so the systems can learn, to creating applications with brain-like capabilities. In this work, we provide a comprehensive survey of the research and motivations for neuromorphic computing over its history. We begin with a 35-year review of the motivations and drivers of neuromorphic computing, then look at the major research areas of the field, which we define as neuro-inspired models, algorithms and learning approaches, hardware and devices, supporting systems, and finally applications. We conclude with a broad discussion on the major research topics that need to be addressed in the coming years to see the promise of neuromorphic computing fulfilled. The goals of this work are to provide an exhaustive review of the research conducted in neuromorphic computing since the inception of the term, and to motivate further work by illuminating gaps in the field where new research is needed.
Realization of CDMA-based IoT Services with Shared Band Operation of LTE in 5G
Venue: arXiv cs.NI 2017
Year: 2017

5G network is envisioned to deploy a massive Internet-of-Things (IoTs) with requirements of low-latency, low control overhead and low power. Current 4G network is optimized for large bandwidth applications and inefficient to handle short sporadic IoT messages. The challenge here spans multiple layer including the radio access and the network layer. This paper focus on reusing CDMA access for IoT devices considering event-driven and latency sensitive traffic profile. We propose a PHY/MAC layer design for CDMA based communication for low power IoT devices. We propose and evaluate coexisting operation of CDMA based IoT network in presence of the exiting LTE network. Our proposed design will integrate IoT traffic with legacy system by minimal modification at the edge network, essentially eNodeB. We show that the underlay CDMA IoT network meets IoT data traffic requirements with minimal degradation (3%) in the LTE throughput. We also implement the proposed design using Software Defined Radios and show the viability of the proposal under different network scenarios.
Cooperative Spectrum Sensing over Generalized Fading Channels Based on Energy Detection
Venue: arXiv cs.IT 2017
Year: 2017

This paper analyzes the unified performance of energy detection (ED) of spectrum sensing (SS) over generalized fading channels in cognitive radio (CR) networks. The detective performance of SS schemes will be obviously affected by fading channel between communication nodes, and ED has the advantages of fast implementation, no requirement of priori received information and low complexity, so it is meaningful to investigate ED that is performed over fading channels such as Nakagami-m channel and Rice channel, or generalized fading channels such as \k{appa}-μ fading distribution and η-μ fading distribution. The α-\k{appa}-μ fading distribution is a generalized fading model that represents the nonlinear and small-scale variation of fading channels. The probability density function (p.d.f.) of instantaneous signal-to-ratio (SNR) of α-\k{appa}-μ distribution is derived from the envelope p.d.f. to evaluate energy efficiency for sensing systems. Next, the probability of detection model with Marcum-Q function has been derived and the close-form detective expressions with moment generating function (MGF) method are deduced to achieve sensing communications over generalized fading channels. Furthermore, novel and exact closed-form analytic expressions for average area under the receiver operating characteristics curve also have been deduced to analyze the performance characteristics of ED over α-\k{appa}-μ fading channels. Besides, cooperative spectrum sensing (CSS) with diversity reception has been applied to improve the detection accuracy and mitigate the shadowed fading features with OR-rule. At last, the results show that the detection capacity can be evidently affected by α-\k{appa}-μ fading conditions, but appropriate channel parameters will improve sensing performance.
Cooperative spectrum sensing with enhanced energy detection under GAUSSIAN noise uncertainty in cognitive radios
Venue: arXiv cs.IT 2017
Year: 2017

This paper presents optimization issues of energy detection (ED) thresholds in cooperative spectrum sensing (CSS) with regard to general Gaussian noise. Enhanced ED thresholds are proposed to overcome sensitivity of multiple noise uncertainty. Two-steps decision pattern and convex samples thresholds have been put forward under Gaussian noise uncertainty. Through deriving the probability of detection (Pd) and the probability of false alarm (Pf ) for independent and identical distribution (i.i.d.) SUs, we obtain lower total error rate (Qe) with proposed ED thresholds at low signal-to-noise-ratio (SNR) condition. Furthermore, simulation results show that proposed schemes outperform most other noise uncertainty plans.
Achievable Information Rates for Coded Modulation with Hard Decision Decoding for Coherent Fiber-Optic Systems
Venue: arXiv cs.IT 2017
Year: 2017

We analyze the achievable information rates (AIRs) for coded modulation schemes with QAM constellations with both bit-wise and symbol-wise decoders, corresponding to the case where a binary code is used in combination with a higher-order modulation using the bit-interleaved coded modulation (BICM) paradigm and to the case where a nonbinary code over a field matched to the constellation size is used, respectively. In particular, we consider hard decision decoding, which is the preferable option for fiber-optic communication systems where decoding complexity is a concern. Recently, Liga \emph{et al.} analyzed the AIRs for bit-wise and symbol-wise decoders considering what the authors called \emph{hard decision decoder} which, however, exploits \emph{soft information} of the transition probabilities of discrete-input discrete-output channel resulting from the hard detection. As such, the complexity of the decoder is essentially the same as the complexity of a soft decision decoder. In this paper, we analyze instead the AIRs for the standard hard decision decoder, commonly used in practice, where the decoding is based on the Hamming distance metric. We show that if standard hard decision decoding is used, bit-wise decoders yield significantly higher AIRs than symbol-wise decoders. As a result, contrary to the conclusion by Liga \emph{et al.}, binary decoders together with the BICM paradigm are preferable for spectrally-efficient fiber-optic systems. We also design binary and nonbinary staircase codes and show that, in agreement with the AIRs, binary codes yield better performance.
Speeding up Memory-based Collaborative Filtering with Landmarks
Venue: arXiv cs.IR 2017
Year: 2017

Recommender systems play an important role in many scenarios where users are overwhelmed with too many choices to make. In this context, Collaborative Filtering (CF) arises by providing a simple and widely used approach for personalized recommendation. Memory-based CF algorithms mostly rely on similarities between pairs of users or items, which are posteriorly employed in classifiers like k-Nearest Neighbor (kNN) to generalize for unknown ratings. A major issue regarding this approach is to build the similarity matrix. Depending on the dimensionality of the rating matrix, the similarity computations may become computationally intractable. To overcome this issue, we propose to represent users by their distances to preselected users, namely landmarks. This procedure allows to drastically reduce the computational cost associated with the similarity matrix. We evaluated our proposal on two distinct distinguishing databases, and the results showed our method has consistently and considerably outperformed eight CF algorithms (including both memory-based and model-based) in terms of computational performance.
Analyzing Privacy Breaches in the Aircraft Communications Addressing and Reporting System (ACARS)
Venue: arXiv cs.CR 2017
Year: 2017

The manner in which Aircraft Communications, Addressing and Reporting System (ACARS) is being used has significantly changed over time. Whilst originally used by commercial airliners to track their flights and provide automated timekeeping on crew, today it serves as a multi-purpose air-ground data link for many aviation stakeholders including private jet owners, state actors and military. Since ACARS messages are still mostly sent in the clear over a wireless channel, any sensitive information sent with ACARS can potentially lead to a privacy breach for users. Naturally, different stakeholders consider different types of data sensitive. In this paper we propose a privacy framework matching aviation stakeholders to a range of sensitive information types and assess the impact for each. Based on more than one million ACARS messages, collected over several months, we then demonstrate that current ACARS usage systematically breaches privacy for all stakeholder groups. We further support our findings with a number of cases of significant privacy issues for each group and analyze the impact of such leaks. While it is well-known that ACARS messages are susceptible to eavesdropping attacks, this work is the first to quantify the extent and impact of privacy leakage in the real world for the relevant aviation stakeholders.
EE-Grad: Exploration and Exploitation for Cost-Efficient Mini-Batch SGD
Venue: arXiv cs.LG 2017
Year: 2017

We present a generic framework for trading off fidelity and cost in computing stochastic gradients when the costs of acquiring stochastic gradients of different quality are not known a priori. We consider a mini-batch oracle that distributes a limited query budget over a number of stochastic gradients and aggregates them to estimate the true gradient. Since the optimal mini-batch size depends on the unknown cost-fidelity function, we propose an algorithm, {\it EE-Grad}, that sequentially explores the performance of mini-batch oracles and exploits the accumulated knowledge to estimate the one achieving the best performance in terms of cost-efficiency. We provide performance guarantees for EE-Grad with respect to the optimal mini-batch oracle, and illustrate these results in the case of strongly convex objectives. We also provide a simple numerical example that corroborates our theoretical findings.
What do We Learn by Semantic Scene Understanding for Remote Sensing imagery in CNN framework?
Venue: arXiv cs.CV 2017
Year: 2017

Recently, deep convolutional neural network (DCNN) achieved increasingly remarkable success and rapidly developed in the field of natural image recognition. Compared with the natural image, the scale of remote sensing image is larger and the scene and the object it represents are more macroscopic. This study inquires whether remote sensing scene and natural scene recognitions differ and raises the following questions: What are the key factors in remote sensing scene recognition? Is the DCNN recognition mechanism centered on object recognition still applicable to the scenarios of remote sensing scene understanding? We performed several experiments to explore the influence of the DCNN structure and the scale of remote sensing scene understanding from the perspective of scene complexity. Our experiment shows that understanding a complex scene depends on an in-depth network and multiple-scale perception. Using a visualization method, we qualitatively and quantitatively analyze the recognition mechanism in a complex remote sensing scene and demonstrate the importance of multi-objective joint semantic support.
Deep Learning as a Tool to Predict Flow Patterns in Two-Phase Flow
Venue: arXiv cs.LG 2017
Year: 2017

In order to better model complex real-world data such as multiphase flow, one approach is to develop pattern recognition techniques and robust features that capture the relevant information. In this paper, we use deep learning methods, and in particular employ the multilayer perceptron, to build an algorithm that can predict flow pattern in twophase flow from fluid properties and pipe conditions. The preliminary results show excellent performance when compared with classical methods of flow pattern prediction.
Softmax Q-Distribution Estimation for Structured Prediction: A Theoretical Interpretation for RAML
Venue: arXiv cs.LG 2017
Year: 2017

Reward augmented maximum likelihood (RAML), a simple and effective learning framework to directly optimize towards the reward function in structured prediction tasks, has led to a number of impressive empirical successes. RAML incorporates task-specific reward by performing maximum-likelihood updates on candidate outputs sampled according to an exponentiated payoff distribution, which gives higher probabilities to candidates that are close to the reference output. While RAML is notable for its simplicity, efficiency, and its impressive empirical successes, the theoretical properties of RAML, especially the behavior of the exponentiated payoff distribution, has not been examined thoroughly. In this work, we introduce softmax Q-distribution estimation, a novel theoretical interpretation of RAML, which reveals the relation between RAML and Bayesian decision theory. The softmax Q-distribution can be regarded as a smooth approximation of the Bayes decision boundary, and the Bayes decision rule is achieved by decoding with this Q-distribution. We further show that RAML is equivalent to approximately estimating the softmax Q-distribution, with the temperature $τ$ controlling approximation error. We perform two experiments, one on synthetic data of multi-class classification and one on real data of image captioning, to demonstrate the relationship between RAML and the proposed softmax Q-distribution estimation method, verifying our theoretical analysis. Additional experiments on three structured prediction tasks with rewards defined on sequential (named entity recognition), tree-based (dependency parsing) and irregular (machine translation) structures show notable improvements over maximum likelihood baselines.
Deep De-Aliasing for Fast Compressive Sensing MRI
Venue: arXiv cs.CV 2017
Year: 2017

Fast Magnetic Resonance Imaging (MRI) is highly in demand for many clinical applications in order to reduce the scanning cost and improve the patient experience. This can also potentially increase the image quality by reducing the motion artefacts and contrast washout. However, once an image field of view and the desired resolution are chosen, the minimum scanning time is normally determined by the requirement of acquiring sufficient raw data to meet the Nyquist-Shannon sampling criteria. Compressive Sensing (CS) theory has been perfectly matched to the MRI scanning sequence design with much less required raw data for the image reconstruction. Inspired by recent advances in deep learning for solving various inverse problems, we propose a conditional Generative Adversarial Networks-based deep learning framework for de-aliasing and reconstructing MRI images from highly undersampled data with great promise to accelerate the data acquisition process. By coupling an innovative content loss with the adversarial loss our de-aliasing results are more realistic. Furthermore, we propose a refinement learning procedure for training the generator network, which can stabilise the training with fast convergence and less parameter tuning. We demonstrate that the proposed framework outperforms state-of-the-art CS-MRI methods, in terms of reconstruction error and perceptual image quality. In addition, our method can reconstruct each image in 0.22ms--0.37ms, which is promising for real-time applications.
State Complexity of Reversals of Deterministic Finite Automata with Output
Venue: arXiv cs.FL 2017
Year: 2017

We investigate the worst-case state complexity of reversals of deterministic finite automata with output (DFAOs). In these automata, each state is assigned some output value, rather than simply being labelled final or non-final. This directly generalizes the well-studied problem of determining the worst-case state complexity of reversals of ordinary deterministic finite automata. If a DFAO has $n$ states and $k$ possible output values, there is a known upper bound of $k^n$ for the state complexity of reversal. We show this bound can be reached with a ternary input alphabet. We conjecture it cannot be reached with a binary input alphabet except when $k = 2$, and give a lower bound for the case $3 \le k < n$. We prove that the state complexity of reversal depends solely on the transition monoid of the DFAO and the mapping that assigns output values to states.
Clustering under Local Stability: Bridging the Gap between Worst-Case and Beyond Worst-Case Analysis
Venue: arXiv cs.DS 2017
Year: 2017

Recently, there has been substantial interest in clustering research that takes a beyond worst-case approach to the analysis of algorithms. The typical idea is to design a clustering algorithm that outputs a near-optimal solution, provided the data satisfy a natural stability notion. For example, Bilu and Linial (2010) and Awasthi et al. (2012) presented algorithms that output near-optimal solutions, assuming the optimal solution is preserved under small perturbations to the input distances. A drawback to this approach is that the algorithms are often explicitly built according to the stability assumption and give no guarantees in the worst case; indeed, several recent algorithms output arbitrarily bad solutions even when just a small section of the data does not satisfy the given stability notion. In this work, we address this concern in two ways. First, we provide algorithms that inherit the worst-case guarantees of clustering approximation algorithms, while simultaneously guaranteeing near-optimal solutions when the data is stable. Our algorithms are natural modifications to existing state-of-the-art approximation algorithms. Second, we initiate the study of local stability, which is a property of a single optimal cluster rather than an entire optimal solution. We show our algorithms output all optimal clusters which satisfy stability locally. Specifically, we achieve strong positive results in our local framework under recent stability notions including metric perturbation resilience (Angelidakis et al. 2017) and robust perturbation resilience (Balcan and Liang 2012) for the $k$-median, $k$-means, and symmetric/asymmetric $k$-center objectives.
PixColor: Pixel Recursive Colorization
Venue: arXiv cs.CV 2017
Year: 2017

We propose a novel approach to automatically produce multiple colorized versions of a grayscale image. Our method results from the observation that the task of automated colorization is relatively easy given a low-resolution version of the color image. We first train a conditional PixelCNN to generate a low resolution color for a given grayscale image. Then, given the generated low-resolution color image and the original grayscale image as inputs, we train a second CNN to generate a high-resolution colorization of an image. We demonstrate that our approach produces more diverse and plausible colorizations than existing methods, as judged by human raters in a "Visual Turing Test".
Adapting Low-Cost Platforms for Robotics Research
Venue: arXiv cs.RO 2017
Year: 2017

Validation of robotics theory on real-world hardware platforms is important to prove the practical feasibility of algorithms. This paper discusses some of the lessons learned while adapting the EvoBot, a low-cost robotics platform that we designed and prototyped, for research in diverse areas in robotics. The EvoBot platform was designed to be a low cost, open source, general purpose robotics platform intended to enable testing and validation of algorithms from a wide variety of sub-fields of robotics. Throughout the paper, we outline and discuss some common failures, practical limitations and inconsistencies between theory and practice that one may encounter while adapting such low-cost platforms for robotics research. We demonstrate these aspects through four representative common robotics tasks- localization, real-time control, swarm consensus and path planning applications, performed using the EvoBots. We also propose some potential solutions to the encountered problems and try to generalize them.
Speedup from a different parametrization within the Neural Network algorithm
Venue: arXiv cs.LG 2017
Year: 2017

A different parametrization of the hyperplanes is used in the neural network algorithm. As demonstrated on several autoencoder examples it significantly outperforms the usual parametrization, reaching lower training error values with only a fraction of the number of epochs. It's argued that it makes it easier to understand and initialize the parameters.
PrivMin: Differentially Private MinHash for Jaccard Similarity Computation
Venue: arXiv cs.DS 2017
Year: 2017

In many industrial applications of big data, the Jaccard Similarity Computation has been widely used to measure the distance between two profiles or sets respectively owned by two users. Yet, one semi-honest user with unpredictable knowledge may also deduce the private or sensitive information (e.g., the existence of a single element in the original sets) of the other user via the shared similarity. In this paper, we aim at solving the privacy issues in Jaccard similarity computation with strict differential privacy guarantees. To achieve this, we first define the Conditional $ε$-DPSO, a relaxed differential privacy definition regarding set operations, and prove that the MinHash-based Jaccard Similarity Computation (MH-JSC) satisfies this definition. Then for achieving strict differential privacy in MH-JSC, we propose the PrivMin algorithm, which consists of two private operations: 1) the Private MinHash Value Generation that works by introducing the Exponential noise to the generation of MinHash signature. 2) the Randomized MinHashing Steps Selection that works by adopting Randomized Response technique to privately select several steps within the MinHashing phase that are deployed with the Exponential mechanism. Experiments on real datasets demonstrate that the proposed PrivMin algorithm can successfully retain the utility of the computed similarity while preserving privacy.
Sparse Matrix Multiplication On An Associative Processor
Venue: arXiv cs.MS 2017
Year: 2017

Sparse matrix multiplication is an important component of linear algebra computations. Implementing sparse matrix multiplication on an associative processor (AP) enables high level of parallelism, where a row of one matrix is multiplied in parallel with the entire second matrix, and where the execution time of vector dot product does not depend on the vector size. Four sparse matrix multiplication algorithms are explored in this paper, combining AP and baseline CPU processing to various levels. They are evaluated by simulation on a large set of sparse matrices. The computational complexity of sparse matrix multiplication on AP is shown to be an O(nnz) where nnz is the number of nonzero elements. The AP is found to be especially efficient in binary sparse matrix multiplication. AP outperforms conventional solutions in power efficiency.