OpenRaiser/Intern-Atlas huggingface dataset:
A case for multiple and parallel RRAMs as synaptic model for training SNNs
Venue: arXiv cs.ET 2018
Year: 2018

To enable a dense integration of model synapses in a spiking neural networks hardware, various nano-scale devices are being considered. Such a device, besides exhibiting spike-time dependent plasticity (STDP), needs to be highly scalable, have a large endurance and require low energy for transitioning between states. In this work, we first introduce and empirically determine two new specifications for an synapse in SNNs: number of conductance levels per synapse and maximum learning-rate. To the best of our knowledge, there are no RRAMs that meet the latter specification. As a solution, we propose the use of multiple PCMO-RRAMs in parallel within a synapse. While synaptic reading, all PCMO-RRAMs are simultaneously read and for each synaptic conductance-change event, the mechanism for conductance STDP is initiated for only one RRAM, randomly picked from the set. Second, to validate our solution, we experimentally demonstrate STDP of conductance of a PCMO-RRAM and then show that due to a large learning-rate, a single PCMO-RRAM fails to model a synapse in the training of an SNN. As anticipated, network training improves as more PCMO-RRAMs are added to the synapse. Fourth, we discuss the circuit-requirements for implementing such a scheme, to conclude that the requirements are within bounds. Thus, our work presents specifications for synaptic devices in trainable SNNs, indicates the shortcomings of state-of-art synaptic contenders, and provides a solution to extrinsically meet the specifications and discusses the peripheral circuitry that implements the solution.
A Scalable Near-Memory Architecture for Training Deep Neural Networks on Large In-Memory Datasets
Venue: arXiv cs.DC 2018
Year: 2018

Most investigations into near-memory hardware accelerators for deep neural networks have primarily focused on inference, while the potential of accelerating training has received relatively little attention so far. Based on an in-depth analysis of the key computational patterns in state-of-the-art gradient-based training methods, we propose an efficient near-memory acceleration engine called NTX that can be used to train state-of-the-art deep convolutional neural networks at scale. Our main contributions are: (i) a loose coupling of RISC-V cores and NTX co-processors reducing offloading overhead by 7x over previously published results; (ii) an optimized IEEE754 compliant data path for fast high-precision convolutions and gradient propagation; (iii) evaluation of near-memory computing with NTX embedded into residual area on the Logic Base die of a Hybrid Memory Cube; and (iv) a scaling analysis to meshes of HMCs in a data center scenario. We demonstrate a 2.7x energy efficiency improvement of NTX over contemporary GPUs at 4.4x less silicon area, and a compute performance of 1.2 Tflop/s for training large state-of-the-art networks with full floating-point precision. At the data center scale, a mesh of NTX achieves above 95% parallel and energy efficiency, while providing 2.1x energy savings or 3.1x performance improvement over a GPU-based system.
Renewing computing paradigms for more efficient parallelization of single-threads
Venue: arXiv cs.DC 2018
Year: 2018

Computing is still based on the 70-years old paradigms introduced by von Neumann. The need for more performant, comfortable and safe computing forced to develop and utilize several tricks both in hardware and software. Till now technology enabled to increase performance without changing the basic computing paradigms. The recent stalling of single-threaded computing performance, however, requires to redesign computing to be able to provide the expected performance. To do so, the computing paradigms themselves must be scrutinized. The limitations caused by the too restrictive interpretation of the computing paradigms are demonstrated, an extended computing paradigm introduced, ideas about changing elements of the computing stack suggested, some implementation details of both hardware and software discussed. The resulting new computing stack offers considerably higher computing throughput, simplified hardware architecture, drastically improved real-time behavior and in general, simplified and more efficient computing stack.
A Survey on Deep Learning Toolkits and Libraries for Intelligent User Interfaces
Venue: arXiv cs.HC 2018
Year: 2018

This paper provides an overview of prominent deep learning toolkits and, in particular, reports on recent publications that contributed open source software for implementing tasks that are common in intelligent user interfaces (IUI). We provide a scientific reference for researchers and software engineers who plan to utilise deep learning techniques within their IUI research and development projects.
Impacts of transport development on residence choice of renter households: An agent-based evaluation
Venue: arXiv cs.MA 2018
Year: 2018

Because of improving accessibility, transport developments play an important role in residence choice of renter households. In this paper, an agent-based model is developed to investigate impacts of different transport developments on residence choice of renter households in Tehran, the capital of Iran. In the proposed model, renter households are considered as agents who make a multi-objective decision and compete with each other to rent a preferred residential zone. Then, three transport development scenarios including construction a new highway, subway and bus rapid transit (BRT) line are simulated and resulting changes in residence choice of agents are evaluated. Results show that transport development scenarios significantly affect residence choice behavior of different socio-economic categories of renter households and lead to considerable changes in the residential demand, composition of residents, mean income level and mean car ownership in their vicinities.
Scalable Algorithms for Parallel Tree-based Adaptive Mesh Refinement with General Element Types
Venue: arXiv cs.DC 2018
Year: 2018

In this thesis, we develop, discuss and implement algorithms for scalable parallel tree-based adaptive mesh refinement (AMR) using space-filling curves (SFCs). We create an AMR software that works independently of the used element type, such as for example lines, triangles, tetrahedra, quadrilaterals, hexahedra, and prisms. Along with a detailed mathematical discussion, this requires the implementation as a numerical software and its validation, as well as scalability tests on current supercomputers. For triangular and tetrahedral elements (simplices) with red-refinement (1:4 in 2D, 1:8 in 3D), we develop a new SFC index, the tetrahedral Morton index (TM-index). Its construction is similar to the Morton index for quadrilaterals/hexahedra, as it is also based on bitwise interleaving the coordinates of a certain vertex of the simplex, the anchor node. We develop and demonstrate a new simplicial SFC and create a fast and scalable tree-based AMR software that offers a flexibility and generality that was previously not available.
LCANet: End-to-End Lipreading with Cascaded Attention-CTC
Venue: arXiv cs.CV 2018
Year: 2018

Machine lipreading is a special type of automatic speech recognition (ASR) which transcribes human speech by visually interpreting the movement of related face regions including lips, face, and tongue. Recently, deep neural network based lipreading methods show great potential and have exceeded the accuracy of experienced human lipreaders in some benchmark datasets. However, lipreading is still far from being solved, and existing methods tend to have high error rates on the wild data. In this paper, we propose LCANet, an end-to-end deep neural network based lipreading system. LCANet encodes input video frames using a stacked 3D convolutional neural network (CNN), highway network and bidirectional GRU network. The encoder effectively captures both short-term and long-term spatio-temporal information. More importantly, LCANet incorporates a cascaded attention-CTC decoder to generate output texts. By cascading CTC with attention, it partially eliminates the defect of the conditional independence assumption of CTC within the hidden neural layers, and this yields notably performance improvement as well as faster convergence. The experimental results show the proposed system achieves a 1.3% CER and 3.0% WER on the GRID corpus database, leading to a 12.3% improvement compared to the state-of-the-art methods.
A Brief Retrospective Look at the Cayley-Purser Public-key Cryptosystem, 19 Years Later
Venue: arXiv cs.CR 2018
Year: 2018

The purpose of this paper is to describe and analyze the Cayley-Purser algorithm, which is a public-key cryptosystem proposed by Flannery in 1999. I will present two attacks on it, one of which is apparently new. I will also examine a variant of the Cayley-Purser algorithm that was patented by Slavin in 2008, and show that it is also insecure.
A Probabilistic Disease Progression Model for Predicting Future Clinical Outcome
Venue: arXiv cs.LG 2018
Year: 2018

In this work, we consider the problem of predicting the course of a progressive disease, such as cancer or Alzheimer's. Progressive diseases often start with mild symptoms that might precede a diagnosis, and each patient follows their own trajectory. Patient trajectories exhibit wild variability, which can be associated with many factors such as genotype, age, or sex. An additional layer of complexity is that, in real life, the amount and type of data available for each patient can differ significantly. For example, for one patient we might have no prior history, whereas for another patient we might have detailed clinical assessments obtained at multiple prior time-points. This paper presents a probabilistic model that can handle multiple modalities (including images and clinical assessments) and variable patient histories with irregular timings and missing entries, to predict clinical scores at future time-points. We use a sigmoidal function to model latent disease progression, which gives rise to clinical observations in our generative model. We implemented an approximate Bayesian inference strategy on the proposed model to estimate the parameters on data from a large population of subjects. Furthermore, the Bayesian framework enables the model to automatically fine-tune its predictions based on historical observations that might be available on the test subject. We applied our method to a longitudinal Alzheimer's disease dataset with more than 3000 subjects [23] and present a detailed empirical analysis of prediction performance under different scenarios, with comparisons against several benchmarks. We also demonstrate how the proposed model can be interrogated to glean insights about temporal dynamics in Alzheimer's disease.
Principal Component Analysis with Tensor Train Subspace
Venue: arXiv cs.LG 2018
Year: 2018

Tensor train is a hierarchical tensor network structure that helps alleviate the curse of dimensionality by parameterizing large-scale multidimensional data via a set of network of low-rank tensors. Associated with such a construction is a notion of Tensor Train subspace and in this paper we propose a TT-PCA algorithm for estimating this structured subspace from the given data. By maintaining low rank tensor structure, TT-PCA is more robust to noise comparing with PCA or Tucker-PCA. This is borne out numerically by testing the proposed approach on the Extended YaleFace Dataset B.
Investigating the Effect of Music and Lyrics on Spoken-Word Recognition
Venue: arXiv cs.SD 2018
Year: 2018

Background music in social interaction settings can hinder conversation. Yet, little is known of how specific properties of music impact speech processing. This paper addresses this knowledge gap by investigating 1) whether the masking effect of background music with lyrics is larger than that of music without lyrics, and 2) whether the masking effect is larger for more complex music. To answer these questions, a word identification experiment was run in which Dutch participants listened to Dutch CVC words embedded in stretches of background music in two conditions, with and without lyrics, and at three SNRs. Three songs were used of different genres and complexities. Music stretches with and without lyrics were sampled from the same song in order to control for factors beyond the presence of lyrics. The results showed a clear negative impact of the presence of lyrics in background music on spoken-word recognition. This impact is independent of complexity. The results suggest that social spaces (e.g., restaurants, cafés and bars) should make careful choices of music to promote conversation, and open a path for future work.
A Survey of Multimedia Streaming in LTE Cellular Networks
Venue: arXiv cs.NI 2018
Year: 2018

With the growing of Long Term Evolution (LTE) cellular networks and the increase in the demand of the video services, it is vital to consider the challenges in the streaming services from a different perspective. A perspective that focuses on the streaming services in light of cellular networks challenges, both per layer basis and across multiple layers as well. In this tutorial, we highlight the main challenges that faces the industry of video streaming in the context of cellular networks with a focus on LTE. We also discuss proposed solutions for these challenges while highlighting the limitations of these solutions and the conditions/assumptions required for these solution to deliver high performance. In addition, we show different work in cross layer optimization for video streaming and how it leads towards a more optimized end to end LTE networking for video streaming. Finally, we suggest different open research areas in the domain of video delivery over LTE networks that can significantly enhance the quality of streaming experience to the end user.
Multi-objective Analysis of MAP-Elites Performance
Venue: arXiv cs.NE 2018
Year: 2018

In certain complex optimization tasks, it becomes necessary to use multiple measures to characterize the performance of different algorithms. This paper presents a method that combines ordinal effect sizes with Pareto dominance to analyze such cases. Since the method is ordinal, it can also generalize across different optimization tasks even when the performance measurements are differently scaled. Through a case study, we show that this method can discover and quantify relations that would be difficult to deduce using a conventional measure-by-measure analysis. This case study applies the method to the evolution of robot controller repertoires using the MAP-Elites algorithm. Here, we analyze the search performance across a large set of parametrizations; varying mutation size and operator type, as well as map resolution, across four different robot morphologies. We show that the average magnitude of mutations has a bigger effect on outcomes than their precise distributions.
OSINT Analysis of the TOR Foundation
Venue: arXiv cs.CR 2018
Year: 2018

This research paper is a study on the TOR Foundation and the TOR Network. Due to the blind confidence over this network and this foundation we have collect data and gather information in open sources to understand how it is organized. We discovered that the US government is very active through the financial aspect and development aspect. Furthermore, we discovered that some critical points of the TOR network are running by unknown people who have special nodes and the foundations never talks about them.
Secure SWIPT for Directional Modulation Aided AF Relaying Networks
Venue: arXiv cs.IT 2018
Year: 2018

Secure wireless information and power transfer based on directional modulation is conceived for amplify-and-forward (AF) relaying networks. Explicitly, we first formulate a secrecy rate maximization (SRM) problem, which can be decomposed into a twin-level optimization problem and solved by a one-dimensional (1D) search and semidefinite relaxation (SDR) technique. Then in order to reduce the search complexity, we formulate an optimization problem based on maximizing the signal-to-leakage-AN-noise-ratio (Max-SLANR) criterion, and transform it into a SDR problem. Additionally, the relaxation is proved to be tight according to the classic Karush-Kuhn-Tucker (KKT) conditions. Finally, to reduce the computational complexity, a successive convex approximation (SCA) scheme is proposed to find a near-optimal solution. The complexity of the SCA scheme is much lower than that of the SRM and the Max-SLANR schemes. Simulation results demonstrate that the performance of the SCA scheme is very close to that of the SRM scheme in terms of its secrecy rate and bit error rate (BER), but much better than that of the zero forcing (ZF) scheme.
Illumination-aware Faster R-CNN for Robust Multispectral Pedestrian Detection
Venue: arXiv cs.CV 2018
Year: 2018

Multispectral images of color-thermal pairs have shown more effective than a single color channel for pedestrian detection, especially under challenging illumination conditions. However, there is still a lack of studies on how to fuse the two modalities effectively. In this paper, we deeply compare six different convolutional network fusion architectures and analyse their adaptations, enabling a vanilla architecture to obtain detection performances comparable to the state-of-the-art results. Further, we discover that pedestrian detection confidences from color or thermal images are correlated with illumination conditions. With this in mind, we propose an Illumination-aware Faster R-CNN (IAF RCNN). Specifically, an Illumination-aware Network is introduced to give an illumination measure of the input image. Then we adaptively merge color and thermal sub-networks via a gate function defined over the illumination value. The experimental results on KAIST Multispectral Pedestrian Benchmark validate the effectiveness of the proposed IAF R-CNN.
On the insufficiency of existing momentum schemes for Stochastic Optimization
Venue: arXiv cs.LG 2018
Year: 2018

Momentum based stochastic gradient methods such as heavy ball (HB) and Nesterov's accelerated gradient descent (NAG) method are widely used in practice for training deep networks and other supervised learning models, as they often provide significant improvements over stochastic gradient descent (SGD). Rigorously speaking, "fast gradient" methods have provable improvements over gradient descent only for the deterministic case, where the gradients are exact. In the stochastic case, the popular explanations for their wide applicability is that when these fast gradient methods are applied in the stochastic case, they partially mimic their exact gradient counterparts, resulting in some practical gain. This work provides a counterpoint to this belief by proving that there exist simple problem instances where these methods cannot outperform SGD despite the best setting of its parameters. These negative problem instances are, in an informal sense, generic; they do not look like carefully constructed pathological instances. These results suggest (along with empirical evidence) that HB or NAG's practical performance gains are a by-product of mini-batching. Furthermore, this work provides a viable (and provable) alternative, which, on the same set of problem instances, significantly improves over HB, NAG, and SGD's performance. This algorithm, referred to as Accelerated Stochastic Gradient Descent (ASGD), is a simple to implement stochastic algorithm, based on a relatively less popular variant of Nesterov's Acceleration. Extensive empirical results in this paper show that ASGD has performance gains over HB, NAG, and SGD.
Resource Allocation in NOMA based Fog Radio Access Networks
Venue: arXiv cs.IT 2018
Year: 2018

In the wake of growth in intelligent mobile devices and wide usage of bandwidth-hungry applications of mobile Internet, the demand of wireless data traffic and ubiquitous mobile broadband is rapidly increasing. On account of these developments, the research on fifth generation (5G) networks presents an accelerative tendency on a global scale. Edge computing draw lots of attention for reducing the time delay and improving the Quality of Service for the networks. While, fog radio access networks (F-RANs) is an emergent architecture, which takes full use of edge computing and distributed storing capabilities in edge devices. In this article, we propose an architecture of non-orthogonal multiple access (NOMA) based F-RANs, which has a strong capability of edge computing and can meet the heterogeneous requirements in 5G systems. NOMA with successive interference cancellation (SIC) is regarded as a critical multi-user access technology. In NOMA, more than one user can access the same time, code domain, and frequency resources. With assigning different power levels to multi-user and implementing SIC, multiple users detection can be achieved. In this article, we provide a description of the NOMA based F-RANs architecture, and discuss the resource allocation in that. We will focus on the power and subchannel allocation in consideration of using NOMA and the edge caching. Simulation results show that the proposed NOMA baesd F-RANs architecture and the resource management mechanisms can achieve the high net utility for the RANs.
Motion optimization and parameter identification for a human and lower-back exoskeleton model
Venue: arXiv cs.RO 2018
Year: 2018

Designing an exoskeleton to reduce the risk of low-back injury during lifting is challenging. Computational models of the human-robot system coupled with predictive movement simulations can help to simplify this design process. Here, we present a study that models the interaction between a human model actuated by muscles and a lower-back exoskeleton. We provide a computational framework for identifying the spring parameters of the exoskeleton using an optimal control approach and forward-dynamics simulations. This is applied to generate dynamically consistent bending and lifting movements in the sagittal plane. Our computations are able to predict motions and forces of the human and exoskeleton that are within the torque limits of a subject. The identified exoskeleton could also yield a considerable reduction of the peak lower-back torques as well as the cumulative lower-back load during the movements. This work is relevant to the research communities working on human-robot interaction, and can be used as a basis for a better human-centered design process.
Approximating Max-Cut under Graph-MSO Constraints
Venue: arXiv cs.CC 2018
Year: 2018

We consider the max-cut and max-$k$-cut problems under graph-based constraints. Our approach can handle any constraint specified using monadic second-order (MSO) logic on graphs of constant treewidth. We give a $\frac{1}{2}$-approximation algorithm for this class of problems.
Joint Rate Allocation with Both Look-ahead And Feedback Model For High Efficiency Video Coding
Venue: arXiv cs.MM 2018
Year: 2018

The objective of joint rate allocation among multiple coded video streams is to share the bandwidth to meet the demands of minimum average distortion (minAVE) or minimum distortion variance (minVAR). In previous works on minVAR problems, bits are directly assigned in proportion to their complexity measures and we call it look-ahead allocation model (LAM), which leads to the fact that the performance will totally depend on the accuracy of the complexity measures. This paper proposes a look-ahead and feedback allocation model (LFAM) for joint rate allocation for High Efficiency Video Coding (HEVC) platform which requires negligible computational cost. We derive the model from the target function of minVAR theoretically. The bits are assigned according to the complexity measures, the distortion and bitrate values fed back by the encoder together. We integrated the proposed allocation model in HEVC reference software HM16.0 and several complexity measures were applied to our allocation model. Results demonstrate that our proposed LFAM performs better than LAM, and an average of 65.94% variance of mean square error (MSE) is saved with different complexity measures.