Query: A Time-Restricted Self-Attention Layer for ASR

Relevant passages:
Attention Is All You Need <=SEP=> The dominant sequence transduction models are based on complex recurrent or convolutional neural networks that include an encoder and a decoder. The best performing models also connect the encoder and decoder through an attention mechanism. We propose a new simple network architecture, the Transformer, based solely on attention mechanisms, dispensing with recurrence and convolutions entirely. Experiments on two machine translation tasks show these models to be superior in quality while being more parallelizable and requiring significantly less time to train. Our model achieves 28.4 BLEU on the WMT 2014 Englishto-German translation task, improving over the existing best results, including ensembles, by over 2 BLEU. On the WMT 2014 English-to-French translation task, our model establishes a new single-model state-of-the-art BLEU score of 41.2 after training for 4.5 days on eight GPUs, a small fraction of the training costs of the best models from the literature. We show that the Transformer generalizes well to other tasks by applying it successfully to English constituency parsing both with large and limited training data.

The Kaldi Speech Recognition Toolkit <=SEP=> We describe the design of Kaldi, a free, open-source toolkit for speech recognition research. Kaldi provides a speech recognition system based on finite-state transducers (using the freely available OpenFst), together with detailed documentation and scripts for building complete recognition systems. Kaldi is written is C++, and the core library supports modeling of arbitrary phonetic-context sizes, acoustic modeling with subspace Gaussian mixture models (SGMM) as well as standard Gaussian mixture models, together with all commonly used linear and affine transforms. Kaldi is released under the Apache License v2.0, which is highly nonrestrictive, making it suitable for a wide community of users.

Show, Attend and Tell: Neural Image Caption Generation with Visual Attention <=SEP=> Inspired by recent work in machine translation and object detection, we introduce an attention based model that automatically learns to describe the content of images. We describe how we can train this model in a deterministic manner using standard backpropagation techniques and stochastically by maximizing a variational lower bound. We also show through visualization how the model is able to automatically learn to fix its gaze on salient objects while generating the corresponding words in the output sequence. We validate the use of attention with state-of-theart performance on three benchmark datasets: Flickr9k, Flickr30k and MS COCO.

Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift <=SEP=> Training Deep Neural Networks is complicated by the fact that the distribution of each layer’s inputs changes during training, as the parameters of the previous layers change. This slows down the training by requiring lower learning rates and careful parameter initialization, and makes it notoriously hard to train models with saturating nonlinearities. We refer to this phenomenon as internal covariate shift, and address the problem by normalizing layer inputs. Our method draws its strength from making normalization a part of the model architecture and performing the normalization for each training mini-batch. Batch Normalization allows us to use much higher learning rates and be less careful about initialization, and in some cases eliminates the need for Dropout. Applied to a stateof-the-art image classification model, Batch Normalization achieves the same accuracy with 14 times fewer training steps, and beats the original model by a significant margin. Using an ensemble of batch-normalized networks, we improve upon the best published result on ImageNet classification: reaching 4.82% top-5 test error, exceeding the accuracy of human raters.

Enhancing the TED-LIUM Corpus with Selected Data for Language Modeling and More TED Talks <=SEP=> In this paper, we present improvements made to the TED-LIUM corpus we released in 2012. These enhancements fall into two categories. First, we describe how we filtered publicly available monolingual data and used it to estimate well-suited language models (LMs), using open-source tools. Then, we describe the process of selection we applied to new acoustic data from TED talks, providing additions to our previously released corpus. Finally, we report some experiments we made around these improvements.

Blockchain challenges and opportunities: a survey <=SEP=> Copyright © 2017 Inderscience Enterprises Ltd. 2 Blockchain has numerous benefits such as decentralization, persistency, anonymity and auditability. There is a wide spectrum of blockchain applications ranging from cryptocurrency, financial services, risk management, Internet of Things to public and social services. Although a number of studies focus on using the blockchain technology in various application aspects, there is no comprehensive survey on the blockchain technology in both technological and application perspectives. To fill this gap, we conduct a comprehensive survey on the blockchain technology. In particular, this paper gives the blockchain taxonomy, introduces typical blockchain consensus algorithms, reviews blockchain applications and discusses technical challenges as well as recent advances in tackling the challenges. Moreover, this paper also points out the future directions in the blockchain technology.

A Real Time Audio Fingerprinting System for Advertisement Tracking and Reporting in FM Radio <=SEP=> In this paper we present a system designed to detect, identify and track commercial segments transmitted by a radio station. The program is entirely written in Visual C++ and uses state of the art audio fingerprinting technologies to achieve a great performance, being able to operate several times faster than real time while keeping a moderate computational load.

Mining process models with non-free-choice constructs <=SEP=> Process mining aims at extracting information from event logs to capture the business process as it is being executed. Process mining is particularly useful in situations where events are recorded but there is no system enforcing people to work in a particular way. Consider for example a hospital where the diagnosis and treatment activities are recorded in the hospital information system, but where health-care professionals determine the “careflow.” Many process mining approaches have been proposed in recent years. However, in spite of many researchers’ persistent efforts, there are still several challenging problems to be solved. In this paper, we focus on mining non-free-choice constructs, i.e., situations where there is a mixture of choice and synchronization. Although most real-life processes exhibit non-free-choice behavior, existing algorithms are unable to adequately deal with such constructs. Using a Petri-net-based representation, we will show that there are two kinds of causal dependencies between tasks, i.e., explicit and implicit ones. We propose an algorithm that is able to deal with both kinds of dependencies. The algorithm has been implemented in the ProM framework and experimental results shows that the algorithm indeed significantly improves existing process mining techniques.

Detecting and discriminating behavioural anomalies <=SEP=> This paper aims to address the problem of anomaly detection and discrimination in complex behaviours, where anomalies are subtle and difficult to detect owing to the complex temporal dynamics and correlations among multiple objects’ behaviours. Specifically, we decompose a complex behaviour pattern according to its temporal characteristics or spatial-temporal visual contexts. The decomposed behaviour is then modelled using a cascade of Dynamic Bayesian Networks (CasDBNs). In contrast to existing standalone models, the proposed behaviour decomposition and cascade modelling offers distinct advantage in simplicity for complex behaviour modelling. Importantly, the decomposition and cascade structure map naturally to the structure of complex behaviour, allowing for a more effective detection of subtle anomalies in surveillance videos. Comparative experiments using both indoor and outdoor data are carried out to demonstrate that, in addition to the novel capability of discriminating different types of anomalies, the proposed framework outperforms existing methods in detecting durational anomalies in complex behaviours and subtle anomalies that are difficult to detect when objects are viewed in isolation. & 2010 Elsevier Ltd. All rights reserved.

Mobile Virtual Network Admission Control and Resource Allocation for Wireless Network Virtualization: A Robust Optimization Approach <=SEP=> Wireless network virtualization is a promising technology in next generation wireless networks. In this paper, motivated by the experience of user equipment (UE) admission control in traditional wireless networks, we propose a novel concept of mobile virtual network (MVN) admission control for wireless virtualization. By limiting the number of MVNs embedded in the physical network, MVN admission control can effectively guarantee quality of service (QoS) experienced by users of MVNs and maximize the utilization of the physical networks at the same time. Specifically, we propose a two-stage MVN embedding mechanism that can decouple short-term physical resource allocation from long-term admission control and resource leasing. With recent advances in robust optimization, we formulate the MVN admission control problem as a robust optimization problem. Both the long-term admission control and short- term resource allocation problems are transformed to convex problems, which can be solved efficiently. Simulation results are presented to show the effectiveness of the proposed scheme.

The Experience of Nature A Psychological Perspective <=SEP=> 

A novel framework for SMS spam filtering <=SEP=> A novel framework for SMS spam filtering is introduced in this paper to prevent mobile phone users from unsolicited SMS messages. The framework makes use of two distinct feature selection approaches based on information gain and chi-square metrics to find out discriminative features representing SMS messages. The discriminative feature subsets are then employed in two different Bayesian-based classifiers, so that SMS messages are categorized as either spam or legitimate. Moreover, the paper introduces a real-time mobile application for Android™ based mobile phones utilizing the proposed spam filtering scheme, as well. Hence, SMS spam messages are silently filtered out without disturbing phone users. Effectiveness of the filtering framework is evaluated on a large SMS message collection including legitimate and spam messages. Following the evaluation, remarkably accurate classification results are obtained for both spam and legitimate messages.

Forecasting stock price directional movements using technical indicators: Investigating window size effects on one-step-ahead forecasting <=SEP=> Accurate forecasting of directional changes in stock prices is important for algorithmic trading and investment management. Technical analysis has been successfully used in financial forecasting and recently researchers have explored the optimization of parameters for technical indicators. This study investigates the relationship between the window size used for calculating technical indicators and the accuracy of one-step-ahead (variable steps) forecasting. The directions of the future price movements are predicted using technical analysis and machine learning algorithms. Results show a correlation between window size and forecasting step size for the Support Vector Machines approach but not for the other approaches.

Scene Graph Generation by Belief RNNs <=SEP=> Understanding and describing the scene beyond an image has great value. It is a 1 step forward from recognizing individual objects and their relationships in isolation. 2 In addition, Scene graph is a crucial step towards a deeper understanding of a visual 3 scene. We present a method in an end-to-end model that given an image generates 4 a scene graph including nodes that corresponds to the objects in the image and 5 an edges corresponds to relationship between objects. Our main contribution is 6 introducing a novel deep structure prediction module Belief RNNs that performs 7 learning on a large graphs in a very efficient and generic way. 8

A new Design Criteria for Hash-Functions <=SEP=> The most common way of constructing a hash function (e.g., SHA-1) is to iterate a compression function on the input message. The compression function is usually designed from scratch or made out of a block-cipher. In this paper, we introduce a new security notion for hash-functions, stronger than collision-resistance. Under this notion, the arbitrary length hash function H must behave as a random oracle when the fixed-length building block is viewed as an ideal primitive. This enables to eliminate all possible generic attacks against iterative hash-functions. In this paper, we show that the current design principle behind hash functions such as SHA-1 and MD5 — the (strengthened) Merkle-Damg̊ard transformation — does not satisfy this security notion. We provide several constructions that provably satisfy this notion; those new constructions introduce minimal changes to the plain Merkle-Damg̊ard construction and are easily implementable in practice. This paper is a modified version of a paper to appear at Crypto 2005.

Ordinary magic. Resilience processes in development. <=SEP=> The study of resilience in development has overturned many negative assumptions and deficit-focused models about children growing up under the threat of disadvantage and adversity. The most surprising conclusion emerging from studies of these children is the ordinariness of resilience. An examination of converging findings from variable-focused and person-focused investigations of these phenomena suggests that resilience is common and that it usually arises from the normative functions of human adaptational systems, with the greatest threats to human development being those that compromise these protective systems. The conclusion that resilience is made of ordinary rather than extraordinary processes offers a more positive outlook on human development and adaptation, as well as direction for policy and practice aimed at enhancing the development of children at risk for problems and psychopathology.

An automated framework for generating variable-accuracy battery models from datasheet information <=SEP=> Models based on an electrical circuit equivalent have become the most popular choice for modeling the behavior of batteries, thanks to their ease of co-simulation with other parts of a digital system. Such circuit models are actually model templates: the specific values of their electrical elements must be derived by the analysis of the specific battery devices to be modeled. This process requires either to measure the battery characteristics or to derive them from the datasheet. In the latter case, however, very often not all information are available and the model fitting becomes then unfeasible.
 In this paper we present a methodology for deriving, in a semi-automatic way, circuit equivalent battery models solely from data available in a battery datasheet. In order to account for the different amount of information available, we introduce the concept of "level" of a model, so that models with different accuracy can be derived depending on the available data. The methodology requires only minimal intervention by the designer and it automatically generates MATLAB models once the required data for the corresponding model level are transcribed from the datasheet.
 Simulation results show that our methodology allows to accurately reconstruct the information reported in the datasheet as well as to derive missing ones.

Spatial Reference in Linguistic Human-Robot Interaction: Iterative, Empirically Supported Development of a Model of Projective Relations <=SEP=> The aim of our research is to enable spontaneous and efficient spatial reference to objects in human-robot interaction situations. This paper presents the iterative, empirically based design of a robotic system that uses a computational model for identifying objects on the basis of a range of spatial reference systems. The efficiency of the system is evaluated by two successive empirical studies involving uninformed users. The linguistic analysis points to the striking variability in speakers’ spontaneous strategies and preferences, and it motivates a number of modifications of the computational model.

Mobile Media Use, Multitasking and Distractibility <=SEP=> Portable media devices are ubiquitous and their use has become a core component of many people’s daily experience, but to what effect? In this paper, the authors review research on the ways in which media use and multitasking relate to distraction, distractibility and impulsivity. They review recent research on the effects of media multitasking on driving, walking, work, and academic performance. The authors discuss earlier research concerning the nature of media’s impact on attention and review cognitive and neuropsychological findings on the effects of divided attention. Research provides clear evidence that mobile media use is distracting, with consequences for safety, efficiency and learning. Greater use of media is correlated with higher levels of trait impulsivity and distractibility, but the direction of causality has not been established. Individuals may become more skilled at media multitasking over time, but intervention is currently required to improve the safe and effective use of mobile media. DOI: 10.4018/ijcbpl.2012070102 16 International Journal of Cyber Behavior, Psychology and Learning, 2(3), 15-29, July-September 2012 Copyright © 2012, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited. and neuropsychological research on the effects of divided attention. Finally, we explore how high levels of media multitasking might alter our general responses to experience in ways that are marked by changes in distractibility and impulsivity. THE DISTRACTING EFFECTS OF MULTITASKING WITH MOBILE MEDIA

Multileaved Comparisons for Fast Online Evaluation <=SEP=> Evaluation methods for information retrieval systems come in three types: offline evaluation, using static data sets annotated for relevance by human judges; user studies, usually conducted in a lab-based setting; and online evaluation, using implicit signals such as clicks from actual users. For the latter, preferences between rankers are typically inferred from implicit signals via interleaved comparison methods, which combine a pair of rankings and display the result to the user. We propose a new approach to online evaluation called multileaved comparisons that is useful in the prevalent case where designers are interested in the relative performance of more than two rankers. Rather than combining only a pair of rankings, multileaved comparisons combine an arbitrary number of rankings. The resulting user clicks then give feedback about how all these rankings compare to each other. We propose two specific multileaved comparison methods. The first, called team draft multileave, is an extension of team draft interleave. The second, called optimized multileave, is an extension of optimized interleave and is designed to handle cases where a large number of rankers must be multileaved. We present experimental results that demonstrate that both team draft multileave and optimized multileave can accurately determine all pairwise preferences among a set of rankers using far less data than the interleaving methods that they extend.

A tensorized logic programming language for large-scale data <=SEP=> We introduce a new logic programming language T-PRISM based on tensor embeddings. Our embedding scheme is a modification of the distribution semantics in PRISM, one of the state-of-the-art probabilistic logic programming languages, by replacing distribution functions with multidimensional arrays, i.e., tensors. T-PRISM consists of two parts: logic programming part and numerical computation part. The former provides flexible and interpretable modeling at the level of first order logic, and the latter part provides scalable computation utilizing parallelization and hardware acceleration with GPUs. Combing these two parts provides a remarkably wide range of high-level declarative modeling from symbolic reasoning to deep learning. To embody this programming language, we also introduce a new semantics, termed tensorized semantics, which combines the traditional least model semantics in logic programming with the embeddings of tensors. In T-PRISM, we first derive a set of equations related to tensors from a given program using logical inference, i.e., Prolog execution in a symbolic space and then solve the derived equations in a continuous space by TensorFlow. Using our preliminary implementation of T-PRISM, we have successfully dealt with a wide range of modeling. We have succeeded in dealing with real large-scale data in the declarative modeling. This paper presents a DistMult model for knowledge graphs using the FB15k and WN18 datasets.

X-band microstrip narrowband BPF composed of split ring resonator <=SEP=> The paper deals with the development of X-band microstrip narrowband bandpass filter (BPF) composed of split ring resonator (SRR). The proposed filter is designed on a grounded 0.508mm thick Roger RT/Duroid® 5880 dielectric substrate with the dimension of 25mm length and 20mm width. Some parametric studies to achieve an optimum design of microstrip narrowband BPF are investigated prior realization by varying the physical parameters of SRR including the width of rings and the gap between rings. From the experimental characterization, it shows that the realized microstrip narrowband BPF has a center frequency of 9.02 GHz with the values of return loss and insertion loss of 15.231 dB and 2.768 dB, respectively. The measured result is 20MHz lower than the simulated one which has center frequency of 9 GHz with the values of return loss and insertion loss of 15.025 dB and 2.242 dB, respectively.

Enhanced Modelling and Performance in Braided Pneumatic Muscle Actuators <=SEP=> Pneumatic technology has been successfully applied for over two millennia. Even today, pneumatic cylinder based technology forms the keystone of many manufacturing processes where there is a need for simple, high-speed, low-cost, reliable motion. But when the system requires accurate control of position, velocity or acceleration profiles, these actuators form a far from satisfactory solution. Braided pneumatic muscle actuators (pMAs) form an interesting development of the pneumatic principle offering even higher power/weight performance, operation in a wide range of environments and accurate control of position, motion and force. This technology provides an interesting and potentially very successful alternative actuation source for robots as well as other applications. However, there are difficulties with this approach due to the following. (i) Modeling errors. Models of the force response are still nonoptimal and for good results these models are highly complex, which makes accurate design difficult. (ii) Low bandwidth—the bandwidth of the actuator—link assemblies are often considered to be too low for practical success in many applications, particularly robotics. In this paper we address these limitations and show how the performance in each area can be enhanced with overall improvements in the response and utility of the braided pMAs. KEY WORDS—actuators, braided pneumatic muscle actuators, McKibben muscles

A Comparison of Features for Android Malware Detection <=SEP=> With the increase in mobile device use, there is a greater need for increasingly sophisticated malware detection algorithms. The research presented in this paper examines two types of features of Android applications, permission requests and system calls, as a way to detect malware. We are able to differentiate between benign and malicious apps by applying a machine learning algorithm. The model that is presented here achieved a classification accuracy of around 80% using permissions and 60% using system calls for a relatively small dataset. In the future, different machine learning algorithms will be examined to see if there is a more suitable algorithm. More features will also be taken into account and the training set will be expanded.

High Dimensional Semiparametric Gaussian Copula Graphical Models <=SEP=> We propose a semiparametric approach called the nonparanormal SKEPTIC for efficiently and robustly estimating high-dimensional undirected graphical models. To achieve modeling flexibility, we consider the nonparanormal graphical models proposed by Liu, Lafferty and Wasserman [J. Mach. Learn. Res. 10 (2009) 2295–2328]. To achieve estimation robustness, we exploit nonparametric rank-based correlation coefficient estimators, including Spearman’s rho and Kendall’s tau. We prove that the nonparanormal SKEPTIC achieves the optimal parametric rates of convergence for both graph recovery and parameter estimation. This result suggests that the nonparanormal graphical models can be used as a safe replacement of the popular Gaussian graphical models, even when the data are truly Gaussian. Besides theoretical analysis, we also conduct thorough numerical simulations to compare the graph recovery performance of different estimators under both ideal and noisy settings. The proposed methods are then applied on a large-scale genomic data set to illustrate their empirical usefulness. The R package huge implementing the proposed methods is available on the Comprehensive R Archive Network: http://cran.r-project.org/.

Chapter 3 Recent Developments in Auxiliary Particle Filtering <=SEP=> State space models (SSMs; sometimes termed hidden Markov models, particularly in the discrete case) are very popular statistical models for time series. Such models describe the trajectory of some system of interest as an unobserved E-valued Markov chain, known as the signal process. Let X1 ∼ ν and Xn|(Xn−1 = xn−1) ∼ f(·|xn−1) denote this process. Indirect observations are available via an observation process, {Yn}n∈N. Conditional upon Xn, Yn is independent of the remainder of the observation and signal processes, with Yn|(Xn = xn) ∼ g(·|xn). For any sequence {zn}n∈N, we write zi:j = (zi, zi+1, ..., zj). In numerous applications, we are interested in estimating, recursively in time, an analytically intractable sequence of posterior distributions {p (x1:n| y1:n)}n∈N, of the form:

Fast time series classification using numerosity reduction <=SEP=> Many algorithms have been proposed for the problem of time series classification. However, it is clear that one-nearest-neighbor with Dynamic Time Warping (DTW) distance is exceptionally difficult to beat. This approach has one weakness, however; it is computationally too demanding for many realtime applications. One way to mitigate this problem is to speed up the DTW calculations. Nonetheless, there is a limit to how much this can help. In this work, we propose an additional technique, numerosity reduction, to speed up one-nearest-neighbor DTW. While the idea of numerosity reduction for nearest-neighbor classifiers has a long history, we show here that we can leverage off an original observation about the relationship between dataset size and DTW constraints to produce an extremely compact dataset with little or no loss in accuracy. We test our ideas with a comprehensive set of experiments, and show that it can efficiently produce extremely fast accurate classifiers.

Blind Source Separation in nonminimum-phase systems based on filter decomposition <=SEP=> This paper focuses on the causality problem in the task of Blind Source Separation (BSS) of speech signals in nonminimum-phase mixing channels. We propose a new algorithm for solving this problem using filter decomposition approach. Our proposed algorithm uses an integrated cost function in which independence criterion is defined in frequency-domain. The parameters of demixing system are derived in time-domain, so the algorithm has the benefits of both time and frequency-domain approaches. Compared to the previous work in this framework, our proposed algorithm is the extension of filter decomposition idea in multi-channel blind deconvolution to the problem of blind source separation of speech signals. The proposed method is capable of dealing with both minimum-phase and nonminimum-phase mixing situations. Simulation results show considerable improvement in separating speech signals specially when the mixing system is nonminimum-phase.

CSI Neural Network: Using Side-channels to Recover Your Artificial Neural Network Information <=SEP=> Machine learning has become mainstream across industries. Numerous examples proved the validity of it for security applications. In this work, we investigate how to reverse engineer a neural network by using only power side-channel information. To this end, we consider a multilayer perceptron as the machine learning architecture of choice and assume a non-invasive and eavesdropping attacker capable of measuring only passive side-channel leakages like power consumption, electromagnetic radiation, and reaction time. We conduct all experiments on real data and common neural net architectures in order to properly assess the applicability and extendability of those attacks. Practical results are shown on an ARM CORTEX-M3 microcontroller. Our experiments show that the side-channel attacker is capable of obtaining the following information: the activation functions used in the architecture, the number of layers and neurons in the layers, the number of output classes, and weights in the neural network. Thus, the attacker can effectively reverse engineer the network using sidechannel information. Next, we show that once the attacker has the knowledge about the neural network architecture, he/she could also recover the inputs to the network with only a single-shot measurement. Finally, we discuss several mitigations one could use to thwart such attacks.

Parking Guidance System Based on ZigBee and Geomagnetic Sensor Technology <=SEP=> Concerning the phenomenon that common parking service could not satisfy the increasing demand of the private vehicle owners, an intelligent parking guidance system based on Zig Bee network and geomagnetic sensors was designed. Real-time vehicle position and related traffic information were collected by geomagnetic sensors around parking lots and updated to center server via Zig Bee network. On the other hand, out-door Liquid Crystal Display screens controlled by center server can display information of available parking places. In this paper, guidance strategy was divided into four levels, which could provide clear and effective information to drivers. The experimental results prove that the distance detection accuracy of geomagnetic sensors was within 0.4m, and the lowest package loss rate of the wireless network in the range of 150m is 0%. This system can provide solution for better parking service in intelligent cities.